title: "Metadata requests no longer tracked in PyPI download counts"

description: PyPI stopped counting requests that were never artifact downloads. Historical counts were inflated, and comparisons across 2026-08-24 will not line up.

authors:

- miketheman

date: 2026-08-31

tags:

- infrastructure

- transparency

meta:

- name: fediverse:creator

content: "@miketheman@hachyderm.io"

On 2026-08-24 I [shipped a change to the configuration that emits PyPI's download logs](https://github.com/pypi/infra/pull/242)

so that only requests for actual distribution artifacts are counted.

A request has to end in `.whl`, `.tar.gz`, or `.zip` to produce a download record.

The counts are more accurate now,

and existing systems based on [PyPI's BigQuery dataset](https://docs.pypi.org/api/bigquery/),

like [pypistats.org](https://pypistats.org/)

will display a different shape going forward.

If you maintain something that reads these numbers from the public dataset,

the discontinuity at 2026-08-24 is expected and permanent.

This effort would not be possible without the continued support from [Alpha-Omega](https://alpha-omega.dev/).

The logs are generated by Fastly's Edge, shipped to an AWS S3 bucket,

parsed and anonymized by our [`linehaul`](https://github.com/pypi/linehaul-cloud-function) functions in Google Cloud,

and published to a Google-managed [public BigQuery dataset](https://docs.cloud.google.com/bigquery/public-data).

Everything PyPI serves for a release lives under the same `/packages/<xx>/<yy>/<hash>/` prefix.

Matching on that prefix alone counted a fetch of any part of a release

as a download of the distribution itself.

- [PEP 658](https://peps.python.org/pep-0658/) `.metadata` sidecars. (estimated at 40%)

Installers may fetch these to read a wheel's metadata *without* downloading the wheel.

Every metadata fetch was being logged as a distribution download.

- `.asc` GPG signatures. PyPI [stopped accepting these in 2023](2023-05-23-removing-pgp.md),

but the ones uploaded before then are still served, and no longer counted.

- `.egg` uploads were separately [deprecated in 2023](2023-06-26-deprecate-egg-uploads.md).

All of these remain downloadable, and none of them are counted.

- Formats frozen for upload since 2016 under [PEP 527](https://peps.python.org/pep-0527/),

such as `.exe`, `.msi`, and `.rpm`, are about 0.2% of the files on PyPI, and are no longer counted.


*Daily download quantity across all of PyPI, 60-day window ending 2026-08-31, from [pypistats.org](https://pypistats.org/).*

Before the change, daily totals cycled between roughly 4.5 billion on weekends

and a little over 7 billion midweek.

The curve steps down after 2026-08-24.

The last few days of that window were still settling when the chart was captured,

so read the shape of the change rather than the final data point.

If you compare download counts across 2026-08-24, the numbers will not line up.

The counts after that date are lower and more accurate.

Nothing was lost from the historical record - the older rows in BigQuery are unchanged,

they were just measuring something broader than "someone downloaded this package" -

they measured "someone downloaded a given file from PyPI".

Most consumers of this dataset today do not filter specific to filename extensions,

leading to inflated download counts for a given package.

However, if you wanted to query the dataset yourself, you can write your own SQL queries

to be able to distinguish a single project's distribution file downloads

from others with a BigQuery statement that looks a bit like this:

DATE(timestamp) AS download_date,

COUNT(*) AS all_objects,

COUNTIF(REGEXP_CONTAINS(file.filename, r'\.(whl|tar\.gz|zip)$')) AS distributions_only,

COUNT(*) - COUNTIF(REGEXP_CONTAINS(file.filename, r'\.(whl|tar\.gz|zip)$')) AS others,

ROUND(100 * (COUNT(*) - COUNTIF(REGEXP_CONTAINS(file.filename, r'\.(whl|tar\.gz|zip)$'))) / COUNT(*), 1) AS others_pct

FROM `bigquery-public-data.pypi.file_downloads`

WHERE project = 'urllib3'

AND timestamp >= TIMESTAMP('2026-08-18')

AND timestamp < TIMESTAMP('2026-08-26')

GROUP BY download_date

ORDER BY download_date

Results:

| download_date | all_objects | distributions_only | others | others_pct |

| ------------- | ----------- | ------------------ | ------ | ---------- |

| 2026-08-18 | 77477790 | 47268787 | 30209003 | 39\.0 |

| 2026-08-19 | 74742551 | 45808710 | 28933841 | 38\.7 |

| 2026-08-20 | 73495894 | 45011207 | 28484687 | 38\.8 |

| 2026-08-21 | 69652309 | 42467783 | 27184526 | 39\.0 |

| 2026-08-22 | 49665300 | 30080582 | 19584718 | 39\.4 |

| 2026-08-23 | 50965479 | 30344490 | 20620989 | 40\.5 |

| 2026-08-24 | 64241532 | 44012651 | 20228881 | 31\.5 |

| 2026-08-25 | 46619401 | 46619401 | 0 | 0\.0 |

For this particular query and time range, ~39% of the download counts are

not the actual distribution files.

Download counts are tricky to get right,

and should not be used as a proxy for criticality or popularity.

Volumes are often driven by misconfiguration of proxies or caches, CI systems running wild,

or any variety of problems that occur with software.

There's even folks out there who drive these numbers up artificially.

This change removes one source of potential confusion and misinterpretation,

but does not mean download counts are an accurate measure of how many people use a package.

An HTTP response code `206 Partial Content` counts exactly the same as a `200 OK` right now.

Reading two bytes out of a wheel to inspect its ZIP central directory

is recorded as a download of the whole wheel,

alongside a client that pulled all 30 MB.

We want to record the request type and the bytes actually transferred,

so that analysis can distinguish bytes served from downloads counted.

That work is not scheduled yet.

Follow <https://github.com/pypi/linehaul-cloud-function/issues/232>

and <https://github.com/pypi/linehaul-cloud-function/issues/252>

for that.