Tutorial5 hours ago

We Rebuilt Backblaze's Q2 2026 Drive Stats in DuckDB, and Three of Its Four Honor Roll Drives Proved Nothing

We matched Backblaze's Q2 2026 table on all 354,415 drives and 31,553,350 drive-days, then added confidence intervals. Only one of four zero-or-one-failure Seagate models is provably better than the 1.75% fleet rate.

The WJS Desk

Oct 1, 2026 · 9 min read

Photo by Pixabay on Pexels

Backblaze published its Q2 2026 Drive Stats on September 29: 354,415 drives, a quarterly annualized failure rate of 1.73 percent, and an "honor roll" of Seagate models with zero or one failure. They also publish every daily row the report is built from. We downloaded the 2.09 GB quarter, rebuilt the headline table in DuckDB on a laptop, and got it to match exactly on drive count and drive-days for all 31 models.

Then we added the one column the report leaves out, a 95 percent confidence interval, and the honor roll mostly fell apart. This is the path from zip file to that table, including the three places we went wrong, so you can check the next report yourself in under 20 minutes.

What you will end up with

A local DuckDB file holding all 31,968,003 daily drive records for April 1 to June 30, 2026, a query that reproduces Backblaze's per-model AFR table, and a short Python script that tells you which of those rates you can actually trust. Here is how our rebuild compared with the published report.

Q2 2026Backblaze publishedWe computed
Drives monitored359,101359,101
Models in the table3131
Drives analysed354,415354,415
Drive-days31,553,35031,553,350
Failures1,4981,516
Quarterly AFR1.73%1.75%

The failure count is the one number we could not reproduce, and we explain below what we ruled out.

Prerequisites

  • Python 3.10 or later. We used 3.14.7 on an Apple M4 Pro running macOS 26.5.1.
  • About 15 GB of free disk: 2.09 GB for the zip, 12 GB unzipped, plus about 430 MB for the DuckDB file.
  • About 3 GB of free RAM. The load peaked at 2.7 GB resident.
  • Time: our download took 615 seconds, unzip 59, the load 49 to 53 across our two runs. Everything after that is instant.

Step 1: Get the data

Backblaze lists every quarter on its Drive Stats data page. The files live on its own B2 storage.

mkdir drivestats && cd drivestats
curl -L -o data_Q2_2026.zip https://f001.backblazeb2.com/file/Backblaze-Hard-Drive-Data/data_Q2_2026.zip
unzip -q data_Q2_2026.zip
ls data_Q2_2026 | wc -l

You should see 91 files, one CSV per day, each with 197 columns: date, serial number, model, capacity, a failure flag, location fields, and SMART attributes. We only need five of them.

python3 -m venv .venv
.venv/bin/pip install duckdb scipy

We used DuckDB 1.5.6 and SciPy 1.18.1.

Step 2: Load it, and the first thing that broke

Our first attempt told DuckDB the types of the five columns we wanted, which felt like good practice:

SELECT date, serial_number, model, capacity_bytes, failure
FROM read_csv('data_Q2_2026/*.csv', header = true,
  columns = {'date':'DATE','serial_number':'VARCHAR','model':'VARCHAR',
             'capacity_bytes':'BIGINT','failure':'INTEGER'});

It failed in 0.2 seconds with "Error when sniffing file", a list of every delimiter DuckDB had tried, and a suggestion to disable strict mode. None of that was the problem. The columns option does not mean "only these columns". It means "this is the full schema of the file", and five names against 197 real columns cannot parse. The fix is to let DuckDB read the whole header and pick the columns in the SELECT. Save this as load.sql:

CREATE OR REPLACE TABLE days AS
SELECT date::DATE AS date, serial_number,
       regexp_replace(model, '^WUH', 'WDC WUH') AS model,
       capacity_bytes, failure
FROM read_csv('data_Q2_2026/*.csv', union_by_name = true, header = true);

And run it:

.venv/bin/python -c "import duckdb; duckdb.connect('drives.duckdb').execute(open('load.sql').read())"

That took 49 seconds on our first run and 53 on a clean rerun, and turned 12 GB of CSV into a 432 MB database. Projection pushdown means DuckDB never materialises the 192 SMART columns you did not ask for. The regexp_replace line is there because of the second failure, below.

Pro tip: once it is loaded, export the five columns to Parquet with COPY days TO 'q2_2026.parquet' (FORMAT parquet, COMPRESSION zstd). It took 2.6 seconds on our clean rerun and produced a file of about 210 MB you can keep and delete the 12 GB of CSVs. A full GROUP BY over the Parquet file ran in 0.02 seconds.

Step 3: Rebuild the per-model table

Backblaze's AFR is failures divided by drive-years: every row in the data is one drive alive for one day, so count(*) / 365 is drive-years. The report only includes models with more than 100 drives and more than 10,000 drive-days in the quarter, and it excludes boot drives.

CREATE OR REPLACE TABLE m AS
SELECT model,
       round(max(capacity_bytes) / 1e12) AS tb,
       count(DISTINCT serial_number)       AS drives,
       count(*)                            AS drive_days,
       sum(failure)                        AS failures,
       round(100.0 * sum(failure) / (count(*) / 365.0), 2) AS afr_pct
FROM days GROUP BY model;

SELECT count(*), sum(drives), sum(drive_days), sum(failures)
FROM m WHERE tb >= 4 AND drives > 100 AND drive_days > 10000;

That returns 31 models, 354,415 drives and 31,553,350 drive-days, identical to the report. Getting there took two more wrong turns.

What broke, and what we could not fix

Our first table had 40 models, not 31. Nine extra rows made the cut, and most were things like CT250MX500SSD1, DELLBOSS VD and several 250 GB SSDs, all with zero failures. These are boot drives. The dataset includes them. The report does not. Nothing in the rows labels a drive as a boot drive, so we filtered by capacity: every model under 4 TB is a boot or legacy laptop drive in this fleet. That rule removed 3,959 drives, and the size thresholds removed another 727 data drives, which together account exactly for the gap between 359,101 and 354,415.

Backblaze's own text gives those two numbers as 3,881 and 705. Those add up to 4,586, not the 4,686 that its own totals imply. Our counts reconcile with the table and theirs are 100 short, so we suspect a slip in the prose rather than in the data.

One Western Digital model appears under two names. 26,208 drives report as WDC WUH721816ALE6L4 on the last day of the quarter and 531 as plain WUH721816ALE6L4. The report merges them. Without the regexp_replace in Step 2 you get two rows, and the smaller one (4.87 percent AFR on 531 drives) looks like a problem model that does not exist.

"Drive count" is not the drive count on June 30. Our first version counted drives on the last day and came up short on every model. ST8000DM002 showed 5,968 against the report's 6,903. The report counts every distinct serial that appeared at any point in the quarter, including drives that failed or were retired. count(DISTINCT serial_number) matches it for all 31 models.

We count 18 more failures than Backblaze, and we do not know why. The gap is spread over 10 models, largest on the 24 TB Seagate ST24000NM002H (82 in the data, 78 in the report). We checked the obvious causes: no serial has more than one failure row, and no drive marked failed reappears later in the quarter. Backblaze may review failures internally before publishing, but the public data does not show that step. The effect on the headline is small, 1.75 percent against 1.73.

Gotcha: if you compare your AFR with Backblaze's for a single model, expect small differences in failures even when drive-days match to the unit. Do not call a model worse on a gap of one or two failures. The next section shows why.

Step 4: Add the column the report leaves out

A model with 239 drives and zero failures in a quarter has not proven it never fails. It has proven that it did not fail in about 59 drive-years. The standard tool for this is an exact Poisson confidence interval on the failure count, which SciPy computes from the chi-squared distribution. Save this as ci.py:

import duckdb
from scipy.stats import chi2

con = duckdb.connect('drives.duckdb', read_only=True)
rows = con.sql("""SELECT model, drive_days, failures FROM m
                  WHERE tb >= 4 AND drives > 100 AND drive_days > 10000""").fetchall()
fleet = 100 * sum(r[2] for r in rows) / (sum(r[1] for r in rows) / 365)

for model, days, f in rows:
    years = days / 365
    lo = 0 if f == 0 else chi2.ppf(0.025, 2 * f) / 2
    hi = chi2.ppf(0.975, 2 * (f + 1)) / 2
    lo, hi = 100 * lo / years, 100 * hi / years
    verdict = 'worse' if lo > fleet else 'better' if hi < fleet else 'cannot tell'
    print(f"{model:22} {100 * f / years:5.2f}%  [{lo:5.2f}, {hi:5.2f}]  {verdict}")
.venv/bin/python ci.py

Here is the honor roll with those intervals, against our fleet rate of 1.75 percent.

Honor roll modelDrive-daysFailures95% interval for AFRBetter than fleet?
Seagate ST12000NM000J97,74600.00 to 1.38%Yes
Seagate ST14000NM000J43,34900.00 to 3.11%Cannot tell
Seagate ST8000NM000A21,49900.00 to 6.26%Cannot tell
Seagate ST16000NM000J14,22110.06 to 14.30%Cannot tell

One of the four is genuinely good. The other three are consistent with rates up to 3.6 times the fleet average, and the one-failure model's interval runs to 14.3 percent. The report's "clean sweep by Seagate" is real as a count and says almost nothing as a measurement. For contrast, Seagate's 35,100-drive ST16000NM001G posted 0.66 percent with an interval of 0.50 to 0.85, which is a result you could buy drives on, and it did not make the honor roll because it had 57 failures.

The intervals also firm up the bad news. All three of the report's outliers stay above the fleet rate at their lower bound, and so do seven more models. The report says 10 of 31 models were above 3 percent. Its own table has 11, and so does ours.

Common mistakes

  • Dividing failures by drive count. 1,516 failures over 354,415 drives is 0.43 percent, a quarterly number. It is not an AFR, and it ignores drives that ran for a week.
  • Using the columns option to pick columns, as in Step 2. Use SELECT.
  • Counting drives on the last day instead of distinct serials across the quarter.
  • Forgetting boot drives. They add nine zero-failure models and drag the fleet rate down.
  • Ranking models by point AFR. Rank by the upper bound if you are buying, and by the lower bound if you are deciding what to retire.
  • Treating one quarter as a verdict. Backblaze's lifetime table uses stricter thresholds (more than 500 drives and 100,000 drive-days) for exactly this reason.

What we would not do yet

We would not use one quarter to pick a drive for a NAS with four bays. Backblaze's drives live in its pods, at its temperatures and workloads, and its own report flags the high AFRs this quarter as mostly old drives: the HGST HUH721212ALN604 is nearly seven years old. Age is the confounder this tutorial does not remove. The next step, if you want it, is to join the SMART 9 (power-on hours) column and compare models at the same age.

We also would not claim Backblaze's failure count is wrong. It is 18 lower than the public rows imply, and we could not find the rule that explains it. That is a question for Backblaze, not a correction.

Rollback

Everything lives in one folder and a virtualenv. Nothing is installed system-wide.

cd .. && rm -rf drivestats

If you want to keep the analysis and lose the bulk, delete data_Q2_2026/, the zip and drives.duckdb, keep q2_2026.parquet and point the queries at it. That takes you from about 15 GB to about 210 MB.

Next steps

Run the same script on the Q1 2026 file, which is on the same page, and see whether the honor roll repeats. Which drive model do you actually run, and does its interval look better or worse than its headline? If you want another data pipeline where the first plausible result was wrong, read our MarkItDown batch run over 698 pages of IRS guidance.

Share

A Seagate drive with 0 failures in Backblaze's Q2 stats could still have a 6.26% failure rate, 3.6x the fleet. We rebuilt the whole table from 32M raw rows to check. #DuckDB #DataScience #Backblaze #Storage

Never miss a ship

The best stuff that shipped this week, delivered every Thursday. Free, no spam. We read all the boring stuff so you get the fun parts.

Keep reading