bzip3 Beat xz on Size and Speed, Then We Found What Its Release Notes Left Out
bzip3 hit the Hacker News front page with 412 points and shipped version 1.5.4 six hours later. We built it, benchmarked it against xz and zstd on 100MB of Wikipedia text, and reproduced the silent data loss bug the release quietly fixed.
The WJS Desk
Sep 8, 2026 · 6 min read
bzip3 spent 7 September on the Hacker News front page, and stood at 412 points and 119 comments when we last checked on 8 September. That is a lot of attention for a repository created on 1 May 2022 that has 1,461 stars and 19 watchers.
So this is not a velocity story. bzip3 is four years old and does not appear anywhere in our trending window, which only tracks repos created in the last 30 days. It is a rediscovery story, and the interesting part is what the maintainer did about it. The thread went up at 13:35 UTC. Version 1.5.4 was tagged at 19:53 UTC the same day, six hours and eighteen minutes later, ending a 13-month gap since 1.5.3.
What it actually is
bzip3 is what bzip2 would be if it were written now: a Burrows-Wheeler transform over suffix arrays, an order-0 context mixing entropy coder, and an LZP prefilter. The README's claim is narrow and worth repeating precisely, because it is the whole pitch: bzip3 excels at compressing text or code. It does not claim to be a general-purpose zstd replacement.
The compressor core is 1,054 lines of C in src/libbz3.c, plus a 242-line header. The CLI is another 814 lines. The only real dependency is libsais.h, a 5,428-line suffix-array library vendored directly into include/. Beyond that it is libc and pthread. Nothing else.
That has a practical consequence. We did not have cmake installed and did not install it. Two source files and one compiler invocation was enough:
cc -O3 -march=native -Iinclude -DVERSION='"1.5.4"' -DPTHREAD \
-o bzip3 src/main.c src/libbz3.c -lpthreadIf this project is ever abandoned, you can vendor the whole compressor into your own tree in an afternoon. That is worth more than most stars.
Gotcha: our first build omitted -DPTHREAD. The resulting binary still printed Usage: bzip3 [-e/-z/-d/-t/-c/-h/-V] [-b block_size] [-j jobs] in its own help text, then answered -j -- unknown option when we passed -j. The usage string is not conditional on the build flag that makes half of it work.
Our numbers, on 100MB of Wikipedia
We compressed enwik8, the standard 100,000,000-byte Wikipedia text corpus, on an M4 Pro with 12 cores running macOS 26.5.1. Every figure below is a single run against a warm page cache, not an average, so treat the times as accurate to roughly a second rather than to the millisecond. The sizes are exact.
| Method | Compressed size | Compress time | Decompress time | Peak RSS |
|---|---|---|---|---|
| bzip3 -b 100 -j 1 | 20,749,611 | 8.15s | 6.15s | 601 MB |
| bzip3 -b 16 -j 8 | 22,677,530 | 2.11s | 1.38s | 722 MB |
| xz -9e -T1 | 24,831,648 | 52.07s | 0.71s | 673 MB |
| zstd --ultra -22 --long=27 | 25,272,471 | 53.10s | 0.08s | 189 MB |
| bzip2 -9 | 29,008,758 | 4.39s | 1.71s | 8 MB |
Read the first column and the last column together, because that is the trade. bzip3 at a 100MB block produced a file 4.08MB smaller than xz -9e and did it in about a sixth of the time. Against its own ancestor it is 28.5 percent smaller. We verified the round trip: the decompressed output was byte-identical to the input, SHA-256 2b49720e.
Now the losses. zstd decompresses in 0.08 seconds against bzip3's 1.38, which is 17 times faster, and that ratio is what matters if you write once and read constantly. And bzip2 finished the whole job in 8MB of resident memory. bzip3 with eight threads wanted 722MB, roughly 90 times more, to compress a 100MB file. On a build agent with a small memory ceiling that is the number that will bite you, not the ratio.
We tested the top complaint in the thread
The sharpest comment on Hacker News was from ot, who called the README's benchmark table "disingenuous, to the point of looking cherry-picked": bzip3 is run with a 512MB block while zstd keeps its default window of about 8MB, on a corpus made of every Perl 5 release concatenated. Near-identical files repeated hundreds of times is the best case for a large window and the worst case for a small one.
That criticism is correct about that corpus. It does not explain our result. enwik8 is a single stream of natural-language text with no long-range file duplication, and we gave zstd the window anyway:
| zstd configuration | Compressed size | Time |
|---|---|---|
| -19 -T1, default window | 26,944,227 | 32.50s |
| -19 -T1 --long=27 (128MB window) | 26,395,074 | 36.81s |
| --ultra -22 -T1 --long=27 | 25,272,471 | 53.10s |
Matching the window bought zstd 1.67MB and cost it 20 seconds, and it still finished 4.52MB behind bzip3. Both things are true at once: the README's headline comparison is unfair, and bzip3 still wins on prose by a wide margin. We would have reported the opposite just as happily.
The bug 1.5.4 fixed without saying so
Version 1.5.4 has an empty release body on GitHub and no entry in NEWS. Comparing tags, it is nine commits ahead of 1.5.3, and two of those are labelled only fix #174 and fix #175.
Issue 175 is a silent data loss bug in the high-level library API. When in_size is an exact multiple of block_size, bz3_compress encodes the final block with orig_size = 0 and still returns BZ3_OK. So does the matching decompress, while handing back fewer bytes than you put in.
We built the reporter's case against both versions. On 1.5.3, compressing exactly 1MiB with a 1MiB block size:
bz3_compress rc=0 (BZ3_OK) compressed=29 bytes
bz3_decompress rc=0 (BZ3_OK) decompressed=0 bytes (expected 1048576)Every byte gone, twice reported as success. The same program against 1.5.4:
bz3_compress rc=0 (BZ3_OK) compressed=129 bytes
bz3_decompress rc=0 (BZ3_OK) decompressed=1048576 bytes (expected 1048576)The fix works. We also checked the blast radius, because this matters for who needs to care: we compiled the 1.5.3 command line tool and round-tripped a 1MiB file with -b 1, and it came back intact. The CLI drives the per-block API directly and never touches the broken path. This is a library bug only. If you shell out to bzip3, you were never exposed. If you link libbz3 and call bz3_compress, you were, and nothing in the release told you.
A compressor that loses your data and returns success is the worst failure mode in the category, and it shipped its fix with an empty changelog.
What it does not do
- Random access. Blocks are independent, but there is no index, so seeking into a large archive means decoding from a block boundary.
- Ecosystem support. The complaint from 8organicbits on Hacker News is the real adoption barrier: they picked gzip for JSONL files despite worse ratios because their tooling supports it transparently. bzip3 has no DuckDB extension, no transparent HTTP content-encoding, no browser support.
- Small inputs. The design is block-oriented and memory-hungry. Nothing here helps you compress a 4KB API response.
- Binary data. The README says text or code and means it. We did not test images or already-compressed archives, and neither should you expect much.
- A guarantee. The README carries the warning inherited from bzip2 in full capitals: do not compress any data with this program unless you are prepared to accept the possibility, however small, that the data will not be recoverable. Issue 175 is that sentence doing its job.
Who maintains it, and should you use it
The bus factor is one. Kamila Szewczyk, as iczelia, authored three of the five commits landed in the six months to 8 September; the other two came from hartwork and arp242 and touched autoconf and documentation. Five commits in six months on a compression library is not neglect, it is a project that is close to finished. There are eight open issues and the repository moved namespace at 1.5.3, which is why older links point at a different owner.
One more detail we enjoyed. At 14:20 UTC, 45 minutes into the thread, sedatk objected that "the latest release is a year ago, the last commit is two months ago, and the build is failing." That was accurate when written. Version 1.5.4 landed five and a half hours later, and the CI fix for Clang on armv6 landed the following morning.
Adopt it today if you are archiving text or source and you control both ends: log tarballs, corpora, code snapshots, anything you write once and read rarely. The ratio is real and we measured it. Wait if your reads are hot, your memory ceiling is tight, or your pipeline needs a compressor other tools already understand. What would change our mind on the memory objection is streaming support that decouples working set from block size, and the block-oriented format does not obviously allow that.
