⏱️ Lectura: 9 min
BZip3 just demonstrated that the same file can be compressed up to 84% smaller than with BZip2, without touching a single line of the original content. The open-source project, published on GitHub by developer iczelia, presents itself as the spiritual successor to BZip2: same general goal, but with a completely different engine inside.
📑 En este artículo
In a public benchmark using the complete Perl 5 source code, bzip3 went from a 3.44 GB file compressed with BZip2 to just 546 MB. That’s the kind of improvement that matters when you manage backups, code repositories, or CI pipelines that move gigabytes of text every day.
TL;DR
- BZip3 is an open-source compressor presented as the spiritual successor to BZip2, published on GitHub by iczelia.
- With the complete Perl 5 source, BZip3 -b511 compressed the file to 546,456,978 bytes versus 3,441,163,911 bytes for BZip2.
- That’s a file nearly 84% smaller than BZip2’s with the same test content.
- LZMA (xz -9) reached 2,056,645,240 bytes: better than BZip2, but far behind BZip3 -b511.
- BZip3 decompressed the corpus in 4min06s versus 9min22s for BZip2 and 4min40s for xz, on an HDD.
- Combined with lrzip for long-range deduplication, BZip3 brought the final result down to just 60,672,608 bytes.
- The engine uses BWT via suffix arrays, RLE+LZ77 with PPM modeling, and a context-mixing entropy encoder.
- The repository has accumulated 1,400 stars and 61 forks on GitHub, and can already be installed with brew install bzip3 on macOS.
What Happened
The iczelia/bzip3 repository has been on GitHub for a while, but it started circulating again in developer communities because of its compression numbers against classic tools like BZip2, LZMA (xz), and Zstandard. The author documented his own benchmark: he downloaded every Perl 5 version published on CPAN, decompressed them, and packed everything into a single .tar file to compare compressors under equal conditions.
The result with BZip2 was a file of 3,441,163,911 bytes. With LZMA (xz -9) it dropped to 2,056,645,240 bytes. But bzip3, running with 511 MiB blocks, reduced it to 546,456,978 bytes: less than a sixth of the size it took up with BZip2.
Context and History
BZip2 was born in 1996 and is still used in Linux distributions, in the .tar.bz2 format, and in old-guard compression tools. Its compressor is based on the Burrows-Wheeler transform combined with Huffman coding, an effective design for its time but one that doesn’t take advantage of multi-core CPUs or the entropy modeling advances of the last two decades. You can review its full history on the BZip2 Wikipedia page.
bzip3 takes the same core idea (BWT plus grouping similar symbols before compressing) but changes almost the entire rest of the pipeline. Instead of Huffman, it uses an entropy encoder with order-0 context mixing. Instead of a single reordering pass, it adds an RLE stage combined with LZ77-style string matching and PPM-style modeling before entering the final encoder.
The project declares tested support on ten different architectures: x86, x86_64, armv6, armv7, aarch64, ppc64le, mips, mips64, sparc, and s390x. That coverage is unusual for a relatively young compression project and suggests it has already run on embedded hardware, ARM servers, and legacy machines alike.
Technical Details of BZip3 and Its Performance
The table below summarizes the benchmark against its main alternatives, compressing the same .tar file with the complete Perl 5 history:
| Method | Compressed Size | Total Compression Time |
|---|---|---|
| LZMA (xz -9, 16 threads) | 2,056,645,240 bytes | 12min 09s |
| BZip2 -9 | 3,441,163,911 bytes | 17min 16s |
| BZip3 -b 256 (12 threads) | 1,001,957,587 bytes | 7min 10s |
| BZip3 -b 511 (4 threads) | 546,456,978 bytes | 7min 08s |
| Zstandard -16 (12 threads) | 3,076,143,660 bytes | 6min 35s |
With blocks of just 256 MiB, bzip3 already compresses to less than a third of BZip2’s size, in half the time. Bumping the block up to 511 MiB reduces the final size to almost a sixth of BZip2’s, though it requires more memory: the process with -b 256 -j 12 used up to 18,301 MiB of RAM during compression, and with -b 511 -j 4 the peak dropped to 12,178 MiB.
In decompression, measured separately on a mechanical WD Blue disk, bzip3 in parallel mode took 4min 06s: faster than BZip2 (9min 22s) and LZMA (4min 40s), and nearly on par with Zstandard (3min 51s).
bzip3’s internal pipeline chains three stages before writing the final file:
flowchart TD
A["Input data"] --> B["BWT via suffix arrays"]
B --> C["RLE + LZ77 with PPM modeling"]
C --> D["Entropy encoder (order-0 context mixing)"]
D --> E[".bz3 file"]Real-world performance depends heavily on the compiler. According to the documentation in the official bzip3 repository, Linux x64 builds compiled with Clang 13 reach up to 17 MiB/s compression and 23 MiB/s decompression per thread. Windows and 32-bit builds tend to perform considerably worse.
💡 Tip: the-jflag controls how many threads bzip3 uses. With large blocks (-b 511) you need fewer threads to saturate the CPU because each block is already a large unit of work; with small blocks it’s better to raise-jto parallelize more.
Getting Started with BZip3
Installing bzip3 doesn’t require any unusual steps: it’s a project built with autotools and CMake, and it has already reached the most common package managers.
# Linux (build from a git clone of the repository)
$ ./bootstrap.sh
$ ./configure
$ make
$ sudo make install
# macOS (via Homebrew)
$ brew install bzip3
# Windows (via CMake, with Visual Studio or MinGW/MSYS2 installed)
> cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
> cmake --build build --config ReleaseOn Linux, if you installed from a source package (not a git clone), you can skip the ./bootstrap.sh step. On Windows and in 32-bit builds, the project itself warns that performance is usually considerably lower than on Linux x64.
Once installed, compressing and decompressing a real file looks like this:
# Compress with 256 MiB blocks and 12 parallel threads
$ bzip3 -e -b 256 -j 12 backup.tar
# Decompress the result
$ bunzip3 backup.tar.bz3The -e flag forces explicit compression mode, -b sets the block size in MiB, and -j sets the number of threads. To confirm the installation went well, run bzip3 --help: it should list those options along with -d for manual decompression.
Impact and Analysis
The use case where bzip3 stands out most is text and source code: packaged git repositories, logs, CSV or JSON datasets, and code trees like the Perl 5 benchmark. The README says it plainly: bzip3, like its predecessor, “excels at compressing text or code”. For already-compressed binaries (JPEG images, videos, nested ZIP files) the advantage shrinks, because no general-purpose compressor gets much out of those.
The combo with lrzip is interesting for giant files with a lot of long-range redundancy: lrzip does the deduplication between distant parts of the file, and bzip3 compresses what’s left. In the benchmark, that combination brought Perl 5’s .tar.lrz down to 60,672,608 bytes, versus 64,774,202 bytes for lrzip+lzma and 75,685,065 bytes for lrzip+bzip2.
⚠️ Heads up: the project itself warns that no compressor can guarantee 100% that a compressed file can always be recovered. Before using bzip3 for critical backups, test the full compress-and-decompress cycle with sample data and verify the result.
The tradeoff with bzip3 is memory. Large blocks mean keeping several hundred megabytes per active thread in RAM during compression, something Zstandard avoids by design (687 MiB peak in the same benchmark, versus more than 12 GB for bzip3 with large blocks). For servers with limited RAM or containers with strict limits, bzip3 with small blocks (-b 256 or smaller) is the more reasonable choice.
What’s Next
The repository remains active, with 456 commits accumulated and official support already packaged for macOS via Homebrew. The README points to additional comparisons against Turbo-Range-Coder and BSC within the repository itself, which suggests the project keeps actively measuring itself against the state of the art in lossless compression.
For teams that currently rely on .tar.bz2 in CI pipelines or package distribution, the logical next step is to run their own dataset (not Perl 5’s) against bzip3 with different block sizes, and compare final size, compression time, and peak memory before migrating any production process.
📖 Summary on Telegram: View summary
Try it yourself: clone the bzip3 repository on GitHub and run bzip3 -e -b 256 -j 4 on your own code repository to compare the final size against the .tar.gz you already use.
Frequently Asked Questions
Is BZip3 compatible with BZip2’s .bz2 files?
No. bzip3 uses its own container format (.bz3 extension) and a different compression pipeline. To open a .bz2 file you need bzip2 or a tool that explicitly supports it; bzip3 doesn’t read that format.
How much memory does BZip3 need to compress?
It depends on the block size. In the Perl 5 benchmark, running bzip3 with -b 256 -j 12 used up to 18,301 MiB of RAM; with -b 511 -j 4 the peak dropped to 12,178 MiB. Smaller blocks consume considerably less memory.
Does BZip3 run on Windows?
Yes, via CMake with Visual Studio or MinGW/MSYS2, though the project itself warns that Windows and 32-bit builds tend to perform considerably worse than Linux x64 with Clang.
What architectures has BZip3 been tested on?
The README lists x86, x86_64, armv6, armv7, aarch64, ppc64le, mips, mips64, sparc, and s390x as already-verified platforms.
Is BZip3 good for compressing images or video?
Not its strong suit. bzip3 is optimized for text and source code; already-compressed binary files (JPEG, MP4, ZIP) don’t benefit as much from its BWT and context-modeling pipeline.
Is it safe to use BZip3 for backups without testing it first?
The project itself doesn’t recommend it: the README’s disclaimer asks that you not compress critical data without accepting the possibility, however small, that the file might not be recoverable. Testing the full compress-and-decompress cycle before trusting it for production is the sensible practice.
References
- iczelia/bzip3 on GitHub: official repository, source code, and complete benchmarks.
- Perl 5 version archive on CPAN: the dataset used in the compression benchmark.
- Burrows-Wheeler transform on Wikipedia: context on the algorithm shared by BZip2 and BZip3.
- BZip2 on Wikipedia: history and context of the predecessor compressor.
📱 Enjoying this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
Imagen destacada: Foto de Sebastian Kanczok en Unsplash
0 Comments