EnigmaConsultant/huntr-poc-avro-decompression-bomb

Apache Avro (Python) — unbounded decompression ("zip bomb") DoS via deflate codec

Status: gated, manual-approval PoC repository. For authorized security research / bug-bounty triage only (huntr.com MFF).

Target

• Project: apache/avro (Python implementation)
• Verified against: the real, official PyPI avro package, version 1.12.1 (reference implementation)
• Component: avro.codecs.DeflateCodec.decompress() / avro.datafile.DataFileReader

Summary

An Avro Object Container File declares its per-block compression codec in the file header and the compressed byte length of each data block in the block header. When the deflate codec is used, DeflateCodec.decompress() calls:

uncompressed = zlib.decompress(data, -15)

with no output-size limit. zlib/DEFLATE can achieve compression ratios up to roughly 1000-1030x on pathological (e.g. all-identical-byte) input. A small, valid .avro file can therefore declare a single data block whose compressed payload — while itself only a few MB — decompresses into gigabytes of data in one unbounded call, exhausting available memory (a classic "decompression bomb" / zip-bomb pattern) before the caller ever gets to inspect a single record.

avro_bomb_poc.avro is a well-formed Avro Object Container File (magic Obj\x01, valid header, valid avro.schema/avro.codec metadata, valid sync markers) containing exactly one data block that declares object_count=1 and a 7,305,374-byte deflate-compressed payload. The block is essentially all-repeated-byte content, chosen for near-maximal deflate compression ratio.

Proof of concept

real_library_run_output.txt — captured output from running the REAL, official avro package (installed via pip install avro==1.12.1, unmodified) against the crafted file inside a process whose address-space was capped at 900MB (ulimit -v 900000) to safely observe the unbounded-allocation behavior without risking the host:

Opening with real avro.datafile.DataFileReader ...
MemoryError after 0 records: Unable to allocate output buffer.
DONE. records=0 elapsed=0.6s maxrss=839MB

The real DataFileReader never returns a single record — it dies inside zlib.decompress() trying to materialize the whole decompressed block in one allocation, before any record-level processing (or object_count sanity check) can even occur.

compression_ratio_analysis.txt — independent confirmation of the underlying compression ratio, using zlib.decompressobj() with a 200MB output cap (to avoid ever actually exhausting host memory in this analysis) instead of the library's real unbounded call:

compressed block size: 7305374
decompressed 209715200 bytes from only 203839 consumed compressed bytes (ratio 1028.8x) before hitting a 209715200 byte safety cap
stream NOT finished (eof=False); 7101535 more compressed bytes remained unprocessed

Only ~204KB of the 7.3MB compressed block was needed to produce 200MB of output (a ~1029x ratio, consistent with DEFLATE's theoretical maximum). Extrapolated across the full block, the real (uncapped) zlib.decompress() call the avro library actually makes would attempt to allocate several GB from a single ~7MB input file.

Impact

Any service that accepts untrusted .avro files (a very common ingestion pattern for ML/data pipelines) and opens them with the stock Python avro library is vulnerable to a memory-exhaustion denial of service from a single small, well-formed-looking file, with no special crafting beyond a highly-compressible payload.

Files

• real_library_run_output.txt — captured run against the real avro 1.12.1 library
• compression_ratio_analysis.txt — independent zlib ratio verification
• test_avro_real_library.py — the test script used to produce the above output

Reproduction

python3 -m venv venv && venv/bin/pip install avro==1.12.1
(ulimit -v 900000; venv/bin/python test_avro_real_library.py)