trevdatastreams/apache-avro-reader-varint-eof-poc

Apache Avro C++ legacy Reader malformed-varint PoC

Tested against Apache Avro commit:

44f7aa35a6b90ac7d39ef0e68dad706561235941

Summary

The public C++ avro::Reader, avro::ValidatingReader, and avro::ResolvingReader path does not check whether BufferReader::read() reached EOF and does not bound the varint shift count.

A 10-byte binary datum containing only continuation bytes (0x81) therefore keeps the legacy decoder inside readVarInt() after the input is exhausted. The last byte remains in val, its continuation bit remains set, and the loop never reaches its exit condition. A process loading the 10-byte file consumes a CPU core until it is terminated.

An 11-byte terminated overlong datum independently reaches a shift exponent of 70. UBSan reports undefined behavior, and an ordinary build silently decodes a bogus negative value. Apache Avro's modern BinaryDecoder rejects the same datum as Invalid Avro varint.

Reproduce

Build current Apache Avro C++ with ASan and UBSan:

git clone https://github.com/apache/avro.git
cmake -S avro/lang/c++ -B avro-build \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer -g" \
  -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address,undefined"
cmake --build avro-build --target avrocpp_s -j

Then run:

AVRO_SRC=/path/to/avro/lang/c++ \
AVRO_BUILD=/path/to/avro-build \
./reproduce.sh

Expected output includes:

runtime error: shift exponent 70 is too large for 64-bit type 'uint64_t'
trigger: modern BinaryDecoder rejected input: Invalid Avro varint
trigger: legacy Reader decoded -4647998506761461825
TIMEOUT: legacy Reader did not return after input EOF
PASS: truncated input caused deterministic non-termination (exit 124)

The non-termination test compiles only the header-defined legacy Reader call site without UBSan. Its two-second alarm is part of the harness, not the library. The same result reproduced in three consecutive runs.

Files

• control.avro: valid one-byte raw Avro long datum, value 1.
• trigger-overlong.avro: 11-byte terminated overlong varint.
• trigger-truncated.avro: 10 continuation bytes with no terminator.
• poc.cpp: control, modern-decoder, and legacy-decoder differential.
• poc_hang.cpp: deterministic non-termination harness.
• reproduce.sh: build and reproduction commands.

Raw binary datums are a supported Avro representation when the schema is provided out of band. The affected API accepts an InputBuffer containing that representation.

Artifact hashes

dbc1b4c900ffe48d575b5da5c638040125f65db0fe3e24494b76ea986457d986  control.avro
6439afb84e6b735dbe05a6b434edc5dd5c7b7cf0e57165e1ffc5430d89a785ab  trigger-overlong.avro
19386bb2036b5a0377ce28d0d069d61131fe668c292128cbd1783f9a29d80129  trigger-truncated.avro

Suggested fix

Check the boolean return from reader_.read(val) and throw an Avro parse exception on EOF. Before shifting, reject shift >= 64, matching BinaryDecoder::doDecodeLong().