currentlyawey/kaldi-nnet3-read-eager-memset-dos

Kaldi Matrix::Read / Vector::Read eager-alloc memset DoS

Target: kaldi-asr/kaldi, nnet3 model loader (C++) Pin: e02e35f0254bb033fab73d1df99fc34123e31d56 (2025-09-22) Sinks:

• src/matrix/kaldi-matrix.cc:1505-1508 Matrix<Real>::Read() — unvalidated ReadBasicType(rows) / ReadBasicType(cols) feed directly into this->Resize(rows, cols) with no bound/sanity check.
• src/matrix/kaldi-matrix.cc:857 / :1335 — Resize() -> Init() + SetZero(), and SetZero()'s memset(data_, 0, sizeof(Real)*num_rows_*num_cols_) at line 1335 is the real, physical commit (touches every page — not a lazy reserve).
• src/matrix/kaldi-vector.cc Vector<Real>::Read() — same shape: unvalidated ReadBasicType(size) -> this->Resize(size) -> SetZero() memset.
• src/nnet3/nnet-simple-component.cc:1289 AffineComponent::Read() calls linear_params_.Read(is, binary) (a Matrix<BaseFloat>) and bias_params_.Read(is, binary) (a Vector<BaseFloat>) with no dimension check before or after — confirmed by reading the pinned source (see "Reachability" below). NaturalGradientAffineComponent, RepeatedAffineComponent, etc. have the same pattern.

Reproduced: YES

A crafted file of 13 bytes ("FM " + int32 rows=46000 + int32 cols=46000, no payload) drives the real, unmodified kaldi::Matrix<float>::Read() to commit ~7.89 GiB via a single blocking memset(), confirmed three independent ways in the same run:

• /proc/self/status VmHWM (peak resident-set high-water mark, sampled immediately after the call returns/throws): 8,269,932-8,270,188 kB (~7.89 GiB), vs. baseline ~4.2 MB.
• /usr/bin/time -v "Maximum resident set size": matches VmHWM.
• Minor page faults: 2,066,565 x 4096 B ~= 8,464,650,240 B, matching the expected 46000*46000*4 = 8,464,000,000 bytes almost exactly (the kernel zero-fills each page on first touch during the memset, one minor fault per page).
• Wall-clock: ~7-10 s (mostly System time, i.e. kernel-side page zeroing) blocking a single thread, from a 13-byte input.

A second crafted file ("FV " + int32 size=536870000, 8 bytes) drives kaldi::Vector<float>::Read() to commit ~2.00 GiB via the same Resize()->SetZero()->memset() path (VmHWM 2,101,512-2,101,612 kB, ~524,446 minor faults x 4096 B ~= the expected 2,147,480,000 B).

Both PoC files intentionally omit the payload bytes: Resize()'s memset happens before the subsequent is.read() of matrix/vector data, so a truncated stream still forces the full commit and then throws — the downstream KaldiFatalError is expected and does not indicate the bug didn't fire; VmHWM (a monotonic, unaffected-by-later-free counter) proves the commit happened regardless of the object being destructed on unwind.

Negative controls (small valid matrix/vector) load correctly (rows=2 cols=3; dim=4), return in <0.15 ms, and never move VmHWM off baseline (~4.2-4.6 MB) — confirming the harness parses valid Kaldi binary streams correctly and the multi-GB commit is specific to the crafted rows/cols/size fields, not a harness artifact.

A related, distinct bug found along the way (documented, not the main PoC)

Vector<Real>::Init() (kaldi-vector.cc) computes MatrixIndexT size = dim * sizeof(Real); where MatrixIndexT is a 32-bit int32 (matrix-common.h:98). The multiplication is evaluated in 64-bit (sizeof() is size_t), but the result is truncated back into the 32-bit size variable. For dim=2,100,000,000, dim*4=8,400,000,000 wraps mod 2^32 to a negative int32 (-189,934,592), which is then reinterpreted as a huge size_t when passed to KALDI_MEMALIGN/malloc, which fails fast with ENOMEM — no memset ever touched, VmHWM stays at baseline. This is captured as kaldi_poc_out/variant_vector_int32_truncation_badalloc.vec for the record; it is a real (separately reportable) 32-bit integer-truncation bug, but it is not the eager-alloc-memset DoS this PoC targets, so the main Vector PoC uses dim=536,870,000 (the practical ceiling that stays inside the positive int32 byte-size range) to reliably trigger the genuine memset commit instead.

Reachability (source-confirmed, not re-built at runtime)

nnet3/nnet-simple-component.cc:1289-1297:

void AffineComponent::Read(std::istream &is, bool binary) {
  ReadUpdatableCommon(is, binary);
  ExpectToken(is, binary, "<LinearParams>");
  linear_params_.Read(is, binary);      // Matrix<BaseFloat>::Read -- no dim check
  ExpectToken(is, binary, "<BiasParams>");
  bias_params_.Read(is, binary);        // Vector<BaseFloat>::Read -- no dim check
  ...
}

This is the exact Matrix<Real>::Read / Vector<Real>::Read entry point our harness calls directly. nnet3-copy (and any tool that loads an nnet3 raw/transition model, e.g. nnet3-am-copy, decoders, etc.) reaches this via Nnet::Read() -> the component-factory dispatch -> whichever Component::Read() the on-disk <ComponentName> token selects. Building the full nnet3 component-registry/factory (plus cudamatrix CPU-fallback, hmm, tree, etc.) was judged infeasible to do reliably within the available time/toolchain (no root/apt access in this WSL environment to install a full BLAS for a complete Kaldi build; see below), so per the task's explicitly sanctioned fallback, the runtime PoC targets Matrix<float>::Read/Vector<float>::Read directly — this is the exact same, unmodified sink code AffineComponent::Read calls, compiled from the same pinned source tree, not a reimplementation or simulation.

Build notes (why this works without a full Kaldi/BLAS build)

No root/sudo is available in this environment, so apt-get install libopenblas-dev etc. was not possible, and building OpenBLAS/ATLAS from source was judged out of scope for what the sink actually needs. Matrix<Real>::Read/Resize/Init/SetZero (the vulnerable code path) never call into BLAS or LAPACK — those are only reached by numeric ops (GEMM, SVD, matrix inversion, ...) our harness never performs. However, kaldi-matrix.cc/kaldi-vector.cc/sp-matrix.cc/qr.cc are each single translation units that also define BLAS/LAPACK wrapper functions (cblas_Xgemm, clapack_Xgetrf2, ...), so the compiled .o files carry unresolved references to real BLAS/LAPACK symbols even though the harness never exercises those branches at runtime. blas_stubs.c satisfies exactly those 41 symbols with abort()-on-call stubs (see file for the full rationale and list). The bundled tools/CLAPACK/{cblas.h,clapack.h,f2c.h} headers already vendored in the kaldi source tree (no download needed) supply the declarations the real Kaldi translation units compile against (-DHAVE_CLAPACK). -DOPENFST_VER=10801 sidesteps the (unrelated) fst/types.h OpenFst dependency by taking the kaldi-types.h branch that typedefs straight off <stdint.h>. base/version.h is generated by running the vendored base/get_version.sh.

This compiles and links the real, unmodified Kaldi source for: base/io-funcs.cc, base/kaldi-error.cc, base/kaldi-math.cc, base/kaldi-utils.cc, matrix/kaldi-vector.cc, matrix/kaldi-matrix.cc, matrix/compressed-matrix.cc, matrix/packed-matrix.cc, matrix/sp-matrix.cc, matrix/tp-matrix.cc, matrix/matrix-functions.cc, matrix/optimization.cc, matrix/qr.cc, matrix/srfft.cc, matrix/sparse-matrix.cc, matrix/numpy-array.cc against harness.cc, which calls kaldi::Matrix<float>::Read() / kaldi::Vector<float>::Read() directly on a crafted std::ifstream, exactly as AffineComponent::Read calls linear_params_.Read(is, binary).

Directory contents

• RUN_LOG.txt — full /usr/bin/time -v console output for all 5 runs (2 negative controls, matrix bomb, vector bomb, int32-truncation variant).
• kaldi_poc_out/ — the crafted PoC/control files:
  • control_valid_matrix.mat (37 B, negative control, 2x3 matrix)
  • control_valid_vector.vec (24 B, negative control, dim=4)
  • poc_matrix_bomb.mat (13 B on disk -> ~7.89 GiB memset commit)
  • poc_vector_bomb.vec (8 B on disk -> ~2.00 GiB memset commit)
  • poc_affine_component_bomb.bin (13 B — same Matrix-bomb shape as would appear inside a real AffineComponent's <LinearParams> section)
  • variant_vector_int32_truncation_badalloc.vec (8 B — documents the distinct truncation/fast-bad_alloc behavior at dim=2.1e9, NOT the memset-commit PoC)
  • craft_kaldi_pocs.py — generator script for all of the above, with inline wire-format documentation and arithmetic comments.
• build_artifacts/
  • harness.cc — the PoC driver source (calls the real Kaldi sink, instruments VmRSS/VmHWM before and after).
  • blas_stubs.c — the 41 abort()-on-call BLAS/LAPACK stub definitions.
  • build_commands.sh — turnkey rebuild script (verified from a clean directory during this run).
  • harness — the built ELF binary (x86-64 Linux).

Usage

./harness <file> [matrix|vector]   # default mode = matrix

Prints /proc/self/status VmRSS/VmHWM before and after the call, the wall-clock time, and whether Read() threw.