modelscan-h5-nested-lambda-bypass

This repository contains a proof of concept for a scanner bypass in legacy Keras HDF5 (.h5) model files.

Vulnerability

The open-source protectai/modelscan HDF5 Keras Lambda detection logic only inspects the first-level config.layers list stored inside the HDF5 model_config attribute. It does not recursively inspect nested model configs such as an inner Sequential.

This means a malicious HDF5 model can hide a real serialized Lambda layer inside a nested model while keeping the top-level layer list apparently benign. Keras still recognizes the nested Lambda as unsafe and, with safe_mode=False, still executes it during load.

Primary PoC

The main PoC file is:

• artifacts/nested_lambda.h5

The payload is harmless. When loaded unsafely, it writes a local marker file:

• H5_MARKER.txt

Included Files

• artifacts/nested_lambda.h5
• artifacts/reproduce_output.txt
• artifacts/sha256sums.txt
• reproduce.py

Tested Environment

• Date tested: July 16, 2026
• Python: 3.13.12
• Keras: 3.15.0
• Backend: NumPy (KERAS_BACKEND=numpy)
• modelscan: protectai/modelscan @ 61fcec9c2a37c24c1fb12d84ede30fe248a364bd

Reproduction

1. Verify the included PoC

Run:

KERAS_BACKEND=numpy python reproduce.py --modelscan-path /path/to/modelscan

If modelscan is already installed in your environment, --modelscan-path is optional.

Expected results:

• top-level layer classes are only InputLayer and Sequential
• recursive config walk still finds one Lambda
• safe_mode=True rejects the model as unsafe
• safe_mode=False loads the model and creates the marker file
• modelscan helper returns []
• full ModelScan().scan(...) reports zero issues

2. Rebuild the artifact from source logic

Run:

KERAS_BACKEND=numpy python reproduce.py --build --modelscan-path /path/to/modelscan

This regenerates artifacts/nested_lambda.h5 and reruns the verification steps.

Observed Results

From artifacts/reproduce_output.txt:

• top_level_layer_classes: ['InputLayer', 'Sequential']
• recursive_lambda_count: 1
• safe_mode=True rejects the model
• safe_mode=False loads successfully
• marker file is created during unsafe load
• modelscan_helper_operators: []
• modelscan_issue_count: 0

Impact

This is a practical scanner bypass for a Keras model format:

• the file is a valid .h5 model
• the malicious operator is still a real serialized Lambda
• Keras still treats it as unsafe
• existing HDF5 scanner logic misses it when the layer is nested instead of top-level

Notes

• The included artifact is intentionally harmless and only writes a marker file.
• The bypass comes from scanner traversal logic, not from a novel Keras execution primitive.