ChristianTeroerde/modelscan-nested-lambda-poc

Security Research PoC — ModelScan coverage gap for nested Keras Lambda

This repository contains a deliberate security-research proof-of-concept submitted through huntr (Protect AI / Palo Alto Networks) under the Model File Vulnerability program. It is NOT a usable model. Do not deploy it.

What this demonstrates

ModelScan detects an unsafe top-level Keras Lambda layer in a .keras archive, but misses a structurally-equivalent unsafe Lambda when it is nested inside a wrapper layer (here TimeDistributed). The Keras loader still reaches and deserializes that nested layer.

Two artifacts are included:

File	Structure	ModelScan result
top-level-lambda.keras	Lambda at top level (config.layers[*])	1 issue (detected)
nested-wrapper-lambda.keras	same Lambda under TimeDistributed.layer	0 issues (missed)

Root cause: ModelScan's Keras scanner (modelscan/scanners/keras/scan.py, _get_keras_operator_names) only iterates top-level config.layers[*] for class_name == "Lambda" and does not recurse into wrapper config.layer (or Bidirectional.forward_layer / backward_layer, etc.).

This is NOT a Keras safe-mode bypass

Keras default safe_mode=True correctly blocks both models with a ValueError about Lambda deserialization. The issue is purely a scanner coverage gap: a tool meant to flag unsafe Keras artifacts returns a clean result for a model that is structurally just as unsafe as one it flags. The risk is that a defender who gates untrusted models on a clean ModelScan result is given false assurance and then loads the artifact in an unsafe-deserialization context.

The payload is benign

The nested Lambda writes a single marker file MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt in the current working directory and returns its input unchanged. It performs no network, shell, file-deletion, or otherwise harmful action. The marker only proves the nested Lambda is reachable.

Reproduce (only in an isolated environment you control)

import keras
# default safe mode blocks both (expected):
keras.saving.load_model("nested-wrapper-lambda.keras")          # -> ValueError

# explicit unsafe load reaches the nested Lambda; calling the model runs it
# and writes the benign marker file:
m = keras.saving.load_model("nested-wrapper-lambda.keras", safe_mode=False)
import numpy as np
m(np.zeros((1, 2, 1), dtype="float32"))   # writes MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt

Scan both files with ModelScan to observe the differential (1 issue vs 0 issues).

Suggested fix

ModelScan should recursively walk Keras configuration objects and flag Lambda wherever it appears in nested layer-like fields (wrapper layer, bidirectional forward_layer/backward_layer, nested preprocessing/pipeline layers, and other deserialize_keras_object() targets), not only top-level config.layers[*].

Disclosure

Reported responsibly via huntr. Generated with Keras 3.15.0 (numpy backend).