modelscan-legacy-pytorch-first-pickle-bypass

Proof of concept for a scanner bypass in legacy (non-ZIP) PyTorch .pt model files against protectai/modelscan (tested on 0.8.8, the current release).

Summary

modelscan fails to scan the payload region of a legacy-format PyTorch file (torch.save(..., _use_new_zipfile_serialization=False)). A file carrying the simplest, already-blacklisted global — os.system (posix.system) — is reported as "No issues found! 🎉", yet torch.load(..., weights_only=False) executes it.

This is not a blacklist-evasion trick. The exact same os.system payload is correctly flagged CRITICAL when saved in the modern ZIP format. The difference is a scanner logic bug in how the legacy multi-pickle stream is parsed.

Root cause

The legacy torch serialization format is a single file containing several pickle streams concatenated back to back:

frame 0: PROTO / LONG1(magic 0x1950a86a20f9469cfc6c) / STOP   <- magic number
frame 1: PROTO / BININT2(protocol_version) / STOP             <- version
frame 2: <system info dict>
frame 3: <the real object graph>   <-- os.system lives HERE
frame 4: <storage keys>
... raw tensor storage bytes follow ...

modelscan dispatches legacy .pt files to scan_pytorch() (modelscan/tools/picklescanner.py), which calls:

return scan_pickle_bytes(model, settings, scan_name, multiple_pickles=False)

With multiple_pickles=False, _list_globals() runs pickletools.genops once and stops at the first STOP opcode — i.e. the end of frame 0, which contains only the magic-number LONG1. It therefore extracts zero globals and never looks at frame 3 where the malicious posix.system reduce lives.

Verified directly:

multiple_pickles=False (what scan_pytorch uses): set()      # <- payload never seen
frame 3 globals: ['torch._utils _rebuild_tensor_v2', 'torch FloatStorage',
                  'collections OrderedDict', 'posix system']  # <- payload is here

The scanner reads ~15 of the file's bytes and declares the rest clean.

Why this is distinct from prior submissions

• modelscan-pytorch-zip-gadget-chain-bypass: modern ZIP .pt; requires an operator.methodcaller + importlib gadget chain to dodge the static blacklist. That is blacklist evasion in a file modelscan does parse fully.
• This finding: legacy non-ZIP .pt; uses the plainest blacklisted global (os.system) with no obfuscation. modelscan misses it because it only parses the first of many concatenated pickle frames. It is a parsing/coverage bug, and it would also defeat any future blacklist additions since the payload region is never scanned at all.

Reproduce

# Python 3.12 (modelscan requires <3.13)
python -m venv venv && source venv/bin/activate
pip install "modelscan==0.8.8" "torch==2.13.0+cpu" \
    --extra-index-url https://download.pytorch.org/whl/cpu

# Build the PoC (harmless: creates a marker file on load)
python make_poc.py artifacts/legacy_first_pickle_bypass.pt MARKER.txt

# 1) modelscan sees nothing
modelscan -p artifacts/legacy_first_pickle_bypass.pt
#   --> "No issues found! 🎉"

# 2) torch.load executes the payload
python -c "import torch,os; \
  torch.load('artifacts/legacy_first_pickle_bypass.pt', weights_only=False); \
  print('executed:', os.path.exists('MARKER.txt'))"
#   --> executed: True

verify.py runs both steps and asserts the bypass automatically.

Control (proves the format is the differentiator)

Saving the identical os.system payload with the modern ZIP format (_use_new_zipfile_serialization=True) makes modelscan report CRITICAL Use of unsafe operator 'system' from module 'posix'. Only the legacy format is missed.

Files

• make_poc.py — generates the legacy-format PoC .pt
• verify.py — end-to-end assertion (scan is clean AND load executes)
• artifacts/legacy_first_pickle_bypass.pt — the PoC model file

Impact

Arbitrary code execution on model load that is invisible to modelscan. Any pipeline that gates untrusted PyTorch checkpoints with modelscan and then loads them with torch.load(..., weights_only=False) (the default in older torch, and common in practice) is exposed. Legacy-format checkpoints are still produced and accepted across the ecosystem, so a malicious file needs no exotic construction — just the old save flag.

Suggested fix

Scan legacy PyTorch streams with multiple_pickles=True (walk every concatenated pickle frame until the stream is exhausted), rather than stopping at the first STOP.