Hugging Face model repository pickle-based malware supply chain
Date: 2024-02
Affected: Any organisation loading models from Hugging Face Hub without verification — ML training environments, inference servers
Attack vector: supply-chain
Category: real-world
Severity: Critical

Security researchers at JFrog and Protect AI identified malicious machine learning models uploaded to Hugging Face's public model repository (Hugging Face Hub). These models used Python's pickle serialisation format to embed arbitrary code that would execute on the victim's machine when the model was loaded — a form of supply chain compromise. JFrog identified over 100 models with malicious pickle payloads. Protect AI's ModelScan tool was developed specifically to address this class of attack. Hugging Face responded by enabling malware scanning and introducing safetensors as a safe serialisation alternative.

Impact: Remote code execution on model loading; potential full system compromise of ML infrastructure; demonstrated against real uploaded models

Mitigations:
Use safetensors format instead of pickle for model serialisation
Scan all downloaded model artifacts with ModelScan or equivalent before loading
Maintain an internal model registry with provenance verification — do not load arbitrary public models
Run model loading in isolated sandboxes with no network access
Verify model checksums against published hashes from known-good sources

Architectural layers (MAESTRO):
label: Foundation Models
notes: Malicious models distributed through trusted model repository — supply chain compromise at model level
role: origin
label: Data Operations
notes: Model artifacts treated as trusted data without integrity verification
role: propagation
label: Deployment & Infrastructure
notes: Code execution at model load time — arbitrary code runs in the victim's ML environment
role: impact