AI-generated CSAM detection evasion — adversarial manipulation of content safety classifiers
Date: 2024-09
Affected: Open-source image generation models with safety filters
Attack vector: adversarial-input
Category: research-demonstrated
Severity: Critical

Researchers demonstrated that image generation safety classifiers (NSFW detection, CSAM detection) could be bypassed using adversarial prompt techniques, negative prompt manipulation, and fine-tuned LoRA models. The attacks allowed generation of content that existing content moderation systems failed to detect, as the adversarial techniques modified the output distribution enough to evade classifiers while maintaining semantic content.

Impact: Content safety bypass enabling prohibited content generation; regulatory compliance risk for model providers

Mitigations:
Multi-layered content safety: pre-generation, during generation, and post-generation checks
Adversarial robustness testing for content safety classifiers
Hardware-level content provenance (C2PA) for generated images
Fine-tuning restrictions on safety-critical model components

Architectural layers (MAESTRO):
label: Foundation Models
notes: Fine-tuned models and prompt manipulation bypass generation safety filters
role: origin
label: Evaluation & Observability
notes: Content safety classifiers fail on adversarially modified outputs
role: blind-spot
label: Security & Compliance
notes: Generated content evades detection infrastructure
role: impact