diverse derived malicious prompts bypass ModerationBERT-En
Hello ModerationBERT-En Maintainer,
We are security researchers evaluating whether prompt safety checkers detect malicious prompts derived systematically from blocked originals. The derived prompts use different wording and token patterns while retaining the original harmful intent when submitted to downstream text-to-image generators.
We evaluated ModerationBERT-En using the same inference configuration and decision rule for original and derived prompts across pornography, illegal activity, disturbing content, hate, and violence. The checker accepted 75.28% of the original malicious prompts and 99.60% of the derived malicious prompts, an increase of 24.32 percentage points. For pornography, acceptance increased from 33.33% to 100%. Illegal-activity acceptance increased from 81.08% to 100%, disturbing-content acceptance increased from 62% to 98%, and hate and violence reached 100% acceptance.
The derived prompts were not optimized against ModerationBERT-En and transferred to several independently developed safety checkers. This indicates a generalization weakness rather than a checker-specific query attack. Applications relying on this model alone may accept transformed malicious inputs that preserve harmful downstream image-generation intent.