ToxicChat: Hidden Toxicity Detection Challenges in User-AI Conversations
Date: 2023-10
Affected: Chatbot moderation systems, content filters
Attack vector: jailbreak
Category: research
Severity: Low

Lin et al. release ToxicChat, a 10K-conversation benchmark drawn from real user-AI interactions exposing the domain gap between chatbot and social-media toxicity. EMNLP 2023.


Mitigations:

Architectural layers (MAESTRO):