Title: SelfGrader: Detecting Jailbreak Attacks on Large Language Models with Token-Level Logit Distribution
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Jailbreak Attack, LLM, Guardrail

Abstract: Large Language Models (LLMs) are powerful tools for answering user queries, but remain highly vulnerable to jailbreak attacks. Existing guardrail methods typically rely on internal features or textual responses to detect malicious queries, which either introduce substantial latency or suffer from the randomness in text generation. To overcome these limitations, we propose $\textbf{SelfGrader}$, a lightweight guardrail method that solves jailbreak detection as a numerical grading problem using token-level logits. SelfGrader prompts the LLM to evaluate an input query with a compact set of {numerical tokens} (NTs) (e.g., 0–9) and interprets their logit distribution as an internal safety signal. To align these signals with human intuition of maliciousness, SelfGrader introduces a novel in-context prompting strategy and a dual-perspective scoring rule that considers both the maliciousness and benignness of the query, yielding a stable and interpretable score that both reflects harmfulness and reduces the false positive rate. Extensive experiments across diverse jailbreak benchmarks, multiple LLMs, and state-of-the-art guardrail baselines demonstrate that SelfGrader achieves accurate and robust detection while maintaining low computational overhead and latency.