T hink G uard: Deliberative Slow Thinking Leads to Cautious Guardrails (2025)

Abstract: Ensuring the safety of large language models (LLMs) is critical as they are deployed in real-world applications. Existing guardrails rely on rule-based filtering or single-pass classification, limiting their ability to handle nuanced safety violations. To address this, we propose ThinkGuard, a critique-augmented guardrail model that distills knowledge from high-capacity LLMs by generating structured critiques alongside safety labels. Fine-tuned on critique-augmented data, the captured deliberative thinking ability drastically enhances the guardrail’s cautiousness and interpretability. Evaluated on multiple safety benchmarks, ThinkGuard achieves the highest average F1 and AUPRC, outperforming all baselines. Compared to LLaMA Guard 3, ThinkGuard improves accuracy by 16.1% and macro F1 by 27.0%. Moreover, it surpasses label-only fine-tuned models, confirming that structured critiques enhance both classification precision and nuanced safety reasoning while maintaining computational efficiency.

## introduction recent years have seen large language models (llms) demonstrate remarkable emergent capabilities @xcite , enabling their use as the backbone of a wide range of intelligent systems @xcite @xcite . therefore, ensuring do not generate harmful content @xcite @xcite has become increasingly critical for their safe deployment. guardrails @xcite @xcite , acting as external safety layers that detect and filter harmful inputs or outputs of llms, have gained attention for their flexible deployment across different llms and their effectiveness in mitigating risks. existing guardrails are typically framed as a discriminative task: given an input (and sometimes an output), they classify it as safe or unsafe or assign it to specific harmful content categories. these methods can be categorized into rulebased and model-based approaches. rule-based guardrails rely on predefined lists of forbidden words or rules to detect harmful content. while simple and efficient, they struggle to adapt to evolving threats and varying guidelines @xcite @xcite . model-based guardrails, on the contrary, are developed by continually finetuning pretrained language models on instruction-based safety datasets, enhancing their contextual awareness and generalizability @xcite . however, many still approach guardrails as a simple classification task, merely predicting labels rather than engaging in deliberative reasoning about intent, context, and potential risks. as a result, these models often fail to distinguish between genuinely harmful content and benign yet sensitive statements, making them particularly vulnerable to adversarial prompts that exploit models' weaknesses @xcite @xcite . moreover, since these models only output labels without explaining their decisions, it is difficult to interpret why a particular prediction was made, limiting transparency and trust in their decisions. to address these limitations, we propose think-guard, a model that enhances guardrails through cautious, deliberative processing. our method is inspired by the cognitive distinction between fast and slow thinking in human decision-making @xcite . while fast thinking, such as simple end-to-end classification, enables quick and intuitive predictions, it often leads to oversimplified or inaccurate assessments, making models vulnerable to adversarial inputs @xcite . in contrast, slow thinking, through deliberative processing, mitigates these issues by enabling more robust, context-aware decision-making @xcite . additionally, it improves transparency by providing detailed explanations for predictions, making it easier for humans to interpret and trust the model's decisions. specifically, as shown in the fig. 1 , we conduct mission-focused distillation from existing llms @xcite by prompting them to engage in slow, step-bystep reasoning before making a prediction. this process generates augmented safety datasets that include not only labels but also detailed reasoning steps. to integrate this deliberative reasoning into guardrail models, we finetune them using a two-round conversational format: in the first round, the model produces an initial prediction, and in the second round, it articulates its reasoning. this approach achieves performance comparable to standard chain-of-thought fine-tuning @xcite while offering an additional advantage-users can opt to receive only the final prediction, preserving the efficiency of traditional llm-based guardrails while retaining interpretability when needed. we evaluate thinkguard on multiple safety benchmarks, where it achieves the highest average f1 and auprc, surpassing all baselines. compared to llama guard 3, it improves accuracy by 16.1% and macro f1 by 27.0%, demonstrating the benefits of critique-augmented fine-tuning. thinkguard also outperforms label-only finetuned llama guard 3, confirming that structured critiques enhance both classification precision and nuanced safety reasoning. these results highlight thinkguard's strong performance and efficiency, making it well-suited for real-world deployment. early guardrail models relied on rule-based filtering @xcite @xcite , which uses predefined keyword lists and heuristic constraints to identify harmful content. while these methods are transparent and efficient, they suffer from rigidity and poor adaptability, leading to often issues of false positives (overblocking benign content; @xcite ) and false negatives (failing to detect nuanced harms; @xcite ). due to these limitations, research has shifted towards llmbased classifiers, which provide greater flexibility by leveraging large models for content assessment. llm-based safety classifiers, such as llama guard @xcite @xcite and aegis guard @xcite , improve moderation by fine-tuning models on safety datasets, enabling them to classify inputs according to predefined safety guidelines. unlike rule-based approaches, these classifiers leverage large models to capture different categories of harmful content, making them more adaptable to diverse regulatory requirements and shifting safety standards. however, most llm-based guardrails operate in a single-pass classification manner, resulting in their providing binary moderation decisions without deeper analysis. this approach makes them vulnerable to adversarial prompts and subtle harmful content that requires nuanced reasoning @xcite . recent research, such as shieldgemma @xcite and wildguard @xcite , attempts to improve guardrail models through larger training datasets and fine-tuning on adversarial examples, but challenges in interpretability and robustness persist. llm inference with slow thinking the concept of "slow thinking" in large language models is inspired by the psychological dual-process theory @xcite , which differentiates between fast, intuitive thought processes (system 1) and slow, deliberate reasoning (system 2). integrating slow thinking into llms aims to enhance their reasoning capabilities by enabling more deliberate and structured problem-solving approaches. recent research has explored various methods for implementing "slow thinking" in llms' decision making. for instance, @xcite evaluated the performance of proprietary models such as gpt-3 on cognitive reflection tests (crt) and found that these models exhibited behavior similar to human intuition, as well as the cognitive errors that come with it. in a related effort, openai introduced o1 (openai, 2024c), a model trained to deliberate longer before responding, thereby improving performance on complex tasks. unlike earlier models that often required explicit chain-of-thought prompting, o1 naturally engages in step-by-step reasoning and dynamically adapts its responses to context. in addition, @xcite proposes a framework for imitation, exploration, and self-improvement that significantly improves the model's performance on complex reasoning tasks through a reward-guided tree search algorithm. another notable work is by @xcite , which adopts a dual-mind mechanism, using two generative llms to handle fast and slow thinking respectively. the fast thinking model is responsible for initial response generation, while the slow thinking model carries out detailed planning, reasoning, and tool use to provide a wellconsidered answer. ## thinkguard in this section, we introduce thinkguard, a critique-augmented guardrail model designed to enhance safety classification and justification. ## experiments to evaluate thinkguard, we conduct experiments on multiple safety benchmarks, focusing on (1) safety classification performance, (2) comparative analysis of model capabilities, and (3) the impact of data scaling on accuracy and reasoning ability. ## conclusion we present thinkguard, a critique-augmented guardrail model that integrates structured critiques to improve both safety classification and interpretability. thinkguard achieves the highest average f1 and auprc across multiple safety benchmarks, outperforming existing baselines. it improves f1 by 14.1% on toxic chat and 6.1% on wildguardmix compared to llama guard 3, demonstrating enhanced robustness against adversarial and ambiguous cases. additionally, it generalizes better than label-only fine-tuned models, particularly on datasets with diverse harm categories. future work will explore extending critique-based fine-tuning to broader safety tasks and optimizing efficiency for real-world deployment. we hope thinkguard contributes to the development of more transparent, accountable, and effective ai safety mechanisms.