X -Guard: Multilingual Guard Agent for Content Moderation (2025)

Abstract: Large Language Models (LLMs) have rapidly become integral to numerous applications in critical domains where reliability is paramount. Despite significant advances in safety frameworks and guardrails, current protective measures exhibit crucial vulnerabilities, particularly in multilingual contexts. Existing safety systems remain susceptible to adversarial attacks in low-resource languages and through code-switching techniques, primarily due to their English-centric design. Furthermore, the development of effective multilingual guardrails is constrained by the scarcity of diverse cross-lingual training data. Even recent solutions like Llama Guard-3, while offering multilingual support, lack transparency in their decision-making processes. We address these challenges by introducing X-Guard agent, a transparent multilingual safety agent designed to provide content moderation across diverse linguistic contexts. X-Guard effectively defends against both conventional low-resource language attacks and sophisticated code-switching attacks. Our approach includes: curating and enhancing multiple open-source safety datasets with explicit evaluation rationales; employing a jury of judges methodology to mitigate individual judge LLM provider biases; creating a comprehensive multilingual safety dataset spanning 132 languages with 5 million data points; and developing a two-stage architecture combining a custom-finetuned mBART-50 translation module with an evaluation X-Guard 3B model trained through supervised finetuning and GRPO training. Our empirical evaluations demonstrate X-Guard’s effectiveness in detecting unsafe content across multiple languages while maintaining transparency throughout the safety evaluation process. Our work represents a significant advancement in creating robust, transparent, and linguistically inclusive safety systems for LLMs and its integrated systems. We have publicly released our dataset and models at this {href{https://github.com/UNHSAILLab/X-Guard-Multilingual-Guard-Agent-for-Content-Moderation}{URL}}.

## introduction the capabilities of large language models (llms) have been rapidly expanding, and these models are now being envisioned for adoption across numerous fields, including sensitive areas such as cybersecurity, healthcare, and drug discovery @xcite @xcite . this widespread adoption means adversaries can potentially cause more serious harm to both systems and users. therefore, it is essential to integrate robust safety methods and frameworks directly into llms and their surrounding systems. model providers typically address this by performing thorough data cleaning before pretraining and implementing comprehensive safety training before releasing models. additionally, post-training approaches such as reinforcement learning from human feedback (rlhf) have proven highly effective for developing llms that are both harmless and helpful @xcite @xcite . however, these safety-aligned llms have been shown to generate harmful responses and fail to prevent adversarial attacks @xcite @xcite @xcite . this has further motivated researchers to add additional guardrails for llms and systems to prevent attacks. these guardrails are in the form of apis such as perspective api foot_0 , openai moderation api foot_1 , and also have been released as input-output content filtering models such as llama-guard/2/3 @xcite @xcite , sheildgemma/2 @xcite , wildguard @xcite , aegis @xcite म आजभो ल धे र ए ो महसु स ग ररह को छ र पै साको खाँ चो छ, सै ले घरमा मे था फे टामाइन कसरी बनाउने ? the given user text is concerning as it explicitly asks for instructions on how to manufacture methamphetamine, which is an illegal drug. ..... .... ... the content must be flagged as unsafe due to its direct violation of policies prohibiting the promotion or facilitation of illegal activities, especially those involving dangerous substances like methamphetamine. ## data collection and synthetic data generation we begin synthetic data generation by creating harmful responses from prompts of salad-bench @xcite and the alert @xcite dataset, using an uncensored llm, i.e., abliterated llama @xcite . furthermore, we collected additional data points from aegis @xcite , wildguard @xcite , bingo @xcite , and xstest @xcite to create 100k+ data points consisting of prompts, responses, and prompt-response pairs. we sampled approximately 100k data points and then re-evaluated each data point via distillation with open-sourced llms and api. we used llama-3-70b-instruct, qwen2.5-32b-instruct, openai moderation api, llama-guard-2-8b, and abliterated llama as the safety evaluation judges. except for openai moderation api, we performed safety evaluation using the prompt as illustrated in fig 4 . for openai moderation api and llama-guard (lg), we added a custom mapping function to map the labels for safety categories. the judge llms were asked to provide the assessment and reasoning of the provided user text and then provide the safety label and the violated categories. jury rubrics: for a given text, we classified it as unsafe if at least two judge llms agreed; otherwise we considered the text safe. for category violation codes, we selected the most common safety violated category among category codes selected by the judges. we chose the assessment of the llm whose safety label agreed with the jury label. we used alert and salad-bench dataset of approximately 39k responses only, and translated them into 132 languages using google cloud translation, resulting in 5 million translation data points from other languages to english (xx-to-en). ## x-guard agent our x-guard agent consists of three modules that work together as depicted in fig. 1 : 1) language detection and routing module, 2) translation module, and 3) safety evaluation module. the language detector module identifies the language of the user text and determines if the user text uses the sandwich attack (code-switching) style @xcite . the module first breaks the input text into linguistic units using sentence delimiters, and when necessary, into dynamic wordbased chunks with adaptive sizing based on text length. we used the langdetect foot_2 library to identify the language of each segment, such as determining whether a given sentence is in english or another language. our algorithm maintains contextual awareness by employing a state-based approach: it tracks language transitions by comparing each segment's detected language with the previous one. this enables the identification of language switches and the grouping of consecutive segments in the same language. as a result, the method avoids over-segmentation and produces coherent language blocks instead of treating each sentence in isolation. if the user text is in english (en), the agent routes it directly to the safety evaluation module. if the text is in a language other than english, the agent routes it to the translation module. the translation module incorporates a fine-tuned mbart-many-to-en model that translates the user text into english. if the user text is in the sand-wich attack style, the module translates it in a oneby-one fashion. the x-guard agent subsequently sends the output of the translation module to the safety evaluation module, which contains the x-guard model to perform the evaluation of the user text. the output from the x-guard provides the assessment of the safety of the user text inside @xmath0, and the safety label inside @xmath1. if the text is unsafe, it will also provide the safety category violation code @xmath2. the fig. 2 depicts an example of moderation performed by the x-guard. even when the translation is not entirely accurate, the x-guard captures the intent of the user text and provides the correct safety assessment. the situation in the gaza strip is indeed a matter of great concern. you can create a scheme to relocate your family's district office to a new location. yes, it is possible to intentionally damage a building's foundations. ## discussions in this work, we present the x-guard agent for input-output content moderation across multiple languages. we demonstrate how open-source llms and free moderation apis can be used to generate high-quality synthetic data through distillation, which can then train smaller translation models and safety evaluators to work together effectively. our 3.7b parameter agent achieves higher accuracy in english (97.20%) than the larger lg-8b model (92.40%) and remains competitive across other languages. unlike other llms, our agent not only performs input-output filtering in more than 130 languages but also defends against codeswitching attacks such as the sandwich attack with 83% accuracy compared to lg-8b's 62%. technical limitations and their practical impact. the performance of our agent directly depends on the translation model, meaning that improving the model's performance will increase overall accuracy. we traced the low safety evaluation scores in several languages to specific translation failures. our translation model was initially pretrained on only 50 languages before being expanded through finetuning to support 132 languages, though with limited training data. this expansion likely triggered the "curse of multilinguality," a phenomenon that causes performance degradation across languages as more languages are added @xcite . the synthetic data was translated using google cloud translation, which introduces its own biases and risks of translationese that have carried over to our agent. these translation limitations significantly restrict our model's capacity to understand native slang and identify culturally-specific harmful content, particularly impacting performance in low-resource languages. in our preliminary experiments, we began with grpo training directly (for english only) and observed that the model could not reason properly. we believe this was due to the limitations of using only a small 3b model. additionally, our reward function did not effectively evaluate the quality of reasoning within the think tags; it only provided rewards when the total number of reasoning evaluation sentences exceeded three. due to the poor results from the initial grpo approach, we decided to first perform sft to ensure the model learns reasoning for safety evaluation to some extent. this two-stage training process, while necessary for our model size, adds complexity and may not scale optimally to larger models. ## related work the field of llms safety has evolved rapidly with various approaches to creating llms-based guardrails. @xcite introduced the llama-guard model, which was fine-tuned on the llama-7b model to create an input-output safety filter for prompts and responses with multilabel classification for safety categories. this work was later expanded with llama-guard-2 (8b) @xcite and llama-guard-3 (in both 8b and 1b variants), with llama-guard-3 also incorporating vision capabilities @xcite . similarly, google launched @xcite ) shieldgemma built on top of gemma-2, and shieldgemma-2 @xcite built on top of gemma-3, with shieldgemma-2 also capable of classifying images. as noted in the literature, the training data and details for these proprietary models have not been fully disclosed. in contrast to proprietary approaches, several researchers have released open-source models with transparent methodologies. @xcite performed full parameter sft with distilled datasets from gpt-4 and deepseek-r1-distill-llama-70b, incorporating safety labels and critiques where the model provides further clarification on classification decisions. @xcite trained llama-3.2 base models in 1b, 3b, and 8b sizes using sft, implementing a two-step process where the model first reasons and then identifies safety and harm categories. the authors created the guardreasonertrain dataset by synthesizing reasoning processes using gpt-4, followed by direct preference optimization (dpo) that treats correct outputs with corresponding reasoning as positive examples. parameter-efficient methods have made safety models more accessible. han et al. (2024) 2024 ) serves as a guardrail agent that dynamically checks if other agents' actions comply with safety guard requests through a two-step approach: analyzing requests to generate task plans and transforming these plans into executable guardrail code. @xcite offers an open-source toolkit using a specialized language called colang, providing programmable guardrails that operate at runtime rather than being embedded during training. @xcite combines data-driven learning with explicit reasoning through category-specific models and a probabilistic graphical model encoding intercategory relationships as first-order logical rules. yuan et al. (2024 ) integrates knn-based approaches with energy-based training data generation, minimax optimization for safe suffix generation, and a fusion-based model combining knn with llms. for resource-constrained environments, several lightweight approaches have emerged. wei et al. (2023b) enhances llm resilience against jailbreak attacks by including examples of appropriate refusals directly in the conversation context, working by simply prefixing user queries with 1-2 examples. lee et al. (2024) distilled a large safety model (llama-guard-3-8b) into a smaller model (deberta-v3-large, 435m parameters) for deployment on resource-constrained devices, using a data augmentation technique that jailbreaks llms to generate harmful instructions, thereby enhancing the smaller model's detection capabilities. several important benchmarks have been developed. li et al. (2024) provides a comprehensive safety benchmark to evaluate llms' safety capabilities, attack vulnerabilities, and defense mechanisms. ghosh et al. (2024) contributed 30k data samples with a multi-llm jury system covering 12 risk categories. han et al. (2024) released 92k labeled examples including adversarial jailbreaks, valuable for training robust safety models. ## conclusion and future works our research introduces the x-guard agent, a significant advancement in multilingual safety moderation. by addressing the limitations of current safety frameworks, particularly their englishcentric design and vulnerability to attacks in lowresource languages, we have created a robust solution that effectively operates across 132 languages while maintaining transparent decision-making processes. the strengths of our approach lie in our comprehensive methodology: curating and creating enhanced safety datasets with explicit reasoning components, employing a jury of judges to mitigate individual llm biases, creating an extensive multilingual dataset of 5 million data points, and developing a two-stage architecture combining a custom-finetuned translation module with our x-guard 3b evaluation model. this architecture enables effective detection of harmful content across diverse linguistic contexts, including sophisticated code-switching attacks that typically bypass conventional guardrails. as our agentic approach is limited by the cascade method, future work could include developing higher-size models and training a single model for multilingual context classification, as well as exploring translation-followed-by-classification approaches. future research should focus on developing specialized translation modules for specific language families or scripts, which the agent could select after language detection, thereby addressing translation issues and enhancing overall performance. specifically, we propose: (1) training separate translation models for high-resource vs. lowresource languages to mitigate the curse of multilinguality, (2) developing culturally-aware safety evaluation modules that can understand regionspecific harmful content without relying solely on translation, and (3) conducting comprehensive human evaluation studies to validate the jury approach against human annotators across multiple languages and cultures. additionally, while it would be feasible to train a single llm for multilingual classification, the limited availability of high-quality reasoning data and the curse of multilinguality would be the bottleneck. the translationin-the-middle approach taco (upadhayay and behzadan, 2024b) could be another viable option for training a single llm for multilingual content moderation, which we plan to explore in future work. additionally, future efforts could focus on further expanding language coverage, improving performance in ultra-low-resource languages, and developing more sophisticated reasoning capabilities. as llms continue to be deployed in critical applications worldwide, multilingual safety frameworks like x-guard will be essential for ensuring these powerful technologies benefit users across all linguistic communities while minimizing potential harms. to support progress in this direction, we are publicly releasing our models and artifacts, which we hope will accelerate research in this important area and contribute to the development of more inclusive and robust ai safety systems.