Unlocking LLM Safeguards for Low-Resource Languages via Reasoning and Alignment with Minimal Training Data (2025)

Abstract: Recent advances in LLMs have enhanced AI capabilities, but also increased the risk posed by malicious requests, highlighting the need for effective LLM safeguards to detect such queries. Existing approaches largely rely on classifier-based methods that lack interpretability and perform poorly on low-resource languages. To address these limitations, we propose ConsistentGuard, a novel reasoning-based multilingual safeguard, which enhances explainability via reasoning and boosts knowledge transfer between languages through alignment. With only 1,000 training samples, our method demonstrates superior performance on three datasets across six languages, outperforming larger models trained with significantly more data, and exhibits strong interpretability and generalization ability. We also contribute a multilingual benchmark extension and release our code to support future research.

## introduction recent advances in large language models (llms) have brought ai applications to a new height, which also makes the defense against malicious prompts increasingly critical. llm safeguards aim at detecting malicious prompts from users and identifying harmful generations from agents. most previous methods work in a simple classifier manner, e.g., llama guard (genai, 2023), shieldgemma @xcite , etc. therefore, making the results less explainable and lacking evidence @xcite . moreover, though these models have superior performance on mainstream languages, it has a significant performance drop on low-resource languages, such as bengali @xcite . to mitigate such issues, recent research has tried to incorporate models' reasoning ability with chainof-thought (cot) prompt engineering @xcite or reinforcement learning (rl), such as guardreasoner @xcite . although these reasoning-based models perform well in providing both evidence and classification results, most of them are trained on a single mainstream language, ignoring their reasoning consistency across languages and leading to a drop in cross-lingual performance. for models' cross-lingual performances, prior research has primarily focused on enhancing their cross-lingual performance by continued pretraining or through alignment methods with supervised fine-tuning (sft) @xcite . more recent research has introduced direct preference optimization (dpo) @xcite alignment for qa tasks @xcite , demonstrating remarkable generalization ability. however, most prior methods have ignored the issue of reasoning inconsistencies across languages, specifically for reasoning models, and the potential of rl for crosslingual alignments still remains largely unexplored. detailed related work is included in app. a. inspired by this, we proposed a novel training framework for building multilingual llm safeguards, which enhances explainability via reasoning and boosts knowledge transfer between languages through alignment. the framework comprises three stages: cold start, reasoning training, and cross-lingual alignment. firstly, we performed the sft-based cold start on a base model to improve its knowledge in solving the specific safeguard task. then, we performed reasoning training via group relative policy optimization (grpo) @xcite , in which we designed two novel rewards to balance length and diversity of the reasoning process. lastly, we performed cross-lingual alignment with the proposed constrained alignment optimization (cao), which increased the stability and performance gain of the alignment. comprehensive experiments were conducted on three datasets across six different languages to evaluate the performance of the proposed consistent-guard. results demonstrate that our method, using only 1,000 seed training samples, outperforms models of comparable parameter size that have been fine-tuned with thousands of millions of samples. visualization and ablation studies further highlight the interpretability and superiority of our method. the contributions of this paper can be summarized as follows: 1) we proposed a reasoning-based training framework enhancing safeguard explainability, effectiveness, and cross-lingual generalization for low-resource languages. 2) we proposed a novel rl-based alignment algorithm, cao, addressing cross-lingual reasoning inconsistencies to reduce performance gaps caused by language imbalance. 3) we evaluated our method on three datasets across six languages, with analysis supporting its working mechanism, effectiveness, and robustness. 4) we released a reasoning-based multilingual safeguard training code and extended three existing english safety benchmarks foot_0 to six languages to support research in this field. ## methodology the general training framework of consistent-guard is illustrated in the fig. 1 . the proposed method comprises three main training stages. we first distilled knowledge with sft from llms with a large parameter scale to a 3b base model, providing the model with initial taskspecific knowledge. then, in the reasoning training stage, we chose grpo as our core algorithm and designed novel rewards based on simple functions, which promote reasoning diversity and length. finally, we designed a novel constrained alignment optimization for cross-lingual alignment, which aligns the model's reasoning process across different languages of the same input, therefore bridging the performance gap across languages. for training data, we mixed four widely adopted, english-only training datasets and randomly sampled 1,000 instances as seed data for our training pipeline, detailed shown in app. c. ## conclusion this work presents a multi-stage training framework combining distillation, reinforcement, and alignment to tackle performance insufficiency and imbalance in multilingual safeguard task. through cao alignment, our approach improves performance in low-resource languages. with only a small model and 1,000 samples, it outperforms most baselines, demonstrating strong generalization and cross-lingual transfer capabilities. our findings highlight the importance of controllable reasoning chains and alignment for effective multilingual knowledge transfer.