PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Authors: Priyanshu Kumar
Devansh Jain
Akhila Yerukola
Liwei Jiang
Himanshu Beniwal
Thomas Hartvigsen
Maarten Sap
Venue: COLM 2025 (2025)
Keywords: ai safety
hate-speech detection
TL;DR: We introduce PolyGuard, a new state-of-the-art multilingual safety model for safeguarding LLM generations along with PolyGuardMix for safety detection training and PolyGuardPrompts for safety guardrail evaluation.

Abstract: Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by a narrow focus on a small set of languages (e.g., English, Chinese) as well as a limited scope of safety definition, resulting in significant gaps in moderation capabilities. To bridge these gaps, we release POLYGUARD, a new state-of-the-art multilingual safety model for
safeguarding LLM generations, and the corresponding training and evaluation datasets. POLYGUARD is trained on POLYGUARDMIX, the largest multilingual safety training corpus to date containing 1.91M samples across 17 languages (e.g., Chinese, Czech, English, Hindi). We also introduce POLYGUARDPROMPTS, a high quality multilingual benchmark with 29K samples for the evaluation of safety guardrails. Created by combining naturally occurring multilingual human-LLM interactions and human-verified machine translations of an English-only safety dataset (WildGuardMix; Han et al., 2024), our datasets contain prompt-output pairs with labels of prompt harmfulness, response harmfulness, and response refusal. Through extensive evaluations across multiple safety and toxicity benchmarks, we demonstrate that POLYGUARD outperforms existing state-of-the-art open-weight and commercial safety classifiers by 5.5%. Our contributions advance
efforts toward safer multilingual LLMs for all global users.

Published as a conference paper at COLM 2025
PolyGuard: A Multilingual Safety Moderation Tool for 17 Languages
Priyanshu Kumar 91
Devansh Jain 91
Akhila Yerukola
Liwei Jiang
Himanshu Beniwal AQ
Thomas Hartvigsen +
Maarten Sap x
@ Carnegie Mellon University
University of Washington
A IIT Gandhinagar
University of Virginia
Allen Institute for AI
Abstract
Truly multilingual safety moderation efforts for Large Language Models (LLMs) have been hindered by narrow focus small set of languages (e.g., English, Chinese) as well as limited scope of safety definition, result- ing in significant gaps in moderation capabilities. To bridge these gaps, we release POLYGUARD, a new state-of-the-ar multilingual safety model for safeguarding LLM generations, and the corresponding training and eval- uation datasets. POLYGUARD is trained on POLYGUARDMIT the largest multilingual safety training corpus date containing 1. M.91M samples across 17 languages (e.g., Chinese, Czech, English, Hindi). We also introduce POLY- GUARDPROMP'H high quality multilingual benchmark with 29K samples for the evaluation of safety guardrails. Created by combining naturally occurring multilingual human-LLM interactions and human-verified ma- chine translations of an English-only safety dataset (WildGuardMix; Han et al., 2024), our datasets contain prompt-output pairs with labels of prompt harmfulness, response harmfulness, and response refusal. Through extensive evaluations across multiple safety and toxicity benchmarks, we demon- strate that POLYGUARD outperforms existing state-of-the-arr open-weight and commercial safety classifiers by 5.5%. Our contributions advance efforts toward safer multilingual LLMs for all global users.
PolyGuard Collection
fitiyanshu256/polyguaaa
1 Introduction
Recent advances in large language models (LLMs), especially their multilingual capabilities, have led to their deployment diverse global user base that spans multiple languages. Despite this global reach, safety research has focused primarily on the English language (Ghosh et al., 2024; Ghosh al.; Han et al., 2024), exposing global users to potential safety risks such harmful content and privacy violations. For instance, studies have shown that multilingual models are more likely to generate hate speech, disinformation, and harmful content when prompted in non-English languages (Kotha et al., 2023; Jain et al., 2024).
The development of robust multilingual safety systems presents several key challenges. First, building multilingual systems is inherently difficult due to challenges such the lack of comprehensive datasets, the "curse of multilinguality" (Aharoni et al., 2019; Conneau et al., 2020; Gurgurov et al., 2024), and the inherent biases embedded in training corpora (Xu et al., 2024). Second, existing multilingual efforts have been limited in their (a) scope by focusing either on a subset of safety (e.g., PerspectiveAA covering only toxicity, ignoring other unsafe content) and/ or on à narrow set of language coverage (e.g.,
Equall contributors, correspondence at msap28cs cmu. edu.

---

Published as a conference paper at COLM 2025
only covering English safety, ignoring toxicity and DuoGuard being evaluated on 4 very high resource languages only; Inan et al., 2023; Jain et al., 2024; Deng et al., 2025), or (b) performance (e.g., Luama-Guard-3 which struggles et al., 2024; PatronusAT, 2024). Finally, most existing safety frameworks address only the single task of classifying safety and often rely on simplistic binary settings (safe/ /unsaff)) which fail to capture the complex spectrum of harmful content that can manifest differently across cultural and linguistic contexts (Sap et al., 2020; Zhou et al., 2023).
multilingual benchmarks; Dubey
To address these gaps, we release POLYGUARD ARD (PG), a new state-of-the-art fine-tuned language model for multi-task safety detection and moderation. As Figure highlights, PG can classify a multilingual input of user prompt and an LLM response on five dimensions.
We also release the first large-scale mul- tilingual corpora for safety detection training, POLYGUARDMIX (PGMix) and safety guardrail evaluation, POLYGUARD- PROMPTS (PGPrompts), comprising 1.91M and 29K user prompt LLM output pairs, respectively, across 17 languages. Our datasets contain binary and categorical la- bels for prompt harmfulness and response harmfulness, and response refusal (if the LLM response complies with the user request). We use systematic labeling process that leverages a panel of English safety classi- fiers and a-d--a-- (proprietary and open-weight LLM) to obtain these labels.
Inputs in 17 languages
para
aebutgeting
(llest actutties ane race creetine you ou reatice sill untra uncads.
Faterggea teeia pringss
crodcct werden?
T
con mesu saluchee bet gao. mociices bestin Slleglle lechert an seter riger wyy suto wa salerii
frowent
misht io gratin det lpen eratation.
de W:: ruch vnik unaan wech son in oy wooh it-t for it antty ott Pi in antt 8
aeot de
at sore 11.8
up
PolyGuard
trained with
Translated WildGuard
In-the-witd multilingual prompts
We create our PGMix dataset by combining both: (a) naturally occurring multilingual human-LLM interactions from In-The-Wiin (ITW) datasets, and (b) machine transla- tions WildGuardMix (Han et al., 2024), to ensure data diversity which is crucial for im- proved model performance (Davani et al., 2024). We utilize multiple LLMs to ensure high-quality translations of WildGuard verified by high average translation score of 81.15 as rated by human annotators.
Hommful Request pas
Anquest vistattoe
waw--olleeeemm crner
Hoomful Beaparss: you
Anquist vistatie
Resporse refusat no
Figure 1: POLYGUARD takes in a user prompt and model response (optional) and lists the safety labels, violations, and model compli- ance following the same safety taxonomy as Llama-Guard-3 (Llama Team, 2024). Take- away: POLYGUARD classifies inputs in 17 differ- ent languages on five different dimensions.
We then use PGMix to train our state-of-the- art POLYGUARD (PG) models, including a fast lightweight model for application use cases. Our empirical results show that PG outperforms existing open-source and proprietary safety detectors on English-only as well as multilingual safety and toxicity benchmarks. Furthermore, we find that the incorporation of ITW samples in the training datasets makes PG models more robust to various data distributions, including code-switched and translated data.
Overall, our datasets and models? serve starting point for building powerful and robust multilingual safety detectors and advance efforts towards multilingual safe Al systems.
2 Dataset
To address the critical need for multilingual safety detection, we introduce POLYGUARDMi' (PGMix) and POLYGUARRPROMPTS (PGPrompts), multilingual datasets specifically de- signed to train and evaluate robust safety classifiers. PGMix comprises 1.91M human-LLM interactions, including 1.47M machine-translated samples from WildGuardMix and 0.43M
Model, code, and data are available under the QDC-BY license.
2

---

Published as a conference paper at COLM 2025
naturally-occurring samples from In-The-Wild datasets, whereas PGPrompts comprises 29K translated samples.
Our datasets cover 17 languages: Arabic (ar), Chinese (zh), Czech (cs), Dutch (nl), English (en), French (fr), German (de), Hindi (hi), Thai (th), Italian (it), Japanese (ja), Korean (ko), Polish (pl), Portuguese (pt), Russian (ru), Spanish (es), and Swedish (sv). This diverse linguistic coverage ensures the representation of languages that span multiple language families and writing systems, facilitating the development of more inclusive safety systems.
Figure 2 shows an overview of our data curation pipeline, whose components we describe in detail in the following subsections.
2.1 Data Sources
Both PGMix and PGPrompts are constructed from the train and test samples of Wild- GuardMix (Han et al., 2024), a dataset of synthetic and nat- ural single-turn human-LLM interactions with fine-grained annotations, respectively, In addition, PGMix also con- tains samples from In-The- Wild datasets: LMSys-Chat- 1M (Zheng et al., 2023) and WildChat (Zhao al., 2024)² We posit that the combination of natural and synthetic sam-
LMG
Inche-Wad
PoyGundils
Rupentee Tin-Ture
Tonndatioe
Sobtyy
GP144 Agent
PuyounnPrennts (29)
(296))
Figure 2: Data curation process for PGMix (safety detec- tion training) and PGPrompts (safety guardrail evaluation). Takeaway: PGMix combines tachine-dr and natu- rally occurring data to improve data diversity and, conse- quently, model performance.
ples improves the diversity of data and consequently improves model performance (Davani et al., 2024).
2.2 Machine Translation Pipeline
We develop an efficient machine translation pipeline using open-weight models to minimize computational costs when translating WildGuardMix for our training data. We employ two state-of-the-art translation models: Iowerrnstruct-7B-W
(Alves et al., 2024) and NLLB-3. 3B (Team al., 2022). For optimal performance, we utilize Tower to translate content into its nine supported languages, where it consistently outperforms NLLB-3. 3B. We then leverage NLLB-3. 3B for the remaining languages, as it has wider language coverage, and TowerInstruct 7B--0. 2 exhibits performance degradation on these out-of-distribution samples. To ensure high-fidelity translations for evaluation, we use GPT-4o in an agentic framework (Ng) to translate the WildGuardMix Test split. We provide details about our translation pipelines and automated quality assessment in Appendix A.
2.3 Safety Annotation
We leverage a panel of English safety classifiers and LLM-as-judges to annotate safety violation categories automatically. We follow Llama-Guard-3--8 (Dubey et al., 2024) and define our safety violation taxonomy according to the MI.Commons Safety Taxonomy". We label English WildGuardMix samples using L -laam-uurdd3----- and GPT-4o as a judge to obtain multiple annotations, thus reducing biases from single model. Furthermore, we use the existing WildGuardMix binary labels and Llama3. 1-405B-Instruct (Dubey et al., 2024) as judge to resolve conflicts and obtain the final annotations Finally, since PGMix and PGPrompts contain translations of WildGuardMix, we propagate safety labels from the
WillCCatt-M is available for modifications under the ODC-BY license. 4https:/ /mmcomons. org htgg//24//4/mmcmma/ via-aaaettttaaae--555555 5-po/ We the same prompt as Haaaa-Guard-3-80 for all L.MM-asjudges.
3

---

Published as a conference paper at COLM 2025
Model
Model
GPT-4o
15000
GPT-4o
15000
Number of samples
Guard
Number of samples
Combined
Combined
10000
10000
5000
Non-ioolent Crimes
Defamation
Privacy
Violent Crimes
Self-Harm
indiscriminatt Weapons
Special Hale
Intellectuall Property
Sexual Content
Exploitation
Crimes
Elections
Speciallzedd Aovice
Self-ttamm
Non-Vielent Crimes
Defamation
Privacy
Violent Criines
Hate
Sexual Content
Sex Crimes
Child Exploitation
Intellectual Property
Cods Interpreter Abuso
indiscriminate Weapons
Elections
Interpreter
Prompt Safety Categories
Model Response Safely Categories
Figure 3: Safety category distribution for user prompts and model responses for WildGuard- Mix train samples. The model name (GPT-40 and Llama-Quard-3-8B) represents the LLM used judge to automatically annotate the safety category. These annotations are then ensembled together, using Llama3.
to break ties (Combined). Takeaway: Final aggregated safety annotations tend to maximize recall.
annotated English samples to other languages. ITW samples contain multilingual prompts and responses, we only GPT-4o for annotation Baaa--Guard-3-B8 performs poorly on multilingual samples.
Figure 3 illustrates the distribution of safety categories across both user prompt harmfulness and model response harmfulness, comparing annotations from Llama-Guard-3--a GPT-4o, and our final consolidated labels. The higher frequency of safety categories in the final annotations stems from Llama3. 1-405B- SUiSB-Instrutt's recall-oriented annotations, which we employed to resolve discrepancies between Llama-Guard-3- the GPT-4o annotated safety categories for the ITW split of our dataset, showing that ITW samples cover different types of unsafe content than WildGuard.Mix non-violent crimes and hate comprise the top-2 categories for WildGuardMix samples, while sex crimes and sexual content comprise the t categories for ITW samples.
and GPT-4o. Figure 4 shows
2.4 Human Validation
To validate the translation quality and the generated safety labels, we conduct human validation across all 16 languages. Due budget constraints, we randomly sample 50 data points per language, ensuring balanced distribution across PGMix (frain) and PGPrompts (lest), harmful and harmless labels, as well as user prompts and model responses. We recruit workers from Prolific, 6 filtering them based on their proficiency in each language. Each data point is evaluated by three annotators.
For each data point, we ask the annotators to assess the following.
1. Translation Quality: Using the Direct Assessment + Scalar Quality Metric (DA+SQM) framework (Kocmi al., 2022), we elicit score between 0 and 100 on continuous sliding scale with seven labeled tick marks.
2. Safety Label for the Source Sentence: Annotators assign a label of either 'harmful' 'safe' for the source sentence in English.
3. Safety Label for the Translated Sentence: Annotators assign 'harmful' or 'safe' label for the corresponding translation.
hnttps://ww.prollffic.com

---

Published as a conference paper at COLM 2025
Irput
Rraporaz
Annotators rated translation quality to be high, with an average score of 81.15 across all 16 languages. The inter- annotator agreement, averaged across all 16 languages, for both source and translated sentence safety labels yielded a Krippendorff's = 0.46. Furthermore, the agreement between the majority-voted source and tar- get safety labels is high, with an average Krippendorff's
30000
30000
10000
Speccated ADACE
Movnitiers Crmas
Xam Creno
Sipual Connent
Prpptty
coe museee Asue Excrens
0.94, indicating that the translations effectively pre- served the original intent of the English source data. We provide details on anguage-specifis scores, the anno- tation scheme, IRB approval, and fair pay in Appendix B.
User hroppt Safddy Categoriza
Figure 4: Safety category distribu- tions for PGMix ITW samples.
3 POLYGUARD: A 17-Language Safety Moderation Tool
To build POLYGUARD, we fine-tune Qwen2. 5-7B-IInstuce (Yang et al., 2024a) and -iasstra--BB-Issructt-
both of which have been shown have state-of-the-art per- formance in multilingual knowledge and commonsense, code, and math settings (Qwen; Mistral). We refer these models as PG Qwen2. and PG Ministral In addition, we also fine-tune Qwen2. 50..BIInnstuc to build PG Smol.
The models are fine-tuned on PGMix using Low-Rank Adapters (Hu et al., 2022). We follow Han et al. (2024) and implement unified text-to-text format for comprehensive safety assessment, which evaluates: (1) prompt harmfulness (binary classification: safe/unsafe /usaaf and categories violated if unsafe), (2) response harmfulness (binary classification: safe/un teneffeafeee and categories violated unsafe), and (3) response refusal (binary classification for compliance with user request). POLYGUARD enables comprehensive safety moderation in 17 major languages. We provide detailed training specifications in Appendix C.
4 Results & Research Questions
multilingual system must be robust; that is, should perform consistently on data belong- ing to different distributions (sources and languages). The performance of multilingual system, in turn, is crucially governed by the distribution training data. Hence, we study the performance of POLYGUARD evaluation benchmarks, and the influence of ITW samples and low-quality translations on model performance. We perform
POLYGUARDPROMPTT and multiple out-of-distribution
run per evaluation due to computational constraints.
Baselines: We compare POLYGUARD with popular open-source safety detection models of similar size (Yang et al., 2024b), namely Llama-Guard-2 (Team, 2024), Llam--Gard-33 (Dubey et al., 2024), Aegis 1.0 Defensive (Ghosh et al., 2024), MD Judge (Li et al., 2024), and DuoGuard (Deng et al., 2025). We also benchmark proprietary models, namely Perspective APP, OpenAl Omni Moderation and Google Moderation
4.1 How do PG models perform on the in-distribution PGPrompts benchmark?
We first evaluate PG and open-source baselines on POL.GUARDDROMPTS benchmark, com- prising 29K samples, using the following metrics: (1) for binary tasks of prompt harmfulness, response harmfulness, and response refusal, we use score for the positive label (unsafe for harmfulness and yes for response refusal), and (2) for the tasks of prompt violations and response violations, we compare the list of ground truth and predicted categories using Exact Match and Jaccard Similarity.
PG models based on Qwen2. 5 and Ministral achieve state-of-the-art performance on PGPrompts with Qwen2.5 performing marginally better. PG Smol outperforms DuoGuard,
7 9nttps://platform opw.ai............m.mmm. wtt.coooocccccc http:///oooooooooooooomoooooooaa
9https://cloud google. htt.://rrwllllaauugugag////////e
5

---

Published as a conference paper at COLM 2025
Harmful Request
Response Refusal
Harmful Response
Prompt Safety
Response Safety
Model
Violations
Violations
F1 Score
Fl Score
F1 Score
Exact Match Jaccard
Exact Match Jaccard
Aegis-Defensive
66.45
MD Judge
43.54
49.12
Llama Guard 2
60.87
63.62
Llama Guard 3
67.98
65.74
71.98
74.59
87.24
88.37
DuoGuard
62.59
37.99
PG Qwen2.5 7B (Ours)
87.12
83.59
74.08
80.87
85.44
86.67
88.79
PG Ministral (Ours)
86.02
84.45
73.75
79.92
84.30
86.85
88.78
PG Smol (Ours)
83.76
81.36
66.82
77.02
81.51
84.05
85.92
Table 1: Evaluation of POLYGUARD models and baselines on POLYGUARDPROMPTS. Take- away: PG models outperform baselines in-distribution data.
its similar size counterpart (Table 1). Aegis Defensive supports only single text input and is hence evaluated for Harmful Request only. Since the remaining baselines do not explicitly support Harmful Respouse, we approximate the prediction by executing them prompt + response. None of the baselines support the Response Refusal task. Out of all baselines, the safety category taxonomy is the same for Llama-Guard- and PG. We observe that Llama-Guard-3 achieves marginally better performance for Response Safety Violations task because it conservatively predicts only one safety category for most of the samples in PGPrompts; PG, on the other hand, predicts multiple violations, thus leading to lower Exact Match and comparable Jaccard similarity scores.
4.2 How does POLYGUARD fare against existing baselines on out-of-distribution multilingual benchmarks?
RTF-
RTP-
XS
XS
MJ
MJ
MJ
MJ
LX
LX
Mod.
Mod.
En.
Mul.
En.
Mul.
En.
Mul.
En.
Mul. (Aegis)
Type
Model
Mul.
Mul.
(Aegis)
(Aegis)
(LG)
(Aegis)
Avg
Aegis- Defensive
84.23
83.21
71.13
59.22
66.59
35.47
69.46
36.75
90.91
79.52
90.61
79.37
70.54
Open
MD Judge
38.60
37.97
58.10
-Weight
Llama Guard 2
39.47
34.99
75.83
72.35
53.70
22.32
50.57
22.56
77.52
62,38
76.86
61.56
54.19
Llana Guard
48.51
79.92
77.52
61.03
Duo Guard
91.83
50.46
70.85
49:44
61.16
26.03
64.83
27.31
89.18
41.84
89.26
41.44
58.66 18.64
Closed
Perspective
97.09
81.97
6.64
44.73
48.73
-Source
OpenAl Ornni
87.52
74.10
74.43
68.08
58.02
22.48
60.11
23.52
82.59
66.94
82.73
66.94
63.95
Google Mod.
83.21
53.89
44.79
83.66
81.00
67.38
PG Qwen2.5
91.34
83.21
74.39
69.31
72.07
35.33
74.93
37.13
93.93
86.44
93.97
86.33
74.88
Ministral
79.58
70.51
95.71
83.11
95.39
83.02
73.87
PG Smol
92.3
71.56
69.3
63.00
70.28
33.22
74.38
35.19
94.39
73.59
93.72
73.34
70.36
Table 2: Fl scores of safety detectors on Multilingual Guardrail Test Suite; metrics are in bold and underlined for the best second-best performing models respectively. Mod.=Moderation, XS=XSafety, Mj=MultiJail, En.=English, Mul.-Muitilingual, LG=Llama Guard. Takeaway: PG models outperform baselines on the Multilingual Guardrail Test Suite benchmarks.
Multilingual Bench: We first benchmark models on datasets inspired by Yang et al. (2024b). This comprises multilingual toxicity and safety datasets, namely RTP-LX (de Wynter et al., 2024), Open Moderation (Markov et al., 2023), 10 XSafety (Wang et al., 2023), and MultiJail (Deng et al., 2024). We mention dataset annotation details in Appendix D, highlighting the need for safety annotations for XSafety and MultiJail benchmarks which measure an LLM's unsafe content generation capability.
Patronus AI Bench: We also evaluate models using the recall score on the benchmarks reported by PatronusAT (2024), consisting of toxic/unsafe unsafe samples from English and multi-
10The OpenAl Moderation dataset comprises only English samples and is extended to multilingual setting using Google Translate.

---

Published as a conference paper at COLM 2025
lingual toxicity and safety datasets. We perform our evaluations on all samples instead of small subset. Appendix E contains details about the benchmark.
Results show that our PG models outperform the baselines most datasets, achieving higher scores for the unsafe class (Table 2). We observe that Perspective API and Google Moderation outperform PG on RTP-LX and XSafety, respectively. This is likely due to the shorter prompts in both datasets, while PG models are trained using longer samples across various safety categories and thus generalize better across different benchmarks. PG models also outperform existing detectors on safety datasets in the Patronus Al benchmark and also achieve the best average performance (Table 3).
thai- toxicity- tweet
toxic- text-en
ukr- toxicity
toxic- text-pt
toxic- chat
Beaver Tails
Salad- Data
Type
Model
jigsaw
Avg
80.32
79.27
62.80
67.29
86.54
91.64
77.98
Open
MD Judge
68.45
73.40
5.80
0.80
56.86
63.54
81.41
96.68
55.87
Llama Guard
23.73
20.67
6.32
4.83
53.51
23.17
59.20
16.14
25.95
-Weight
Llama Guard
40.03
27.20
9.60
11.50
53.78
27.30
52.68
29.42
31.43
Duo Guard
93.65
93.18
0.72
9.27
74.22
54.17
87.54
70.70
60.43
Perspective API
77.20
86.20
93.00
15.89
23.00
1.80
37.14
Closed
OpenAI Omni
54.20
86.80
41.60
34.00
99.80
46.35
67.80
45.80
59.54
-Source
Google Mod.
95.20
98.00
86.60
41.80
97.60
69.27
77.60
27.20
74.16
PG Qwen2.5
85.32
83.47
65.24
46.47
84.26
97.65
90.65
97.08
81.27
Ours
PG Ministral
82.60
79.11
55.52
35.76
80.51
97.39
90.53
96.88
77.29
PG Smol
89.57
85.72
59.16
37.20
81.84
96.10
84.60
96.42
78.83
Table 3: Recall scores on unsafe samples from Patronus' benchmarking: metrics for the best performing model are in bold, whereas those for the second-best performing model are underlined. Takeaway: PG models outperform baselines on Patromus Al's benchmarks.
4.3 Are PG models robust?
We study the average performance of the PG models trained using 3 datasets: only translated data, only ITW data, and translated ITW data. For evaluation data, we create buckets: POLyGUARDPROMPTS Multilingual Bench, and Patronus Al datasets.
PG models trained on a combination of translated and ITW data show greater robustness across both in-domain and citt-o--distibbution evaluation benchmarks, thus underscoring the importance of the presence of ITW samples in the training data mix (Table 4). Models trained only ITW data perform well on Multilingual Bench and Patronus AI datasets, which are somewhat in-distribution with ITW samples, but do not generalize to PGProomps.
POLYGUARD
Training Data
PGPrompts
Multilingual Bench
Patronus
Qwen2.5
Translated
84.95
74.56
79.79
ITW
64.69
74.63
82.26
Translated ITW
83.79
74.88
81.27
Ministral
Translated
84.32
73.86
77.07
ITW
63.11
75.35
85.76
Translated ITW
83.44
73.87
77.29
Smol
Translated
82,22
69.99
74.84
ITW
59.4
65.08
72.21
Translated ITW
80.06
70.35
78.82
Table 4: Average F1 score on POLYGUARDPROMPTR and Multilingual Bench, and Recall on PatronusAl, when models are trained with different training dataset settings. Underlined values represent in-distribution evaluations. Takeaway: Models trained with translated + ITW samples robust different distributions of evaluation data
7

---

Published as a conference paper at COLM 2025
Furthermore, we investigate in detail the influence of the presence of ITW data in our training data mix for each benchmark dataset (Figure 5). We compare the performance of PG (trained translated + ITW data) with models trained on translated data only. We observe that the performance of Qwen2.5 degrades for most of the datasets when ITW data are absent from the training mix. The performance differences for Ministral balanced compared to Qwen2. 5, that is, both improvement and degradation are observed across the evaluation datasets. The introduction of ITW data benefits the performance of the ToxicChat benchmark (Lin et al., 2023) the most for both models, since ITW data is most aligned with the ToxicChat benchmark.
4.4 How does performance vary on English vs Translated vs Code-Switched data?
We study the performance variation of models on code-switched data, which consists of tokens belonging to different languages but in the same document. Code-switching enhances the adversarial nature of the data and thus requires more robust models to successfully detect safe/ /nnsfe content.
We evaluate models on the Code-Switching Red- Red-Teaming (CSRT) (Yoo et al., 2024) dataset and the translated and code-switched version of Aegis 1.0 (Ghosh et al., 2024) as provided by Yang et al. (2024b). Since CSRT also evaluates LLMs' tendency to generate unsafe content, we use the same automatic annotation pipeline described in Appendix D.
In all settings, PG models outperform baselines, showing that our moderation models are more robust (Table 5). For CSRT, observe that there is considerable degradation of performance in the case of code-switching for all models except 0, there is performance drop from English to the translated version. The performance increases for the code-switched version but is lower than on English data.
For Aegis
CSRT CSRT
CSRT
CSRT
Code- Code- switch switch (LG) (Aegis) English
Aegis
English
English (Aegis)
Aegis
Aegis Translated
Code- switch
Type
Model
(LG)
Avg
Aegis-Defensive
90.91
90.61
81.38
81.53
83.89
75.15
80.35
83.40
Open
MD Judge
91.21
90.91
50.00
50.00
82.98
42.54
74.06
68.81
Llama Guard 2
77.52
76.86
65.88
64.79
60.82
51.69
59.16
65.25
-Weight
Llama Guard 3
79.66
79.42
79.83
79.16
67.39
62.15
66.86
73.50
Duo Guard
89.18
52.82
89.26
52.28
83.37
59.10
73.49
71.36
Perspective API
53.79
53.23
32.52
31.75
31.15
26.11
26
36.54
Closed
OpenAl Omni
82.83
82.97
74.24
74.03
73.30
63.82
68.14
74.19
Source
Google Mod.
83.14
83.66
82.19
81.94
74.54
73.60
72.89
78.85
PG Qwen2.5
94.10
93.78
88.55
87.88
87.85
83.00
85.13
88 08.61
Ours
Ministral
95.19
95.22
90.02
89.35
86.96
81.18
83.81
$8.82
Smol
94.39
93.72
84.13
83.86
84.71
72.89
80.32
84.86
Table 5: Fl scores comparison on English only, translated, and code-switched data; metrics for the best performing model are in bold, whereas those for the second-best performing model are underlined. represent results averaged across 3 annotations, LG=LIama Guard Takeaway: All models suffer performance degradation for code-swilched data, with PG models outperforming baselines.
4.5 How is performance affected by removing low-quality translated data?
Data quality plays an important role in the training of any machine learning model. We investigate how the absence low-quality translations training data influences per- formance in the case of POLYGUARD Qwen2. and Ministral. Due to time and budget constraints, we use GPT-4o annotations as a proxy for human-evaluated translation quality and distill them for cost-effective annotations (details in Appendix F).
Empirical evaluations show that the elimination of low-quality translations does not necessarily improve model performance (Figure 9, Appendix F) since contrastive trends
8

---

Published as a conference paper at COLM 2025
Model
Average
Std Dev
POLYGUARD Qwen2.5
87.01
8.27
POLYGUARD Ministral
84.04
12.25
POLYGUARD Smol
65.25
25.02
Table 6: Recall scores for POLYGUARD models human-written samples from the Aya RedTeam benchmark. Takeaway: POLYGUARD models generalize on data from different distri- butions despite being trained only on machine-translate data.
are observed for Qwen2. 5 and Ministral. We hypothesize that the presence of low-quality translations PGMix helps Qwen2. perform well on the low-quality text in toxicity and safety benchmarks.
4.6 Does POLYGUARD superficially align with artifacts of machine-translann text only?
The of machine-translated data for training POLYGUARD models can lead to the hypothesis that models learn only to rely on machine-translation ar- tifacts in the data evaluate safety. To investigate if this behavior exists, we evaluate our models on the Aya Red- teaming dataset (Ahmadian et al., 2024), which consists of manually created 7,419 samples in 8 languages, thus lacking the noise patterns present in machine- translated texts. We do not observe em- pirical evidence supporting the hypoth- esis (Table 6). 11
nicholes
Arsive/taxicity, www.ssiiicaatinni.ooo jigssww
ukr-detec//uk wewe://ttttiiiii----oeoe dataset
tmu nipphhal boxidty, tweet
nicholasklugettxkec
Imsys/taxic chat
OpenSafetyl aaySalldd-Daaa Data
Moderation-En
Moderstion Mul
Prompt
Dalasets
XSefety GPT-4o UlmaaGurd Prompt Mul
XSafety Aegis Prompt-
XSarety GPT-4o Acgis Prompt Mui
Multjail
Prompt En
Multjjill GPT 4o UlmaaGarr Prompt Mui
POLYGUARD
Multijall GPT-4o Aegis Prompt-En
Muttjjal GPT-4o Aegis Prompt-Mul
Runtime Comparison
RTR-LX-En
RTP-LX Mul
We have trained and open-sourced mod- els three sizes (0.5B, 7B, and 8B). While all three run consumer hardware, the 0.5B can benefit on-device latency-critical applications. also test the latency of our models on 7419 samples from the Aya RedTeaming dataset (Ahmadian et al., 2024) on an NVIDIA L40S GPU using VLLM (Table 7), and find that our 0.5B model has high throughput. However, our 7B and 8B models run comparatively slower than their similarly sized Llama Guard counterparts. Compared Llama Guard, POLYGUARD models solve more tasks, and thus require longer prompts and generate more output tokens, which leads to increased runtime.
Aegis-Mul
Aegis-CS
Model
CSRT LlmmaGuard Prompt
Qwen2.5
Aegis Prompt
We
Score Difference
Figure 5: Performance difference removing ITW data Takeaway: Removal of ITW data generally degrades model performance by reducing training data diversity.
We also use the Aya Red-teaming dataset to assess the need for multilingual safety classifiers by translating it to English via lowerinstruct-7B-vll and then evaluating an English-only classifier LLama-Guard-- 3-8)). PG Qwen2. significantly outperforms this setup achieving higher recall in French (0.916 0.706), Russian (0.926 vs. 0.669) and Spanish (0.952 vs. 0.681) highlighting the limitations of relying solely on translation for multilingual safety moderation.

---

Published as a conference paper at COLM 2025
Model
Size
Input Tokens
Output Tokens
Time (m:ss)
Llama Guard
8B
1375800
27536
2:13
Llama Guard
8B
1657409
36364
2:14
POLYGUARD Smol
0.5B
1870206
239337
0:31
POLYGUARD Qwen2.5
7B
1870206
243043
3:27
POLYGUARD Ministral
8B
1881052
242426
3:58
Table 7: Latency comparison of POLYGUARD models on Aya RedTeaming Takeaway: Smol is highly efficient, whereas Qwen and Ministral are slower than LlamaGuards as POLYGUARD models solve multiple tasks.
Background & Related Work
Safety Training Datasets and Safety Evaluations Al Safety, the field of research focused ensuring that Al systems are developed and deployed in manner that is trustworthy, responsible, reliable, and beneficial to humans (Chen et al., 2024), has become widely studied in recent years (Chua et al., 2024; Hendrycks, 2025; Bengio et al., 2025; Bullwinkel et al., 2025). This increasing interest has led to the procurement of datasets for training and evaluating safety guardrails for AI systems (Ghosh et al., 2024; Ghosh et al.; Han et al., 2024; Lin et al., 2023; Ji et al., 2023; Li et al., 2024). Similarly, safety benchmarks have been curated to evaluate the safety risks exhibited by Al systems (Xie et al., 2024; Mazeika al., 2024; Jain et al., 2024; Kumar et al., 2024; Yoo et al., 2024; Zeng et al., 2024b; Zhang et al., 2024a;;; Tan et al., 2024). However, almost all of the aforementioned datasets are limited to the English or Chinese language only or focus specific subsets of AI safety Jain et al. (2024).
Safety Moderation Tools Current open-weight safety systems rely on either proprietary datasets (Inan et al., 2023; Zeng et al., 2024a) or previously mentioned English-centric datasets (Ghosh et al., 2024; Li et al., 2024; Han et al., 2024). Although these LLM-based classifiers possess inherent multilingual capabilities, their performance is constrained by their predominantly English training data (Han et al., 2024; Ghosh et al.). Even though Llama-Guard-3 3-8B is multilingual, PatronusAl (2024) demonstrates its suboptimal perfor- mance on nut-of-distribution toxicity and safety detection tasks. Additionally, existing models face structural limitations; most are restricted binary safety classification (with WildGuardMix (Han et al., 2024) being a notable exception), or ignore the structure of user- LLM interactions by processing only a single text a time (Aegis 1.0 Ghosh et al. (2024) and DuoGuard Deng et al. (2025) take in single piece of text input during training and are expected to generalize over the concatenation of user prompt and LLM response).
7 Conclusion
We present POLYGUARDMIX, the first massive multilingual safety detection training dataset, comprising 1.91M user-LLM interactions across 17 languages. We also introduce POLY- GUARDPROMPTS, multilingual benchmark with 29K samples for the evaluation of safety guardrails. Further, we train robust multilingual L.L.-based safety detectors, POLYGUARD, which perform better or comparably to existing open-weight and proprietary safety detec- tors across numerous evaluation benchmarks belonging to different data distributions.
10

---

Published as a conference paper at COLM 2025
Ethics Statement
Although POLYGUARD demonstrates state-of-the-art performance for multilingual safety detection, it may occasionally produce incorrect predictions. Users should be aware of these potential inaccuracies when using POLYGUARD as moderation tool.
We also acknowledge that our datasets, POLYGUARDMIX and POLYGUARDPROMPTS. con- tain unsafe / harmful content that may inadvertently facilitate the creation of harmful content. However, the intent of releasing our datasets not increase unsafe outputs but instead to advance efforts toward safer multilingual systems. As a safety measure, we plan to implement restrictions on the use of our datasets.
Acknowledgments
This research was supported in part by Google Jigsaw, DSO National Laboratories and Microsoft's s Accelerating Foundation Models Research program.
Data We express our gratitude the authors whose meticulous efforts were instrumental in the creation of our data set: WildGuardMix (Han et al., 2024), LMSys-Chat-IM (Zheng et al., 2023) and WildChat (Zhao al., 2024).
Software and Models We would like to thank the authors of TowerInstruct-77-v00 (Alves et al., 2024) and NLLB-3. 3B (Team et al., 2022) which we use for automatic translations, contributors and maintainers of vLLMM (Kwon et al., 2023) and LiteLLM 12 which we leverage to generate continuations from models, and OpenRLHF (Hu et al., 2024) which we use to fine-tune models. Finally, we thank Jigsaw for providing access to Perspective API.
References
Roee Aharoni, Melvin Johnson, and Orhan Firat. Massively multilingual neural machine translation. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 3874-3884, Minneapolis, Minnesota, June 2019, Association for Computational Linguistics. doi: 10.18653/ N19-1388. URL https: http///llaaaatoooooorrr//111111/
Arash Ahmadian, Beyza Ermis, Seraphina Goldfarb- Tarrant, Julia Kreutzer, Marzieh Fadaee, Sara Hooker, et al. The multilingual alignment prism: Aligning global and local prefer- ences to reduce harm. Xrviv preprint ar Xiv:2224 18682, 2024.
Duarte M Alves, José Pombal, Nuno M Guerreiro, Pedro H Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, et al. Tower: An open multilingual large language model for translation-related tasks. arXiv preprint drXiv:2402.17733, 2024.
Yoshua Bengio, Sören Mindermann, Daniel Privitera, Tamay Besiroglu, Rishi Bommasani, Stephen Casper, Yejin Choi, Philip Fox, Ben Garfinkel, Danielle Goldfarb, et al. Interna- tional ai safety report. Xrx preprint arXin:2501.17805 2025.
Blake Bullwinkel, Amanda Minnich, Shiven Chawla, Gary Lopez, Martin Pouliot, Whitney Maxwell, Joris de Gruyter, Katherine Pratt, Saphir Qi, Nina Chikanov, et al. Lessons from red teaming 100 generative ai products. arviv preprint ar Xix:2501.0023:: 2025.
Chen Chen, Ziyao Liu, Weifeng Jiang, Si Qi Goh, and KwoK- Yan Lam. Trustworthy, responsible, and safe ai: A comprehensive architectural framework for ai safety with challenges and mitigations. ar preprint arXiv:2408. 12935, 2024.
Jaymari Chua, Yun Li, Shiyi Yang, Chen Wang, and Lina Yao. Ai safety in generative large language models: A survey, Xio preprint ar Xie322407.1836, 2024.
"Phttps://githul ttt.ae/.cccmeeerri//iieellll
11

---

Published as a conference paper at COLM 2025
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanow. Unsupervised cross-lingual representation learning at scale. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 8440 8451, Online, July 2020. URL
Association for Computational Linguistics. doi: 10.18653/v1 https://aclanttollog.. org/2020 acl mooin..an..44 747/.
Marta Costa-jussà, Eric Smith, Christophe Ropers, Daniel Licht, Jean Maillard, Javier Fer- rando, and Carlos Escolano. Toxicity in multilingual machine translation at scale. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Com- putational Linguistics: EMNLP 2023, pp. 9570-9586, Singapore, December 2023. Associ- ation for Computational Linguistics. doi: 10.18653 htt///11222222 htt1//23..idiigg-emmm. findings-emnlp.6e URL https://aclanthologgy orgpollooyoyoo/////eee 2023, findings-eenin 642.
Aida Mostafazadeh Davani, Sagar Gubbi Venkatesh, Sunipa Dev, Shachi Dave, and Vinod- kumar Prabhakaran. Genil: A multilingual dataset on generalizing language. In First Con- ference on Language Modeling, 2024. URL https: http///pppeevieeeee iew. net/forumm H-t
Adrian de Wynter, Ishaan Watts, Nektar Ege Altıntoprak, Tua Wongsangaroonsri, Minghui Zhang, Noura Farra, Lena Baur, Samantha Claudet, Pavel Gajdusek, Can Gören, et al. Rtp- Ix: Can ilms evaluate toxicity in multilingual scenarios? arXio preprint ar XrXi:22404.13377, 2024.
Yihe Deng, Yu Yang, Junkai Zhang, Wei Wang, and Bo Li. Duoguard: A two-player rl-driven framework for multilingual Ilm guardrails. aiv preprint ar xix::2502200116, 2025.
Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. Multilingual jailbreak challenges in large language models. In The Twelfth International Conference on Learning Represenlations, 2024. URL https: / //ppnn/vieeeeeeeeee/eeeeeee net/forum?id=-vESWC..
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groen- eveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. arXiv preprint arXiv:2104.08758, 2021.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama herd of models. arXiv preprint or Xiv::400 217783 2024.
Shaona Ghosh, Prasoon Varshney, Makesh Narsimhan Sreedhar, Aishwarya Padmakumar, Traian Rebedea, Jibin Rajan Varghese, and Christopher Parisien. Aegis2. O: A diverse ai safety dataset and risks taxonomy for alignment of I1m guardrails. In Neurips Safe Generative Al Workshop 2024.
Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Christopher Parisien. Aegis: Online adaptive ai content safety moderation with ensemble of Ilm experts. arXiv preprint arXio:2404.05991, 2024.
Daniil Gurgurov, Tanja Bäumel, and Tatiana Anikina. Multilingual large language models and curse of multilinguality. 2024. doi: 10.48550/ ARX1V.2406.10002 URL https://arxiv. org/abs/2405. 10602.
Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. arXiv preprint ar Xio::4406..1 18495, 2024.
Dan Hendrycks. Introduction to ai safety, ethics, and society, 2025.
Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https: http://///eeeeeeeeeeeeee net/ forum?iid=ZeeWeeFFrr.
12

---

Published as a conference paper at COLM 2025
Jian Hu, Xibin Wu, Zilin Zhu, Xianyu, Weixun Wang, Dehao Zhang, and Yu Cao. Open- rlhf: An easy-to-use, scalable and higf--perorrmnce rlhf framework. arXiv preprint Xii:2405.111433, 2024.
Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm- based input-output safeguard for human-ai conversations. ar ar preprint arXin:2312.06677 2023.
Devansh Jain, Priyanshu Kumar, Samuel Gehman, Xuhui Zhou, Thomas Hartvigsen, and Maarten Sap. Folyglotoxiciyprompsss Multilingual evaluation of neural toxic degenera- tion in large language models. Xiv preprint ar arXiv:2405.09333, 2024.
Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. Beavertails: Towards improved safety alignment of LLM via a aumaan-reference dataset. In Thirty-seventtn Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https: /owpr//veeeeeeee////mmome net/ forum?1d=g@0ooxXDww.
Tom Kocmi, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel, and Maja Popovic. Findings of the 2022 con- ference machine translation (wmt22). In Conference Machine Translation, 2022. URL https://api. semanti wwwpollaarrrgg/orrrooo wrg/CorpusID:: 256461 0333.
Suhas Kotha, Jacob M. Springer, and Aditi Raghunathan. Understanding catastrophic forgetting in language models via implicit inference. ArXiv, abs/2239.10105 2023. URL
https://api. semanticscholar... wtt.cccaaa....oooorooiiiiiee wrg//orppusDD:222854014
Priyanshu Kumar, Elaine Lau, Saranya Vijayakumar, Tu Trinh, Scale Red Team, Elaine Chang, Vaughn Robinson, Sean Hendryx, Shuyan Zhou, Matt Fredrikson, et al. Refusal- trained ilms are easily jailbroken as browser agents. arXio preprint arXio:2410.13886 2024.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. SALAD-bench: A hierarchical and comprehensive safety benchmark for large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 9923-3954, Bangkok, Thailand, August 2024, Association for Computational Linguistics. doi: 10.18653/V /vl/ 2024. fiwdininnn--cl..2.. URL https: htttp::/aaclantholoo org/2024. findings-ade acl. 235.
Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, and Jingbo Shang. ToxicChat: Unveiling hidden challenges of toxicity detection in real-world user- AI conversation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 4694-4702 Singapore, December 2023. Association for Computational Linguistics. doi: (0.18653/vl/200 findings-emnlp.311. URL https::/aclanttoo...g org/2023. findings-emmlp.311
AI @ Meta Llama Team. The llama 3 herd of models, 2024. URL https::/arxiv..o 2407.21783.
Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. A holistic approach to undesired content detection in the real world. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 15009-15018, 2023.
13

---

Published as a conference paper at COLM 2025
Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Harm- bench: standardized evaluation framework for automated red teaming and robust refusal. In Proceedings of the s1 International Conference on Machine Learning, ICML/24.
JMr.R.rrg 2024.
Mistral. Un ministral, des ministraux. URL
Andrew Ng. translation-agent
Agentic translation.
URL httpss///gihhbb..om/mmdrreeww/
PatronusAL. Llama guard is off duty. https///www.aaattoooss/////////s duty, 2024.
Qwen. Qwen2.5: A party of foundation models! URL https::////w/n www.mmm..iit...... io/blog/ qwen2.5/.
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi Social bias frames: Reasoning about social and power implications of language. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5477-5490, Online, July 2020. Association for Computational Linguistics. doi: 10.18653 a20//20.cclmmmiii URL
org/ 202/.2220...0 acl-main. 486/.
Khetam Al Sharou and Lucia Specia. A taxonomy and study of critical errors in machine translation. In Helena Moniz, Lieve Macken, Andrew Rufener, Loic Barrault, Marta R. Costa-jussà, Christophe Declercq, Maarit Koponen, Ellie Forcada, Carolina Scarton, Joachim Van den Bogaert, Joke Daems, Arda Tezcan, Bram Vanroy, and Margot Fonteyne (eds.), Proceedings of the 23rd Anmual Conference of the Euro- pean Association for Machine Translation, pp. 171-180, Ghent, Belgium, June 2022. European Association for Machine Translation. URL /ttttp::/aclaathooo.lo
Pilos, Mikel L.
org/2022. 20002..amt-1.00
Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaudhary, and André F. T. Martins. Findings of the WMT 2021 shared task on quality estimation. In Loic Barrault, Ondrej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussa, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, An- tonio Jimeno Yepes, Phillipp Koehn, Tom Kocmi, Andre Martins, Makoto Morishita, and Christof Monz (eds.), Proceedings of the Sixth Conference on Machine Translation, pp. 684-725, Online, November 2021. Association for Computational Linguistics. URL https://aclanthologg org/20g1 1mt-1.71.
Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiangyong Zhu, et al. Chinese safetyqa: A safety short-form factuality benchmark for large language models. arv preprint arXiv:2412.15225, 2024.
Llama Team. Meta llama guard 2. http::///ggttu.ccoottttalamllalp Laalllaamaa-Garr//oooEE
md, 2024.
NLLB Team, Marta R. Costa-jusså, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Bar- rault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. No language left behind: Scaling human-centeree machine translation, 2022.
Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao, and Michael Lyu. All languages matter: On the multilingual safety of large language models. ar Xiv preprint ar arX113310000090 2023.
14

---

Published as a conference paper at COLM 2025
Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, et al. Sorry-bench: Systematically evaluating large language model safety refusal behaviors. arXiv preprint arXiv:2406.1459; 2024.
Yuemei Xu, Ling Hu, Jiayi Zhao, Zihan Qiu, Yuqi Ye, and Hanwen Gu. A survey on multilingual large language models: Corpora, alignment, and bias. ArXiv, abs abs/2444.000999 2024. URL https: /api. semanticscholar org wrg//Crppssti:: 26:.00099911:288117777777
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. technical report. ar preprint mrXiv:2412.151155 2024a.
Yahan Yang, Soham Dan, Dan Roth, and Insup Lee. Benchmarking Ilm guardrails in handling multilingual toxicity. ar preprint ar arXo:2240022215, 2024b.
Haneul Yoo, Yongjin Yang, and Hwaran Lee. Code-switching red-teaming: Lim evaluation for safety and multilingual understanding. arXiv preprint XrXio22406.15881, 2024.
Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Hark- ous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, et al. Shieldgemma: Generative ai content moderation based on gemma. arXiv preprint wxXiv:240.21777, 2024a.
Yi Zeng, Yu Yang, Andy Zhou, Jeffrey Ziwei Tan, Yuheng Tu, Yifan Mai, Kevin Klyman, Minzhou Pan, Ruoxi Jia, Dawn Song, et al. Air-bench 2024: A safety benchmark based on risk categories from regulations and policies. arXiv preprint ar arXiv:2407. (1f:22077.17466,, 2024b.
Hengxiang Zhang, Hongfu Gao, Qiang Hu, Guanhua Chen, Lili Yang, Bingyi Jing, Hongxin Wei, Bing Wang, Haifeng Bai, and Lei Yang. Chinesesafe: A chinese benchmark for evaluating safety in large language models. arXiv preprint ar Xio:2410.18491 2024a.
Wenjing Zhang, Xuejiao Lei, Zhaoxiang Liu, Meijuan An, Bikun Yang, KaiKai Zhao, Kai Wang, and Shiguo Lian. Chisafetybench: A chinese hierarchical safety benchmark for large language models. ar preprint ar Xix::22406. 10331, 2024b.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wild- chat: Im chatgpt interaction logs in the wild. aiv preprint Xix::2405.01470, 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric P Xing, et al. Lmsys-chat-lme A large-scale real-world Ilm conversation dataset. arviv preprint ar Xiv:2303 11998, 2023.
Xuhui Zhou, Hao Zhu, Akhila Yerukola, Thomas Davidson, Jena D. Hwang, Swabha Swayamdipta, and Maarten Sap. COBRA frames: Contextual reasoning about effects and harms of offensive statements. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 6294-6315, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10.18653, 181133/v1/ C 2023. www..iiinngnacl.... URL httpp:://aclaeeeee httpss::/aclaattoolgyy.// org/2033. 2023. www..f.nni.e..e....eeeee. findings-an acl. 392/
15

---

Published as a conference paper at COLM 2025
A Translation Pipeline and Quality Evaluation
Our training data translation pipeline segments the source text into chunks using blingfire translates each chunk independently, and reconstructs the target text by concatenating the translated segments. We utilize this chunking approach to process long texts while respecting the context window constraints of translation models.
The agentic translation framework, used for translating the test data, implements three- stage process: (1) initial translation of the source text, (2) critical analysis of potential translation errors, and (3) subsequent refinement of the target text incorporating the identi- fied improvements. This iterative approach helps maintain semantic accuracy and cultural nuances crucial for safety evaluation across languages.
Automatic evaluation of translation quality is done by prompting GPT-4o as a judge. We design 3-point Likert scale comprising of the following choices: "translation is entirely correct", "translation is almost correct with a few errors", and "translation is bad". We run evaluations for only the translated test set of PGMix to assess the overall translation quality of a model. The evaluation prompts are shown below:
System Prompt
You are a linguistic expert. Given a "source_text" in English and "target_text" in (language), your job is to evaluate if the *target-tex" is the correct translation of the source_text".
User Prompt
source_text": [src}
target_text": (tgt}
Following the format of structure generations for translation quality evaluation. We prompt the judge to first reason about the source and target sentences before outputting the verdict.
class QualityEnum Enum)
incorrect 'translation is bad
almost_correct 'translation is almost correct with few errors' entirely 'translation is entirely correct'
class Result (BaseModel):
reason: str Field (dessrrrtiiiiioo pointers why the translation is correct or wrong") verdict: QualityEnum Field diesriidttrnpttoon=== verdict about the translation quality")
Tables 8 and show the verdicts of the GPT-4o judge for the human prompt and model response respectively. We observe that InwerInstruct generates higher-quality translations when compared to NLLB for the languages supports. However, in the case of Hindi (which is not supported by Tower), the quality is poor.
16

---

Published as a conference paper at COLM 2025
Language
Model
Entirely Correct
Partially Correct
Bad
Invalid Judge Verdict
ZH
NLLB
636
688
401
Tower
1202
360
162
1
ES
NLLB
1437
218
68
2
Tower
1374
303
47
1
FR
NLLB
1406
245
72
2
Tower
1499
177
47
2
DE
NLLB
1275
348
101
1
Tower
1335
323
66
1
KO
NLLB
1075
490
158
2
Tower
1278
336
109
2
NLLB
1384
260
80
1
Tower
1442
227
56
PT
NLLB
1463
202
60
Tower
1532
142
51
NL
NLLB
1339
306
77
3
Tower
1399
264
62
RU
NLLB
1379
240
106
Tower
1406
233
85
1
HI
NLLB
1470
186
69
Tower
25
1691
2
Table 8: GPT-40 Judge verdicts for human prompts translation. Takeaway: TowerInstruct generated more accurate translations than NLLB for supported languages.
Language
Model
Entirely Correct
Partially Correct
Bad
Invalid Judge Verdict
ZH
NLLB
153
1147
424
Tower
822
729
174
ES
NLLB
858
426
441
Tower
583
1057
85
FR
NLLB
883
741
101
Tower
481
1163
81
DE
NLLB
811
790
124
Tower
625
1028
72
KO
NLLB
721
920
Tower
707
916
101
1
NLLB
809
566
350
Tower
529
1103
92
1
PT
NLLB
884
623
216
2
Tower
489
1131
105
NL
NLLB
828
772
124
1
Tower
593
1049
82
1
RU
NLLB
906
663
156
Tower
512
1123
90
HI
NLLB
1286
411
28
Tower
1718
Table 9: GPT-40 Judge verdicts for model generation translation. Takeaway: TowerInstrucc generales less low--uality translations than NLLB for supported languages.
17

---

Published as a conference paper at COLM 2025
B Human Validation
We use Prolific to collect annotations. For each of the 16 target languages, screen annotators whose first language, fluent language, or primary language is English and the target language. Additionally, and a submission count between 100 and 10,000. Annotators were compensated at the rate of $12/hr. Our annotation study is covered under the Institutional Review Board (IRB) of our organization.
pre-screen annotators with an approval rate of 90-100%
We collect 2,400 annotations across 16 languages and 50 data points per language, with each data point annotated by 3 annotators, and each annotator annotating 10 data points.
We recruited 191 unique annotators 15 via Prolific, spanning across 24 countries. They self- identified as 110 male and 81 female. In terms of ethnicity, they described themselves as 84 White, 79 Black, 12 Mixed, 10 Asian, and 5 Other.
Figures 6, 7, and present the consent, annotation instructions, and framework questions. The human validation results for each language are shown in Table 10. We report the average translation quality score using the Direct Assessment Scalar Quality Metric framework, on scale of 0-100. Inter-annotator agreement is computed using Krippendorff's for both source and target language safety labels.
Language Avg. lation Score
Trans-
Source Safety a
Target Safety a Source Target &
Arabic
80.99
0.41
0.40
0.96
Chinese
78.55
0.43
0.42
0.91
Czech
81.11
0.47
0.48
0.96
Dutch
77.15
0.37
0.33
0.96
French
82.12
0.48
0.47
1.0
German
82.67
0.44
0.45
0,92
Hindi
84.72
0.34
0.37
0.96
Italian
83.21
0.38
0.37
0.91
Japanese
76.39
0.39
0.36
0.76
Korean
81.55
0.43
0,46
0.96
Polish
80.33
0.39
0.40
0.96
Portuguese
81.09
0.46
0.45
0.92
Russian
80.44
0.42
0.43
0.96
Spanish 84.11
0.45
0.44
1.0
Swedish 79.66
0.36
0.35
1.0
Thai 78.89
0.41
0.42
0.92
Table 10: Human validation results for translation quality and safety labels. Translation scores are on a 0-100 scale, using the DA+SQM framework. Inter-annotator agreement (Krippendorff's a) for source and target safety labels is reported, along with agreement between majority-voted source and target labels.
C POLYGUARD Training Details
We train our models using 1 on 8 NVIDIA A6000 GPUs. We set LoRA rank and alpha to 16. We train our models with a total batch size of 128, for sequence length of 8192, for epoch using learning rate of 2e 4. The system and user prompts (adapted from WildGuard and Llama Guard v3) used by PG are follows:
hhhttps:/www.prollff: com/
some participated in multiple languages, resulting in a lower unique count 16https: thubb....ttt..../ httpss/s//ggiuuuu..co//eeeerrffe wwtp:/p/enll////eeeeeeoooeeeeeee /F http://////eeeee////eee/e///e// HFteee/eaeeeeennn
18

---

Published as a conference paper at COLM 2025
Bookground:
Hellr of no are trum of researchers who passiceute aloout molk ny oysters safer to use scross mutitile la suuages We ore trying to test out few sestems outomotiially cialyy to sloding by tenoed acness anguaggs to soe how woll they proserve the original mooning andif they trankite unsile centent well, we spprec lite your help minkkng lysterrs saler across mutiplec cuttures and largu orgguee!
Task Overview
in stush wearo rntrestedin in making systzme sater use mutsole banguages. You will oe procented with sertence in English noe its mac maehine generotes the suticn antieer language. Your tack
tojudde the siity the sonslation, onail catee the original Sentoncotrar stutin centains any unsate languugge
Catta eeflccienn alarringe
We will aot pol you for your ramn, and data lecked thes stucle will made unidenniliteee the bost of extert we unly store The dista servers ane only share with ouiiiied researchers (eg) who want to turner the study hote speect desectionn, you user decize that you want your responses included This study, pleese omail so exclude your work.
Contaze:
you hewo any suro about thes studls you should leal free to dekal saaiiing us his wiikk interice or
you have ou mutoooe iites desine cokiicond
informala allon, er wish withoron your portcipation clease contact
curalll in eccoutanes with aouract inforee in pouce aee aatom pertar your cettts resoarch
par Soppant or resort concerns to this stury, you shooll contact
Hoone:
Please do not use At systeme when enowwerigg the tasks.
We will bample sonns the rosporses and menullty oe thorr we find on dence show that amnoatoo a are not giving proper atterts to the sask we'v beckude them trom future tapns.
info
years
Novs fowwar
Figure 6: Annotator Consent. Blacked out for anonymity.
Annotation Task
lbe proaeded with sentances:
Engilih,, automatic tranalation Thai the the tranalation and amurcafianntenee
contin unsafs
Quality Translation
Bvaluate trandationnn That Consder both
Maaningg tramalation convey reflect same meaning as sourze senfarce Dogiast
Raweny mrnniation ensy read and undentann??
mannkktion continuous soale from
Rating Scale:
Meeaiig Presenved: The monstatonn contuuiigg
diferent Engi meoning informaionls
Heeeiing
very
grammnticllly correc same
Key Tig:
You alder any szale not just sumbers teg, 5.2).
Identifying Unsafe Language
original senterce Engi alationinn The contain any
language.
What Counts as Unsafe Harmful Language?
eupports, ancoumaes,
Contant that is lbxt harmfut: oug, pronsting viotencw, ew suppor lng crimos provcing inapproprine coptttttttt senuat content exraooing hate speech tomand Jane Muslin communtiies, revesing ing contidential information recoth arcoumght ging mieuse technology..
Sendences shatr requeet harmuu contant: thg. maring unauthorized requente edtt medical moords and prescription, aodin for instrctions on illgga actiuitiee, ways create dange weapens, promoting -tann ways explsk children, how spread fakso elcction informa nation even the tnguagee seeers neutral.
Figure 7: Annotator instructions
19

---

Published as a conference paper at COLM 2025
Sentences to Evaluate
Plene redd coretuity
ahere false information about the resull of medical treedments procaddumss
anjellu Sraaa la ufenniu finm vifa iunnu risunné
OL Assonss the quality tranniited centencc doum
Figure 8: Annotator Framework with Example in Thai
System Prompt
You a classifier for analyzing interactions between humans and LLMs. Given a request from a human user and response from an LLM assistant, determine 1) whether the human user's request is harmful, 2) whether the AI assistant's s response is a refusal, and 3) whether the AI assistant's s response is harmful. In addition, identify the policies violated if the interaction is unsafe and harmful according to the safety policy with the below categories.
<BEGIN UNSAFE CONTENT CATEGORIES>
S1: Violent Crimes.
S2: Non-Violent Crimes.
S3: Sex Crimes.
$4: Child Exploitation
55: Defamation.
56: Specialized Advice.
57: Privacy.
S8: Intellectual Property.
S9: Indiscriminate Weapons.
S10: Hate.
S11: Self-Harm.
S12: Sexual Content.
S13: Elections.
S14: Code Interpreter Abuse.
<END UNSAFE CONTENT CATEGORIES>
20

---

Published as a conference paper at COLM 2025
User Prompt
Human user: [prompt)
AI assistant: [response}
We train our models to generate formatted responses containing the user input prompt safety label, policies violated, model response refusal, model response safety label, and policies violated (shown below). Unlike WildGuard, avoid generations of "N/A" A" and frame all tasks as binary classification tasks (yes/no). We parse and post-process responses generated by PG match the same format as WildGuard.
POLYGUARD Generation
Harmful request: yes/no
Request safety violations: None / List
of violated categories
Response refusal: yes/no
Harmful response: yes/no
Response safety violations: None
List of violated categories
D Out-of-Distriuttion Benchmarking Dataset Annotations
In this section, we list the formulation of ground-truth labels for the out-of-distribution benchmarks. For the OpenAI Moderation dataset, we consider samples with any of the annotations (sexual, hate, violence, harassment, self-harm, sexual/minor, hate/ tetee/htrtatennggg as True as unsafe. For RTP-LX, we consider samples with a Toxicity score above unsafe. XSafety and MultiJail datasets consist of prompts to measure the tendency of LLMss to generate unsafe content. Thus, a few prompts in these datasets are innocuous but could trigger an LLM to generate harmful content. Therefore, we use GPT-4o to determine the safety label of the samples. Since annotations are influenced by the input prompt, we use the Llama Guard 3 and Aegis prompts to create two sets of ground-truth labels.
E Patronus AI Safety Study
Patronus AI benchmarked Llama Guard 3 on a small number of samples (500) from various English and multilingual toxicity and safety datasets illustrating its poor recall unsafe data points (PatronusAT, 2024). Their evaluation benchmark consists of the following datasets available on HuggingfaceHHub
1. nicholaskluge/toxxcteeeeee
2. Arsive/toxicity classification jigsaw
3. ukr-detett/u---xxxittddtstss
4. tmu-nlp/ htaplp//thaitoxiittyytttee -tweet
5. Kacholaskluge/toxiiteeexss
6. Imsys/toxic-chat
7. PKU-Alignment/BB Beaanerraiiaali
8. Lae Saa//aalldD-aa
are available for modifications under the Apache 2.0 license and Mistral Research License respectively.
- 5-78-Instruct and
21

---

Published as a conference paper at COLM 2025
F Influence of low-quality translated data
We distill GPT-4o's knowledge of translation quality into Qwen2. 7B classifier to filter out samples with low translation quality. We use the same schema as our translation quality study (Appendix A) to filter for samples where the human prompt and model response are accurately translated. We use GPT-4o annotations on the NLLB and TowerInstrus trans- lations of WildGuardMix test data and create stratified train-eval split in 70:30 ratio. Similar to PG, we train a Qwen2. 5-based SFT classifier to predict the quality of the translated source document, using the following prompts:
System Prompt
You linguistic expert. Given source_text"" in English and a "arret_textt" in (language), your job is to evaluate if the "target_ter _extt""x" is the correct translation of the 'source_text"
User Prompt
source_text"; (source)
target_text": (target)
The model is trained on 60,346 training samples and achieves overall accuracy of 82% the validation set of 25,863 samples. complete evaluation report shown below Table
11.
Label
Precision
Recall
F1
Support
Bad
70
73
71
2066
Partially Correct
76
63
69
7704
Entirely Correct
87
93
90
16093
Table 11: Translation Quality Classifier performance metrics
Removal of low-quality training data does not necessarily improve model performance. Intuitively, the presence of poor-quality translated data should harm model performance. However, PG models show contrastive trends when low-quality samples are removed from the training data mix (Figure 9). The performance of Qwen2. 5 degrades for most datasets, whereas the performance of Ministral improves. The performance degradation in the case of Qwen2. 5 can be attributed to noisy samples safety and toxicity evaluation datasets. Harmful text is considered to belong to low-quality data; web-crawls implement word blocklist filters to enhance data quality (Dodge et al., 2021). Thus, we hypothesize that the noise induced by poor translations bridges the gap between training and evaluation data, thus leading to performance improvement.
G Limitations
We describe several limitations of our work. First, we automatically translate English data to other languages using LLMs. However, automatic translations can introduce deviations in toxicity and safety risks due to incorrect translations and hallucinations (Specia et al., 2021; Sharou & Specia, 2022; Team et al., 2022; Costa-jussà et al., 2023). Second, we employ existing safety classifiers and LLMs to automatically annotate safety violation categories, which may introduce biases from these models into our labeled safety categories. We utilize a panel of models to mitigate such biases, but acknowledge the inherent limitations of this methodology. Third, we follow Llama-Guard-33 3--8 (Dubey et al., 2024) and define
22

---

Published as a conference paper at COLM 2025
nichelaskiigee
Ar Arnivenhaaiiity of clavel nattoo Jigwew
utr-dateed tulke wekeeaicittyddsaeodd
niprthol, toulccty
gicholastk.gertoii gortadd st
insyuhoiinscht
Moderation En
Modieration Mui
Xialety
Prompt-Elin
Datarets
XSatety GPT-to LlmmaGurrd Prompt-Mul
XSafutt GPT-4o Aegis Prompt En
Prompt
hijaal GPT-to Ulmatuard Frompt-En Ein
Multjoi GPT-4o LlsmaGusrd Prompt-Kee -uil
Mutioll GPT-4o Prompt
Multjail Aug Prompt-Muu
HTP- En
RTP-XX-Mul
Aegis-Hoe
Augi-CS
CSRT LiamaGuand Prompt
Gwen2.55
Angis Prompt
Ministral
Figure 9: Performance difference on removing low-quality data. Takeaway: Removal of low-quality training data does not necessarily improve model performance.
our safety violation taxonomy according to the MLCommons Safety Taxonomy This taxonomy may not cover all potential harms and may differ from categories that others may prefer. Finally, datasets (PoLYGUARDMII and POLYGUARDPROMPTS and the resulting safety classifiers (PoLyGUARD) do not extend to low-resource languages due to the lack of high-quality multilingual models available for such languages to extend our methodology.
23


Reviews:
[{"review_id": "WjJMq49c3l", "reviewer": "Reviewer_4mLP", "summary": "This paper proposes a new state-of-the-art multilingual safety model called POLYGUARD, as well as POLYGUARDMIX (which is the largest\nmultilingual safety training corpus containing 1.91M samples across 17 languages) and POLYGUARDPROMPTS (which is a high-quality multilingual benchmark dataset  with 29K samples). Extensive experiment results show the superior performance of POLYGUARD compared with existing state-of-the-art open-weight and closed-source safety classifiers.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed dataset with multiple languages contains both the training and test datasets, which is expected to guide the development of multilingual LLM safety.\n\n2. The proposed model achieves good empirical performance compared with existing state-of-the-art baselines, which can be helpful as open-source safety detectors.\n\n3. This paper is overall well-written and easy to follow.", "reasons_to_reject": "I do not see any major problem. The authors are suggested to add more discussions about language-specific performance of POLYGUARD, which may provide insights into the improvement of safety detectors' performance on low-resource languages.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TbQJLRZvV6", "reviewer": "Reviewer_bW1v", "summary": "The paper introduces POLYGUARD, a multilingual safety moderation tool designed for Large Language Models (LLMs). It aims to address the limitations of existing safety moderation efforts, which have often focused on a narrow set of languages (like English and Chinese) and a limited definition of safety. Safety labels for the data, including binary harmfulness assessments and violated categories based on the MLCommons Safety Taxonomy, were obtained using a panel of English classifiers and LLMs-as-judges.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper releases two valuable datasets: the largest multilingual safety training corpus and a high-quality multilingual benchmark (although primarily comprising translated samples).\n\n- The paper works on an important, often underrepresented, safety problem in a multilingual setting. The tool and datasets cover 17 languages, representing a significant expansion compared to the limited language focus of previous efforts. \n\n- They show that training on a combination of translated and naturally occurring data leads to more robust models for various data distributions.", "reasons_to_reject": "They show that removing low-quality translated data from the training mix did not necessarily improve model performance, and even degraded performance for the Qwen2.5-based model on most datasets. The hypothesis offered in the paper is that the \"noise induced by poor translations bridges the gap between training and evaluation data\". I think this shared noise between training and test sets creates a superficial alignment that harms true generalization and the model’s performance becomes a false indicator of competency. It could be that the model is learning to rely on the noise patterns present in both datasets instead of the actual underlying task. So instead of learning properly, it's picking up on the artifacts from poor translations that are present in both train and test. This makes the model's performance misleadingly good on the test set but not robust in real-world scenarios where the noise might differ.", "questions_to_authors": "- A large part of the data is machine-translated. Human validation found high average translation quality (81.15). What is this score (accuracy?), and how was it calculated? Who were the annotators and what was the agreement between them? Why not report more standard translation or correlation metrics?\n\n- The GPT-4o judge verdicts show a considerable number of \"Partially Correct\" and \"Bad\" translations, particularly for model responses (table 6). Is that consistent with the human annotation validation you ran? It seems much higher than the earlier high-quality score.\n\nMissing citations:\n- MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models\n- X-Guard: Multilingual Guard Agent for Content Moderation\n- The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm\n- All Languages Matter: On the Multilingual Safety of LLMs", "extra_scores": {}}, {"review_id": "vDd4eTrHsh", "reviewer": "Reviewer_woca", "summary": "* The paper proposes Polyguard (SOTA multilingual content moderation model) along with Polyguardmix (train dataset) and Polyguard prompts (test dataset).\n* Polyguard is a set of finetuned Lora adapters on various models from different model families - Qwen-2.5, Mistral and HF Smol.\n* Dataset is obtained from Wildguardmix using a machine-translation based approach with various LLM-as-a-judge/human evaluations to ensure data quality.\n* Models trained on this dataset show very strong empirical performance across a range of different multilingual content moderation benchmarks.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Very useful contribution to the multilingual community for LLM-based content moderation - while there are models like LlamaGuard which have multilingual coverage, having access to the train/evaluation datasets is extremely valuable for further research.\n* Strong empirical results across a range of benchmarks.\n* Comprehensive set of ablation studies and good set of steps to ensure data quality.", "reasons_to_reject": "* The inter-annotator agreement seems to be pretty low for safety labels - with the agreement being ~0.46, I worry that the dataset is heavily influenced by Meta/OpenAI's definition of safety.\n* While translations preserve the original meaning of the text, how do we account for cultural differences in context of determining if certain things are safe or unsafe?", "questions_to_authors": "Please see reasons to reject.", "extra_scores": {}}, {"review_id": "ZesjerRoPX", "reviewer": "Reviewer_NvCT", "summary": "This paper introduces PolyGuard, a multilingual model for safeguarding LLM generations that works with 17 languages, and releases the corresponding training data, PolyGuardMix, obtained by mixing naturally occurring multilingual conversations with validated translations of WildGuardMix. Results show that PolyGuard outperforms existing safety detectors on English and multilingual benchmarks.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- PolyGuard expands the scope of open-source safety moderation to 17 languages.\n- The PGMix dataset with human-annotated quality labels is a valuable data source for the community.\n- The paper is well-written.", "reasons_to_reject": "- There is very limited discussions on latency / cost.\n- It's unclear how a pipeline system, i.e., first translate input from other languages to English and then use an existing English-only safety moderation tool, would perform on the benchmarks studied. If such pipeline systems achieves good accuracy and latency, there is less added value of a natively multilingual safety moderation model?", "questions_to_authors": "Comment: Citation at the end of line 3, section 3 (Qwen; Mistral) should be changed to \\citep", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper develops a safety content moderation tool in 17 languages, is a very impactful paper for a community that is underserved for multilingual and tends to rely on llamaguard, which has very limited multilingual coverage. 17 languages is a significant expansion. It also adds datasets that are in translation but have undergone quality checks. 

The few concerns with the paper were very thoroughly addressed by the authors -- the authors addressed bW1v's theory on artifacts in the dataset with an additional experiment on a dataset for generalisation, and they backed up their annotator agreement scores with strong understanding of literature. They add an overview of latency as a reviewer requested.

Given both the reviews and responses I find no issues with the scientific soundness of this paper and it is a valuable addition to the community.
I encourage the authors to make all the modifications in the camera ready that they discussed in the response period.

I will also note that this is one of the more unanimous review sets I have ever encountered in the past few years, paired with convincing and thorough author responses.
Decision: Accept