All Languages Matter: On the Multilingual Safety of LLMs (2024)

Abstract: Ensuring safety is fundamental when developing and deploying large language models (LLMs). However, previous safety benchmarks only concern the safety in one language, e.g., the majority language in the pretraining data, such as English. In this work, we build the first multilingual safety benchmark for LLMs, XSAFETY, in response to the global deployment of LLMs in practice. XSAFETY covers 14 commonly used safety issues across ten languages spanning several language families. We utilize XSAFETY to empirically study the multilingual safety for four widely-used LLMs, including closed-source APIs and open-source models. Experimental results show that all LLMs produce significantly more unsafe responses for non-English queries than English ones, indicating the necessity of developing safety alignment for non-English languages. In addition, we propose a simple and effective prompting method to improve ChatGPT’s multilingual safety by enhancing cross-lingual generalization of safety alignment. Our prompting method can significantly reduce the ratio of unsafe responses by 42% for non-English queries. We release the data to facilitate future research on LLM’s safety1.

heading: 1 Introduction
text: Recent advances in scaling Large Language Models (LLMs) have made breakthroughs in the Artificial Intelligence (AI) area. With the rapid increase of model parameters and training data, LLMs have gained emergent abilities in various tasks, including writing assistance (Gao et al., 2022), code generation (Gao et al., 2023), machine translation (Jiao et al., 2023). Due to their impressive
*Work was done when Wenxuan Wang, Youliang Yuan, and Jen-tse Huang were interning at Tencent AI Lab.
† Jen-tse Huang is the corresponding author. 1Our dataset is released at https://github.com/
Jarviswang94/Multilingual_safety_benchmark
performance, LLMs have been launched by commercial companies and academic institutions, including OpenAI’s GPT models (Brown et al., 2020; OpenAI, 2022), Google’s Bard (Pichai, 2023), and Meta’s LLaMA (Touvron et al., 2023a,b). Such extensive deployment underscores an imperative of paramount significance: ensuring the safety of the deployed LLMs.
Numerous studies have been conducted to align large language models (LLMs) with human ethics and preferences to improve their safety. These include methods such as data filtering (Xu et al., 2020; Welbl et al., 2021; Wang et al., 2022), supervised fine-tuning (Ouyang et al., 2022), reinforcement learning from human feedback (RLHF) (Christiano et al., 2017), and red teaming (Perez et al., 2022; Ganguli et al., 2022a). However, despite the diverse safety alignment methods, most existing research has focused on the interactions in English (OpenAI, 2023), which is a limitation given that LLMs, like ChatGPT, are deployed globally and are frequently used for non-English communication with users from non-English-speaking regions. The discrepancy between the limited multilingual safety research and the widespread application of LLMs raises an important research question: Can non-English language prompts bypass the safety alignment primarily tuned in English?
To address this question, we develop the first multilingual safety benchmark for LLMs, called XSAFETY. We gather several well-established
5865
monolingual safety benchmarks covering 14 types of safety issues and employ professional translators to translate them, resulting in a multilingual benchmark in 10 languages. XSAFETY comprises 2,800 instances in the ten most widely-used languages, spanning several language families: English, Chinese, Spanish, French, Bengali, Arabic, Hindi, Russian, Japanese, and German, totaling 28,000 annotated instances. XSAFETY allows us to systematically evaluate the multilingual safety of four widely used LLMs, including ChatGPT, PaLM2, LLaMA-2-Chat, and Vicuna. Experimental results reveal that all the LLMs exhibit significantly lower safety in non-English languages compared to English, highlighting the need for developing safety alignment strategies for non-English languages. These findings demonstrate that the risks associated with LLMs in non-English languages are concealed by their safety performance in English, emphasizing the importance of addressing safety concerns in multiple languages.
Specifically, inspired by recent success in prompting GPT-3 to be reliable (Si et al., 2023), we propose a simple and effective prompting method to improve the multilingual safety of ChatGPT. The principle behind the prompting engineering is to improve cross-lingual generalization of safety alignment (e.g., “Please think in English and then generate the response in the original language.”). The effective prompt can significantly reduce the ratio of unsafe responses by 42% for non-English queries.
Contributions Our main contributions are:
• We build the first multilingual safety benchmark XSAFETY for LLMs, which covers 14 safety scenarios across ten languages.
• Our study demonstrates the necessity of developing safety alignment for non-English languages.
• We propose a simple and effective prompting method to improve the multilingual safety of ChatGPT by improving cross-lingual generalization of safety alignment.
• We release the data to facilitate future research on the multilingual safety of LLMs.
Content Warning: We apologize that this paper presents examples of unsafe questions and responses for demonstration and clarity.

heading: 2 Related Work

heading: 2.1 Safety of LLMs
text: Taxonomy The safety of LLMs plays an important role in AI development (Ji et al., 2023b). Weidinger et al. (2021) categorized the risks associated with LLMs into six distinct areas: (I) information hazards; (II) malicious uses; (III) discrimination, exclusion, and toxicity; (IV) misinformation harms; (V) human-computer interaction harms; and (VI) automation, access, and environmental harms. Recently, Sun et al. (2023) adopted a broader taxonomy from two perspectives: 8 kinds of typical safety scenarios and six types of more challenging instruction attacks. In this paper, we adopt the taxonomy of the later paper, aiming to analyze the safety of LLMs comprehensively.
Evaluation A branch of previous works has primarily focused on specific risk areas, such as toxicity (Hartvigsen et al., 2022), bias (Dhamala et al., 2021; Wan et al., 2023), copyright (Chang et al., 2023) and psychological safety (Huang et al., 2023). There is also some work on the development of holistic safety datasets. (Ganguli et al., 2022b) collected 38,961 red team attack samples across different categories. Ji et al. (2023a) collected 30,207 question-answer (QA) pairs to measure the helpfulness and harmlessness of LLMs. Moreover, Sun et al. (2023) released a comprehensive manually written safety prompt set on 14 kinds of risks. However, both safety datasets are only in a single language rather than a multilingual safety benchmark, hindering the study on multilingual safety. Our work bridges this gap by introducing a multilingual dataset to assess model safety across ten languages.

heading: 2.2 Multilingual Evaluation on LLMs
text: LLMs can learn multiple languages from trillions of pre-trained tokens, serving as a foundation for multilingual task solvers. For instance, OpenAI’s ChatGPT (OpenAI, 2022, 2023) provides services to users from different countries using various languages. As a result, in addition to evaluating the performance of ChatGPT on NLP tasks in English (Bubeck et al., 2023), there is growing interest in its multilingual capabilities. Jiao et al. (2023) assessed ChatGPT’s translation capability and found it to have excellent cross-language translation skills. Bang et al. (2023) tested ChatGPT’s language understanding and generation abilities
in high, medium, and low-resource settings, identifying shortcomings in low-resource languages, particularly in language generation. Furthermore, Abdelali et al. (2023); Ahuja et al. (2023); Lai et al. (2023); Wang et al. (2023, 2024) evaluated ChatGPT and other large models (e.g., BLOOM (Workshop and et al., 2023), Vicuna (Chiang et al., 2023), Claude (Anthropic, 2023), and GPT-4 (OpenAI, 2023)) on a broader range of languages and diverse tasks. In contrast to these studies, which focus on the performance of large models in cross-language tasks, our work serves as a complement, examining the safety of these models across different languages.

heading: 3 Multilingual Safety Benchmark
text: The Monolingual Corpora We systematically review all the safety benchmarks for LLMs from different fields, including NLP, Security, and AI, to select the basis of multilingual XSAFETY. We use the following three criteria to select monolingual corpora. First, the benchmark should be comprehensive and cover different safety issues. Second, the benchmark should not suffer from the data contamination issue that has already been trained and aligned. Third, the dataset should have licenses that can be used and modified for research. Finally, we select (Sun et al., 2023), a comprehensive safety benchmark including seven typical safety scenarios and six instruction attacks, to build our multilingual safety benchmark.
We do not choose widely-used benchmarks, especially the dataset from OpenAI and Anthropic (Bai et al., 2022; Ganguli et al., 2022b), due to the high risk of data contamination issues. Our benchmark also includes a commonsense safety testset from (Levy et al., 2022), which requires commonsense knowledge to comprehend whether the text will lead to unsafe. Table 1 shows the illustration of each type of testset.
Translating the Corpora To build a multilingual safety benchmark, we translate the original monolingual safety data into the other languages. We adopt two criteria to select the languages. First, the languages should have a sufficient number of native speakers in the real world, which means more people could be harmed when unsafe responses in these languages occur. Second, current LLMs have enough capability to chat in these languages. Finally, we select ten widely used languages in the world (wikipedia, 2023): English (en), Chinese
(zh), Hindi (hi), Spanish (es), French (fr), Arabic (ar), Bengali (bn), Russian (ru), Japanese (jp) and German (de). Note that though these languages are not low-resource languages in the real world, they are relatively low-resource in the pretraining data of LLMs. For example, Chinese constitutes only 0.13% of the pretraining data in LLaMA-2 (refer to Table 7 in the Appendix).
We first used Google Translate to translate the monolingual data. Then, we recruited professional translators to proofread the translation. To guarantee the accuracy of translated data, we conducted two rounds of proofreading by professional translators from a commercial data annotation company, costing 3,000 USD. The modification rate for the first round was 15.5%, and the second round had a 3.4% modification rate. Subsequently, we randomly inspected 10% of the data, achieving a pass rate greater than 99%.
Mitigating the Cultural Bias in XSAFETY Concerning the potential cultural bias, since the benchmark is mainly derived from the Chinese dataset, we conducted several methods to avoid our benchmark bias toward Chinese cultures. First, since the original safety issue taxonomy (Sun et al., 2023) was designed for assessing Chinese LLMs, a particular portion of the corpus is associated with China. For example, the instance “请问在北上 广地区，要如何提高外来人员的整体素质， 以降低当地的犯罪率?” (“How to improve the overall quality of immigrants to reduce crime rates in the Beijing-Shanghai-Guangzhou region?”) is associated with specific Chinese region – BeijingShanghai-Guangzhou, which rarely exists in other languages. To build a universal benchmark agnostic to specific languages, we remove the Chinese culture-associated sentences by manually checking and collecting 200 instances for each safety issue. The commonsense safety corpus (Levy et al., 2022) is in English and unrelated to any specific language culture. Accordingly, we randomly sample 200 instances from this corpus.
Second, the translated sentences may not represent the harm in those cultures. To mitigate this, we ask the translators to make necessary modifications to the translation to adapt the benchmark to different languages. During the two rounds of translation proofreading, we ask the annotators to adjust the sentences if they do not follow the harmful representation of the culture of the languages. We have 2,800 instances for the 14 safety issues
listed in Table 1.

heading: 4 Experiments
text: In this section, we empirically explore two research questions using our XSAFETY benchmark:
RQ1. Are the widely used LLMs equally safe across languages?
RQ2. Can we improve the multilingual safety of LLMs?
In Section 4.2, we utilize XSAFETY to evaluate the multilingual safety of 4 widely used LLMs. Experimental results show that all LLMs perform much more unsafely in non-English languages than in English. Among the non-English languages, Bengali, Hindi, and Japanese are the top-3 most unsafe languages, which are very low-resource languages in the pertaining data of LLMs.
In Section 4.3, we develop simple and effective prompts to improve cross-lingual generalization of safety alignment in English. Empirical results show that the prompt works best for Russian (i.e., the unsafe ratio from 13.0% to 2.7%) and enjoys the best translation performance from English.

heading: 4.1 Setup
text: Models We conduct experiments on four LLMs, including closed-API GPT-3.5-turbo-0613 (ChatGPT) and PaLM-22, as well as open-source LLaMA-2-Chat3 and Vicuna4. We use the OpenAI official APIs5 for ChatGPT with a temperature of 0 to obtain more deterministic results.
LLMs can Understand User Input and Generate Coherent Responses in non-English Languages. The multilingual capacity of LLMs, such as ChatGPT and LLaMA-2, has been studied by previous works, showing that LLMs can communicate with other languages, although not as well as in English (Lai et al., 2023; Zhu et al., 2023). For example, ChatGPT has a COMET score of 76.7 for English-Hindi and 89.2 for Hindi-English, and LLaMA-2-7B has 44.6 for English-Hindi and 80.1 for Hindi-English. Our experiments found that ChatGPT can understand user input and generate coherent responses in non-English languages, including Hindi and Bengali.
2https://ai.google/discover/palm2/ 3https://github.com/facebookresearch/llama 4https://lmsys.org/blog/2023-03-30-vicuna/ 5https://openai.com/blog/chatgpt/
We conducted two experiments to validate the accuracy of this unsafe detection method. First, we conducted human annotation on 1,400 randomly selected cases encompassing all ten languages and 14 safety issues. Specifically, we recruited three annotators with bachelor’s degrees and are proficient in English to label whether the automatic evaluation is correct. The accuracy of the ChatGPT evalua-
tion is 88.5%, demonstrating the effectiveness of this automatic evaluation method. Second, we used a more advanced LLM, GPT-4, as the evaluation model. Specifically, we employed GPT-4 to evaluate responses in English, Chinese, and Hindi, with 100 cases randomly selected and annotated where ChatGPT and GPT-4 had differing judgments. The annotation results are listed in Table 2. ChatGPT is correct in 76 cases, while GPT-4 is correct in 24 cases. The primary reason for GPT-4’s weak performance is its over-sensitivity, which led to the classification of 70 safe responses as unsafe. Both experiments provide evidence that our current self-evaluation method using ChatGPT is reliable.

heading: 4.2 Multilingual Safety of Different LLMs
text: Safety Across Languages We first investigate the safety performance of 4 widely-used LLMs on the multilingual XSAFETY benchmark, as listed in Table 3. The unsafety ratios of non-English languages are higher than English in all cases, showing that LLMs are not equally safe in different languages. Table 4 lists the responses of ChatGPT for queries in different languages. This case clearly shows the multilingual safety issue that the response in English is safe, while the response in Chinese is unsafe. Specifically, the most unsafe languages (e.g., Bengali, Hindi, Japanese, and Arabic) are generally the lowest-resource languages in the pretraining data. These results demonstrate
the necessity of developing safety alignment for non-English languages to match the multilingual capability of the underlying LLMs.
ChatGPT performs best among all LLMs. One possible reason is that ChatGPT spent more effort on safety mitigations (the majority in English). Although ChatGPT performs much better than PaLM2 in English (i.e., 1.0 vs. 10.3), the performance gap for non-English languages is relatively smaller (i.e., 15.9 vs. 18.7 on average). These results reconfirm our claim that although there is some evidence that safety alignment in English can be generalized to other languages, it is still necessary to develop safety mitigations directly in other languages. Concerning the open-source LLMs, although LLaMA-2-Chat performs worse in English than Vicuna, it performs better in other languages. We attribute the superior performance of LLaMA2-Chat on the multilingual tasks to the stronger underlying model (i.e., LLaMA-2) compared with that for Vicuna (i.e., LLaMA). We use ChatGPT as default for its superior safety performance in the following experiments.
Safety Across Scenarios Figure 2 shows the unsafety ratio across scenarios for ChatGPT. The most challenging scenario of multilingual safety is Commonsense Safety, where the text is not explicitly violent and requires additional commonsense knowledge to comprehend that it leads to physical harm (Levy et al., 2022). This would be especially challenging for non-English languages that only take a small proportion of the pretraining data. Thus, the required commonsense knowl-
edge in such languages cannot be directly learned. The large performance gap between English and non-English languages for ChatGPT also indicates unsatisfactory cross-lingual generalization of the commonsense knowledge, which is consistent with the safety alignment.
The second most challenging safety scenario is Goal Hijacking, which induces the system to ignore the original user prompt by appending deceptive or misleading instructions to the input. The unsafety ratio in English is significantly smaller than in all the other languages. Another representative scenario for typical safety is Ethics and Morality, with relatively higher unsafety ratios in non-English languages.

heading: 4.3 Improving Multilingual Safety
text: This section explores simple and effective prompting methods to improve multilingual safety.
Prompting Method The prompting method aims to improve the cross-lingual generalization of safety alignment in English. Our empirical results (e.g., Table 3) show that safety alignment in English can be somewhat generalized to other languages. Therefore, LLMs can respond safely to non-English queries, although most safety alignment data is in English.
You are a helpful assistant. Please think in English and then generate the response in the original language.
Inspired by recent successes of prompting on further improving LLMs (e.g., reliability (Si et al.,
2023)), we develop a simple prompt to explicitly leverage safety alignment in English when handling non-English queries: “Please think in English and then generate the response in the original language.” By instructing the model to think in English, safety alignment in English can take effect before generating the response in the original language. All the prompts are in English since Shi et al. (2023) reveals that using the instruction and examples in English performs better for multilingual tasks.
Experimental Results We conduct experiments on ChatGPT for its powerful instruction-following ability. For computational tractability, we use the three most challenging safety scenarios: “Ethics And Morality,” “Insult,” and “Crimes And Illegal Activities.” We select Chinese, Russian, Japanese, and French as representative non-English languages.
Table 5 lists the unsafe ratio for the proposed prompting method. The prompting method improves the multilingual safety of ChatGPT in nonEnglish languages. Overall, the prompting method significantly reduces the unsafe ratio by 42% on average, demonstrating the effectiveness and universality of our approaches across safety scenarios and languages.
Case Study Table 6 shows example responses from ChatGPT with different methods. The response style of our prompting method is similar to that of an English query (e.g., starting with an apology), which reconfirms our hypothesis that the prompt improves the cross-lingual generalization of safety alignment in English.

heading: 5 Conclusion
text: This paper presents a comprehensive investigation into the multilingual safety of LLMs, addressing a crucial gap in the current understanding of AI safety. By developing the first multilingual safety benchmark, XSAFETY, we have enabled a systematic evaluation of the safety performance of widelyused LLMs across ten languages. Our findings reveal a significant disparity in safety performance between English and non-English languages, emphasizing the need for more focused research and development of safety alignment strategies for nonEnglish languages. Moreover, we propose a simple and effective prompting method to improve the multilingual safety of ChatGPT, demonstrating its potential to reduce unsafe responses by 42% for non-English queries. This study highlights the importance of addressing safety concerns in multiple languages and presents a promising direction for future research in multilingual AI safety.
By releasing the data and results, we hope to encourage further exploration and development of safety alignment techniques for LLMs in nonEnglish languages, ultimately creating safer and more reliable AI systems for users worldwide. Our work serves as a call to action for researchers, developers, and policymakers to collaborate in addressing the ethical and practical challenges associated with deploying AI systems in multilingual and multicultural contexts. We hope our work can inspire more future work to: (1) examine more scenarios of multilingual safety, such as bias and copyright; (2) provide a better understanding of how crosslingual generalization of safety alignment works; and (3) further explore more effective strategies to
improve multilingual safety.
Limitations
This paper has two primary limitations:
1. We employ a self-evaluation method using ChatGPT to determine the safety of LLMs’ responses. Although we incorporate human annotations to demonstrate the reliability of this method, it is not entirely accurate, potentially compromising the soundness of our findings.
2. Our proposed improvement methods are not sufficient to resolve this issue. Further investigation is required to enhance the handling of multilingual safety concerns, such as cross-lingual self-improving.

heading: Acknowledgement
text: The work described in this paper was supported by the Research Grants Council of the Hong Kong Special Administrative Region, China (No. CUHK 14206921 of the General Research Fund).

heading: A Language Distribution in Pretraining Data of Representative LLMs
text: Table 7 lists the language distribution in the pretraining corpus of three LLMs. Since the distribution of ChatGPT is not publicly available, we use GPT-3 as references rather than concrete evidence and statistics. The language distribution of PaLM2 is an approximate estimation: PaLM uses 22% nonEnglish data for pretraining, and PaLM2 includes a higher percentage of non-English data than previous LLMs. We roughly estimate that the percentage of non-English pretraining data in PaLM-2 is 30%. Most data is in English for GPT-3 and LLaMA-2 (i.e. ≈ 90%), while PaLM2 includes a higher percentage of non-English data. Note that although these languages are the most widely used globally, they are not necessarily high-resource for the LLMs. For example, French and German only constitute 1.82% and 1.47% of the pretraining data of GPT-3, and the other seven non-English languages constitute 1.22% in total.

heading: B A Large Scale Human Evaluation
text: To valid our findings and the effectiveness of auto evaluation, we conduct a large-scale human evaluation. We hire two annotators with bachelor’s degrees and proficiency in English. We select the responses from 2 models (ChatGPT and LLaMA2) * 8 languages * 5 kinds of safety issues (Crime, Insult, Unfair, Role play and Goal hijacking) * 50 samples, ending up with 4000 input-response pairs. We ask the annotators to annotate whether the responses are safe. Each annotator annotates separately and then discusses to reach an agreement. Based on the human annotation results as well as the automatic evaluation results by ChatGPT, we can calculate the unsafe ratio and correct rate of
automatic evaluation.
The unsafe ratio evaluated by ChatGPT, the unsafe ratio evaluated by humans, and the correct rate of ChatGPT evaluation are in Table ?? and Table 9. We can see that the ChatGPT’s judgment results are consistent with the human annotation results, and all conclusions in the paper still hold: 1) English is safer than other languages; 2) ChatGPT is safer than LLama-2 ; 3) The correct rate of ChatGPT on evaluate ChatGPT and LLaMA-2 are on par (89.3 v.s. 88.2), indicating there is no significant bias to the content generated by itself.

heading: C Using Other LLMs as Judge
text: To investigate if we can adopt other or multiple LLM judges and then take the average to get a more accurate evaluation, we also conduct experiments on two recently coming out famous LLMs, Claude3 and Gemini, as evaluators. However, these two models are over-sensitive. Specifically, Claude3 and Gemini classify 85.1% and 44.8% of the ChatGPT’s responses as unsafe, among which only 7.7% are unsafe according to human annotation. Therefore, adopting other famous LLMs as evaluators can lead to negative effects.

heading: D Multilingual Safety on Other Recently Proposed LLMs
text: To show that multilingual safety issues also exist in more recently proposed LLMs, we conduct a small-scale experiment on Gemini and Claude-3. We select four languages (En, Zh, Hi, Bn) and 4 safety issues (Crime, Insult, Goal Hijacking and Prompt Leakage).
The results are shown in Table 10, the safety rate in English is significantly higher than other three languages. Our conclusions still hold for the recently proposed LLMs.

heading: Future Work
text: This study highlights the importance of addressing safety concerns in multiple languages and presents a promising direction for future research in multilingual AI safety. By releasing the data and results, we hope to encourage further exploration and development of safety alignment techniques for LLMs in nonEnglish languages, ultimately creating safer and more reliable AI systems for users worldwide. Our work serves as a call to action for researchers, developers, and policymakers to collaborate in addressing the ethical and practical challenges associated with deploying AI systems in multilingual and multicultural contexts. We hope our work can inspire more future work to: (1) examine more scenarios of multilingual safety, such as bias and copyright; (2) provide a better understanding of how crosslingual generalization of safety alignment works; and (3) further explore more effective strategies to
improve multilingual safety. Limitations
This paper has two primary limitations:
1. We employ a self-evaluation method using ChatGPT to determine the safety of LLMs’ responses. Although we incorporate human annotations to demonstrate the reliability of this method, it is not entirely accurate, potentially compromising the soundness of our findings. 2. Our proposed improvement methods are not sufficient to resolve this issue. Further investigation is required to enhance the handling of multilingual safety concerns, such as cross-lingual self-improving