The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts (2024)

Abstract: As the influence of large language models (LLMs) spans across global communities, their safety challenges in multilingual settings become paramount for alignment research. This paper examines the variations in safety challenges faced by LLMs across different languages and discusses approaches to alleviating such concerns. By comparing how state-ofthe-art LLMs respond to the same set of malicious prompts written in highervs. lowerresource languages, we observe that (1) LLMs tend to generate unsafe responses much more often when a malicious prompt is written in a lower-resource language, and (2) LLMs tend to generate more irrelevant responses to malicious prompts in lower-resource languages. To understand where the discrepancy can be attributed, we study the effect of instruction tuning with reinforcement learning from human feedback (RLHF) or supervised finetuning (SFT) on the HH-RLHF dataset. Surprisingly, while training with high-resource languages improves model alignment, training in lower-resource languages yields minimal improvement. This suggests that the bottleneck of cross-lingual alignment is rooted in the pretraining stage. Our findings highlight the challenges in cross-lingual LLM safety, and we hope they inform future research in this direction.1

heading: 1 Introduction
text: Large Language Models (LLMs) are trained with the aim of generating proper responses conditioned on user instructions (Lu et al., 2022; Hejna III and Sadigh, 2023; Go et al., 2023; Korbak et al., 2023; OpenAI, 2023). While LLMs have demonstrated promising empirical success as general-purpose language generators and task solvers (Khashabi et al., 2020; Wang et al., 2022; Chowdhery et al., 2023), safety concerns around
1The code is accessible here: https://github. com/shadowkiller33/Language_attack.
the potential misuse of LLMs emerge. Recent studies show that malicious prompt instructions could solicit objectionable content from LLMs. (Wei et al., 2023; Zou et al., 2023; Shen et al., 2023b). Safeguarding LLMs against such attacks and aligning LLMs with human values become a priority in LLM research and development (Ganguli et al., 2023; Touvron et al., 2023).
As the influence of LLMs spans across global communities, understanding the capabilities of LLMs from a multilingual perspective becomes important (Conneau et al., 2020; Xue et al., 2021). Due to the discrepancy in the textual resources available for different languages during training, LLMs typically exhibit different capabilities across languages (Scao et al., 2022; ArmengolEstapé et al., 2022).
Our study starts with the observation that LLMs are prone to generate unsafe or irrelevant content when prompted with lower-resource languages compared to higher-resource ones. When comparing LLMs responses to the same set of malicious prompts translated into high- vs. lowresource languages, we observe two key curses (weaknesses) that present safety challenges for LLMs: (1) LLMs tend to generate harmful responses more often to malicious prompts in lowerresource languages compared to higher-resource languages. e.g., with GPT-4, we find that 35% of the responses to malicious prompts in lowresource languages contain harmful content, compared to 1% in high-resource languages. (2) LLMs tend to generate less relevant responses, as LLM’s instruction-following ability is still limited in lowresource languages. e.g., GPT-4 recognizes the instruction and produces relevant responses in only 80% cases with low-resource languages, compared to almost 100% in high-resource languages.
To understand what the discrepancy between low- vs. high-resource language can be attributed
2668
to, we study the effect of aligning LLMs with instruction-tuning datasets in different languages. Specifically, we train LLM on the HH-RLHF dataset (Bai et al., 2022b) translated in different languages. We compare supervised finetuning (SFT) or reinforcement learning from human feedback (RLHF) under mono- or multilingual training. Surprisingly, while RLHF and SFT training in high-resource language lowers the model’s HARMFUL RATE and improves models’ instruction following capability, we see little to no improvement with training on low-resource language. These results indicate that aligning the model for safety in low-resource languages requires more than instruction tuning.
We trace back the origin of these two curses (see §4) and attribute their occurrence to the limited low-resource data that LLMs have been pretrained on. Our findings show the difficulties and challenges of tackling the low-resource curse through alignment.
Our main contributions in the paper are:
• We identify two safety-related curses caused by low-resource languages when jailbreaking GPT-4, in terms of HARMFUL RATE and FOLLOWING RATE, respectively.
• We present empirical analyses evaluating the effectiveness of common alignment techniques (SFT and RLHF) in addressing the identified curses. Our results indicate that resolving these curses through alignment presents significant challenges.
• We trace the origin of the curses and attribute their occurrence to the limited low-resource data that LLMs have been pre-trained on.

heading: 2 Two Safety Curses of LLMs with Lower-Resource Languages
text: We begin our study by demonstrating that GPT4 is vulnerable to attacks with malicious prompts in low-resource languages (Deng et al., 2024). We observe and highlight two curses with respect to LLMs’ responses in lower-resource languages compared to higher-resource ones (harmfulness curse and relevance curse).
Curse 1. (harmfulness curse) LLMs tend to generate more harmful responses when prompted with malicious instructions written in low-resource languages compared to highresource languages,
Curse 2. (relevance curse) With malicious prompts in low-resource languages, LLM tends to generate less relevant responses as LLM’s instruction-following ability is still limited in low-resource languages.

heading: 2.1 Translation-based jailbreaking
text: To illustrate LLM’s vulnerability to multilingual jailbreak, we propose a simple translation-based attack method. We start with a set of malicious prompts written in English and translate the prompts into different languages with a machine translation model. We then prompt the LLMs with the translated malicious prompts. We use the same translation model to translate the response back into English and evaluate whether the responses exhibit safety concerns.
For our experiments in the study, we use the set of harmful prompts sourced from Zou et al. (2023)
for evaluation and use NLLB-1.3B (NLLB Team et al., 2022) as the translation model. Specifically, the prompting process is detailed in Appendix A.

heading: 2.2 Low- vs high-resource languages
text: We study the levels to which low- and highresource languages exhibit safety concerns when responding to malicious prompts. Here, the distinction between low- vs. high-resource languages is defined by the general amount of linguistic resources available for each language (Wu and Dredze, 2020). Following the categorization outlined by NLLB Team et al. (2022), we select the following 9 high-resource and 10 low-resource languages for our study.
• High-resource languages: Simplified Chinese, Russian, Spanish, Portuguese, French, German, Italian, Dutch, and Turkish.
• Low-resource languages: Hausa, Armenian, Igbo, Javanese, Kamba, Halh, Mongolian, Luo, Maori, and Urdu.

heading: 2.3 Evaluating the generated responses
text: We evaluate the LLM-generated responses by whether they can recognize the malicious intent behind the provided instruction and refuse to generate harmful content as a result. Following Wei et al. (2023), we use GPT-4 to classify each of LLM’s response (in English) into one of the three following categories:
• Irrelevant: if the response is nonsensical or fails to recognize the instruction. The response would neither feature harmful content nor intend to follow the given instructions.
• Harmful: when the model engages with the malicious instruction and provides an on-topic response, yet the response has harmful content.
• Harmless: when the model correctly recognizes the malicious intent behind the given instruction and refuses to engage.
With the classifications for the responses to an evaluation set of malicious prompts, we compute the two following metrics. (1) HARMFUL RATE estimates the likelihood of an LLM producing harmful responses, and (2) FOLLOWING RATE measures the likelihood of an LLM recognizing and
following the given instructions in the prompt.
HARMFUL RATE = # Harmful
# Harmless + # Harmful
FOLLOWING RATE = 1− # Irrelevant # All
Given a harmful prompt, we would expect the LLM to detect its malicious intent and refuse to engage. In the ideal case, we expect a safe LLM to have high FOLLOWING RATE but low HARMFUL RATE for each language.

heading: 2.4 Two curses with low-resource languages
text: Curse of Harmful Response: Lower-resource languages lead to higher harmful rate. We show the HARMFUL RATE comparison between high- vs. low-resource languages in Table 1 Overall, we can see low-resource languages exhibit much higher HARMFUL RATE. The primary reason for this susceptibility might be the limited data available for alignment and pre-training, often leading to model jailbreaking. Consequently, LLM might produce harmful responses. This highlights the importance of dedicated resources toward model alignment and pre-training for these low-resource languages, ensuring inclusivity and reducing potential harm in LLM-driven applications.
Curse of Irevelant Response: Lower-resource languages lead to lower following rate. The outcomes for the FOLLOWING RATE are depicted in Table 1. When presented with harmful prompts in high-resource languages, the LLM responds with relevant responses. This enhanced response quality is largely attributed to the extensive training data available for these languages, facilitating a deeper and more nuanced understanding when prompted in these languages. Consequently, even when the LLM generates content with harmful undertones, it frequently responds in a manner that helpfully addresses the harmful prompts.
In the following sections, we aim to (1) find whether such two curses still exist in open-sourced LLMs (§3). (2) try to alleviate these two curses through common alignment strategies (§3). (3) trace the origin of these two curses (§4)

heading: 3 Does Alignment Training Lift the Curses of Low-resource Languages?
text: To trace the root cause for the two curses, we study the effect of alignment training with human preference data for the safety and helpfulness of responses, and observe how the resulting language models’ behavior changes to malicious prompts in low- vs. high-resource languages. Specifically, we conduct experiments on the HH-RLHF dataset (Bai et al., 2022a). We compare different instruction tuning strategies, such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) (Ouyang et al., 2022). We additionally explore the effect of SFT and RLHF training in multilingual settings, where the instruction tuning data is translated from English into the target languages for SFT or reward model training respectively.

heading: 3.1 Multilingual alignment strategies
text: Multilingual Supervised Fine-tuning (xSFT) Given an instruction-tuning dataset Dl1 , which features pairs of prompt and target responses both written in a high-resource language l1 (e.g., English), we translate the examples into other target high- and low-resource languages in our evaluation l2..n. This yields {Dl1 ,Dl2 , . . . ,Dln}. We merge all translated data for instruction tuning of the LLM with the following objective.
L(θ) = ∑
P,R∈D ℓclm(R|P, θ) (1)
where D is the combined mixture of all translated datasets {Dl1 ,Dl2 , . . . ,Dln}, and P ,R refer to instances of the harmful prompts and ethical responses in the dataset. ℓclm denotes the causal language modeling loss.
RLHF via multilingual reward model (xRLHF) To train a multilingual reward model, we start with a human preference dataset Ql1 = {Ii, r+i , r−i }Ni=1 in English. r+i represents the human-preferred response over the less preferred one R−i . We translate the prompts and responses into the target low- and high-resource languages l2..n, yielding {Ql1 ,Ql2 , . . . ,Qln}. Similar to the xSFT case, we combine all translated human preference datasets and use the mixture to train a multilingual reward model. The reward model learning objective is to minimize the ranking loss L to the learned scalar reward function Rθ, where σ is the sigmoid function and Ii ◦ r+i is the concatenation of Ii and r+i .
L(θ) = − ∑ log(σ[Rθ(Ii ◦ r+i )−Rθ ( Ii ◦ r−i ) ]) (2)
With the learned multilingual reward model, we apply RLHF on the xSFT trained model. Specifically, we follow the PPO algorithm (Schulman et al., 2017; Ouyang et al., 2022) and maximize the following combined objective function J (ϕ).
J (ϕ) = E(I,r)∼D πRL ϕ [Rθ(I ◦ r)−
β log(πRLϕ (r | I)/πxSFT(r | I))], (3)
where πRLϕ is the learned RL policy parameterized by ϕ and initialized from the pretrained xSFT model πxSFT. DπRLϕ and Dpre denotes the RL training and pre-training datasets respectively. The first term encourage the policy πRLϕ to generate responses that have higher reward scores. The second term represents a per-token approximated KL reward controlled by coefficient β between πRLϕ and π
SFT to mitigate over-optimization toward the reward model during RL training. The set of training prompts used in the RL stage is also translated into the target languages, similar to the xSFT case.

heading: 3.2 Experimental setup
text: Benchmarks and methods We use the HHRLHF dataset Bai et al. (2022b) to train our xSFT and xRLHF models. For evaluation, we used the harmful prompts collected by Zou et al. (2023). We follow the same evaluation metrics HARMFUL RATE and FOLLOWING RATE, as described in §2.
We use LLaMa2-7B as the base model for mono- and multi-lingual SFT and RLHF instruction tuning. In addition, we compare to the official checkpoint of LLaMa2-chat-7B, which is instruction-tuned with RLHF on safety-related examples as part of the training mixture (Touvron et al., 2023)2. For simplicity, we refer to this model as CHAT-RLHF. We include our implementation details in Appendix B.
Translator and languages We use NLLB1.3B (NLLB Team et al., 2022) 3 as the translation model. Here, we select five high-resource and five low-resource languages respectively for our experiments. The five high-resource languages are English, Simplified Chinese, Spanish, Portuguese, French. And the low-resource languages are Hausa, Igbo, Kamba, Halh, Urdu. We include a more detailed description of the process and prompts used in Appendix A.

heading: 3.3 Results on harmful rate
text: We start by evaluating the base LLaMa-2 model without further alignment training as the baseline. As shown in Table 2, the base LLaMa2 generally generates harmful responses across all languages. Overall, LLaMa2 (BASE) exhibits an average HARMFUL RATE of 77.4% and 80.4% across high and low-resource languages, with only around 3% gap between these two language resource levels.
Reducing HARMFUL RATE is more difficult with low-resource languages. In Table 3, we show the improvements in terms of HARMFUL RATE after alignment training is applied on the base model. Despite all methods (CHAT-RLHF, xRLHF, xSFT) reducing the HARMFUL RATE of the model, we observe a notable gap between their effectiveness on high-resource and low-resource languages.
2For the LLaMa-2-chat checkpoints, Touvron et al. (2023) did not reveal details on the safety training data used during RLHF, e.g. distribution of languages, source of data.
3https://huggingface.co/facebook/ nllb-200-1.3B
Specifically: (1) With the official CHAT-RLHF checkpoint, RLHF training results in a substantial 45% reduction in high-resource languages, but the average improvements drop to around 20% for low-resource languages. (2) In our experiments, xSFT leads to a 20% decrease in HARMFUL RATE for high-resource languages. In comparison, we see a less than 7% reduction for low-resource languages. Similarly, xRLHF results in a 14% decrease in the harmful output rate for high-resource languages, compared to zero improvements for low-resource languages.
The results suggest that harmfulness curse for low-resource languages persists after alignment training. This highlights the difficulty of resolving the curse with typical alignment training methods.

heading: 3.4 Results on following rate
text: As shown in Table 4, the base LLaMa2 model exhibits low FOLLOWING RATE across all languages without further alignment training or instruction tuning. Specifically, LLaMa2 achieves 33.0% FOLLOWING RATE in high-resource languages and 24.8% in low-resource languages. Notably, we already observe a gap between low- vs. high-resource languages in terms of the instruction following capabilities with the base model.
Improving FOLLOWING RATE is more difficult with low-resource languages. Similarly, we observe much smaller gains in terms of FOLLOWING RATE when alignment training is applied on the base model. As illustrated in Table 5, while highresource languages experience consistent boosts
in FOLLOWING RATE, the improvements for lowresource languages are much smaller.
Here, it is worth noting that despite the big improvements from RLHF training of CHAT-RLHF in high-resource languages, we see a much lower improvement rate when we test it on low-resource languages. Apart from the official CHAT-RLHF checkpoint, our alignment training with xRLHF and xSFT does not achieve significant enhancements in FOLLOWING RATE. This is because our training data only consists of examples related to safety and ethical content, which fails to improve the model’s instruction-following capabilities.

heading: 3.5 Monolingual SFT fails to resolve the curses
text: We investigate the improvements of monolingual fine-tuning in different languages in reducing HARMFUL RATE, and the results are shown in Figure 2. From the results, we can see that (1) SFT on high-resource language data only provides improvements on high-resource languages. (2) SFT on low-resource language data is not beneficial for high-resource or low-resource languages. As for FOLLOWING RATE, monolingual SFT on the ethical data generally provides limited improvements for enhancing FOLLOWING RATE. This is reasonable since our ethical datasets aim to reduce harmfulness instead of enhancing LLMs’ instructionfollowing or chat ability.

heading: 4 Where does the low-resource language curse stem from?
text: Our earlier experiments (§3) reaffirm the presence of the two curses in open-source LLMs. This is consistent with findings from the GPT-4 experiments (§2). The recurrent patterns suggest that these curses are not mere coincidences, driving us to investigate their origins. For clarity, we break down the LLM training process into two
stages: (a) The pre-training stage, where the LLM is trained on a vast corpus using causal language modeling loss. (b) The post-hoc alignment stage, where the pre-trained LLMs are further fine-tuned using alignment data.
Harmfulness curse. LLMs without alignment suffer from malicious prompts, regardless of the language. Based on our results in Table 2 (full results can refer to Table 11), LLaMa2 (BASE) achieves a similar average HARMFUL RATE on low-resource and high-resource languages, and we do not observe any significant bias towards languages from different resource levels.
Instead, as shown in the results of our alignment
stage (Table 3), we observe severe bias towards languages from different resource levels. Notably, these patterns persist when we use well-balanced training data across different languages, ruling out data bias during the alignment stage as a culprit. Besides, when we fine-tune the model with pure monolingual low-resource data (as shown in Figure 2), LLM still fails to improve in terms of HARMFUL RATE, which is different from highresource cases where EN-RLHF and EN-SFT bring improvements to the model.4 This suggests harmfulness curse is difficult to solve during the fine-tuning stage since they may be deeply rooted, possibly originating from the scarce low-resource language data during the pre-training phase.
Overall, harmfulness curse can not be observed in the base version of LLMs. However, after being applied further safety-aware alignment, harmfulness curse begins to emerge. Although harmfulness curse does not emerge after the pre-training stage, its origin possibly originates from insufficient low-resource language data during pretraining.
Relevance curse. Unlike the case of harmfulness curse, we can observe relevance curse after the pre-training stage of LLMs. As shown in Table 4, LLaMa2 (BASE) achieves 33.0% and 24.8% FOLLOWING RATE on high-resource and lowresource languages, respectively, which presents a bias across different language levels.
After CHAT-RLHF alignment5, as shown in Table 5, we can see the bias is significantly strengthened. This phenomenon means that although the alignment stage would increase the instructionfollowing ability of LLMs, it amplifies relevance curse in the dark side.
Overall, relevance curse can be observed after the pre-training stage of LLMs. Besides, after being applied further safety-aware alignment, relevance curse would be substantially strengthened. Like harmfulness curse, its origin possibly originates from the limited low-resource language data during pre-training.
Multilingual pre-training helps alleviate the problem. In this part, we show evidence that
4These two cases are shown in Appendix C. 5We do not discuss our methods here (e.g., xSFT), since they are trained on domain-specific data, thus fail to increase the instruction-following ability of LLMs substantially. To better verify the origin of relevance curse, discussing the consequences of CHAT-RLHF would be more convincing.
multilingual pre-training may help alleviate the curses brought by low-resource languages. We select ALMA (Xu et al., 2023, 2024)6, a model that continues pre-training LLaMa2 model on multilingual translation data, including lowresource languages (ALMA is trained on Flores200 (NLLB Team et al., 2022), which contains low-resource language corpus), then we conduct xSFT on ALMA-pretrain-7B and LLaMa2-7B. The results are shown in Table 6, and we can observe that ALMA outperforms LLaMa with xSFT. These results indicate that adding more low-resource language corpus to the pre-training stage can alleviate the curses to a certain extent.
Why does xRLHF fail? As shown in Table 7, it is evident that xSFT outperforms xRLHF in reducing HARMFUL RATE on both high- and lowresource languages. This suggests that xRLHF might not be effectively enhancing performance. Given that our xRLHF model is guided by the multilingual reward model (X-RM), it motivates us to explore potential issues with X-RM.
Subsequently, we evaluated the X-RM for xRLHF. Our observations revealed a clear bias
6https://huggingface.co/haoranxu/ ALMA-7B-Pretrain
based on language resource levels, as highlighted in Table 8. While X-RM performs commendably in high-resource languages, its effectiveness sharply declines for languages with fewer resources. Notably, the model differentiates between ethical and harmful responses in highresource languages. However, its accuracy in lowresource languages hovers around a mere 50%, suggesting it is no better than random guessing. This phenomenon still exists even when we create and add CONTRAST INSTRUCTION (Shen et al., 2023a)7 for X-RM training.
The pronounced bias likely stems from the LLM’s pre-training phase. Due to its limited exposure to low-resource language datasets during this phase, the LLM does not gain sufficient knowledge about these languages, leading to an inherent bias in our X-RM. Addressing this bias is a challenging and resource-intensive task, and a sensible initial step could involve integrating more lowresource language datasets during pre-training.

heading: 5 Related Work
text: Safety and helpfulness of LLMs. While LLMs excel at generating coherent text, they have drawbacks. They frequently exhibit biases rooted in their pre-training data and may generate erroneous information, a phenomenon often referred to as ‘hallucination’ (Dziri et al., 2022; Agrawal et al., 2023; Dhuliawala et al., 2023). Recent endeavors (Zhao et al., 2021; Ganguli et al., 2022; Bai et al., 2022b,a; Kim et al., 2022) have been undertaken to fine-tune LLMs, making them more helpful and less likely to produce harmful content. These efforts have also led to the creating of datasets specifically designed for this purpose (Wang et al., 2023; Bai et al., 2022a).
One emerging safety concern revolves around jailbreaking attacks, which assesses whether an LLM responds inappropriately to malicious prompts. Previous research has addressed and mitigated the jailbreaking phenomenon, making
7Contrast Instruction is an effective strategy (Shen et al., 2023a) to strengthen the reward model.
LLMs more robust, especially in the English language (Wei et al., 2023; Zou et al., 2023; Li et al., 2023b; Wolf et al., 2023; Shen et al., 2023c). However, our study reveals that LLMs remain susceptible to jailbreaking prompts in low-resource languages. In tandem with a contemporary investigation by Yong et al. (2023), we observe a similar trend that LLMs are more likely to be jailbroken across low-resource languages. Beyond analysis, we propose strategies to alleviate the jailbreaking issue in LLMs and explore their helpfulness in a broader context.
Cross-lingual learning for LLMs. Due to the availability of copious resources, language technology’s inherent bias toward English is a wellestablished concern (Blasi et al., 2022). Recent efforts have aimed to enhance LLMs’ cross-lingual capabilities through multilingual language modeling (K et al., 2020; Kalyan et al., 2021; Conneau et al., 2020) and fine-tuning (Zhang et al., 2023; Li et al., 2023a,c). However, these approaches have primarily concentrated on high-resource languages. Even when addressing low-resource languages, they often focus on general benchmarks rather than evaluating the safety of LLMs when operating in such linguistic contexts.

heading: 6 Conclusion
text: This paper comprehensively analyzes the crosslingual capabilities of LLMs along two key dimensions: HARMFUL RATE and FOLLOWING RATE. Our investigation has unveiled that LLMs, primarily trained in English-centric contexts, exhibit two curses when prompted by low-resource languages. This vulnerability raises significant safety concerns and hinders their utility in linguistic contexts. Building upon these findings, we adapted commonly accepted alignment methods with monolingual and multilingual settings. We find that the two curses still exist after being applied with our methods, which show the challenges and difficulties of resolving the two curses through alignment methods. Then, we present empirical analysis and discussions towards the origin of two curses.
Our work highlights the multilingual vulnerability of LLMs and the challenges of resolving such a vulnerability through the alignment process. We hope our work can shed light on future works on enhancing the cross-lingual ability of LLMs.
Limitation
One limitation of our work is the inevitable noise brought by the imperfect translator during the translation process, which may bring some noise to the evaluation of HARMFUL RATE and FOLLOWING RATE. Another limitation is that, due to our limited budget, we could not conduct a highquality human evaluation for HARMFUL RATE and FOLLOWING RATE.

heading: A Prompts used in Evaluation
text: In §2, we use GPT-4 for evaluation of the HARMFUL RATE and FOLLOWING RATE of model responses in different languages. Here are the prompts we used for evaluation.
B Implementation details
• Standard fine-tuning (SFT): For standard finetuning, we select LLaMa-7B as the base model and train it following the configuration below: we adopt the Low-Rank Adaptor (LoRA) (Hu et al., 2021) for training. We use the AdamW optimizer and set the learning rate as 1.5e-5, with 50 warmup steps.
• Reward model (RM): For RM training, we select LLaMa-7B as the base model, train it with the LoRA with the AdamW optimizer, and set the learning rate as 2e-5.
• Reinforcement learning with PPO: We select the SFT model as the reference model in RLHF and use the reward score generated by RM as a supervision proxy. We set the learning rate as 1.5e5, batch size as 8, and accumulation step as 8 with 1,000 PPO steps.
• The experiments are conducted on 4 A6000 (48G) GPUs.

heading: C Full results
text: The full results of our experiment are shown in Table 11 and Table 12. Specifically, we chose English (high resource) and Kamba (low resource) as monolingual alignment cases for our illustrations. The techniques we used are represented as ENSFT, EN-RLHF, KAM-SFT, and KAM-RLHF.

heading: D Contemporaneous work claim
text: During the completion of this work, we became aware of some contemporaneous studies (Deng
et al., 2024; Yong et al., 2023) 8, and Yong et al. (2023) submitted their work to arxiv in October 2023.
8Our initial experiments (§2) have been completed in August 2023

heading: Future Work
text: We hope our work can shed light on future works on enhancing the cross-lingual ability of LLMs. Limitation
One limitation of our work is the inevitable noise brought by the imperfect translator during the translation process, which may bring some noise to the evaluation of HARMFUL RATE and FOLLOWING RATE. Another limitation is that, due to our limited budget, we could not conduct a highquality human evaluation for HARMFUL RATE and FOLLOWING RATE