Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation
Authors: Collin Zhang
Fei Huang
Chenhan Yuan
Junyang Lin
Venue: ICLR 2026 Poster (2026)
Keywords: llm
decoding intervention
language confusion
Primary area: foundation or frontier models, including LLMs

Abstract: Large language models (LLMs) often experience language confusion, which is the unintended mixing of languages during text generation. Current solutions to this problem either necessitate model retraining or cannot differentiate between harmful confusion and acceptable code-switching. This paper introduces the \textbf{Language Confusion Gate} (LCG), a lightweight, plug-in solution that filters tokens during decoding without altering the base LLM. The LCG is trained using norm-adjusted self-distillation to predict appropriate language families and apply masking only when needed. Our method is based on the findings that language confusion is infrequent, correct-language tokens are usually among the top predictions, and output token embedding norms are larger for high-resource languages, which biases sampling. When evaluated across various models, including Qwen3, GPT-OSS, Gemma3, Llama3.1, LCG decreases language confusion significantly—often by an order of magnitude—without negatively impacting task performance.

Published as a conference paper at ICLR 2026
LANGUAGE CONFUSION GATE: LANGUAGE-AWARE DECODING THROUGH MODEL SELF-DISTILLATINN
Collin Zhang!:2 22 Fei Huang', Chenhan Yuan', Junyang Lin' !wwen Team, Alibaba Group "Corelll University
ABSTRACT
Large language models (LLMs) often experience language confusion, which is the unintended mixing of languages during text generation. Current solutions to this problem either necessitate model retraining or cannot differentiate between harm- ful confusion and acceptable rode-switching. This paper introduces the Language Confusion Gate (LCG), a lightweight, plug-in solution that filters tokens during decoding without altering the base LLM. The LCG is trained using norm-adjusted self-distillation to predict appropriate language families and apply masking only when needed. Our method is based on the findings that language confusion is in- frequent, correct-language tokens are usually among the top predictions, and out- put token embedding norms are larger for high-resource languages, which biases sampling. When evaluated across various models, including Qwen3, GPT-OSS, Gemma3, Llama3.1, 1, LCG decreases language confusion significantly, often by an order of magnitude, without negatively impacting task performance.
INTRODUCTION
Large language models have made remarkable strides in multilingual understanding and genera- tion, with state-of-the-art systems like Qwen3 and GPT-5 now supporting over 100 languages and achieving strong performance on benchmarks such as FLORES 200 (Team et al. 2022) and XL- Sum (Hasan et al. (021). These models demonstrate impressive cross-lingual transter capabilities, enabling applications ranging from translation to multilingual creative writing. However, despite the most advanced L.LMs occasionally make seemingly elementary errors: generating text that mixes languages inappropriately. This phenomenon, known as language con- fusion (Marchisio et al. (224), occurs when a model outputs tokens from an unintended language family (e.g., inserting Chinese characters into an Hebrew sentence), undermining reliability and user experience. We show three examples of language confusion mistakes on left side of Figure [
their sophistication,
While recent improvements have reduced confusion rates in some models, the trend of Large Rea- soning Models seem to reintroduce the problem. As discussed in Guo et al. (2025). DeepSeek-RI exhibited significant language mixing during training, and applying language consistency reward led to measurable performance degradation, indicating trade-off between language consistency and reasoning capability. [Wang al. (202)) shows that the reasoning capability of LLM degrades when thinking in low resource languages, which explains why a reward purely based on outcome correctness encourages language confusion.
Further, our evaluation reveals that language confusion remains widespread, even among leading commercial systems. For instance, GPT-5-Chat exhibits 0.57% Chinese/Japanese (CJ) character confusion and 0.67% Latin-script confusion, while Qwen3-235B 2-5--A22B- A-----A22-I-nnneee Instruct-2507 suffers 2.27% CJ and 5.07% Latin confusion. These results confirm that language confusion is far from solved and affects both open-source and proprietary models.
The challenge of mitigating language confusion is the lack of an automatic way of evaluation, rule- based detectors struggle in distinguishing erroneous mixing from legitimate code-switching. a com- mon and often necessary linguistic behavior. In many practical scenarios such as writing English
Work done during internship at Qwen. Corresponding authors emails: Collin Zhang r
edu
Fei Huang Feihu. hf bbaaa- inc.com

---

Published as a conference paper at ICLR 2026
Transiate the to lollwing Chiness loal into Thai, Only
provide the tranclatinn
st 22 11 224) 15
Takan
Prob(5)
GPT-5 Chat
19.36
Wrong
18:36
minigskira
de fitur Ahuurnog anerigus www.uuuuuuuufffeeoo.
LLM
LM Head
floss nurertts 100 fffaaa
10.21
L9150 whraLE
11.13
Trandato to lowinng Engliel tn text into Korean Only provida translation
Ona
examples
Heruulatuuem vervin wrra??)
North Altenicc compriggn WWiil
Americans wero Erying move man and matoriata
scccso Fhe Azanto Ocean help Britin. Korcart
Qwen3-22588-222 Instruct
With Intervention
919 DiA
2200
0V (1
Next Taken Pred
01.21 2 SL
Allowed
Taken
Probe5)
peovide the mensdattonn
Trandiate lowing Eng th text into Heorew Only
Conlusion
Chinssse No
Languages
10.36
19.36
Correct Result: minuuttrra #emik/hunn
eggs,
legs
Symbool No
Tollwe Hestower
LowRes Yes
0.989
with
traphop
Figure I: Left: examples of language confusion in three LLMs on the left. Right: LCG takes the LLM's hidden state, predicts permissible language families (CJ, Latin, Symbols, LowRes), and dynamically masks logits for disallowed tokens only when necessary to correct the confusion on the left without altering the base model.
code with Chinese comments, using technical terms like ReLU or Python in non-English text, or explaining foreign phrases, the ability blend languages enhances expressivity and utility. Con- sequently, simply restricting LLM to output in single language doesn't 't work. Furthermore, the high number of supported languages requires find a method that be easily scaled large number of languages.
To address this, we analyze generation behavior at confusion points and make three key obser- vations: (1) language confusion occurs rarely soeelysuggesting the model generally knows the correct language; (2) the confusion point, the correct-language token typically ranks within top-5 candidates, indicating that L.MM knows the correct answer; and (3) A mechanistic observation that output layer token embedding norm imbalance makes LLM biased towards high- high-resource languages.
Leveraging these insights, propose the Language Confusion Gate (LCG): lightweight, plug- intervention that dynamically filters inappropriate tokens at decoding time without modifying the base LLM. LCG consists of a small two-layer MLP trained via self seff-distillation the frozen model's own top-k/p predictions, with norm adjustment used to debias inclination to high-resource language tokens. At inference, the gate predicts which language families (Chinese/Japanese, Latin, Symbols, Low Resource Languages) are pernissible at each step, and applies masking only when necessary.
Results show that LCG reduces language confusion by order of magnitude across multiple models and tasks. For example, on FLORES-NO-LATIH benchmark, our method reduces Chinese/lapanese confusion from 1.0% to 0.0% and Latin confusion from 4.4% 0.4% Qwen3-30B-A3B.
We make three key contributions that build upon our analysis of language confusion in LLMs: Firstly, we propose the Language Confusion Gate (LCG), efficient intervention mechanism that dynamically filters inappropriate tokens during generation without modifying the base LLM. Second, we introduce norm-adjusted self-distillation, leveraging mechanistic insights about token embedding norm imbalance to train the gate using the model's 's own debiased top-k/p predictions. Third, we collect and open-source specialized training and evaluation datasets, and evaluate LCG on open-source models covering diverse architectures and both thinking and no-think modes.
2 RELATED WORKS
Language confusion has emerged critical challenge in multilingual large language models (LLMs). Marchisio et al. (2024) formalized the concept of language confusion and introduced the I Language Confusion Benchmark (LCB), providing the first standardized evaluation framework for
2

---

Published as a conference paper at ICLR 2026
measuring cross-lingual interference in LLMs. Their analysis revealed that confusion often occurs at specific "confusion points" in the generation process, motivating targeted intervention strategies. They show that greedy decoding can help reduce language confusion but not climinate it, at the cost of potentially degraded L.MM performance.
Building on this foundation, Nie al. (2025) conducted mechanistic interpretability analysis to iden- tify neurons responsible for language switching behavior. They found that suppressing these critical neurons during inference reduces unwanted language mixing, suggesting that confusion is localized in the model's internal representations. Similarly, Ji et al. (2025) focused on Korean-language se- tups where Chinese character intrusion was observed, proposing a post-hoc smoothing method that identifies and suppresses Chinese tokens during decoding. Other approaches have explored different mitigation strategies.
Li et al. (2025) took a unique perspective by studying whether language mixing between English and Chinese could actually benefit reasoning performance. Rather than targeting suppressing lan- guage confusion, they trained a gating mechanism to predict when mixing helps harms task performance, selectively encouraging or discouraging it accordingly. proposed training models to prefer language-consisten responses through Odds Ratio Preference Optimization (ORPO), aligning model outputs with preferences for linguistic coherence.
contrast, [ee et al. (2025)
While these works represent important progress, they face limitations in practical deployment: some require model retraining or fine-tuning, others lack the ability to distinguish legitimate code- switching (e-g., technical terms bilingual education contexts) from erroneous confusion. Our work addresses these gaps by introducing lightweight, plug-in intervention that operates at confu- sion points without modifying the base model, while preserving valid multilingual behaviors.
3 CLOSER LOOK INTO LANGUAGE CONFUSION
3.1 CONFUSION POINT
Large language models (LLMs) generate text autoregressively by producing a probability distribu- tion over the vocabulary at each step. The next token is usually sampled using hybrid of top-k and top-p (nucleus) sampling. As demonstrated in Marchisio et al. (2024) and Nie et al. (2025), a confusion point arises when token with language different from the last token appears within the sampling tokens. We define the token in the different language as confusion token.
To better understand behavior of LLMs at confusion points, we use the LLORES-NO-LATIA dataset as described in Section 5.2 to trigger language confusion in Qwen3-8B. We inspect the token probability distribution of LLM at the confusion point, and find that the confusion token is the top-1 token 64%74% of the time, which makes greedy decoding ineffective to prevent language confusion. Further, we find that language consistent tokens appear within top-3 99. 99.2%% of the time. This suggests that language switching errors are not due to a complete absence of correct-language candidates in the model's output distribution, but rather to the model assigning insufficient prob- ability mass to them relative to competing tokens from the confused language. This observation motivates a logits based intervention strategy without modifying weights of the model. We can simply mask tokens in the undesired language families.
3.2 TOKEN EMBEDDING NORM ANALYSIS.
The magnitude of output token embeddings plays critical role in language confusion by favoring tokens from high high-resource languages.
Language models compute hidden states at each generation step and project them to vocabulary- sized logits through linear layer Wout E R.duude x Vl;
logits Wout" Th
Wost collection of individual column vectors, [co, ci. 1 where each vector e; repre- sents the output embedding for a specific token in the vocabulary, we define these as output token embeddings. Then we can decompose logit, as
logits, h e; =|[>|| ||ea|] cos sim(h, es).
3

---

Published as a conference paper at ICLR 2026
That is, the logit of each token logit; is the dot product between the hidden state h and that to- ken's embedding E,, and thus be decomposed into its geometric components: magnitude (norm) and direction (cosine similarity).
Since the norm of the hidden state, ||a|] is constant for all tokens at a given generation step, this decomposition reveals critical, often-overlooked factor: the output token embedding norm, |/e/|- shows that a token can achieve a high logit simply by having a large embedding norm. This creates systemic bias where tokens from high-resource languages develop larger norms, which sometimes causes language confusion.
We categorize the vocabulary by language family: CJ (Chinese/lapanese)) Latin, and Low-Res (low- resource languages). For each language family, we compute the fraction of tokens whose embedding norms lie in the top 5% of all token norms the model's vocabulary. As shown in Table [ the results confirm a significant imbalance: high-resource languages like Latin and CJ consistently dominate the high-norm group, while low-resource languages are heavily underrepresented.
Table I: Percentage of tokens in each language family with embedding norms among top 5% of all token embedding norms.
Model
CJ%
Latin
Low-Res%
Qwen3-8B 10.74
4.61
0.14
6.52
5.50
0.07
Llama3.1 1-8B
4.38
5.95
1.34
Gemma3-128
0.94
5.04
2.40
GPT-OSS
0.00
7.00
0.00
Adjustment
Adjustment
Rank
Token
Prob[%)
Norm
Token
Prob(%)
Norm
Translate the following English Hebrew. Only provide the translation. English: The views presented are often cursory, general and oversimplifled compared to the more detailed information available elsewhere.
25.17
1.6406
1.0703
'nn'
20.41
1.0703
25.56
1.0547
11.82
1.7268
'd'
9.77
1.0898
9.62
1.0547
2.04
1,2188
5.49
1.0838
'2'
1.40
0.9766
Hebrew:
Qwen3-8B
1,2188
.nitto nanp o'uuh aomn vna'' nemuna тт nibresi nioraoo J 1nx nipna pnpu union.
1911
2.03
1.5359
mme/
0.45
1.0196
1.6250
0.8633
'ub'
0.74
1.3047
"
0.38
1.1562
1.3780
1.3388
Figure 2: Top-10 Logits before and after applying confusion point Qwen3-8B
Heall [/flll-co_ssm// e;) removes the embedding norm bias, allowing tokens be ranked purely by their cosine similarity with the hidden state as shown in Figure 2 We can see that the initial highly ranked language confusion tokens disappear from the top tokens. This shows that norm-adjusted top-k tokens provides signal for correct next-token language family, and can use this signal train gate that predicts language family of next token as discussed in Section such errors but cannot fully explains language confusion. For example, it can't explain language confusion between English and Chinese since they both have high norm, or between low resource languages since they both have low norm, it can't 't be directly used for intervention.
Adjusting the logits by the token embedding norm: logit free
Norm bias can account for a subset of
We studied how percentage of low resource tokens in training data affects the output token norm in Appendix G

---

Published as a conference paper at ICLR 2026
3.3 LANGUAGE CONFUSION V.S. NATURAL LANGUAGE MIX
Language mix, code-switch, has been observed and discussed in both NLP and linguistic area (Dogruöz et al. 202T Winata et al. 2023). We show several examples that language mixing the context is necessary. 1) the use of English abbreviations or terms like Python, Java, iPhone. 2) Coding tasks. The user may prompt in Chinese to ask LLM to write code, while most programming languages are based on English characters. 3) Language study. We show examples of natural code- switch contexts in Appendix H The user may ask the language model to explain phrases in another language in English. In that case, the ability to several language in response should be pre- served, so simply constrain the LLM to output in single language won't t work. In that case, simply enforcing rule-based language consistency constraint won't work, since we have two objectives: suppressing unnormal language confusion while maintaining normal code-switch capabilities.
3.4 DOES SOTA COMMERCIAL LLMS SHOW LANGUAGE CONFUSION?
Even state-of-the-ee commercial LLMs exhibit non-negligible language confusion, confirming it as a widespread challenge across both open-source and proprietary models. We evaluate commercial models using our FLORES-NO-LATIN as described in Section 5.2 (Table 2 The results reveal that language confusion is widespread even among state-of-then art closed models. Notably, we are not sure that if similar intervention mechanism like Language Confusion Gate has been applied to any commercial models, but we can observe that all models show non-negligible Latin Confusion and CJ Confusion (except -.ouled-.) We show the full table covering more Commercial LLM confusion rates in Appendix
Table 2: Language Confusion Rates on the FLORES-NO-LATIN Benchmark for Leading Commercial LLMs. This table displays the percentage of responses containing erroneous Chi- nese/Japanese (CJ%) and Latin (Latin%) characters, alongside the task-specific BLEU score. These results highlight that language confusion persistent issue across various SOTA models.
Model
CJ%
Latin%
BLEU
GPT-5-Chat
0.57
0.67
10.66
Claude- 45odet
0.00
0.35
21.77
Gemini-2.5-Pra
0.04
0.50
19.11
DeepSeek-v3.
0.67
1.06
18.11
Bwen3-235B-Inm
2.27
5.07
15.43
4 METHOD
4.1 LANGUAGE CONFUSION GATE
We propose lightweight intervention mechanism to address language confusion without modify- ing the base LLM architecture or requiring model retraining. Our approach introduces Language Confusion Gate as shown in Figure D two layer MLP that determines language families allowed at each generation step. then masks inappropriate tokens from the logits during sampling. The gate itself is two layer MLP that takes the LLM's final hidden state h, input and produces language family logits 2, = (h, E R.. At each generation step, for each new token, the gate predicts more language families allowed for next token, and mask tokens in banned language. Inter- vention only happens when the tokens could be sampled under current top-k, top-p, and temperature parameters contain language families disallowed by the gate, Since language confusion happens rarely, it has minimal impact on overall generation.
Classify Tokens into Language Families. To enable the Language Confusion Gate, we clas- sify each token in the entire vocabulary into one of four mutually exclusive families: Chinese and Japanese (CJ), for tokens primarily composed of Chinese and Japanese characters; Latin, for to- kens representing Latin scripts; Symbuls, for punctuation, numbers, and special characters; and Low Resource Languages (Low- Res), the category for all other tokens.
5

---

Published as a conference paper at ICLR 2026
The classification is performed using prioritized heuristic. For each token in the vocabulary, first attempt to decode it from its byte-pair encoding (BPE) into Unicode characters. If the resulting characters contain any Chinese characters consist only of Latin script and symbols, it is classified as Latin. If a token decodes exclusively to symbols, is classified Symbols. All other tokens that decode to valid characters from other scripts are categorized as Low-Res.
Japanese script, the token is classified
CJ. If not, and the
known challenge with BPE is that some tokens may represent incomplete Unicode characters. In these cases, we analyze the partial byte sequence to infer its language family based on Unicode's continuous block structure. We discuss in more detail this method in Appendix [ If the family cannot be reliably determined, the token is conservatively classified as Symbols. Applying this methodology to the Qwen3 tokenizer (151,936 total tokens) yields the following distribution: 27,658 CJ. 94,666 Latín, 10,355 Symbols, and 19,257 Low-Res tokens.
4.2 TRAINING
Normeadjusted self-distillation. We train the gate with norm- norm-adjusted self-distillation, use the model's language prediction pscudo-targets targets, and remove the systemic advantage of high- norm tokens with norm-adjustment.
For the logit vector logits E IRI produced at a given step, we compute norm-adjusted logits, logits,djustr by dividing each token's logit by the norm of its output embedding vector ||e,]|2.-
With these debiased logits, we create multi-label pseudo-targets y for each generation step I. First, identify set of high-confidence candidate tokens, Sx.p filtering to the norm-adjusted logits. Then, we determine which language families are present in this candidate set. The pseudo-targe target for language family i is set to 1 if any token from that family appears in the set, and otherwise. This is formally expressed as:
by applying top-k/top-p
Vi.i 1 S
n.F; + 0],
where Fi the set of tokens belonging to language family (as defined Section 4.1
We train the gate to predict the pseudo-targets using standard binary cross-entropy (BCE) loss: Σr i-l BCE( (2).))), where the sigmoid function and n is the number of language families. We freeze weights of the LLM during training.
4.3 INTERVENTION RULES
During inference, we apply the LCG to dynamically mask tokens from disallowed language families each generation step. To mitigate potential side effects, supplement the gate's 'ss prediction with several intervention rules: (1) Symbols and Low-Res tokens are never masked. It's very rare for high-resource language to mix low-resource languages, We never mask symbols since they don't cause language confusion. (2) No intervention if the gate's prediction contradicted by high-confidence model output. If neither of the two high- probability candidate sets defined by (top-k = 5, top-p 0.999) or (top-k = 20, top-p = 0.95) contains any token from the preeepredeeted language family, we refrain from applying any mask. (3) Persistence of the previous token's 's language. To maintain linguistic coherence, we always allow the language family of the immediately preceding non-symbol token.
never mask Low-Res tokens.
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Models. Our experiments include both standard ("no-think")) and sesooing-focusee ("thinking")) large language models to ensure the LCG is effective across different architectures and capabilities. For no-think models, applied our intervention Qwen3-30B-A Beeen330B-A3B--Me 1..-00--3B--nntruut-222 Ins--nuuttttt2500 (Yang et al.)
2025), Qwen3-8B, Llama 3.1-8B (Dubey et al., 2024). and Gemma3-12B
(Team et al. al.] (025). For Qwen3-8B,
thinking models, evaluated our intervention and GPT-OSS-20B (OpenAT 2025). Notably, Qwen3-8B is a hybrid model and was used both experimental setups. In experiments, refer the gate trained with norm-adjusted self-distillation

---

Published as a conference paper at ICLR 2026
LCG-adjusted, while the gate trained only with self-distillation without norm-adjusttmnt LCG- unadjusted.
Training Data for the Gate. We trained the LCG composite dataset of approximately 78,000 samples covering over 200 languages to ensure it learns to handle a wide variety of linguistic con- texts. This same dataset was used to train the gate for both thinking and no-think models. The data was aggregated from several sources, including the Aya Dataset (Singh et al., (024) for diverse top- ics, the FLORES+ Dataset (NLLB Team et (2024) to generate translation pairs for low-resource languages, the DeepSeek Distill Dataset (Lightblue KK. 2022) for multilingual reasoning contexts, and the Alpaca (Taori et al. 2023 datasets (Chinese English) maintain strong performance in high-resource languages.
5.2 EVALUATION STRATEGY
Our evaluation is designed to confirm that LCG reduces language confusion without degrading task performance. We use different benchmarks for "thinking" and "no-think" models align with their distinct behavioral patterns.
Evaluation Datasets. We evaluate no-think models using the translation dataset FLORES+ for Arabie, Hebrew, Korean, and Thai, and the INCLUDE benchmark (Romanou al. 2024). a multi- lingual knowledge and reasoning dataset for Arabic, Hebrew, Greek, Russian, and Vietnamese. For thinking models, use Python problems from Humaneval-XL. (Peng et al. 2024) in Arabic and Hebrew, repeating each prompt 10 times to reliably detect confusion in reasoning-intensin tasks. Across all datasets, we measure both language confusion rate and standard task performance.
Evaluation Metrics. We define the language confusion rate as the percentage of model responses that contain at least one character from an unintended language script. Our evaluation focuses two primary types of confusion: Chinese/Japanese (CJ) confusion and Latin confusion. CJ confusion is straightforward measure using a rule- based detector, as legitimate code-switching into Chinese or Japanese characters is exceedingly rare in the target languages of our benchmarks. Consequently, we evaluate CJ confusion across datasets.
In contrast, Latin confusion presents more nuanced challenge due to the frequent and valid use of Latin-script tokens contexts such as programming code or mathematical notation. To address this, partition the FLORES+ dataset into two subsets: FLORES-NO-LATEE ground-truth references contain no Latin characters, so any Latin script in model output is considered erroneous. FLORES-WITH-LATTE
translations whose
translations where Latin characters appear in the reference and are thus permissible. This partitioning is performed by examining ground- truth translations from English into five target languages: Arabic, Hebrew, Korean, Thai, and Chinese, and flagging those that include Latin-script characters. We restrict our Latin confusion evaluation to the FLORES- NO-LATIN subset, where rule-based detection reliably identifies unintended language mixing.
Rationale for Not Using LCB. We use established multilingual benchmarks for our evaluation instead of the Language Confusion Benchmark (LCB) (Marchisio al. (024) for two reasons: (1) We observed that some LCB queries require natural code-switching, which could lead to unreliable confusion metrics. (2) Its language detector sometimes produce wrong results, which may result in false positives. Our methodology of using standard benchmarks with targeted filtering provides more robust and practical evaluation.
5.3 EVALUATION RESULTS
We evaluate the Language Confusion Gate (LCG) across both standard ("no-think")) and reasoning- focused ("thinking") models to assess its effectiveness in mitigating unintended language mixing and validate the importance norm adjustment. To examine LCG's impact on legitimate code- switching behavior, we conduct comparative analysis against reference models. Furthermore, benchmark LCG against established baseline methods to demonstrate its advantages in reducing language confusion while maintaining appropriate multilingual capabilities.
7

---

Published as a conference paper at ICLR 2026
Table Effectiveness of LCG Intervention on "No-Think" Models. No LCG is the case without intervention. BLEU scores are for PLORES-NO-LATIA, accuracy is for INCLUDE.
Qwen3-30B Llama3.1-8B
Gemma3-122
Qwen3-8B
FLORES-NO-LTTIN
CJ% (No LCG)
1.0
3.0
0.2
4.5
CJ% (LCG-unadjusted)
0.2
2.0
0.1
0.5
(L.G-adjusted)
0.0
0.4
Latin% (No LCG)
4.4
8.4
1.0
12.1
Latin% (LCG-unadjusted)
0.7
5.7
6.2
Latin% (I.CG-adjusted)
0.4
2.9
0.5
2.0
BLEU (No LCG)
13.2
11.3
16.9
12.1
BLEU (I.CGunnadjutted)
13.3
12.2
17.0
11.9
BLEU (LCG-adjusted)
13.4
12.3
17.1
12.1
INCLUDE
CJ% (No LCG) CJ% (LCG-unaa CGunndjjusted)
2.21
0.87
0.00
1.67
0.22
0,51
0.00
0,44
CJ%
(LCG-a) Cdjajsstted)
0.11
0.07
0.00
0.18
Accuracy (No LCG)
71.12
46.12
64.95
61.43
Accuracy LLCG-unadjustedd
71.55
46.12
65.02
62.84
Accuracy (LCG-adjusted)
70.83
46.34
65.75
61.76
Experiments on No-Think Models Intervention. LCG drastically reduces language confusion in standard (no-think) models by an order of magnitude, while maintaining task performance. We eval- uate the intervention effectiveness of the language confusion gate on nothink models on the FL.O- RES+ dataset and the INCLUDE dataset. On the FLORES+ dataset (Table [ the gate drastically reduces both CJ and Latin confusion. For example, Qwen3-30B-A3B-22
reduces CJ confusion from 1.0% to 0.0%, and Latin confusion from 4.4% to 0.4%, while maintaining stable BLEU scores. Llama3. 1-BB and Qwen3-8B show high language confusion rate without intervention, with LCG- adjusted intervention, Llama3. 1-8B's Latin% drops from 8.4% 2.9% and Qwen3-8B's Latin% falls from 12.1% to 2.0%. Results on the INCLUDE dataset (Table 3 also show significant reduc- tions in CJ confusion- from 2.21% % 0.1 1% in Qwen3-30B without degradation in task accuracy.
Frequency of Intervention. The intervention performed by LCG is sparse and precise, so it has minimum influence normal token generations. We measured the frequency of intervention for Qwen3-8B and 1lama3.1-8F on the PLORES-NO-LATII dataset. For Qwen3-8B, the intervention rate is 0.38%, which is 523 among 139354 tokens generated. For Llama3. 1-BB, the intervention rate is 0.33%, which is 545 among 162846 tokens generated.
Ablation of norm-adjustment. We compare the LCG-unadjusted against the LCG-adjusted and find that LCG-adjusted consistently achieves better performance. As shown in Table 3 LCG- adjusted further reduces both CJK and Latin confusion while preserving or slightly improving BLEU scores. For instance, on Llama3. 1-8B, Latin confusion drops from 5.7% (LCGGunadjjusedd) to 2.9% LLCG-adjusted), and on Qwen3-30B, it decreases from 0.7% to 0.4%. This demonstrates that train- ing with norm-adjustment produces gate with higher accuracy, leading to more precise suppression of language confusion. Thus, LCG-adjusted represents our final, optimized intervention.
Experiments on Thinking Model Intervention. LCG can also effectively reduce language con- fusion on thinking models. For reasoning-capable models, we evaluate on the humaneval-xi dataset (Table 4). Our intervention successfully eliminates Chinese character confusion-reduciin renunnirednnggg it from 0.38% to 0.06% in GPT-OSS and from 0.12% to 0.00% in Qwen3-30B whhle maintaining com- petitive Pass@1 and Pass @ 10 scores. This indicates that the gate effectively prevents language confusion during complex reasoning tasks without degrading the model's reasoning capability, its effect on reasoning length is also very small.
8

---

Published as a conference paper at ICLR 2026
Table 4: Effectiveness of LCG Intervention on "No-Think" Models measured on Humaneval-XI.. Length refers to the average Length of reasoning tokens used.
No LCG
LCG-adjusted
Model
CJ%
Pass 1
Pass@ 10
Length
CJ%
Pass
Pass 10
Length
Qwen3-8B
1.50
83.81
97.01
3327
0.06
83.13
96.67
3361
Qwen3-30B
0.12
91.25
97.83
2528
0.00
90.50
97.97
2534
GPT-Oss
0.38
85.88
98.07
501
0.06
84.56
98.32
507
Impact on normal code-switch. A critical challenge in mitigating language confusion ensuring that the intervention does not penalize legitimate code-switching, which is natural and often nec- essary aspect of multilingual communication. We find that although LCG reduces the frequency of legitimate code-switching, it preserves the model's code-switch ability. We measured LCG's impact on the WLLORE-WITH-LLTTE dataset, a subset of the FLORES benchmark where ground-truth translations contain Latin characters, indicating possibility of code-switch.
In our first experiment, we ran the Qwen3-8B No LCG on the FLORES-WITH-LATE WITH-LATIN dataset to generate translations. From these outputs, we select cases where the model's use of English was judged by human annotators to be natural, appropriate code-switch. We then applied Qwen3-8B LCG-adjusted on to these outputs determine whether would permit the English tokens each confusion point. We find that Qwen3-8B LCG-adjusted allows the English code-switch in 86.7% of these human-validated examples, indicating that it largely preserves legitimate code-switch.
In our second experiment, we ran the models with LCG on the FLORES-WITH--AE WITH-LATIN dataset. We define the "code-switch rate" as the percentage of responses that contain Latin characters. We compare the models' rates before and after intervention to two baselines: the rate in the ground-truth answers ("Answer Rate") and the rate of a strong baseline model: Claude Sonnet 4. Notice that these two baselines just references for comparison but not ground truth optimal code-switch rate.
As shown in Table 5 LCG does reduce the rate rode-switching across all models. For instance, the code-switch rate for Qwen3-8B from 46.34% to 25.90%. However, the most-intervention rates remain higher than the Claude Sonnet 4 baseline (23.29%) and not much lower than the ground-truth answer rate (38.36%) This suggests that while LCG makes models more cautious about mixing languages, it does not eliminate their ability to perform necessary code-switching. The intervention effectively moderates the behavior, preserving the model's 's capacity for legitimate language blending while suppressing erroneous confusion. We show examples that our LCG avoids language confusion and maintains natural code-switch Appendix
Table 5: Impact of LCG on legitimate swithsnig behavior. addition to these numbers, at the token level, LCG-adjusted allows English tokens 86.7% of the confusion points.
Model
No LCG
LCG-adjusted Answer Rate
Claude Sonnet 4
Llama3. 1-8B
42.51
31.60
38.36
23.29
Qwen3-8B
46.34
25.90
38.36
23.29
Gemma3-
30.94
25.57
38.36
23.29
Comparison with baseline intervention mechanisms. We compared our LCG-adjusted approach with three baseline intervention mechanisms: in-context learning (ICL., greedy decoding, and ORPO tuning as deseribed in[Lee et al. (2025). We show the prompt we used for ICI. in Appendix For the ORPO method, prepare multilingual dataset, and synthesize samples with language confusion as rejected samples similar as Lee et al. (2025). The results Figure demonstrate that, LCG most effectively reduce the language confusion rate while preserving model performance.
For instance, with the Qwen3-8B 3BB model, ICL only offers a marginal improvement, reducing the Chi- nese/Japanese character confusion (CJ%) from 4.5% to 4.2%. Greedy decoding provides similarly limited benefits, lowering the CJ% to just 4.2%. Since greedy decoding is the most conservative sampling strategy, this result implies that merely tuning other decoding parameters like temperature

---

Published as a conference paper at ICLR 2026
Ower2-43 INCJUD
Lamio ROLUCE
1.6
5.8
Figure 3: Comparison of LCG with baseline methods. CJ% and Latin% show CJ confusion rate and Latin confusion rate respectively.
or top-p would also be insufficient to resolve the language mixing issue. In contrast, our LCG- adjusted mechanism achieves a substantial and consistent reduction errors across all models. For Qwen3-8B, it decreased the CJ% from 4.5% down to 0.1 % and the Latin% from 12.1% to 2.0%. This shows our learned gate more targeted and effective solution than simple prompting or decoding-based interventions.
LCG also outperforms vaining-based methods. For instance, while ORPO achieves performance comparable to LCG on Llama3.1-8P 1-8B, it performs significantly worse on Qwen3-8B when evaluated on the FLORES-NO-LATIN dataset. Moreover, we observe that ORPO can degrade the model's general capabilities: on Qwen3-8B. INCLUDE accuracy drops from 61. to 57.3, and on Llama3. 8B, it declines from 46.1 to 43.2. This suggests that ORPO may sacrifice overall language under- standing in its attempt to reduce language confusion. We have also ablated the intervention rules as discussed Section [43 in the "No Rule" setup. We can see that LCG can still reduce language con- fusion without the additional rules, but the combination of rules and LCG further reduces language confusion rate. This shows the necessity both LCG and intervention rules.
DISCUSSION AND CONCLUSION
The Language Confusion Gate (LCG) is a lightweight, plug-in intervention that effectively miti- gates language confusion without altering the base model's parameters. Its primary advantage its practicality: as a small MLP with sparse intervention rate, it adds minimal computational overhead and avoids the performance trade-offs common in methods that require retraining.
LCG is very efficient. In our production system, we benchmarked the performance of Qwen3-30B: A3B-Irstt 2507 with and without I.G.. We set the input length to 2000 tokens with a concurrency of 8 samples, and let the model outputs 100 tokens. We find the time taken for each generation step 15.95ms without the gate, and 15.99 with the gate, with a minimum increase of 0.4%. Further, LCG is compatible with speculative decoding discussed in Appendix
However, the current approach is limited by its script-level granularity. By grouping tokens into broad families like "Latin" or "Low-Res", the gate cannot resolve more nuanced confusion be- tween languages that share the same script (e.g., Spanish vs. English) or between two different low-resource languages. Future work could explore more fine-grained and language-specific gates.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our results, we have uploaded all code necessary for training and evaluating the models described in this paper the supplementary materials. The provided code- base includes detailed instructions for data preprocessing, model training, hyperparameter settings, and evaluation procedures. All experiments can be replicated using the included scripts and config- urations.
LLM USAGE
The authors acknowledge the of large language model (LLM) technology to assist in the prepa- ration of this manuscript. Specifically, an LLM was employed to aid in refining language, improving clarity, and polishing the prose of certain sections.
REFERENCES
A. Seza Doğruöz. Sunayana Sitaram, Barbara Bullock, and Almeida Jacqueline Toribio. A survey of code-switching: Linguistic and social perspectives for language technologies. In Chengqing Zong. Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pp. 1654-1666, Online, August 2021. Association for Computational Linguistics. doi: 10. 1t.118663/1/// 10253//11222/// 1. acl-long.131 131. URL https:
//aclartholooy.ozz//22222 acl- .
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 2025.
Tahmid Hasan, Abhik Bhattacharjee, Md Saiful Islam, Kazi Samin, Yuan-Fang Li, Yong-Bin Kang. M. Sohel Rahman, and Rifat Shahriyar. XI-sum: Lcrge multilingual abstractive summariza- tion for 44 languages, 2021. URL https: http::///axxvvvvo org/ 1b822216.1382
SeungWon Ji, Jungyup Lee, Jemin Kim, Sang Park, and Seunglae Lee. Smoothie-qwen: Post-hoc smoothing reduce language bias in multilingual lims, 2025. URL httpss:/arxiv..or abs 2os%/207..m 05686
Nahyun Lee, Yeongseo Woo, Hyunwoo Ko, and Guijin Son. Controlling language confusion in multilingual Ilms, 2025. URL (tRLfhttts://arxiit /5t.aaawwww..r/aaa//555..... 25s5
org/
Yihao Li, Jiayi Xin, Miranda Muqing Miao, Qi Long, and Lyle Ungar. The impact of language mixing on bilingual IIm reasoning, 2025. URL UR..https../aaxwv..rr......
15849
Lightblue URL
KK.
ressonnn--multiiignaa-----------
2024.
https::/bf-mirre wttp://atttttttttteteaaaaaaaa reasoning-ae mu htt.://wmi.iiiiiaa-aaa-aaa------
Accessed: 2025-09-21.
Kelly Marchisio, Wei-Yin Yin Ko, Alexandre Berard, Théo Dehaze, and Sebastian Ruder. Under-
standing and mitigating language confusion in LLMs. In Yaser Al-Onaizan, Mohit Bansal,
and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Nat-
ural Language Processing, pp. 6653-6677, Miami, Florida, USA, November 2024. Associa-
tion for Computational Linguistics. doi: 10. 10..1665//1/2024.cmm
www.pmmmin..3... URL https:
7/ //aclanthology wrg. org. 2024. emnlp sannp-mmain main... 300./.....0/11.1
Ercong Nie. Helmut Schmid, and Hinrich Schütze. Mechanistic understanding and mitigation of
language confusion in english-centric large language models, 2025. URL https: : /arxivv.
hrr//bbs/2505: 16338
11

---

Published as a conference paper at ICLR 2026
NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shan- non Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Kochn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. Scaling neural machine translation to 200 languages. Nature, 630(8018):844 -46, 2024. ISSN 1476-4687 doi: 10.1038/ s4 $41586-024-07333-- URLpttts::/ wrg. 1tt.:////00.000001011100000000-0 02005588-244--7-8
OpenAL gpt-oss- 120b & gpt-oss-20b model card, 2025. URL https://arxiv.org// 2508. 10925
Qiwei Peng, Yekun Chai, and Xuhong Li. Humaneval-xl: A multilingual code generation bench- mark for cross-lingual natural language generalization. arXiv preprini urXiv:2402.16660 2024.
Angelika Romanou, Negar Foroutan, Anna Sotnikova, Zeming Chen, Sree Harsha Nelaturu, Shiv- alika Singh, Rishabh Maheshwary, Micol Altomare, Mohamed A Haggag, Alfonso Amayuelas, et al. Include: Evaluating multilingual language understanding with regional knowledge. arXiv preprint arXiv:2411.19799 2024.
Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F Karlsson, Abinaya Mahendiran, Wei-Yr Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, et al. Aya dataset: An open-access collection for multilingual instruction tuning. arXiv preprint arXiv:2402.06119, 2024.
Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. https: / thub. com/ tatsu- atabb//tanford_appaaae
2023.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shan- non Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. No language left be- hind: Scaling human-centered machine translation, 2022. URL httht:tt//aaatt org. /bs/ 2207 04672
Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bha- gia, Yuling Gu, Shengyi Huang. Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Valentina Py- atkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. OLMo 2 Furious, 2024. |t /ttpss///aaxvvv/ org abs aataabs/2500 2501/2511111 00656
Yiming Wang, Pei Zhang, Jialong Tang, Haoran Wei, Baosong Yang, Rui Wang, Chenshu Sun, Feitong Sun, Jiran Zhang, Junxuan Wu, et al. Polymath: Evaluating mathematical reasoning in multilingual contexts. arXiv preprint arXiv:2504. 18428, 2025.
Genta Winata, Alham Fikri Aji, Zheng Xin Yong, and Thamar Solorio. The decades progress on code-switching research in NLP: A systematic survey on trends and challenges. In Anna Rogers, Jordan Boyd-Graber, and Nacaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 2936-2978, Toronto, Canada, July 2023. Association for Computa- tional Linguistics. doi: 10.18653/v1/2022 ftt.:/3..iniin--ac. acl. URL https: httpp::::/caaaaaaaaataaaa /aa.aaannnoooyy..... org/2023. f f ias 1857
12

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXIv:2505.009388 2025.
A DETAILS OF THE TOKEN CLASSIFICATION ALGORITHMS
A known challenge with Byte-Pair Encoding (BPE) is that some tokens may represent incomplete Unicode characters, making them difficult to classify directly. When a BPE token cannot be decoded into a valid Unicode string, we employ more sophisticated method to infer its language family by analyzing its underlying byte structure. This approach allows us to conservatively classify even partial tokens.
Our methodology relies on the continuous block structure of the Unicode standard. Even if token only contains the first few bytes of multi-byte character, those bytes can significantly narrow down the range possible code points could belong to. The process is as follows:
1. Byte Sequence Decomposition. First, the ambiguous token is converted back into its raw byte sequence. This sequence is then fed into a UTF-8 state machine that breaks down into logically complete or partial UTF-8 units. The key is identify what we term a "left partial" token: a byte sequence that correctly starts a multi-byte character but is incomplete.
2. Inferring Code Point Bounds. With "left partial" sequence identified, its first two bytes are used to determine the possible range of Unicode code points could represent if com- pleted. Based on the rules of UTF-8 encoding, code points that can be formed. For 3-byte sequence, the final code point is formed from bit patterns across all three bytes. Since the third byte is missing from our partial token, we can calculate the full range of possibilities by assuming the third byte is at its minimum valid value 80x80) and then its maximum valid value (CxBF).
calculate the lowest and highest possible
3. Overlapping with Language Blocks. Finally, we check if this inferred Unicode range overlaps with any predefined language blocks. The calculated bounds are compared against map of known Unicode ranges, which contains the official start and end code points for scripts like CJK Ideographs, Hiragana, and Katakana.
If token is identified CJK blocks, it is classified as CJ token. This robust, byte-level analysis allows us to handle the ambiguity of partial BPE tokens and improve the accuracy of our language family classification.
"left partial" and its inferred Unicode range overlaps with any of the
B MORE EVALUATION ON IMPACT GENERAL CAPABILITIES
Beyond the henchmarks we evaluated in Section 5 we also evaluated and compared the performance of Owen3-30B-AS3e 33BB-Instruct-2 with and without the Language Confusion Gate in Table B We can see that the incorporation of LCG doesn't affect general Capabilities of LLM.
Benchmark
Qwen3-30B No LCG
Qwen3-30B LCG
MMLU-ProX-13
71.95
71.74
MMLU
85.04
85.03
MMLU-pro
78.44
78.25
GPQA
70.35
70.45
SuperGPQA
53.38
53.24
AIME24
74.32
74.90
AIME25
61.25
61.88
Table 6: Performance comparison between Wreen---0BB-A3--nnstmnttet- without and with LCG.
13

---

Published as a conference paper at ICLR 2026
C COMPUTE RESOURCES REQUIRED FOR EXPERIMENTS
All of the training and evaluation experiments we talked about could be run two NVIDIA-A100 GPUs, for which smaller models like Qwen3-8B, Llama3. and Gemma3-12B could be trained on single NVIDIA-AI00 GPU. It takes about 12 hours trained the largest model Qwen3-30B-A3B we experimented with on A100 GPUs.
D ICL PROMPT
As discussed in Section 5.3 we tried prompt to teach LLM what is language confusion, and provide an example to it avoid. We show the ICL prompt we use Figure
Language confusion in the context of large language models (LLMs) refers to the phenomenon where a model mixes confuses elements multiple languages inappropriately during generation. attention to prevent it. For example. "a ! = a yard 0Il
E) 21 4
20 & incorrect. should "2 U =
a aeas = 0 5 9 & 7 g +
TJ Pay attention to avoid language confusion during generation.
Figure 4: Prompt we used for ICL as described in Section 5.3
E COMMERCIAL LLMS CONFUSION RATE
We show the full list of commercial LLMs we tested with FLORES-NO-LATIN in Table 7
Table 7: Full list of language confusion rate commercial chat LLMs
Model
CJ (%)
Latin (%)
BLEU
GPT-5-Chat
0.57
0.67
10.66
GPT-5-mini
0.04
1.06
18.07
GPT-5-nano
0.04
1.24
16.79
Qwen3 Swen3-235B-Be 235B-Innsrucc
2.27
5.07
15,43
Qwen3-30B- -0---30--Inttrct
0.92
4.68
13.42
4
0.00
0.35
21.77
Gemini-2.5 -25--Pro
0.04
0.50
19.11
Gemini- Dommini-2.5-Flas 2.5-Flash
0.07
0.82
19.93
DeepSeck-v3.
0.67
1.06
18.11
DeepSeek-v3-0333
0.14
0.57
20.72
GLM-4.55
0.14
0.99
13.35
GLM-445-air
0.14
1.31
15.18
Grok-4
0.18
1.21
20.75
Doubao-1.6
0.99
1.74
14.09
F COMPATIBILITY WITH SPECULATIVE DECODING
The Language Confusion Gate (LCG) is designed to be lightweight module that integrates lessly with modern inference optimizations, including speculative decoding. This appendix details why LCG is compatible with this technique and does not compromise its efficiency gains.
14

---

Published as a conference paper at ICLR 2026
Speculative Decoding Overview. Speculative decoding accelerates inference by using smaller, faster "draft" model to generate a candidate sequence of several tokens. The larger, more powerful "target" model then processes this entire sequence in a single, parallel forward pass to verify the draft tokens. This allows multiple tokens to be accepted for the cost single forward pass of the large model, significantly improving throughput.
Integration of LCG. LCG's compatibility stems from its ability to be integrated directly into the target model's verification step with minimal overhead. The process works follows:
1. Draft Generation: The small draft model generates candidate sequence of tokens as it normally would.
2. Target Verification: put combined with the draft sequence, producing hidden states and logits for each token position in parallel.
The large target model performs its single forward pass on the in-
3. LCG Masking: At each position, the LCG takes the corresponding hidden state generated by the target model and computes a language mask. Because the LCG is a small MLP, this step is extremely fast and can be performed in parallel for all draft tokens.
4. Logit Modification:
The LCG's mask is applied to the target model's logits for each token. Since intervention rare, the logits are often left unchanged, ensuring minimal impact on the model's 's standard behavior.
5. Final Validation: The standard speculative decoding algorithm proceeds with its validation check, but it uses the ICG-modified logits accept or reject the draft tokens.
Because the ICG is fast, parallelizable operation that adds negligible computational cost to the target model's pass, it does not create bottleneck or negate the speed improvements offered by speculative decoding. This ensures that language confusion can be mitigated without sacrificing inference performance.
G INVESTIGATION OF HOW TRAINING DATA AFFECTS OUTPUT TOKEN NORM
As discussed in Section 3.2, the imbalance output token embedding norms contributes to language confusion. We studied how training data affects token norm in Olmo (Team OLMo el al. 2024). Olmo released their training data and checkpoint in different stages for research purpose. Here we compared the output token norm of Olmo-3-1025. and -0mo-3-7B-InsstuuctSSii
(Pretrain), which is the pretraining checkpoint, (SFT), which is the SFT checkpoint. According to Olmo 3's tech report, the training data is filtered in the pretraining stage to retrain contain English texts. While in the SFT stage, they incorporate the aya dataset (Singh et al.) 2024) for multilingual capabilities. Since the percentage of low resource tokens in SFT stage is much higher than that of Pretrain stage (which is close to 0), we expect the low res token norm to increase.
In Table 8 compare the percentage of tokens whose embedding norms lie in the top 5% of all token norms the model's vocabulary of Olmo 3 Pretrain and Olmo 3 SFT, similar to Table We can see that after the SFT stage, the percentage of Low-Res tokens with top 5% norm increase from 3.79% to 7.89%. Notice that since the pretraining stage of Olmo 3 only contains English, Chinese is also relatively "low resource" in this case. As result, observe that the percentage of CJ tokens with top 5% norm increases from 0.87% to 2.14% after the SFT stage.
Table 8: Percentage of tokens in each language family with embedding norms among top 5% of all token embedding norms Olmo 7B pretrain and SFT stage.
Model
CJ%
Latin%
Res%
Olmo 3 7B Pretrain
0.87
5.25
3.79
Olmo 3 7B SFT
2.14
5.22
7.89
Notice that the percentage of low-res tokens in in top 5% of all tokens norm in Olmo is higher than the 5 models
measured in Table This is because Olmo only tokenize very high frequency
15

---

Published as a conference paper at ICLR 2026
Swnel
O' ciaaetatie meme pridued (8680) 10n
Tous theigho lo faiic targ Dyneery poot (710, 19h-1g
meeeied Agjial
30 1e Uiieeee
rided futAl Aubis Mack
mosiige geenec
SAAA Albue High
bnltial in And poltur of
Imspery daar volees accollrss boodee shring of
Figure 5: Left: User asking in Arabic to write a python program that computes factorial, Right: User asking in English to let LLM introduce a Chinese poem.
grams in low resource languages, which are mostly single characters. As a comparison, there are 19,257 Low-Res tokens in Qwen's tokenizer among of vocab size of 131,936, while only 1268 Low- Res tokens in Olmo 3's tokenizer among vocab size of 100,278. Since Olmo 3 only tokenize high frequency Low-Res tokens, the average frequency of all Low-Res tokens higher, which explains the relatively higher norm.
H EXAMPLES OF NATURAL CODE-SWITCH
We show examples of user queries that require LLM to naturally blend two languages in Figure
EXAMPLES OF INTERVENTION V.S. NO INTERVENTION
We show examples of Qwen3-8B responses to query with and without LCG intervention in Fig- u The example the left shows example that incorrect language mixing avoided, and the example on the right shows an example that correct code-switch is maintained.
From Figure 7 to Figure 10 We show qualitative examples showing that models with LCG maintains natural code-switch capabilities under context that re- quries multilingual communication. For each of the Figures, the left show the response of 30--A30----Sntttt
without intervention (NoGate), on the right we show the response of the same model with Intervention (Gate). We can that model with LCG preserves capability to switch between languages.
J FALIURE CASES OF LCG
It's possible for LCG to fail in preventing language confusion. This happens when the LCG thinks token in a different language acceptable here, while it turns out to be incorrect. We show two examples of this case in Figure [I]
However, it's impossible for LCG to increase rate language confusion, because of the intervention rule of "persistence of the previous token's language" in Section 4.3 Language confusion
16

---

Published as a conference paper at ICLR 2026
QUERY:
QUERY:
Translate the following English text into Modern Standard Arabic. Only provide the translation. English: Layton had asked for changes to the conservatives' environmental bill during the meeting with the PM, asking for "thorough and complete rewriting" of the Conservative party's environmental bill. Modern Standard
Translate the following English text into Korean, Only provide the translation. English: Also, make sure to pronounce and rr differently: caro means dear, whereas carro means chariot. Korean:
Arabic:
Qwen3-8B No LCG:
Qwen3-8B No LCG: E rst m= K 2 ~ # caro W E carro xi 20
+ البنةة قكرن عل تساات بهر نفرن طلب " سااا الززااد رفن + الأجتاع غال الماتتينن شااة كنبة sلelpiete تنري البينة تفقترن المفااينن
Qwen3-8B LCG-adjusted:
Qwen3-8B LCG-adjusted: _ r2) r= QE>I E ONOP such
التحنب البنةة فاررن علي تسيلتت اجباء افخون طلب بأادةة مطليا الززرادء ينفس مع التتاا ماال المحلف الحلفه اتحزب البينة لققنرن وناناة شناة ككبب
caro A All 0 carro 전 L/CC.
Figure 6: Comparison between language confusion and natural code-switch.
Create Arabic restuurant menu with Arabic and Engliih dish names
Seeen3-30B-Noaate
Qwen3-30B-Gate
jen't pla lugfl Jاول 3M pils
Au Saouh plu
Aumbl Sel
J A Jalk wie s Say ai ge the who
الyl Lilr wilh with light Sos Ragal
M.y (Cold Kibbeb)
Chicken Shawarma glaci logil
gla (Chickon Mansof)
2u Jult
d Ju, Jr perell upll gldde
Monsal
ui ge μ stsl glwo t
youdd
paly (aam Fattoush)
Tabbouleh 4,
Lale jidl
JA Cumllyy yomc Loudl الااي فلتي dlas
JuP geull prioeey Gard
Nabale (Hammus with Tukinl)
Moutabal -20g
job zie (Pickded Vegeiables)
Pattoush J.
الرر Judyy unنl Man Jibn
Sa Jusnd
[Ablawd)
[Ablated]
Figure 7: Qwen3-30B without and with LCG under multilingual context of Arabic and English.
17

---

of Published as a conference paper at ICLR 2026
Explain Greek expression in Engilsh
Swven3-30B-NoGG
Qwen3-3388 Bwen3-30B-Gatt
Herp's Greek
English:
Sunel Hamw's commen Graek eoploned English:
Greek Expression: Fiza (Pasii miins))
Greek expression: MAr mapoc ora dodapa.
Literal Translation: "The city goes"
Litrral translition: "Dirrl lain serioualy
Meaning Englishh: " over. "I's done"" or "t's Eriahed."
English equivatent: 'Do''t take is heart" "Don't get ucset about
This expression is used ind cate that shuation has ended, often with seroee resignation rekerl, fi For ecample. semoone frustroedd ofter long day probiems,, they right "Wiell,
Explanation:
dealing
This ecpeossion otten used when someone saya sorwihing (ak nilht sarcastically or lightly, and they want to meke sure the other person docsn misundenstand feel offended. 1'm kitsinng -ontinng-oonn-n overthine
English
conversation
eligttty
demmalic humroaaae lore, depardiing on contact.
For example. a friend tacses you about sp your coffee, you repy senousty y-
cottosd, DDn'
cullural londenny igte-heer everyday
Figure 8: Qwen3-30B without and with LCG under multilingual context of Greek and English
American student
Nodete
Gwer3-388-00e
Of coumel
neouttll meoninghi Komen
Atookuaey Lar's ectice beourhh and acoeoi Korean poirs
logetve- "Spring
rosil twed madem posts. Kim Sowel (Ald
supctio-- "AlA"" (Bee-a0 by Kins So wot BB) This pourt onar nost fonous and boknnod Korcon atuuuro and beout canture themee naturn, time, and sulet reflccion untvrna emetions zut
perfed denanaan skudent becauee blnds simale, vird imagery with deep notion much like the sorts of merton poets such as Rizbett Phost Mary Ofier.
any akabent inclucing stinng Amarcca
"Spring Sower
Km So-wcl (2912- 1954)
(Seove)) "Dwering Song"
1/b: OICLE 21
IMM 52.8t
20 3ilee 21 9/
2:1 d B3- 76:135
B6.. 010 24 06
J10 (801 2 3000
231 01 3im
21 1 11212 112
S/
Lite UiLEs J 187
300 Jitz 12 9922
-B0 4 1121 21213-9
(1:55:0 14
5:22
-T2 1:- 39-13/9
01:5000 #M
JIN 01
DE
225..1 .252.
718K
2012 1/7
English
(Poetic, liberat:
spring
Enggiihh Translation (Peetic Netural::
The oven rg ow si spon ricurtann shadow partly ootiee
Astumn thouugh grans,
you?
The
Quietly descends.
Would meet you?? my hear
The
wind
The earth plain, the hoam
Would go most
When leswe ihts woith.
Kaa rill lioe leaf upon grouns--
Quilts quility Key my heaft sottl carh.
[Abrated]
[Ablated]
Figure 9: Qwen3-30B without and with LCG under multilingual context of Korean and English
18

---

Published as a conference paper at ICLR 2026
Wrtte dialogue between Russian teacher and Amer ican studdett
Qwen3-306-NoGate
Qwen3-30B-Gaid
Corversation Across
Charecters:
Setting asaarom university in Wasdengtan, D.G. R Jodan tencher. krna Perrova revinwing a leacc with her Arrrizan enadent Jass Thompson, arter rówa. The moom rilee with tooks, masp, and anall Phooker mesang cull dealc
language
Cartar an osiagg akadent earl 20x.
ontiraicaoo strugiigg with pronuncation.
infiloed arotunciation
[Betting: cleaaroore, whiteboard noeaanorrr" (Welcomel) servously flipping through
something
Cyrilli,
inde
(augting) aorr. Phrrows, two bean aatiiingg
Ante: jamilngg Good momnnn Jumeel Now are you tocay?
saying live Maybo
momnnng
Inteac 1ch (shuoding) Pertupel four no, the louue in not your aftort it the aitoa, in Hooese:: "rooeer" pronouneac with acil, alrend w waappryy souadd-iiiee live "rneeggs aistt "prvstt" Try wich Pree- syyt
Anne: (shuchles) Don't be nerooss We're hore to tom, not to be pwttect Rerwebbw won moin minakee hen spank mg Enghen
Jamma:
(beaghal Wadt
roumenate
Jake: Dryvng) Proessyet. Like winnper?
at sposk Englnn ho lime.
Intox: Eoactyy Lik ancret, (She smilles You Americnn--et you strong, chear acurciss
(giiiiing) your
wony--yo
say good
spooking to
soom
Jake: That's aoluaily lind besutll. L nover thought of that way. in ting lin, we day thai loud and fact. But inpweee" feait...
James: Groat know "rpwoor" and "asoecroyyin."" tve beon pract cing
pensonal
langsaage
Exceblent "Tpueer" irformal 'tll friend Supracrryyive karrad-lbe "tallo teacher Nuw, "How you?" Russien?
like foo I'n twan meding Reacoe-ene, cen sederstarrl-err theniing toon Tobilyy doaply poilry.
James:: (firking king) Uh... acro??
[Aobbad]
Figure 10: Qwen3-30B without and with LCG under multilingual context of Russian and English.
Query
Translate the following English text into Korean. Only provide the Iranslation.
English: Afler seeing horrors and atrocilies war during World War I, nations desired to avoid such situation again the future.
Qwen3-8B
22 Di
CLAI
2::8: 0
Ct.
Query
Transiale the following English text into Korean. Only provide
translation.
English: There's tradition pass the Easter night awake at some exposed point
Korean:
Qwen3-8B
00 2101
3 01
RICI.
Figure 11: Examples of L.C fail to prevent language confusion on Qwen3-8B.
only happens when the language of token is different from the last token, while the rule ensures that the probability of tokens with same language as last token would never decrease.
19


Reviews:
[{"review_id": "BqhRSy1HBb", "reviewer": "Reviewer_7Ns2", "summary": "Authors propose language confusion gate (LCG), which uses a 2-layer MLP to predict the language class of next generated tokens and mask the unwanted language tokens. Results show that language confusion behaviors are reduced by 10x by applying the proposed method.", "questions": "1. Could authors measure if there is any effect on inference speed?\n2. Is there potentially a cleaner way to obtain the expected class of the next token? For example rule-based/heuristics-based on logits/last hidden states. Would like to see such a simple baseline to be experimented with to justify a two-layer MLP is needed.\n3. Does the incorporation of LCG affect the length of reasoning traces?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. Authors propose an effective fix on the problem of language confusion based on reliable observations.\n2. Method is simple and intuitive.\n3. That the method does not require modifying the base model alleviates the concern of the potential forgetting happening.", "weaknesses": "1. In section 5.4, there is slight performance degradation on Qwen series after adjusting for LCG (no degradation on GPT-OSS though). Further error analysis on this matter is needed. For example, some qualitative analysis on some reasoning traces: does LCG suppress the diversity/exploration in language models?\n2. In light of the above, more experiments could be done evaluating general capabilities after applying LCG.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H0caMrW28y", "reviewer": "Reviewer_KkFU", "summary": "This paper addresses the language confusion issue in LLMs, where models inappropriately mix languages during generation. It proposes the LCG (Language Confusion Gate), a lightweight plug-in module that filters tokens during decoding without modifying the base LLM. Trained via norm-adjusted self-distillation, LCG predicts possible language families and applies masking when necessary. Experiments across various models show that LCG reduces language confusion while maintaining task performance.", "questions": "1. Could the token classification method be optimized to handle more fine-grained language distinctions rather than just broad families?\n\n2. What is the specific computational overhead of LCG in large-scale inference?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The proposed LCG is a practical and lightweight solution, avoiding the need for model retraining and adding minimal computational overhead.\n\n2. The norm-adjusted self-distillation method effectively mitigates the bias towards high-resource languages caused by token embedding norm imbalance.\n\n3. The work distinguishes between harmful language confusion and legitimate code-switching, ensuring the model retains necessary multilingual expression capabilities.", "weaknesses": "1. LCG only classifies tokens into broad language families (CJ, Latin, Symbols, Low-Res), failing to resolve confusion between languages sharing the same script (e.g., Chinese vs. Japanese).\n\n2. The evaluation’s reliance on rule-based detectors for language confusion may have limitations, especially in complex multilingual contexts.\n\n3. The paper does not discuss the generalization of LCG to more low-resource languages beyond the tested ones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hAf8NeSwmP", "reviewer": "Reviewer_53Zx", "summary": "The paper proposes the Language Confusion Gate (LCG), injecting a lightweight MLP layer into LLM to reduce unintended mixing of scripts/languages during LLM decoding. This is based on the observations (i) language confusion is rare and the correct-script token is usually ranked in the top-k, (ii) embedding-norm imbalance favors high-resource scripts. The MLP layer is trained to predict which language families should be permitted at each step and mask others in the logits LCG reduces both Chinese/Japanese and Latin confusion by up to an order of magnitude on translation and QA settings (FLORES+, INCLUDE) and on “thinking” models for code generation (HumanEval-XL), with negligible or no degradation in BLEU/accuracy/Pass@k. This method works for both thinking and non-thinking models.", "questions": "1. How do you perform the threshold calibration for the sigmoid function in the MLP? What is the criteria to determine that?\n2. Do you try any multilingual tasks, where the model need to respond in at least two different languages? Does the proposed method help in that scenarios?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper introduces a norm-adjusted self-distillation training signal for a language-family gate. I consider it as a very simple idea distinct from weight edits or reward finetuning.\n\n2. The paper demonstrates empirical gains of the proposed method across open and closed models.", "weaknesses": "1. The paper calculates “code-switch rate” on FLORES-WITH-LATIN but does not include human judgments or llm-as-a-judge on the responses. It could offer more evaluation on whether the improved responses are also preferred by humans. \n\n2. Are there cases that the proposed method do not help to correct the language confusion and could even increase the problem? It would be helpful to showcase some negative cases as well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v8jqS85RaM", "reviewer": "Reviewer_EcN8", "summary": "This paper introduces the Language Confusion Gate, which is a mechanism to prevent language confusion. Language confusion is the scenario in which a model mistakenly uses tokens from an inappropriate language - such as including Latin text in a language that does not typically have code switching (e.g. Chinese or Japanese). This paper distinguishes between \"harmful confusion and acceptable code-switching.\" The authors carefully construct test scenarios by analyzing token distributions, properties of unicode, and task relevant exceptions such as english technical terms in otherwise non-english text. The authors construct a classifier trained on final hidden states, accounting for interesting observations about HRL token norms, and use this classifier to gate allowed tokens at model decoding. This method is effective at reducing harmful confusion, permitting intentional language mixing, and maintains task performance. I found the analysis of why language confusion happens (top logit distributions, insufficiency of greedy decoding, high norm bias) particularly interesting and insightful.", "questions": "- Could you provide statistics indicating how many tokens are disallowed at certain timestamps? \n\nLine 206 contains a typo", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "This paper identifies a rare, but important, failure mode for LLMs. They provide interesting analysis to motivate studying and solving this problem, then introduce several specific technical innovations to address it. \n\nS1. This paper is extremely well motivated. I was initial skeptical of the prevalence of the problem, but the paper expands on prior work by Marchisio et al. showing that the problem setting is realistic. Sections 2 and 3 are particularly effective in exploring why harmful language confusion happens (e.g. showing that the language confusion token is the top-1 choice 56% time). \n\nS2. Paper introduces high resource language (HRL) norm bias. The authors show that output tokens with very high norm are naturally higher in the distribution (230-241), and that HRL tokens are over-represented in the top 5% of token output norms. In 264-265 the authors are very precise about the claims, stating they do not claim that this phenomena accounts for all confusion and thus cannot be used as a direct signal. Rather they use this to normalize measurements when training the classifiers. \n\nS3. The classifier and gating mechanism are very carefully designed - section 4.3 lists deterministic rules of when the classifier can override the expected token prediction. These ensure that the masking does not prevent fluent outputs. \n\nS4. The evaluations are broad and relevant, including both translation focused tasks derived from FLORES and tasks suitable for reasoning models. The evaluations also carefully distinguish between harmful and intentional code mixing (e.g. WITH-LATIN scenario).\n\nS5. Appendix A details the analysis of partial unicode characters present in model vocabulary - these are details often missed in other papers that study tokenization. The examples in the appendix and discussion of connections between token gating and speculative decoding are also interesting. \n\nOverall this is a very strong and well-executed paper.", "weaknesses": "None of these weaknesses should be taken as a reason to avoid publication.\n\n- It would be interesting to see how experimental results change if the additional deterministic intervention rules were not used to further control the gating mechanism \n\n- If there exist a lightweight way to do experiments going beyond the CJ/Latin split, it would be quite interesting to see those results. Perhaps leveraging further unicode metadata? However this is not necessary, it would simply further strengthen the paper. \n\n- Further investigation of the high-norm tokens would also be interesting. Could these be tied to token frequencies in pretraining data, for models with known data (e.g. olmo models?). Also, the causality direction between high norms <> HRL tokens could be further investigated.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)