The Emergence of Abstract Thought in Large Language Models Beyond Any Language
Authors: Yuxin Chen
Yiran Zhao
Yang Zhang
An Zhang
Kenji Kawaguchi
Shafiq Joty
Junnan Li
Tat-Seng Chua
Michael Qizhe Shieh
Wenxuan Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Models
Language-Agnostic Neuron
Abstract Thought
Primary area: deep_learning

Abstract: As large language models (LLMs) continue to advance, their capacity to function effectively across a diverse range of languages has shown marked improvement. Preliminary studies observe that the hidden activations of LLMs often resemble English, even when responding to non-English prompts. This has led to the widespread assumption that LLMs may ``think'' in English. However, more recent results showing strong multilingual performance, even surpassing English performance on specific tasks in other languages, challenge this view. In this work, we find that LLMs progressively develop a core language-agnostic parameter space—a remarkably small subset of parameters whose deactivation results in significant performance degradation across all languages. This compact yet critical set of parameters underlies the model’s ability to generalize beyond individual languages, supporting the emergence of abstract thought that is not tied to any specific linguistic system. Specifically, we identify language-related neurons—those are consistently activated during the processing of particular languages, and categorize them as either shared (active across multiple languages) or exclusive (specific to one). As LLMs undergo continued development over time, we observe a marked increase in both the proportion and functional importance of shared neurons, while exclusive neurons progressively diminish in influence. These shared neurons constitute the backbone of the core language-agnostic parameter space, supporting the emergence of abstract thought. Motivated by these insights, we propose neuron-specific training strategies tailored to LLMs' language-agnostic levels at different development stages. Experiments across diverse LLM families support our approach. Our codes are available at https://anonymous.4open.science/status/S-C393.

The Emergence of Abstract Thought in Large Language Models Beyond Any Language
Yuxin Chen
Yiran Zhao
Yang Zhang?
An Zhang'
Kenji Kawaguchi' Wenxuan Zhang*t
Shafiq Joty²
Junnan Li
Tat-Seng Chua'
Michael Qizhe Shieh
National University of Singapore
Salesforce Al Research
3 Peeking University
Singapore University of Technology and Design
Abstract
As large language models (LLMs) continue to advance, their capacity to function effectively across diverse range of languages has shown marked improvement. Preliminary studies observe that the hidden activations LLMSs often resem- ble English, even when responding non-English prompts. This has led the widespread assumption that LLMs may "think" in English. However, more re- cent results showing strong multilingual performance, even surpassing English performance on specific tasks in other languages, challenge this view. In this work, we find that LLMs progressively develop core language-agnostic param- eter space- a remarkably small subset of parameters whose deactivation results in significant performance degradation across all languages. This compact yet critical set of parameters underlies the model's ability to generalize beyond indi- vidual languages, supporting the emergence of abstract thought that not tied to any specific linguistic system. Specifically, we identify language-related neu- rons-those are consistently activated during the processing of particular languages, and categorize them either shared (active across multiple languages) or exclusive (specific to one). As LLMs undergo continued development over time, we observe marked increase in both the proportion and functional importance of shared neurons, while exclusive neurons progressively diminish in influence. These shared neurons constitute the backbone of the language-agnen agnostic parameter space, supporting the emergence of abstract thought. Motivated by these insights, we propose neuron-specific training strategies tailored to LLMs' language-agnostin levels at different development stages, Experiments across diverse L.LM families support approach
Introduction
As large language models (LLMs) continue to advance (OpenAl 2023 Touvron et al. 2023a Hurst
al. 22244] Yang 2024a Team al. 2024), their performance across wide range of languages
(known as multilingual capability) has markedly improved over the past years (Le Scao et al. 2023
Yang al. 2024b Üstlin et al. 2024). Despite this progress, several studies have observed that L.MSS
tend to "think in English", often using it as an internal language of thought even when processing
inputs in other languages (Wendler et al.) 2024 ZZhao al.] 2024b) [Schut et al.) (022a)). This
phenomenon has led to the hypothesis that LLM performance in non non-Enennnn English languages is inherently
constrained by their capabilities English (Qin et 2023
Liu al. 2024).
"Equal Contribution. Corresponding Authors.
Our codes are available at http:p///ghhhh..co///////a//////
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Yet, more recent findings complicate this narrative: some studies found that LLMs can actually outperform their
performance on certain tasks in other languages (Zhao al. 2025c Gemma Team et al. 2024b 2025), indicating that non-English processing may not always rely on English as an intermediate language. These conflicting observations raise a deeper research question: Do LLMSs think in the distinct space of each language, or Do they operate in higher-level language-agnostic space beyond any specific language? In other words, whether the trend of non- English performance compared to English indicates the emergence of abstract thought within LLMs?
In this work, we explore the existence and develop- ment of abstract thought LLMs by analyzing how individual neurons, responsible for models' thinking. respond to multilingual queries. Here each neuron corresponds to a row or column in the model's 's pa- rameter matrices and is considered activated if its re- moval significantly alters the model's 's output (Fran- kle and Carbin 2018 Tang et al., 2024 Wang et al. 2025a). We begin by identifying neurons activated when the model processes inputs in specific languages, referred to Language- Saeauagg--Rlatedd Neurons. To inves- tigate whether these neurons become increasingly specialized for specific languages potentially ex- hibit more general and language-agnostic functional- ity, we distinguish between Language Sinnuagg-Excllsive Neu- rons, which are activated only for one language, and Language-Shared Neurons, which are consistently activated across all languages considered. Figure [ (top) shows positive correlation between multilin- gual ability and the proportion of language-share neurons across different generations of LLMs. This suggests that as multilingual performance improves, greater proportion of language-related neurons are shared across languages.
Proportion of Shared Neuron on
0.58
1.5-1.88
3-48
Owen2 5-0 50
7B
Overall Trandline
Qwin
Qwen2.5 28
Qeea2-
Qwiz
Geenl 1:43
Feb 2024
Jun
Sep 2024
Model Release Date
Importance of Shared Neuron
Qwee2.5- 1on2.5-1:30 1.50
10-
1.5-1, 8B
5
3-48
Quer2
8
Overall Trendline
uwen22
@Am11 5-40
Qwan2.5.7
oweR 3.50
Guegaer-55 5.se
Qwen2-78
@wen1.5:70
Feb 2024
Jun 2024 Model Release
Sep 2024
Building the observed positive relationship between language-shared neurons and multilingual capability, as well the finding that LLMs increasingly outper- form in non-English languages on certain tasks, we hypothesize that shared neurons may gradually as- sume more fundamental roles beyond merely support- ing multilingual processing, Accordingly, rather than focusing solely on how the proportion of language- shared neurons evolves across model generations, it is essential to evaluate their functional sig- nificance relative to fanguage-exclusive neurons. If shared neurons contribute more critically to multilingual processing than language-exclung neurons wwhch also participate language tasks and should be comparably important in trinciple-this would indicate that shared neurons have evolved into Language-Agnosie Neurons, which go beyond shared activation patterns to support abstract functions like semantic reasoning and generalization. As these neurons evolve, they support increasingly abstract thought that transcends linguistic boundaries. As shown in Figure (bottom), language-shan shared neurons exhibit markedly growing importance in multilingual processing relative to Lunguage-xxclusee neurons, signaling the emergence of language-agnostic properties and potentially, the development of abstract thought in LLMs.
Figure 1: (Top) The trendline of shared neu- ron proportion rises with model release date (see Section 2.2 (Bottom) The trendline of shared neuron importance also grows, in- dicating their increasing language-agnostic
property (see Section 2.3
Inspired by the insights discussed above, we propose a set of targeted neuron training strategies aimed at enhancing the multilingual capabilities of LLMs. These methods are tailored based on the presence or absence language-agnostic neurons, which serve as an indicator of the emergence of abstract thought within the model. For LLMs that lack language-agnostic neurons, the model is likely still under-trained; thus, training any language-related neurons can contribute to improving multilingual performance. In contrast, in LLMs where abstract thought has emerged, language-shan neurons have evolved into language-agnostic ones. As these neurons have reached form of generalization,
2

---

further improvement through additional training is limited. In such cases, enhancing multilingual capabilities requires focusing on training kanguage-exclusvy neurons better support language- specific nuances. We validate our approach through comprehensive experiments across diverse model series and release time. The results demonstrate that our training method, guided by the presence of language-agnostic properties, effectively enhances multilingual performance.
2 Metrics for Exploring A Thought
In this section, we identify neurons associated with language processing, referred to as Language- Related Neurons, and, based them, define several metrics to quantify and analyze the emergence of abstract thought LLMs.
2.1 Relaee Neurons
We identify language-related neurons as those that are consistently activated when processing inputs in particular language, where a neuron is defined as a single row or column within the model's parameter matrices. Building on prior work in identifying important neurons in neural networks (Fran
kle and Carbin 2018 8 Ni et 2023 Tang et al. 2024, Zhao al., 2024b), we consider a neuron to be activated if its removal leads to a significant change in the resulting embedding. Formally, given an input sequence x: in a specific language, a neuron V is considered activated if
[|CLM()) - LCMex(r)]l 20
(I)
where (.) denotes the output embedding when processing æ, and (LMMw((r) denotes the output when neuron N is deactivated, i.e., its parameters are set to zero. The threshold a specifies the minimum magnitude of change required to consider neuron activated.
Furthermore, language-related neurons Miang for a specific language are identified through
lang == {N' CCM - 2 a, Vx E
(2)
Since sequentially deactivating neurons in Equation] 2 is computationally expensive, employ the parallel neuron detection methods proposed Zhao et al. (2024b); Wang et al. (2025a). Further implementation details are provided in Appendix A Details of how the activation threshold a is selected and validated are provided in Appendix
2.2 Language-Shared and Language-Exchert Neurons
To investigate whether neurons become increasingly specialized for specific languages or exhibit sunguage-annoste behavior, we conduct preliminary analysis of the proportion of Language- Shared Neurons, defined as language-related neurons that are consistently activated across all languages considered, and Language-Exclusive Exclusive Neurons, defined as language-related neurons that are uniquely activated for individual languages and not shared across all languages. Formally, language-shared and language-exclusive neurons are defined as follows: Ne Vexclusive == N \ Waharedd
Nahared := -N W and
REC
where denotes the set of all languages under consideration. other word, Nasared consistently exhibit high importance across inputs from different languages, while A eexelusive is the set of neurons specific to that language but not part of the shared set. Furthermore, we examine the proportion of angguage-haaree neurons relative to language- ciclusvv neurons, defined as
[Wshared
Shared Neuron Ratio == [ Eeec W excluusive
which quantifies the extent to which individual neurons are shared across all languages opposed to being specialized for specific ones. A higher ratio indicates a greater number neurons that are commonly activated across languages, while a lower ratio suggests that more neurons are uniquely responsive individual languages. A more detailed analysis of the shared and exclusive neurons, including their layer-wise and component-level distributions, can be found in Appendix C

---

2.3 Language-Agnostic Neurons
As LLMs continue improve in their ability to handle multiple languages, and even outperform their English capabilitiiss on specific tasks, we hypothesize that shared neurons may gradually serve more fundamental functions beyond the processing of multiple languages. Consequently, rather than solely examining the evolution of the proportion of language-shared neurons across successive model generations, it also crucial to assess their relative functional significance in comparison to Lunguage-excluivve neurons. Specifically, if language-sharee neurons contribute significantly more to multilingual processing than language-exclusive neurons, this indicates functional difference between the two, since both types are involved in language-related tasks and identified using the same criteria, they should be equally important if their roles are analogous. The discrepancy suggests that shared neurons have evolved into Language-Agnosiee Neurons. Note that while language-shared neurons are activated across multiple languages, language-agnosie of abstraction. Rather than encoding sanguage-speciing features, they are hypothesized to support cognitive functions that transcend individual languages. such as semantic abstraction, reasoning, and generalization.
neurons reflect a higher level
To investigate whether language-agnostie property emerge in language-shared neurons, we intro- duce the metric Lunguage-haree Neuron Importance, which quantifies the impact of deactivating language-shared neurons versus language-exclusive neurons on the model's performance in a given language. This is operationalized by measuring the change in perplexity (APPL) when each neuron group is ablated. A disproportionately larger increase in perplexity upon deactivating shared neurons would suggest their greater functional importance. Formally, define the sinuuage-shares neuron importance for a language as:
APPL shhared/ / |N
Imp's
APPL, eexclusive/
|
exclusivel
where APPI. 'shared and APPL. exxlusive and corresponding exclusive neurons are deactivated, respectively, and |/ahared and W Mexclusivel represent the number of neurons in each group. A higher value of Imp' indicates that shared neurons contribute more significantly than language-exclusive neurons, thereby providing evidence for their language-agnostic role, since both types of neurons should exhibit comparable importance if their functions were equivalent.
"exclusive denote the changes in perplexity for language & when shared
To obtain an overall model-level estimation reflecting this trend across different languages, we compute the average importance across all languages and apply logarithmic transformation to mitigate scale sensitivity. We refer to the resulting quantity as the Language Agnostic Score:
Language Agnostic Score log
1+
Σ Imp' REC
C
which quantifies the average degree which language-shared neurons contribute in language- agnostic manner across the evaluated languages. In contrast to the shared neuron ratio defined in Equation 4 which solely quantifies the number of shared neurons, the language-agnostic score incorporates the functional importance of neurons. Higher values suggest not only stronger language- agnostic behavior but also hint at the emergence of abstract thought in LLMs.
3 Emergence of Abstract Thought
3.1 Experiment Setup
Evaluated Models To comprehensively evaluate the emergence of abstract thought in LLMs
throughout their development, we examine 20 open-source models encompassing diverse model families, release periods, and sizes. Specifically, we evaluate Llama series including LLaMA1-
7B (Touvron et al. 2023a), Llama2-7B (Touvron et al.) 2023b), Llama3.2-1B: Llama3.2-3B, Llama3-
8B, Llama3. 1 18B Grattafiori et al. 2024), Qwen .5.0..B, Owenl.5-1.88p Owen1.5-4PP Qwen1.5- 7B (Bai et al. 2023), Qwen2-0.5B, Qwen2-1.551 1.5BB, Qwen2-7B (Yang et al. (2024)). Qwen2.5-0.5B,
Qwen2.5-1.5h Qwen2.5-3B, Qwen2.5-7B (Yang al. 2024b), Gemma-71 (Gemma Team et al.
2024a), Gemma2-9p (Gemma Team al. 2024b), Gemma3-4B (Kamath et al. 2025).

---

Owen (0.58)
Qwen Series (1.5/1.88)
Qwen (3/4B)
0wan2-0.58
0.25
Qwan2-1.0
0wen2.533
0.30
Q..32.5-0.88
Qwin2.61.18
0.20
0.25
Fraction of All Neurons (%)
0.15
0.15
0.35
Qwen Series (7B)
Series
Gemma Series
0.30
Qwen2-78
0.60
Uama-2278
332-11
0.40
Gemma-2-00 2-tb-g
0.25
Qwen2.5-78
Lama-248
Gemma--200 2-48
0.30
0.20
0.30
0.35
0.20
0.20
0.30
0.10
Shared
En
2h
Th Fr
Dn
SM
Shared
En
221
Th D4
SM
Shared En
25
17
Fr
D4
SW
Nauron Type
Nouron Type
Nauron Type
Figure 2: Neuron distribution across language-shared and language-exclusive neurons for six lan- guages (En, Zh, Th, Fr, Dr, Sw) in various model series and scales. For each model, we present the fraction of shared neurons and exclusive neurons to the total number of neurons.
Multilingual Benchmark We evaluate models across six typologically and resource-diverse lan- guages: Chinese (Zh), English (En), Thai (Th), Swahili (Sw), French (Fr), and German (De). This selection spans high-resource, medium-resource and low-resource languages, enabling a represen- tative analysis of language-related neuron behaviors. For our analysis, we utilize the Multilingual Massive Multitask Language Understanding (MMMLU) dataset (OpenAl) 2024), a cumaa-traassatted extension of the original MMLU benchmark (Hendrycks al. (021). available in 14 languages. In addition, we incorporate the Multilingual Grade School Math (MGSM) dataset (Shi et al. (022), a translated version of GSM8K (Cobbe et al., (021), which covers 10 languages. Together, these datasets provide quantitative measures of the models" multilingual capabilities.
Neuron Detection Corpus For each language, we identify language-related neurons by analyzing activation patterns on 1000 sentences sampled from the OSCAR corpus (Abadji et al. 2022). To quantify the functional contribution of these neurons, we further compute perplexity changes caused by deactivating them, using the same language-specific OSCAR data. This unified framework allows us assess both the proportion and the importance of seggaaee-spciffee and shared neurons across languages and model generations. More detailed illustration can be found in Appendix D
3.2 Analysis on Shared Neuron Ratio
Language-related neurons account for only small proportion in LLMs. To develop a prelimi- nary understanding of language-shared and language-exclusive neurons, we begin by analyzing the distribution of shared and language-exclusive parameters across all neurons within the model. For each language, we compute the proportion of language-shared and language -anguage-exclusive neurons rela- tive to the total number of neurons the model. Specifically, we calculate the ratios / and AV e LLMM), where é denotes specific language. The results, illustrated in Figure 2 encompass six languages across multiple model series. It shows that only a small fraction of neurons, often fewer than 1% play critical role in processing language, underscoring the sparsity and selectivity of language-relevant neural activations. Furthermore, the quantities of language-shared and language-exclusive neurons are of similar magnitude, each coarsely estimated around 0.3% of the total number of neurons in the LLM.
To further explore the evolution of shared and exclusive neurons across models and over time, we compute the overall shared neuron ratio for each model, as defined in Equation 4 relate it to multilingual performance measured by MMMLU and MGSM, and present the results Figure
The proportion of shared neurons increases with model evolution. We first group models from the same series and with similar parameter scales, as indicated by the shaded color regions in Figure 3 Within each group (e.g., Qwen1 7B, Qwen2-7B, and Qwen2.5-7B). we observe a steady and neuron ratio across generations. This growth closely
consistent increase in the shared-to-exclusee parallels improvements in the model's 's multilingual ability, with an average Pearson correlation coefficient of R 0.92 and a Spearman rank correlation of p 0.88, indicating strong and reliable
5

---

Trandline
Gartna--2 2-48
2025-01-12
2024-10-04
Shared Neuron Ratio
2.5
2024-06-26
Date
2.0
Mena 8.3
Owes2-d
2024-03-18
Uame-3.2-10
2023-12-09
Dwen1.S-0.30
Queen b-
2023-08-31
Lamn 1738
Owan1.5-7
0.5
Average Pearson R: 0.94
2023-05-23
Average Spearman p: 0.90
0.0
Multilingual Ability
Figure 3: The relationship between multilingual ability and shared neuron ratio (as defined in Equation across various models. Each point represents model, color-coded by its release date. Shaded regions indicate groups models within the same series and of comparable scale. The gray dashed line (-- illustrates the overall trend: models evolve, those with greater multilingual capabilities tend to exhibit higher proportion of shared neurons.
relationship. other words, later generations within the same series show a strong trend toward engaging more shared neurons for processing different languages.
The increase shared neuron proportion generalizes across model families. Beyond individual model series, observe that the positive relationship between the proportion of shared neurons and multilingual capability generally persists across different model families, as illustrated by the gray dashed line (----) in Figure Despite differences in architecture design and pretraining corpora, models with stronger multilingual ability tend to activate a larger proportion of shared neurons. For instance, the Gemma series exhibits both the most strong multilingual performance and the highest toared-to-exclusive neuron ratio. This consistency across diverse architectures suggests that progressively leveraging shared neurons may be general strategy adopted multilingual LLMs, regardless of their specific design choices.
3.3 Analysis on Language Agnostic Score
The above observations raise further question: whether the shared neurons not only occupy a larger proportion of the language-related neuron set, but also contribute more crit- ically to multilingual process- ing, effectively functioning sunguage-agnosse neurons. To address this question, inves- tigate how the language-shared neurons importance, i.e., lan- guage agnostic score defined Equation [6 evolves alongside multilingual capability across different generations of large language models.
log (1 Perplcsity) Across Language
8,000
Random
Shared
Exclusive
4.033
4,000
3,893
2,000
6.425
1.113
3,007
0.166
0.122
5,0Om
0.0338
0.2538
0.0008
0,000
Owen2.5-1.30
Quend.3 38
Det 3-78
Liama 3.2-8
Liama 3.2.38
Liana 3.1-88
gamma- pt.
gramma 2:3h
Figure 4: Perplexity changes caused by deactivating random neuron sets (Random), language-share shared neurons (Shared) and language- exclusive neurons (Exclusive). Notice that Random deactivation barely affects models' perplexity, while Shared and Exclusive de- activation break the models' abilities.
Deactivating shared and exclusive neurons both leads to model degradation. Before contrasting language-share shared neurons with language-exclusive neurons, conduct control experiment in

---

Gwen1 0nee3.5-1.5
10
2025:01-12
Language Agnostic Score
QweRt
2024-10-04
Oesetied
2024-06-26
Release Date
Oren
2024-03-18
2023-12-09
Liara
Uame-3
2023-08-31
Unrt-3-78
Owent
Average Pearson R: 0.77
2023-05-23
Deats
Average Spearman p:
0.3
0.5
Multilingual Ability
Figure 5: The relationship between multilingual ability and language-agnostic score (as defined in Equation 6 across various language models. Each point represents model, colored by model size. The red dashed line (- -) indicates where shared neuron influence surpasses that of exclusive neurons, i.e., Imp
and Language Agnostic Score log(2). Shaded regions group models within the same series and of similar scale, while the dashed line (----) indicates the overall trend: as LLMs evolve, successive generations with enhanced multilingual capabilities tend to achieve higher language-agnosta scores. This trend suggests that shared neurons increasingly support not only multilingual processing but also the emergence of more language-agnostic, abstract thought.
which deactivate equal number of randomly selected neurons, matching the quantity of both language-sharee and language-excluse exclusive neurons. As illustrated in Figure this random deactivation results in minimal changes in perplexity across languages. In contrast, deactivating language-shure or language-exclut exclusixcee ncurons leads to significant performance degradation. These results confirm that the identified language-related neurons indeed specialized for language processing, and that our neuron importance metrics are robust to random perturbations.
To further investigate whether language-shared neurons evolve into language-agnostic neurons, analyze the evolution of the language-agnostic score, as defined Equation relation to the models' multilingual capabilities, as shown in Figure
Shared neurons in early-stage models reflect superficial overlap without supporting higher-level cognition. In earlier models such as the Qwen1.5 series and LLaMA-1-7B, deactivating shared neurons has comparable effect to deactivating exclusive neurons, with Lunguage-agnoste scores around This suggests that early-stage models, shared neurons have the similar importance with exclusive neuron, and shared neurons largely reflect superficial overlaps between language-relate related neuron across languages, rather than representing distinet, functionally meaningful shared space.
Shared neurons in recent models become central and exhibit language-agnostic properties. In contrast, recent models, such as those the Qwen2.5 series, exhibit a dramatically different pattern. Deactivating shared neurons leads a sharp and disproportionate increase in perplexity across all languages, often several orders of magnitude greater than the increase caused by removing language- exclusive neurons. In other words, shared neurons contribute far more critically to multilingual processing than exclusive neurons, despite both being part of the language-related neuron set. This disproportionate degradation reveals that shared neurons in recent models have evolved beyond serving merely intersections of ainuage-speciftt components: they now fulfill more fundamental, language-agnostic roles. such shared neurons have indeed evolved into language-agnostic neurons, they may be operating within conceptual space that abstracts away from surface-leve linguistic variations. Such space would allow the model to perform high-level reasoning, semantic alignment, and tross-lingus generalization haapriiizttioo-aalllaa..eee of abstract thought in multilingual LLMs.
7

---

Table 1: Multilingual performance improvements on MGSM (primarily involving abstract thought) and MMMLU (requiring both abstract thought and domain knowledge) across five languages. Models were trained only on 100,000 general documents without reasoning-related data and evaluated using 1.lmm-3.11-8 (high language-agnosticg Llama-3.2-33 (medium language-agnostic), and Llama-3.2. 1B (low aegguage-agnostte under various targeted neuron tuning strategies.
MGSM
MMMLU
Neuron
7h
Fr
De
Th
Sw
A Aug
7h
Fr
De
Th
Sw
A Aee
Llama-3 L-8B
None
52.4
51.6
54.4
46.8
38.8
53.8
58.4
56.9
48.8
40.9
Shared
52.0
52.8
55.6
L12
45.6
39.6 10.6
0.3
54.6
57.2
56.5
48.9
42.3
0.1
Exclusive
56.8
57.2+5.6
$7.2
50,4+3.8
42.4+36
4.0
55.6
59.2
59.1
59.1+22
49.9
43,742.8
1.7
Random
50.4-20
2.0 51.2
0.4
54.4
-0.0
47.2
37.6 1.2
-0.6
52.4
1.4 58.3
0.1
57.2
47.1
1.7
41.3+0.4
-0.5
1.2a--322111 Llame-3 3.2.3.333 2.233
None
40.8
42.4
57.2
35.2
30.8
45.2
49.0
47.1
40.6
34.1
Shared
42.8+2.0
45.6
+3.2
66.4 19:2
40.4
H6.2
39.6** 8..
5.7
44.9
0.3
49.8
10.8 47.3
41.0
0.4
34.8 34.8+0.7
+0.7
0.4
Exclusive
42.4+16
43.2
65.6
37.2+2.0
36.0+5/2
3.6
44.9
0-3
48.9
0.1 47.1
+0.0
40.9
-0.9 34.7+0.0
Random
40.4-0.4
41.6-0.6
41.6
63.2+6.0
34.8-0.4
34.8
30.0
0.7
44.5
49.0
46.9
40.3
40.3-
34.1 +0.0
-0.2
None
26.4
26.0
29.2
20.0
22.8
29.0
27.8
28.8
28.8
26.6
Shared
30.0
30.4
30.8
22.4
26.4
3.1
29.2
28.7
29.5
29.5+0.7
29.4*0.6
26.8
0.5
Exclusive
27.6+1.2
30.0++.0
34,4+5.2
23,2+3.2
30,4+76
4.2
29.0
28.0+0.2
29.3
28.2
26.8
0.1
Random
26.8+004 26.4 0.4
26.4-0.4
29.6+0.1
21,2+1.
26.4+3.
1.0
28.8-0.2
0.2
28.3+0.5
29.1
+0.3
28.6
26.8
+6.2+0.2
0.1
4 Multilingual Enhancement via Neuron-Targeted Training
4.1 Language Agnostic Score Guided Multilingual Enhance
Inspired by above insights, we propose various targeted neuron training methods to enhance models' multilingual capability according to their language agnostic score.
LLMs with low language agnostic score can train any language-related neurons. These models exhibit limited multilingual capabilities, indicating that all language-related neurons require improve- ment. To enhance their performance across languages, we propose training all language-related neurons, whether they are shared across languages specific to individual ones.
LLMs with middle language agnostic score should train language-shared neurons. These models demonstrate degree of multilingual capability; however, the language-shared neurons have not yet evolved to become truly maaguage-agnostic Given that language-shane neurons are more prevalent than Ianguage-exclusiv ones in these models, it is essential to further train and refine them to more effectively enhance the models' multilingual performance.
LLMs with high language agnostic score should train language-exclusive neurons The language-shared neurons in these models have evolved into language-agnostic neurons, responsible for abstract thought. They already well-trained and offer limited room for further improve- ment. Therefore, to enhance multilingual performance, it is necessary to focus on training the language-cxclusive neurons in these LLMs.
4.2 Experiment Setup
Dataset To further validate our hypothesis and explore how to utilize our findings to efficiently enhance multilingual capability in L.Ms,, we conduct continuous pretraining specific neurons using multilingual corpora. Specifically, we construct a training set by sampling 100,000 examples per language from mixture of three widely used multilingual datasets: Culturax (Nguyen al.) 2024), MADLAD (Kudugunta et al. 2023). and Wikipedia (Guo et al. 2020).
Training Settings We utilize Llama3.22 2- IB Grattafiori et al. 2024 Llama3.2-3B and Lamma- 3. --B as representative LLMs with low, medium, and high language-agnosii scores, respectively. We conduct experiments under three training settings: language-share shared neurons, iagguage-exclusin neurons, and an equal number of randomly selected neurons. To evaluate multilingual capability, we employ the MMMLU and MGSM benchmarks.
8

---

Experiment Results Table demonstrates that the multilingual capabilities of language models can be effectively enhanced through targeted neuron-specific tuning. For Llama-3.2-1BB which exhibits relatively low language-agnostin score, tuning both shared and exclusive neurons significantly improves the model's cross-lingual reasoning performance, yielding average gains of 3.1 and 4.2 points on the MGSM benchmark, respectively. In the case of Llama-3.2-3B, which has a moder- ate languagc-agnostic score, tuning language-shared neurons results in the greatest performance improvement-an average gain of 5.7 points on MGSM. This is likely because these neurons are more numerous and less well-trained than exclusive ones. Finally, for Llama-3.2- 3.2--B, which already possesses high sanguage-agnostee score, the language-shared neurons appear to be sufficiently trained; thus, tuning exclusive neurons leads to further enhancement of multilingual performance, with observed improvement of 4.0 points on MGSM. Compared to the improvement observed on relies more heavily on knowledge extraction-is
MGSM, the performance gain on MMLU- relatively smaller. This suggests that our training approach primarily enhances the model's thinking capabilities rather than its factual recall. Moreover, since we exclusively utilize general documents without incorporating ceasoning-specffie data, the substantial improvement further validates the elfectiveness of our neuron-targeted training methodology. Additional experimental settings, results on different backbone LLMs, comparisons with baseline methods (e.g., LoRA), and cross-lingual evaluation analyses are provided in Appendix D and Appendix
5 Related Work
Thinking Language of LLMs
Large language models (LLMs) Touvron et al. 2023b OpenAI
2023 Zhang et al. [2024a] Chen et al. 2024] Liu et al. 2025c Gemma Team et al. (025) demonstrate
strong multilingual reasoning and transfer abilities (iies ct al. 2019; Wu and Dredze 2019 You
cct al. 2025 Cai et al. 2025; Nooralabzadeh et al., 2020), rasing questions about whether these
models operate in language-agnostic
sanguage-specitng concept space (Nanda al. 2023 Schut
iet al. 2025a Zhao et al. (024b). and which language would the model "think" in. One stream of work supports the hypothesis that LLMs "think" in concept space centered the predominant training language. Zhong et al. (2024) analyzed LLMs trained predominantly on English or Japanese (Fujii et al., 2024 LLM-jp et al.) 2024) for their mainly activated languages; FFerro al. (2025)
showed language dependence in object retrieval; and [Schut et al. (2025b), found representations align more closely with English even on foreign inputs. On the other hand, a language-agnostic
view is supported by either probing studies (Pires et al. 2019; Stanczak et al. (2022), neuron-level manipulations (Dumas et al.) 2024 Brinkmann
2025 Ding al. 2024) both (Wu et al. 2025
Wendler et al. 2024). Our work falls in line with Dumas et al. (2024): Wendler et al. (2024); ) Wu
jtt al. (2025). with more fine- -eeiggaiied neuron-level results and a novel activation-anc mond-training-aae analysis method.
Multilingual Enhancement Early-on, multilingual enhancement is mainly approached from pre- training in works such as XLM, XLM-R M(MnCCCnneau et al., 2020 Lample and Conneau 2019) and
M-BERT (Devlin et al. 2018). More post-training work, ranging from continual pre-training (Zhang
ct al. 2021| 2021| et al. [2024] et al. 2025b Husain et al. [024 Kuulmets et al. 2024) to fine-tuning
MMuennighofi et al. 2023] Chen et al. 2023 Ahuja et al. 2024] Lai et al. 2023 Indurthi et al. 2024
Lai and Nissim| 2024; Zhao et al. 2024c) have emerged to effectively improve models' ingal
abilities, though rather sensitive training corpus and settings. A parallel body of work focuses
prompt-based methods, either leaning on language alignment (Zhang et al. [022bb [Exaniz et al. cestruction-folloviin
2023 Zhao et al., 2024a)
and attention (Wang al.] 2005b; Zhao et al.
20256 a). However, Liu et al. (2024) points out the suboptimality in translation-
prompting
pipelines. Our neuron-specific tuning strategy answers the academic call (Liu et al.
2024 2025a) for
a more comprehensive approach to multilingual enhancement than translation-
prompting, and
provides more efficient and mask-neutral alternative than the post-training based methods.
Conclusion & Discussion
In this work, we explore the emergence of abstract thought large language models through the lens of neuron behavior. By identifying and categorizing language-related neurons as either shared or exclusive, we uncover a consistent trend across model development: shared neurons not only increase in proportion but also grow in functional importance, eventually forming compact yet

---

critical set of tonguage-agnossee neurons. These neurons underpin the model's 's ability to generalize across languages and support abstract reasoning that transcends linguistic boundaries. Motivated by this insight, we introduce neuron-specifir training strategies that adapt to the developmental stage of an LLM,, whether or not it exhibits language-agnostic behavior. Extensive experiments confirm that our targeted training approach effectively enhances multilingual performance across diverse models. We believe this neuron-centric perspective opens new avenues for understanding and improving the generalization capabilities of LLMs in multilingual and cross-lingual contexts.
Our study is limited by computational resources, which restricts evaluation on the larger LLMs and prevents full exploration of the potential of neuron-centric training at larger scales. We leave these directions for future work. Nonetheless, our findings shed light on the emergence of multilinguua and abstract reasoning in LLMs, which may promote language equity but also raise risks like cross- -ooss-lnguual misinformation, calling for responsible deployment.
Acknowledgements
This research supported by the National Research Foundation, Singapore under its National Large Language Models Funding Initiative (AISG Award No: AISG-NMLP-2024--00 and National Large Language Models Funding Initiative (AISG Award No: AISG-NMLP-2024-000
Any opinions, findings and conclusions recommendations expressed in this material are those of the authors and do not reflect the views National Research Foundation, Singapore. This project was also partially supported by the Singapore Ministry of Education Academic Research Fund Tier (Award Number: Ti 25 251RES25144 14) and TPU Research Cloud.
References
Julien Abadji, Pedro Javier Ortiz Suárez, Laurent Romary, and Benoît Sagot. 2022. Towards a cleaner document-oriented multilingual crawled corpus. In LREC, pages 4344 4355. European Language Resources Association.
Sanchit Ahuja, Kumar Tanmay, Hardik Hansrajbhai Chauhan, Barun Patra, Kriti Aggarwal, Lu- ciano Del Corro, Arindam Mitra, Tejas Indulal Dhamecha, Ahmed Awadallah, Monojit Choudhary, Vishrav Chaudhary, and Sunayana Sitaram. 2024. (sphinx: Sample efficient multilingual instruction fine-tuning through n-shot guided prompting
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge,
Yu Han, Fei Huang, et al. 2023. Qwen technical report. arXiv preprint
Jannik Brinkmann, Chris Wendler, Christian Bartelt, and Aaron Mueller. 2025, Large language models share representations of latent grammatical concepts across typologically diverse languages
Yuchen Cai, Ding Cao, Xin Xu, Zijun Yao, Yuqing Huang, Zhenyu Tan, Benyi Zhang, Guiquan Liu, and Junfeng Fang. 2025. On predictability of reinforcement learning dynamics for large language models. arXiv preprint arXh::2510.0055.
Wuyang Chen, Yanqi Zhou, Nan Du, Yanping Huang, James Laudon, Zhifeng Chen, and Claire Cu. 2023. Lifelong language pretraining with distributiion-spcciaiizz experts
Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. 2024. On softmax direct preference optimization for recommendation. In NewriPS.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman, 2021. Training verifiers solve math word problems. arXiv preprint arXiv:2110.1411
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale, In Proceedings of the 58th Annual Meeting of the Association for Computational Linguisties, pages 8440-8451, Online, Association for Computational Linguisties.
10

---

Yiming Cui, Ziqing Yang, and Xin Yao. 2024. Efficient and effective text encoding for chinese llama and alpaca
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding CoRR, abs/ 18..4800.
Chenlu Ding, Jiancan Wu, Yancheng Yuan, Jinda Lu, Kai Zhang, Alex Su, Xiang Wang, and Xiangnan He. 2024. Unified parameter-efficient unlearning for llms. arXiv preprint arXiv:24/2.00388
Clément Dumas, Veniamin Veselovsky, Giovanni Monea, Robert West, and Chris Wendler. 2024. How do Ilamas process multilingual text? a latent exploration through activation patching In
ICML 2024 Workshop on Mechanistic Interpretability.
Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lopez de Lacalle, and Mikel Artetxe. 2023. Do multilingual language models think better english?
Constanza Fierro, Negar Foroutan, Desmond Elliott, and Anders Søgaard. 2025, How do multilingual
language models remember facts?
Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations.
Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki lida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. 2024. Continual pre-training for cross. lingual IIm adaptation: Enhancing japanese language capabilities.
Gemma Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej. Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. 2025. Gemma 3 technical report. arXiv preprint vrXiv:2503.19788
Gemma Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rívière, Mihir Sanjay Kale, Juliette Love, et al. 2024a. Gemma: Open models based gemini research and technology. arXiv preprint arXiv:2403.08295.
Gemma Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari. Alexandre Ramé, et al. 2024b. Gemma 2: Improving open language models at practical size. arXiv preprint arXiv:2408.00) 18.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint 27X.v:2407.21783
Mandy Guo, Zihang Dai, Denny Vrandecic, and Rami Al-Rfou. 2020. Wiki-40b: Multilingual language model dataset. In LREC, pages 2452. European Language Resources Association.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multiaskk language understanding
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
Jaavid Aktar Husain, Raj Dabre, Aswanth Kumar, Jay Gala, Thanmay Jayakumar, Ratish Puduppully. and Anoop Kunchukuttan. 2024. Romansetu: Efficiently unlocking multilingual capabilitie of large language models via romanization!
Sathish Reddy Indurthi, Wenxuan Zhou, Shamil Chollampatt, Ravi Agrawal, Kaiqiang Song, Lingxiao Zhao, and Chenguang Zhu. 2024. Improving multilingual instruction finetuning via linguistically natural and diverse datasets)
Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Lean-Bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Cashon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas Beyer, Xiaohai Zhai Anton
11

---

Tsitsulin, Róbert Busa-Fekete, Alex Feng, Noveen Sachdeva, Benjamin Coleman, Yi Gao, Basil Mustafa, lain Barr, Emilio Parisotto, David Tian, Matan Eyal, Colin Cherry, San-Thorteen Peter, Danila Sinopalnikov, Surya Bhupatiraju, Rishabh Agarwal, Mehran Kazemi, Dan Malkin, Ravin Kumar, David Vilar, Idan Brusilovsky, Jiaming Luo, Andreas Steiner, Abe Friesen, Abhanshu Sharma, Abheesht Sharma, Adi Mayrav Gilady, Adrian Goedeckemeyer, Alaa Saade, Alexander Kolesnikov, Alexei Bendebury, Alvin Abdagic, Amit Vadi, András György, André Susano Pinto, Anil Das, Ankur Bapna, Antoine Miech, Antoine Yang, Antonia Paterson, Ashish Shenoy, Ayan Chakrabarti, Bilal Piot, Bo Wu, Bobak Shahriari, Bryce Petrini, Charlie Chen, Charline Le Lan, CJ Carey, Cormac Brick, Daniel Deutsch, Danielle Eisenbud, Dee Cattle, Derek Cheng, Dimitris Paparas, Divyashree Shivakumar Sreepathihalli, Doug Reid, Dustin Tran, Dustin Zelle, Eric Noland, Erwin Huizenga, Eugene Kharitonov. Frederick Liu, Gagik Amirkhanyan, Glenn Cameron, Hadi Hashemi, Hanna Klimezak-Plucinkka, Harman Singh, Harsh Mehta, Harshal Tushar Lehri, Hussein Hazimeh, Ian Ballantyne, Idan Szpektor, and Ivan Nardini. 2025. Gemma 3 technical report. CoRR, 0bs/2503.19986
Christopher A. Choquette-Chonl
Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2023. MADLAD-400: multilingual and decument-level large audited dataset. In NeurIPS.
Hele-Andra Kuulmets, Taido Purason, Agnes Luhtaru, and Mark Fishel. 2024. Teaching llama new language through cross-lingual knowledge transfer
Huiyuan Lai and Malvina Nissim. 2024. mcot: Multilingual instruction tuning for reasoning
consistency in language models
Viet Dac Lai, Chien Van Nguyen, Nghia Trung Ngo, Thuat Nguyen, Franck Dernnncourt, Ryan A. Rossi, and Thien Huu Nguyen. 2023. Okapi: Instruction-tuned large language models in multiple languages with reinforcement learning from human feedback
Guillaume Lample and Alexis Conneau. 2019. Cross-lingual language model pretraining
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilié, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2023. Bloom: A 176b-parameter open-access multilingual language model.
Chaoqun Liu, Wenxuan Zhang, Yiran Zhao, Anh Tuan Luu, and Lidong Bing. 2024. Is translation all you need? a study on solving multilingual tasks with large language models. arXiv preprint arXiv:2403.1023
Chaoqun Liu, Wenxuan Zhang, Yiran Zhao, Anh Tuan Luu, and Lidong Bing. 2025a. Is translation all you need? a study on solving multilingual tasks with large language models,
Xiaohao Liu, Xiaobo Xia, See-Kiong Ng, and Tat-Seng Chua. 2025b. Principled multimodal representation learning. arXiv preprint 2507
Xiaohao Liu, Xiaobo Xia, Weixiang Zhao, Manyi Zhang, Xianzhi Yu, Xiu Su, Shuo Yang, See-Kiong Ng, and Tat-Seng Chua. 2025c. L-mip: Leap multi-token prediction beyond adjacent context for large language models. arXiv preprint arXiv:2505. 175.5
LLM-ip, :, Akiko Aizawa, Eiji Aramaki, Bowen Chen, Fei Cheng, Hiroyuki Deguchi, Rintaro Enomoto, Kazuki Fujii, Kensuke Fukumoto, Takuya Fukushima, Namgi Han, Yuto Harada, Chikara Hashimoto, Tatsuya Hiraoka, Shohei Hisada, Sosuke Hosokawa, Lu Jie, Keisuke Kamata, Teruhito Kanazawa, Hiroki Kanezashi, Hiroshi Kataoka, Satoru Katsumata, Daisuke Kawahara, Seiya Kawano, Atsushi Keyaki, Keisuke Kiryu, Hirokazu Kiyomaru, Takashi Kodama, Takahiro Kubo, Yohei Kuga, Ryoma Kumon, Shuhei Kurita, Sadao Kurohashi, Conglong Li. Taiki Maekawa, Hiroshi Matsuda, Yusuke Miyao, Kentaro Mizuki, Sakae Mizuki, Yugo Murawaki, Akim Mous- terou, Ryo Nakamura, Taishi Nakamura, Kouta Nakayama, Tomoka Nakazato, Takuro Nittsuma, Jiro Nishitoba, Yusuke Oda, Hayato Ogawa, Takumi Okamoto, Naoaki Okazaki, Yohei Oseki, Shintaro Ozaki, Koki Ryu, Rafal Rzepka, Keisuke Sakaguchi, Shota Sasaki, Satoshi Sekine, Kohei Suda, Saku Sugawara, Issa Sugiura, Hiroaki Sugiyama, Hisami Suzuki, Jun Suzuki, Toyotaro Suzumura, Kensuke Tachibana, Yu Takagi, Kyosuke Takami, Koichi Takeda, Masashi Takeshita, Masahiro Tanaka, Kenjiro Taura, Arseny Tolmachey, Nobuhiro Ueda, Zhen Wan. Shuntaro Yada,
12

---

Sakiko Yahata, Yuya Yamamoto, Yusuke Yamauchi. Hitomi Yanaka, Rio Yokota, and Koichiro Yoshino. 2024. Llm-jp: A cross-organizational project for the research and development of fully open japanese llms.
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong. Hailey Schoelkopf, Xiangru Tang. Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel. 2023. [Crosslingual generalization through multitask finetuning.
Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. Progress measures for grokking via mechanistic interpretability
Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai, Hieu Man, Nghia Trung Ngo, Franck Dernoncourt, Ryan A. Rossi, and Thien Huu Nguyen. 2024. Culturax: A cleaned, enormous, and multilingual dataset for large language models 167 languages. In LREC/COLING, pages 4226-4237 ELRA and ICCL.
Jinjie Ni, Rui Mao, Zonglin Yang, Han Lei, and Erik Cambria. 2023. Finding the pillars of strength for multi-head attention. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14526-14540.
Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva, and Isabelle Augenstein. 2020. sero-shot cross-lingual transfer with meta learning] In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4547-4562, Online. Association for Computational Linguisties.
OpenAl. 2023.
Gpt-4 technical report
OpenAL. 2024. Multilingual massive multitask language understanding (mmmlu) Hugging Face.
Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguisties, pages 4996-5001, Florence, Italy. Association for Computational Linguistics.
Libo Qin, Qiguang Chen, Fuxuan Wei, Shijue Huang, and Wanxiang Che. 2023. Cross-lingua prompting: Improving zero-shot chain-of-thought reasoning across languages. In The 2023 Conference on Empirical Methods in Natural Language Processing.
Lisa Schut, Yarin Gal, and Sebastian Farquhar. 2025a. Do multilingual lims think in english? arXiv preprint arXiv:2502.15603
Lisa Schut, Yarin Gal, and Sebastian Farquhar. 2025b. Do multilingual ilms think in english?
Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2022. Language models are multilingual thain-of-thought reasoners. In The Eleventh International Conference on Learning Representations.
Karolina Stanczak, Edoardo Ponti, Lucas Torroba Hennigen, Ryan Cotterell, and Isabelle Augenstein. 2022. Same neurons, different languages: Probing morphosyntax in multilingual pre-trained models In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1589-1598, Seattle, United States. Association for Computational Linguistics.
Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Wayne Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. Language-speccfic neurons: The key to multilingual capabilities in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), pages 5701-5715
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arX5v:2403.05530
13

---

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.139:
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b. Llama 2: Open
foundation and tne-tuned chat models. arXiv preprint arXiv:2307.09288
Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, et al. 2024. Aya model: An instruction finetuned open-access multilingual language model. arXiv preprint arXiv:2402.07827
Chengxin Wang, Yiran Zhao, Shaofeng Cai, and Gary Tan. 2025a. Investigating pattern neurons in urban time series forecasting In The Thirteenth International Conference on Learning Representa- tions.
Teng Wang, Zhenqi He, Wing- Yin Yu, Xiaojin Fu, and Xiongwei Han. 2025b. Large language models are good multi-lingual learners: When ilms meet cross-lingual prompts
Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. Do llamas work in english? on the latent language of multilingual transformers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1; Long Papers), pages 15366-15394.
Shijie Wu and Mark Dredze. 2019. Beto, bentz, becas: The surprising cross- looss-lingual effectiveness of BERT In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP- IJCNLP), pages 833-844, Hong Kong, China. Association for Computational Linguisties.
Zhaofeng Wu, Xinyan Velocity Yu, Dani Yogatama, Jiasen Lu, and Yoon Kim. 2025. The semantic hub hypothesis: Language models share semantic representations across languages and modalities
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu. Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang. Jiamhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang. Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. 2024a. Qwen2 technical report
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, al. 2024b. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115.
Runyang You, Yongqi Li, Meng Liu, Wenjie Wang. Liqiang Nic, and Wenjie Li. 2025. test-time scaling for latent reasoning models.
Parallel
An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang. and Tat-Seng Chua. 2024a. On generative agents in recommendation. In SIGIR, pages 1807-1817. ACM.
Yongheng Zhang, Qiguang Chen, Min Li, Wanxiang Che, and Libo Qin. 2024b. Autocap: Towards automatic cross-lingual alignment planning for zero-shot chain-of-thought
Zhengyan Zhang, Yuxian Gu, Xu Han, Shengqi Chen, Chaojun Xiao, Zhenbo Sun, Yuan Yao, Fanchao Qi, Jian Guan, Pei Ke, Yanzheng Cai, Guoyang Zeng, Zhixing Tan, Zhiyuan Liu, Minlie Huang, Wentao Han, Yang Liu, Xiaoyan Zhu, and Maosong Sun. 2021. Cpm-2: Large-scale cost-effective pre-trained language models.
Weixiang Zhao, Jiahe Guo, Yang Deng, Tongtong Wu, Wenxuan Zhang, Yulin Hu, Xingyu Sui, Yanyan Zhao, Wanxiang Che, Bing Qin, et al. 2025a. When less language is more: Language-reasoningg disentanglement makes lims better multilingual reasoners. arXiv preprint arX5v.2505.15257.
14

---

Weixiang Zhao, Yulin Hu, Yang Deng, Tongtong Wu, Wenxuan Zhang, Jiahe Guo, An Zhang, Yanyan Zhao, Bing Qin, Tat-Seng Chua, et 2025b. Mpo: Multilingual safety alignment via reward gap optimization. arXiv preprint 2rXiv:2505.
Weixiang Zhao, Yulin Hu, Jiahe Guo, Xingyu Sui, Tongtong Wu, Yang Deng, Yanyan Zhao, Bing Qin, Wanxiang Che, and Ting Liu. 2024a. Lens: Rethinking multilingual enhancement for large language models. arXiv preprint 2411:2441.04407.
Yiran Zhao, Chaoqun Liu, Yue Deng. Jiahao Ying. Mahani Aljunied, Zhaodonghui Li, Lidong Bing, Hou Pong Chan, Yu Rong, Deli Zhao, et al. 2025c. Babel: Open multilingual large language models serving over 90% of global speakers. arXiv preprint arXiv:2503. 00865.
Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024b. How do large language models handle multilingualism? In NeurIPS.
Yiran Zhao, Wenxuan Zhang, Huiming Wang, Kenji Kawaguchi, and Lidong Bing. 2024c.
Adamergex: Cross-lingual transfer with large language models via adaptive adapter merging
Chengzhi Zhong. Fei Cheng, Qianying Liu, Junfeng Jiang, Zhen Wan, Chenhui Chu, Yugo Murawaki, and Sadao Kurohashi. 2024. Beyond english-centric llms: What language do multilingual language models think in?
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction elearly articulate the paper's key contributions, including the discovery of maguaae--gnnosee neurons and the proposal of neuron-centric training strategies, which are thoroughly supported by the analyses and experiments in the main text.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We explicitly discuss the limitations of our work in Section 6
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification: While the main paper focuses on empirical findings, we provide the full set of assumptions and complete proof for necessary theoretical analysis, such the parallel neuron detection method, in the supplementary material, ensuring theoretical soundness.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide all necessary details for reproducing our main experimental results in Section 3.1 and Section 4.2 Additionally, we include an anonymous link to our code in the abstract to further support reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
17

---

reproducible, they 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: An anonymous link to the code is provided in the abstract, and it includes sufficient instructions to reproduce the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
public/ guides/ Codeeu Codee/edeeeeeeeesmmiimmccce wtt.:/oooooisoooooooooooooooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https //nips.. cc/ pttpppullcc/aa htips.cccpublic////ccccco
quides////// /Coeeoooeuulll Submi commissionn cesionoolieyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide all necessary details Section 3.1 and Section 4.2 will be included in the supplementary materials.
Further details
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: We report error bars where applicable and include both Pearson and Spearman correlation coefficients ensure the statistical reliability and robustness of our findings.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide the necessary information compute resources used for each experiment. Further details are included in the supplemental material.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't itdn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://pww....a.........aaaaaaaa
Answer: [Yes]
Justification: Our research fully adheres the NeurIPS Code of Ethics. We ensure trans- parency, reproducibility, and responsible use of models, and our work does not involve sensitive data, human subjects, or harmful applications.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We explicitly discuss the broader impacts of work in Section
Guidelines:
19

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Our work relies solely on publicly available open-source models and datasets. We do not release any new models data that would require additional safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All external assets used in this work, including models and datasets, are open-source and properly credited. Their licenses and terms of use have been respected as per their original distribution.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
20

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, (papersw superswithoooes com/ datase has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We include all necessary details about the dataset and codes in Section 3.1 Section 4.2 and the anonymous link.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for rescarch with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [Yes]
Justification: This study does not involve human subjects and thus does not require IRB approval.
Guidelines:
21

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were only used for minor writing and editing purposes, and did not contribute to the core methodology or scientific content of the research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer LLM policy (https: ctteerrreppsccc//oofeeeeeeee///e LLM) for what should or should not described.
22

---

Appendix
A Parallel Neuron Detection Algorithm
24
Feed-Forward Network (FFN) Neurons
24
A.2 Self-Attention Network Neurons
24
Neuron Detection Threshold
26
Neuron Analysis
26
Loeponent-Level Distribution
26
C.2 Layer-wise Distribution
26
Unique Language Neurons
27
D Neuron Detection Corpus
28
D.I OSCAR Corpus
28
D.2 Illustration of Sample Sentences
29
Multilingual Enhancement
29
Iuuuttaaionn of Random Neurons
29
Results on Additional Backbones
30
Comparison with LoRA
31
Cross-lingual Evaluation
31
Broder Impacts
32
Limitations
32
23

---

A Parallel Neuron Detection Algorithm
Inspired by Zhao al. (2024b) Wang al. (2025a, the neuron detection method in Equation 2 can be done parallel. While Equation 2 considers the change the final output embedding, the parallel methods described here efficiently calculate the change in the output of the specific layer containing the neuron when that neuron is deactivated. This layer-wise impact serves as a proxy or component for the overall impact.
In this context, let E RR(xdmode be the input hidden states to a given layer, where / is the sequence length and dmodet is the hidden dimension of the model. For neuron / within this layer, its impact measured as (I; Q) - f(X; ew))l where f(X; @) is the layer's output with parameters e and f(X; ewx) is the output when neuron N (a specific row column in () is deactivated (its parameters set to zero).
A.1 Feed-Forward Network (FFN) Neurons
A standard FFN layer in modern transformer models can be expressed as:
FXN = (SiLU(XW, gate) ) O (X Wap)) Wdown
where X E Rixhmods is the input to the FFN layer, W Wgate: Wup E R.dmodel Xdinter and W down E Pditee X Xemodol Here, dinter is the intermediate dimension of the FFN. The symbol @ denotes element-wise multiplication. Let aet SiLJU(X gate) (X up) be the intermediate activation matrix, Huct [xxdetter Thus, the FFN output is YFFN Hast Wdown Rixdmodel,
We consider a neuron Ninter,& be associated with the k-th dimension of the intermediate repre- sentation Hos. Deactivating such neuron means that the k-th column of Hoet, denoted Hoct k). effectively zeroed out before the multiplication with Wdown. This deactivation corresponds to zeroing out the parameters that produce this k-th intermediate feature, e.g. the k-th column of W up (i.c., ncuron N W Wupl: k)) and W gate. or by zeroing out parameters that read from it, c.g., the k-th row of W Wdown (i.e., neuron N is W Woown (k.
Let Yerrre be the output when the k-th intermediate neuron deactivated. The change in the layer's s output is:
AYFFN, YFFN YPFN.eNiuter.t
If H.
oct is Hact with its k-th column zeroed, then YPrN.GNmeen.. H W down- So, oct
AYFENA (Hact HHaot)
The matrix (Hoet - H'at) is zero everywhere except for its k-th column, which consists of the elements Hactli [i, k). Let this difference matrix be òl1g. Then AYFFN, &HH W down. This resulting I x dmodel matrix is formed by the outer product of the k-th column of Hact and the k-th row of W down:
AYFFN. Hacth |:. (Wdoww
The impact of the k-th intermediate FFN neuron is then the L2 norm of this change:
||Y|Fn = ||Hacel [:, down)..
(8)
This computation can be performed in parallel for all & E {l,..., dinter) to obtain the impact of all intermediate neurons in the FFN layer.
A.2 Self-Attention Network Neurons
The output of self-attention layer (for simplicity, we describe single attention head; multi-head attention involves similar computations per head) can be given by:
(XX2)
YAuR Softmax
(XWv)
Vdl.
Let Q XWQ Iixddut. K XWK E Bixdau,, and V XWv E I!xdaun where dattn is the dimension of queries, keys, and values for the attention mechanism. d2 the scaling factor, typically the dimension of the key/query vectors (e.g.. de datt)). Let A Softmax [KK" Vaz
Rixl The layer output is YAttn = AV E [Rlxdan,, (An additional output projection Wo might follow this, which would be multiplied subsequently).
24

---

A.2.1 Neurons in Wy
Consider neuron Nv, defined as the k-th column of Wv, i.e., Wv[. kj. Deactivating this neuron sets Wv[: k] to zero, which in turn makes the k-th column of V XWy, denoted [[t, zero. Let V" be the matrix V with its k-th column zeroed. The change in the layer's output is:
AY AV AV' AV
The matrix (V - VV) zero everywhere except for its k-th column, which V[; Let this difference matrix be oV.. Then AY Attn,k A(VV.). This x dattn matrix has AV[: k] (the matrix A multiplied by the vector V [i k)) as its k-th column, and zeros in other columns. The impact of neuron Nv.A is:
AY
Acem.b |12
||AV[ [:, kill2
(10)
where the norm effectively taken over the vector AVI: k] that forms the k-thh column of the change matrix. This can be calculated in parallel for all &o E {1...., dattn
A.2.2 Neurons in WqQ
Consider neuron Niq.: defined the k-th column of Wg. i.e., Wol: k). Deactivating this neuron sets Wol: k] to zero. This makes the k-th column of Q = XWo. denoted Q k). zero. Let Q' be the matrix Q with its k-th column zeroed. The original unnormalized attention scores are Sraw = OK
vdc"
The new unnormalized attention scores with NQq deactivated are Syow Q'K" dda
The change in
the unnormalized scores due to deactivating NQ, is ASraw, = Sraw S (Q-9)k² The matrix (Q Q') is everywhere except for its k-tth column, which is Q(: k). Thus,
/dc
ASrow,k [Q (K)
dd
This / matrix represents the change in raw attention scores attributable to the interaction involving the k-th column of Q and the k-th column of K.
softmax(Sraw) be the original attention probability matrix. Let AgNig softmax (Sraw AS ))) be the attention probability matrix when neuron No, is deactivated. The change in the layer's output is:
Let Aorig
AY
(Q) Ammm, Aorig -ANNa. V= (Aoria AANa,
The impact of neuron No. ViQuiia is:
AYYOO
Attn, 1 = (Aorig AZNo M122
(11)
To calculate this efficiently for all A: {1,..., ) (corresponding to each column neuron in Wq):
Compute the original Sraw
QKT and Aorig Saitmax(Sraell Vdl
2. For each k compute the specific change term AS QlAA((K(.))) This step can be
Vdd
parallelized by constructing a tensor ASraw Rduci xixt where the slice A.Sawllkii = ASraw
3. For each k compute the adjusted scores Sadjustedd, Sram ASram i,
4. For each k, compute AeNa, = softmax poftmax(Sadjnetedaa)
5. For each k calculate the impact norm [[(Aorig AeNa
A.2.3 Neurons in Wx
The impact of deactivating a neuron NK. (the k-th column of Wk) is calculated symmetrically to that of No. The same change term ASraw,k = ((K))()))) is used, reflecting the idea that this
vag term captures the interaction component associated with the k-th features of both Q and K. The procedure then follows steps 3-5 as outlined for Wo neurons, using this ASraw find the adjusted attention matrix and the resulting impact.
25

---

B Neuron Detection Threshold
An important implementation detail in identifying language-related neurons lies in the selection of the activation threshold ø. Rather than adopting a fixed global scalar, we employ dynamic thresholding mechanism that adapts to each query. Specifically, as shown in our released implementation, for every query in given language, we rank neurons based on their computed importance scores and select the top 1% activated neurons. Subsequently, for each language E, its tonguage-speciine neuron set lang is defined as the intersection of these top-ranked neurons across all queries belonging to that language.
This dynamic top-1% strategy ensures consistent sensitivity across languages with different overall activation magnitudes, allowing the model to capture meaningful variations without being biased by language-specific activation scales. The choice of 1 1% is empirically determined through set of calibration experiments designed to balance selectivity and stability.
To validate the appropriateness of this threshold, we conduct sanity check using random baselines. For each language, we compare the model degradation caused by deactivating the selected language- specific neurons with that caused by deactivating an equal number of randomly chosen neurons. In all cases, we observe that removing the identified neurons results in a drastic performance drop-ofter removing random neurons yiclds negligible effects.
exceeding a 1 increase in perplexity- This substantial performance disparity confirms that the selected neurons are functionally meaningful and that the threshold effectively distinguishes critical neurons from background noise.
If, conversely, random neuron deactivation were cause a comparable decline in performance, it would suggest that the threshold is too lenient, allowing excessive neurons to be classified as important. In such cases, the percentile threshold would be systematically reduced until the random baseline no longer impacts model behavior. This adaptive validation process ensures that the threshold a remains both rigorous and empirically grounded across all examined languages and model families.
C Neuron Analysis
To further understand the internal organization of multilingual representations within large language models, we conduct comprehensive neuron-level analysis. This section explores how language- shared, language-exclusive, and strictly unique neurons are distributed across different architectural components and model layers, offering insight into how multilingual models balance generalization and specialization.
C.1 Component-Level Distribution
We first analyze how neurons are distributed across major architectural components. Neurons are categorized into three groups: query-key (QK), value-output (VO), and feed-forward network (FFN). As shown in Table 2 shared neurons are primarily concentrated in the QK components, aligning with the general attention mechanism responsible for capturing cross-lingual relational patterns. In contrast, exclusive neurons are more prevalent VO and FFN layers, indicating their more important role in language-spe specific transformations and output generation. This decomposition suggests that shared and exclusive neurons perform complementary roles in multilingual processing.
We find that the component-level trend remains consistent across model families: shared neurons con- centrate within the attention's QK submodules, supporting cross-lingual abstraction, while exclusive neurons appear more prominently in VO and FFN blocks, handling language -epecgfi-- representations and refinements.
C.2 Layer-wise Distribution
We further examine how these neurons are distributed across model layers. Figures 6 and illustrate the proportion of shared and exclusive neurons across all layers for Gemma2-99 2B and LLaMA3.1-8B respectively. To quantify the trend, we group layers into early, middle, and late stages and report the average proportions in Table
The layer-wise distribution reveals distinct allocation patterns for shared and exclusive neurons. Exclusive neurons are more concentrated in the early and late layers, suggesting that language-
26

---

Table Component-lewe neuron distribution across models. Percentages represent the proportion of shared or exclusive neurons (averaged across languages) within each component.
Model
Component
Shared (%)
Exclusive (%)
LLaMA3.1-8B
QK
92.50
59.48
VO
2.90
23.80
FFN
4.59
16.71
Qwen2.5-3B
QK
76.40
53.06
VO
16.72
30.36
FFN
6.88
16.58
Gemma2-9B
QK
65.32
35.40
VO
22,02
40,46
FFN
12.66
24.16
16
Figure 6: Layer-wise distribution analysis of Gemma2-9B.
specific processing primarily occurs near the input and output boundaries, where lexical and syntactic variations are handled. In contrast, shared neurons maintain stable proportion across all layers, reflecting their role in capturing transferable, cross-lingual representations throughout the network.
Overall, these analyses demonstrate that while both neuron types are crucial for multilingual pro- cessing, shared neurons form stable representational backbone that supports language-agnostic reasoning, whereas exclusive neurons enable fine-graines grained, language-speciff adjustments near the model periphery. These findings consistent across LLLMA, Qwen, and Gemma series, reinforcing the robustness of this observation.
C.3 Unique Language Neurons
To further refine understanding of neuron selectivity, investigate strictly unique neurons- those that respond exclusively to one language. While our definition of language-exclle neurons allows for activation across subset of languages, this stricter criterion provides additional insight into the specialization of multilingual models.
Table 4 reports the proportion of strictly unique neurons per language. These neurons constitute only a small percentage of the total population, suggesting that the model predominantly relies shared neurons for multilingual understanding. Interestingly, higher values for Swahili and Thai-both both lower-resource languages- indicate a stronger reliance on language-speciffe neurons, likely to compensate for limited training data.
We also analyze their layer-wise distribution by grouping the model layers into three stages: early (0-7), middle (8 (8-23), and late (24-31) The averaged proportions of unique, shared, and exclusive neurons each group are shown in T The early and late stages exhibit higher proportions of unique and exclusive neurons, implying that language-specinn the input and output boundaries. The middle layers, dominated by shared neurons, correspond to cross- lingual abstraction stage responsible for Linguage-indpennde reasoning.
encoding and decoding occur near
27

---

stand cxlusit
exblaing
Figure 7: Layer- Layer-wise distribution analysis of LLaMA3.1-8B.
Table Liser-wise neuron distribution across representative models. Percentages indicate average proportions of shared and exclusive neurons within each layer range.
Model
Layer Range
Shared (%)
Exclusive (%)
LLaMA3.1-8B
0-7
3.07
4.40
8-23
3.11
2.89
24-31
2.83
3.83
Qwen2.5-3B
0-5
3.69
4.45
6-29
2.70
2.43
30-35
2.20
2.48
Gemma2-9B
0-5
2.41
3.13
6-34
2.39
1.92
35-41
2.29
3.65
Finally, examine the component-level distribution of strictly unique neurons (Table 6). Shared neurons cluster within QK, reinforcing their role in pooss-inggual alignment. Exclusive neurons, as well as strictly unique ones, more concentrated in VO and FFN, highlighting their specialization in language-specific semantic transformation and output projection.
These findings reinforce our broader conclusion: shared neurons underpin cross-lingual generaliza- tion, capturing transferable semantics; exclusive neurons encode language-specific nuances, crucial for accurate understanding and generation; and strictly unique neurons-being the most selective reflect the model's fine-grained adaptation to individual languages, typically concentrated at the model's periphery where input encoding and output generation occur.
D Neuron Detection Corpus
This section provides additional details regarding the corpus used for neuron detection, as mentioned in the main text. Our methodology relies on the OSCAR corpus for both identifying language-related neurons through activation patterns and quantifying their functional contribution via perplexity changes upon deactivation.
D.1 OSCAR Corpus
The OSCAR (Open Super-large Crawled Aggregated coRpus) corpus (Abadji al. 2022) massive
multilingual collection of texts obtained by language classification and filtering of the Common Crawl dataset. Common Crawl is publicly available web crawl spanning petabytes of data. OSCAR further processes this raw data to produce monolingual corpora across a wide range of languages, making it a valuable resource for training large language models and conducting cross- cross-lingual research.
Key characteristics of the OSCAR corpus include:
Large Scale: It contains hundreds of gigabytes to terabytes of text data for many languages.
28

---

Table 4: Proportion of strictly unique neurons (responding to only one language).
Language
Strictly Unique Neurons (%)
English (en)
0.02
Chinese (zh)
0.03
Thai (th)
0.06
French (fr)
0.03
German (de)
0.02
Swahili (sw)
0.07
Table 5: Layer-wist distribution of unique, shared, and exclusive neurons in LLaMA3,1-8B
Layer Group
Unique (%)
Shared (%)
Exclusive (%)
Early (0-7)
3.69
3,07
4.40
Middle (8-23)
2.28
3.11
2.89
Late (24-31)
4.62
2.83
3.83
Multilingual Coverage: It supports a vast number of languages, facilitating studies that require diverse linguistic data.
Data Cleaning: Efforts are made to clean and filter the crawled data, though the quality can vary depending on the language and the nature of web content.
Accessibility: OSCAR is publicly available, promoting reproducibility and broader research in NLP.
For our study, sample 1000 sentences for each target language from its respective monolingual section within the OSCAR corpus. This sampled data serves as the basis for analyzing neuron activations and evaluating perplexity changes. The diversity and scale of OSCAR help in capturing a wide array of linguistic phenomena necessary for robustly identifying language-specifie neural correlates.
D.2 Illustration of Sample Sentences
To provide a concrete illustration of the data used, Table 7 presents conceptual example sentences from the OSCAR corpus for the five languages central to analysis: English (en), Chinese (zh), Swahili (sw), German (de), and French (fr).
The sentences sampled for each language are then further used to observe which neurons are consistently activated during processing. similar of sentences is then used to measure the perplexity of the model when specific neurons or sets of neurons are deactivated, thereby quantifying their functional importance that language.
E
Multilingual Enhancement
This section provides additional details on multilingual enhancement experiments, including (1) the construction of random neuron baselines, (2) results on additional backbones, and (3) comparisons with LoRA, famous parameter-efficient fine-tuning method. Together, these analyses validate the robustness and efficiency of our neuron-level enhancement strategy.
E.1 Illustration of Random Neurons
Regarding the random neurons presented in Figure 4, we carefully designed the sampling process to ensure fair comparison. Specifically, we sampled two sets of random neurons-each matching the total number of shared and exclusive neurons, respectively. These random neurons were uniformly sampled across all layers and components of the model, under the assumption of homogeneous distribution.
29

---

Table Component-level comparison among unique, shared, and exclusive neurons in LLaMA3.1-
8B.
Component
Unique (%)
Shared (%)
Exclusive (%)
QK
18.40
92.50
59.48
VO
50.96
2.90
23.80
FFN
30.64
4.59
16.71
Table 7: Illustrative sample sentences from the OSCAR corpus for the selected languages. These are conceptual examples, actual sentences randomly sampled.
Language
Conceptual Example
English (en)
The quick brown fox jumps over the lazy dog.
Chinese (zh)
S AT TI J (
Swahili (sw)
Mbweha mwepesi wa kahawia anaruka juu ya mbwa mvivu.
German (de)
Der schnelle braune Fuchs springt über den faulen Hund.
French (fr)
Le renard brun rapide saute par-dessus chien paresseux.
We then selected the random set that exhibited stronger influence on model performance, and used it consistently in both Figure and Table 1. This ensures comparability across evaluations. It is indeed expected that the distribution and activation patterns of random neurons differ from those of the identified language-specifer neurons, as the latter capture semantically grounded linguistic features rather than arbitrary activation patterns.
E.2
Results on Additional Backbones
Figure presents simplified cross-model analysis within the generation but across different model sizes. We observe that larger models generally exhibit lower proportion and reduced importance of shared neurons. This aligns with prior findings showing that as models scale up. parameter specialization increases, leading to fewer neurons being shared across languages.
Although main focus lies in analyzing the evolution of abstract thought model development rather than size scaling, we include this discussion for completeness. To further verify the generality of our findings, conducted additional analyses on the Qwen-2.5 family, particularly the 1.5B variant. The results are summarized below:
Table Performance comparison shared, exclusive, and random neuron sets on Owen-2.5-1.5BB. Metrics represent accuracy (%) on MGSM and MMMLU datasets.
MGSM
MMMLU
Model
de
fr
th
Qwen2.5-1.55
63.60
57.20
61.20
50.80
28.00
53.95
48.47
50.96
44.00
30.49
Exclusive
65.20
57.60
61.60
52.00
29.60
+1.04
53.81
48.65
51.29
44.49
31,57
+0.39
Shared 63.20
56.80
62.00
49.20
31.20
+0.32
53.72
48.38
51.30
44.42
31.48
+0.29
Random 62.80
54.00
60.40
46.80
27.60
-2.88
53.71
48.62
51.09
44.17
31.36
-0.17
These results indicate that both shared and exclusive neuron adjustments consistently improve multilingual reasoning, whereas random neuron updates negatively affect performance.
We further tested our findings on the Gemma2-9B model, which represents a different model family and high language-agnostic score the global level. The results are presented in Table 9
We observe consistent trends across backbones: fine-tuning exclusive neurons enhances reasoning in target languages, while shared neurons contribute general stability. In contrast, random or unstructured modifications fail to improve multilingual alignment.
Furthermore, our GSM8K experiments confirm that fine-tuning model specific language im- proves reasoning in that language but may degrade performance in suhers--upporting our hypothesis that language-speciff adaptation often comes at the cost of reduced cross-lingual transfcrability.
30

---

Table Performance of neuron subsets on Gemma2-9B across languages.
Subset
Zh
Fr
De
Th
Sw
Avg A
None
58.4
58.0
58.8
57.2
51.2
Shared
56.8
57.6
58.8
54.8
48,4
-1,4
Exclusive
61.6
60.8
62.4
58.4
55.6
+3.0
Random
56.0
57.6
57.2
56.0
50.8
-1.2
E.3 Comparison with LoRA
Although our method focuses on neuron-level tuning rather than introducing new fine-tuning layers, we further conduct comparison with LoRA to verify the efficiency and effectiveness of our approach. Unlike LoRA and other parameter-efficies fine-tuning (PEFT) methods, which insert additional adapter layers and train extra parameters, our method adjusts only small subset of existing shared neurons, This enables multilingual enhancement without any architectural modification or parameter growth, highlighting a distinct trade-off between targeted internal adaptation and parameter-efficient extension.
For fair comparison, we implement LoRA-basee fine-tuning on LLaMA3.2-D3 under similar parameter budget (rank 48). As shown Table [0] LoRA improves performance in certain MGSM cases (e.g.. German, Swahili) but shows limited generalization and even degradation on MMMLU. Moreover, LoRA requires longer training time (2.2 hours on 22H200 GPUs) compared our neuron-level method (1.5 hours), demonstrating that our approach achieves competitive multilingual gains with higher efficiency and simpler implementation.
Table 10: Comparison between our neuron-level fine-tuning and LoRA on LLaMA3.2-3B.
MGSM
MMMLU
Model
zh
de
fr
th
SW
A
zh
de
fr
th
SW
A
LLaMA3,2-3B
40.80
$7.20
42.40
35.20
30.80
45.20
47.10
49.00
40.60
34.10
-LORA
38.80
68.80
44.00
31.20
37.20
+2.72
44.40
45.79
47.74
39.58
32.50
-1.12
Overall, LoRA demonstrates partial improvements but lacks consistency across benchmarks and languages. In contrast, neuron-level approach achieves stable multilingual enhancement with lower computational overhead, highlighting its simplicity and interpretability as complementary direction to PEFT methods.
F
Cross-lingual Evaluation
To further explore the effect of language-specifie fine-tuning on multilingual generalization, we conduct cross-lingual evaluation experiments. While primary focus is understanding how fine-tuning with a single language corpus can enhance performance that language, it is equally important to assess how such adaptation influences the model's capabilities across other languages.
We fine-tune LLaMA-3.2 3.B-BB using corpora from individual languages and then evaluate its per- formance all target languages. Two metrics are examined: (1) accuracy improvement on the multilingual GSM8K (MGSM) benchmark, reflecting reasoning capability; and (2) change lan- guage perplexity (PPI.), reflecting language understanding and fluency.
Table II reports the results of fine-tuning on one language and testing across all others. "Target Lan- guage Acc" denotes the improvement on the language used for fine-tuning, while "Other Languages Acc (Avg)" shows the average accuracy change over the remaining four languages.
We also measure the change in perplexity (PPL) before and after fine-tuning as an intuitive indi- cator of the model's linguistic understanding. A decrease in PPL indicates improved fluency and comprehension in that language. The results are summarized in Table
These results demonstrate consistent trade-off: fine-tuning on single language improves both reasoning ability and linguistic understanding in that language, but often the expense of reduced performance on others. This observation suggests that language-specific adaptation repurposes part
31

---

Table 11:
Cross loossll evaluation on MGSM for LLaMA-3.2-3BB Fine-tuning on single language
improves reasoning in that language but moderately reduces performance in others.
Language Trained On
Target Language Ace
Other Languages (Avg)
Chinese (Zh)
+2.0
-166
French (Fr)
+3.2
2,4
German (De)
49.2
-3.2
Thai (Th)
-2.8
Swahili (Sw)
+8,8
-2.0
Table 12: Negative values indicate reduced perplexity (better language modeling).
Change in language perplexity (PPL) before and after fine-tuning on LLaMA-3.2-3B.
Language Trained On
Target Language PPL
Other Languages PPL (Ayg)
Chinese (Zh)
-3.15
+0.89
French (Fr)
-2.03
+0.77
German (De)
-2.76
+0.64
Thai (Th)
-0.41
+0,75
Swahili (Sw)
-12.40
+1.84
of the shared neuron subspace better align with the target language, consequently weakening cross-lingual generalization.
In summary, aunuagg--pecifit fine-tuning enhances targeted capabilities while moderately com- promising multilingual balance, implying that shared neurons serve as a critical mechanism for maintaining cross- ringull consistency.
G Broder Impacts
This work contributes to deeper understanding of how LLMs develop multilingual and abstract reasoning capabilities, which may help improve language equity in AI systems. By enhancing perfor- mance across diverse languages, our methods could benefit underrepresented linguistic communities. However, stronger multilingual models also carry risks, such as enabling more sophisticated misinfor- mation in multiple languages. We encourage responsible use and further research into safeguards for multilingual LLM deployment.
H
Limitations
While study provides compelling evidence for the emergence of abstract, language-agnostic thought LLMs and demonstrates the effectiveness of neuron-centric training strategies, several limitations remain: First, due resource constraints, model families and scales scales-hat has not been extended to the larger LLMs. Whether the observed patterns of neuron sharing and functional importance generalize such scales remains open question. Second, although our proposed training strategy yields consistent gains, the scope of our experiments remains limited by computational cost. We have not fully explored the upper bound of performance improvements that could be achieved with larger-scale or longer-term neuron-centric training. We leave these limitations future work.
analysis mhoysis-thouuh conducted across diverse
32


Reviews:
[{"review_id": "JNgnvbn0Mk", "reviewer": "Reviewer_PaXX", "summary": "This paper investigates the emergence of \"abstract thought\" in LLMs by analyzing how individual neurons respond to multilingual inputs. The authors identify and categorize language-related neurons into language-exclusive neurons (activated only for specific languages) and language-shared neurons (activated across multiple languages). Through extensive analysis of 20 open-source models across different families and generations, they discover that as LLMs evolve, shared neurons not only increase in proportion but also grow disproportionately in functional importance, eventually forming a compact set of language-agnostic neurons that support abstract reasoning beyond linguistic boundaries.\n\nThe key contributions include: (1) empirical evidence showing that shared neurons evolve from superficial overlaps in early models to critical language-agnostic components in advanced models, with their deactivation causing orders of magnitude greater performance degradation than exclusive neurons; (2) the introduction of a language-agnostic score metric that quantifies this evolution; and (3) a practical application of these insights through neuron-targeted training strategies that adapt based on a model's developmental stage.", "questions": "Suggestions:\n- Perform an analysis of the statistics of the different neurons (where they exist in the model, their activation statistics compared to random, etc).\n- Analyze multilinguality in terms of parameters, training tokens, and FLOPs instead of release date (deep research can probably accumulate these figures for you)\n- Compare your neuron training to a PeFT method with a similar computation budget.", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n- Very comprehensive comparison across model families and sizes.\n- This is one of only a few papers that leverages an observation about model internals to develop a practical method.\n\nWeaknesses\n- I don't understand the emphasis on release date as an interesting axis. Parameter count, total tokens trained, and FLOPs are all more interesting covariates.\n- In Figure 3 the stated pearson R ≈ 0.94 seems optimistic given visual scatter\n- There is no analysis in terms of what layer the neurons are in the model, or there average activation magnitude/sparsity. I could imagine Figure 4 is fully explained by random neurons simply having different statistics from those specifically filtered to have a large effect on the final activations.\n- Figure 2 is unnatural as a line plot and the grouping by size rather than generation obscures the more interesting comparison (how much shared neurons change as a function of size).\n- Inconsistencies with prior results: In \"On the Biology of a Large Language Model\", Lindsey et al., showed that larger models have a greater degree multilingual overlap (intuitive). In contrast, Figure 1, shows no trends with respect to model size (not intuitive). The paper is also missing a reference to this paper.C\n- I don't think the training method has much practical utility, compared to LoRA or other PEFT methods, as it requires an expensive preprocessing stage to find the relevant neurons. Comparing to LoRA is an easy baseline that is missing.", "was_revised": true, "final_justification": "The authors have given a detailed rebuttal, which have addressed several of my concerns. I am raising my score from 2 to 3.", "rebuttal": "# Response to Reviewer PaXX\n\nWe appreciate your comments. Below we provide the point-to-point responses to address your concerns and clarify the misunderstandings of our proposed method. If you have additional questions, we would be pleased to discuss them with you.\n\n> **Comment 1:  Confusion of axis.** — “  I don't understand the emphasis on release date as an interesting axis…”\n\nThank you for the thoughtful comment. Our primary focus is to investigate how the proportion and importance of shared neurons evolve with improving multilingual capabilities. In this context, the central axis of interest is a model’s *multilingual ability*. We introduced the release date as a proxy to provide an intuitive and chronological perspective on this evolution—particularly since, within the same model family (i.e., architecture and size), newer models tend to exhibit stronger multilingual performance due to improvements in training data quality, scale, or objective functions.\n\nWe fully agree that parameter count and FLOPs are important covariates. In fact, parameter count is annotated in Figures 3 and 5. Since FLOPs are approximately proportional to parameter count for models of the same architecture and training length, we chose not to repeat this information. Figure 1 is intended to be a simplified summary of Figures 3 and 5, designed to visually convey the high-level trend in a concise and accessible way—hence the use of release date as the x-axis.\n\nWe initially considered *training tokens* as a covariate, but found them often underreported or ambiguous (e.g., LLaMA series). In some cases (e.g., Qwen-1.5 series), models of different sizes share the same token count, making it hard to isolate the effect of training tokens from model size and release time. \n\n---\n> **Comment 2:  Confusion of R.** — “In Figure 3 the stated pearson R ≈ 0.94 seems optimistic given visual scatter …”\n\nIn Figure 3 and Figure 5, our primary goal is to examine the relationship between shared neuron ratio and multilingual ability **within the same model family and at comparable parameter scales**, as we believe that **models across different sizes and families are not directly comparable**. The reported Pearson and Spearman coefficients are the averages computed across these individual model groups.\n\nAlthough our focus is not on global analysis, we nonetheless report both the global correlation and the model familiy-level group-average correlation coefficients for completeness:\n\n- **Figure 3**\n\n  - Global Pearson: 0.427 (*p* = 0.060), Spearman: 0.409 (*p* = 0.073)\n  - Group-average Pearson: 0.591, Spearman: 0.494\n\n- **Figure 5**\n  - Global Pearson: 0.279 (*p* = 0.234), Spearman: 0.226 (*p* = 0.339)\n  - Group-average Pearson: 0.312, Spearman: 0.270\n\nWe can see that, **even though our primary focus is not on global or family-level average correlations, there still exist correlations**, particularly for the **shared neuron proportion**, which partially supports the general trend we aim to highlight.\n\n\n---\n> **Comment 3:  Lack of neuron analysis.** — “There is no analysis in terms of what layer the neurons are in the model …”\n\nThank you for this insightful suggestion. We agree that analyzing the distribution of language-specific neurons across layers and components can provide further insight. To address the question of where language-shared and language-exclusive neurons exist in the model, we conducted a detailed analysis on LLaMA-3.1-8B.\n\n**Layerwise Distribution**:\n\nWe calculate the proportion of **shared neurons** (common to all languages) and **exclusive neurons** (specific to one language) across layers:\n\n| Layer Range | Avg Shared (%) | Avg Exclusive (%) |\n| - | - | - |\n| 0–5         | 3.07           | 4.40              |\n| 6–25        | 3.11           | 2.89              |\n| 26–31       | 2.83           | 3.83              |\n\nThe **layerwise distribution** highlights distinct allocation patterns for shared and exclusive neurons. We observe that **exclusive neurons** are more concentrated in the **early** and **late** stages of the model, whereas their proportion is relatively lower in the **middle layers**. This suggests that the model tends to encode **language-specific features** near the input and output boundaries.\n\nIn contrast, **shared neurons** maintain a relatively stable presence across all layers. This indicates that shared neurons are more involved in processing **generalizable, cross-lingual representations**, and serve as a backbone for multilingual understanding throughout the network.\n\n**Component-level Distribution**:\n\n| Component | Shared (%) | Exclusive (%) (avg over languages) |\n| - | - | - |\n| QK        | 92.50      | 59.48                              |\n| VO        | 2.90       | 23.80                              |\n| FFN       | 4.59       | 16.71                              |\n\nShared neurons are mainly found in **QK**, aligning with general attention patterns. Exclusive neurons are relatively more prominent in **VO** and **FFN**, indicating their more important role in language-specific output transformations. This distribution indicates that **shared and exclusive neurons may serve different roles, although they both belong to language-specific neurons.**\n\n**Illustration of random neurons**:\n\nRegarding the random neurons in Figure 4, we carefully designed the selection process to make a fair comparison. Specifically, we sampled two sets of random neurons—each matching the total number of shared and exclusive neurons respectively. These random neurons were uniformly sampled across all layers and components of the model under the assumption of a homogeneous distribution. We then selected the group that had a higher influence on model performance and reported this in Figure 4. This same group was also used in the ablation experiments as “random neurons” reported in Table 1. It is indeed expected that the distribution and activation behavior of random neurons may differ from those of the language-specific neurons we detected. \n\n\n---\n> **Comment 4:  Size as axis.** — “Figure 2 is unnatural as a line plot and the grouping by size rather than generation…”\n\nThank you for your insightful suggestion. Figure 1 presents a simplified analysis of models within the same generation but of different sizes. Specifically, we observe that larger models tend to have a lower proportion and reduced importance of shared neurons. We attribute this to increased parameter specialization in larger models [1], which naturally leads to fewer shared neurons.\n\nAlthough our current work primarily focuses on the evolution of abstract thought over time rather than across model sizes, we appreciate this valuable observation and will incorporate it into the revised version of the paper.\n\n\n---\n> **Comment 5:  Inconsistent results.** — “Inconsistencies with prior results: In On the Biology of a Large Language Model…”\n\nThank you for pointing this out. We acknowledge the omission of the reference and will include it in the revised version.\n\nWe also wish to clarify a key methodological difference. Lindsey et al. analyze multilingual generalization from the **feature space**—focusing on hidden state reuse across languages. In contrast, our study operates in the **parameter space**, identifying **shared neurons** that are functionally important across languages and fixed in model structure.\n\nDue to these fundamental differences in approach—feature-level versus parameter-level analysis—the two works may lead to different empirical trends. For instance, our Figure 1 does not show a monotonic increase in shared neuron ratio with model size, which may reflect that **parameter sharing** across languages does not always scale proportionally with model capacity.\n\nWe thank the reviewer again for highlighting this, which allowed us to more clearly distinguish our methodological contribution.\n\n\n---\n> **Comment 6:  Additional baseline.** — “I don't think the training method has much practical utility, compared to LoRA or other PEFT methods…”\n\nWe clarify that our training-stage intervention is not proposed as a new fine-tuning method, but rather as a way to validate our core finding: multilingual capability in LLMs can be enhanced by understanding and adjusting neuron-level contributions.\n\nWhile LoRA and other PEFT methods are widely adopted, they introduce additional parameters and adapter layers. In contrast, our method fine-tunes a small subset of existing shared neurons, aiming to produce a **fully multilingual model** without any added parameters. This reflects a different trade-off—targeted internal adjustment vs. parameter-efficient extension.\n\nFollowing your suggestion, we conducted LoRA experiments on Llama3.2-3B under similar parameter budgets. LoRA improved performance in some MGSM cases (e.g., German, Swahili), but degraded it in others, especially on MMMLU. Moreover, LoRA increased training time (2.2h on 2×H200, rank=48) compared to our method (1.5h), highlighting our method’s simplicity and efficiency without modifying the model architecture.\n\n|                   | MGSM   |        |        |        |        |               | MMMLU  |        |        |        |        |               |\n| - | - | - | - | - | - | - | - | - | - | - | - | - |\n|                   | zh     | de     | fr     | th     | sw     | average delta | zh     | de     | fr     | th     | sw     | average delta |\n| Llama-3.2-3B      | 40.80% | 57.20% | 42.40% | 35.20% | 30.80% | -             | 45.20% | 47.10% | 49.00% | 40.60% | 34.10% | -             |\n| Llama-3.2-3B-LoRA | 38.80% | 68.80% | 44.00% | 31.20% | 37.20% | 2.72%         | 44.40% | 45.79% | 47.74% | 39.58% | 32.50% | -1.12%        |\n\n[1] The Rise of Parameter Specialization for Knowledge Storage in Large Language Models. Arxiv 2025"}, {"review_id": "To8C2P4uIC", "reviewer": "Reviewer_AGNp", "summary": "This paper studies neurons in multilingual large language models that are either language shared (they activate for all the languages tested) or language exclusive (they activate for some languages, but not for all of them). Within a family of LLMs, the authors show that the better the performance of the LLM, the greater the proportion of language shared neurons. They propose that these language shared neurons become language agnostic, forming the basis of \"abstract thoughts\" in the best models, which is supposed to explain the better performance of these LLMs in multilingual benchmarks. According to the authors, intervening on the specific types of neurons that are not developed enough in a certain model makes this model comparatively better at multilingual tasks.", "questions": "See questions and suggestions in the Strengths And Weaknesses section above.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "This work tackles the important question of whether LLMs think in English or in another lingua franca, possibly a non existing language, that might be called a language of thought.\n\nWhile the ambition of the paper is interesting, there are several shortcomings that strongly limit its impact.\nMany \"details\" are lacking in order to reproduce the experiments (the authors provide the code though). This goes beyond the question of reproducibility, as these details impact the way we understand the paper. First, the definition of language shared and language exclusive neurons depends on a threshold, notated $\\sigma$ in the paper (see Eqs. 1 and 2), and, as far as I can tell, the specific value that was used is not provided, and its impact on the results is not investigated. This seems important as many results, notably the proportion of language exclusive vs language shared neurons, hence also the language agnostic score, might depend on this value. It is far from obvious how the results depend on the specific value of $\\sigma$. See the claim \"Language-related neurons account for only a small proportion in LLMs.\" -- this typically should depend heavily on $\\sigma$.\nWhat is the value of $\\sigma$? How was it chosen? And what is its impact on the results? This seems crucial, why not discuss it in the paper?\nThe difference between language shared and language exclusive neurons does not seem to be as clear cut as presented in the paper, and might justly depend on the threshold that is used. Depending on the definition, we might also see effects of the distance between languages, which could be an interesting thing to investigate. Also, it might be interesting to look at the existence of neurons that are exclusive to only one language.\nLikewise, the difference between language shared and language agnostic is not so clear, and the empirical demonstration of their existence is not compelling.\n\nIt would be nice to have more details about the location of each type of neuron within the model, i.e. in which layer (first vs middle vs close to the outputs), or in which type of component (self-attention vs. feed-forward network). As we already know a lot about those, this knowledge might help the community having a more complete picture.\n\nHere are some additional comments, page by page.\n\np.3 \nThe main definition of the importance of a neuron entails how the ablation of a specific neuron yields a difference in the output of the LLM for a given language. But in practice, the output of the layer of this neuron is used \"as a proxy or component for the overall impact\" (see appendix A). Why not directly state the definition that is used in practice ? It is not obvious how the impact at a given layer impacts the final output, and it is an important question, as it might depend on the depth of the layer.\nAlso, the term \"layer\" seems to be used in a way different to common practice. Usually a layer is the combination of the self attention and feedforward network, but here each neuron is evaluated as the way it impacts the \"layer\" it belongs to, but it seems that the \"layer\" here refers to either the self-attention component or the feedfoward network. \nBeyond the minor question of terminology, this impacts here the way the importance of a given neuron is evaluated (see p.15).\n\np.4, Eq 5\nNormalizing by the number of neurons implicitly assumes that $\\Delta$PPL scales linearly with $|\\mathcal{N}|$. Is this true? \n\np.5, Fig. 2\nInterestingly, English is always the language with the lowest fraction of associated language-exclusive neurons -- that seems to go with the idea that use of English is actually shared among all languages, hence being the lingua franca, contrary to one of the main claims of the paper. \n\np.6, Fig. 3\nThe average correlation may not be very meaningful here, as there are very few points per family -- correlation based on three data points can easily be very high.\nThe \"overall trendline\" is plotted, making it looks like the overall correlation in the relationship between shared neuron ratio and multilingual ability is significant; but it looks quite fragile in fact, with a correlation quite low and possibly non significant. Could you report it, with both the value of the correlation and its p-value? \n(same remark for Fig. 5, p.7)\n\np.6\n\"we deactivate an equal number of randomly selected neurons\"\nAgain there is a lack of details that are actually important. How many neurons? What is the rationale this choice? How does it affect the results?\nSame thing for the experiment conducted p.8.\n\np.7 & p.8\n\"Shared neurons in early-stage models reflect superficial overlap without supporting higher-level cognition.\"\n\"the language-shared neurons in these models have evolved into language-agnostic neurons, responsible for abstract thought. They are already well-trained and offer limited room for further improvement.\"\nThose are strong and interesting takes, and although I understand the intuition, I do not think it is really supported by empirical evidence.\n\np.8\nMaybe I'm missing something here, but Llama3.1-8B is taken as a representative LLM with high language agnostic score, but in Fig. 5 we can see that its score is actually pretty low, almost like the LLM with low language agnostic score, and below the medium one. That makes this experiment difficult to follow.\n\nMinor comments\n\nReferences Liu et al 2024;2025 point to the same reference (arXiv:2403.10258); same for Schut et al. (2025a) and Schut et al. (2025b).\n\np.5 The values of the average correlations are not the same in the main text and in the corresponding figure (Fig. 3)\n\np.8 typo Lamma -> Llama\n\np.15 appendix A. Two notations for the same thing does not ease the reading (FFN(X) and Y_FFN)", "was_revised": true, "final_justification": "I appreciate all the clarifications from the discussion, especially regarding the methods, as well as the new analyses, notably on the location of the different types of neurons. The notion of low vs high agnostic score is still pretty fuzzy to me, due to its relativity between family, and the distinction between language-shared and language agnostic neurons is also still not very convincing, and I find the results and discussion insufficient on this aspect, given its importance in the narrative of the paper. But overall, I think the paper is improved, and I raise my rating from 3 to 4.", "rebuttal": "We thank the reviewer for the thorough and valuable feedback. To address your concerns, we present the point-to-point responses as follows.\n\n> **Comment 1:  Illustration of threshold.** \n\nThank you for the question. The threshold σ is not a fixed scalar but a dynamic, query-level mechanism. For each query in a language, we select the top 1% of neurons by importance score, then define language-specific neurons as those consistently appearing across all queries for that language. This 1% threshold is validated via a sanity check: deactivating these neurons causes a significant performance drop (e.g., 100× increase in perplexity), unlike deactivating the same number of random neurons. If random deactivation also hurts performance, we lower the threshold. We will clarify this in the revised version, along with supporting empirical results.\n\n---\n> **Comment 2:  Lack of neuron analysis.** \n\nThank you for the suggestion. We analyzed the distribution of language-shared and language-exclusive neurons in **LLaMA-3.1-8B** to better understand where they reside in the model.\n\n### **Layerwise Distribution**\n\nWe computed the proportion of **shared** (across all languages) and **exclusive** (language-specific) neurons per layer group:\n\n| Layer Range | Shared (%) | Exclusive (%) |\n| - | - | - |\n| Early (0–7) | 3.07 | 4.40 |\n| Middle (8–23) | 3.11 | 2.89 |\n| Late (24–31) | 2.83 | 3.83 |\n\n**Exclusive neurons** are more concentrated in **early** and **late** layers, suggesting language-specific processing near input/output. **Shared neurons** are relatively stable across layers, supporting generalizable cross-lingual features.\n\n### **Component-Level Distribution**\n\nWe also analyzed neuron distribution across model components:\n\n| Component | Shared (%) | Exclusive (%) (avg over languages) |\n| - | - | - |\n| QK      | 92.50      | 59.48   |\n| VO  | 2.90       | 23.80   |\n| FFN   | 4.59       | 16.71     |\n\n**Shared neurons** are concentrated in **Q** and **K**, aligning with general attention mechanisms. **Exclusive neurons** are more spread across **V**, **O**, and **FFN**, indicating roles in language-specific transformation and output.\n\nThese findings are consistent with prior work and highlight distinct functional roles for shared vs. exclusive neurons.\n\n---\n> **Comment 3:  Lack of illustration in detection method.** \n\n**“It is not obvious how the impact at a given layer impacts the final output”**\n\nWe adopt an intuitive definition of neuron importance, measuring it by the change in the layer’s embedding after deactivating a neuron. This layer-wise approach is widely used in prior work analyzing neurons in large language models [1][2][3]. Importantly, we do not assess neuron influence based on the final model output, as influence tends to accumulate across layers. This accumulation can lead to an overestimation of the importance of neurons in earlier layers, which motivates our layer-wise evaluation strategy.\n\n**“Also, the term \"layer\" seems to be used in a way different to common practice.”**\n\nWe will revise the terminology to eliminate ambiguity in our use of the word *layer*. Specifically, we will use *structure* to refer to components such as self-attention or Q/K/V, and reserve *layer* exclusively for the architectural layers in LLMs.\n\n---\n> **Comment 4:  The validity of the assumption.** — “Normalizing by the number of neurons |N|.”\n\nWe acknowledge that our formulation of the language-agnostic score in Eq. 5 involves an implicit linearity assumption—that the change in perplexity scales proportionally with the number of ablated neurons ∣N∣.\n\nOur motivation for this normalization is to **approximately remove the effect of neuron count**, in order to make a fairer comparison between shared and exclusive neurons. While this introduces a simplifying linear assumption, we believe it is a **reasonable approximation** in our setting for two reasons:\n\n1. As shown in Figure 2, the total number of shared and exclusive neurons is **not drastically different** in scale across models and languages.\n2. In contrast, the observed changes in perplexity caused by ablation are often **orders of magnitude larger**, suggesting that the **magnitude of impact is primarily driven by functional importance**, not merely neuron count.\n\nThus, we consider this normalized score a practical and relatively fair metric for comparing the contribution of different types of neurons. That said, we acknowledge its limitations and will clarify this assumption in the revised version.\n\n---\n> **Comment 5:  Lowest exclusive neuron ratio of English.** \n\nThank you for the observation. It's true that English often shows the lowest fraction of language-exclusive neurons, but this doesn't contradict our main claim.\n\nWe argue that the model doesn’t \"think\" in English, but rather develops a **shared, high-dimensional semantic space**. This is supported by the **growing proportion and importance of shared neurons**, indicating increasingly abstract, language-agnostic representations.\n\nOur **GSM8K experiments** further support this: fine-tuning on one language improves reasoning in that language, but slightly reduces performance in others.\n\n**GSM8K Accuracy:**\n\n| Trained Language | Target Acc | Other Acc (Avg) |\n| --- | --- | --- |\n| Zh | +2.0 | -1.6 |\n| Fr | +3.2 | -2.4 |\n| De | +9.2 | -3.2 |\n| Th | +5.2 | -2.8 |\n| Sw | +8.8 | -2.0 |\n\nThis trade-off suggests the model reallocates representational resources (e.g., exclusive neurons) during fine-tuning, reinforcing the idea that reasoning occurs in a **shared semantic space**, not in English.\n\n---\n> **Comment 6:  Report of global correlation.** \n\nIn **Figure 3** and **Figure 5**, our primary goal is to examine the relationship between shared neuron ratio and multilingual ability **within the same model family and at comparable parameter scales**, where models are directly comparable. We agree that **correlation coefficients may not be well-suited for capturing the trends we aim to highlight in this context**, especially given the small number of data points per group.\n\nAlthough our focus is not on global analysis, we nonetheless report both the global correlation and the model familiy-level group-average correlation coefficients for completeness:\n\n- **Figure 3**:\n    - Global Pearson: 0.427 (*p* = 0.060), Spearman: 0.409 (*p* = 0.073)\n    - Group-average Pearson: 0.591, Spearman: 0.494\n- **Figure 5**:\n    - Global Pearson: 0.279 (*p* = 0.234), Spearman: 0.226 (*p* = 0.339)\n    - Group-average Pearson: 0.312, Spearman: 0.270\n\nWe can see that, **even though our primary focus is not on global or family-level average correlations**—as we believe models across different scales and families are not directly comparable—**there still exist moderate correlations**, particularly for the **shared neuron proportion**, which partially supports the general trend we aim to highlight.\n\n---\n> **Comment 7:  Lack of random neuron details.** \n\nFor the random neuron selection, we chose two sets of neurons, each with the **same total number** as the **shared neurons** and **exclusive neurons**, respectively. We assumed that these random neurons are **uniformly distributed across all layers and components** of the LLM, and we performed random sampling with various random seeds. The results reported in **Figure 4** reflect the set of random neurons that had a **greater impact** on the LLM's performance. This set of random neurons was then used in the ablation experiments presented in **Table 1**.\n\nIn general, we found that **comparable numbers of random neurons** had minimal impact on the model's ability. We tried several different random seeds, and the results were quite stable, showing that the randomness did not significantly alter the overall conclusions.\n\n---\n> **Comment 8:  Illustration of model selection.** \n\nTo clarify, our aim was to train models within the same series to ensure comparability. However, we argue that models of significantly different sizes might not be directly comparable, especially when considering the impact of scaling. Specifically, Llama3.2-1B and Llama3.2-3B are relatively similar in size and exhibit low and medium language-agnostic scores, respectively. In contrast, Llama3.1-8B, which is closer in size to the Llama 7B models, actually demonstrates a high language-agnostic score within that model family.\n\nTo provide more generalizable insights, we also trained **Gemma2-9B**, a model that, at the global level, clearly qualifies as having a high language-agnostic score. The results are as follows: \n\n|  | Zh | Fr | De | Th | Sw | $\\Delta_{Avg}$ |\n| --- | --- | --- | --- | --- | --- | --- |\n| None | 58.4 | 58.0 | 58.8 | 57.2 | 51.2 | - |\n| Shared | 56.8 | 57.6 | 58.8 | 54.8 | 48.4 | -1.4 |\n| Exclusive | 61.6 | 60.8 | 62.4 | 58.4 | 55.6 | 3.0 |\n| Random | 56.0 | 57.6 | 57.2 | 56.0 | 50.8 | -1.2 |\n\nThis additional explanation clarifies the model selection process. Moreover, we provide empirical support from our GSM8K experiments. When fine-tuned on a specific language, the model’s reasoning ability in that language improves, but often at the cost of performance on others:\n\n**GSM8K Accuracy (Reasoning Ability):**\n\n| Language Trained On | Target Language Acc | Other Languages Acc (Avg) |\n| --- | --- | --- |\n| Zh | +2.0 | -1.6 |\n| Fr | +3.2 | -2.4 |\n| De | +9.2 | -3.2 |\n| Th | +5.2 | -2.8 |\n| Sw | +8.8 | -2.0 |\n\nThis performance trade-off suggests that the model’s reasoning capacity is distributed and not inherently anchored in English. Training on a single language reallocates representational resources (e.g., exclusive neurons) to that language, which slightly reduces generalization. Together, these findings reinforce our claim that the model reasons in a shared semantic space beyond English.\n\n[1] Language-Specific Neurons. ACL 2024\n\n[2] Investigating Pattern Neurons in Urban Time Series Forecasting. ICLR 2025\n\n[3] Finding Safety Neurons in Large Language Models. Arxiv 2024"}, {"review_id": "6EVLFbG67m", "reviewer": "Reviewer_oHcr", "summary": "This paper investigates the extent to which language-agnostic processing happens in multilingual models, then relating this to model performance. On experiments spanning several model series, the authors isolate language-specific and language-agnostic neurons. They show that a high proportion and importance of language-agnostic neurons correlates to better LLM performance. Finally, they selectively finetune language-specific and -agnostic neurons on several multilingual datasets, showing that doing so improves performance more than  training random neurons.", "questions": "See weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n1. Very well-written, easy to follow, and easy to see what the core contributions are.\n2. IMO the findings are significant, with an implication that better models will abstract away knowledge that is not language-specific.\n\n## Weaknesses\nThough I found the paper easy to follow, I believe the paper needs further experiments to be accepted. In particular, current experiments appear to be run on a single random seed, and there are several methodological details missing. If my comments are addressed, I will raise my score to a 4 or 5.\n\n** Note: I am not well-versed in the multilingual LLMs literature and so can't speak to the originality of this work.** \n\n### Major (impacted score)\n1. It appears all experiments are run on one random seed or data split. Results in Fig 2-4, Table 1 should be reported across random seeds or random data splits with errors. \n2. What was the $\\sigma$ chosen in Eq. 1? How was it chosen? I'm missing t-tests (corrected for multiple hypothesis testing) to show that the difference is statistically significantly greater than $\\sigma$.\n3. Does finetuning on language-specific neurons for a single language degrade the performance on other languages? \n\n### Minor\n1. Missing citations on language localizer paradigms, see e.g. AlKhamissi et al., NAACL 2025.", "was_revised": true, "final_justification": "The authors answered all of my questions satisfactorily and additional experiments to strengthen their claim.", "rebuttal": "We sincerely thank you for your time and valuable comments. To address your concerns, we present the point-to-point responses as follows.  \n\n> **Comment 1:  Different random seeds.** — “It appears all experiments are run on one random seed or data split…”\n> \n\nThank you for your valuable suggestion. In response, we **vary the random seed** and **perform three independent repetitions** of each experiment to validate the stability of our results.\n\n- For **Table 1** and **Figure 4** (random neuron experiments), we **change the random seed** used for selecting random neurons.\n- For **Figures 2-5** and **Table 1** (shared and exclusive neurons), we **alter the data split** used for selecting language-specific neurons.\n- Additionally, for **Table 1**, we **modify both the training and inference random seeds** to ensure consistency and stability in the results.\n\nThe results from these repeated experiments are as follows.\n\nFor Figure 2:\n\nDue to rebuttal formatting constraints, we can only present the results in **markdown table form**. To keep the presentation concise and readable, we report the **mean and standard deviation** of the neuron ratios for each neuron type, **aggregated over all models within each family**. \n\n| Neuron Type | Qwen (%) | LLaMA (%) | Gemma (%) |\n| --- | --- | --- | --- |\n| Shared | 0.23 ± 0.02 | 0.34 ± 0.03 | 0.42 ± 0.03 |\n| En | 0.11 ± 0.02 | 0.14 ± 0.03 | 0.13 ± 0.01 |\n| Zh | 0.13 ± 0.01 | 0.17 ± 0.02 | 0.14 ± 0.01 |\n| Th | 0.21 ± 0.03 | 0.19 ± 0.02 | 0.17 ± 0.01 |\n| Fr | 0.15 ± 0.04 | 0.16 ± 0.03 | 0.16 ± 0.02 |\n\nFor Figure 3:\n\n| Model | Shared Neuron Ratio |\n| --- | --- |\n| Llama-1-7B | 0.6737 ± 0.0967 |\n| Llama-2-7B | 2.3847 ± 0.1231 |\n| Llama-3-8B | 2.6200 ± 0.1172 |\n| Llama-3.1-8B | 2.4833 ± 0.2437 |\n| Llama-3.2-1B | 1.4092 ± 0.1275 |\n| Llama-3.2-3B | 2.0881 ± 0.1084 |\n| Qwen1.5-0.5B | 1.1854 ± 0.2320 |\n| Qwen1.5-1.8B | 0.9754 ± 0.2005 |\n| Qwen1.5-4B | 0.8370 ± 0.1300 |\n| Qwen1.5-7B | 0.3808 ± 0.1289 |\n| Qwen2-0.5B | 1.8239 ± 0.1030 |\n| Qwen2-1.5B | 1.5486 ± 0.0924 |\n| Qwen2-7B | 1.6780 ± 0.1324 |\n| Qwen2.5-0.5B | 1.9457 ± 0.2137 |\n| Qwen2.5-1.5B | 1.5101 ± 0.1321 |\n| Qwen2.5-3B | 1.8499 ± 0.0072 |\n| Qwen2.5-7B | 1.8599 ± 0.1275 |\n| Gemma-2-9B | 3.2607 ± 0.0627 |\n| Gemma-3-4B-pt | 2.7653 ± 0.1628 |\n| Gemma-7B | 2.4826 ± 0.0549 |\n\n| Metric | Mean ± Std |\n| --- | --- |\n| Average Pearson | 0.9026 ± 0.0139 |\n| Average Spearman | 0.8334 ± 0.0067 |\n\nFor Figure 4:\n\n| Model | PPL Change of Random Neuron |\n| --- | --- |\n| Llama-1-7b | 0.001452 ± 0.000149 |\n| Llama-2-7b-hf | 0.000105 ± 0.000023 |\n| Llama-3-8B | 0.001149 ± 0.000071 |\n| Llama-3.1-8B | 0.000778 ± 0.000295 |\n| Llama-3.2-1B | 0.009100 ± 0.000055 |\n| Llama-3.2-3B | 0.001805 ± 0.000035 |\n| Qwen1.5-0.5B | 0.014462 ± 0.000274 |\n| Qwen1.5-1.8B | 0.001326 ± 0.000077 |\n| Qwen1.5-4B | 0.004029 ± 0.000063 |\n| Qwen1.5-7B | 0.000389 ± 0.000085 |\n| Qwen2-0.5B | 0.007883 ± 0.000063 |\n| Qwen2-1.5B | 0.001214 ± 0.000245 |\n| Qwen2-7B | 0.000282 ± 0.000029 |\n| Qwen2.5-0.5B | 0.041696 ± 0.000192 |\n| Qwen2.5-1.5B | 0.003737 ± 0.000260 |\n| Qwen2.5-3B | 0.001117 ± 0.000122 |\n| Qwen2.5-7B | 0.000117 ± 0.000131 |\n| gemma-2-9b | 0.003656 ± 0.000246 |\n| gemma-3-4b-pt | 0.006731 ± 0.000136 |\n| gemma-7b | 0.000011 ± 0.000176 |\n\nFor Figure 5:\n\n| Model | Language Agnostic Score |\n| --- | --- |\n| Llama-1-7B | 0.3816 ± 0.1405 |\n| Llama-2-7B | 6.0818 ± 0.2390 |\n| Llama-3-8B | 1.0102 ± 0.2854 |\n| Llama-3.1-8B | 3.7728 ± 0.5139 |\n| Llama-3.2-1B | 1.7169 ± 0.6402 |\n| Llama-3.2-3B | 4.9557 ± 0.4094 |\n| Qwen1.5-0.5B | 2.4951 ± 0.6952 |\n| Qwen1.5-1.8B | 0.678 ± 0.2605 |\n| Qwen1.5-4B | 4.9751 ± 1.1123 |\n| Qwen1.5-7B | 0.7296 ± 0.1484 |\n| Qwen2-0.5B | 7.2774 ± 0.3304 |\n| Qwen2-1.5B | 5.1656 ± 0.4643 |\n| Qwen2-7B | 4.1291 ± 0.4153 |\n| Qwen2.5-0.5B | 9.4350 ± 0.2096 |\n| Qwen2.5-1.5B | 9.7219 ± 0.8226 |\n| Qwen2.5-3B | 7.1878 ± 0.6007 |\n| Qwen2.5-7B | 5.7474 ± 1.1062 |\n| Gemma-2-9B | 7.8019 ± 0.5050 |\n| Gemma-3-4B | 6.5917 ± 0.3133 |\n| Gemma-7B | 6.2285 ± 0.6756 |\n\n| Metric | Mean ± Std |\n| --- | --- |\n| Average Pearson | 0.8120 ± 0.0208 |\n| Average Spearman | 0.8667 ± 0.0331 |\n\nFor Table 1, we conduct multiple experiments on Llama-3.2-1B due to limited time:\n\nMGSM\n\n| Method | Zh | Fr | De | Th | Sw |\n| --- | --- | --- | --- | --- | --- |\n| Exclusive | 27.7 ± 0.2 | 29.7 ± 0.2 | 34.5 ± 0.2 | 23.3 ± 0.2 | 30.3 ± 0.2 |\n| Random | 26.8 ± 0.2 | 26.4 ± 0.2 | 29.7 ± 0.2 | 21.1 ± 0.2 | 26.3 ± 0.2 |\n| Shared | 29.9 ± 0.2 | 30.1 ± 0.2 | 30.8 ± 0.4 | 22.0 ± 0.4 | 26.3 ± 0.2 |\n\n\nMMMLU\n\n| Method | Zh | Fr | De | Th | Sw |\n| --- | --- | --- | --- | --- | --- |\n| Exclusive | 29.0 ± 0.2 | 28.0 ± 0.0 | 29.2 ± 0.1 | 28.4 ± 0.1 | 26.8 ± 0.1 |\n| Random | 28.8 ± 0.1 | 28.4 ± 0.1 | 29.2 ± 0.1 | 28.4 ± 0.1 | 26.8 ± 0.0 |\n| Shared | 29.1 ± 0.1 | 28.8 ± 0.1 | 29.5 ± 0.0 | 29.5 ± 0.1 | 26.8 ± 0.1 |\n\nWe believe that this additional analysis strengthens the reliability of our findings.\n\n---\n\n> **Comment 2:  Illustration of threshold.** — “ What was the threshold chosen in Eq. 1? …”\n> \n\nThank you for raising this important question. The threshold σ is not a fixed scalar, but a dynamic selection mechanism applied at the **query level**. Specifically, as shown in our released code, we select the top 1% of neurons for each query in a given language based on their computed importance scores. Then, for each language, we define its language-specific neurons as the intersection of these top neurons across all queries from that language.\n\nThe choice of the 1% threshold is motivated by a **sanity check against random baselines**. For each language, we compare the effect of deactivating the selected language-specific neurons versus deactivating the same number of randomly selected neurons. When the deactivation of identified neurons causes a **substantial degradation in performance**—e.g., more than 100× increase in perplexity—while random neurons have little to no effect, this suggests that the selected neurons are indeed functionally meaningful and the threshold is appropriately calibrated.\n\nIf, on the other hand, removing the same number of random neurons also leads to a noticeable performance drop, this would indicate that the threshold is too large and not selective enough—prompting us to reduce it accordingly.\n\nWe will include this clarification and the supporting empirical analysis in the revised version to improve the transparency and reproducibility of our method.\n\n---\n\n> **Comment 3:  Question about cross-languages evaluation.** — “Does finetuning on language-specific neurons …”\n> \n\nThank you for this excellent question. While our primary focus is on exploring how fine-tuning with a single language's corpus can enhance the LLM's capabilities in that language, we agree that examining the impact of fine-tuning on other languages provides valuable insights.\n\nTo investigate this, we fine-tuned the model using a language-specific corpus and then tested the performance on all languages. The results of Llama-3.2-3B on GSM8K are as follows:\n\n| Language Trained On | Target Language Acc | Other Languages Acc (Avg) |\n| --- | --- | --- |\n| Zh | + 2.0 | - 1.6 |\n| Fr | + 3.2 | - 2.4 |\n| De | + 9.2 | - 3.2 |\n| Th | + 5.2 | - 2.8 |\n| Sw | + 8.8 | - 2.0 |\n\nAdditionally, we observed the change in language perplexity before and after fine-tuning, which serves as an intuitive metric for assessing the model’s understanding of a given language. The results on LLaMA-3.2-3B are as follows:\n\n| Language Trained On | Target Language PPL   | Other Languages PPL  (Avg) |\n| --- | --- | --- |\n| Zh | - 3.15 | + 0.89 |\n| Fr | - 2.03 | + 0.77 |\n| De | - 2.76 | + 0.64 |\n| Th | - 0.41 | + 0.75 |\n| Sw | - 12.4 | + 1.84 |\n\nOur findings demonstrate that fine-tuning on a single language improves the model’s understanding ability and reasoning performance in that language, but often comes at a moderate cost to the performance on other languages. This indicates that while language-specific fine-tuning enhances targeted capabilities, it may compromise multilingual generalization. We hypothesize that this is because the adaptation process repurposes some shared neurons to better serve the target language, thereby reducing their effectiveness for others."}, {"review_id": "Q3TTGXA2mQ", "reviewer": "Reviewer_ysL3", "summary": "This work investigates language-shared and language-specific neurons in multilingual LLMs. The authors identify neurons whose removal has a significant impact on the resulting embeddings. For some of these neurons, the impact is apparent for all languages, while for other neurons, the effect is not specific to a (subset of) language(s). The authors define a \"language agnostic score\" score that quantifies the impact (in terms of changes in perplexity) of language-shared neurons.\n\nThe analyses, performed with several families of models (qwen , llama, gemma), reveal that the proportions of language-selective and language-shared neurons are small (around 0.3%).  Morevover, the relative proportions of shared vs selective neurons increases with the model generation and performance on a multilingual benchmark. The language agnostic score, that is, the importance of the shared-neurons, also increases with multilingual ability. In recent models, deactivating shared neurons increases perplexity more than deactivating language-exlusive neurons (Fig.4)  \n\nFinally, the authors further trained language--shared and language-exclusive neurons in models of the Llama family, and assess them on the Multi-task Language Understanding (MMMLU)  and Multilingual Grade School Math (MGSM) benchmarks. They report that the model with the highest agnostic score benefits more from training language-specific neurons, while the model with the  lowest agnostic scores benefits from training both share and exclusive neurons.  These leads them to claim that the language-shared neurons support abstract thinking.", "questions": "- For a given release date,  the proportion of shared neurons and their importance decrease with model size  in the qwen 2.5 family while the multilingual ability increases with model size.  Any idea why? Would the analysis of section 4 performed on the llama-3.2 family apply to qwen-2.5?\n\n-  the number of language sensitive neurons depends on the threshold \\sigma  (eq.1) but I did not see in the paper how this theshold was set/determined (?)\n\n- The proposed definition of \"language-exclusive neurons\" (eq.3) includes neurons that may respond to several languages, but not all of them (which belong to N_shared). Another possibility would have been to \"language-selective neurons as neuron that respond to only to one language\". It could be nice to have the distribution of count of neurons responding  to 1, 2, 3, ...n language.\n\n-   The conclusion/discussion is somewhat short.  As the authors claim that their study  shows the emergence of \"abstract thought\", and not solely of language agnostic neurons, it would be nice to reiterate the arguments spread in the paper and discuss their potential limitations.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "The paper reads quite well. The authors' convincingly demonstrate that the proportion and importance of *language-agnostic* neurons increase with the  LLMs  generation (although  not systematically with model size or multilingual ability within a generation).\n\nThe approach of focusing the training on subsets of neurons (language-speicfic or language agnostic one) is particularly original.  \n\nA more thorough investigation of \"abstract thought\" with more benchmarks dissociating between linguistic and reasoning abilities, could have strengthened the conclusion that this is evidence for the emergence of \"abstract thought\".", "was_revised": true, "final_justification": "The first version of the paper was quite good paper, and the clarifications, especially the crucial one about the determination of the sigma threshold, are convincing.  I therefore raised the scores to 4.", "rebuttal": "We thank the reviewer for the thorough and valuable feedback. To address your concerns, we present the point-to-point responses as follows.\n\n> **Comment 1:  More benchmark.** — “ A more thorough investigation of \"abstract thought\" with more benchmarks …”\n> \n\nThank you for the insightful suggestion. We fully agree that distinguishing between linguistic and reasoning abilities is essential for substantiating our hypothesis regarding the emergence of abstract thought.\n\nTo address this, we evaluated the model from both perspectives:\n\n1. **Linguistic Benchmark:** We used **perplexity** as a direct metric to assess the model’s language understanding.\n2. **Reasoning Benchmark:** We measured performance on **GSM8K**, a reasoning-intensive task.\n\nSpecifically, we fine-tuned LLaMA-3.2-3B on single-language corpora and tested the resulting models across all five target languages. The results are summarized below:\n\n**GSM8K Accuracy (Reasoning Ability):**\n\n| Language Trained On | Target Language Acc | Other Languages Acc (Avg) |\n| --- | --- | --- |\n| Zh | +2.0 | -1.6 |\n| Fr | +3.2 | -2.4 |\n| De | +9.2 | -3.2 |\n| Th | +5.2 | -2.8 |\n| Sw | +8.8 | -2.0 |\n\n**Language Perplexity (Linguistic Understanding):**\n\n| Language Trained On | Target Language PPL  | Other Languages PPL (Avg) |\n| --- | --- | --- |\n| Zh | -3.15 | +0.89 |\n| Fr | -2.03 | +0.77 |\n| De | -2.76 | +0.64 |\n| Th | -0.41 | +0.75 |\n| Sw | -12.4 | +1.84 |\n\nThese results clearly dissociate the effects on **linguistic competence** (as measured by perplexity) and **reasoning competence** (as measured by GSM8K accuracy). Across both dimensions, we observe that fine-tuning in one language significantly improves performance in that language, while moderately impairing it in others.\n\nRather than indicating that the model \"thinks in English,\" our findings support the opposite hypothesis:\n\nThe model does not rely on English as a latent reasoning space, but instead gradually develops a higher-dimensional, language-agnostic reasoning space.\n\nThis is further corroborated by our neuron analysis, which reveals a growing proportion of **shared neurons** that dominate in importance, indicating that abstract reasoning emerges independently of any single language.\n\n---\n\n> **Comment 2:  Illustration of phenomenon.** — “For a given release date, the proportion of shared neurons and their importance decrease with model size …”\n> \n\nThank you for your insightful suggestion. Figure 1 presents a simplified analysis of models within the same generation but of different sizes. Specifically, we observe that larger models tend to have a lower proportion and reduced importance of shared neurons. We attribute this to increased parameter specialization in larger models [1], which naturally leads to fewer shared neurons.\n\nAlthough our current work primarily focuses on the evolution of abstract thought over time rather than across model sizes, we appreciate this valuable observation and will incorporate it into the revised version of the paper.\n\nReferring to whether the analysis of section 4 applies to the Qwen-2.5 family, with the limited rebuttal time we were able to show that Qwen-2.5-1.5B indeed exhibits a similar trend to its counterpart in the Llama-3.2 family, with results shown as follows.\n\n|  | MGSM_zh | MGSM_de | MGSM_fr | MGSM_th | MGSM_sw | average_delta | MMMLU_zh | MMMLU_de | MMMLU_fr | MMMLU_th | MMMLU_sw | average_delta |\n| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |\n| Qwen-2.5-1.5B | 63.60% | 57.2% | 61.20% | 50.80% | 28.00% |  | 53.95% | 48.47% | 50.96% | 44% | 30.49% |  |\n| Qwen-2.5-1.5B-exclusive | 65.20% | 57.60% | 61.60% | 52% | 29.60% | 1.04% | 53.81% | 48.65% | 51.29% | 44.49% | 31.57% | 0.39% |\n| Qwen-2.5-1.5B-shared | 63.2% | 56.80% | 62% | 49.20% | 31.20% | 0.32% | 53.72% | 48.38% | 51.30% | 44.42% | 31.48% | 0.29% |\n| Qwen-2.5-1.5B-random | 62.80% | 54% | 60.4% | 46.80% | 27.6% | -2.88% | 53.71% | 48.62% | 51.09% | 44.17% | 31.36% | -0.17% |\n\n---\n\n> **Comment 3:  Illustration of threshold.** — “The number of language sensitive neurons depends on the threshold…”\n> \n\nThank you for raising this important question. The threshold σ is not a fixed scalar, but a dynamic selection mechanism applied at the **query level**. Specifically, as shown in our released code, we select the top 1% of neurons for each query in a given language based on their computed importance scores. Then, for each language, we define its language-specific neurons as the intersection of these top neurons across all queries from that language.\n\nThe choice of the 1% threshold is motivated by a **sanity check against random baselines**. For each language, we compare the effect of deactivating the selected language-specific neurons versus deactivating the same number of randomly selected neurons. When the deactivation of identified neurons causes a **substantial degradation in performance**—e.g., more than 100× increase in perplexity—while random neurons have little to no effect, this suggests that the selected neurons are indeed functionally meaningful and the threshold is appropriately calibrated.\n\nIf, on the other hand, removing the same number of random neurons also leads to a noticeable performance drop, this would indicate that the threshold is too large and not selective enough—prompting us to reduce it accordingly.\n\nWe will include this clarification and the supporting empirical analysis in the revised version to improve the transparency and reproducibility of our method.\n\n---\n\n> **Comment 4:  Distribution of possible unique neurons.** — “Another possibility would have been to language-selective neurons as neuron that respond to only to one language…”\n> \n\nThank you for the thoughtful suggestion. While our definition of \"language-exclusive neurons\" allows for neurons that are shared across a subset of languages (i.e., not strictly one), we agree that analyzing neurons activated by exactly 1, 2, ..., *n* languages can provide additional insight.\n\nTo complement this, we present the distribution of *strictly unique neurons*—neurons that respond exclusively to only one language. As shown below, these strictly unique neurons account for only a small percentage of the total neurons across all languages:\n\n| Language | Strictly Unique Neurons (%) |\n| --- | --- |\n| en | 0.02 |\n| zh | 0.03 |\n| th | 0.06 |\n| fr | 0.03 |\n| de | 0.02 |\n| sw | 0.07 |\n\nThis small percentage suggests that the model primarily uses shared neurons for general language understanding. Interestingly, higher values for Swahili and Thai—both lower-resource languages—indicate more allocation to language-specific features, likely to compensate for limited training data.\n\nWe also examined the **layer-wise distribution** of strictly unique neurons. To avoid clutter, we grouped the 32 transformer layers into three stages: *early (Layer 0–7)*, *middle (Layer 8–23)*, and *late (Layer 24–31)*. The averaged proportions of different neuron types across these groups are summarized below:\n\n| Layer Group | Avg. Unique (%) | Avg Shared (%) | Avg Exclusive (%) |\n| --- | --- | --- | --- |\n| Early (0–7) | 3.69 | 3.07 | 4.40 |\n| Middle (8–23) | 2.28 | 3.11 | 2.89 |\n| Late (24–31) | 4.62 | 2.83 | 3.83 |\n\nThese results reveal a clear **functional asymmetry** across the model. The **early and late layers** contain more **unique and exclusive neurons**, suggesting that **language-specific processing** occurs at the **input and output stages**. This aligns with prior findings that early layers capture syntactic patterns, while late layers focus on output generation. In contrast, the **middle layers** are dominated by **shared neurons** and show the lowest uniqueness, indicating a **cross-lingual abstraction core** where language-independent reasoning takes place.\n\nFinally, we analyzed the **component-wise** breakdown of strictly unique neurons:\n\n| Component | **Unique (%)** | **Shared (%)** | **Exclusive (%)** |\n| --------- | -------------- | -------------- | ----------------- |\n| QK        | 18.40          | 92.50          | 59.48             |\n| VO        | 50.96          | 2.90           | 23.80             |\n| FFN       | 30.64          | 4.59           | 16.71             |\n\nShared neurons cluster in Q and K, supporting their role in cross-lingual alignment. Exclusive neurons are more concentrated in V, O, and FFN, indicating that language-specific specialization primarily occurs during semantic transformation and output projection. Strictly unique neurons further amplify this pattern, highlighting the most language-specific activations.\n\nThese findings further supports our broader claim: **shared neurons are primarily responsible for cross-lingual generalization**, enabling the model to capture abstract, language-agnostic semantics that are transferable across different languages. **Exclusive neurons**, on the other hand, are responsible for encoding language-specific features, which are crucial for correctly interpreting and generating content unique to a given language—primarily located in the early and late layers. **Unique neurons**, by definition, represent a stricter subset of exclusive neurons—those that respond to *only* one language—which further strengthens the language-specialized nature of the exclusive category. \n\n---\n\n> **Comment 5:  Conclusion is too short.** — “The conclusion/discussion is somewhat short.…”\n> \n\nThank you for the helpful suggestion. We will revise the conclusion to better summarize the main findings and more clearly support our claim regarding the emergence of *abstract thought*. We will also include a brief discussion of the study’s limitations to provide a more balanced and comprehensive interpretation.\n\n[1] The Rise of Parameter Specialization for Knowledge Storage in Large Language Models. Arxiv 2025"}]

Author rebuttal:

Meta-review: This papers investigated an important question on how LLMs think by analyzing how individual neurons respond to multi-lingual inputs. Based on the findings and observation of language-specific and language-agnostic neurons, the authors further proposed neuron-specific tuning method to improve the multilingual ability of LLMs. Overall, most reviewers are positive about the novelty of the work, the observed insights, and the extensive evaluations. The authors did a good job on addressing reviewers questions with additional empirical evidence, analysis, and clarifications. Although there are still some remaining concerns on the notion of agnostic score and discussions consistency (reviewer AGNp), it is expected that the authors can address them to further improve the clarity of the revised manuscript. As a result, an acceptance is recommended.
Decision: Accept (poster)