Benford’s Curse: Tracing Digit Bias to Numerical Hallucination in LLMs
Authors: Jiandong Shao
Yao Lu
Jianfei Yang
Venue: NeurIPS 2025 poster ()
Keywords: Digit Bias
Benford's Law
Numerical Hallucination
Pretraining Bias
Large Language Models
Primary area: deep_learning
TL;DR: We investigate how digit bias learned from skewed pretraining data lead to systematic numerical hallucinations in LLMs.

Abstract: Large Language Models (LLMs) exhibit impressive performance on complex reasoning tasks, yet they frequently fail on basic numerical problems, producing incorrect outputs. Inspired by Benford’s Law, a statistical pattern in which lower digits occur more frequently as leading digits, we hypothesize that the skewed digit distributions in web-collected corpora may be learned by LLMs during pretraining, leading to biased numerical generation. To investigate the hypothesis, we first examine whether digits frequencies in pretraining corpus (OLMo2) follows Benford's law. We then construct an evaluation benchmark in which the ground-truth digits are uniformly distributed within each of the seven numerical reasoning tasks. Our evaluation results demonstrate that leading open-source LLMs show a consistent pattern of digit bias that resembles Benford's law. Through logit-lens tracing and neuron-level dissection, we identify that this bias arises predominantly from a small subset of highly digit-selective feed-forward network (FFN) neurons in the deeper layers. Finally, we demonstrate that pruning these neurons mitigates imbalanced overgeneration and partially corrects erroneous outputs, providing causal evidence that fine-grained pretraining digit bias can propagate into model behavior. Our findings reveal a fundamental connection between corpus-level statistics and symbolic failure modes in LLMs, offering a new lens for diagnosing and mitigating hallucinations in numerical tasks.

Benford's Curse: Tracing Digit Bias to Numerical Hallucination in LLMs
Jiandong Shao'; Yao Lu², Jianfei Yanglt
Naayang Technological University, Universityy College London shamyshao@gmail. com, yao. lu@cs uul. ac. uk, jianfei yangentu senggnnt..duu..gg.
Abstract
Large Language Models (LLMs) exhibit impressive performance on complex rea- soning tasks, yet they frequently fail on basic numerical problems, producing incorrect outputs. Inspired by Benford's Law, a statistical pattern which lower digits occur more frequently as leading digits, we hypothesize that the skewed digit distributions in web-collected corpora may learned by LLMs during pretraining, leading to biased numerical generation. To investigate the hypothesis, we first ex- amine whether digits frequencies pretraining corpus (OLMo2) follows Benford's law. We then construct an evaluation benchmark in which the ground-truth digits are uniformly distributed within each of the seven numerical reasoning tasks. Our evaluation results demonstrate that leading open-source LLMs show consistent pattern of digit bias that resembles Benford's law. Through logit-lens tracing and neuron-level dissection, identify that this bias arises predominantly from a small subset of highly digit-selective feed-forward network (FFN) neurons the deeper layers. Finally, we demonstrate that pruning these neurons mitigates imbalanced overgeneration and partially corrects erroneous outputs, providing causal evidence that ginengraeeed pretraining digit bias can propagate into model behavior. Our findings reveal a fundamental connection between corpus-level statistics and symbolic failure modes LLMs, offering a new lens for diagnosing and mitigating hallucinations in numerical tasks. Code and data is made available here: Intps: //gith/bb.bbt com/ homm/shmy22///// /ate////eenorrree Curse
1 Introduction
Large Language Models (LLMs) have demonstrated impressive performance not only on daily reason- ing tasks but also on challenging domains such as mathematical competition problems, showcasing strong symbolic and logical capabilities [[]
6 77.. Yet. despite these amazing success, producing mathematically incorrect or logically inconsistent outputs X 9 10 This discrepancy prompts a fundamental and
2 3,
4
5
LLMSs frequently fail on surprisingly simple numerical reasoning-ore
pressing question: why are L.LMs so prone to hallucinations when dealing with basic numbers?
Previous studies [I] 12, 13 [4] have identified duplication bias in pretraining corpus as a major contributor hallucinations in natural language tasks. For instance, when prompted with "list some red fruits, excluding apples", LLMs frequently still mention "red apples" likely due to the high frequency of co-occurring phrases such as "red apples, watermelon, cherries, and strawberries" in the training data. This over-reliance on memorized patterns causes the model to overlook explicit instructions in the prompt [TT] Given this strong cinttrrncoomlettoonn tendency in language, a natural
"Thi work conducted during the author's research internship MARS Lab, NTU. Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Task: Identifying the Last Term in a Sequence
What is the result of multiplying the sequence's last term by two? [..., 2.95. 1.81, 8.26] Model Generation: To find the result of multiplying the sequence's 's last term by two, we need identify the last term in the sequence and then multiply it by 2.
Prompt 1:
The last term in the sequence is 1.81 x
(Incorrect: the model mistakenly identifies the second-to-lax number the last term.)
What is the result of multiplying the sequence's last term by two? [.., 7.43, 8,51, 1.05] Model Generation: To find the result of multiplying the sequence's 's last term by two, we need to identify the last term in the sequence and then multiply it by 2.
Prompt 2:
The last term the sequence is 1.05 (Correct: the model correctly identifies the last number in the sequence.)
Figure I: Illustration of digit bias in identifying the last term of a numerical sequence. LLaMA3. 1-B. 11-8--Insstuce
is asked to multiply the final term of sequence by two, but first must identify the last item. In the first case, the model incorrectly selects smaller intermediate value (1.81) instead of the actual final term (8.26). In the second, correctly selects 1.05. This asymmetry suggests hias toward smaller digits when the model is uncertain, revealing subtle form of numerical hallucination. Detailed accuracy comparisons are shown in Figure 2
question arises: might similar memorization-die -driiie biases emerge in numerical reasoning tasks, potentially leading systematic numerical hallucinations?
Benford's Law [5], a striking statistical phenomenon, was first observed in 1881 by astronomer Simon Newcomb, who noticed that the earlier pages of logarithmic tables-thon starting with digit were more worn than others [16]. This empirical observation was later formalized by physicist Frank Benford in 1938, who verified the pattern across over 20 diverse datasets (17]. Benford's Law states that the probability that a number begins with digit d E (l...., 9} is given by P(d) (1 2). This implies that digit occurs as the leading digit in about 30% of cases, while digit 9 appears less than 5%. This law has since been observed across various domains such as economic records, scientific measurements, and population statistics, and thus naturally emerges in comman-written text [15] 18 19 Given this ubiquity, it is natural to ask whether the pretraining corpora of LLMs, which are largely composed of web-scraped data, also reflect this skewed digit distribution. If so, could LLMs internalize such patterns during training. and might this lead to form of digit bias that contributes to numerical hallucinations, even on otherwise simple numerical reasoning tasks?
To investigate this question, begin by examining the digit distribution in the Olmo-mix-I 124 pretraining corpus [20] and confirm a strong digit skew consistent with Benford's Law. To test whether this corpus-level skew propagates into model behavior, we construct a diagnostic benchmark with uniformly distributed ground digits, eliminating task-induced priors. Empirical results show that LLMs consistently overgenerate smaller digits, despite uniform targets. Further analysis reveals that when the model generates incorrect answers, the distribution of first error digits(e.g.. for ground truth 758 and prediction 714, the first error digit is 1) exhibits an even stronger skew toward smaller values. To better understand the underlying mechanisms, we analyse the model's 's internal representations. First, we apply the Logit Lens 21 trace how digit preferences evolve across layers and find that the preference for smaller digits often emerges strongly in the later layers. Second, we disentangle the contributions of FFN and self-attention, finding that the bias is primarily driven by the FFN. Third, by quantifying the digit selectivity of FFN neurons, we find that their aggregated preferences form a skewed distribution favouring smaller digits, closely mirroring the statistics of the pretraining corpus. Motivated by these insights, we explore a lightweight neuron-level intervention that prunes a small set of biased neurons. This intervention can partially mitigate overgeneration and correct certain hallucinated outputs, offering further evidence of the causal role digit bias plays in numerical hallucination.
The contributions of this work are summarized as follows: (1) We formulate and investigate fundamental question chain: does the skewed digit distribution LLMs' pretraining corpus induce digit generation bias, leading to numerical hallucination? (2) To explore this, we construct a diagnostic benchmark with uniformly distributed target digits and observe that L.Mss consistently overgenerate smaller digits. Notably, the first error digits exhibit even stronger skewed distribution, suggesting

---

bias-driven mechanism behind numerical hallucination. (3) Using logit lens tracing and neuron-level analysis, we identify that this digit bias primarily originates from a subset of highly digit-sclective feedforward (FFN) neurons, particularly concentrated in the later layers of the model. (4) Finally, we show that pruning these neurons can partially mitigate hallucination, offering causal evidence that digit bias is a contributing factor to numerical hallucination. By identifying fine-grained statistical artifact as a mechanistic failure point, our work highlights a critical yet underexplored source of errors in numerical reasoning and underscores the importance of addressing pretraining-induced biases to develop more reliable language models.
2 Related Works
Numerical Hallucinations in LLMs. While there is no universally accepted definition of numerical hallucination, the term broadly refers to a language model's tendency to generate numerically incor- rect, inconsistent, or implausible outputs, despite syntactically or semantically coherent completions. These include phenomena such as miscounting, incorrect number reproduction, inappropriate number substitution, or generating fabricated values. Prior work has largely attributed these numerical halluci- nations to reasoning deficiencies [22], suboptimal tokenization schemes [23], and misalignment (24). Razeghi al. (25] further showed that model accuracy in numerical tasks correlates with token frequency in pretraining data, suggesting a statistical basis for some failures. However, these studies do not examine how digit-level statistical patterns manifest during generation. We highlight this as distinct and measurable form of numerical hallucination overlooked in prior work.
Dataset Bias as Source of Hallucination. Prior studies have increasingly identified dataset bias 1 25 [2]. suggesting that many hallucinations
as a key driver of hallucination in LLMs [44 IT stem not from architectural flaws but from imbalances in the pretraining corpus. For example, overrepresented entities or linguistic patterns in large-scale web data can lead models to repeat them even in inappropriate contexts []I]. However, existing work [26] 27 28 has primarily focused on semantic-level biases, such as spurious facts or misattributions, while overlooking lower-level statistical regularities. In particular, the extent to which digit-level statistical pattern in the training data influences the model's numerical generation remains largely unexplored. Our work fills this gap by showing that frequency pattern over digits in the pretraining corpus is systematically reflected model generations, providing a concrete statistical basis for a subset of numerical hallucinations.
Small Digit
3 Investigating Digit Bias in LLMs
80
PWN
Big Digit
60
Benford's Law describes a logarithmic distribution over leading digits in naturally occurring numerical data, where smaller digits appear with higher frequency than larger ones. This prompts critical question: does the pretrain- ing corpus, largely sourced from real-world Internet text, also exhibit similar skewed digit distribution? If so, how might such a skewed distribution influence digit generation in L.MS?? To motivate this investigation, we begin with a simple diagnostic experiment illustrated in Figure [] In this basic sequence identification task, the model demon- strates noticeably higher accuracy when the final digit to be predicted is small (1 or 2) compared when it is large (8 or 9), as shown in Figure 2 This asymmetry suggests a preference for generating small digits, hinting at po- tential internal digit bias that could underlie numerical hallucinations.
Score
20-
LlaMA3.1 8D
OLMo 2-7D
Figure 2: Accuracy in the sequence last term identification task shown in Figure] [ Digits in the range 1-3 (low-value) are recognized with significantly higher accuracy compared to digits in the range 8-9 (high-value), indicat- ing generation bias toward smaller digits.
Digit Distribution in Pretraining Corpus. To empirically assess whether the digit distribution in
pretraining corpus aligns with Benford's 's Law, we analyze the Olmo-Mix-1 124 corpus (00 widely- used 22.4TB data collection curated for training open-source L.LMs. As shown in Figure 4a| the digit frequencies strongly align with Benford's Law, exhibiting pronounced logarithmic distribution that
3

---

Table Examples tasks in the Digit Bias Benchmark, Covering seven numerical reasoning tasks.
Task Category
Examples
1. What 0..121789 minus -6?
Add or Sub
Add 4292 and 597069553.5.
1. Multiply 13862 and 0.5.
Multiplication
2. What is -464 times -3?
Solve 81190 divided by 2.
Division
2. Calculate the division of 40380 by 5.
1. Let c(a) = - 10. What is c(-1)?
Evaluate
2. Let I(f) = 3*f*02 f 3. Give 1(2).
What the fifth root of 29889504 to the nearest integer?
Nearest Integer Root
2. What is 4528941 to the power of 1/9, to the nearest integer?
1. Solve 33 8 137 712 for
Linear ld
Solve 309 + 23940 -356*j for
1. What comes next: 532118, 1064232, 15963467
Sequence Next Term
2. What is next in 704, 506, 334, 188, 68?
favors smaller digits. These findings suggest that LLMs are likely to encounter digit distributions that are heavily biased toward smaller values during training.
Benchmark for Measuring Digit Bias. investigate whether skewed digit distribution in pretraining data leads to generation bias, we introduce the Digit Bias Benchmark, suite of seven numerical reasoning tasks designed to yield uniformly distributed ground-truth digits. Tasks include add or sub, mulriplication, division, evaluate, nearest integer root, linear_ _Id and sequence next term, primarily adapted from DeepMind's Mathematics Dataset [29]. Each task contains over 1,000 examples, with answer sets carefully constructed to ensure uniform digit distribution: when pooling all digits from all positions across all answers within a task (e.g.. the answer "132" contributes three digits: 1, 3, and 2), each digit 0-9 appears approximately 10% of the time. This design enables to disentangle generation bias from insk-induced digit distribution effects, allowing a more controlled evaluation of digit-level preferences in LLMs. Representative examples for each task are listed in
Table 1
Empirical Evidence of Digit Bias. We evaluate six open-source LLMs on the proposed Digit Bias Benchmark, including models from LLaMA [00, Qwen [3I,, Gemma [32]. OLMo (33 and Mistral 33) families. By design, the benchmark enforces uniform digit distribution in its ground truth, any deviation in the model's output distribution directly reflects inherent generation bias. As shown in Figure 3a Mistral-7B exhibits a strong and consistent over-generation of smaller digits. For example, digit often appears over 12% of the time, while digits such as 8 and 9 are severely underrepresented. This trend closely parallels the skew found in the pretraining corpus, reinforcing the hypothesis that the bias originates from corpus-level statistics. To further probe the behavioral
35
6.14
30-
25
Propertonn
6.12
0.10
10
Digit
(a)
(b)
Figure Digit bias observed in the Digit Bias Benchmark with Mistral-7B. (a) The boxplot shows the distribution of digits in generated answers across all lasks. revealing significant overrepresentation of smaller digits despite the benchmark's 's uniform ground-truth distribution. (b) The distribution of digits at the first error position exhibits an even stronger skew toward smaller values, closely following Benford's s Law. Together, these results suggest that digit bias shapes not not preferences but also the numerical hallucination.

---

impact of this bias, we conduct fine-graine error analysis. For each incorrect output answer, we identify the first digit where the model's output diverges from the ground truth and record its value (e.g., for ground truth 758 and prediction 714, the first error digit is 1). As shown in Figure 3bj the distribution of these "first error digits" exhibits an even stronger skew toward smaller values, closely following Benford's Law. This finding suggests that digit bias not only affects overall preferences but also distorts the model's generation trajectory when it deviates from the correct answer, implicating as potential driver of numerical hallucination. More results are shown in Figure
4 Analyzing the Mechanisms of Digit Bias
4.1 Probing Layerwise Behavior with Logit Lens
Logit Lens. The LLMs used in this paper are based decoder-only architecture composed of stacked Transformer layers [35] with residual connections [6]. During a standard
Method:
forward pass, the hidden representation 4%2 E Hg for the final input token 2n is retrieved from learned embedding table. This representation is then iteratively updated through each Transformer block, incorporating outputs from both self-attention and feed-forward submodules via residual addition. After the final layer, the hidden state n,4") (with I. denoting the total number of layers) normalized and projected via the unembedding matrix U gx yielding logits over the vocabulary of size v. Since hidden states across all layers share the same dimensionality. intermediate representations can also be projected via U to obtain layer-wise token distributions. This technique, known as the logit lens [21]. allows us to visualize how the model's token predictions evolve across layers, providing interpretable insights into the generation process. To explore the origins of digit bias, we apply the logit lens to trace layer-wise changes in predicted digits.
Datasets. As observed in Section B] LLMs consistently tend to favor smaller digits in numerical reasoning tasks. This suggests that, under conditions of uncertainty or hallucination, the model is more likely to generate smaller digits. Therefore, trace the internal origins of this bias, hallucinated outputs should be the focus. However, hallucinations in multi-step reasoning tasks are often hard to localize, the precise point of failure is difficult identify. To simplify this analysis, begin with single-step reasoning tasks (basic arithmetic tasks from the benchmark), where hallucination can be precisely localized by identifying the first incorrectly generated digit. Then, expand this analysis beyond a limited set of hand-selected cases, we develop an entropy-based automated sampling strategy grounded observations from these hand-analyzed examples identify uncertain samples. Specifically, we compute the entropy of the model's output digit token distribution at each layer. Samples with entropy exceeding threshold (e.g.. >3.0 at layer 26) are flagged as potentially biased. Applying this criterion to the Evaluate task yields a larger set of high- samples, enabling more systematic investigation of digit bias within the model's 's internal dynamies.
30
Dertlee's Law Predicio GR663 Pretraling Corpus
Distrbbttin555)
38
335
30
1000
3322
Digits (a)
Digits
(b)
Figure 4: (a) The histogram compares the digit distribution predicted by Benford's Law with that of the OlMo-mix- 1124 corpus, showing their degree similarity. (b) The heatmap digit probabilities across layers obtained via Logit Lens. Smaller digits remain relatively undistinguished in early and middle layers, whereas larger digits show sharper activations earlier. This indicates that the overgeneration of small digits is driven by preferences formed in the final layers.
5

---

Layerwise Visualization of Digit Bias. By applying the logit lens [21] to high-uncertainty samples that ultimately generate different digits, we obtain the layer-wise digit preference heatmap shown in Figure 4b The visualization reveals a consistent pattern: under similar levels of uncertainty, smaller digits tend to exhibit stronger generation signals only in the later layers, whereas larger digits show earlier and more gradual emergence. For instance, digit 1 typically shows little preference in intermediate layers but becomes strongly favored in the final few. These results indicate that digit bias is not uniformly distributed across the network, but instead predominantly arises in the final layers.
Layerwise Bias Trends. To gain deeper insight into how digit bias emerges in the later layers, conduct layer-wise analysis focusing how the model's digit preferences evolve across later layers. Specifically, we examine samples where digit token probabilities are approximately equal at an intermediate layer and then trace how these probabilities shift in subsequent layers. As shown in Figure [3 the model begins to exhibit a clear preference for smaller digits in later layers, with their probabilities increasing more sharply compared larger digits. This further supports our observation that digit bias is primarily concentrated and amplified in the final layers of token generation.
Digil 1
Digit 1
Digil: 3
Digit 3
Normalteed Probebiiity
Digit 6
Diptt
Eigit
Layer
Layer
Layer
Figure 5: Digit probability trajectorics in later layers. Starting from layers where digits 1, 3, and have equal probabilities (layers 17, 22, and 25 respectively). we trace their subsequent evolution. Digit consistently gains prohability more rapidly than the others, suggesting that the model amplifies its bias toward smaller digits during final token prediction stages.
4.2 Digit Selectivity Score
To quantify how strongly a hidden vector favors a specific digit more precisely, we propose the Digit Selectivity Score (DSC). Given hidden vector h", we first project it into vocabulary logits via the model's 's unembedding matrix. For each digit token E {0..... 9), we compute its rank within the full vocabulary. Let S denote the sum of ranks of all digit tokens. Then, the DSC for digit i is defined as DSC; = S/rank(i). Higher values indicate stronger selectivity for that digit. While normalized probabilities can indicate digit preference to some degree, they often fail to provide meaningful differentiation when all digit probabilities are low, leading to misleading or noisy interpretations. In contrast, DSC offers more sensitive and robust measurement of digit selectivity for any vector in the model's hidden space.
4.3 Self-Attention vs. FFN
Our goal in this section is to determine whether digit bias primarily originates from the self-attention the FFN, or their combined effect. Prior work [37 38 39 using causal mediation analysis has shown that, in arithmetic tasks, digit generation is predominantly driven by the FFN, while only small number of attention heads contribute by storing simple arithmetic facts or attending to operands and operators. To examine whether digit bias follows a similar pattern, we compare the correlation between the DSCs of each layer's 's residual stream and the DSCs of its self-attention and FFN. Let d (d), d' ....... d²)) denote the DSCs of the residual stream across all layers. dm (din d'im. .i....... dfi.) the DSCs computed from FFN outputs, and d (dam d ... dh those from self-attention outputs. We then compute the Spearman correlation between d and each of d and dam to assess which component more closely aligns with the overall digit bias the model's hidden representations.
Comparison Results. Figure 6 presents the Spearman correlation results across layers. In interme- diate layers, the residual DSC is strongly correlated with the self-attention output, while in the later layers, the residual DSC shows a much stronger correlation with the FFN output and very weak

---

correlation with the self-attention output. Given that digit bias emerges most prominently in later layers, this suggests that the FFN module in the later layers the main contributor to such bias.
(a)
20
Layer
Layer
Figure 6: (a) Self-attention DSC vs. residual DSC. (b) FFN DSC vs. residual DSC. Self-attention exhibits moderate correlation in middle layers, while the FFN shows stronger correlation in the later layers, highlighting the FFN's role in driving digit bias.
4.4 FFN-Ievel Analysis
Having established that digit bias primarily emerges from the FFN in later layers, we now investigate why FFNs in these layers tend to induce such bias.
Interpreting the FFN. The feed-forward network (FFN) at layer mn produces its output as:
dua
p(m)
FFN (FFN((
w-(m)
-f(m))
W
int
(1)
Here, f(())
E Rd and ((()) E Rd are the input and output vectors of the FFN at layer m, p(m) E Re
the post-activation intermediate vector, and W(m)) e -2dxd is the down-projection matrix. The pair (mm,", m.n. represents the activation and output direction of the n-th neuron in the layer. This formulation aligns with the view of Geva et [40], who interpret each FFN neuron as a "key-value memory" unit, where the output direction w encodes semantic information. When projected through the unembedding matrix U, reveals token preference distribution that can be interpreted as the neuron's 's functional role.
What Drives Digit Bias in the FEN? The output of an FFN layer is shaped not only by the input representation but also by the directional structure its learned parameters. While inputs are often complex and difficult to interpret, the output directions of individual neurons offer more interpretable handle. Building on the interpretability analysis above, we examine whether the skewed digit distribution from the pretraining corpus is reflected in the FFN's parameters. To this end, we compute the DSC of each neuron with respect to each digit, and then aggregate these scores obtain model-wide selectivity profile. We find strong correlation between this profile and the digit frequency distribution the pretraining data (r = 0.949, Pearson), suggesting that the FFN not only encodes general numerical knowledge but also internalizes corpus-level frequency biases. Figure shows the DSC distributions of the top 1000 most selective neurons for digit and digit 7, Neurons associated with digit consistently exhibit higher selectivity scores than those for digit indicating that the model dedicates larger portion of its representational space to more frequent digits. This uneven allocation likely contributes directly to the emergence of digit bias in generation.
Debiasing Method: Probing the Causal Role of Digit Bias
Our analysis shows that LLLMs not only overgenerate smaller digits but also exhibit a stronger tendency to generate them when first deviating from the correct answer, following pat- tern consistent with Benford's Law. This suggests that digit bias may shape not just gen- eration preferences but also the trajectory of numerical hallucination. However, statistical alignment alone does not establish whether digit bias plays direct causal role numeri- cal hallucination. To probe this link, we introduce lightweight neuron pruning interven- tion. Rather than aiming to improve overall accuracy, this approach tests whether selec- tively suppressing neurons most biased toward digit 1 reliably correct erroneous outputs.

---

[a] LlaMA2-7B
Qwen2.5-7B
(c) G Gemma2-9B
(d) Mistral 7B
Ogen
Dap
10 12
14
16
16
Figure 7: FPN neuron-level selectivity distributions for digit '1" (blue) and digit '7' (orange) four open-source
LLMs: (a) LLaMA2-7B, (b) Qwen2.5-7B, (c) Gemma2-9B, and (d) Mistral-7B. each digit, we independently select the top 1000 neurons with the highest selectivity. Across all models, neurons most selective for digit 'I' exhibit higher selectivity scores than those most selective for digit "7', revealing stronger model-intenee bias toward lower digits.
Table Effect pruning top 0.01 most digit-1-selectivy neurons. Coreected Prop. denotes the proportion of all test samples that were originally incorrect but become correct after pruning. Original Prop. denotes the original frequency of digit in model outputs. Pruned Prop. denotes the digit 1 frequency after pruning.
Multistep Reasoning Tasks
Model
Evaluate
Linear ld
Nearest Integer
Sequence Next term
Coreected Prop.
1.36 %
0.76 %
0.19 %
0.18 %
Llama2-7B
Original Prop.
16.26 %
16.21 %
11.91 %
11.70 %
Pruned Prop. 11.17 %
13.10 %
6.6 %
9.74 %
Coreected Prop.
1.22 %
1.94 %
0.35 %
0.54 %
Mistral-7B
Original Prop.
15.63 %
14.49 %
21.64 %
11.90 %
Pruned Prop. 11.85 %
10.92 %
10.71 %
10.61 %
Coreected Prop.
3.49
5.05 %
4.96 %
4.73 %
Qwen2.5-7B
Original Prop.
16.45 %
15.85 %
16.25 %
14.06 %
Pruned Prop.
14.72%
13.86 %
13.13 %
12.29 %
Coreected Prop.
1.69
2.14 %
1.94 %
1.16 %
Gemma2-9B
Original Prop.
13.66
11.04 %
17.49 %
11.36 %
Pruned Prop. 12.53 %
10.91 %
14.22 %
11.23 %
successful, such targeted correction would provide concrete evidence of a causal link between digit bias and numerical hallucination.
Table 3: Corrected Prop. on arith- metic henchmarks after pruning.
Model
GSM8k
SVAMP
Llama2-7B
2.35 %
1.60 %
5.1 Locating and Pruning Biased Neurons
Mistral-7B
1.97 %
1.40 %
Owen2.5-7B
2.12 %
0.70
Each FFN neuron's output contributes directly to the residual stream and thus influences final token prediction. We estimate neuron's contribution digit bias isolating its standalone output and computing its Digit Selectivity Score toward each digit.
Gemma2-9B
0.08 %
0.50 %
Formally, for the n-th neuron in layer we define its bias score toward digit as:
biscore... n. = DSC, (f
m.R.
(2)
Here, 1i is the neuron's 's activation, and w m.n is its corresponding output direction. This scalar product represents the neuron's 's individual contribution the final output, which can be interpreted in terms of digit selectivity via the DSC. Given that individual neurons may encode multiple features or functions [41), indiscriminate pruning across all decoding steps may disrupt general model behavior. Therefore, we prune only the top 0.01 % most digit-1-selectivy neurons, and activate this intervention only during the generation of digit tokens. Furthermore, to minimize the disruption to the model's reasoning path, the generation token after pruning is restricted to digit token only.

---

Comparing Model Behavior With and Without Pruning
Prompt 1: Solve -2005 p4553* 1977*p + 3235 for p.
Generation (no pruning): ... Simplify the left side: [[-2200 524p [ 3. **Solve for paa by by dividing both sides by 524: V[p \frac(-52411 524))4(4))......
Generation (pruning):
Simplify the left side: \[-5240 = 524p\] 3. **Solve for p** by by
dividing both sides by 524: A[p = frrac(62400-6624)V /
Prompt 2: Solve 366*k + 9029 = 14885 for
Generation (no pruning): To solve for k in the equation 366k 9029 = 14885, we can use the following steps: 1.Subtract 9029 from both sides of the equation get: 366k 5856 2. Divide both sides of the equation by 366 to get: k 161 Therefore, the value of is 161. Generation (pruning):
solve for k the equation 366k 9029 14885, we can use the following steps: Step I: Subtract 9029 from both sides of the equation to get: 366k = 5856 Step 2: Divide both sides of the equation by 366 to get: k 5856 / 366 16 Therefore, the value of k is 16. /
Prompt 3: Solve 4 = * + 14100 for w.
Generation (no pruning): To solve the equation -47613t = -47331 0 + 14110 for w.... X Generation (pruning): To solve the equation -47613w w47331 + 14100 for w....
Figure 8:
Examples showing model responses before and after pruning the top 0.01% of neurons biased toward digit 1. In each cases, pruning corrects an originally erroncous sample by rectifying an intermediate step, demonstrating causal relationship between digit bias and numerical hallucination.
5.2 Debiasing Results and Causal Insight
Table[]report two key effects of pruning the top 0.01% most digit-1- neurons: a notable reduction in the generation frequency of digit 1, and a measurable correction rate among previously erroneous outputs. Manual inspection confirms that most of these corrected outputs originally featured small-digit hallucinations as shown in Figure 8 Consistent with these findings, Table shows that similar correction effects also emerge on real-world mathematical reasoning benchmarks (GSM8k (42) and SVAMP (43). Given the Complexity of multi-step numerical reasoning, such corrections are unlikely to be coincidental. Rather, these corrections suggest that digit bias plays an active role in driving the model off its correct reasoning path by skewing generation toward over- frequent digits like 1. If digit bias were not causal factor in hallucination, such targeted suppression would not yield consistent improvements on biased outputs. This result provides empirical support for a causal link: digit bias is not merely a statistical artifact, but a mechanistic contributor to numerical hallucination.
Takeaway. While our pruning method is coarse and may negatively affect some previously correct predictions, its ability to reliably correct biased errors underscores the causal link between digit bias and hallucination. Rather than being general-puppose debiasing tool, this method serves as a probing mechanism to deepen our understanding on how low-level statistical priors manifest as systematic reasoning failures in L.MMs.
Conclusions
This work investigates the overlooked role of digit bias in LLMs. We begin by showing that pretraining corpus exhibit a logarithmic digit distribution consistent with Benford's Law. LLMs internalize this skew, as evidenced by their tendency to overgenerate smaller digits during numerical generation. Notably, in incorrect answers, the first erroneous digit often falls among the smaller digits, suggesting that this bias not only shapes output preferences but also contributes numerical hallucination. Mechanistically,, we trace this bias to the final digit-selective neurons encode preferences aligned with corpus statistics. Finally, we propose lightweight neuron pruning strategy that corrects portion of biased errors, offering causal evidence that fine- me-iirai digit biases can directly cause numerical hallucination. These findings highlight how lewelevel statistical priors from pretraining data can affect high-level behavior in LLMs.
layers of LLMs, where a subset of highly

---

Limitations and future work. While our work reveals a compelling link between digit bias in the pretraining corpus and numerical hallucinations LLMs, it has several limitations. Most importandy. although we observe a strong correlation between digit frequencies in the training data and biased generation behavior, we do not claim a causal relationship; establishing causality would require controlled interventions during training, which we leave for future work. In addition, our experiments are conducted on relatively small-scale decoder-only LLMs (7B-9B parameters) with standard MLP architectures. Whether similar patterns of bias and internal activation dynamics persist in larger models or those employing Mixture-of-Experts (MoE) remains an open question. Finally, our pruning strategy offers causal insights into digit-selective neurons but is coarse, potentially disrupting correct generations and limiting accuracy gains. We believe that more fine-gres ame-grained or adaptive debiasing methods could yield stronger performance improvements. However, developing such techniques is beyond the scope of this work and is left for future investigation.
7 Acknowledgements
This work is supported by Start-up Grant from Nanyang Technological University and jointly funded by the Singapore Ministry of Education (MOE) under a Tier-1 research grant.
References
[1] Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin. Large language models for mathematical reasoning: Progresses and challenges. ArXiv, abs/2402.001557, 2024.
[2] Simon Frieder, Julius Berner, Philipp Petersen, and Thomas Lukasiewicz. Large language models for mathematicians. ArXiv, 1,,055,,, 2023.
[3] Xinyu Guan, Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. rstar-math: Small llms can master math reasoning with self-evolved deep thinking. ArXiv, abs/2501.0459 100519, 2025.
[4] Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos. Stephen Marcus McAleer. Albert Q. Jiang, Jia Deng, Stella Biderman, and Sean Welleck. Llemma: An open language model for mathematics. ArXiv, abs/2310. 10631, 2023.
[5] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, F. Xia, Quoe Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. ArXiv, abs/2201. 1903, 2022.
[6] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zcro-shot reasoners. ArXiv, abs/2205. 11122005.1996. 1916, 2022.
[7] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155. 2022.
[8] Safal Shrestha, Minwu Kim, and Keith Ross. Mathematical reasoning in large language models: Assessing logical and arithmetic errors across wide numerical ranges. ArXiv, abs/2500208666 2025.
[9] Iman Mirzadeh, Keivan Alizadeh- Vahid, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. Gismssybbolic: Understanding the limitations of mathematical reasoning in large language models. ArXiv, abs/2410.05229, 2024.
[10] Qintong Li, Leyang Cui, Xueliang Zhao, Lingpeng Kong, and Wei Bi. Gsm-plus: A comprehensive benchmark for evaluating the robustness of Ilms mathematical problem solvers. ArXiv, 192555.0..1925. 2024.
[11] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng. Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ArXiv, abs/23 11 1002322, 2023.
[12] Nick McKenna, Tianyi Li, Liang Cheng, Mohammad Javad Hosseini, Mark Johnson, and Mark Steedman. Sources of hallucination by large language models on inference tasks. In Conference on Empirical Methods in Natural Language Processing, 2023.
10

---

[13] Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang. Enbo Zhao,
Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi Siren's song in the ocean: survey on hallucination large language models. ArXiv, abs/2309.01219, 2023.
[14] Katja Filippooaa. Controlled hallucinations:lemming to generate faithfully from noisy data. In Findings of EMNLP 2020, 2020.
[15] Theodore P. Hill. A statistical derivation of the significant-dee diggt law. Statistical Science, 10:354-366 1995.
[16] Simon Newcomb. Note on the frequency of use of the different digits in natural numbers. American Journal of Mathematics, 4:39, 1881.
[17] F. Benford and I. Langmuir. The Law of Anomalous Numbers. Proceedings of the American Philosophical Society, American Philosophical Society, 1938.
[18] Mark Nigrini. Benford's s Law: Applications for forensic accounting, auditing, and fraud detection. John Wiley & Sons, 2012.
[19] Joseph Deckent, Mikhail Myagkov, and Peter C. Ordeshook. Benford's law and the detection of election fraud. Political Analysis, 1963):245-268, 2011.
[20] AllenAI. Olmo-mix- 124 dataset. httpp: 2024.
1124
[21] Nostalgebraist. Interpreting gpt: The logit lens. LessWrong, 2020.
[22] Kaixuan Huang, Jiacheng Guo, Zihao Li, Xiang Ji, Jiawei Ge. Wenzhe Li, Yingqing Guo, Tianle Cai, Hui Yuan, Runzhe Wang, Yue Wu, Ming Yin, Shange Tang, Yangsibo Huang. Chi Jin, Xinyun Chen, Chiyuan Zhang, and Mengdi Wang. Math-perturb: Benchmarking ilms' math reasoning abilities against hard perturbations. ArXiv, abs/2502.06453. 2025.
[23] Aaditya K. Singh and DJ Strouse. Tokenization counts: the impact of tokenization on arithmetic in fronticr Ilms. ArXiv, abs/2402. 14903,, 2024.
[24] Sean McLeish, Arpit Bansal, Alex Stein, Neel Jain, John Kirchenbauer, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, Jonas Geiping, Avi Schwarzschild, and Tom Goldstein. Transformers can do arithmetic with the right embecldings. ArXiv, abs/2405. 17399, 2024.
[25] Yasaman Razeghi, Robert I. Logan IV. Matt Gardner, and Sameer Singh. Impact of pretraining term frequencies few-shot reasoning. ArXiv, abs/2202.072066, 2022.
[26] Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, and Xindong Wu. Unifying large language models and knowledge graphs: A roadmap. IEEE Transactions on Knowledge and Data Engineering. 2023.
[27] Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference Fairness, Accountability, and Transparency. FAceT '21, page 610-623, 623, New York, NY, USA, 2021. Association for Computing Machinery.
[28] Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Mcasuring how models mimic human falschoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 3214-3252, Dublin, Ireland, May 2022, Association for Computational Linguistics.
[29] David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical reasoning abilities of neural models. ArXiv, abs/ 100.01994.1557. 2019.
[30] Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko lay Bashlykov. Soumya Baira. Prajjwal Bhargava. Shruti Bhosale. Daniel M. Bikel, Lukas Blecher, Cris tian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams. Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang. Angela Fan, Melissa Hall Melanie Kambadur, Sharan Narang, Aur' elien Rodriguez, Robert Stojnic. Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and tneet chat models. ArXiv, hbs/2307.09288. 2023.

---

[31]
Qwen Team. Qwen2.5: A party of foundation models, September 2024.
[32]
Gemma Team Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju. L'eonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram's e. Johan Ferret, Peter Liu, Pouya Dehghani Tafti, Abe Friesen. Michelle Casbon, Sabela Ramos, Ravin Kumar. Charline Le Lan, Sammy Jerome, Anton Tsitsulin, Nino Vieillard, Piotr Stańezyk, Sertan Girgin, Nikola Momehev, Matt Hoffman, Shantanu Thakoor, cean-Bastien Grill, Behnam Neyshabur, Alanna Walton, Aliaksei Scveryn, Alicia Parrish, Aliya Ahmad, Allen Hutchison, Alvin Abdagic, Amanda Carl, Amy Shen, Andy Brock, Andy Coenen, Anthony Laforge, Antonia Paterson, Ben Bastian, Bilal Piot, Boxi Wu, Brandon Royal, Charlie Chen, Chintu Kumar, Chris Perry. Christoper A. Welty, Christopher A. Choquette-Choo, Danila Sinopalnikov, David Weinberger. Dimple Vijaykumar, Dominika Rogozi nska, D. Herbison, Elisa Bandy, Emma Wang. Erie Noland, Erica Moreira, Evan Senter. Evgenii Eltyshev, Francesco Visin, Gabriel Rasskin, Gary Wei, Glenn Cameron, Gus Martins, Hadi Hashemi, Hanna Klimezak- nska, Harleen Batra, Harsh Dhand, Ivan Nardini, Jacinda Mein, Jack Zhou, James Svensson, Jeff Stanway, Jetha Chan, Jin Zhou, Joana Carrasqueira, Joana Nljazi, Jocelyn Becker, Joe Fernandez, Joost R. van Amersfoort, Josh Gordon, Josh I. Lipschultz, Joshua Newlan, Junsong Ji, Kareem Mohamed, Kartikeya Badola, Kat Black, Katie Millican, Keelin McDonell, Kelvin Nguyen, Kiranbir Sodhia, Kish Greene, Lars Lowe Sjoesund, Lauren Usui, L. Sifre. Lena Heuermann, Leti cia Lago, Lilly McNealus, Livio Baldini Soares, Logan Kilpatrick, Lucas Dixon, Luciano Martins, Machel Reid. Manvinder Singh, Mark Iverson, Martin Gorner. Mat Velloso, Mateo Wirth, Matt Davidow, Matt Miller, Matthew Rahtz, Matthew Watson, Meg Risdal, Mchran Kazemi, Michael Moynihan, Ming Zhang, Minsuk Kahng, Minwoo Park, Mofi Rahman, Mohit Khatwani, Natalic Dao, Nen shad Bardoliwalla, Nesh Devanathan, Neta Dumai, Nilay Chauhan, Oscar Wahltinez, Pankil Botarda, Parker Bames, Paul Barham, Paul Michel, Peng chong Jin, Petko Georgiev, Phil Culliton, Pradeep Kuppala, Ramona Comanescu, Ramona Merhej. Reena Jana, Reza Ardeshir Rokni, Rishabh Agarwal, Ryan Mullins, Samaneh Saadat, Sara Marie Mc Carthy, Sarah Perrin, Sébastien M. R. Arnold, Se bastian Krause, Shengyang Dai, Shruti Garg. Shruti Sheth, Sue Ronstrom, Susan Chan, Timothy Jordan, Ting Yu, Tom Eccles, Tom Hennigan, Tomás Kociský, Tulsee Doshi, Vihan Jain, Vikas Yadav, Vilobh Meshram, Vishal Dharmadhikari, Warren Barkley, Wei Wei, Wenming Ye, Woohyun Han, Woosuk Kwon, Xiang Xu, Zhe Shen, Zhitao Gong. Zichuan Wei, Victor Cotruta, Phoebe Kirk, Anand Rao, Minh Giang, Ludovic Peran, Tris Warkentin, Eli Collins, Joelle Barral, Zoubin Ghahramani. Raia Hadsell, Daniel Sculley, Jeanine Banks, Anca Dragan, Slav Petrov, Oriol Vinyals, Jeffrey Dean, Demis Hassabis, Koray Kavukcuoglu, Clément Farabet, Elena Buchatskaya, Schastian Borgeaud, Noah Fiedel, Armand Joulin, Kathleen Kenealy, Robert Dadashi, and Alek Andreev. Gemma 2: Improving open language models practical size. ArXiv, abs/2408. 0010800011111.111111111111111110 2024.
[33] Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu. Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord. Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi. Nouha Dziri, Michal Guerquin, Hamish Ivison, Pang Wei Koh, Jiacheng Liu, Saumya Malik, William Merrill, Lester James V. Miranda, Jacob Morrison, Tyler Murray, Crystal Nam, Valentina Pyatkin, Aman Rangapur, Michael Schmitz, Sam Skjonsberg, David Wadden, Christopher Wilhelm, Michael Wilson, Luke Zettlemoyer, Ali Farhadi, Noah A. Smith, and Hannaneh Hajishirzi. 2 olmo 2 furious. 2024.
[34] Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L'elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William El Sayed. Mistral 7b. ArXiv, abs/23 10.06825, 2023.
[35] Ashish Vaswani. Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Neural Information Processing Systems, 2017.
[36] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEFE Conference on Computer Vision and Pottern Recognition (CVPR), pages 770-778, 2015.
[37] Yaniv Nikankin, Anja Reusch, Aaron Mueller, and Yonatan Belinkov. Arithmetic without algorithms: Language models solve math with a bag of heuristics. ArXiv, abs/2410.21272, 2024.
[38] Zeping Yu and Sophia Ananiadou. Interpreting arithmetic mechanism in large language models through comparative neuron analysis. In Conference on Empirical Methods in Natural Language Processing, 2024.
[39] Wei Zhang, Chaoqun Wan, Yonggang Zhang, Yiu ming Cheung. Xinmei Tian, Xu Shen, and Jieping Ye. Interpreting and improving large language models in arithmetic calculation. ArXiv, abs/2409.01 00669,, 2024.
[40] Mor Geva, R. Schuster, Jonathan Berant, and Omer Levy. Transformer feed- layers key-value memories. ArXiv. abs/2012.149// 149113, 2020.
[41] Anthropic. On the biology of large language model. https://transt circcitscctttttii pub/2025/ attribution- http://ionnngrraapri/boooooooooo httll March 2025. Accessed: May 7, 2025.
12

---

[42] Karl Cobbe. Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10.1111.441. 14168, 2021.
[43] Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? Proceedings the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080-2094, Online, June 2021. Association for Computational Linguistics.
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly summarize the key contributions and scope of the paper, aligning with the detailed methodology and results presented in later sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The paper explicitly addresses limitations, discussing potential constraints,
ensuring transparency about the work's ' boundaries.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
13

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: We do not have any theoretical result, but do provide a complete mathematics proof for our method in appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides sufficient methodological details, code and data to allow reproduction of key results, supporting the validity of its claims.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
14

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The paper clearly states open access data and code in supplemental material, along with sufficient instructions for replication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
gublic/guiden hwtp///eueiesioosooooo..oeee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper clearly specifies all key experimental details needed to interpret the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
15

---

Answer: [Yes]
Justification: The paper appropriately reports error bars and statistical significance where relevant to support the experimental claims.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative етгог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide the relvant information in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research adheres to the NeurlPS Code of Ethics by ensuring transparency
in methodology, using publicly available datasets, and avoiding any biased or harmful applications.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
16

---

10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The creators and original owners of all assets (e.g., code, data, models) used in the paper are properly credited with appropriate citations, and the license and terms of use are explicitly mentioned and respected.
17

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All new assets introduced in the paper are thoroughly documented with detailed descriptions, usage instructions, and metadata.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
18

---

Answer:
[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not non
involve LLMs as any important, original, non- stanstandard components.
Please refer to our LLM policy for what should or should not be described.
19

---

A Additional Experiment Results
A.1 Result Digit Bias Benchmark of across Models
We plot the digit distribution in generated outputs across all models on the Digit Bias Benchmark as shwon in Figure All models show significant overgeneration of small digits, with digit being especially dominant. This consistent trend highlights the generality of digit-level generation bias across open-source LLMs.
0.18
6.14
16 13
Propottinn
0.100
15
0.08
Dapta
Hamm2-7b-chatee
1lama2-7-cchat-ht
0.34
Maans Durge
0.13
0.12
0.11
0.10
22
Digis
Jemmaa-9b-it
gemmaa-9h-i
0.13
deeed Truth
30
0.12
14
0.11
0.10
0.09
0.08
Digit
Qwen2.5-76b 70-
Qwen2,5-7b
Tals
0.16
0.12
Propertin
0.14
Propertoon
6.11
6.19
0.12
6.09
0.08
6.08
6.07
Digit
OLMo2-2-1124-77 t
1lama3.1 Sb-Innstuct
Figure 9: Digit generation bias across models on the Digit Bias Benchmark. Since OLMo and LLaMA3. 18B employ multi-digit tokenization schemes, first-error-igiit distribution analyses not applicable and thus omitted for these models.
20

---

A.2 Digit Selectivity
Figure I0 visualizes the selectivity of FFN neurons across digit tokens. The distributions are clearly skewed, indicating that more neurons are specialized toward frequent digits like '1', suggesting an uneven allocation of model capacity that may underlic observed generation biases.
(a) LlaMA2-7B
(b) Qwen2.5-7B
(c) Gemma2-9B
(d) Mistral- 7B
10
14
12
96
18
20
10
12
20
12
16
10
12
14
10
12
16
12
14
18
12
16
10
12
14
16
20
12
16
Dig
8
10
12
14
16
20
10
12
14
16
D
10
12
18
10
12
14
16
18
10
14
16
20
10
12
18
12
14
16
18
10
12
16
20
10
12
16
18
10
12
14
16
18
14
16
10
Figure 10: FFN neuron-level selectivity distributions for digit '1' (blue) versus all other digits (orange) across four open-source LLMss: (a) I.aMA2-7B, (b) Qwen2.5-7B, (c) Gemma2-9B, and (d) Mistral-7B. For each digit, the top 1000 neurons with the highest selectivity scores are selected independently.
21

---

A.3 Additional Pruning Examples and Results
Figure I1 presents additional examples of samples corrected after pruning. Many of the original errors occur in surprisingly simple scenarios, such repeated or trivial arithmetic, highlighting how digit bias can subtly but significantly affect digit generation and lead failure even in straightforward cases.
Comparing Model Behavior With and Without Pruning
Prompt 1: Solve 0 ~90 12 708 for
Generation (no To solve SwS, we need to isolate variable $w$ on one side
equation. First, we can simplify the equation by combining like terms: SSO = -90w - 12 708S$. $SO -102w - 720S$. Next, we can divide both sides of the equation by 5-1025 get: $Sw = -200 12225. X Generation (pruning): To solve for $wS, need isolate the variable $wS equation. First, we can simplify
side of the equation by combining like terms: 5$0 990w 7085$. $$0=
-90w 720SS. Next, we can divide both sides equation hy $-90$ to get: $Sw (720) (90]SS. /
Prompt 2: Solve 38* Generation (no pruning): Let the equation given is: A[ 38w 75 75
75 1026 w.
1126 Al... X
Generation (pruning): Let the equation given is: A[ 38w 75 75 1026 \....
Prompt 3: Solve 78 + 491 + 817 28 + 266 for x. Generation (no pruning): 4. **Perform the division:** M[x 40 \] So, the solution for x: is is
11. X
Generation (pruning): ** **Pefform the division:** [x
\] So, the solution for is is 40.
Prompt 4: Solve 70 -- -1600m 6852 3518 for Generation (no pruning): To solve the equation 1161m 6852
3518 for
Generation (pruning): solve the equation 71 -1160m 6852 3518 for m
a for Generation (no pruning): ... Simplify left side: A. 104a 1664
Prompt 5: Solve 38 1664 - 69
o\] **Isolate term
with a*** Add 1664 to both sides to move the constant to the right side:
[104a 1164X]...
Generation (pruning): Simplify the left side: A[ 104a 1664
ON] 3.
""Ioolate the term
with a:** Add 1664 to both sides to move the constant to the right side:
\[104a 1664X]... ....
Prompt 6: What is the next term in -106577, -1 166453, -06453,-1063799 -106379?
Generation (no pruning): The pattern is adding 104 to each term. 106527 104 -066553 -106453 + 104 = 106379 * -106379 104 ** ** 1#-2755 Therefore. the next term is ** #*.106275
Generation (pruning): The pattern is adding 74
previous term. * -106527 + 74 = -00653 *
-106453 + -06379
1063779 = **_ 10-306 =* Therefore, the next term is *4-106305**,
Figure 11: Examples showing model responses before and after pruning the top 0.01 % neurons biased toward digit 1. In each cases, pruning corrects an originally erroneous sample by rectifying intermediate step, demonstrating causal relationship between digit bias and numerical hallucination.
22

---

B
Additional Implementation Details
B.1 Experiment Setup
To ensure the accuracy and reproducibility of all results, we employed greedy decoding for generation. Additionally, to achieve fully accurate statistical outcomes, we utilized the DeepSeek API to extract the answers model generated for each sample, thereby preventing potential omissions in script-based statistics due to variations in the position of answers within the responses.
B.2 Tokenization Method
LLMs divide numbers into segmented tokens rather than representing the entire number as single token. Different LLMs employ various tokenization methods, including one-digit to- kenizers and multi-digit tokenizers. Benford's Law states that in many real-life sets of nu- merical data, the leading digit is likely be small. In other words, regardless of the type of tokemizer, the distribution of number tokens in pretraining data is likely to be skewed. Therefore, in this paper, we use six models with dif- ferent tokenizers to investigate this phenomenon of digit bias as shwon in Figure 12 LLLMMA-7B, Mistral-7B, Qwen2.5-7B, and Gemma2-99 9B employ single-digit tok-
(@) 271 828
828 459 045
271 8
enizers, whereas LLaMA3.1 1-BB and OLMo2-7B utilize multi-digit tokenization schemes.
Figure 12: (a) multi-digit tokenizer. (b) single-digit tokenizer.
B.3 Prompt Templates
We provide the exact prompt templates used for Identifica- tion task and Digit Bias Benchmark in table 2 and table 5
Table 4: Prompt Templates Used Identification Task
Identification Prompt Template
1. What is the reault when the last term of the sequence is multiplied
by two? [...]
2. What is the outcome when the final term of the sequence is doubled?
[...]
3. What the product the sequence's last term and two? [...]
4. What is the result of multiplying the sequence's s last term by two?
C Use of existing assets
C.1 Models
Table 6: The list of models used in this work.
Model
Accessed via
License
Qwen2.557BB-Inttmun
Link
Apache license 2.0
gemma-2-9h 2-9b-it
Gemma Terms of Use
Mistral-7B-Inn 7B-Instruct-vO.3
Link
Apache license 2.0
Llama-3 3.1-BB- -1-8B-Innsrrce
Link
Llama 3.1 Community License Agreement
llam-227b-ccht-h
.ink
Ilama 2 Community License Agreement
OLMo-2-1 Insteet
Link
Apache license 2.0
23

---

Table 5: Prompt Templates Used in Digit Bias Benchmark
Addition
Division
[p} (q}
Calculate the division of {q} by
{p}+(q}
{p}.
Work out {p} + {q}.
Divide {q} by {p}.
Add {p} and {q}.
What the quotient {q}
Put together {p} and {q}. Sum {p} and {q}.
divided by {p}?
What is {q} divided by {p}?
Total of {p} and {q}.
Find {q} divided by {p}.
Add together {p} and fq). What is {p} plus {q}?
Compute {q} {p}.
Solve {q} divided by (p}.
Calculate {p} + {q).
What is {p} {q}?
Subtraction {p} {q}
Multiplication
{p} {q}
Work out {p} {q}.
Calculate {p} x {q}
What (p} minus {q}?
Work out {p} {q).
What is {p} take away {q}? What {q} less than {p}?
Multiply {p} and {q)
Product {p} and {q}
Subtract {q} from (p}.
What is the product of {p} and {q}?
Calculate {p} - {q}.
What is {p} {q}?
{p} times {q}
What {p} times {q}?
Evaluate
Let {x(x) f(x)). What is {c(a)}?
Let {c(x) fxx). Determine (c(a))
Let (x(x) f(x). Give (c(a)}.
Let (x(x) = f()). Calculate (c(a))
Nearest Integer Root What is the {num-th} root of {p} to the nearest integer?
What is {p} to the power of {1/num-th}, to the nearest integer? Linear 1d Solve {eqution} for {r}. Sequence Next Term
What comes next: (sequence)?
What next in {sequence}?
What is the next term in (sequence]?
C.2 Dataset
Table 7: The list of datasets used in this work.
Dataset
Accessed via
License
olmo-mix-1 124 mathematics dataset
Link
Open Data Commons License Attribution Apache license 2.0
D Compute statement
All experiments presented in this paper were run on cluster of four NVIDIA GeForce RTX 3090 GPUs with 24GB of memory and using single 24GB memory GPU. Each model requires an average of 50 hours to complete full run across the entire benchmark.
24


Reviews:
[{"review_id": "gUKrx7apDy", "reviewer": "Reviewer_QLC4", "summary": "The paper focus on analysing and addressing the issues of numerical representation and generation in LLMs. The phenomenon of digit bias was pointed out by analysing LLMs in extensive evaluation in numerical reasoning tasks. Through logic lens technique, the digit bias mechanism is discovered and the intervention by pruning to debias is then proposed. The experiments are conducted across different LLMs.", "questions": "In the author's opinion, will this debiasing approach also improves performance of the LLMs in other tasks that require numerical processing, such as mathematical reasoning task, or tasks that involve time-series?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "The paper presented very interesting phenomenon via the lens of Benford's law: lower digit often appeared as leading digit. The analysis in web-collected corpora and the answer from LLMs clearly showed this (Figure 2 to 4). This finding potentially reveals a way to look at digit bias/numerical hallucination in LLMs, which is important for reasoning task that involves numbers. Findings from line 171 to 177 gives insight to the phenomenon. It is further substantiated when intervention can be applied to debias. The research is conducted with very careful and well-designed experiments. Experiment is convincing and showing the effectiveness of the approach.", "was_revised": true, "final_justification": "In the rebuttal phase, my questions and comments in the review have been addressed. I maintain the overall score of 5: Accept.", "rebuttal": "We sincerely thank the reviewer's acknowledgement of our work. We are greatly encouraged by your positive assessment of our work. Below, we provide our thought to the raised question.\n\n**Q1:Will this debiasing approach also improve performance of the LLMs in other tasks that require numerical processing, such as mathematical reasoning tasks, or tasks that involve time-series?**\n\nThank you for your insightful question. We believe our method is indeed generalizable, and our benchmark was designed to test precisely this.\n\nOur approach counteracts the phenomenon of \"digit bias,\" which our research shows is a widespread issue in LLMs. To validate this, we constructed the **Digit Bias Benchmark**, a suite of seven diverse tasks that explicitly cover the areas you mentioned. Six of our tasks (e.g., 'Add or Sub', 'Evaluate', 'Linear_ld') are forms of mathematical reasoning, while the 'Sequence Next Term' task is a direct analogue for time-series analysis.\n\nOur evaluations showed that the digit bias is consistently present across these tasks. Crucially, our intervention, which targets the root cause of this bias—specific digit-selective neurons in the model's deeper layers—proved effective. As shown in Table 2, this method successfully corrected a portion of previously erroneous samples across these varied tasks.\n\nTherefore, because our method targets a fundamental, task-agnostic cause and has already demonstrated its effectiveness on a benchmark representative of both mathematical reasoning and time-series tasks, we expect its success to generalize broadly.\n\nWe agree that testing on standard, real-world benchmarks is crucial to validate our approach. Therefore, we extended our evaluation to public benchmarks GSM8K and SVAMP.\n\nThe results confirm that our method achieves a positive correction rate on these tasks as well. It is worth noting that on the SVAMP benchmark, the improvements for models like Qwen and Gemma were modest. We attribute this to their already high baseline accuracy on this dataset, which left minimal room for error correction. Nevertheless, the method's effectiveness on other models and tasks demonstrates its broader potential.\n| Benchmark | Llama2-7B | Mistral-7B | Qwen2.5-7B | Gemma2-9B |\n|:---------:|:---------:|:----------:|:----------:|:---------:|\n| GSM8K     |   2.35%   |    1.97%   |    2.12%   |   0.08%   |\n| SVAMP     |    1.6%   |    1.4%    |    0.7%    |    0.5%   |"}, {"review_id": "OJWfyeTaEF", "reviewer": "Reviewer_gChx", "summary": "This paper investigates how large language models (LLMs) exhibit systematic biases in digit generation, particularly favoring lower leading digits, in ways that resemble Benford’s Law. By analyzing pretraining corpora and evaluating models on custom benchmarks with uniformly distributed digits, the authors show that this bias is learned during pretraining. Through neuron-level analysis, they identify a small subset of digit-selective neurons responsible for the effect. Pruning these neurons reduces the bias and improves numerical accuracy, offering causal evidence that pretraining statistics can shape symbolic failures in LLMs.", "questions": "see weakness", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength\n\n- The paper introduces a novel perspective by linking statistical patterns in pretraining data, such as Benford’s Law, to systematic digit bias in LLM outputs.\n\n\n- It designs a controlled benchmark with uniformly distributed digits, enabling clear and reliable evaluation of model bias in numerical tasks.\n\n\n- The use of logit-lens analysis and neuron pruning offers strong causal evidence and practical tools for identifying and mitigating fine-grained model biases.\n\nWeakness\n\n- How do you ensure that the overall distribution of digits (0–9) in the benchmark is truly uniform?\n\n\n- Please clarify the term “first error digits”; what exactly do the values reported in Table 3 B represent?\n\n\n- In the logit-lens analysis, are the logits produced by attaching the model’s final prediction head to each intermediate layer?\n\n\n- The paper claims that digit bias primarily emerges in the final layers, yet the layer-wise digit-preference heat-map in Figure 4b shows strong color intensities for earlier layers as well. How is the conclusion that bias is concentrated only in the last few layers justified?\n\n\n- For the procedure that traces samples with approximately equal digit probabilities at an intermediate layer, the paper should provide the exact computation and formulas; similar details are needed for how the values in Figure 4b are calculated.\n\n\n- The proposed pruning strategy is coarse and targets bias toward only digit 1, which harms previously correct predictions and limits the method’s generality.\n\n\n- The study addresses only digit-frequency bias, overlooking other possible sources such as digit co-occurrence biases and operation biases (e.g., addition, subtraction, multiplication, division).", "was_revised": true, "final_justification": "I gave Borderline accept (4) initially and the authors partially addressed my concerns, but I don't think it is worth Accept (5) due to the limited scope.", "rebuttal": "Thank you very much for your time, effort, and thoughtful feedback on our paper. Below, we provide a point-by-point response to the concerns and questions you raised, addressing the weaknesses you identified and offering clarifications where needed.\n\n---\n**Q1: how to ensure that the digit distribution (0–9) in our benchmark is truly uniform？**\n\nThe “uniform digit distribution” refers to the answer set digits (0–9) in our seven benchmark tasks, each constructed from the synthetically generated DeepMind Mathematics dataset. To ensure uniformity, we apply an iterative filtering process that removes overrepresented digits samples until the answer distribution across each task closely approximates uniform. This controlled setup ensures that any observed bias stems from the model, not from the benchmark’s label distribution.\n\n---\n**Q2: Clarify the term “first error digits”; what exactly do the values reported in Figure 3 B represent?**\n\nThe term \"first error digit\" refers to the first digit in the model's final generated answer that differs from the corresponding digit in the ground-truth answer. For example, if the correct answer to a problem is \"72\" and the model outputs \"12\", the \"first error digit\" would be \"1\" because it is the first digit in the model's output that deviates from the ground-truth answer. Similarly, if the correct answer is \"144\" and the model generates \"134\", the \"first error digit\" would be \"3\".\n\nThis analysis serves a critical diagnostic purpose: **since the distribution of ground-truth answer digits in our benchmark is explicitly controlled to be uniform, we would expect the distribution of \"first error digits\"—if errors were purely random—to also approximate a uniform distribution.** However, our results show a clear deviation from uniformity: the \"first error digits\" are significantly skewed toward smaller digits and closely follow Benford’s Law.  Therefore, Figure 3 B suggests that digit bias not only influences the model’s overall preferences but also distorts its generation trajectory when deviating from the correct answer, implicating it as a potential driver of numerical hallucinations.\n\n---\n**Q3: Are the logits produced by attaching the model’s final prediction head to each intermediate layer in the logit-lens analysis?**\n\nYes, that is correct. In the logit-lens analysis, we generate the logits for each intermediate layer by multiplying the model's final prediction head—the unembedding matrix—with that layer's output.\n\n---\n**Q4: If early layers also show strong color intensities in Figure 4b, how can the paper claim that digit bias mainly arises in the final layers?**\n\nWe apologize for the confusion, and we appreciate the opportunity to clarify how to interpret Figure 4b.\n\nFigure 4b shows a layer-wise heatmap of normalized digit probabilities. In earlier layers, the model does not exhibit a strong preference for any specific digit, so the normalized probabilities for each digit are approximately uniform, around 0.1. Therefore, the strong color intensities in early layers of Figure 4b reflect that all digits have similar probabilities—around 0.1—due to the model's uncertainty, not due to a strong preference for specific digits.\n\n**Interpret Figure 4b** : Figure 4b focuses on uncertain digit generation steps, defined as those where the digit probability entropy at layer 26 is greater than 3.0. We observe a crucial asymmetry in how different digits are predicted in these uncertain cases:\n- When the model ultimately generates small digits (e.g., digit '1'), the intermediate layers often do not exhibit a strong preference for that digit. Instead, its probability spikes only in the final few layers—indicating that later layers alone are sufficient to push it over the decision boundary.\n\n- In contrast, when generating large digits (e.g., digit ‘7’), the model needs to build up confidence gradually, often requiring a visible preference for the digit already in the mid-layers.\n\nThis behavioral asymmetry suggests that the bias toward small digits like '1' is injected primarily in the final layers. In other words, digit '1' can be “promoted late,” whereas larger digits require sustained evidence from middle layers to be generated at all.\nThus, while early layers show uniform digit probabilities due to high uncertainty, the actual emergence of generation-relevant digit bias—especially the overgeneration of small digits—is concentrated in the final layers. This is consistent with our logit-lens tracing (Section 4.1) and corroborated by neuron-level pruning experiments in Section 5.\n\n---\n**Q5: The paper should provide the exact computation and formulas for tracing samples with equal digit probabilities in intermediate layers, as well as details on how the values in Figure 4b are calculated.**\n\nThank you for your thoughtful suggestion.  We have outlined the specific process below and will include the revision in the new version.\n\n1.  Figure 4b\n\nTo identify uncertain digit generation steps for Figure 4b, we proceed as follows:\n\n\nLet $U \\in \\mathbb{R}^{v \\times d}$ denote the unembedding matrix, where $v$ is the vocabulary size and $d$ is the hidden dimension. Let $h^{(26)} \\in \\mathbb{R}^{d}$ denote the hidden state at Layer 26 for the generation step. We compute the pre-softmax logits via matrix multiplication:$ z = U \\cdot h^{(26)} \\in \\mathbb{R}^{v} $\n\n\nWe apply softmax to obtain the full vocabulary distribution:$ P = \\frac{\\exp(z_i)}{\\sum_{j=1}^v \\exp(z_j)} $\n\n\nWe extract the probabilities corresponding to digit tokens ${0, 1, \\dots, 9}$, and normalize them:$ P^{\\text{(digit)}}i = \\frac{P_i}{\\sum_{j=0}^{9} P_j} \\quad \\text{for } i \\in {0, \\dots, 9} $\n\n\nFinally, we compute the entropy of this digit distribution: $ H = -\\sum_{i=0}^{9} P^{\\text{(digit)}}_i \\cdot \\log P^{\\text{(digit)}}_i $\n\n\nWe define a digit generation step as an uncertain sample if $H \\geq 3.0$, and include it in the analysis for Figure 4b.\n\n\nFor each digit, we identify the uncertain samples where the model outputs that digit, apply logit lens decoding at each layer for those steps, and compute the average predicted probability of that digit. These digit-specific averages across layers form the curves shown in Figure 4b.\n\n\n2. Figure 5\n\nBased on the same logit lens approach described above, we select samples in which the normalized probability $P^{\\text{(digit)}}_i$ of the generated digit at layer 17 satisfies $|P^{\\text{(digit)}}-x|<=0.01$, with $x$=0.12 at layer 17 (Figure 5a), $x$=0.15 at layer 22 (Figure 5b), and $x$=0.20 at layer 25 (Figure 5c). We then plot how these probabilities evolve across subsequent layers.\n\n---\n**Q6:The proposed pruning strategy is coarse and targets bias toward only digit 1, which harms previously correct predictions and limits the method’s generality.**\n\nWe acknowledge the reviewer's concern about the specificity of our pruning strategy and we'd like to take this opportunity to clarify the rationale behind its design.\nThe primary purpose of this intervention was to simultaneously investigate two critical causal links:\n- **The Source of the Bias**: Is the imbalanced distribution of digit-selective neurons the direct cause of the digit bias observed in generation?\n- **The Consequence of the Bias**: Does this digit bias, in turn, act as a mechanistic cause of numerical hallucination?\n\nWhile such a pruning intervention can have unavoidable side effects such as potentially harming previously correct predictions, we believe its primary contribution is not diminished by this. Its value as a highly effective and straightforward way to prove causality remains. By successfully correcting a portion of previously erroneous samples and measurably reducing the output bias toward digit '1', our pruning method provides strong empirical support for our central claim: that digit bias is a mechanistic contributor to numerical hallucination and originates from later layer FFN neurons.\n\nWe thank the reviewer for this valuable point, as it highlights a promising direction for future work. We plan to design more fine-grained or adaptive pruning strategies to mitigate such side effects while retaining the debiasing benefits, as noted in our paper's limitations section.\n\n---\n**Q7: The study addresses only digit-frequency bias, overlooking other possible sources such as digit co-occurrence biases and operation biases.**\n\nWe thank the reviewer for raising this insightful point. However, our work aims to take an important first step in understanding the relationship between data bias and model behavior, focusing on a particularly fundamental and measurable case: digit-frequency bias. Indeed, various forms of numerical bias—such as digit co-occurrence patterns or operation-specific tendencies—are potentially interesting and worth further investigation. We hope our study provides a foundation for broader inquiries into how pretrained models internalize and manifest different types of bias."}, {"review_id": "bxGJE6QqlI", "reviewer": "Reviewer_YkuC", "summary": "Tracing Digit Bias to Numerical Hallucination in LLMs examines why language models hallucinate numbers. Rooted in Benford’s Law, the authors reveal that LLMs overgenerate smaller digits—even with uniform targets—and trace this bias to training data and FFN modules. They introduce a \"neuron pruning\" fix and demonstrate its effectiveness.", "questions": "While the paper introduces a useful quantification of digit bias via Benford’s Law, the core observation that “training data bias leads to hallucinated numeric bias” is not fundamentally new. (and authors acknowledged this point)  \n\nHuman brain also learns from biased inputs but typically develops internal models that avoid hallucination—this points toward architectural issues in LLMs rather than training data alone. The proposed FFN pruning strategy, while empirically effective in this narrow context, feels ad hoc and may not generalize to broader settings of numeric reasoning or language generation. \n\nDoes pruning affect non-numerical reasoning? Would there be any structural mitigations?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nProblem definition and solutions are clearly stated.\nThorough analysis tracing bias source to FFN layers via logit lens and neuron selectivity. \nOffers a lightweight intervention (pruning few neurons) to correct bias experimentally.\n\nWeaknesses\n\nFocused *only* on digit bias; does not link findings to broader numeric reasoning or real-world LLM applications.\n\nThe mitigation strategy (pruning neurons) lacks evaluation on whether it affects other capabilities.", "was_revised": true, "final_justification": "The authors have provided additional clarification on the generalizability of the proposed method, which is an important aspect of the work. Furthermore, their justification for adopting a data-centric approach rather than a structural solution is reasonable and well-argued.", "rebuttal": "We sincerely appreciate the reviewer’s thoughtful evaluation and valuable feedback on our work. Below, we aim to address the specific concerns and suggestions raised.\n\n---\n**W1: Focused only on digit bias; does not link findings to broader numeric reasoning or real-world LLM applications.**\n\nWe thank the reviewer for this crucial point regarding the link to broader reasoning and applications. We agree that making this connection explicit is vital. In fact, our proposed Digit Bias Benchmark (Section 3.2) includes seven distinct categories of numerical reasoning tasks, such as arithmetic operations, root approximation, linear equation solving, and sequence prediction. Although we have not yet conducted explicit real-world application evaluations, we believe that the observed digit bias is highly likely to manifest in real-world applications as well. This belief is grounded in the breadth and diversity of tasks covered by our benchmark, which simulates many of the numerical reasoning demands encountered in practical settings. We agree that testing on real-world benchmarks is indeed crucial. Therefore, we provide the corrected proportion results on the following benchmarks, where Qwen and Gemma show minimal improvement due to their already high accuracy on SVAMP.\n| Benchmark | Llama2-7B | Mistral-7B | Qwen2.5-7B | Gemma2-9B |\n|:---------:|:---------:|:----------:|:----------:|:---------:|\n| GSM8K     |   2.35%   |    1.97%   |    2.12%   |   0.08%   |\n| SVAMP     |    1.6%   |    1.4%    |    0.7%    |    0.5%   |\n\n---\n**W2: The mitigation strategy might affect other capabilities in LLMs.**\n\nWe thank the reviewer for the thoughtful question. Our pruning strategy is specifically designed to target digit bias, and therefore, we apply it only at the digit generation step, as this is the only stage where mitigation is necessary. This design choice is not merely to avoid unintended side effects, but rather reflects the task-specific nature of the intervention—we do not expect the debiasing mechanism to be relevant outside digit prediction.\n\n\nThat said, we acknowledge the importance of assessing broader impacts. To this end, we evaluated the pruned model on two general NLP benchmarks:TruthfulQA and MMLU, and present the results below to demonstrate that overall model performance remains largely unaffected. (To ensure the robustness of our evaluation, we selected the non-numerical tasks anatomy, electrical engineering, formal logic and philosophy from the various MMLU task categories for testing.)\n\n|  Benchmark  |   Llama2-7B   |  Mistral-7B |   Qwen2.5-7B  |   Gemma2-9B   |\n|:-----------:|:-------------:|:-----------:|:-------------:|:-------------:|\n| TruthfulQA  | 23.75 (-0.12) | 48.23 (0.0) | 63.28 (+0.12) | 74.05 (0.0)   |\n| MMLU        | 47.41 (-0.08) | 53.69 (-0.21) | 67.36 (-0.28)   | 66.95 (0.0) |\n\n---\n**Q1:Training data bias causes hallucinated numeric bias is not fundamentally novel.**\n\nTo the best of our knowledge, the most related study (cited in our Related Works) is [1] (referred to as IPTF), which primarily demonstrates that higher term frequencies in the data correlate with better few-shot reasoning accuracy—a frequency-to-accuracy relationship. In contrast, our work focuses on the relationship between frequency and generation bias. The table below highlights the differences between the two papers.\n\n|          Aspect         |                                                           Benford’s Curse                                                          |                                                       IPTF                                                       |\n|:-----------------------:|:----------------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------:|\n| Research Focus          | Investigates digit-level bias in LLMs leading to hallucinated numerical errors, rooted in Benford-like distributions.              | Investigates term-level frequency effects on reasoning accuracy.                                                 |\n| Main Hypothesis         | Pretraining data's digit bias is learned and causes systematic numerical errors in generation.                                     | High-frequency terms in pretraining data lead to higher accuracy in reasoning tasks.                             |\n| Investigated Phenomenon | The focus is on the model's propensity to generate specific single digits (0-9), particularly a preference for lower-value digits. | The focus is on how the frequency of entire numbers (e.g., \"24\" vs. \"23\") as operands relates to model accuracy. |\n| Methodology             | - Designed digit-balanced benchmarks to isolate bias. - Analyzed internal digit probabilities via Logit Lens.                      | - Correlational analysis between number frequencies and accuracy. - No internal model probing or intervention.   |\n| Findings                | 1. LLMs prefer smaller digits like in Benford's Law.  2. Bias localizes to a few FFN neurons.                                      | 1. Accuracy is correlated with number frequency. 2. Frequency effects persist across model scales.               |\n\n---\n**Q2: 1. The digit bias issue might lie in LLM architecture rather than training data and 2. the proposed FFN pruning method feels ad hoc and unlikely to generalize to broader tasks.**\n1. We thank the reviewer for the thoughtful point. While we agree that architectural improvements could, in principle, mitigate biases more effectively, our focus is on understanding how such biases arise and persist in current state-of-the-art LLMs. Architectural changes are non-trivial to implement, often lack interpretability, and may introduce new trade-offs. In contrast, starting from data allows us to isolate and analyze how distributional skew is internalized, and to identify specific mechanisms—such as a small set of FFN neurons—responsible for bias amplification. This data-driven approach does not preclude architectural solutions. Rather, it provides actionable insights into where and how bias is encoded, offering a foundation for principled architectural design in the future.\n\n2. As we mentioned in W2, since a single neuron may serve multiple functions, to avoid affecting other tasks, we activate this intervention only during the digit generation step. Therefore, this approach can generalize to a broader range of tasks, as demonstrated by the experiments in W2.\n\n---\n**Q3: 1. Does pruning affect non-numerical reasoning? 2. Would there be any structural mitigations?**\n\n1. As illustrated in W2 and Q2.2\n.\n2. We believe that the observed digit bias phenomenon is largely due to the presence of a significant number of meaningless digits in the pretraining corpus. For example, digits that appear frequently without any meaningful context, such as ISBN numbers, telephone numbers or postal codes, can skew the model's learning process. If the pretraining corpus were composed of high-quality, contextually relevant numeric reasoning digit, we believe that even a non-uniform distribution of digits would not lead to this issue. In fact, [2] demonstrated that selectively ignoring meaningless tokens during pretraining significantly improved the model's mathematical capabilities. We analyzed the tokens excluded in that study and found that a large portion of the ignored tokens were meaningless digits, further supporting our hypothesis. Therefore, one potential structural mitigation strategy is to modify the model architecture to avoid learning from meaningless digits. This could be achieved through architectural improvements, such as attention mechanisms that can dynamically identify and suppress the influence of meaningless digits, or by embedding constraints that prevent such digits from being used in critical tasks like numeric reasoning. These approaches would allow the model to focus more effectively on meaningful numeric patterns while minimizing the impact of noisy or irrelevant digits.\n\n[1] Impact of Pretraining Term Frequencies on Few-Shot Reasoning, EMNLP 2022      \n[2] Rho-1: Not All Tokens Are What You Need, Neurips 2024"}, {"review_id": "6197eQUL86", "reviewer": "Reviewer_TDjj", "summary": "This paper investigates the origin of digit bias in LLMs - the empirical observation that LLMs are more likely to be accurate on mathematical problems when the leading digit is small. The authors link this observation in LLMs to observations from naturalistic text data, where it's been shown empirically that leading digits in written text are more likely to be sampled from smaller numbers (Benford's law).  The authors conjecture that the distribution of digits in pretraining data might as well follow Benford's law and argue that this data bias may contribute to digit bias in LLMs. They also design a benchmark of numerical tasks with even digit distribution to further substantiate the bias effects. Using a combination of existing and novel interpretability techniques they further diagnose the origin of the digit bias in the model components, and propose a neural pruning strategy based on identifying selective neurons to mitigate these bias effects.", "questions": "1. Have you conducted any experiments/analyses on any publicly released mathematical benchmarks and do you have any findings that you could share that extend your current results?\n\n2. The digit distribution of pretraining corpora has an interesting trend deviation for digits 7+ (and particularly 9) compared to the trends predicted by Benford's law, almost resembling mild U-shape. Do you have any guesses/ideas why this is the case?\n\n3. Your selected approach to mitigating digit bias involves identifying neurons and pruning them - have you also considered how model post-training could affect/mitigate bias in the pretraining data? If not, can you speculate on whether this could be an effective approach?\n\n4. You highlight the best improvement in Table 2 for Qwen2.5-7B - do you have any idea why this particular model benefited the most from neuron pruning?\n\n5. In numerical cognition a phenomenon of subitizing is well characterized in humans and animals - the rapid, confident and accurate recognition of small numbers (up to 3 or 4). Do you see any reason to believe that there might be such phenomenon in LLMs beyond data statistics?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths:\n* Well-defined and empirically demonstrated problem of digit bias in LLMs, with extensive and technically sound experiments and analyses to illustrate the bias effects across different types of LLMs, including a controlled benchmark with word math problems ranging in difficulty\n* Comprehensive mechanistic interpretability analyses to study how the bias evolves across different layers/modules in the transformer architecture\n* Proposed neural pruning method based on aforementioned analyses to improve performance on the proposed benchmark, with positive results on the proposed benchmark\n\n### Weaknesses:\n\n* The paper effectively highlights LLM bias through its analyses and experiments, but its relevance to a broader audience is currently limited due to a lack of demonstrated impact on widely used benchmarks like MATH or GSM8K, or downstream applications such as VQA where numerical reasoning might be important. To significantly broaden its impact, the work should incorporate analyses using established benchmarks and demonstrate improved performance more generally, in its current form this paper might be more of interest to an audience at a targeted workshop\n\n* While positive, the suggested mitigation strategy to deal with digit bias is specifically done only with the digit 1, and the results are variable (ranging from 0.18%-5.05% improvement) and model/task-dependent, suggesting limited significance. Also it involves identifying selective neurons (a method that may be susceptible to changes in parameters), limiting generality that may be present in alternative approaches (e.g. better pretraining data curation). It is unclear how and if this would affect previously correct results", "was_revised": true, "final_justification": "Concerns have been addressed with additional experimental data.", "rebuttal": "We sincerely appreciate the reviewer’s time and effort in reviewing our paper. Your thoughtful comments have been invaluable in helping us improve our work. Below, we provide our responses to each of the issues mentioned.\n\n**W1: The paper offers insightful bias analysis, but its impact is constrained without validation on standard benchmarks or practical applications. (Q1: More mathmetical benchmarks)**\n\nWe thank the reviewer for the thoughtful suggestion. As this work is a fundamental study exploring if data bias leads to generation bias and, subsequently, hallucinations in LLMs, our primary focus is on understanding the phenomena, rather than on demonstrating the effectiveness or generalizability of debiasing methods, which denotes a secondary concern at this stage. However, we still agree with the reviewer’s point that evaluating the impact on broader benchmarks would strengthen the paper’s relevance. We have extended our evaluation to include GSM8K and SVAMP. The corrected proportion results are as follows. Qwen and Gemma had already achieved high accuracy on SVAMP, so the improvement was minimal.\n| Benchmark | Llama2-7B | Mistral-7B | Qwen2.5-7B | Gemma2-9B |\n|:---------:|:---------:|:----------:|:----------:|:---------:|\n| GSM8K     |   2.35%   |    1.97%   |    2.12%   |   0.08%   |\n| SVAMP     |    1.6%   |    1.4%    |    0.7%    |    0.5%   |\n\n---\n**W2: The mitigation strategy is limited to digit 1, with modest and task-dependent improvements. The method's reliance on selective neurons may hinder generalizability, and its impact on previously correct results remains unclear.**\n\nWe would like to discuss them one by one to clarify our methodology, the interpretation of our results, and the overall contribution of our work.\n\n1. **On the Strategy Being Limited to Digit '1'.** We acknowledge the roughness of our approach. However, this focus was driven by the main objective of our study: to investigate the origin of digit bias and its potential causal role in numerical hallucination. While the method may appear coarse, it remains simple and effective for validating our central hypothesis. Also, the intervention is technically not limited to digit ‘1’. In fact, we have tested the cosine similarity between digits in the embedding space and found that the closer two digits are, the higher their similarity (for example, in LLaMA2, the cosine similarities for digit 1 with 0-9 digits are: 0.68, 1, 0.78, 0.72, 0.68, 0.68, 0.65, 0.64, 0.62, 0.62). Furthermore, we observed that a neuron's preference score towards each digit follows a unimodal distribution, meaning that when we prune neurons for digit '1', we are effectively intervening in the direction of smaller digits, not just digit '1'. Therefore, while our current focus is on digit '1', the underlying mechanism we address extends beyond this digit and has broader implications for smaller digits in general.\n\n2. **On the Variable Results and Their Significance.** The observed variability across tasks and model is likely due to the different difficulty levels and the performance characteristics of the models. For instance, in the Nearest Integer Root task with LLaMA2, pruning resulted in a substantial improvement—from answering only two questions correctly to answering six. While this demonstrates a significant relative gain, in this larger-scale task with over two thousand problems, the improvement was only 0.18%. This suggests that the results’ variability is more related to the nature of the tasks and the model's performance on them, rather than the effectiveness of the pruning approach itself.\n\n3. **On the Reliance on Selective Neurons and Generalizability.** We appreciate and agree with the reviewer’s insightful comment. However, in this paper, based on the analyses conducted in Section 4, we believe that identifying selective neurons is a natural and effective method to validate the causal link between digit bias and the imbalanced distribution of selective neurons. This aligns closely with our original research goal, which was to explore the causal mechanism behind digit bias.\n\n4. **On the Unclear Impact on Previously Correct Results.** When pruning is applied excessively, there is a risk of replacing small digits that are used as step numbers or sequence identifiers (e.g., the digit '1' being replaced by newline characters such as \\n\\n). This could lead to unintended errors in the final generated content, affecting the model’s overall accuracy and output.\n\n---\n**Q2: The digit distribution for digits 7 and above (especially 9) in the pretraining corpus shows a U-shape pattern, deviating from Benford's law. What could explain this trend?**\n\nWe hypothesize this is not due to a single factor, but rather a two-fold effect stemming from the composition of our source data and the selection biases.\n\n- Intrinsic Properties of Web and Technical Data: olmo-mix-1124 pre-training dataset is a heterogeneous mixture of diverse data domains. It is primarily composed of web pages from the DCLM-Baseline corpus [1], but is also supplemented with specialized corpora for code and mathematics. These source domains contain inherent numerical patterns—such as years (e.g., 1990s) and commercial prices (e.g., ending in \".99\")—that naturally deviate from a pure Benford's Law distribution. This composition provides the \"raw material\" for the observed over-representation of high digits.\n\n- Pretraining Corpus Processing: The largest component of olmo-mix-1124, DCLM-Baseline, underwent a rigorous selection process using an algorithm to assign a \"quality score\" to a large pool of web documents. Only the top 10% with the highest scores were retained, focusing on structured, information-dense content like tutorials and factual articles. This filtering process likely amplifies numerical patterns, particularly high digits, which might be common in such documents, contributing to the mild U-shaped curve.\n\n---\n**Q3: Has post-training been considered as a potential method to mitigate biases in the pretraining data?**\n\nWe do agree that post-training approaches, such as fine-tuning on a debiased dataset or employing adversarial training to specifically target digit bias, could help mitigate this issue effectively.\n\nDespite this, we ultimately chose pruning as our approach for two key reasons:\n- Causal Evidence: Pruning allowed us to provide direct causal evidence linking digit bias to model behavior, which was central to our study.\n\n- Resource Constraints: Due to limited computational resources (only four RTX 3090 GPUs), conducting post-training or large-scale data curation to pretraining was not feasible within the scope of this work.\n\nWe believe that exploring data curation methods in future studies, in combination with post-training strategies, could further improve the model's ability to mitigate biases.\n\n---\n**Q4: Why Qwen benefited the most from neuron pruning?**\n\nWe think this is because Qwen exhibits a more extreme FFN neuron-level digit selectivity bias compared to the other models. As the comparison in Figure 7 illustrates, the separation between the selectivity score distributions for digit '1' and digit '7' is visibly more pronounced in Qwen2.5-7B (Figure 7b) than in the other tested models (Figures 7a, 7c, and 7d). This suggests its internal digit bias is stronger and more concentrated.\n\nPlausibly, this stronger bias leads the model to produce more errors driven by biased generation, including very low-level mistakes such as miscopying the digits of the problem itself. For instance, when prompted with \"Let s(b) = -149\\*b - 1467. Calculate s(**-10**).\", the model mistakenly answered: \"To calculate  s(**-11**)  for the function ( s(b) = -149b - 1467 )\". Therefore, the debiasing was most impactful on Qwen precisely because it had the most severe digit bias problem, which manifested all the way from the neuron-level to the final generated content.\n\n---\n**Q5: Is there any reason that subitizing exists in LLMs beyond data statistics?**\n\nWe appreciate the inspiring question. If we define subitizing simply as the ability to quickly recognize small quantities without actually counting them, we believe it’s possible that LLMs could exhibit a similar behavior, and here’s why:\n\nWe have done the experiments before: when we provided the LLM with a numerical sequence and asked it to identify the length of the sequence, models like Mistral and Qwen typically did not count the numbers but gave the answer directly. We found that when the sequence length was below 16, the models usually gave correct answers, but when the sequence length increased up to 128, the models often provided an approximate but incorrect answer. This behavior is strikingly similar to the human ability to subitize small numbers and estimate larger quantities.\n\nHowever, it is hard for us to envision how this ability could be directly encoded in the training data or emerge through any other means. A similar study [2] has observed cognitive phenomena such as the SNARC effect and the Distance effect in LLMs, but explaining how these abilities emerge remains a challenge.\n\nFrom a cognitive perspective, subitizing is widely regarded as an innate ability in humans, appearing in infants just a few weeks old [3]. Unlike other cognitive phenomena in LLMs, which are generally influenced by later developmental factors, subitizing is thought to be an inborn skill. Therefore, it is unlikely that LLMs would naturally exhibit this characteristic.\n\nWe believe that further research and experiments are needed to explore whether LLMs possess specialized mechanisms that allow for such rapid and accurate recognition of small quantities.\n\n [1] DataComp-LM: In search of the next generation of training sets for language models   \n [2] Cognitive Effects in Large Language Models, ECAI 2023   \n [3] Are Subitizing and Counting Implemented as Separate or Functionally Overlapping Processes?, NeuroImage Volume 15, Issue 2, February 2002, Pages 435-446"}]

Author rebuttal:

Meta-review: This paper shows that LLMs exhibit systematic digit bias following Benford's Law patterns from pretraining, in particular demonstrating causal relationship between training data statistics and numerical hallucinations through logit-lens analysis and neuron-level interventions. The work provides technical rigor in connecting corpus statistics to model failures. The authors are encouraged to address generalizability issues as well as the intervention scope in the camera-ready version.
Decision: Accept (poster)