Venue: NAACL 2025

# LLM The Genius Paradox: A Linguistic and Math Expert's Struggle with

Simple Word-based Counting Problems

 Nan Xu, Xuezhe Ma

University of Southern California

{nanx,xuezhema}@usc.edu

###### Abstract

Interestingly, LLMs yet struggle with some basic tasks that humans find trivial to handle, e.g., counting the number of character \(r\)'s in the word "strawberry". There are several popular conjectures (e.g., _tokenization_, _architecture_ and _training data_) regarding the reason for deficiency of LLMs in simple word-based counting problems, sharing the similar belief that such failure stems from model pretraining hence probably inevitable during deployment. In this paper, we carefully design multiple evaluation settings to investigate validity of prevalent conjectures. Meanwhile, we measure transferability of advanced mathematical and coding reasoning capabilities from specialized LLMs to simple counting tasks. Although specialized LLMs suffer from counting problems as well, we find conjectures about inherent deficiency of LLMs invalid and further seek opportunities to elicit knowledge and capabilities from LLMs that are beneficial to counting tasks. Compared with strategies such as _finetuning_ and _in-context learning_ that are commonly adopted to enhance performance on new or challenging tasks, we show that engaging _reasoning_ is the most robust and efficient way to help LLMs better perceive tasks with more accurate responses.

We hope our conjecture validation design could provide insights into the study of future critical failure modes of LLMs. Based on challenges in transferring advanced capabilities to much simpler tasks, we call for more attention to model capability acquisition and evaluation. We also highlight the importance of cultivating consciousness of "reasoning before responding" during model pretraining. 1

Footnote 1: Codes and datasets are available at [https://github.com/xunannancy/LLMParadox.git](https://github.com/xunannancy/LLMParadox.git).

0
Footnote 2: footnotemark:

## 1 Introduction

Recently, large language models (LLMs) are able to achieve human-level performance on tasks such as complex reasoning, taking proficiency exams, code generation, multilingual understanding, and math problem solving LlamaWebsite (2024); OpenAI (2024). They even obtain expert-level performance on more challenging tasks like Mathematical Olympiad DeepMind (2024).

Surprisingly, LLMs yet struggle with some basic tasks that are easy or trivial for humans to handle, where no extensive word knowledge or complicated reasoning is required Ball et al. (2024); Shin and Kaneko (2024); Yehudai et al. (2024). For instance, GPT-4o generates a wrong answer to the questions of counting the number of character \(r\)'s in the word "strawberry" Karpathy (2024b).

The research community has discussed actively over the mysterious reason for such unexpected failures. The top-voted conjecture attributes such deficiency in counting characters to the _subword tokenization algorithm_ adopted by prevalent LLMs Shin and Kaneko (2024); Karpathy (2024b). Other researchers speculate that LLMs haven't seen _sufficient character-level data_ during pretraining, hence lack the capability to understand character-level relationships Shin and Kaneko (2024). Yehudai et al. (2024) theoretically proved that the capability of transformer-based models to count characters is _constrained by their embedding size_, and the more unique characters in words the worse performance achieved by LLMs. All the prior conjectures suggest that the deficiency of LLMs in solving easy word-based counting tasks originates from the design of LLM systems (i.e., tokenization or model size) or the pretraining procedure (i.e., lack of character-level training), hence **inevitable** during model deployment.

Considering the broader impacts of word-based counting tasks on important research areas such as morphological analysis Shin and Kaneko (2024), we focus on investigating validity of above conjectures on LLM failures by carefully designing multiple evaluation settings:



**Conjecture I:** By evaluating LLMs with character-level perturbation and explicit character tokenization rather than the default subword input, we do not observe noticeable performance improvement of LLMs, hence invalidating the conjecture regarding _subword tokenization_.

**Conjecture II:** We consider classification tasks (e.g., sentiment analysis) that LLMs are proficient in with natural input. With input phrased by rarely seen character format instead, we observe performance well beyond random guess. This implies strong capability of LLMs in performing character-level reasoning, which conflicts with _lack of character-level training_ conjecture.

**Conjecture III:** We conduct comprehensive analysis on the impact of unique characters within queried words, but observe no clear connection to LLM performance, implying invalidness of the conjecture regarding _model size_.

We further evaluate specialized models, such as Qwen2 Math [34] and CodeGemma [19], aiming to measure transferability of advanced mathematical and coding reasoning capability to simple word-based counting tasks. Unfortunately, neither math nor coding LLMs are able to improve performance over their base model trained on general domains in the open-ended setting. In contrast, python codes explicitly requested from coding LLMs can complete counting tasks with perfection. Such undesired failure from powerful specialized LLMs trained on way more challenging task data calls for more research in training strategies for capability acquisition and benchmark construction for comprehensive capability evaluation.

Motivated by invalidation of prevalent conjectures regarding inherent deficiency of LLMs, we seek opportunities to elicit knowledge and reasoning capabilities from LLMs that are beneficial to simple counting tasks. Specifically, we evaluate effectiveness of strategies commonly used to enhance LLM performance on new or challenging tasks, i.e., widely adopted reasoning methods such as _chain-of-thought_[50] and _self-consistency_[68], finetuning on counting data as well as in-context learning (ICL) [7, 51]. Compared with the other two directions, we find engaging reasoning is the most robust and efficient way to help LLMs better perceive the task and enhance final performance. With the aid of reasoning, GPT-4o is able to address all studied counting tasks with perfection, which is consistent with the core idea of "complex reasoning before responding" underlying recently announced model OpenAI o1 [35].

In summary, we analyze existing conjectures over failure modes of LLMs on simple word-based counting problems. We hope our conjecture validation procedure could also give insightful guidance in studying other unsolved deficiencies of LLMs such as the lost-in-the-middle phenomenon [13], distraction by irrelevant context [52, 53], etc. We also show inability of specialized math or coding LLMs to transfer advanced capabilities to much simpler tasks, calling for more attention and research in model capability acquisition during training and comprehensive capability evaluation during benchmarking. Lastly, we find effectiveness of reasoning strategies to help elicit knowledge and problem-solving capabilities from LLMs, highlighting importance of cultivating consciousness of reasoning during model pretraining.

## 2 Background

We introduce **related work** in Appx. SSA.1.

### Tokenization

_Word_-based tokenization algorithms used in earlier non-transformer models such as Word2Vec [26], FastText [10] and GloVe [24], split texts into words (probably with some extra rules) and find numerical representation for each of them. Words that are unseen in the training corpus or ignored due to limited vocabulary size are typically represented by an unknown token, hence models lose their sensible information. On the contrary, _character_- and _byte_-based tokenization algorithms lead to much smaller vocabularies and far fewer out-of-vocabulary tokens by splitting texts into characters (e.g., CharBERT [18] and Char2Subword [1]) and bytes (e.g., Canine [17] and

Byt5 (Xue et al., 2022)), respectively. However, character- and byte-based token representations are less meaningful with sequence length drastically expanded, posing the challenging efficiency issue for modeling.

To take advantage of both worlds, _subword_ tokenization algorithms, such as Byte-Pair Encoding (BPE) (Sennrich, 2015), WordPiece (Wu, 2016) and UnigramLM (Kudo, 2018), decompose rare words into meaningful subwords while keep frequently used words intact. With the aid of open-source fast tokenization tool itktoken (OpenAI, 2024), BPE has become the dominant tokenization algorithm adopted by recent large language models like Llama 3 (Dubey et al., 2024) and GPT-4o (OpenAI, 2024). We visualize how tokenizers utilized by LLMs split the text differently, _How many r's in the word "strawberry"_, as follows.

**GPT-4o:** How many r s in the word

**Llama 3:** How many r s in the word

**Clema 1:** How many r s in the word

**Mistral v0.3:** How many r S in the word

**DeepSee V2:** How many r S in the word

**Yi 1.5:** How many r s is the word

**Tai w1.5:** How many r s is the word

**Ryan berny ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?? ? ? ? ? ? ? ?? ? ? ? ? ?? ? ? ? ? ? ? ?? ? ? ? ?? ? ? ? ?? ? ? ?? ? ? ?? ? ? ?? ? ?? ? ?? ? ? ?? ? ?? ? ?? ? ?? ? ?? ? ?? ? ?? ?? ? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ??? ??? ?? 

GSM8K.

In this work, we mainly focus on the English domain, but observe similar issues (shown in Tab. 8) when asked character occurrence questions in words from other Germanic (i.e., German and Swedish) and Romance (i.e., French, Spanish, Italian and Portuguese) Languages. We leave _multilingual_ analysis for future work.

Besides static prompts used for four counting tasks, we find that _prompt engineering_ does not enhance llm capabilities to circumvent such failure modes. We study two types of prompts per task conveying similar semantic meanings to evaluate effectiveness of prompt engineering: 1) four paraphrased prompts created by human heuristics and 2) one improved prompt provided by Claude 3. We provide detailed analysis in Appx. SSB.

Footnote 3: [https://www.anthropic.com/news/prompt-improver](https://www.anthropic.com/news/prompt-improver)

## 4 Why LLMs Struggle with Simple Counting Problems

There are three major conjectures trying to explain LLMs deficiency in simple word-based counting problems, detailed as follows:

**Conjecture I: Tokenization Issues**

As introduced in SS2.1, subword tokenization has become the dominant algorithm to convert text into numerical representations, making it challenging to perceive intrinsic characteristics and nuances of individual characters within words [75]; Shin and Kaneko (2024). Moreover, the relationship between individual characters within a word can hardly be captured by the attention mechanism.

**Conjecture II: Lacking Character-level Training**

Existing large language models are mainly pre-trained and post-trained on word-level data [69, 68, 67, 66], hence not optimized for tasks that require character-level analysis.

**Conjecture III: Excessive Unique Characters within Words**

Recent literature theoretically proves that the capability of transformers to count letters in words is upper bounded by their embedding size [76]. Empirically they find that Gemini [67] tends to make more mistakes when distinct characters in words increase.

In this section, we verify the above conjectures one-by-one by designing characteristic settings and comparing LLM performance with the default one introduced in SS3.

### Conjecture I: Tokenization Issues

To verify the conjecture "LLMs fail on simple word-based count problems due to the subword tokenization", we

\(\bullet\)_Design settings where the tokenizer has to implicitly or explicitly split texts into **characters** rather than subwords._

\(\bullet\)_Check the performance of LLMs on four counting tasks within the new tokenization._

\(\bullet\)_Analyze the implications:_

\(1)\) _If we observe noticeable improvement, then the conjecture is correct;_

\(2)\) _If the performance maintains similar or even degrades, then the conjecture is invalid._

SettingsTo expose more character-level information of the studied word to LLMs, we follow literature that studies robustness of NLP models [69, 68, 67] by conducting character-level word perturbations: 1) delete, 2) insert, 3) repeat, 4) replace, 5) swap, 6) left shift, 7) right shift, 8) shuffle, 9) map

\begin{table}
\begin{tabular}{l c c c c c c c c} Task & GPT-4o & Llama 3 & Qwen 1.5 & Gemma 1 & InternLM 2 & Phi 3 & Mistral v0.3 & DeepSeek V2 & Yi 1.5 \\ \hline
**I: Char Occur** & 82.4 & 34.6 & 30.6 & 41.2 & 60.8 & 39.0 & 35.4 & 27.2 & 46.6 \\
**II: Substring Occur** & **87.4** & 58.2 & 58.9 & 50.8 & 50.3 & 73.0 & 57.7 & 61.5 & 59.6 \\
**III: Word Len** & **92.0** & 74.6 & 42.4 & 26.0 & 55.8 & 64.4 & 41.0 & 36.6 & 58.0 \\
**IV: Distinct Char** & **89.2** & 57.8 & 27.8 & 4.4 & 21.8 & 70.2 & 34.4 & 28.4 & 36.6 \\ \hline
**MMLU** (0-shot) & 85.0 & 64.2 & 58.5 & 50.4 & 59.2 & 75.7 & 59.3 & 53.0 & 67.0 \\
**GSMSK** (0-shot) & 86.3 & 78.9 & 58.0 & 38.6 & 67.9 & 82.2 & 47.8 & 70.6 & 81.8 \\ \end{tabular}
\end{table}
Table 1: Performance of LLMs on simple word-based counting problems, as well as on the general benchmark MMLU and the math benchmark GSM8K for comparison. Both open-source and proprietary LLMs struggle with answering the correct numbers (i.e., Task I, III and IV) or identifying existence of subtrings in words (i.e., Task II). After marking counting accuracy (\(2\text{nd}\) to \(5\text{th}\) rows) higher than general and math benchmarks (bottom two rows) in **boldface**, we find LLMs can hardly achieve much better performance than that on the more challenging benchmark MMLU and GSM8K.

 

ping to alphabetical character, and 10) mapping to special character 4. Besides, we also manually split characters among the word without altering the final answer by adding special characters in between: 1) dash, 2) space, and 3) comma. In Tab. 9, we present an example for every character-level word perturbation method.

Footnote 4: The answers may change after character-level perturbations like _delete_, _insert_, _repeat_, and _replace_, while keeping unaltered after perturbations such as _swap_, _left/right shift_, _shuffle_ and _alphabetically/special mapping_.

Beyond above implicit character-level tokenization, we explicitly interfere with the tokenization process during inference so that the studied word is tokenized into a list of individual character tokens while other words are tokenized to subwords.

ResultsIn Fig. 1, we demonstrate performance comparison when LLMs are provided with the original subword tokens and our proposed implicit (top) or explicit (bottom) character tokens. Noticeably, LLMs do not benefit from inputs represented by either implicit or explicit character tokens to better perceive character-level information of key words, leading to similar or even worse performance than that given subword input. Therefore, we empirically **refute** the popular conjecture that the subword tokenization leads to LLMs failure in word-based counting tasks.

### Conjecture II: Lacking Character-level Training

In order to validate the correctness of the conjecture, "LLMs haven't trained on sufficient character-level data, hence lack ability to understand and handle tasks requiring character-level reasoning," we

\(\bullet\)_Evaluate performance of LLMs on tasks they are proficient in, but with character input._

\(\bullet\)_Compare the performance of LLMs between natural text and the rarely seen format of character input._

\(\bullet\)_Analyze the implications:_

\(1)\) _If we observe significant performance drop, then the conjecture is correct;_

\(2)\) _If the performance maintains similar or become slightly worse, then the conjecture is invalid._

SettingsWe consider three sentiment analysis benchmarks where LLMs are able to achieve much

Figure 1: Performance comparison on **Task I** with different tokenization strategies to verify **Conjecture I** that deficiency in word-based counting is caused by subword tokenization of LLMs.**Top:** There is no perturbation that benefits all studied models identifying core character-level information (above \(0\)). **Bottom:** Directly feeding character tokens instead of subword tokens does not help LLMs perceive individual characters within words. Above comparison as well as results on other three studied tasks shown in Fig. 7 refuse the conjecture regarding LLM tokenization.

 

higher accuracy than random guess in zero-shot setting: _1) Emotion_: a dataset of \(2000\) English Twitter messages with six basic emotions: anger, fear, joy, love, sadness, and surprise Saravia et al. (2018); _2) IMDB_: a movie review dataset 5 for binary sentiment classification Maas et al. (2011); _3) SST-2_: \(872\) single sentences 6 extracted from movie reviews for binary classification Socher et al. (2013). We present sentiment classification tasks as multiple-choice questions to LLMs, with options randomly ordered per question to avoid model bias towards specific options.

Footnote 5: We randomly sample \(500\) instances from the \(25\)k testing set for efficient inference. Note that we drop reviews containing more than \(4000\) characters to ensure the input within the context length of all LLMs (i.e., \(8192\)).

Footnote 6: We evaluate on the validation set since labels on the testing set are not publicly available.

ResultsWe demonstrate performance comparison between natural word and character input in Fig. 2. Without further tuning, all studied LLMs can perform sentiment analysis with accuracy above \(90\%\) on binary classification and above \(50\%\) on 6-way classification. Meanwhile, we observe minor performance drop when input format switches from natural words to rarely seen characters, which is still well above random guess performance. This suggests that pretrained LLMs have the capability to perform character-level reasoning, although similar data is not sufficiently seen during model pretraining or fine-tuning. Therefore, we **deny** the conjecture that deficiency of LLMs in simple word-based counting tasks is attributed to lack of training on similar data.

### Conjecture III: Excessive Unique Characters within Words

Yehudai et al. (2024) regard the counting problem as a more difficult one compared with the popular "needle in haystack" Kamradt (2023); Ivgi et al. (2023), since the former requires considering multiple occurrences of a given string, while the latter aims to retrieve only one appearance in a long context. They further find that _the more unique characters showing up in the string, the more challenging for transformer-based LLMs to count the occurrence_. On the contrary, _model performance is barely sensitive to the extension of string length_. We conduct systematic evaluation of model capabilities to handle word-based counting tasks when character uniqueness and total counts vary.

SettingsWe select the closed-source model GPT-4o and open-source model Llama 3, then evaluate on four sets of words by keeping the total number of characters or the number of distinct characters fixed while varying the other: 1) \(500\) words with \(8\) characters, 2) \(500\) words with \(12\) characters, 3) \(500\)

Figure 3: Performance variation from GPT-4o on four tasks. 1**st & 2nd column:** there is no noteworthy correlation between model performance and number of unique characters in queried words. 3**rd & 4th column:** we observe clear trend of performance drop from both models when the word length keeps on increasing since 10. We leave similar trend from Llama 3 in Fig. 9.

Figure 2: Performance comparison of LLMs between natural word and character input on classification dataset _IMDB_. Although questions represented by individual characters are rarely seen, all studied models are able to achieve accuracy much higher than random guess denoted by - The minor performance drop compared with natural word input implies that LLMs have the ability to handle tasks requiring character-level understanding, rejecting the conjecture relevant to lack of character training. Figure 8 shows similar observations on other datasets.

 

words with \(4\) distinct characters, and 4) \(500\) words with \(8\) distinct characters.

ResultsAs shown in Fig. 3, opposite to observations discovered in prior work Yehudai et al. (2024), the increasing number of distinct characters in queried words does not lead to degraded performance on word-based counting problems. Instead, when the total number of characters reaches 10 and keeps increasing, we find obvious accuracy drop in both models. Hence, the conjecture that excessive unique characters in queried words lead to poor word-based counting performance is **incorrect**.

## 5 Whether Math/Code Train Data Helps

Recently, many open-resource base LLMs have been further tuned on billions or trillions Nath (QwenLM, 2024b; Ying et al., 2024; MistralAI, 2024b; Shao et al., 2024) or formal theorem-proving data Wu et al. (2024); Xin et al. (2024) in order to solve advanced mathematical problems that require complex, multi-step logical reasoning. Similarly, quite a few code models QwenLM (2024a; CodeGemmaTeam, 2024; MistralAI, 2024a; Zhu et al., 2024; 01AI, 2024) have been built on top of base LLMs and additionally trained on diverse programming language datasets, demonstrating significant advancements in various aspects of code-related tasks such as code generation Chen et al. (2021); Austin et al. (2021), completion Liu et al. (2023) and insertion Allal et al. (2023).

In this section, we focus on evaluating whether additional training on mathematical or coding data helps LLMs understand and improve reasoning over word-based counting tasks.

ResultsWe provide detailed introduction to **evaluated models** and **implementation details** in Appx. SSC.1. We visualize performance of models with different capabilities in Fig. 4. We observe that models additional trained on mathematical reasoning can not bring obvious improvement over those trained on general-domain data. This indicates that their acquired reasoning capability over math problems is not sufficient to handle word-based counting tasks. On the other side, code models are able to solve the counting tasks successfully when prompted to generate Python codes explicitly, suggesting that the studied tasks are of easy level. Interestingly, the powerful code models fail when prompted in open-ended setting, implying that they do not distill problem-solving capabilities during training on code-specific tasks.

Although specialized LLMs substantially enhance coding or mathematical reasoning capabilities over general LLMs, they still struggle in solving easy word-based counting problems that require easy-level reasoning.

## 6 How to Make LLMs Experts Again

As we have verified in SS4, the popular conjectures, such as tokenization and lack of character-level training, are not the true barriers for LLMs to solve counting tasks. Meanwhile, LLMs achieve competitive performance on far more challenging reasoning Clark et al. (2018); Zellers et al. (2019); Rein et al. (2023) and mathematical Cobbe et al. (2021); Hendrycks et al. (2021) benchmarks. Therefore, we believe LLMs possess the knowledge and skills to solve counting problems if guided properly. We investigate whether reasoning strategies Wei et al. (2022); Wang et al. (2022); Maadan et al. (2024); Sprague et al. (2024); Yao et al. (2024) could elicit strong capabilities from LLMs to help perceive, reason and finally solve the problem.

Reasoning StrategiesWe investigate the following reasoning methods that have demonstrated

Figure 4: Performance comparison among LLMs trained on general and special domain data for _Task II_. Models specialized in mathematical reasoning () or theorem proving () can not better handle word-based counting tasks than general () ones. Although code models are able to write Python codes () and solve tasks successfully, they fail to reason and answer accurately in open-ended setting () . We leave results containing similar observations on three other tasks in Fig. 10.

 

great improvement in math and reasoning (Sprague et al., 2024): _1) CoT_: chain-of-thought (Wei et al., 2022) encourages models to reason before providing the final answer, which becomes the de facto method for eliciting reasoning capabilities from LLMs. _2) self-consistency_: first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by majority voting (Wang et al., 2022). _3) self-refine_: uses a single LLM as the generator, refiner, and feedback provider (Madaan et al., 2024). _4) ToT_: tree-of-thought actively maintains a tree of thoughts, where each thought is a coherent language sequence that serves as an intermediate step toward problem solving (Yao et al., 2024). In contrast, we also append the instruction, "Directly answer the number" after each question, to request direct numeric answers from LLMs.

Other StrategiesIn supervised finetuning (SFT), collecting and mixing instruction tuning data are important steps to improve performance for specific capabilities Dubey et al. (2024). Hence we **finetune** open-source LLMs with task-specific train data 7 and evaluate on both in-distribution test data and widely adopted benchmarks. By providing similar examples as context, **in-context learning** (ICL) Brown (2020); Wei et al. (2023) has become another popular train-free method to efficiently improve LLM performance. We describe **implementation details** in Appx. SSD.1.

Footnote 7: Motivated by benefits of reasoning procedures to counting tasks demonstrated in §6.1, we provide detailed reasoning before correct answers in the ground-truth responses.

### Reasoning

In Fig. 5, we compare diverse reasoning strategies introduced before with baseline strategies, i.e., directly responding with numeric values and open-ended generation. We find that all studied reasoning approaches are helpful to greatly improve performance over those without reasoning across four counting tasks, among which _self-consistency_ exhibits consistent advantage over other reasoning strategies for diverse LLMs. In addition, we show **scaling law** of _self-consistency_ in Fig. 12, where no clear trend of performance boost as utilizing more reasoning paths is observed 8. We provide case study from baseline strategies and CoT in Tab. 10.

Footnote 8: The observation is quite different from that shown in Wang et al. (2022), where using \(40\) reasoning paths achieved the best performance by LaMDA-137B and GPT-3 code-davinci-001. We speculate that the model scale and task difficulty level may be the major reason.

With the aid of reasoning procedures, the most powerful model GPT-4o is capable of solving

Figure 5: Benefits of applying different reasoning strategies to LLMs for _Task I Char Occur_. We observe noticeable improvement from all studied reasoning strategies over baselines that particularly request numeric responses () or open-ended answers (). GPT-4o with the additional reasoning procedure can even solve tasks with perfection. We show similar improvement on other three tasks in Fig. 11.

 

counting tasks with accuracy approaching \(100\%\), indicating that the model can leverage its possessed knowledge and problem-solving abilities individually without external assistance. We also notice considerable performance margin from some LLMs between directly answering numerical value and open-ended generation, implying that they consciously invoke the reasoning process before providing the final answer for certain instances. We expect performance improvement in future LLMs if reasoning-related training is strengthened.

### Supervised Finetuning

In Tab. 2, we evaluate capabilities of Llama 3 models finetuned on different task data. When training and testing on the same distribution, we do observe significant accuracy boost in _Task I_ (from \(34.6\%\) to \(70.4\%\)) and _Task IV_ (from \(57.8\%\) to \(87.2\%\)), while minor performance drop on the other two tasks. However, the acquired specific counting capability from training on _Task I_ or _IV_ can hardly transfer to other evaluated counting tasks, resulting in even much worse performance than the untuned model. Moreover, we find undesired lowered accuracy on benchmarks evaluating important capabilities such as reasoning and math, manifesting negative impacts of solely training on specific domains without considering other aspects. This emphasizes the importance of careful design for the proportion of different data sources, which is consistent with discoveries in literature Bai et al. (2023); Dubey et al. (2024) and leaves the finetuning strategy a less efficient way to improve performance of LLMs on new or challenging tasks compared with reasoning.

### In-context Learning

We demonstrate the influence of demonstrations on counting tasks in Fig. 6. For _Task I_, open-source LLMs achieve much higher accuracy in few-shot settings than zero-shot one, and more demonstrations exhibit further performance improvement. However, benefits of demonstrations are not always guaranteed. For example, additional example context greatly hurts performance of GPT-4o and the majority of open-source LLMs for _Task II_ (in Fig. 12(a)) and _IV_ (in Fig. 5(b)).

## 7 Conclusions

By carefully designing multiple evaluation settings, we first show that prevalent conjectures regarding such unexpected failures are invalid. We further show that specialized models with advanced mathematical or coding reasoning capabilities also suffer from addressing simple counting problems. We also find that _reasoning_ is the most robust and efficient way to aid models in better perceiving and solving tasks, highlighting more research into "reasoning before responding" during pretraining.

Figure 6: In-context learning performance of LLMs. Providing similar examples as demonstrations helps slightly improve performance of open-source models on Task I, while lead performance drop for most models on Task IV. We leave results on Task II and III in Fig. 13.

 



### Limitations

We investigate deficiency of diverse open-source LLMs as well as GPT-4o to address word-based counting problems. This work may have the following limitations: _1) Lack of analysis on more proprietary LLMs:_ for the sake of cost, we only consider GPT-4o and use it as the representative of other models of similar strong capabilities. Some online discussion has revealed similar issues from closed-source models such as Claude and Gemini. We hope researchers who develop these proprietary models can get insights from our conjecture validation procedure and reasoning-driven solutions, hence further boosting capabilities of top LLMs. _2) Reasoning incorporated in pretraining:_ we find that reasoning before providing the final answer during inference is effective in solving counting problems, while leaving training design of incorporating reasoning into pretraining as future direction.

## Ethics Statement

This paper presents comprehensive study of LLMs from diverse families that have gone through ethical reviews in prior works. Therefore, we believe our work does not pose additional ethical issues.

## References

* O. Abdin, S. Jacobs, A. Awan, J. Aneja, A. Awadallah, H. Awadalla, N. Bach, A. Bahree, A. Bakhtiari, I. Behl, et al. (2024)Phi-3 technical report: a highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219. Cited by: SS5.
* G. Aguilar, B. McCann, T. Niu, N. Rajani, N. Keskar, and T. Solorio (2021)Char2Subword: extending the subword embedding space using robust character compositionality. In Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic, pp. 1640-1651. External Links: Link Cited by: SS1.
* L. Ben Allal, R. Li, D. Kocetkov, C. Mou, C. Akiki, C. Munoz Ferrandis, N. Muennighoff, M. Mishra, A. Gu, M. Dey, et al. (2023)Santacoder: don't reach for the stars!. arXiv preprint arXiv:2301.03988. Cited by: SS1.
* J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dolan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. (2021)Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Cited by: SS1.
* J. Bai, S. Bai, Y. Chu, Z. Cui, Kai Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. (2023)Qwen technical report. arXiv preprint arXiv:2309.16609. Cited by: SS1.
* T. Ball, S. Chen, and C. Herley (2024)Can we count on llms? the fixed-effect fallacy and claims of gpt-4 capabilities. arXiv preprint arXiv:2409.07638. Cited by: SS1.
* L. Berglund, M. Tong, M. Kaufmann, M. Balesni, A. Soercikland, T. Korbak, and O. Evans (2023)The reversal curse: clmns trained on a is b" fail to learn" b is a". arXiv preprint arXiv:2309.12288. Cited by: SS1.
* S. Bird, E. Loper, and E. Klein (2009)Natural language processing with python. O'Reilly Media Inc.. Cited by: SS1.
* P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov (2017)Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics5, pp. 135-146. Cited by: SS1.
* T. B. Brown (2020)Language models are few-shot learners. arXiv preprint arXiv:2005.14165. Cited by: SS1.
* Z. Cai, M. Cao, H. Chen, K. Chen, K. Chen, X. Chen, Z. Chen, P. Chu, et al. (2024)Internlm2 technical report. arXiv preprint arXiv:2403.17297. Cited by: SS1.
* M. Chen, J. Tworek, H. Jun, Q. Yuan, H. Ponde De Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. (2021)Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Cited by: SS1.
* X. Chen, R. A. Chi, X. Wang, and D. Zhou (2024)Premise order matters in reasoning with large language models. arXiv preprint arXiv:2402.08939. Cited by: SS1.
* J. H. Clark, D. Garrette, I. Turc, and J. Wieting (2022)Canine: pre-training an efficient tokenization-free encoder for language representation. Transactions of the Association for Computational Linguistics10, pp. 73-91. Cited by: SS1.
* P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord (2018)Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457. Cited by: SS1.
* K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Juh, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. (2021)Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.



CodeGemmaTeam. 2024. Codegemma: Open code models based on gemma. _arXiv preprint arXiv:2406.11409_.
* DeepMind (2024) DeepMind. 2024. Ai solves IMO problems at silver medal level. Accessed: 2024-10-06.
* DeepSeek-AI (2024) DeepSeek-AI. 2024. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. _Preprint_, arXiv:2405.04434.
* Dubey et al. (2024) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. _arXiv preprint arXiv:2407.21783_.
* Gao et al. (2023) Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Pal: Program-aided language models. In _International Conference on Machine Learning_, pages 10764-10799. PMLR.
* Guo et al. (2024) Daya Guo, Qiaho Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseok-coder: When the large language model meets programming-the rise of code intelligence. _arXiv preprint arXiv:2401.14196_.
* Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. _arXiv preprint arXiv:2103.03874_.
* Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Llora: Low-rank adaptation of large language models. _arXiv preprint arXiv:2106.09685_.
* Iygi et al. (2023) Maor Iygi, Uri Shaham, and Jonathan Berant. 2023. Efficient long-text understanding with short-text models. _Transactions of the Association for Computational Linguistics_, 11:284-299.
* Jiang et al. (2023) Albert Q Jiang, Alexandre Sablayrolles, Arthur Mengsch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lampple, Lucile Saulinier, et al. 2023. Mistral 7b. _arXiv preprint arXiv:2310.06825_.
* Kamradt (2023) Greg Kamradt. 2023. Llmtest_needleinahaystack. Accessed: 2024-10-10.
* Karpathy (2024a) Andrej Karpathy. 2024a. Tweet: Jagged intelligence. Accessed: 2024-10-06.
* Karpathy (2024b) Andrej Karpathy. 2024b. Tweet: To help explain the weirdness of llvm tokenization. Accessed: 2024-10-08.
* Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Michel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. _Advances in neural information processing systems_, 35:22199-22213.
* Kudo (2018) Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. _arXiv preprint arXiv:1804.10959_.
* Liu et al. (2020) Hui Liu, Yongzheng Zhang, Yipeng Wang, Zheng Lin, and Yige Chen. 2020. Joint character-level word embedding and adversarial stability training to defend adversarial text. In _Proceedings of the AAAI conference on artificial intelligence_, volume 34, pages 8384-8391.
* Liu et al. (2024) Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Parangape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. _Transactions of the Association for Computational Linguistics_, 12:157-173.
* Liu et al. (2023) Tianyang Liu, Canwen Xu, and Julian McAuley. 2023. Repobench: Benchmarking repository-level code auto-completion systems. _arXiv preprint arXiv:2306.03091_.
* LjannaWebsite (2024)LlamaWebsite. 2024. Introducing llama 3.2. [https://www.llama.com/](https://www.llama.com/). Accessed: 14-Oct-2024.
* Ma et al. (2020) Wentao Ma, Yiming Cui, Chenglei Si, Ting Liu, Shijin Wang, and Guoping Hu. 2020. CharBERT: Character-aware pre-trained language model. In _Proceedings of the 28th International Conference on Computational Linguistics_, pages 39-50, Barcelona, Spain (Online). International Committee on Computational Linguistics.
* Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_, pages 142-150, Portland, Oregon, USA. Association for Computational Linguistics.
* Madanu et al. (2024) Aman Madanu, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegrefte, Uri Alon, Nouha Dziri, Shimai Prabhumoye, Yuning Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. _Advances in Neural Information Processing Systems_, 36.
* Mikolov (2013) Tomas Mikolov. 2013. Efficient estimation of word representations in vector space. _arXiv preprint arXiv:1301.3781_.
* Mustrial (2024a) MustrialAI. 2024a. Codestral: Mistral ai code generation model. [https://mistral.ai/news/codestral/](https://mistral.ai/news/codestral/). Accessed: 2024-10-10.
* Mustrial (2024b) MustrialAI. 2024b. Mathstral: Mistral ai mathematical model. [https://mistral.ai/news/mathstral/](https://mistral.ai/news/mathstral/). Accessed: 2024-10-10.
* Moradi and Samwald (2021) Milad Moradi and Matthias Samwald. 2021. Evaluating the robustness of neural language models to input perturbations. In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_, pages 1558-1570, Online and Punta Cana 

Dominant Republic. Association for Computational Linguistics.
* OpenAI (2024) OpenAI. 2024. Hello gpt-4. [https://openai.com/index/hello-gpt-4o/](https://openai.com/index/hello-gpt-4o/). Accessed: 2024-10-06. Cited by: SS3.2, SS5.1.
* OpenAI (2024) Learning to reason with lms. Note: [https://openai.com/index/learning-to-reason-with-llms/Accessed](https://openai.com/index/learning-to-reason-with-llms/Accessed): 2024-10-14 External Links: Link Cited by: SS1.
* OpenAI (2024) Tiktoken: efficient tokenization library. Note: [https://github.com/openai/tiktoken/tree/mainAccessed](https://github.com/openai/tiktoken/tree/mainAccessed): 2024-10-06 External Links: Link Cited by: SS1.
* Jeffrey Pennington, Richard Socher, and Christopher Manning (2014) GloVe: global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1532-1543. External Links: Link, Document Cited by: SS1.
* QwenLM (2024a) Code with codeqwen1.5 External Links: Link Cited by: SS1.
* QwenLM (2024b) Qwen2-math: enhancing large language models for mathematical reasoning. Note: [https://qwenlm.github.io/blog/qwen2-math/Accessed](https://qwenlm.github.io/blog/qwen2-math/Accessed): 2024-10-10 External Links: Link Cited by: SS1.
* M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittweiser, et al. (2024) Gemini 1.5: unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 External Links: Link, 2403.05530 Cited by: SS1.
* David Rein, B. Li Hou, A. C. Stickland, J. Petty, R. Yuanzhe Pang, J. Dirani, J. Michael, and S. R Bowman (2023) Gpqa: a graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022. Cited by: SS1.
* E. Abad Rocamora, Y. Wu, F. Liu, G. Chrysos, and V. Cevher (2018) Revisiting character-level adversarial attacks for language models. In Forty-first International Conference on Machine Learning, External Links: Link Cited by: SS1.
* Elvis Saravia, H. T. Liu, Y. Huang, J. Wu, and Y. Chen (2018) CARER: contextualized affect representations for emotion recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 3687-3697. External Links: Link, Document Cited by: SS1.
* Rico Sennrich (2015) Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. Cited by: SS1.
* Z. Shao, P. Wang, Q. Zhu, Runxin Xu, J. Song, M. Zhang, Y. Li, Y. Wu, and D. Guo (2024) Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: SS1.
* Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, D. Dolan, Ed H Chi, Nathanael Scharli, and Denny Zhou (2023) Large language models can be easily distracted by irrelevant context. In International Conference on Machine Learning, pp. 31210-31227. External Links: Link, Document Cited by: SS1.
* Andrew Shin and K. Kaneko (2024) Large language models lack understanding of character composition of words. arXiv preprint arXiv:2405.11357. Cited by: SS1.
* R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts (2013) Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Washington, USA, pp. 1631-1642. External Links: Link, Document Cited by: SS1.
* Z. Sprague, F. Yin, J. Diego Rodriguez, D. Jiang, M. Wadhwa, P. Singhal, X. Zhao, X. Ye, K. Mahowald, and G. Durrett (2024) To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. arXiv preprint arXiv:2409.12183. Cited by: SS1.
* Mirac Suzgun, N. Scales, N. Scharli, S. Gehrmann, Y. Tay, H. Won Chung, A. Chowdhery, Q. V. Le, Ed H. Chi, D. Zhou, , and J. Wei (2022) Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261. Cited by: SS1.
* G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Riviere, M. Sanjay Kale, J. Love, et al. (2024) GEMDA: open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. Cited by: SS1.
* Xuezhi Wang, J. Wei, D. Schuurmans, Quoc Le, E. Chi, S. Narang, A. Chowdhery, and Denny Zhou (2022) Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. Cited by: SS1.
* Jason Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. (2022) Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems35, pp. 24824-24837. Cited by: SS1.
* Jerry Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou, et al. (2023) Larger language models do in-context learning differently. arXiv preprint arXiv:2303.03846. Cited by: SS1.
* Yonghui Wu (2016) Google's neural machine translation system: bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. Cited by: SS1.
* Z. Wu, J. Wang, D. Lin, and K. Chen (2024) Lean-github: compiling github lean repositories for a versatile lean prover. Preprint arXiv:2407.17227.

 

Zikai Xie. 2024. Order matters in hallucination: Reasoning order as benchmark and reflexive prompting for large-language-models. arXiv preprint arXiv:2408.05093. Cited by: SS1, SS2.
* H. Xin, Z. Ren, J. Song, Z. Shao, W. Zhao, H. Wang, B. Liu, L. Zhang, X. Lu, Q. Du, et al. (2024)DeepSeek-prover-v1. 5: harnessing proof assistant feedback for reinforcement learning and monte-carlo tree search. arXiv preprint arXiv:2408.08152. Cited by: SS1.
* L. Xue, A. Barua, N. Constant, R. Alfou, S. Narang, M. Kale, A. Roberts, and C. Raffelt (2022)ByT5: towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics10, pp. 291-306. External Links: Document, Link Cited by: SS1.
* S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, and K. Narasimhan (2024)Tree of thoughts: deliberate problem solving with large language models. Advances in Neural Information Processing Systems36. Cited by: SS1.
* G. Yehudai, H. Kaplan, A. Ghandeharioun, M. Geva, and A. Globerson (2024)When can transformers count to n?. arXiv preprint arXiv:2407.15160. Cited by: SS1.
* H. Ying, S. Zhang, L. Li, Z. Zhou, Y. Shao, Z. Fei, Y. Ma, J. Hong, K. Liu, Z. Wang, et al. (2024)Internlm-math: open math large language models toward verifiable reasoning. arXiv preprint arXiv:2402.06332. Cited by: SS1.
* A. Young, E. Chen, C. Li, C. Huang, G. Zhang, G. Zhang, H. Li, J. Zhu, J. Chen, J. Chang, et al. (2024)Yi: open foundation models by 01. ai. arXiv preprint arXiv:2403.04652. Cited by: SS1.
* R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi (2019)Hellawag: can a machine really finish your sentence?. arXiv preprint arXiv:1905.07830. Cited by: SS1.
* W. Zhong, R. Cui, Y. Guo, Y. Liang, S. Lu, Y. Wang, A. Saied, W. Chen, and N. Duan (2024)AGIEval: a human-centric benchmark for evaluating foundation models. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, New York, NY, USA, pp. 2299-2314. External Links: ISBN 978-1-4503-3213-X, Link Cited by: SS1.
* Q. Zhu, D. Guo, Z. Shao, D. Yang, P. Wang, Runxin Xu, Y. Wu, Y. Li, H. Gao, S. Ma, et al. (2024)Deepsek-coder-v2: breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:2406.11931.

 

Appendix

### Related Work

Failure Modes of LLMsAlthough LLMs have exhibited strong capabilities to complete tasks requiring extensive world knowledge and complex reasoning, they still present some unexpected failures. Berglund et al. (2023) discovered the _reversed curve_, where an LLM that recognizes "A is B" does not necessarily learn that "B is A." Another challenging posed to LLMs is irrelevant context, which distracts models from completing tasks as normal. For instance, Shi et al. (2023) found that adding irrelevant context in the problem statement leads to a noticeable performance drop on multiple reasoning benchmarks. Moreover, Chen et al. (2024) show that including irrelevant rules degrades the logical reasoning performance of LLMs. Sensitivity to text order is another challenge that LLMs struggle with. For example, Chen et al. (2024) observed that in deductive reasoning tasks, presenting the premises in the same order as the ground truth proof in the prompt (as opposed to random ordering) drastically increases the model's accuracy, while permuting the premise order can cause a performance drop of over \(30\%\). Another example is the lost-in-the-middle phenomenon in the long-context scenario, in which LLM performance drops drastically when they need to utilize input context in the middle rather than that in the beginning or the end Liu et al. (2024).

Word-based CountingFailure to count the number of specific character within the queried word is a recently emergent problem that most LLMs struggle with Karpathy (2024). Yehudai et al. (2024) attributed such deficiency to constraints from LLM architecture, emphasizing that it is likely impossible for a size-limited transformer to complete the counting task. Ball et al. (2024) examined capabilities of GPT-4 on character occurrence task and showed sensitivity of task-accuracy both to query phrasing and input parameter population. Shin and Kaneko (2024) observed significant performance contrast between character and token (i.e. subword) input. They also proposed the tokenization issue and lack of training on similar data as potential reasons for such failure.

Different from prior literature that focuses on demonstrating the failure mode or proposing potential reasons, we carefully design multiple evaluation settings and empirically show invalidness of major conjectures. More importantly, we investigate promising strategies and show that reasoning is a promising direction to solve word-based counting problems.

### Experimental Setup

Language ModelsFor comprehensive evaluation of LLMs capabilities on simple word-based counting problems, we consider 9 prevalent families of powerful instructed or chat models including both open-source and proprietary ones: Llama 3 (8B-instruct) Dubey et al. (2024), Qwen 1.5 (7B-chat) Bai et al. (2023), Gemma 1 (7B-instruct) Team et al. (2024), InternLM2 (7B-chat) Cai et al. (2024), Phi 3 (small-128k-instruct) Abdin et al. (2024), Mistral v0.3 (7B-instruct) Jiang et al. (2023), DeepSeek V2 (Lite-chat) DeepSeek-AI (2024), Yi 1.5 (9B-chat) Young et al. (2024), and GPT-4o OpenAI (2024). Unless otherwise stated, we follow prior benchmark literature Suzgun et al. (2022); Zhong et al. (2024) by adopting greedy decoding 9 to minimize the noise for open-ended text generation. We list the checkpoint resource of tested open-source LLMs in Tab. 7.

Footnote 9: We set the maximum number of tokens for generation to 128.

Evaluation MetricsFor Task II Substring Cccur where the ground-truth answer is "Yes" or "No", we measure accuracy using _soft match_, computed by checking whether the true answer appears in models' responses or not. For the other three tasks where models are expected to answer a number, we extract the last digits from model responses automatically and examine whether they are identical to the true answer. We also consider the verbal representation of numbers (e.g., "two" and "twice" for "2") with soft match by comparing the generated output with word form of true numbers.

## Appendix B Prompt Engineering

In Tab. 3, we list human prompts and improved prompts from Claude 10 on four counting tasks. Different from concise human prompts, prompts 

automatically improved by Claude normally contain detailed instructions and explicitly specify expected formats of responses.

We present LLM performance in response to different task prompts in Tab. 5. In contrast to stable performance enhancement from reasoning-based strategies, neither human heuristics-driven nor automatically improved prompts from Claude are able to elicit better capabilities to address the studied word-based counting problems. We conclude that failures of LLMs for solving counting tasks are irrelevant to how we phrase prompts.

\begin{table}
\begin{tabular}{l l} \hline \hline
**Source** & **Prompt** \\ \hline
**Original Prompt** & How many \{‘char’\}’s in the word “[{word}]”? \\ \cline{2-3}
**Human Prompt I** & How many times does the letter ‘[{char}]’ appear in the word ”[{word}]”? \\ \cline{2-3}
**Human Prompt II** & Can you count the number of ’[{char}]’s in ”[{word}]”? \\ \cline{2-3}
**Human Prompt III** & How many \{‘char’\}’s can be found in \(\lceil\)[{word}]”? \\ \cline{2-3}
**Human Prompt IV** & How often does the letter ‘{‘char’\}’s how up in the word “[{word}]”? \\ \hline  & You are tacked with counting the occurrences of a specific letter in a given word. Here are the inputs: \\ \multicolumn{1}{c}{} & Letter to count: \\ \cline{2-3}  & \textless{letter} \\ \cline{2-3}  & \textless{letter} \\ \cline{2-3}  & \textless{letter} \\ \cline{2-3}  & Word to analyze: \\ \cline{2-3}
* The letter is case-sensitive. \\ \multicolumn{1}{c}{} &
- Spaces and punctuation marks, if any, should be ignored. \\ \multicolumn{1}{c}{} & Please provide your response in a clear, concise sentence stating the count. \\ \hline \hline \multirow{4}{*}{**Claude Improved Prompt**} & \multicolumn{1}{c}{**Task II: Substring Occur**} \\ \cline{2-3}  & Is the substring –[{substring}]” part of the word “[{word}]”? \\ \cline{2-3}
**Human Prompt I** & Does the word “[{word}]” contain the substring —[{substring}]”? \\ \cline{2-3}
**Human Prompt II** & Does the sequence “[{substring}]" appear in the word “[{word}]”? \\ \cline{2-3}
**Human Prompt III** & Does the word “[{word}]” contain the sequence of letters “[{substring}]”? \\ \cline{2-3}
**Human Prompt IV** & Is ”[{substring}]” present as a substring in the word “[{word}]”? \\ \hline \multirow{4}{*}{**Claude Improved Prompt**} & \multicolumn{1}{c}{You are tacked with determining whether a given substring is part of a specified word. Here are the inputs:} \\ \cline{2-3}  & \textless{substring}\textgreater{} \\ \cline{2-3}  & \textless{word}\textgreater{} \\ \cline{2-3}  & Example output structure: \\ \cline{2-3}  & [Yes/No], the substring [is/is not] part of the word. \\ \multicolumn{1}{c}{} & Please provide your answer based on the given inputs. \\ \hline \hline \multirow{4}{*}{**Claude Improved Prompt**} & \multicolumn{1}{c}{**Task III: Word Len**} \\ \cline{2-3}  & How many characters in the word “[{word}]”? \\ \cline{2-3}
**Human Prompt I** & What is the total number of characters in the word “[{word}]”? \\ \cline{2-3}
**Human Prompt II** & Can you count the letters in the word “[{word}]”? \\ \cline{2-3}
**Human Prompt III** & Could you tell me how many letters are in the word “[{word}]”? \\ \cline{2-3}
**Human Prompt IV** & How many alphabetic characters does the word “[{word}]” contain? \\ \hline \multirow{4}{*}{**Claude Improved Prompt**} & \multicolumn{1}{c}{You are tacked with counting the number of characters in a given word.} \\ \cline{2-3}  & Here is the word: \\ \cline{2-3}  & \textless{word>} \\ \cline{2-3}  & \textless{|word>} \\ \cline{2-3}  & Note: Include all characters in your count, including letters, numbers, and any special characters that may be present. \\ \hline \end{tabular}
\end{table}
Table 3: Diverse prompts used to evaluate impact of prompt engineering on LLM performance. Human prompts are brief while prompts improved by Claude contain more detailed instructions and response format requirements. See prompts for Task IV in Tab. 4.

 



## Appendix C Whether Math or Code Training Data Helps

### Setup

Math/Code ModelsWe compare LLMs fine-tuned on general instruction/chat data (described in SS3) with their counterparts specialized in math-or code-related tasks: 1) Qwen2 Math (QwenLM, 2024b) and CodeQwen 1.5 (QwenLM, 2024a), 2) CodeGemma (CodeGammaTeam, 2024), 3) InternLM2 Math Plus (Ying et al., 2024) and InternLM2 Step Prover (Wu et al., 2024), 4) Mathstral v0.1 (MistralAI, 2024b) and Codestral v0.1 (MistralAI, 2024a), 5) DeepSeeK Math (Shao et al., 2024), DeepSeek Prover V1.5 (Xin et al., 2024) and DeepSeek Coder V2 (Zhu et al., 2024), 6) Yi Coder (01AI, 2024). We list detailed model information in Tab. 7.

ImplementationsBesides prompting LLMs to answer the word-based counting tasks defined in SS3 in the open-ended setting, we also explicitly request code LLMs to generate Python codes 11. We then measure correctness by executing codes and comparing output with the ground-truth.

Footnote 11: We adopt few-shot prompting used in Gao et al. (2023); Guo et al. (2024) so that models follow code formats demonstrated in provided examples, which makes code extraction easier and evaluation more accurate.

\begin{table}
\begin{tabular}{l l} \hline \hline
**Source** & **Prompt** \\ \hline  & \multicolumn{3}{c}{**Task IV: Distinct Char**} \\
**Original Prompt** & How many distinct characters in the word ”[{word}]”? \\
**Human Prompt I** & How many different letters are found in the word ”[{word}]”? \\
**Human Prompt II** & What is the number of unique letters present in the word ”[{word}]”? \\
**Human Prompt III** & What’s the count of distinct characters in the word ”[{word}]”? \\
**Human Prompt IV** & Can you identify how many unique characters make up the word ”[{word}]”? \\ \hline  & You are tasked with counting the number of distinct characters in a given word. Here is the word: \\  & ¡word> \\
**Claude Improved Prompt** & ¡word¿ \\  & ¡word¿ \\ \end{tabular}
\end{table}
Table 4: Diverse prompts used to evaluate impact of prompt engineering on LLM performance for Task IV.

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**Models** & **Original** & **Human I** & **Human II** & **Human III** & **Human IV** & **Claude** \\ \hline  & \multicolumn{8}{c}{**Task I: Char Occur**} \\ GPT-4o & 82.4 & 91.6 & 89.6 & 91.2 & 93.4 & 85.2 \\ Llama-3 & 34.6 & 26.4 & 55.4 & 24.6 & 40.4 & 22.4 \\ \hline  & \multicolumn{8}{c}{**Task II: Substring Occur**} \\ GPT-4o & 87.4 & 94.7 & 89 & 94 & 87.4 & 88.1 \\ Llama-3 & 58.2 & 71.7 & 59.7 & 65.3 & 61.4 & 52.8 \\ \hline  & \multicolumn{8}{c}{**Task III: Word Len**} \\ GPT-4o & 92.0 & 92.4 & 96.2 & 95.4 & 96.8 & 97.6 \\ Llama-3 & 74.6 & 92.6 & 91.0 & 79.2 & 77.8 & 96.8 \\ \hline  & \multicolumn{8}{c}{**Task IV: Distinct Char**} \\ GPT-4o & 89.2 & 87.2 & 78.0 & 90.6 & 95.8 & 52.8 \\ Llama-3 & 57.8 & 56.8 & 82.6 & 63.2 & 43.2 & 21.6 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Impact of prompt engineering on word-based counting tasks. Neither human heuristics-driven prompts nor improved prompts from Claude are able to consistently elicit better capabilities from LLMs for solving word-based counting problems.





## Appendix D How to Make LLMs Experts Again

### Implementations

We use greedy decoding and adopt the zero-shot setting 12 for model generation as introduced in SS3 for most strategies. For _self-consistency_ and _ToT_, we follow the practice in literature Wang et al. (2022); Yao et al. (2024) by applying temperature sampling with \(T=0.7\) and truncating at the top-k (\(k=40\)) tokens with the highest probability, we set the reasoning path to \(5\) unless otherwise specified. We finetune Llama 3 with Lora Hu et al. (2021) on \(10,000\) training instances and set the learning rate to \(3e-4\), epoch to \(1\) and batch size to \(128\) on a single A100 80G device 13. We also measure the impact of finetuning on existing capabilities with finetuned models evaluated on general (MMLU and IFEval), reasoning (GPQA and Hellaswag), math (GSM8K) and coding (HumanEval) benchmarks following LlamaWebsite (2024). For ICL, we randomly sample 4 and 8 demonstrations per testing instance from the training set.

Footnote 12: To avoid impact of demonstrations on generation, we use zero-shot rather than few-shot to evaluate effectiveness of different reasoning strategies, where the expected format of responses is included in questions.

[FOOTNOTE:1][ENDFOOTNOT



\begin{table}
\begin{tabular}{l c l} \hline LLMs & \#Params & Download Links/Version \\ \hline Llama 3 & 8B & [https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) \\ \hline Qwen 1.5 & 7B & [https://huggingface.co/Qwen/Qwen1.5-7B-Chat](https://huggingface.co/Qwen/Qwen1.5-7B-Chat) \\ Qwen2 Math & 7B & [https://huggingface.co/Qwen/Qwen2-Math-7B-Instruct](https://huggingface.co/Qwen/Qwen2-Math-7B-Instruct) \\ CodeQwen 1.5 & 7B & [https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat](https://huggingface.co/Qwen/CodeQwen1.5-7B-Chat) \\ \hline Gemma 1 & 7B & [https://huggingface.co/google/gamma-7b-it](https://huggingface.co/google/gamma-7b-it) \\ CodeGemma & 7B & [https://huggingface.co/google/codegemma-7b-it](https://huggingface.co/google/codegemma-7b-it) \\ \hline InternLM2 & 7B & [https://huggingface.co/internlm/internln2-chat-7b](https://huggingface.co/internlm/internln2-chat-7b) \\ InternLM2 Math Plus & 7B & [https://huggingface.co/internlm/internln2-math-plus-7b](https://huggingface.co/internlm/internln2-math-plus-7b) \\ InternLM2 Step Prover & 7B & [https://huggingface.co/internlm/internlm2-step-prover](https://huggingface.co/internlm/internlm2-step-prover) \\ \hline Phi 3 & 7B & [https://huggingface.co/microsoft/Phi-3-small1-128k-instruct](https://huggingface.co/microsoft/Phi-3-small1-128k-instruct) \\ Mistral v0.3 & 7B & [https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3) \\ Mathstral v0.1 & 7B & [https://huggingface.co/mistralai/Matshrtal-7B-v0.1](https://huggingface.co/mistralai/Matshrtal-7B-v0.1) \\ Codestral v0.1 & 22B & [https://huggingface.co/](https://huggingface.co/) mistralai/Codestral-22B-v0.1 \\ \hline DeepSeek-V2 & 16B & [https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat) \\ DeepSeeKath & 7B & [https://huggingface.co/deepseek-ai/deepseek-math-7b-rl](https://huggingface.co/deepseek-ai/deepseek-math-7b-rl) \\ DeepSeek Prover V1.5 & 7B & [https://huggingface.co/deepseek-ai/DeepSeek-Prover-V1.5-RL](https://huggingface.co/deepseek-ai/DeepSeek-Prover-V1.5-RL) \\ DeepSeek Coder V2 & 16B & [https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct) \\ \hline Yi 1.5 & 9B & [https://huggingface.co/@1-ai/Vi-1.5-9B-Chat](https://huggingface.co/@1-ai/Vi-1.5-9B-Chat) \\ Yi Coder & 9B & [https://huggingface.co/@1-ai/Vi-Coder-9B-Chat](https://huggingface.co/@1-ai/Vi-Coder-9B-Chat) \\ \hline GPT-do & - & gpt-4o-2024-05-13 \\ \hline \end{tabular}
\end{table}
Table 7: Information of tested LLMs. We list their model sizes and the download links if available or the model version for the proprietary model.

\begin{table}
\begin{tabular}{l r r r r r r r r} \hline \hline
**Task** & **GPT-4o** & **Llama 3** & **Qwen 1.5** & **Germa 1** & **InternLM2** & **Phi 3** & **Mistral v0.3** & **DeepSeek V2** & **Yi 1.5** \\ \hline  & & & \multicolumn{6}{c}{**Germanic Languages**} & & \\ English & 82.4 & 34.6 & 30.6 & 41.2 & 60.8 & 39.0 & 35.4 & 27.2 & 46.6 \\ German & 69.6 & 27.2 & 20.6 & 5.2 & 50.6 & 40.2 & 21.8 & 34.8 & 38.4 \\ Swedish & 80.6 & 39.0 & 18.8 & 5.8 & 61.2 & 38.0 & 34.4 & 55.6 & 42.6 \\ \hline  & & & \multicolumn{6}{c}{**Romance Languages**} & & & \\ French & 75.6 & 38.0 & 16.4 & 10.0 & 63.6 & 45.0 & 26.0 & 40.2 & 52.6 \\ Spanish & 76.2 & 32.6 & 25.4 & 10.4 & 64.6 & 45.6 & 28.0 & 38.2 & 50.4 \\ Italian & 71.4 & 24.8 & 22.0 & 15.6 & 55.2 & 37.6 & 20.4 & 37.0 & 49.6 \\ Portuguese & 65.0 & 31.2 & 21.4 & 23.0 & 65.4 & 52.6 & 26.0 & 45.8 & 47.4 \\ \hline \hline \end{tabular}
\end{table}
Table 8: Performance of LLMs on _Task I Char Occur_ in different languages from Germanic and Romance language families. LLMs cannot better identify occurrence of characters in less common words.





\begin{table}
\begin{tabular}{l c l} \hline
**Perturbation** & **Perturbed Word** & **Description** \\ \hline delete & strawberry & “w” is deleted \\ insert & strawbektry & “k” is inserted \\ repeat & strawberry & “t" is repeated \\ replace & strswberry & “a” replaced with ”s” \\ swap & striywerra & “a” and “y” are swapped \\ left shift & trawwberrys & all letters shift left with the first letter ”s” moving to the end \\ right shift & ystrawberrr & all letters shift right with the last letter ”y” moving to the start \\ shuffle & rasbretyrw & all letters arranged in random order \\ mapping & abcdefghi & letters from left to right replaced by ”a”, “b”, “c”, etc. \\ mapping & !@#5\%\&’(()\) & letters from left to right replaced by ”!”, “@”, “\#”, etc. \\ (special) & +dash & s-t-r-a-w-b-e-r-r-y & dash ’-’ inserted between every two letters \\ +space & s t r a w b e rr y & space ’ ’ inserted between every two letters \\ +comma & s,t,r,a,w,b,e,r,r,y & comma ’, inserted between every two letters \\ \hline \end{tabular}
\end{table}
Table 9: Character-level perturbation examples on the word ”strawberry” when the question is ”How many r’s in the word ”strawberry”?

* [26] M. C. Bader, A. Krizhevsky, and S. Vedaldi. Learning from incomplete data: a survey of machine learning models. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 1-40, 2015.
* [27] J. D. Friedman, R. G. Friedman, and P. W. Zisserman. The MIT Press' Cambridge University Press, Cambridge, USA, 2009.
* [28] Y. Bengio et al. Generative adversarial nets for large scale image recognition. In _Advances in neural information processing systems_, volume 3, pages 103-113, 2016.
* [29] H. Chen, X. Li, and L. Zhang. Deep residual learning with deep convolutional networks. In _NIPS_, pages 1398-1399, 2015.
* [30] T. Deng, F. Ren, and Q. Weinberger. Adam: Large-scale hierarchical network architectures. In _ICML_, pages 1398-1406, 2015.
* [31] I. Goodfellow, I. Sutskever, and J. Pouget. Imagenet classification by jointly connected layers. In _NIPS_, volume 3, pages 1339-1340, 2016.

[MISSING_PAGE_POST]

 

Figure 8: Performance comparison of LLMs between natural word and character input on two classification datasets.

Figure 9: Performance variation from Llama 3 on four tasks.

 

* [26] M. C. Bader, J. Sivic, and R. Girshick, "A unified framework for image recognition with deep convolutional neural networks," in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 1-4, 2015.
* [27] D. P. Kingma and A. Ba, "Adam: a method for stochastic optimization," in _Advances in Neural Information Processing Systems (NIPS'98)_, volume 3, pages 1349-1350, 2014.
* [28] K. L. Li, Y. Zhang, Z. Liu, and X. Sun, "Deep residual learning for image classification," in _IEEE International Conference on Acoustics and Speech (ICASSP)_, pages 1-10, 2015.
* [29] H. Lin, W. Wang, and G. Xu, "ImageNet: A simple approach to image segmentation," in _IEEE International Conference on Computer Vision and Pattern Recognition (CVPR)_, pages 1-10, 2015.

[MISSING_PAGE_POST]

 

Figure 12: Scaling trend of the _self-consistency_ strategy by different LLMs on four tasks.

Figure 10: Impact of different reasoning strategies on LLM performance for three word-based counting tasks.

 

* [26] M. C. Bader, A. D. Krizhevsky, and S. I. Sutskever. Learning from incomplete data: a survey of machine learning models. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_, pages 1-40, 2015.
* [27] J. L. Friedman and R. G. Hinton. Visualizing data using deep networks. _Journal of Machine Learning Research_, 3(8):199-202, 1994.
* [28] P. F. Donoho, N. Srivastava, and V. Savona. Deep residual learning for image recognition. In _Advances in neural information processing systems_, volume 1, pages 1-10, 2015.
* [29] T. E. Dinh, Y. Bengio, and H. Zisserman. Very accurate representations of natural language. _arXiv preprint arXiv:1406.1120_, 2014.
* [30] W. Dong, X. Li, and Q. Weinberger. Unsupervised learning with generative adversarial nets. In _NIPS Workshop on Neural Networks_, pages 1-10, 2015.
* [31] W. Dong, X. Li, and Q. Weinberger. Generative adversarial nets. In _Advances in neural information processing systems_, volume 1, pages 1-10, 2015.
* [32] W. Dong, X. Li, and Q. Weinberger. Adversarial nets: an introduction to artificial intelligence. _Neural Information Processing Systems_, 2(1):1-10, 2015.

[MISSING_PAGE_POST]

 



**Question:** How many a's in the word "confusable"?

\begin{table}
\begin{tabular}{l l} \hline \hline
**True Answer:** 1 \\ \hline
**Direct Answer Number:** 2 � \\ \hline
**Open-ended:** There are 2 a’s in the word ”confusable”. � \\ \hline
**CoT:** Here’s the step-by-step reasoning: \\ Reasoning: & 1. Start by looking at the word “confusable”. \\
2. Identify each letter in the word: c-o-n-f-u-s-a-b-l-e. \\
3. Count the number of “a”s in the word: there is one ”a” in the word. \\ Answer: & \\
1 \(\blackvee\) & \\ \hline \end{tabular}
\end{table}
Table 10: Case study of baseline and CoT generation from Llama 3 on _Task I Char Occur_. We mark correct answer by \(\blackdiamond\) and wrong ones by �. We provide examples on other three datasets in Tab. 11.





**Task II: Substring Occur**

**Question: Is substring "Elvp" part of the word "Elvis"? True Answer: No. Direct Answer Yes/No: Yes, (Wrong) Open-ended: A simple question! Yes, the substring "Elvp" is indeed part of the word "Elvis". (Wrong) CoT: Here's the step-by-step reasoning: Reasoning: 1. The word "Elvis" is a sequence of characters: E-L-V-I-S. 2. The substring "Elvp" is a sequence of characters: E-L-V-P. 3. To determine if "Elvp" is a part of "Elvis", we need to check if all the characters in "Elvp" are present in the same order in "Elvis". 4. Comparing the characters, we can see that the first three characters "Elv" are present in both "Elvp" and "Elvis". 5. However, the fourth character "P" is not present in "Elvis", which has an "I" instead. 6. Since the characters in "Elvp" are not present in the same order in "Elvis", "Elvp" is not a part of "Elvis". Answer: No, the substring "Elvp" is not part of the word "Elvis". (Correct) Task III: Word Len Question: How many characters in the word "Missourianism"? True Answer: 13 Direct Answer Number: 12 (Wrong) Open-ended: Let me count them for you! The word "Missourianism" has 12 characters: M-I-S-S-O-U-R-I-A-N-I-S-M. (Wrong) CoT: Here's the step-by-step reasoning: Reasoning: To count the number of characters in the word "Missourianism", I will break it down into individual letters and count them one by one.

\begin{table}
\begin{tabular}{l l}
1. M & \\
2. i & \\
3. s & \\
4. s & \\
5. o & \\
6. u & \\
7. r & \\
8. i & \\
9. a & \\
10. n & \\
11. i & \\
12. s & \\
13. m & \\ \end{tabular}
\end{table}
Table 11: Case study of Llama 3 on Task II and III. See examples on Task IV in Tab. 12.





**Task IV: Distinct Char Question: How many distinct characters in the word "mutch"? True Answer: 5 Direct Answer Number: 5 (Correct) Open-ended: The word "mutch" contains 5 characters: m-u-t-c-h. There are 4 distinct characters: m, u, t, and c. (Wrong) CoT: Reasoning:To find the number of distinct characters in the word "mutch", I will start by writing down the word and identifying each character: m-u-t-c-h Next, I will look for any duplicate characters. In this case, I see that there is no repeated character. Finally, I will count the number of unique characters, which are: m, u, t, c, h

Answer: 5 (Correct)