Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models (2024)

Abstract: Neural-symbolic methods have demonstrated efficiency in enhancing the reasoning abilities of large language models (LLMs). However, existing methods mainly rely on syntactically mapping natural languages to complete formal languages like Python and SQL. Those methods require that reasoning tasks be convertible into programs, which cater to the computer execution mindset and deviate from human reasoning habits. To broaden symbolic methods’ applicability and adaptability in the real world, we propose the Meta-Reasoning from a linguistic perspective. This method empowers LLMs to deconstruct reasoning-independent semantic information into generic symbolic representations, thereby efficiently capturing more generalized reasoning knowledge. We conduct extensive experiments on more than ten datasets encompassing conventional reasoning tasks like arithmetic, symbolic, and logical reasoning, and the more complex interactive reasoning tasks like theory-of-mind reasoning. Experimental results demonstrate that MetaReasoning significantly enhances in-context reasoning accuracy, learning efficiency, outof-domain generalization, and output stability compared to the Chain-of-Thought technique. Code and data are publicly available at https: //github.com/Alsace08/Meta-Reasoning.

heading: 1 Introduction
text: Tom has 3 apples, and David has 5 bananas, how many fruits do they have in total?
Original Forms (Infinite)
Wendy walked 3 kilometers in the morning and 5 kilometers in the afternoon. How many kilometers did she walk today?
There are 3 bags of biscuits at home. Mom bought 5 bags in the market. How many bags of biscuits are there now?
……
LLM
The answer is -2. ×
Meta Form (Finite)
It is known that A = 3 and B = 5. Add B to A, now what is the value of A?
The answer is 8.0. √
The answer is 8. √ ……
LLM
A = 8. √
acquire more generic knowledge with enhanced data learning efficiency, as shown in Figure 1.
Motivated by the insight above, we introduce Meta-Reasoning, a novel reasoning paradigm aimed at deconstructing the semantics of entities and operations in questions into generic symbolic representations. Meta-Reasoning enables LLMs to learn generalized reasoning patterns across various semantics-wrapped scenarios, enhancing learning efficiency and reasoning accuracy. We apply MetaReasoning to in-context learning by designing demonstrations integrating semantic resolution with the CoT technique. This empowers LLMs to deconstruct questions and effectively capture more generalized knowledge autonomously.
To assess the efficacy of our method, we conduct experiments on over ten datasets, spanning both conventional reasoning scenarios, which involve arithmetic, symbolic, and logical reasoning tasks, and interactive reasoning scenarios, which involve theory-of-mind reasoning. We mainly compare our method with the CoT method upon GPT3 and ChatGPT. Experimental results show that Meta-Reasoning consistently outperforms the FewShot-CoT method across all tasks, demonstrating significant performance improvements. In the conventional reasoning scenarios, Meta-Reasoning achieves an average performance gain of +20% across all datasets with fewer demonstrations. In more complex interactive reasoning scenarios, Meta-Reasoning surpasses CoT across all levels of theory-of-mind reasoning with just a single demonstration. Moreover, Meta-Reasoning demonstrates remarkable out-of-domain generalization and output stability, indicating its scalability and user-friendly nature as a reasoning paradigm.
To our knowledge, we are the first to establish an equivalence mapping from semantics to symbols within natural language. This innovation facilitates in-context learning for LLMs, significantly enhancing their capacity for generalized reasoning. We expect to extend the reasoning ability boundary of LLMs based on this research.

heading: 2 Preliminary: Why Meta-Reasoning?
text: Meta-Reasoning is an idealized reduction-based reasoning paradigm defined in this work, whose goal is to reduce the infinite semantic concepts in the world’s languages to a finite symbolic system, thus allowing machines to generalize to many semantically wrapped problems through the
acquisition of universal laws. This paradigm is best suited for such a reasoning scenario: the final reasoning results are independent of the particular semantic representations and are only related to the underlying reasoning skeletons.
The core of Meta-Reasoning lies in SemanticSymbolic Deconstruction, which we simplify as Semantic Resolution. This process conveys the semantics of the original problem via symbols with generalized meanings, without affecting the final results. However, why deploying Semantic Resolution in LLMs is a key issue, we must consider the advantages it brings to the reasoning process.
We explore this issue from two perspectives: (i) the human reasoning speed when responding to different questions, and (ii) the machine reasoning accuracy when responding to different questions. We select MultiArith (Roy and Roth, 2015) and GSM8K (Cobbe et al., 2021), two arithmetic datasets, and rephrase 100 questions in each dataset according to the semantic resolution rules that will be introduced in Section 3.1, thereby creating metaquestions. Subsequently, we distribute the original and meta-questions to both human volunteers and LLMs to obtain corresponding results of metrics.
2.1 Response Speed Test For Human
We assess the response speed of three human volunteers by measuring the total time taken from receiving the question to providing the answer.1
As shown in Figure 2, human response speed significantly improves when solving meta-questions, particularly evident in GSM8K. This acceleration is attributed to the removal of unimportant semantic information in meta-questions, which enables quicker recognition of the reasoning skeleton by humans. Moreover, the more concentrated distribution of human reaction times suggests
1Samples with incorrect answers are excluded from the analysis due to their negligible impact, given the low difficulty level of the math problems for adults.
623
a similarity in reasoning frameworks for such problems, indicating that semantic resolution fosters consistency in reasoning patterns.

heading: 2.2 Accuracy Test For Machine
text: MultiArith (original → meta) Zero-Shot 28% → 31% (+) Zero-Shot-CoT 70% → 100% (+)
GSM8K (original → meta) Zero-Shot 22% → 13% (-) Zero-Shot-CoT 41% → 97% (+)
2The prompt is “A:”. 3The prompt is “A: Let’s think step by step.”.

heading: 3 Meta-Reasoning Paradigm
text: We have observed notable performance improvements in LLMs when tackling questions after semantic resolution in the last section. In this section, we formally introduce the Meta-Reasoning paradigm employed in LLMs. Section 3.1 defines the specific rules for semantic resolution. Then, we put this process through in-context learning for LLMs to imitate, and Section 3.2 formalizes the demonstration design form of in-context learning.

heading: 3.1 Definition: Semantic Resolution Rules
text: Semantic resolution corresponds to the many-toone mapping from various semantic representations to the most intrinsic symbolic representation. We focus on two types of elements within text sequences that structure the entire reasoning skeleton but whose semantics do not change the reasoning path: (i) Entity, it represents the subjects on which the reasoning task acts, but it is not critical what or who exactly it is; (ii) Operation, it establishes connections and changes between subjects, but the exact form of that is not important. For example, “he ate 3 apples” and “he threw 3 apples” are both essentially forms of subtraction. Examples are shown in Figure 3.
Entity. Intuitively, entity representations with natural semantics can be treated as the expansion products of an exhaustive set of non-empty symbols. Given a native symbol set4(alphabet),
4Take examples in the English language system, regardless of lowercase or uppercase.
624
Σ1 = {A,B, ..., Z}, the positive closure Σ+ =⋃∞ i=1Σ
i of Σ1 contains the set Q of all symbolic representations with natural semantics in the English language system, i.e., Q ⊂ Σ+, where Σi(i > 1) = Σj × Σi−j(1 ≤ j ≤ i) and × denotes the Cartesian product operation. We consider the opposite form of the symbol-semantics expansion, i.e., semantics-symbol resolution, and construct the mapping fe : Q → Σ1 to transform these complex semantic representations to their primitive symbolic form in the alphabet. Since the symbols in the alphabet are meaningless, the mapping results are not required to be specified — we default to mapping them one by one in alphabetical order without duplication.5
Back to reasoning scenarios, given a sequence of original question S = [s1:n], we first manually locate all the entity spans [si:j ] ⊂ S (e.g. apple, mom), and later apply the mapping fe to them to obtain the single characters σij = fe([si:j ]), respectively, which will be embedded back into the original position of the sequence S so that it will be modified into S = [s1:i−1 ◦ σij ◦ sj:n].
5For example, there are three semantic representations x1, x2, x3 that need to be mapped, and the mapping can be done by default as x1 → A, x2 → B, x3 → C.
Operation. Entities constitute the set of subjects on which the reasoning task acts, while the definition and change of entity states determine the reasoning path: (i) definitions of entity states can usually be reduced to assignment and logical association operations, i.e., O1 = {=,→}, and O1 is a finite set; (ii) changes in entity states can be reduced to arithmetic operations, i.e., O2 = {+,−,×,÷}, and O2 is a finite set.6 Conveniently, these arithmetic symbols can correspond to natural semantics, e.g., “+” corresponds to “add”, which allows symbols to be more closely integrated with natural language. Similar to the resolution of entities, we construct the mapping fo : Q→ (O1∪ O2), and transform all manually-located operation representation [si:j ] (e.g. eat, have) into single symbols ρij = fo([si:j ]), which will be embedded into the original position of the sequence S so that it will be modified into S = [s1:i−1 ◦ ρij ◦ sj:n].
Appendix A provides some mapping examples. After semantic resolution, the original questions maximally remove semantically irrelevant terms and simplify the need for semantic reasoning.
6There may be some extraordinary operations, but generally finite. We leave this for future work.
625

heading: 3.2 Deployment: Synthetic Demonstration Design for In-context Learning
text: However, manual annotation of entities and operations one by one is time-consuming. We expect LLMs to autonomously learn generic reasoning patterns for certain reasoning tasks by automatically simplifying complex questions into equivalent and simpler forms. This can drive dataefficient learning. Therefore, we consider the incontext learning. Furthermore, inspired by the demonstrated significance of the CoT technique in enhancing reasoning capabilities in prior works (Wei et al., 2023; Kojima et al., 2022a), we are dedicated to devising a fusion strategy of semantic resolution and CoT, which aims to maximize the performance potential of LLMs in reasoning.
We focus on two fusion modes: Completelyserial and Crossly-serial. The primary distinction between the two modes lies in whether Semantic Resolution (SR) and CoT appear overlappingly. The pipeline and case are illustrated in Figure 4, with further details provided below:
Completely-serial. We first conduct SR to obtain the meta-question form, then draft the CoT for the corresponding meta-question. In this case, the rationale is [SR ◦ CoT]. Crossly-serial. We first split the original question into n sub-steps, where n may vary depending on the specific context. For each sub-step i, the sub-rationale is represented as [SRi ◦ CoTi]. Finally, we concatenate all the sub-rationales. In this case, the rationale is [[SR1 ◦ CoT1] ◦ [SR2 ◦ CoT2] ◦ · · · ◦ [SRn ◦ CoTn]], where [SR1 ◦ SR2 ◦ · · · ◦ SRn] = SR and [CoT1 ◦ CoT2 ◦ · · · ◦ CoTn] = CoT.

heading: 4 Experiments

heading: 4.1 Setup
text: Tasks and Datasets. We conduct experiments on two categories: (i) conventional reasoning, involving basic reasoning scenarios like arithmetic, symbolic, and logical reasoning. This includes the following datasets: MultiArith (Roy and Roth, 2015), AddSub (Hosseini et al., 2014), Last Letter Concatenation (Letter) (Wei et al., 2022), Coin Flip (Coin) (Wei et al., 2022), Web of Lies (Lies) (Srivastava et al., 2022), Tracking Shuffled Objects7 (Track) (Srivastava et al., 2022), and (ii)
7Divided into 3 subsets based on the number of objects and shuffler operations (3/5/7).
interactive reasoning, which involves reasoning scenarios of multi-agent mental gaming, including Hi-ToM8 (He et al., 2023a). Refer to Appendix B for detailed information on datasets.
Language Models. We utilize publicly available 175B GPT-3 models (text-davinci-002 and text-davinci-003) (Brown et al., 2020), as well as ChatGPT (gpt-3.5-turbo).9 Additionally, for comparison purposes, we include other robust closed-API LLMs: 175B Codex (code-davinci-002) (Chen et al., 2021) and 540B PaLM (Chowdhery et al., 2022).
Implementation and Baselines. In our MetaReasoning (MR) paradigm, we use the completelyserial mode for arithmetic tasks and the crosslyserial mode for symbolic and logical tasks. We also compare our method with three other paradigms: (i) Fine-tuning; (ii) Standard prompting, including Zero-Shot and Few-Shot; (iii) Chain-of-Thought (CoT) prompting, including Zero-Shot-CoT (Kojima et al., 2022a) and Few-Shot-CoT (Wei et al., 2022). Refer to Appendix G for demonstrations.

heading: 4.2 Main Results I: Conventional Reasoning
text: Overall Performances. Table 2 presents the results.10 Our MR consistently outperforms FewShot-CoT and notably excels on complex tasks challenging for LLMs. This trend is particularly evident for the relatively capacity-constrained textdavinci-002. Notably, on intricate tasks where pure CoT struggles, our MR effectively alleviates the reasoning bottleneck, resulting in significantly higher accuracy (+27.0% in Letter and +37.7% in Track). This indicates that our MR facilitates LLMs in learning general principles for specific task types, automatically reducing reasoning difficulty across various semantic representations.
Fewer Demonstrations, Better Performances. Figure 5 shows comparisons between the performance of CoT and MR paradigms with varying numbers of demonstrations. MR consistently achieves superior performance across almost all datasets while utilizing fewer demonstrations, particularly evident in symbolic and logical reasoning tasks. For example, in the Letter task,
8Divided into 5 subsets based on the number of mental gaming orders (1/2/3/4/5).
9https://chat.openai.com/ 10Experimental results of GPT-3 were obtained in March 2023 via the OpenAI API interface, while the results of ChatGPT were obtained in November 2023.
626
MR results in a +27.0% improvement for LLMs with 1/2 demonstrations compared to the CoT paradigm. Similarly, in the Track(7) task, using only 1/3 demonstrations (i.e., one demonstration) leads to a remarkable +39.6% boost. This indicates that LLMs can acquire general solutions for specific tasks with minimal demonstrations, facilitating learning through analogy.

heading: 4.3 Main Results II: Interactive Reasoning
text: The real-world reasoning environment is more intricate than these conventional reasoning scenarios. Therefore, we consider more complex interactive scenarios and introduce the Theory-of-Mind (ToM) reasoning. In ToM reasoning, the objects involved in reasoning require subjective observation or cognitive abilities, and their observation and thought directly influence the reasoning outcomes. Therefore, LLMs are susceptible to interference.
The variable parameter “Order” determines ToM’s difficulty level, which refers to the layer number of the mental game involved. For example, in 3-order reasoning, the structure might be “A thinks B thinks C thinks xxx”. Notably, 1-order reasoning does not entail any interaction and is categorized as low-order reasoning. On the other hand, reasoning with an order greater than 1 involves a mental game between multiple observers and is classified as high-order reasoning.
When solving lower-order ToM questions, both 1-shot CoT and MR achieve nearly 100% accuracy, indicating that LLMs can accurately comprehend the reasoning text itself. But when solving highorder ToM, CoT exhibits a notable performance decline, with an about 40% decrease in joint accuracy when transitioning from 1 to 2-order, and with a nearly 0% accuracy remaining at 5-order. In contrast, MR maintains stable performance as the
627
MultiArith AddSub Letter Coin Lies Track(3) Track(5) Track(7)
60
70
80
90 100 A cc ur ac y (% ) 91.7
81.3
59.0
97.2 92.0
62.8 60.8 59.6
94.5
86.6 86.0
100.0 99.2 97.2 100.0 99.2
Meta-Reasoning (Acc) Chain-of-Thought (Acc)
0
2
4
6
8
10N um
ber of D em onstrations 8 8 4 8 3 3 3 3 6 5 2 3 3 3 1
Chain-of-Thought (Num) Meta-Reasoning (Num)
3

heading: 5.1 Boundary Test: OOD Generalization

heading: 5 Advantage Analysis
text: 0 2 4 6 8 10 Number of Reasoning Units
0.0
0.2
0.4
0.6
0.8 1.0 B R at e
Track(3) MR Track(3) CoT Lies MR Lies CoT ToM MR ToM CoT
Figure 7: Boundary test of out-of-domain generalization under CoT and MR paradigms, where the number of reasoning units is larger than three (the right area of the vertical gray line in the figure) means out-of-domain.
0 6000
40
80
120 MultiArith
Chain-of-Thought Meta-Reasoning
0 3950
40
80
120 AddSub
0 250150
200
250 Web of Lies
0 250300
400
500
600 Tracking Shuffled Objects (7)
Sample ID
N um
be r o
f A PI
O ut
pu t T
ok en
s
Figure 8: The token number distributions of the text generated by GPT-3 text-davinci-002 when using the Chain-of-Thought and Meta-Reasoning paradigms.
We draw BRate curves of each D. The larger the area enclosed by the curve and the x-axis, the stronger the OOD generalization of the method. Figure 7 shows the BRate curves of each dataset under CoT and MR paradigms, respectively. We note that as the number of reasoning units grows beyond the domain, the CoT curves exhibit a sharp decline, while the MR curves maintain relative smoothness, with Lies and Track tasks achieving nearly 100% BRate. This indicates that our MR facilitates strong OOD generalization for LLMs.

heading: 5.2 Output Stability Test
text: In addition to performance, user experience is another crucial consideration. Currently, access to LLMs like GPT-3 involves paywalls. Unexpected outputs, such as endless looping text or random guessing, can increase user fees, so making a stable output space is essential. We analyze the number
of API output tokens generated by MR and CoT paradigms for each sample to evaluate this stability, as illustrated in Figure 8. When employing the MR, the output scales of different samples are much closer. Conversely, under the CoT, outputs scatter widely, increasing the likelihood of encountering unexpected and abnormal situations.

heading: 6 Discussion
text: We conduct ablation studies to examine the role of semantic resolution in the reasoning process. Moreover, we compare our method with existing work in language programming (Chen et al., 2022; Gao et al., 2023b), highlighting Meta-Reasoning’s broader applicability across diverse scenarios. These extended analyses are shown in Appendix C.

heading: 7 Related Work
text: Our work is related to the research lines of neuralsymbolic methods and chain-of-thought reasoning. Please refer to Appendix F for full details.
Neural-Symbolic Methods in LLMs. Symbolic learning (Chen et al., 2021) significantly improves LLMs’ reasoning performance. Prior works focus on converting natural languages into programming languages (Gao et al., 2022; Cheng et al., 2022) and accessing external interpreters for execution (Schick et al., 2023); or using symbolic tasks for post-tuning (Liu et al., 2023; Wei et al., 2023), leading to performance improvements. However, these symbols are well-defined formal languages completely independent of natural languages. Our work jumps out of this framework and further enhances the efficiency of the symbolic methods.
Chain-of-Thought Reasoning. Intriguing chainof-thought techniques (Wei et al., 2022; Kojima et al., 2022b; Wang et al., 2022b; Zhang et al., 2023b) have effectively leveraged the emergent ability of LLMs to decompose multi-step reasoning. It can improve the performance of general-purpose and even domain-specific reasoning (Zhang et al., 2023c; Wang et al., 2023; He et al., 2023b; Zhang et al., 2023a).

heading: 8 Conclusion
text: We propose Meta-Reasoning, a semantic-symbol deconstruction paradigm for reasoning. Through the semantic resolution of the original questions, we enable LLMs to grasp meta-forms and general solutions for specific types of reasoning tasks. This
629
approach requires fewer demonstrations to expand the upper limit of their reasoning accuracy, out-ofdomain generalization, and output stability.
Limitations
Semantic resolution dictates that Meta-Reasoning tasks must disregard the intrinsic properties of entities. Consequently, Meta-Reasoning may not be well-suited for reasoning tasks reliant on world knowledge in semantics, such as commonsense reasoning. However, Meta-Reasoning shows potential in real-world agent reasoning scenarios (Gao et al., 2023a; Tang et al., 2023). When agents are impeded by irrelevant properties, Meta-Reasoning can effectively circumvent such obstacles. We aim to explore more comprehensive reasoning scenarios to further justify its applicability in future work.
Ethics Statement
We use publicly available datasets for experiments, so the ethics issues of the source texts are nonexistent. For the generated contents with LLMs, prior work (Brown et al., 2020; Chan, 2023) has elaborated on their inevitable potential toxicity, such as issues of bias and fairness. We completely keep the prompts neutral and task-specific to avoid toxic language generation, and there were no toxic texts that appeared in our experiments.

heading: Acknowledgements
text: Yiming and Rui are with MT-Lab, Department of Computer Science and Engineering, School of Electronic Information and Electrical Engineering, and also with the MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai 200204, China. This paper is mainly supported by the Alibaba-AIR Program (22088682). Yiming and Rui are also supported by the National Natural Science Foundation of China (62176153), and the Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102). This work is also partially supported by the Joint Funds of the National Natural Science Foundation of China (U21B2020).

heading: A Semantic-Symbol Operation Rulebase
text: Table 4 shows operation mapping examples. Due to the lack of automatic methods, the rule base is continuously revised and improved with the annotation process.

heading: B Dataset Details
text: To measure the generalizability of our approach, we consider conventional and interactive reasoning:
Conventional Reasoning. In this scenario, reasoning information is globally accessible to all observers. We adopt three categories of reasoning as our testbed: (i) Arithmetic reasoning, we choose MultiArith (Roy and Roth, 2015) and AddSub (Hosseini et al., 2014) tasks, with 600 and 395 test instances separately; (ii) Symbolic reasoning, we follow Wei et al. (2022) to use Last Letter Concatenation and Coin Flip tasks, they both include 500 test instances; (iii) Logical reasoning, We choose Web of Lies and Tracking Shuffled Objects tasks from BIG-bench (Srivastava et al., 2022) — a more challenging reasoning task collection. In particular, the Tracking Shuffled Objects task is divided into three datasets according to the number of objects and shuffler operations (3/5/7). each dataset includes 250 test instances.
Interactive Reasoning. In this scenario, individual observers are limited to observing distinct local reasoning information, necessitating reliance on interaction and mental gaming for their reasoning processes. We select the Theory-of-Mind (ToM) task as our testbed and choose Hi-ToM (He et al., 2023a) as a benchmark for it involves the complex higher-order mind. This dataset contains a collection of multiple subsets ranging from 1 to 5 orders, each subset has 20 test instances.
632

heading: C Extended Analysis
text: C.1 Bridge the Gap between LLMs’ Capabilities.
We conduct longitudinal analyses of performance gaps between four LLMs. Figure 9 visualizes the performance gaps between LLMs for the same dataset and paradigm. Observations are as below:
• GPT-3 text-davinci-002 (the worst original performance among the four LLMs) greatly outperforms the remaining three LLMs under the CoT paradigm on five datasets after adopting the Meta-Reasoning paradigm.
• Performance gaps between text-davinci-002 and -003 on all datasets are greatly reduced compared to under the CoT paradigm after adopting the Meta-Reasoning paradigm.
These findings indicate that our Meta-Reasoning paradigm further bridges the gap in the LLMs’ capability themselves, allowing the weaker LLMs (e.g. text-davinci-002) to approximate the stronger LLMs (e.g. text-davinci-003) in reasoning ability.
C.2 Ablation Study We perform ablation studies to explore the role of semantic resolution in the whole reasoning process. Table 4 reports the error rates of all datasets under both paradigms and error reason rates (caused by semantic resolution or pure reasoning) in the wrong samples for each dataset.
We note that the causes of errors are inconsistent in different reasoning scenarios. For symbolic and logical reasoning, LLMs hardly produce any semantic resolution errors, only errors in the reasoning process (of course, the error rate of their reasoning itself is extremely low). This shows that semantic reasoning fully plays a positive role in reducing the complexity of reasoning for LLMs. But in arithmetic reasoning, semantic resolution errors often occur, and exceed the errors in the reasoning process itself. This shows that LLMs cannot reduce all types of questions under specific arithmetic datasets well. Intuitively, symbolic and logical reasoning questions are easier to logicalize than arithmetic reasoning questions, and the combination of reasoning units under arithmetic reasoning is more flexible. How to fully push the upper limit of LLM’s semantic resolution ability, so as to further improve its reasoning ability, is a promising future work.
C.3 Formal Pattern Flexibility
So far, most symbolic reasoning work focuses on mapping natural semantics to formal languages with complete grammar (such as Python and SQL). However, this grammatical completeness actually limits the form conversion, and it has higher requirements for the abstraction of the original reasoning tasks. To verify the flexibility of our paradigm, we contrast Program-of-Thought (PoT), a Text-to-Python reduction approach for reasoning tasks (Chen et al., 2022; Gao et al., 2022).
633
Meta-Reasoning Program-of-Thought0
20
40
60
80
100 94.5
46.7
86.6 86.8
99.2
44.0
16.8 MultiArith AddSub Lies Track (7)
Paradigm
A cc
ur ac
y (%
)
Meanwhile, to keep the settings consistent, we eliminate the call of an external interpreter in the PoT paradigm but utilize the LLMs themselves to complete the entire reasoning step, and select the same demonstrations for the two paradigms.
Figure 10 shows the performance comparisons of PoT and MR paradigms on four datasets. For the two arithmetic reasoning tasks (MultiArith and AddSub), the performance of PoT fluctuates wildly after removing the external interpreter. For the two symbolic reasoning tasks (Lies and Track), PoT is almost completely ineffective. In contrast, MR has stronger flexibility when encountering reasoning tasks that are not easily programmed.

heading: D Details of Boundary Test
text: D.1 Reasoning Unit Division Sample reasoning units for three datasets are as below. The smallest reasoning unit is highlighted in blue .
• Lies. Andree lies. Delfina says Andree lies .
Jim says Delfina tells the truth .
D.2 Computation of Boundary Length
The algorithm of Boundary Length (BL) Computation is shown in Algorithm 1.

heading: E Case Study
text: Table 5 compares output examples under FewShot-CoT and Meta-Reasoning paradigms. We find that when the original question requires reasoning about a large number of entities and the corresponding relationships, the reasoning process in the Few-Shot-CoT paradigm tends to be more chaotic, leading to errors and potential confusion between entities. In contrast, Meta-Reasoning offers a clearer and more concise reasoning process, reducing interference caused by a large number of natural semantics.

heading: F Additional Related Work
text: This work is inspired by two directions. First is the neural-symbolic methods, which have shown great promise in improving LLMs’ reasoning performance. Current work mainly focuses on converting natural languages into programming languages, however, the symbols that most of these works focus on artificially defined formal languages completely independent of natural languages, which makes it hard to establish the mapping facing complex real-world scenarios. Therefore, our research concentrates on human natural language, delving into semantic resolution at the semiotic level, and pushing the boundaries of LLMs in handling problems within the realm of natural language. Second is the Chain-of-Thought, an important technique for in-context learning reasoning. However, in-context learning with CoT is limited to learning from the reasoning process of the sample itself. Our optimization is highlevel, and We hope to promote the efficiency and generality of sample learning by generalizing the features of a single sample to the general features of the entire dataset. Our objective is to enhance the efficiency and generalizability of sample learning upon the CoT framework.
Neural-Symbolic Methods in LLMs. Starting from Codex (Chen et al., 2021), symbolic learning has shown great promise in improving LLMs’ reasoning performance. Afterward, a series of works further explored symbolic approaches in LLMs’ reasoning, and they can be broadly classified into two categories: (i) converting natural
languages into programming languages (Chen et al., 2022; Gao et al., 2022; Cheng et al., 2022), such as Python or SQL, and using the powerful code capabilities of LLMs to parse and even access external interpreters for execution (Schick et al., 2023); (ii) using symbolic tasks for posttuning of LLMs (Liu et al., 2023), which was found to lead to unexpected improvements in the overall performance of the models. However, the “symbols” that most of these works focus on are artificially defined formal languages completely independent of natural languages. These works establish sample-specific one-to-one mappings between two languages (natural language → formal language). Obviously, formal languages are learned by LLMs with less ambiguity due to their syntactic rigor, but they are divorced from the study of human natural language itself. Recently, Wei et al. (2023) design a novel symbol tuning scheme by replacing natural language labels with semantically-unrelated symbols, but the symbol system they define is not complete. This approach is different from the symbols under formal languages used in previous studies but has not been explored further in depth. Our work closely focuses on human natural language, resolute the semantics at the semiotic level, and explores the upper limit of LLM reasoning in dealing with problems under natural language.
Chain-of-Thought Prompt for Reasoning. Intriguing chain-of-thought (CoT) techniques have effectively leveraged the emergent ability of LLMs to decompose multi-step reasoning. Recent work in this field can be broadly classified into four categories: (i) Improving the performance of general-purpose reasoning tasks (Wei et al., 2022; Kojima et al., 2022b; Wang et al., 2022b; Zhou et al., 2022; Zhang et al., 2023b; Fu et al., 2022), i.e., arithmetic, symbolic, logical, and commonsense reasoning; (ii) Applying to domain-specific reasoning, such as multi-modality (Zhang et al., 2023c), or some purely linguistic tasks, such as translation (He et al., 2023b), summarization (Wang et al., 2023), sentiment analysis (Fei et al., 2023), question-answer (Li et al., 2022), etc; (iii) Analyzing the mechanics and interpretability of CoT (Wang et al., 2022a; Shi et al., 2023; Lyu et al., 2023); (iv) Distilling CoT techniques for smaller models (Ho et al., 2022; Kim et al., 2023).
635
Question (Tracking Shuffled Objects)
Q: Alice, Bob, and Claire are dancers at a square dance. At the start of a song, they each have a partner: Alice is dancing with Lola, Bob is dancing with Rodrigo, and Claire is dancing with Patrick. Throughout the song, the dancers often trade partners. First, Alice and Bob switch partners. Then, Claire and Bob switch partners. Finally, Bob and Alice switch partners. At the end of the dance, Alice is dancing with Options: (A) Lola (B) Rodrigo (C) Pa
Few-Shot-CoT
A: Let’s think step by step. (0) At the start: Alice: Lola, Bob: Rodrigo, Claire: Patrick. (1) Alice and Bob switch partners: Alice: Rodrigo, Bob: Lola, Claire: Patrick. (2) Claire and Bob switch partners: Alice: Rodrigo, Bob: Patrick, Claire: Lola. (3) Bob and Alice switch partners: Alice: Lola, Bob: Patrick, Claire: Rodrigo. At the end of the dance, Alice is dancing with Lola. So the answer is (A).%(Entity information interferes with reasoning)
Meta-Reasoning
A: The question can be simplified to: It is known that A = 1, B = 2, C = 3. First, B and A swap books: B and A → (B = 2, A = 1 → B = 1, A = 2) → A = 2, B = 1, C = 3. Then, A and C swap books: A and C → (A = 2, C = 3 → A = 3, C = 2) → A = 3, B = 1, C = 2. Finally, C and B swap books: C and B → (C = 2, B = 1 → C = 1, B = 2) → A = 3, B = 2, C = 1. At the end of the semester, Alice has: Alice → A = 3, 3 → the 3-rd option → the answer is (C). ✓
Question (Web of Lies)
Q: Sherrie tells the truth. Ryan says Sherrie lies. Bernita says Ryan tells the truth. Tamika says Bernita lies. Jerry says Tamika lies. Does Jerry tell the truth?
Few-Shot-CoT
A: Let’s think step by step. (1) Sherrie tells the truth. So, we know that Sherrie tells the truth. (2) Ryan says Sherrie lies. Since we know from (1) that Sherrie tells the truth, if Ryan says Sherrie lies, then Ryan lies. (3) Bernita says Ryan tells the truth. Since we know from (2) that Ryan lies, if Bernita says Ryan tells the truth, then Bernita lies. (4) Tamika says Bernita lies. Since we know from (3) that Bernita lies, if Tamika says Bernita lies, then Tamika tells the truth. (5) Jerry says Tamika lies. Since we know from (4) that Tamika tells the truth, if Jerry says Tamika lies, then Jerry tells the truth. Now, the question asks: Does Jerry tell the truth? We know from (5) that Jerry tells the truth. So the answer is Yes. %(Complex entity relationships and long reasoning chains interfere with reasoning)
Meta-Reasoning
A: The question can be simplified to: It is known that A = 1. Ryan says Sherrie lies: lies → A’ = 0. Since A = 1, A is not equal to A’, so B = 0. Bernita says Ryan tells the truth: truth → B’ = 1. Since B = 0, B is not equal to B’, so C = 0. Tamika says Bernita lies: lies → C’ = 0. Since C = 0, C is equal to C’, so D = 1. Jerry says Tamika lies: lies → D’ = 0. Since D = 1, D is not equal to D’, so E = 0. Since E = 0, so the answer is: no. ✓

heading: G Demonstration Design

heading: Q: For Halloween Debby and her sister combined the candy they received. Debby had 32 pieces of candy while her sister had 42. If they ate 35 pieces the first night, how many pieces do they have left?

heading: A: The question can be simplified to: It is known that A = 32, B = 42, C = 35. Add B to A, and then subtract C from A, now what is the value of B?

heading: Q: For Halloween Katie and her sister combined the candy they received. Katie had 8 pieces of candy while her sister had 23. If they ate 8 pieces the first night, how many pieces do they have left?

heading: A: Let¡¯s think step by step. Katie and her sister have a total of 8 + 23 = 31 pieces of candy. If they eat 8 pieces the first night, they have 31 - 8 = 23 pieces left. The answer is 23.

heading: Q: A pet store had 78 puppies. In one day they sold 30 of them and put the rest into cages with 8 in each cage. How many cages did they use?

heading: A: Let¡¯s think step by step. There are 78 puppies. 30 are sold, so that means there are 48 left. 48 divided by 8 is 6, so that means there are 6 cages with 8 puppies in each. The answer is 6.

heading: Q: A waiter had 14 customers to wait on. If 3 customers left and he got another 39 customers, how many customers would he have?

heading: A: Let¡¯s think step by step. The waiter had 14 customers to wait on. If 3 customers left, that means he would have 11 customers left. If he got another 39 customers, that means he would have 50 customers in total. The answer is 50.

heading: Q: Bianca was organizing her book case making sure each of the shelves had exactly 8 books on it. If she had 5 shelves of mystery books and 4 shelves of picture books, how many books did she have total?

heading: A: Let¡¯s think step by step. There are 5 shelves of mystery books. Each shelf has 8 books. So that¡¯s 40 mystery books. There are 4 shelves of picture books. Each shelf has 8 books. So that¡¯s 32 picture books. 40 + 32 = 72 books. The answer is 72.

heading: Q: Wendy uploaded 45 pictures to Facebook. She put 27 pics into one album and put the rest into 9 different albums. How many pictures were in each album?

heading: Q: A trivia team had 7 members total, but during a game 2 members didn¡¯t show up. If each member that did show up scored 4 points, how many points were scored total?

heading: A: Let¡¯s think step by step. There were 7 members on the team, but 2 members didn¡¯t show up. That means that there were 5 members that did show up. Each member that showed up scored 4 points. So if 5 members each scored 4 points, then the total number of points scored would be 5*4=20. The answer is 20.

heading: Q: Mike made 69 dollars mowing lawns over the summer. If he spent 24 dollars buying new mower blades, how many 5 dollar games could he buy with the money he had left?

heading: A: Let¡¯s think step by step. Mike made $69 from mowing lawns. He spent $24 on new mower blades. That means he has $45 left. Each game costs $5, so he could buy 9 games. The answer is 9.

heading: Q: The school cafeteria ordered 8 red apples and 43 green apples for students lunches. But, if only 42 students wanted fruit, how many extra did the cafeteria end up with?

heading: A: Let¡¯s think step by step. The school cafeteria ordered 8 red apples and 43 green apples for students lu
text: 637

heading: Q: A carpenter bought a piece of wood that was 0.41 meters long . Then she sawed 0.33 meters off the end . How long is the piece of wood now ?

heading: A: The question can be simplified to: It is known that A = 0.41, B = 0.33. Subtract B from A, now what is the value of A?

heading: A = A - B = 0.41 - 0.33 = 0.08

heading: So the answer is: 0.08

heading: Q: Melanie picked 4 plums , Dan picked 9 plums , and Sally picked 3 plums from the plum tree . How many plums were picked in total ?

heading: A: The question can be simplified to: It is known that A = 4, B = 9, C = 3. Add B to A, and then add C to A, now what is the value of A?

heading: Q: Sally picked 7 lemons and Mary picked 9 lemons from the lemon tree . How many lemons were picked in total?

heading: Q: Gordon bought 3.42 pounds of fruit for a class party . The class ate 2.2 pounds of the fruit . How much fruit is ?

heading: A: Let’s think step by step. Gordon bought 3.42 pounds of fruit. The class ate 2.2 pounds of the fruit. That means that there is 3.42 - 2.2 = 1.22 pounds of fruit left. The answer is 1.22.

heading: Q: Mary had 21 dimes and 38 pennies in her bank . Her dad borrowed 18 pennies from Mary . How many pennies does she have now ?

heading: A: Let’s think step by step. Mary had 21 dimes and 38 pennies in her bank. Her dad borrowed 18 pennies from Mary. So now, Mary has

heading: 21 dimes and 20 pennies in her bank. The answer is 20.

heading: Q: Joan found 72 seashells and 12 starfishes on the beach . She gave Alyssa some of her seashells . She has 28 seashell . How many seashells did she give to Alyssa ?

heading: Q: Last year , 90171 people were born in a country , and 16320 people immigrated to it . How many new people began living in the country last year ?

heading: A: Let’s think step by step. First, 90171 people were born in the country. Then, 16320 people immigrated to the country. That means there were 90171 + 16320 = 106491 new people living in the country last year. The answer is 106491.

heading: Q: A construction company is repaving a damaged road . So far , they have repaved a total of 4938 inches of the road . Today , they repaved 805 inches of the road . How many inches of the road had they repaved before today ?

heading: A: Let’s think step by step. The company has repaved a total of 4938 inches of the road. Today, they repaved 805 inches of the road.

heading: So before today, they had repaved 4938 - 805 = 4133 inches of the road. The answer is 4133.

heading: Q: There are 5 scissors and 3 pencils in the drawer . Jason placed 4 scissors in the drawer . How many scissors are now there in total ?

heading: A: Let’s think step by step. There are 5 scissors and 3 pencils in the drawer. Jason placed 4 scissors in the drawer. Now, there are 9 scissors in the drawer. The answer is 9.

heading: Q: There are 37 short bushes and 30 tall trees currently in the park . Park workers will plant 20 short bushes today . How many short bushes will the park have when the workers are finished ?

heading: A: Let’s think step by step. There are 37 short bushes and 30 tall trees currently in the park. Park workers will plant 20 short bushes today. This means that when the workers are finished, there will be a total of 57 short bushes in the park. The answer is 57.
text: 638
639

heading: Q: A coin is heads up. Jeff does not flip the coin. Jen flips the coin. Giselle flips the coin. Noel does not flip the coin. Is the coin still heads up?

heading: A: The question can be simplified to: It is known that S = A = 1. Jeff does not flip the coin, so A = A = 1; Jen flips the coin, so A = -A = -1; Giselle flips the coin, so A = -A = -(-1) = 1; Noel does not flip the coin, so A = A = 1. Since now A is equal to S, so the coin is still head up.

heading: So the answer is: yes

heading: Q: A coin is heads up. Wilmer flips the coin. Valerie does not flip the coin. Melissa flips the coin. Eloisa flips the coin. Is the coin still heads up?

heading: A: The question can be simplified to: It is known that S = A = 1. Wilmer flips the coin, so A = -A = -1; Valerie does not flip the coin, so A =

heading: A = -1; Melissa flips the coin, so A = -A = -(-1) = 1; Eloisa flips the coin, so A = -A = -1. Since now A is not equal to S, so the coin is not still head up.

heading: So the answer is: no

heading: Q: A coin is heads up. Ka flips the coin. Sherrie flips the coin. Is the coin still heads up?

heading: A: The coin was flipped by Ka and Sherrie. So the coin was flipped 2 times, which is an even number. The coin started heads up, so after an even number of flips, it will still be heads up. So the answer is yes.

heading: Q: A coin is heads up. Jamey flips the coin. Teressa flips the coin. Is the coin still heads up?

heading: A: The coin was flipped by Jamey and Teressa. So the coin was flipped 2 times, which is an even number. The coin started heads up, so after an even number of flips, it will still be heads up. So the answer is yes.

heading: Q: A coin is heads up. Maybelle flips the coin. Shalonda does not flip the coin. Is the coin still heads up?

heading: A: The coin was flipped by Maybelle. So the coin was flipped 1 time, which is an odd number. The coin started heads up, so after an odd number of flips, it will be tails up. So the answer is no.

heading: Q: A coin is heads up. Millicent does not flip the coin. Conception flips the coin. Is the coin still heads up?

heading: A: The coin was flipped by Conception. So the coin was flipped 1 time, which is an odd number. The coin started heads up, so after an odd number of flips, it will be tails up. So the answer is no.

heading: Q: A coin is heads up. Sal flips the coin. Raymond does not flip the coin. Is the coin still heads up?

heading: A: The coin was flipped by Sal. So the coin was flipped 1 time, which is an odd number. The coin started heads up, so after an odd number of flips, it will be tails up. So the answer is no.

heading: Q: A coin is heads up. Conception flips the coin. Kristian does not flip the coin. Is the coin still heads up?

heading: A: The coin was flipped by Conception. So the coin was flipped 1 time, which is an odd number. The coin started heads up, so after an odd number of flips, it will be tails up. So the answer is no.

heading: Q: A coin is heads up. Inga does not flip the coin. Elanor does not flip the coin. Is the coin still heads up?

heading: A: The coin was flipped by no one. So the coin was flipped 0 times. The coin started heads up, and it was not flipped, so it is still heads up. So the answer is yes.

heading: Q: A coin is heads up. Ryan flips the coin. Shaunda flips the coin. Is the coin still heads up?

heading: A: The coin was flipped by Ryan and Shaunda. So the coin was flipped 2 times, which is an even number. The coin started heads up, so after an even number of flips, it will still be heads up. So the answer is yes.
text: 640

heading: Q: Sherrie tells the truth. Vernell says Sherrie tells the truth. Alexis says Vernell lies. Michaela says Alexis tells the truth. Elanor says Michaela tells the truth. Does Elanor tell the truth?

heading: Q: Jerry lies. Millicent says Jerry tells the truth. Sal says Millicent tells the truth. Amberly says Sal tells the truth. Shenna says Amberly lies. Does Shenna tell the truth?

heading: Q: Kristian lies. Sherrie says Kristian lies. Delbert says Sherrie lies. Jerry says Delbert tells the truth. Shalonda says Jerry tells the truth. Does Shalonda tell the truth?

heading: Q: Question: Fidel tells the truth. Jerry says Fidel tells the truth. Vina says Jerry tells the truth. Millicent says Vina lies. Raymond says Millicent lies. Does Raymond tell the truth?

heading: Q: Question: Kristian lies. Millie says Kristian lies. Maybelle says Millie tells the truth. Fidel says Maybelle lies. Leda says Fidel lies. Does Leda tell the truth?

heading: Q: Question: Kristian tells the truth. Michaela says Kristian lies. Raymond says Michaela tells the truth. Osvaldo says Raymond tells the truth. Jamey says Osvaldo tells the truth. Does Jamey tell the truth?
text: 641

heading: At the end of the dance, Alice is dancing with Patrick. So the answer is (C).

heading: A: Let's think step by step.(0) At the start: Alice: Lola, Bob: Rodrigo, Claire: Patrick.(1) Alice and Bob switch partners: Alice: Rodrigo, Bob: Lola, Claire: Patrick.(2) Claire and Bob switch partners: Alice: Rodrigo, Bob: Patrick, Claire: Lola.(3) Bob and Alice switch partners: Alice: Patrick, Bob: Rodrigo, Claire: Lola.

heading: At the end of the game, Alice has the pink ball. So the answer is (C).

heading: A: Let's think step by step.(0) At the start: Alice: white, Bob: purple, Claire: pink.(1) Bob and Alice swap balls: Alice: purple, Bob: white, Claire: pink.(2) Bob and Claire swap balls: Alice: purple, Bob: pink, Claire: white.(3) Bob and Alice swap balls: Alice: pink, Bob: purple, Claire: white.

heading: Q: Alice, Bob, and Claire are playing a game. At the start of the game, they are each holding a ball: Alice has a white ball, Bob has a purple ball, and Claire has a pink ball.

heading: At the end of the game, Bob has the yellow ball. So the answer is (A).

heading: A: Let's think step by step.(0) At the start: Alice: yellow, Bob: blue, Claire: pink.(1) Claire and Alice swap balls: Alice: pink, Bob: blue, Claire: yellow.(2) Alice and Bob swap balls: Alice: blue, Bob: pink, Claire: yellow.(3) Claire and Bob swap balls: Alice: blue, Bob: yellow, Claire: pink.

heading: Q: Alice, Bob, and Claire are playing a game. At the start of the game, they are each holding a ball: Alice has a yellow ball, Bob has a blue ball, and Claire has a pink ball.

heading: A: The question can be simplified to: It is known that A = 1, B = 2, C = 3.

heading: A: The question can be simplified to: It is known that A = 1, B = 2, C = 3.

heading: Q: Alice, Bob, and Claire are friends and avid readers who occasionally trade books. At the start of the semester, they each buy one new book: Alice gets The Fellowship of the Ring, Bob gets Frankenstein, and Claire gets The Great Gatsby.

heading: A: The question can be simplified to: It is known that A = 1, B = 2, C = 3.

heading: Q: Alice, Bob, and Claire are on the same team in a soccer match. At the start of the match, they are each assigned to a position: Alice is playing fullback, Bob is playing goalkeeper, and Claire is playing right winger.

heading: Q4: Where does Elizabeth think Isabella thinks Hannah thinks Jacob thinks the tomato is?

heading: Q3: Where does Isabella think Hannah thinks Jacob thinks the tomato is?

heading: Q2: Where does Hannah think Jacob thinks the tomato is?

heading: Q1: Where does Jacob really think the tomato is?

heading: 14 Isabella, Carter, Elizabeth, Hannah and Jacob entered the waiting_room. Q0: Where is the tomato really?

heading: 13 Jacob exited the bedroom.

heading: 12 Jacob made no movements and stayed in the bedroom for 1 minute.

heading: 11 Hannah exited the bedroom.

heading: 10 Jacob lost his watch.

heading: 9 Hannah made no movements and stayed in the bedroom for 1 minute.

heading: 8 Elizabeth exited the bedroom.

heading: 7 Elizabeth moved the tomato to the blue_container.

heading: 6 Carter exited the bedroom.

heading: 5 Carter made no movements and stayed in the bedroom for 1 minute.

heading: 4 Isabella exited the bedroom.

heading: 3 Isabella moved the tomato to the green_box.

heading: 2 The tomato is in the green_bottle.

heading: 1 Isabella, Carter, Elizabeth, Hannah and Jacob entered the bedroom.

heading: Q4: Where does Elizabeth think Isabella thinks Hannah thinks Jacob thinks the tomato is?

heading: Q3: Where does Isabella think Hannah thinks Jacob thinks the tomato is?

heading: Q2: Where does Hannah think Jacob thinks the tomato is?

heading: Q1: Where does Jacob really think the tomato is?

heading: 14 Isabella, Carter, Elizabeth, Hannah and Jacob entered the waiting_room. Q0: Where is the tomato really?

heading: 13 Jacob exited the bedroom.

heading: 12 Jacob made no movements and stayed in the bedroom for 1 minute.

heading: 11 Hannah exited the bedroom.

heading: 10 Jacob lost his watch.

heading: 9 Hannah made no movements and stayed in the bedroom for 1 minute.

heading: 8 Elizabeth exited the bedroom.

heading: 7 Elizabeth moved the tomato to the blue_container.

heading: 6 Carter exited the bedroom.

heading: 5 Carter made no movements and stayed in the bedroom for 1 minute.

heading: 4 Isabella exited the bedroom.

heading: 3 Isabella moved the tomato to the green_box.

heading: 2 The tomato is in the green_bottle.

heading: 1 Isabella, Carter, Elizabeth, Hannah and Jacob entered the bedroom.

heading: Future Work
text: How to fully push the upper limit of LLM’s semantic resolution ability, so as to further improve its reasoning ability, is a promising future work. C. 3 Formal Pattern Flexibility
So far, most symbolic reasoning work focuses on mapping natural semantics to formal languages with complete grammar (such as Python and SQL). However, this grammatical completeness actually limits the form conversion, and it has higher requirements for the abstraction of the original reasoning tasks. To verify the flexibility of our paradigm, we contrast Program-of-Thought (PoT), a Text-to-Python reduction approach for reasoning tasks (Chen et al. , 2022; Gao et al. , 2022). 633
Meta-Reasoning Program-of-Thought0
20
40
60
80
100 94. 5
46. 7
86. 6 86. 8
99. 2
44. 0
16. 8 MultiArith AddSub Lies Track (7)
Paradigm
A cc
ur ac
y (%
)
Meanwhile, to keep the settings consistent, we eliminate the call of an external interpreter in the PoT paradigm but utilize the LLMs themselves to complete the entire reasoning step, and select the same demonstrations for the two paradigms. Figure 10 shows the performance comparisons of PoT and MR paradigms on four datasets. For the two arithmetic reasoning tasks (MultiArith and AddSub), the performance of PoT fluctuates wildly after removing the external interpreter. For the two symbolic reasoning tasks (Lies and Track), PoT is almost completely ineffective. In contrast, MR has stronger flexibility when encountering reasoning tasks that are not easily programmed