GraphReason: Enhancing Reasoning Capabilities of Large Language Models through A Graph-Based Verification Approach (2024)

Abstract: Large Language Models (LLMs) have showcased impressive reasoning capabilities, particularly when guided by specifically designed prompts in complex reasoning tasks such as math word problems. These models typically solve tasks using a chain-of-thought approach, which not only bolsters their reasoning abilities but also provides valuable insights into their problem-solving process. However, there is still significant room for enhancing the reasoning abilities of LLMs. Some studies suggest that the integration of an LLM output verifier can boost reasoning accuracy without necessitating additional model training. In this paper, we follow these studies and introduce a novel graph-based method to further augment the reasoning capabilities of LLMs. We posit that multiple solutions to a reasoning task, generated by an LLM, can be represented as a reasoning graph due to the logical connections between intermediate steps from different reasoning paths. Therefore, we propose the Reasoning Graph Verifier (GraphReason) to analyze and verify the solutions generated by LLMs. By evaluating these graphs, models can yield more accurate and reliable results.Our experimental results show that our graph-based verification method not only significantly enhances the reasoning abilities of LLMs but also outperforms existing verifier methods in terms of improving these models’ reasoning performance.

heading: 1 Introduction
text: Large Language Models (LLMs) have demonstrated exceptional capabilities in a variety of human tasks (Zhao et al., 2023). Among the many abilities LLMs possess, their reasoning capacity is of paramount importance (Kojima et al., 2023; Huang and Chang, 2023). This has been substantiated by recent progresses (Wei et al., 2022; Zhou et al., 2023; Lampinen et al., 2022a). Equipped with the ability to reason, especially in a multistep manner, LLMs can decompose complex prob-
lems into simpler tasks, thereby facilitating their resolution. In everyday life, many complex tasks typically require multi-step solutions. A prime example of a reasoning task is arithmetic reasoning, also known as solving math word problems (Zhang et al., 2019). These math word problems represent simplified versions of complex real-life situations.
The reasoning ability is inherent in Large Language Models (LLMs), but it necessitates specific methods for manifestation. To activate the robust reasoning capability of LLMs, the use of specially designed prompts should be considered. Numerous methods have been proposed to tap into this potential, among which chain-of-thought reasoning (Wei et al., 2022) and in-context learning (Lampinen et al., 2022b) are two notable approaches. Chainof-thought reasoning can elucidate the reasoning paths during the process. In-context learning fur-
1
nishes LLMs with exemplary cases, thereby enabling them to learn from and simulate these examples for improved results. In the arithmetic reasoning scenario, GPT-4 can achieve an accuracy of 92% on the GSM8K dataset using 5-shot chainof-thought prompts (Cobbe et al., 2021a). This represents a level of difficulty that a bright middle school student should be capable of handling. As depicted in Figure 1, this illustrates a multi-step arithmetic reasoning process in LLMs.
In addition to further training of LLMs and prompt design, some methods have been proposed to enhance the reasoning capabilities of LLMs from the perspective of output verification. The primary idea is to have LLMs generate reasoning paths multiple times, and then design a verifier to evaluate these paths and deliver the final results. (Wang et al., 2023) introduces the concept of self-consistency, based on the intuition that a complex reasoning problem usually allows for multiple thought processes, all leading to a unique correct answer. (Li et al., 2023) also proposes All Roads Lead to Rome, which introduces a step-aware verifier to analyze reasoning paths not just through the entire path, but at every step. However, both methods treat each reasoning path as an independent entity and do not consider the potential interrelation and interaction between different reasoning paths. Once reasoning paths are disassembled into steps, intermediate steps from one path may bear reasoning relations to other reasoning paths. These methods do not perceive all LLM outputs for a given input as a collective entity, thereby failing to analyze the internal relations of all candidate paths in depth.
Inspired by these observations, we propose Reasoning Graph Verifier (GraphReason) in this paper. We posit that reasoning paths of one question can form reasoning graphs, where similar intermediate reasoning steps can be merged into the same node. With a graph structure, we can more effectively model and capture the reasoning logic between intermediate steps from different reasoning paths. Specifically, we first construct a reasoning graph based on all outputs from LLMs, and then train a verifier to learn the relationship between the graph structure and the final answer. During the prediction stage, we process the data in the same way as in the training stage, and use the verifier to evaluate each reasoning graph. We then select the reasoning graph with the highest score, using its answer as the final answer. To the best of our
knowledge, we are the first to approach reasoning logic of LLMs from a graph perspective. We conduct extensive experiments to demonstrate the improvements over the original LLMs, and show that our method outperforms other verifiers.
In summary, our contributions are as follows:
• We propose a graph-based verification method, GraphReason, aimed at significantly enhancing the reasoning capabilities of large language models without the need for additional training of LLMs.
• We establish an arithmetic reasoning benchmark using three Math Word Problem datasets to illustrate the fundamental reasoning performance of large language models, and to provide a fair comparison of the performance of various existing verifiers.
• Our experimental results indicate that the method proposed in this paper outperforms other enhancement methods. We also provide an extensive analysis of the limitations and future potential of GraphReason.

heading: 2 Related Works
text: Reasoning of Fine-tuning Models has been extensively studied. It focuses on addressing reasoning tasks using a general sequence-to-sequence approach, enhanced by reasoning-aware pre-training or fine-tuning of language models. (Cobbe et al., 2021a) proposed training a verifier to rank solutions sampled from fine-tuned language models. (Yoran et al., 2022; Wang et al., 2022) suggested equipping language models with reasoning abilities by generating training examples with human-designed templates. (Pi et al., 2022) proposed injecting reasoning capabilities into language models by continually pre-training on program execution data.
Several studies have focused on imbuing PLM with reasoning ability for specific tasks, such as arithmetic reasoning (Cobbe et al., 2021a; Miao et al., 2020; Patel et al., 2021), commonsense reasoning (Talmor et al., 2019), and inductive reasoning (Sinha et al., 2019). For instance, various strategies have been proposed to improve language models’ performance on arithmetic reasoning tasks, often referred to as math word problems. (Xie and Sun, 2019) proposed a tree-structured decoder to generate an equation tree, while (Zhang et al., 2020) applied graph convolutional networks to extract relationships
of quantities in math problems. (Li et al., 2022) used contrastive learning to better learn patterns in math word problems. However, (Valmeekam et al., 2023; Rae et al., 2022) suggested that reasoning, particularly multi-step reasoning, is often a weakness in language models and other NLP models.
Reasoning of Large Language Models has garnered significant attention and demonstrated immense potential. Recent advancements in LLMs suggest that the ability for multi-step reasoning is already embedded within these large-scale models (Kojima et al., 2023; Huang and Chang, 2023), such as PaLM (Chowdhery et al., 2022), GPT4 (OpenAI, 2023). Therefore, providing an adequate prompt is sufficient to utilize this reasoning ability. For example, the prompting method proposed by (Kojima et al., 2023; Wei et al., 2022), which is based on a chain-of-thought, could aid LLMs in generating text with arithmetic reasoning and common factual knowledge. Following (Wei et al., 2022), experiments on current language models demonstrated that chain-of-thought prompting could enhance the accuracy of solving math problems from 18% to 57%. (Lampinen et al., 2022b) included explanations in the in-context examples and tested the influence of explanations by evaluating the score between explain-then-predict and predict-then-explain. Moreover, (Zhou et al., 2023) suggested a two-stage prompting strategy, leastto-most prompting, which breaks down a complex problem into a series of subproblems and solves them step-by-step. (Li et al., 2023) proposed sampling multiple times from diverse prompts to enhance the variety of responses.
In addition to designing prompts, adopting additional strategies like verifier has contributed to enhancing the performance of reasoning abilities of large language models. For instance, (Wang et al., 2023) proposes self-consistency, which involves sampling different reasoning paths from the language model, and then returning the most consistent final answer via majority voting. (Li et al., 2023) used a step-aware voting verifier to enhance the reasoning ability of LLMs from two perspectives. These methods strive to augment the reasoning abilities or yield superior reasoning results without necessitating additional training of LLMs. Our work continues this research direction, with a specific focus on developing a novel graph-based verifier to boost the reasoning capabilities of LLMs.
3 Methodology
3.1 GraphReason Framework Problem 1 (Reasoning to Solve Problems) Given a set of n math word problems Q = {Q1, Q2, ..., Qn}, where each Qi is represented by the text description of a single math word problem, the goal of reasoning to solve math word problems is to generate the answers A = {A1, A2, ..., An} for these problems. Here, each Ai represents the generated text of the corresponding answer. During the process of large language models generating answers, a set of n reasoning paths for solutions S = {S1, S2, ..., Sn} is also produced. Each solution Si is represented as Si = {Q,Step1, Step2, ..., Stepl, A}, where each Stepi denotes the intermediate steps in the step-by-step solutions.
We propose GraphReason to verify the solutions generated by LLMs in order to improve the final answer accuracy. This method is a graph-based verification technique that analyzes reasoning paths from generated solutions from a graph perspective. The final answer is obtained without modifying the original LLMs, functioning much like a plugin. As illustrated in Figure 2, there are two steps in the training stage: Graph Construction and Graph Classification. In the Graph Construction step, we obtain the generated solution from LLMs with the specific designed prompt and group them according to their final answers. We split reasoning paths by steps and then merge intermediate steps with identical expression to form reasoning graphs. In the Graph Classification step, we classify these reasoning graphs with the additional feature of the sum of scores from the base verifier to train the integrated verifier model. In the prediction stage, the candidate solutions are first generated by LLMs. We process them in the same manner as in the training stage, then we use trained verifier to evaluate the scores of each candidate solution. The best solution, denoted by the highest score, is selected as the final predicted answer. We will now provide a detailed introduction to the entire process.

heading: 3.2 Prompt Design
text: To improve the output of Language Models (LLMs) in providing solutions, it is essential to design effective prompts. We incorporate chain-of-thought and in-context learning to enable LLMs to generate step-by-step answers for math word problems. The language models generate output y based on the
input x using the following equation:
p(y|C,x) = |y|∏
t=1
pLM (yt|C,x,y < t), (1)
where, C represents the input provided to the LLMs prior to the current math word problem’s question. C is a concatenation of k exemplars, denoted as:
C = [(Q1, S1, A1); (Q2, S2, A2), ...; (Qk, Sk, Ak)], (2) where, Qi represents the question, Si represents the intermediate steps of the solution, and Ai represents the answer. We set k to five in this study, resulting in a prompt that consists of five questionanswer pairs sampled from the training split of a math word problem dataset. Therefore, the prompt can be denoted as:
Prompt = [C;Q], (3)
where Q represents the question of the current math word problem.
Using a greedy decoding approach to sample one output from LLMs may not be robust. It can lead to instability and occasional errors. To address this, (Wang et al., 2023) propose the concept of self-consistency. This approach involves sampling different reasoning paths from the language model and then selecting the most consistent final answer through majority voting. Instead of using greedy decoding to sample only once and verify, they utilize sampling decoding to sample N1 times. We also follow the idea presented by (Li et al., 2023) in their work named All Roads Lead to Rome. This approach involves generating N2 diverse prompts for LLMs to produce multiple outputs. By employing multiple sampling decodes on diverse prompts, we can obtain generated solutions from different sources. Specifically, we obtain N = N1 × N2
diverse reasoning paths for each question. In our main experiments, we set N1 = 10 and N2 = 3. These solutions will be further processed and verified using our designed verifier.

heading: 3.3 Reasoning Graph Construction
text: After generating multiple solutions for a question, it becomes necessary to construct reasoning graphs based on the reasoning paths taken by these solutions.
As shown in Figure 3, we begin by grouping all the generated solutions for a particular question according to their final answer. Since these solutions originate from the same question, their reasoning paths will share the same starting point. Similarly, solutions with the same final answer will have the same endpoint, as their reasoning paths converge. Therefore, a group of generated solutions with the same final answer can form a reasoning graph with a uniform start node (question node) and end node (answer node). We define this division process as follows:
S = {SA1 , SA2 , ..., SAn}, (4)
where S represents the set of generated solutions for a question, and SAi = {S1, S2, ..., Sm} is the subset of generated solutions that all have the same final answer Ai.
For each subset of generated solutions SAi , we construct a reasoning graph. This construction is motivated by the understanding that each step in the reasoning path of a generated solution does not exist in isolation from the other solutions. The steps from one solution’s reasoning path can impact the steps from another solution, enhancing the overall reasoning process. We utilize the graph structure to model and capture these relationships between steps from different solutions. As the different reasoning paths can benefit each other, we construct
a reasoning graph to link these paths together. As shown in Figure 4, the primary operation here is the merging of identical intermediate nodes in reasoning paths into a single graph node. We first compare the reasoning steps from any two solution reasoning paths. If they have the same intermediate steps of arithmetic expression, we merge them into the same node, and if they differ, we do not. For reasoning math word problems here, we define reasoning steps as the current arithmetic expression without other language text in the current reasoning step for clarity. It can help us simplify construction of reasoning graphs in the reasoning task. The detailed algorithm for constructing a reasoning graph is shown in Algorithm 1.
The generated solutions, divided by their final
answers {SA1 , SA2 , ..., SAn}, can be transformed into n reasoning graphs of generated solutions {GA1 , GA2 , ..., GAn}.
Regarding the node features in the graph, we select the score from the Base Verifier and the node degree. We believe the score from the Base Verifier encapsulates the semantic information of solutions, and the node degree contains information about the graph structure. The Base Verifier is trained independently from the whole framework. It is designed to judge whether a single reasoning path of one solution is correct, which is a binary text classification task. After training, it can be used to verify any single solution and assign a score ∈ (0, 1) to evaluate the likelihood of the solution being correct, where score = 0.99 suggests a 99% probability of the solution being correct. We use the score from the Base Verifier to better incorporate solution semantic information because, according to our experiments, it is challenging to model semantic information while modeling reasoning logic information. The score of a step is the same as its solution score. Therefore, for one step node V , it has many scores {scorea, scoreb, ..., scorec} from different solutions. The feature of one node Vi in the graph is then concatenated by the selected feature, which can be represented as:
V = [scoremeani , score max i , score min i ,
scorenumi , in_degreei], (5)
where V ∈ R5, scoremeani is the mean of all scores of one step Vi, scoremaxi is the maximum score, scoremini is the minimum score, score num i is the number of scores, and in_degreei is the in-degree of the step node Vi.
Algorithm 1 Reasoning graph construction algorithm Input: generated solutions SAi which have the same final answers Output: a reasoning graph GAi
1: node_num← 0 2: node2id← dict() 3: edges← list() 4: for each reason_path in SAi do 5: for each step in reason_path do 6: if step not in node2id.keys() then 7: node2id[step]← node_num 8: node_num← node_num+ 1 9: end if
10: end for 11: end for 12: for each reason_path in SAi do 13: for each step in reason_path do 14: start_node← node2id[last_step] 15: end_node← node2id[step] 16: if (start_node, end_node) not in edges then 17: edges.add((start_node, end_node)) 18: end if 19: last_step← step 20: end for 21: end for 22: GAi ← graph(node2id, edges)
In this way, we can obtain multiple reasoning graphs to represent all generated solutions from LLMs for a single math word problem question.

heading: 3.4 Verifier Design
text: Our designed verifier GraphReason,is used to evaluate the answer of a generated solutions group, which is also represented as a reasoning graph. This verifier has two inputs: the graph and the sum of solution scores. We employ the Graph Isomorphism Network (GIN) (Xu et al., 2019) to perform node feature propagation, thereby encoding the information from the reasoning graphs we obtained. The node feature is propagated and aggregated as follows: h(k)v = MLP (k)((1 + ε(k)) · h(k−1)v + ∑ u∈N(v) h(k−1)u ) ,
(6)
where h(k)v represents the state of node v after the kth update. MLP (k) refers to a multi-layer perceptron in the kth layer. N(v) denotes all the neighbors of node v and ε is a learnable parameter. Then,
we perform a sum readout to obtain the representation of the reasoning graph:
hG = ∑
v∈G h(k)v , (7)
where hG ∈ R5. We set k to 3, signifying the application of three layers of GIN. Concurrently, the sum of the scores of solutions with the same final answer, Ai, denoted as scoreAi , is represented as follows:
scoreA = ∑
i∈SA scorei. (8)
Then a reasoning graph can then be represented as:
G = [hG, scoreA], (9)
where G ∈ R6. The target label of the graph y ∈ {0, 1} indicates whether the final answer matches the correct final answer. We compute the loss and train the verifier model by:
L = n∑
i=1
LBCE(labeli, f(Gi)), (10)
where i represents the number of solution subset among all n subsets after grouping solutions. The corresponding reasoning graph for this subset is denoted by Gi, and f() is a linear classifier.

heading: 3.5 Answer Verification
text: During the prediction stage, all generated solutions are processed in the same way as in the training stage. The trained verifier is then used to evaluate the scores of each reasoning graph, each of which represents a group of solutions that yield the same final answer. The final answer associated with the highest score is selected as our final predicted answer:
ŷ = Answer[argmax i scorei], (11)
where scorei denotes the score of the reasoning graph Gi, as determined by our verifier. Answer represents the list of all candidate final answers. By predicting the number of the optimal reasoning graph, we can determine the final predicted result of the current reasoning task.

heading: 4 Experiments
text: In this section, we conducted extensive experiments to demonstrate the performance of GraphReason, along with a more in-depth analysis. Universally, we reproduced all types of verifiers to report their results based on the same generated solutions. Our experiments are conducted in two settings: Arithmetic Reasoning and Commonsense Reasoning. We ensured a fair comparison by setting the same random seed, using the same hardware environment, and applying similar hyperparameters. We used accuracy as the metric to evaluate the ability of solving math word problems, which determines whether the final answer is correct or not.

heading: 4.1 Training Details
text: For LLMs sampling, we use gpt-3.5-turbo as our base LLMs and set the temperature t to 1. All verifiers use the same LLMs’ output. Regarding verifier training, we fine-tune on bert-base-uncased (Devlin et al., 2019). We employ the AdamW optimizer (Loshchilov and Hutter, 2019) to optimize the model parameters during training. We apply differential learning rates, setting the learning rate of the final linear classifier to 4e-2, while the other graph neural network layers are set to 4e-3. The activation layer between them is ReLU (Agarap, 2019). The batch size in each training step is set to 2. The batch size is small because the verifier needs to verify multiple reasoning graphs for a single question.
To ensure a fair comparison between the Voting Verifier, Simple Verifier, and GraphReason, we use the same trained base verifier for all three approaches.
The details of the datasets and baselines are provided in Appendix A and Appendix B, respectively.

heading: 4.2 Main Results
text: We present the main results in Table 1. As can be seen from the table, GraphReason significantly enhances the original gpt-3.5-turbo’s reasoning abilities across all three datasets, for instance, improving accuracy by 13.0% (72.7%→ 85.7%) on GSM8K. It is also evident that our method surpasses other verifier methods with the same output from LLMs and achieves the state-of-the-art on all three datasets.
Additionally, the Step-aware Voting Verifier improves upon the Voting Verifier by recognizing that not all steps in an incorrect reasoning path are equally erroneous, and some steps may still be useful for reasoning. We believe this hypothesis is overly simplistic and cannot describe complex logical relationships among steps. According to Table 1, it leads to some metric decline, and the same finding also observed in the original paper. Furthermore, it does not perform well in the StrategyQA task, because there are no gold reasoning paths for the training of this commonsense reasoning task. In this task, the reasoning paths are generated and pseudo, indicating a requirement for gold labels at each step of the reasoning process. However, our paper consistently improves upon the Voting Verifier by considering complex relationship between different reasoning paths through reasoning graphs. We enhance the previous method, which did not consider relations in steps between different solutions, by 0.3% (85.4%→ 85.7%), 0.3% (85.1%→ 85.4%), 0.1% (96.9%→ 97.0%), and 0.5% (70.7% → 71.2%) across the four datasets.
Moreover, GraphReason yields only a slight improvement in performance on ASDiv-a, and the results are nearly identical. One reason for this is that the math word problems from ASDiv-a are sim-
pler compared to those in the other two datasets, based on our observations. In most cases, these problems do not require complex reasoning from a graph perspective to generate a satisfactory answer. It demonstrates that our method is particularly well-suited for such situations. We believe that GraphReason can offer more substantial improvements in the more complex scenario.

heading: 4.3 Ablation Study
text: We conducted an ablation study to evaluate the impact of each component on the overall performance of our method. Table 2 presents the results of this study, highlighting how these modules contribute to the improvement of the base model in distinct ways. It can be observed that the omission of any component leads to a decline in the final result. The solution semantics from the base verifier appear to be most crucial to GraphReason. The current method still relies on semantic information, which is reasonable since reasoning steps from different solutions require semantic information for better reasoning. We also notice that reasoning graphs bring a slight improvement to the entire method, thereby proving effectiveness of graph structure. The improvement is not substantial because we do not model the graph structure and semantic information simultaneously, and create a training gap here. Another essential factor is the complexity of graph classification, compounded by the presence of noise and limitations in our training data.
4.4 GraphReason with Different LLMs
To evaluate the compatibility of GraphReason and its effectiveness across various models, we additionally include gpt-4 (OpenAI, 2023) and PaLM-2 (Google, 2023) in our experiments. Given our limited computing resources, we utilize the same training data previously sampled from gpt-3.5-turbo. For testing in the GSM8K task, we select samples from 100 pieces of data from gpt-4 and PaLM-2
respectively. We conduct the sampling 10 times using three types of five exemplars, maintaining the same settings as in our previous experiments. Our method aims to enhance the original reasoning capabilities. Therefore, we do not include smallsized LMs, which typically exhibit weaker reasoning abilities.
From Table 3, it is evident that our method enhances the original reasoning performance of both GPT-4 and PaLM-2. However, there is a performance decline in gpt-4 when compared with the best baselines. The performance of GraphReason is comparable to that of the voting method. We hypothesize that this is because the reasoning patterns of GPT-4 differ from those of GPT-3.5-Turbo, and our verifier is trained specifically on GPT-3.5Turbo samples in this setting.

heading: 5 Conclusion
text: In this paper, we propose GraphReason, a novel and general method to enhance the reasoning abilities of large language models. Our method is the first to approach reasoning logic of large language models from a graph perspective and verifies candidate reasoning paths accordingly. We demonstrate the superiority of GraphReason through extensive experiments.
Limitations
There are several limitations in the current research that contribute to performance that is not as good as expected:
• Computing Resources. Despite the impressive performance it achieves, our framework requires large language models like GPT3.5. Inference with these models is more time-consuming and costly than fine-tuning models like BERT(Devlin et al., 2019). Some experiments, such as hyperparameter analysis, have already been conducted in related previous work and are not replicated here. Furthermore, due to limited computing resources, we have not conducted experiments with additional LLMs. We have chosen solely to use the representative LLM, GPT3.5, to compare the performance of the verifiers.
• Labeled CoT data. GraphReason is a complex verifier method that builds on graph classification, which requires more labeled data with wellannotated chain-of-thought reasoning paths for training. In the training of GraphReason, we use reasoning paths from LLMs’ output which may introduce significant noise. If the training data included labeled reasoning graphs, the performance would improve significantly.
• Other Reasoning Tasks. There are many types of reasoning tasks beyond math word problems, such as Commonsense Reasoning (Talmor et al., 2019), Inductive Reasoning (Sinha et al., 2019), etc. Given that graph construction is a complex process, we have focused mainly on solving math word problems (Arithmetic Reasoning). This focus allows for a more convenient implementation of the merging of intermediate steps. In other cases, identifying similar steps can be challenging. On the other hand, a math word problem typically presents a greater variety of potential solutions.
Nevertheless, we believe that future studies, conducted by us or others, can overcome these limitations and further improve upon our approach.

heading: A Datasets
text: We compared GraphReason with other methods on three different math word problem datasets: GSM8K (Cobbe et al., 2021a), SVAMP (Patel et al., 2021), and ASDiv-a (Miao et al., 2020) and one commonsense reasoning dataset: StrategyQA (Geva et al., 2021). We selected the subset ASDiv-a (arithmetic) from the original dataset ASDiv, which only involves arithmetic operations.
These three arithmetic reasoning datasets are more challenging than other math word problem datasets, making them more suitable for testing
the reasoning capability of LLMs with a verifier. As the GSM8K dataset is the only one providing step-by-step solutions as chain-of-thought exemplars, we chose exemplars from the GSM8K training dataset and tested them on all three datasets. Additionally, the training data for the verifier also used the GSM8K training data. In this setting, we could also demonstrate the transfer learning and generalization ability of our method. The size of the training split from GSM8k is 1000. The test data sizes for GSM8K, SVAMP, and ASDiv-a are 1319, 1000, and 1218, respectively.
In the StrategyQA commonsense reasoning task, we set the number of exemplars to 8 and select pseudo-exemplars from (Li et al., 2023). Additionally, we conduct five sampling iterations for each context of LLMs. From the entire dataset, we select a subset of 1,000 instances, allocating 700 for training and 300 for testing.

heading: B Baselines
text: In our evaluation, we consider the following baselines:
• Greedy Decode is a simple method that uses a greedy decoding strategy to sample once.
• Self-Consistency (Voting) (Wang et al., 2023) samples multiple times and selects the final answers based on majority voting.
• Simple Verifier (Cobbe et al., 2021b), which is also known as the Sampling and Re-ranking strategy, uses a verifier to assign scores to sampled solutions and selects the final answer with the highest score.
• Voting Verifier (Li et al., 2023) combines the Voting and Verifier approaches. It assigns total scores to answers from scores of all candidate solutions and selects the final answer with the highest score.
• DIVERSE (Step-aware Voting Verifier) (Li et al., 2023), which is the state-of-the-art method, considers the reasoning steps throughout the entire reasoning path. It recognizes that not all steps in an incorrect reasoning path are equally wrong and that some steps may still be useful for reasoning.
We primarily compare GraphReason with other verifiers using the same generated solutions from gpt-3.5-turbo. Additionally, we include some previous Fine-tuning state-of-the-art methods to reflect
the strong reasoning ability of LLMs. The previous Fine-tuning SOTA methods are denoted as follows: a: (Cobbe et al., 2021b), b: (Pi et al., 2022), c: (Miao et al., 2020), d: (Chowdhery et al., 2022).

heading: Future Work
text: Nevertheless, we believe that future studies, conducted by us or others, can overcome these limitations and further improve upon our approach