Paper: Forward-Backward Reasoning in Large Language Models for Mathematical Verification (ICLR-2024)

Reviewer comment (Experiments):
The main weakness in my opinion is the relatively narrow scope of the method (mathematical reasoning, and even then mostly on more numerical tasks). While I can see the idea being applicable to other domains, it's not obvious how to do so (if there were other compelling examples, e.g. in logical reasoning tasks, I believe the authors would have likely shown some of them). This is one disadvantage compared to Self-Consistency which, while less effective for math as shown here, is very widely applicable.

Author response:
> Q1. "The main weakness in my opinion is the relatively narrow scope of the method (mathematical reasoning, and even then mostly on more numerical tasks). While I can see the idea being applicable to other domains, it's not obvious how to do so (if there were other compelling examples, e.g. in logical reasoning tasks, I believe the authors would have likely shown some of them). This is one disadvantage compared to Self-Consistency which, while less effective for math as shown here, is very widely applicable."
> 
> "Do you believe FOBAR can be applied outside of numerical mathematical reasoning tasks? If so, what other tasks could be the closest potential targets?"

A1. (i) In this paper, **our focus is on mathematical tasks**, which often require a number of reasoning steps to reach the answer and are thus challenging. The proposed backward verification is **simple, general, and can be applied out-of-the-box to any mathematical task.**

(ii) **Extension to non-arithmetic tasks is possible** and follows a similar line. Similar to that for mathematical tasks, we **mask informative word/token/character** and then ask the LLM to predict. We give **two examples (Date Understanding reasoning task [1, 2] and Last-Letter-Concatenation reasoning task [1])** as follows.

`Example 1: Date Understanding`

*Question*: Yesterday was April 30, 2021. What is the date today in MM/DD/YYYY?

*Candidate Answer*: 05/01/2021 (correct), 05/02/2021 (wrong)

*Backward question for the candidate answer 05/01/2021*:
Yesterday was April 30, 2021. What is the date **x** in MM/DD/YYYY? If we know the answer to the above question is **05/01/2021**, what is the English word at **x**?

*Backward question for the candidate answer 05/02/2021*:
Yesterday was April 30, 2021. What is the date **x** in MM/DD/YYYY? If we know the answer to the above question is **05/02/2021**, what is the English word at **x**?

The LLM is more likely to predict the word "today" given the correct candidate answer 05/01/2021. We leave the details as future work.

`Example 2: Last-Letter-Concatenation`

*Question*: Take the last letters of each word in "Whitney Erika Tj Benito" and concatenate them.

*Candidate Answer*: yajo (correct), yaji (wrong)

*Backward question for candidate answer yajo*:
Take the last letters of each word in "Whitney Erika Tj Benit[]" and concatenate them. If we know the answer to the above question is **yajo**, what is the character at []?

*Backward question for candidate answer yaji*:
Take the last letters of each word in "Whitney Erika Tj Benit[]" and concatenate them. If we know the answer to the above question is **yaji**, what is the character at []?

Note that we use "[]" to mask the character instead of "x" (which is also a character). The LLM is more likely to predict "o" correctly at [] given the candidate answer yajo than yaji. Hence, the proposed FOBAR can be used in other types of tasks by masking the informative word/token/character in the questions.

(iii) Note that one contribution of the proposed FOBAR is a **novel** method to combine forward and backward reasoning for verification (i.e., $\mathbb{P}(\hat{A}_c) \propto (\mathbb{P}\_\text{forward}(\hat{A}_c) )^\alpha (\mathbb{P}\_\text{backward}(\hat{A}_c) )^{1-\alpha}$). The proposed method is **general** and **can be integrated into existing verification methods** (such as RCoT [3] and Self-Verification [4]) for non-arithmetic reasoning tasks.

Reviewer comment (Evaluation):
For results, I think the paper currently misses a qualitative discussion on what failure modes FOBAR addresses. While the idea is intuitive, it's not obvious where exactly self-consistency fails as a verification method, when FOBAR would be expected to succeed.

Author response:
> Q2. "For results, I think the paper currently misses a qualitative discussion on what failure modes FOBAR addresses. While the idea is intuitive, it's not obvious where exactly self-consistency fails as a verification method, when FOBAR would be expected to succeed."
> 
> "Are there any examples where self-consistency fails but FOBAR succeeds that help give insights into the failure modes that FOBAR addresses?"

A2. As suggested, we conducted a qualitative analysis on SingleEq using text-davinci-003 with CoT prompting. We observe that a common characteristic of problems that  Self-Consistency fails but FOBAR succeeds is: **Problems are difficult to solve in a forward direction,
but the correctness of a candidate answer can be easily verified in a backward manner**. An example is shown below.

*Question*: The sum of three consecutive odd numbers is 69. What is the smallest of the three numbers?

*Ground-truth answer*: 21

*Candidate answers generated by Self-Consistency*: 21 (16 times), 23 (24 times)

*An example of wrong forward reasoning chains*: The sum of three consecutive odd numbers is 69. We can use the formula n + (n+2) + (n+4) = 69 to solve for n. We get n = 23. This is the smallest of the three numbers. The answer is 23.

*Question for backward verification*: The sum of three consecutive odd numbers is ${\bf x}$. What is the smallest of the three numbers? If we know the answer to the above question is $21$, what is the value of unknown variable ${\bf x}$?

For the above backward question, it is easy for the LLM to calculate the value of **x**. We sample 10 backward reasoning chains to solve the question, and all of them predict ${\bf x}=69$. Hence, in this case, **backward verification is simpler than solving the original question.**

Reviewer comment (Experiments):
One baseline that is missing is the recent paper from Ling et al, 2023 on "Deductive Verification of Chain-of-Thought Reasoning". Their method is a forward verification method, and it would be interesting to see if (1) the gains from FOBAR + Self-Consistency alone are higher than what they get, and (2) if their method could even be a better drop-in replacement for the "forward" part in FOBAR. If there are directly comparable numbers on their paper, I'd strongly suggest citing them here. If cost is an issue, even doing this just for AquA, where you have the lowest absolute results and thus most room for improvement, could be already interesting.

Author response:
> Q3. "One baseline that is missing is the recent paper from Ling et al, 2023 on "Deductive Verification of Chain-of-Thought Reasoning". Their method is a forward verification method, and it would be interesting to see if (1) the gains from FOBAR + Self-Consistency alone are higher than what they get, and (2) if their method could even be a better drop-in replacement for the "forward" part in FOBAR. If there are directly comparable numbers on their paper, I'd strongly suggest citing them here. If cost is an issue, even doing this just for AquA, where you have the lowest absolute results and thus most room for improvement, could be already interesting."

**A3.** Thanks for bringing this recent paper [5] to our attention. As suggested, we replace the forward part in our FOBAR with the forward verification proposed in [5], and conducted additional experiments on AddSub, GSM8K, and AQuA using the GPT-3.5-Turbo. Table below shows the testing accuracy. As can be seen, **combining our backward verification with their forward verifications consistently boosts the performance.**

\begin{array}{l c c c}
\hline
&  \text{AddSub} & \text{GSM8K} & \text{AQuA} \newline
\hline
\text{Self-Consistency}  & 88.1 & 86.4 & 63.0 \newline
 \text{Self-Consistency + Backward Verification} &\textbf{ 88.4} & \textbf{87.4} & \textbf{63.4}\newline
\hline
\text{NP [1]}  & 93.67 &  87.05 & 70.34 \newline
\text{NP [1] + Backward Verification}  & \textbf{93.92} & \textbf{87.89} & \textbf{71.65} \newline
\hline
\text{NP + DV + UPV [1]} & 93.54 & 86.01 & 69.49  \newline
\text{NP + DV + UPV [1] + Backward Verification}& \textbf{93.92} &  \textbf{87.19} & \textbf{70.86} \newline	
\hline
\end{array}

Reviewer comment (Writing):
A minor point that I'd suggest improving on is the mathematical notation. This is not a barrier to understanding, since the idea is very simple, but will just help standardize papers in the community. For each candidate answer, there is an associated "correct" random variable. Equations 2 and 3 describe estimators for this unknown quantity. The estimator's name is what you'd put a hat on, not the variable. I'd indicate forward/backward as subscripts in the estimator's name, rather than as a "parameter".

Author response:
> Q4. "A minor point that I'd suggest improving on is the mathematical notation. This is not a barrier to understanding, since the idea is very simple, but will just help standardize papers in the community. For each candidate answer, there is an associated "correct" random variable. Equations 2 and 3 describe estimators for this unknown quantity. The estimator's name is what you'd put a hat on, not the variable. I'd indicate forward/backward as subscripts in the estimator's name, rather than as a "parameter"."

**A4.** Thanks for your suggestions, and we revised the paper accordingly.

Reviewer comment (Evaluation):
Are there any examples where self-consistency fails but FOBAR succeeds that help give insights into the failure modes that FOBAR addresses?

Author response:
> Q5. "When self-consistency fails, is it usually when the model consistently proposes a wrong answer, or proposes a completely divergent set of answers that are then ultimately sampled at random?"

**A5.** We conducted an additional analysis on Self-Consistency using the GPT-3.5-Turbo with ComplexCoT prompting. Table below shows the number of failure problems in Self-Consistency, and the number of failure problems with no correct chains or at least one correct chain. We can see that, in total, about 60\% of the failure problems have correct chains in Self-Consistency, while about 40\% of the problems have no correct ones and thus cannot be solved by backward verification.

\begin{array}{cccccccc}
\hline
& \text{AddSub} & \text{MultiArith} & \text{SingleEQ} & \text{SVAMP} & \text{GSM8K} & \text{AQuA} & \text{Total}\newline
\hline
\text{\\#fails} & 47 & 7 & 28 & 150 & 179 & 94 & 505\newline
\hline
\text{\\#has no correct answers} & 28 & 0 & 14 & 57 & 60 & 52 & 211 \newline
\text{\\#has the correct answer} & 19 & 7 & 14 & 93 & 119 & 42 & 294 \newline
\hline
\end{array}

Reviewer comment (Evaluation):
When self-consistency fails, is it usually when the model consistently proposes a wrong answer, or proposes a completely divergent set of answers that are then ultimately sampled at random?

Author response:
> Q6. "Have you observed cases where backward verification is especially hard, while forward verification is not?"
	
**A6.** As suggested, we analyzed the questions in GSM8K (using GPT-3.5-Turbo with ComplexCoT) that  Self-Consistency succeeds but FOBAR fails. There are 24 such questions. We give an example below, where backward reasoning fails to predict the masked numbers.
	
*Question*: Manolo bought 5 lollipops and 4 candies that cost \\$3.20. If each lollipop costs \\$0.40, how much will 10 lollipops and 10 candies cost him?

*Ground-truth answer*: 7

Candidate answers generated by Self-Consistency:  12 (once), 16 (4 times), 7 (5 times)

Number of times that the masked number is correctly predicted in backward reasoning: 12 (6 times), 16 (0 times), 7 (0 times)

Reviewer comment (Experiments):
Do you believe FOBAR can be applied outside of numerical mathematical reasoning tasks? If so, what other tasks could be the closest potential targets?

Author response:
> Q1. "The main weakness in my opinion is the relatively narrow scope of the method (mathematical reasoning, and even then mostly on more numerical tasks). While I can see the idea being applicable to other domains, it's not obvious how to do so (if there were other compelling examples, e.g. in logical reasoning tasks, I believe the authors would have likely shown some of them). This is one disadvantage compared to Self-Consistency which, while less effective for math as shown here, is very widely applicable."
> 
> "Do you believe FOBAR can be applied outside of numerical mathematical reasoning tasks? If so, what other tasks could be the closest potential targets?"

A1. (i) In this paper, **our focus is on mathematical tasks**, which often require a number of reasoning steps to reach the answer and are thus challenging. The proposed backward verification is **simple, general, and can be applied out-of-the-box to any mathematical task.**

(ii) **Extension to non-arithmetic tasks is possible** and follows a similar line. Similar to that for mathematical tasks, we **mask informative word/token/character** and then ask the LLM to predict. We give **two examples (Date Understanding reasoning task [1, 2] and Last-Letter-Concatenation reasoning task [1])** as follows.

`Example 1: Date Understanding`

*Question*: Yesterday was April 30, 2021. What is the date today in MM/DD/YYYY?

*Candidate Answer*: 05/01/2021 (correct), 05/02/2021 (wrong)

*Backward question for the candidate answer 05/01/2021*:
Yesterday was April 30, 2021. What is the date **x** in MM/DD/YYYY? If we know the answer to the above question is **05/01/2021**, what is the English word at **x**?

*Backward question for the candidate answer 05/02/2021*:
Yesterday was April 30, 2021. What is the date **x** in MM/DD/YYYY? If we know the answer to the above question is **05/02/2021**, what is the English word at **x**?

The LLM is more likely to predict the word "today" given the correct candidate answer 05/01/2021. We leave the details as future work.

`Example 2: Last-Letter-Concatenation`

*Question*: Take the last letters of each word in "Whitney Erika Tj Benito" and concatenate them.

*Candidate Answer*: yajo (correct), yaji (wrong)

*Backward question for candidate answer yajo*:
Take the last letters of each word in "Whitney Erika Tj Benit[]" and concatenate them. If we know the answer to the above question is **yajo**, what is the character at []?

*Backward question for candidate answer yaji*:
Take the last letters of each word in "Whitney Erika Tj Benit[]" and concatenate them. If we know the answer to the above question is **yaji**, what is the character at []?

Note that we use "[]" to mask the character instead of "x" (which is also a character). The LLM is more likely to predict "o" correctly at [] given the candidate answer yajo than yaji. Hence, the proposed FOBAR can be used in other types of tasks by masking the informative word/token/character in the questions.

(iii) Note that one contribution of the proposed FOBAR is a **novel** method to combine forward and backward reasoning for verification (i.e., $\mathbb{P}(\hat{A}_c) \propto (\mathbb{P}\_\text{forward}(\hat{A}_c) )^\alpha (\mathbb{P}\_\text{backward}(\hat{A}_c) )^{1-\alpha}$). The proposed method is **general** and **can be integrated into existing verification methods** (such as RCoT [3] and Self-Verification [4]) for non-arithmetic reasoning tasks.

Reviewer comment (Evaluation):
Have you observed cases where backward verification is especially hard, while forward verification is not?

Author response:
> Q6. "Have you observed cases where backward verification is especially hard, while forward verification is not?"
	
**A6.** As suggested, we analyzed the questions in GSM8K (using GPT-3.5-Turbo with ComplexCoT) that  Self-Consistency succeeds but FOBAR fails. There are 24 such questions. We give an example below, where backward reasoning fails to predict the masked numbers.
	
*Question*: Manolo bought 5 lollipops and 4 candies that cost \\$3.20. If each lollipop costs \\$0.40, how much will 10 lollipops and 10 candies cost him?

*Ground-truth answer*: 7

Candidate answers generated by Self-Consistency:  12 (once), 16 (4 times), 7 (5 times)

Number of times that the masked number is correctly predicted in backward reasoning: 12 (6 times), 16 (0 times), 7 (0 times)

Reviewer comment (Evaluation):
Do you have a sense of how often backward verification discards correct solutions? In other words, of the remaining failures, how many are cases where the base model fails to give any correct answer at all, versus cases where one of the answers is correct but ends up discarded by FOBAR?

Author response:
> Q7. "Do you have a sense of how often backward verification discards correct solutions? In other words, of the remaining failures, how many are cases where the base model fails to give any correct answer at all, versus cases where one of the answers is correct but ends up discarded by FOBAR?"

**A7.** As suggested, we did an analysis on the failure cases of FOBAR using the GPT-3.5-Turbo with ComplexCoT prompting to study how often FOBAR discards correct solutions. Table below shows the results. As shown, in total, there are 400 problems  FOBAR fails. Among them, nearly half of them have no correct candidate answers, which cannot be improved by enhancing backward verification. Furthermore, there are 189 questions that have forward chains leading to the correct answer but FOBAR fails.

\begin{array}{cccccccc}
\hline
& \text{AddSub} & \text{MultiArith} & \text{SingleEQ} & \text{SVAMP} & \text{GSM8K} & \text{AQuA} & \text{Total} \newline
\hline
\text{\\#samples} & 395 & 580 & 508 & 1000 & 1319 & 254 & 4056\newline
\hline
\text{\\#succeeds} & 349 & 579 & 479 & 885 & 1153 & 161 & 3606\newline
\text{\\#fails} & 46 & 1 & 29 & 115 & 116 & 93 & 400 \newline
\hline
\text{\\#has no correct answers} & 28 & 0 & 14 & 57 & 60 & 52 & 211  \newline
\text{\\#has the correct answer} & 18 & 1 & 15 & 58 & 56 &  41 & 189 \newline
\hline
\end{array}

Reviewer comment (Experiments):
Related to the previous question, one suggestion that should be simple to implement is computing an "oracle verifier", which takes any correct answer from the model if there is one, and otherwise returns any answer. This might help show the potential to improve results by improving the verifier (as FOBAR/self-consistency try to do), versus cases where we would only see improvements if the base model were to improve.

Author response:
> Q8. "Related to this last question, one suggestion that should be simple to implement is computing an "oracle verifier", which takes any correct answer from the model if there is one, and otherwise returns any answer. This might help show the potential to improve results by improving the verifier (as FOBAR/self-consistency try to do), versus cases where we would only see improvements if the base model were to improve."

**A8.** We conducted an additional analysis to show the potential of improving performance by enhancing the verifier. Table below shows the accuracy of Oracle Verifier (i.e., pick the ground-truth answer if it is a candidate answer generated by Self-Consistency). We can see that there is a large gap between Oracle Verifier and FOBAR/Self-Consistency. Hence, **improving the verifier is a potential method to boost performance.**

\begin{array}{c|c |c c c c c c c}
\hline
 & \text{method} & \text{AddSub} & \text{MultiArith} & \text{SingleEQ} & \text{SVAMP} & \text{GSM8K} & \text{AQuA} & \text{Avg} \newline
\hline
 & \text{Self-Consistency} & 91.7 &   95.9 & 94.5 & 83.1 &67.9& 55.1& 81.4 \newline
\text{text-davinci-003+CoT}&\text{FOBAR}   &91.9 & 100.0 & 96.1& 86.8 & 70.8 & 55.1 & 83.5 \newline
& \text{Oracle Verifier} & 97.0 & 100.0  & 98.0 & 92.8 & 84.6 & 85.8 & 93.0 \newline
\hline
  & \text{Self-Consistency} & 89.4 &98.5  & 91.1& 82.7 & 79.1 & 58.7 & 83.2 \newline
\text{text-davinci-003+ComplexCoT} & \text{FOBAR}   & 90.6 &100.0& 95.3 &87.0 & 78.7 & 58.7 & 85.0 \newline
& \text{Oracle Verifier} & 95.4 & 100.0 & 97.4 & 94.5 & 92.2 & 82.7 & 93.7 \newline
\hline
 & \text{Self-Consistency} &90.6 & 98.6 & 93.1 & 86.4 & 81.9 & 62.6 & 85.5\newline
\text{GPT-3.5-turbo+CoT} & \text{FOBAR} & 89.4 & 99.3 & 94.5 & 88.9 & 85.1 & 62.6 & 86.6 \newline
& \text{Oracle Verifier} & 93.9 & 100.0 & 96.5 & 93.9 & 92.2 & 89.8 & 94.4\newline
\hline
 & \text{Self-Consistency} & 88.1 &  98.8 & 94.5 & 85.0&86.4 & 63.0 &  86.0 \newline
\text{GPT-3.5-turbo+ComplexCoT} &\text{FOBAR} & 88.4 & 99.8& 94.3 & 88.5 & 87.4 & 63.4 & 87.0 \newline
& \text{Oracle Verifier} & 92.9 & 100.0 & 97.2 & 94.3 & 95.5 & 79.5 & 93.2 \newline
\hline
 & \text{Self-Consistency} & 92.2 &  99.0 & 95.9 & 93.3 & 94.8 & 71.3 & 91.1\newline
\text{GPT-4+CoT}& \text{FOBAR} & 92.4 & 99.0 & 96.1& 94.1 & 95.4 & 71.3 & 91.4\newline
& \text{Oracle Verifier} & 94.4 & 99.1 & 97.0 & 96.0 & 97.3 & 89.8 & 95.6 \newline
\hline
 & \text{Self-Consistency} &  91.4 & 98.5 & 94.7 & 93.4  & 96.2 & 75.2 & 91.6 \newline
\text{GPT-4+ComplexCoT} & \text{FOBAR} & 91.9 & 98.6 & 94.7 & 94.4 & 96.4 & 75.2 & 91.9\newline
& \text{Oracle Verifier} & 94.4 & 98.8 & 96.3 & 95.7 & 98.5 & 91.3 & 95.8 \newline
\hline
\end{array}

Reviewer comment (Evaluation):
Does the proposed backward reasoning have the potential for extension to more complex settings? For some questions, given a candidate number, there can be multiple correct values for the masked number in the question statement. The model may output a number that makes sense, even if it's different from the number in the original question statement. Then, how can we measure the accuracy of the backward reasoning?

Author response:
**A1.** Yes, our proposed backward reasoning **still works** for the mentioned complex setting.
In backward reasoning, we select the candidate answer that leads to the most backward reasoning chains that predict the masked number successfully. **When there are multiple correct values for the masked number, the proposed backward reasoning can work as long as the correct candidate answer is easier for the LLM to predict the masked number than the wrong candidate answers.** We give an example below.

*Question*: Emily and Sophia are sisters. Emily is 30 years old, and Sophia is 43 years old. Calculate the square of their age difference.

*Candidate Answers*: 169 (correct), 139 (wrong)

*Backward question for the candidate answer 169*: Emily and Sophia are sisters. Emily is 30 years old, and Sophia is ${\bf x}$ years old. Calculate the square of their age difference. If we know the answer to the above question is **169**, what is the value of unknown variable ${\bf x}$?

*Backward question for the candidate answer 139*: Emily and Sophia are sisters. Emily is 30 years old, and Sophia is ${\bf x}$ years old. Calculate the square of their age difference. If we know the answer to the above question is **139**, what is the value of unknown variable ${\bf x}$?

Obviously, for the correct candidate answer $169$, the correct value of ${\bf x}$ is not unique, can be 43 or 17. The proposed backward reasoning for verification still works as the LLM is more likely to predict ${\bf x} = 43$ given the candidate answer $169$ than given $139$. To validate it, we conduct an experiment as follows. For each candidate answer, we sample 100 reasoning chains to predict ${\bf x}$. For the candidate answer 169, there are 24 backward reasoning chains that predict ${\bf x}=43$ (35 chains lead to ${\bf x}=17$, and the remaining 41 chains lead to either 43 or 17 is ok). However, for the candidate answer 139, none of the backward reasoning chains predict ${\bf x}=43$, thus, the candidate answer 169 wins. Hence, in the cases where the correct value of ${\bf x}$ is not unique, the proposed backward reasoning method still works for verification.

Reviewer comment (Evaluation):
Can backward reasoning be more accurate than forward reasoning? If backward reasoning isn't simpler than forward reasoning, poor performance in backward reasoning could negatively impact the accuracy of the final answer.

Author response:
**A2.** (i) In Section 4.3 of the submission, we conducted an ablation experiment to compare the performance of forward and backward reasoning. As shown in Table 2 there, backward reasoning is more accurate than forward reasoning in 5 of 6 settings.

(ii) It is possible that backward reasoning tasks can be complex. However, **even when the masked number is difficult to predict, as long as the correct candidate answer can better predict the masked number than the incorrect candidate answers, this correct candidate will still be picked.** In other words, the correct candidate answer may still lead to a low prediction accuracy on the masked number, but as long as its accuracy is still higher than those of the incorrect candidate answers, the proposed backward reasoning procedure still works. 

(iii) By combining forward and backward reasoning, the proposed **FOBAR is a general method to improve forward verification**. As shown in Table 2 of the submission, FOBAR consistently performs better than Self-Consistency, which uses forward reasoning alone.

Reviewer comment (Novelty):
Recent work proposes using step-by-step verification for answer verification. The paper also suggests backward reasoning, which involves step-by-step analysis. Is there any relationship between backward reasoning and step-by-step verification?

Author response:
A3. (i) Note that the mentioned papers [1, 2] focus on verifying the steps of forward reasoning chains and **DO NOT** use backward reasoning in verification. In [1], the authors label exclusively steps of forward reasoning chains generated by LLMs. The labeled data are then used to train an LLM for verification. Compared with [1], which is computationally expensive in training an LLM and labor-intensive in labeling data, our backward reasoning is training-free for verification and requires no additional data annotation. 

(ii) In [2], the authors propose a natural language-based deductive reasoning format that allows the LLM to verify **forward** reasoning steps. In our paper, we use **backward** reasoning to **verify the candidate answers** instead of the steps in forward chains. As backward and forward reasoning are complementary, our backward verification can be combined with their step-by-step forward verifications. We replace the forward verification in our FOBAR (Eq. (4) in our paper) with step-by-step forward verification proposed by [2], and conducted additional experiments on AddSub, GSM8K, and AQuA using GPT-3.5-Turbo. Table below shows the testing accuracy. As can be seen, **combining backward verification with their forward verifications consistently boosts performance**.

\begin{array}{lccc}
\hline 
&  \text{AddSub} & \text{GSM8K} & \text{AQuA} \newline
\hline
\text{NP [2]} & 93.67 &  87.05 & 70.34 \newline
\text{NP + Backward Verification} & \textbf{93.92} & \textbf{87.89} & \textbf{71.65} \newline
\hline
\text{NP + DV + UPV [2]} & 93.54 & 86.01 & 69.49  \newline
\text{NP + DV + UPV + Backward Verification} & \textbf{93.92} &  \textbf{87.19} & \textbf{70.86} \newline
\hline
\end{array}

---

## **References**

[1] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. Preprint arXiv:2305.20050, 2023.

[2] Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. Deductive veriﬁcation of Chain-of-Thought reasoning. In *Neural Information Processing Systems*, 2023.