Large Language Models have Intrinsic Self-Correction Ability

## 1 Proof on why SC is effective

Let’s assume that for a benchmark $Q$ where each question $q\in Q$ has $k>=2$ possible answers, some LLM has the true ability of $correct(A\in Q)=\lambda>\frac{1}{k}$. Theoretically speaking, such true ability could be achieved when the LLM is performing a beam search with infinite length and a perfect prompt. However, since such decoding and prompting is impossible, the LLM has a hallucination possibility $h>0$. Additionally, we assume that hallucination randomly changes the answer (originally either correct or incorrect) with equal possibility. No matter what the previous answer was, after the hallucination, the answer is random but not the original answer. Then, we could devise the following Lemma. LABEL:lemma1.
Lemma 1.
LLMs are generally under-performing compared to their real abilities because hallucination will cause the overall accuracy to decrease.
{proof}[Proof of Lemma LABEL:lemma1]
During the response generation, $h$ of responses are hallucinated. This means that $h*\lambda$ answers will be changed from correct to incorrect. On the other side, $\frac{h*(1-\lambda)}{k-1}$ answers will be changed from incorrect to correct.
Adding these two parts together, we get that the final accuracy after hallucination shall be $correct(A^{\prime}\in Q)=\lambda-h*\lambda+\frac{h*(1-\lambda)}{k-1}$. It is not hard to see that $h*\lambda>\frac{h*(1-\lambda)}{k-1}$, as this is the same equation of $\lambda>\frac{(1-\lambda)}{k-1}=\lambda*(k-1)>1-\lambda=\lambda*k>1$, which is the assumption we make above.
Since $h*\lambda>\frac{h*(1-\lambda)}{k-1}$, we could conclude that $correct(A^{\prime}\in Q)<correct(A\in Q)$. In other words, hallucination causes LLMs to perform worse.
Thus, we could model an LLM’s performance on a single instance as a biased Gaussian distribution of $L\sim N(\lambda+\beta,\sigma)$, where bias $\beta$, which is related to hallucination possibility $h$, is strictly less than 0.
Existing works such as self-verification and CoT are different techniques that reduce hallucination. Similarly, as argued in Section 2, SC, which is another form of self-verification and CoT, also has a similar outcome.

## 2 Experiments not included in main paper

### \thesubsection Ablation study on biased prompt

We provide the second ablation study of Section 4 on the effects of temperature changes on the biased prompt. As shown in Table 2 and 2, while the result seems more unstable due to the biased nature of the prompt, we see that there is still an overall trend of losing SC ability as temperature increases for GPT-3.5, whereas GPT-4 stays robust. This shows that temperature’s effect is only contingent on the response’s decomposition, but not on the prompt.

**Table \thetable: Temperature’s effect on GPT3.5 using biased prompt from [huang2024large].**
| CommonSense QA (biased prompt on GPT 3.5) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 75.68 | 68.06 | -7.62 |
| 0.3 | 75.35 | 68.63 | -6.72 |
| 0.5 | 75.43 | 68.55 | -6.88 |
| 0.8 | 75.51 | 66.99 | -8.52 |
| 1.0 | 75.92 | 67.08 | -8.84 |
| 1.2 | 74.2 | 66.18 | -8.02 |
| 1.5 | 72.81 | 61.51 | -11.3 |

**Table \thetable: Temperature’s effect on GPT4 using biased prompt from [huang2024large].**
| CommonSense QA (biased prompt on GPT 4) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 81 | 81 | 0 |
| 0.3 | 85.5 | 77.5 | -8 |
| 0.5 | 86 | 79.5 | -6.5 |
| 0.8 | 84 | 77 | -7 |
| 1.0 | 85 | 79 | -6 |
| 1.2 | 83 | 79 | -4 |
| 1.5 | 61 | 45 | -16 |

### \thesubsection Numerical results for Figure LABEL:fig:temp_all and Figure LABEL:fig:temp_only_sc

In this section, we provide detailed numerical results for the analysis in Section 4 for the reader’s reference.

#### \thesubsubsection Numerical results for Figure LABEL:fig:temp_all

Table 2-2 include numerical results evaluated by four models on CommonSense QA dataset [talmor-etal-2019-commonsenseqa]. The trend of accuracy decrement is very clear on GPT-3.5, but not so much on the other three models because of the response’s decomposition. However, we could still see an overall downward trend for all of the models.

**Table \thetable: Temperature’s effect on GPT3.5 using unbiased prompt. Temperature is applied to the entire generation process.**
| CommonSense QA (unbiased prompt on GPT 3.5) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 75.92 | 76 | 0.08 |
| 0.3 | 75.02 | 75.02 | 0 |
| 0.5 | 75.51 | 74.86 | -0.65 |
| 0.8 | 74.86 | 73.96 | -0.9 |
| 1.0 | 74.45 | 72.56 | -1.89 |
| 1.2 | 74.69 | 72.4 | -2.29 |
| 1.5 | 71.5 | 69.7 | -1.8 |

**Table \thetable: Temperature’s effect on GPT-4 using unbiased prompt. Temperature is applied to the entire generation process.**
| CommonSense QA (unbiased prompt on GPT 4) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 81 | 84.5 | 3.5 |
| 0.3 | 86.5 | 86.5 | 0 |
| 0.5 | 86.5 | 87 | 0.5 |
| 0.8 | 84.5 | 84.5 | 0 |
| 1.0 | 84.5 | 84 | -0.5 |
| 1.2 | 85.5 | 85.5 | 0 |
| 1.5 | 69.5 | 69 | -0.5 |

**Table \thetable: Temperature’s effect on Mistral using unbiased prompt. Temperature is applied to the entire generation process.**
| CommonSense QA (unbiased prompt on Mistral-7B-Instruct-V0.2) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 66.01 | 66.67 | 0.66 |
| 0.3 | 68.47 | 68.63 | 0.16 |
| 0.5 | 68.8 | 68.8 | 0 |
| 0.8 | 68.96 | 68.8 | -0.16 |
| 1 | 69.04 | 69.37 | 0.33 |
| 1.2 | 68.8 | 68.88 | 0.08 |
| 1.5 | 68.06 | 68.22 | 0.16 |

**Table \thetable: Temperature’s effect on Phi3 using unbiased prompt. Temperature is applied to the entire generation process.**
| CommonSense QA (unbiased prompt on Phi3-Mini-4k-Instruct) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 76.74 | 76.99 | 0.25 |
| 0.3 | 76.33 | 76.41 | 0.08 |
| 0.5 | 76.99 | 76.99 | 0 |
| 0.8 | 76.74 | 77.07 | 0.33 |
| 1 | 76.49 | 76.09 | -0.4 |
| 1.2 | 76.66 | 76.58 | -0.08 |
| 1.5 | 76.17 | 75.51 | -0.66 |

#### \thesubsubsection Numerical results for Figure LABEL:fig:temp_only_sc

Table 2-2 include numerical results evaluated by the two GPT models on CommonSense QA dataset [talmor-etal-2019-commonsenseqa] during the first ablation study. The first stage is fixed with the response generated with zero temperature. Similar to the previous study, the trend is obvious for GPT-3.5, but not for GPT-4 because of the response’s decomposition.

**Table \thetable: Temperature’s effect on GPT3.5 using unbiased prompt. Temperature is applied to only the intrinsic SC process.**
| CommonSense QA (unbiased prompt on GPT 3.5) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 75.92 | 76 | 0.08 |
| 0.3 | 75.92 | 75.59 | -0.33 |
| 0.5 | 75.92 | 75.43 | -0.49 |
| 0.8 | 75.92 | 74.69 | -1.23 |
| 1.0 | 75.92 | 73.55 | -2.37 |
| 1.2 | 75.92 | 72.89 | -3.03 |
| 1.5 | 75.92 | 72.4 | -3.52 |

**Table \thetable: Temperature’s effect on GPT-4 using unbiased prompt. Temperature is applied to only the intrinsic SC process.**
| CommonSense QA (unbiased prompt on GPT 4) |  |  |  |
| --- | --- | --- | --- |
| Temperature | Before SC | After SC | $\Delta SC$ |
| 0 | 81 | 84.5 | 3.5 |
| 0.3 | 81 | 84 | 3 |
| 0.5 | 81 | 84 | 3 |
| 0.8 | 81 | 84 | 3 |
| 1.0 | 81 | 84.5 | 3.5 |
| 1.2 | 81 | 85.5 | 4.5 |
| 1.5 | 81 | 81.5 | 0.5 |

### \thesubsection Numerical results on the number of changed answers

In this section, we provide detailed numerical results for the analysis on the number of changed answers for different settings. Results match the theoretical analysis that non-zero temperature and the biased prompt will cause LLMs to hallucinate and make unnecessary changes to their answers, which is detrimental to their intrinsic SC ability. As we can see, the biased prompt set (Prompt Set 1) has many more invalid changes (C2I) compared to the unbiased prompt set (Prompt Set 3), and the amount is significantly more compared to the valid changes (I2C).

#### \thesubsubsection Notation Reference

As mentioned above, there are five outcomes for the SC process. We provide a reference in Table 2.

**Table \thetable: Reference sheet for the labels that will be used below.**
| Label | Meaning (before SC to after SC) |
| --- | --- |
| C2C | correct to correct |
| C2I | correct to incorrect |
| I2C | incorrect to correct |
| I2I-C | incorrect to incorrect, but answer changes |
| I2I-NC | incorrect to incorrect, answer does not change |

#### \thesubsubsection Change of distribution of labels in response to temperature changes

In Table 2 to 2, we provide the number of each class for four models during the SC process as temperature changes. It is clear that as temperature increases, the number of changed responses increases, and C2I increases much faster than I2C. This aligns with the theoretical analysis.

**Table \thetable: Number of changes SC make for GPT-3.5 with unbiased prompt, with each temperature.**
| CommonSense QA (unbiased prompt on GPT 3.5) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Temperature | C2C | C2I | I2C | I2I-C | I2I-NC |
| 0 | 916 | 11 | 12 | 17 | 265 |
| 0.3 | 901 | 15 | 15 | 20 | 270 |
| 0.5 | 896 | 26 | 18 | 23 | 258 |
| 0.8 | 880 | 34 | 23 | 24 | 260 |
| 1.0 | 859 | 50 | 27 | 26 | 259 |
| 1.2 | 861 | 51 | 23 | 37 | 249 |
| 1.5 | 807 | 66 | 44 | 50 | 254 |

**Table \thetable: Number of changes SC make for GPT-4 with unbiased prompt, with each temperature.**
| CommonSense QA (unbiased prompt on GPT-4) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Temperature | C2C | C2I | I2C | I2I-C | I2I-NC |
| 0 | 162 | 0 | 7 | 2 | 29 |
| 0.3 | 173 | 0 | 0 | 1 | 26 |
| 0.5 | 171 | 2 | 2 | 1 | 24 |
| 0.8 | 167 | 2 | 2 | 1 | 28 |
| 1.0 | 167 | 2 | 1 | 0 | 30 |
| 1.2 | 169 | 2 | 2 | 1 | 26 |
| 1.5 | 119 | 20 | 19 | 12 | 30 |

**Table \thetable: Number of changes SC make for Mistral with unbiased prompt, with each temperature.**
| CommonSense QA (unbiased prompt on Mistral) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Temperature | C2C | C2I | I2C | I2I-C | I2I-NC |
| 0 | 803 | 3 | 11 | 12 | 392 |
| 0.3 | 833 | 4 | 5 | 13 | 366 |
| 0.5 | 837 | 3 | 3 | 9 | 369 |
| 0.8 | 836 | 6 | 4 | 20 | 355 |
| 1 | 842 | 1 | 5 | 18 | 355 |
| 1.2 | 837 | 3 | 4 | 9 | 368 |
| 1.5 | 828 | 3 | 5 | 15 | 370 |

**Table \thetable: Number of changes SC make for Phi3 with unbiased prompt, with each temperature.**
| CommonSense QA (unbiased prompt on Phi3) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Temperature | C2C | C2I | I2C | I2I-C | I2I-NC |
| 0 | 936 | 1 | 4 | 2 | 278 |
| 0.3 | 930 | 2 | 3 | 1 | 285 |
| 0.5 | 936 | 4 | 4 | 3 | 274 |
| 0.8 | 934 | 3 | 7 | 3 | 274 |
| 1 | 928 | 6 | 1 | 2 | 284 |
| 1.2 | 931 | 5 | 4 | 3 | 278 |
| 1.5 | 917 | 13 | 5 | 3 | 283 |

#### \thesubsubsection Change of distribution of labels in response to prompt changes

In Table 2 and 2, we provide the number of each class for four models during the SC process as the prompt changes. The results validate the theoretical analysis. Biased prompt will cause more changes during the SC process. However, since the possibility of changing from a correct answer to an incorrect answer is much higher than in the other direction, the biased prompt will in fact harm the SC ability of LLM. We also get a better understanding of the models that do not suffer accuracy degradation on certain benchmarks. As we can see from I2C and C2I values in those cases, they remain very small with the biased prompt, indicating that $\gamma\simeq 0$. Such robustness might be coming from fine-tuning, or simply because the bias prompt is not “bias” enough.

**Table \thetable: The number of occurrences five classes of behaviors during the intrinsic SC process on the CommonSense QA dataset [talmor-etal-2019-commonsenseqa]. As we switch from biased to unbiased prompt, it is clear that the number of changed answers decreased significantly.**
| \multirow2* | GPT-3.5 |  |  |
| --- | --- | --- | --- |
|  | Prompt Set 1 | Prompt Set 2 | Prompt Set 3 |
| C2C | 778 | 870 | 916 |
| C2I | 149 | 57 | 11 |
| I2C | 32 | 16 | 12 |
| I2I-C | 43 | 39 | 17 |
| I2I-NC | 219 | 239 | 265 |
| \multirow2* | GPT-4 |  |  |
|  | Prompt Set 1 | Prompt Set 2 | Prompt Set 3 |
| C2C | 146 | 161 | 162 |
| C2I | 16 | 1 | 0 |
| I2C | 16 | 8 | 7 |
| I2I-C | 2 | 2 | 2 |
| I2I-NC | 20 | 28 | 29 |
| \multirow2* | Mistral |  |  |
|  | Prompt Set 1 | Prompt Set 2 | Prompt Set 3 |
| C2C | 775 | 805 | 803 |
| C2I | 31 | 1 | 3 |
| I2C | 41 | 9 | 11 |
| I2I-C | 44 | 16 | 12 |
| I2I-NC | 330 | 390 | 392 |
| \multirow2* | Phi3 |  |  |
|  | Prompt Set 1 | Prompt Set 2 | Prompt Set 3 |
| C2C | 910 | 932 | 936 |
| C2I | 27 | 5 | 1 |
| I2C | 17 | 7 | 4 |
| I2I-C | 8 | 5 | 2 |
| I2I-NC | 259 | 272 | 278 |

**Table \thetable: The number of occurrences five classes of behaviors during the intrinsic SC process on the GSM8K [GSM8K] dataset. As we switch from biased to unbiased prompt, it is clear that the number of changed answers decreased significantly.**
| \multirow2* | GPT-3.5 |  |  |
| --- | --- | --- | --- |
|  | Prompt Set 1 | Prompt Set 2 | Prompt Set 3 |
| C2C | 1015 | 1024 | 1024 |
| C2I | 9 | 0 | 0 |
| I2C | 17 | 7 | 4 |
| I2I-C | 27 | 8 | 8 |
| I2I-NC | 251 | 280 | 283 |
| \multirow2* | GPT-4 |  |  |
|  | Prompt Set 1 | Prompt Set 2 | Prompt Set 3 |
| C2C | 175 | 179 | 187 |
| C2I | 14 | 10 | 2 |
| I2C | 4 | 5 | 3 |
| I2I-C | 4 | 5 | 2 |
| I2I-NC | 3 | 1 | 6 |
| \multirow2* | Mistral |  |  |
|  | Prompt Set 1 | Prompt Set 2 | Prompt Set 3 |
| C2C | 570 | 597 | 606 |
| C2I | 47 | 20 | 11 |
| I2C | 51 | 29 | 26 |
| I2I-C | 224 | 112 | 107 |
| I2I-NC | 427 | 561 | 569 |

## 3 Prompt and example outputs

We use the prompt formatting strategy as previous works in this area [kim2023language, shinn2023reflexion, huang2024large]. We use the exact same prompt for stage 1 as [huang2024large].

### \thesubsection Prompt Set 1 [ huang2024large ] (biased) on CommonSense QA

Q: Where do you find the most amount of leafs?
Answer Choices:(A) floral arrangement (B) ground (C) forrest (D) field (E) compost pile
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
The most amount of leaves would typically be found in a (C) forest. This is because forests are densely populated with trees, which have many leaves that fall to the ground, especially during autumn when deciduous trees shed their leaves. While leaves can be found in all the other options provided, the sheer number of trees and the volume of leaves they produce in a forest would likely surpass the number of leaves found in a floral arrangement, on the ground in non-forested areas, in a field, or even in a compost pile.
Final answer: (C)
Review your previous answer and find problems with your answer.
Upon reviewing the previous answer, one potential issue that arises is the assumption that the forest will always have the most leaves without considering the other options in certain contexts. For example, during certain times of the year or in a particularly large compost pile that has been accumulating leaves for a long time, the number of leaves in the compost pile could potentially rival or exceed the number of leaves on the forest floor. Additionally, the term ””ground”” is quite vague and could technically include the forest floor as well, which might lead to some confusion.
However, despite these considerations, the reasoning provided in the original answer still holds strong for the most part. Forests are vast areas with a high density of trees, and therefore, they are likely to have a larger accumulation of leaves than the other options listed, especially when considering a typical scenario over a large area and over time. Fields generally have fewer trees and more open space, floral arrangements are small and contain a limited number of leaves, and while compost piles can have a significant number of leaves, they are usually not as vast as a forest.
Final answer: (C)
Based on the problems you found, improve your answer. You must choose only one option from A to E. Please reiterate your answer, with your final answer a single letter from A to E, in the form (answer).
Upon further consideration, it is important to clarify that the term ””ground”” in option (B) is ambiguous and could encompass a variety of landscapes, including the forest floor. However, if we interpret ””ground”” to mean general outdoor areas that are not specifically forested, then the forest (C) still remains the most likely place to find the greatest accumulation of leaves. This is because forests are ecosystems with a high density of trees, and the natural lifecycle of these trees involves shedding leaves, particularly in deciduous forests during the fall season. While a compost pile (E) can indeed contain a significant number of leaves, it is typically confined to a smaller area compared to the expansive spread of a forest. Fields (D) are often open spaces with fewer trees and therefore fewer leaves. Floral arrangements (A) are designed and contain only a small number of leaves for aesthetic purposes.
Taking into account the potential for seasonal variations and the size of compost piles, the forest still represents the most consistent and extensive source of leaves due to the sheer number of trees and the continuous process of leaf shedding and accumulation over time.
Final answer: (C)

### \thesubsection Prompt Set 1 [ huang2024large ] (biased) on GSM8K

Question: Can you solve the following math problem? Jill gets paid $20 per hour to teach and $30 to be a cheerleading coach. If she works 50 weeks a year, 35 hours a week as a teacher and 15 hours a week as a coach, what’s her annual salary? Explain your reasoning. Your final answer should be a single numerical number, in the form \boxed{answer}, at the end of your response.
Answer: To solve this problem, we need to calculate Jill’s earnings from teaching and cheerleading coaching separately, and then add them together to find her total annual salary.
First, let’s calculate her earnings from teaching:
Jill gets paid $20 per hour as a teacher. She works 35 hours a week as a teacher.
Her weekly earnings from teaching are: $20/hour * 35 hours/week = $700/week.
Since she works 50 weeks a year, her annual earnings from teaching are:
$700/week * 50 weeks/year = $35,000/year.
Next, let’s calculate her earnings from cheerleading coaching:
Jill gets paid $30 per hour as a cheerleading coach. She works 15 hours a week as a coach.
Her weekly earnings from coaching are: $30/hour * 15 hours/week = $450/week.
Since she also works 50 weeks a year as a coach, her annual earnings from coaching are:
$450/week * 50 weeks/year = $22,500/year.
Now, we add her earnings from teaching and coaching to find her total annual salary:
Total annual salary = Earnings from teaching + Earnings from coaching
Total annual salary = $35,000/year + $22,500/year
Total annual salary = $57,500/year.
Therefore, Jill’s annual salary is \boxed{57500}.
Review your previous answer and find problems with your answer.
Upon reviewing the previous answer, there doesn’t appear to be any problems with the calculations or the reasoning. The steps taken to calculate Jill’s annual salary from both teaching and cheerleading coaching are correct, and the final answer is accurately computed by adding her earnings from both jobs.
To summarize the correct calculations:
- Annual earnings from teaching: $20/hour * 35 hours/week * 50 weeks/year = $35,000/year.
- Annual earnings from coaching: $30/hour * 15 hours/week * 50 weeks/year = $22,500/year.
- Total annual salary: $35,000/year (teaching) + $22,500/year (coaching) = $57,500/year.
The final answer remains \boxed{57500}.
Based on the problems you found, improve your answer. Please reiterate your answer, with your final answer a single numerical number, in the form \boxed{answer}.
Upon reviewing the previous answer, there is no problem with the calculations or the reasoning. The steps taken to calculate Jill’s annual salary from both teaching and cheerleading coaching are correct, and the final answer is accurately computed by adding her earnings from both jobs.
To summarize the correct calculations:
- Annual earnings from teaching: $20/hour * 35 hours/week * 50 weeks/year = $35,000/year.
- Annual earnings from coaching: $30/hour * 15 hours/week * 50 weeks/year = $22,500/year.
- Total annual salary: $35,000/year (teaching) + $22,500/year (coaching) = $57,500/year.
The final answer remains \boxed{57500}.

### \thesubsection Prompt Set 2 (less biased) on CommonSense QA

Q: If one needed the bathroom they needed a key, to get it they had to also buy something from the what?
Answer Choices:(A) school (B) convenience store (C) rest area (D) mall (E) theater
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
Based on the given information, the person needing the bathroom needs a key. This suggests that the bathroom is likely locked or restricted access. To obtain the key, they have to buy something from a specific place.
Out of the given options, the most likely place where one would need to buy something to obtain a key for the bathroom is a convenience store. Convenience stores often have restrooms that are locked or require a key for access, and it is common for them to require a purchase in order to obtain the key.
Therefore, the answer is (B) convenience store.
Do you think the question and answer pair above is correct? The answer has a logical chain. If you think the answer is incorrect, please identify where the logic is wrong in the logical chain.
The logical chain in the answer is correct. The reasoning provided is sound and aligns with common practices in many convenience stores. Therefore, the answer (B) convenience store is correct.
Based on the responses above, answer the original question. Remember that your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
The answer to the original question is (B) convenience store.

### \thesubsection Prompt Set 2 (less biased) on GSM8K

Question: Can you solve the following math problem? Eliza’s rate per hour for the first 40 hours she works each week is $10. She also receives an overtime pay of 1.2 times her regular hourly rate. If Eliza worked for 45 hours this week, how much are her earnings for this week? Explain your reasoning. Your final answer should be a single numerical number, in the form \boxed{answer}, at the end of your response.
Answer: To solve this problem, we need to calculate Eliza’s earnings for the first 40 hours and then calculate her overtime earnings for the additional hours worked.
Eliza’s regular hourly rate is $10. For the first 40 hours, her earnings are:
40 hours * $10/hour = $400
For overtime, Eliza earns 1.2 times her regular hourly rate. So, her overtime rate is:
$10/hour * 1.2 = $12/hour
Eliza worked 45 hours in total, which means she worked 5 hours of overtime:
45 hours - 40 hours = 5 hours of overtime
Now, we calculate her earnings for the overtime hours:
5 hours * $12/hour = $60
Finally, we add her regular earnings to her overtime earnings to find her total earnings
for the week:
$400 (regular earnings) + $60 (overtime earnings) = $460
So, Eliza’s total earnings for the week are \boxed{460}.
Do you think the question and answer pair above is correct? The answer has a logical chain. If you think the answer is incorrect, please identify where the logic is wrong in the logical chain.
The question and answer pair above is correct. The logic used to calculate Eliza’s earnings for the week is sound and follows these steps:
1. Calculate regular earnings for the first 40 hours at $10 per hour.
2. Determine the overtime rate as 1.2 times the regular rate.
3. Calculate the overtime earnings for the additional hours worked beyond 40 hours.
4. Add the regular earnings and overtime earnings to get the total earnings for the week.
The calculations provided are accurate:
- Regular earnings: 40 hours * $10/hour = $400
- Overtime rate: $10/hour * 1.2 = $12/hour
- Overtime earnings: 5 hours * $12/hour = $60
- Total earnings: $400 + $60 = $460
The final answer is correctly presented in a box as \boxed{460}.
Based on the responses above, answer the original question. Please reiterate your answer, with your final answer a single numerical number, in the form \boxed{answer}.”
Eliza’s total earnings for the week are \boxed{460}.

### \thesubsection Prompt Set 3 (unbiased) on CommonSense QA

Q: What do people typically do while playing guitar?
Answer Choices: (A) cry (B) hear sounds (C) singing (D) arthritis (E) making music
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
People typically do (E) making music while playing guitar. Playing the guitar involves strumming or picking the strings to create melodies, chords, and rhythms, resulting in the production of music.
Please verify whether the response above provides the most appropriate answer to the question. Provide your reasoning and justification to your verification.
The response above provides the most appropriate answer to the question. Playing the guitar is primarily about creating music, whether it is through strumming, picking, or fingerpicking the strings. While some people may cry or experience arthritis while playing the guitar, these are not typical activities associated with playing the instrument. Hearing sounds is a passive action that can occur while playing the guitar, but it does not capture the active process of creating music. Singing can be done in conjunction with playing the guitar, but it is not the primary activity being asked about in the question. Therefore, the most appropriate answer is (E) making music.
Based on the responses above, answer the original question. Remember that your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
People typically (E) make music while playing guitar.

### \thesubsection Prompt Set 3 (unbiased) on GSM8K

Question: Can you solve the following math problem? Terry eats 2 yogurts a day. They are currently on sale at 4 yogurts for $5.00. How much does he spend on yogurt over 30 days? Explain your reasoning. Your final answer should be a single numerical number, in the form \boxed{answer}, at the end of your response.
Answer: Since Terry eats 2 yogurts a day, he would need 2 yogurts * 30 days = ¡¡2*30=60¿¿60 yogurts over 30 days.
Since the yogurts are on sale at 4 for $5.00, Terry would need to buy 60 yogurts / 4 yogurts = ¡¡60/4=15¿¿15 sets of 4 yogurts.
Each set of 4 yogurts costs $5.00, so Terry would spend 15 sets * $5.00=$¡¡15*5=75.00¿¿75.00 on yogurt over 30 days.
Therefore, Terry would spend $75.00 on yogurt over 30 days. Answer: \boxed{75}.
Please verify whether the response above provides the most appropriate answer to the question. Provide your reasoning and justification to your verification.
Yes, the response above provides the most appropriate answer to the question. The reasoning is clear and accurate. It correctly calculates the number of yogurts Terry needs over 30 days and then determines the number of sets of 4 yogurts he would need to buy. Finally, it calculates the total cost by multiplying the number of sets by the cost per set. The final answer of $75.00 is the correct amount Terry would spend on yogurt over 30 days.
Based on the responses above, answer the original question. Please reiterate your answer, with your final answer a single numerical number, in the form \boxed{answer}.
\boxed{75}

### \thesubsection Section 2.2 Case 2

Q: The hostess greeted the employees to the program, she then led them to their what?
Answer Choices:(A) group people (B) welcome guests (C) occupations (D) work room (E) seat customer
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
The hostess greeted the employees to the program, indicating that she is welcoming them and guiding them to a specific location. The most logical option would be (D) work room, as employees typically have designated areas or rooms where they carry out their tasks. Therefore, the answer is (D).
Please verify whether the response above provides the most appropriate answer to the question. First, provide your reasoning and justification to your verification. Then, clearly state the aswer only after your reasoning and justification.
The response above provides a reasonable justification for the answer choice (D) work room. The hostess is greeting the employees to the program, indicating that she is welcoming them and guiding them to a specific location. The most logical option would be a designated work room where the employees can carry out their tasks. Therefore, the answer is (D) work room.
Based on the responses above, answer the original question.Remember that your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
The response above provides a reasonable justification for the answer choice (D) work room. Therefore, the final answer is (D).

### \thesubsection Section 2.2 Case 3

Q: A revolving door is convenient for two direction travel, but it also serves as a security measure at a what?
Answer Choices:(A) bank (B) library (C) department store (D) mall (E) new york
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
A: bank
Please verify whether the response above provides the most appropriate answer to the question. Provide your reasoning and justification to your verification.
Yes, the response (A) bank is the most appropriate answer to the question. A revolving door serves as a security measure at a bank because it helps control the flow of people entering and exiting the building. The design of a revolving door allows only one person to enter or exit at a time, making it more difficult for unauthorized individuals to enter the bank quickly or without notice. Therefore, a revolving door is commonly used in banks as a security measure.
Based on the responses above, answer the original question.Remember that your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
The most appropriate answer to the question is (A) bank.

### \thesubsection Section 2.2 Case 4

Q: A revolving door is convenient for two direction travel, but it also serves as a security measure at a what?
Answer Choices:(A) bank (B) library (C) department store (D) mall (E) new york
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
A: bank
Please analyze the question and answer above. Provide your reasoning. Then, verify whether the response above provides the most appropriate answer to the question. Clearly state your judgement as yes or no.
The question asks about the purpose of a revolving door as a security measure. Among the answer choices, a revolving door is commonly used as a security measure at a bank. Banks often have revolving doors to control the flow of people entering and exiting the building, providing an added layer of security by allowing only one person to enter or exit at a time.
The response above, which selects option A: bank, provides the most appropriate answer to the question. Therefore, the judgment is yes.
Based on the responses above, answer the original question.Remember that your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
A: bank

## 4 Response differences by each of the model

In this section, we provide example responses of stage 2 for each of the model. Among the five tested model, GPT-3.5 puts its final judgement in the beginning, whereas other models put them towards the end. GPT-4 starts directly with reasoning in the beginning, and Mistral (sometimes) and Phi3 performs selective copying in the beginning.

### \thesubsection GPT-3.5

Q: A revolving door is convenient for two direction travel, but it also serves as a security measure at a what?
Answer Choices:(A) bank (B) library (C) department store (D) mall (E) new york
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
A: bank
A revolving door is often used as a security measure at banks. The design of a revolving door allows for controlled entry and exit, making it difficult for unauthorized individuals to enter the building. Additionally, the rotating motion of the door helps to prevent tailgating, where multiple people try to enter or exit through a single door at the same time. Therefore, the correct answer is (A) bank.
Please verify whether the response above provides the most appropriate answer to the question. Provide your reasoning and justification to your verification.
Yes, the response above provides the most appropriate answer to the question. A revolving door is commonly used as a security measure at banks due to its controlled entry and exit design, which helps prevent unauthorized individuals from entering the building. The rotating motion of the door also helps to prevent tailgating. Therefore, the correct answer is (A) bank.

### \thesubsection GPT-4

Q: A revolving door is convenient for two direction travel, but it also serves as a security measure at a what?
Answer Choices:(A) bank (B) library (C) department store (D) mall (E) new york
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
Revolving doors are commonly used in various types of buildings for their convenience and energy efficiency, as they help to prevent drafts, thus maintaining the interior climate. However, when considering the aspect of security, banks are the type of establishment among the options provided that are most likely to use revolving doors as a security measure. Banks often deal with higher security needs due to the nature of their business, which involves handling large amounts of money and sensitive financial transactions. While libraries, department stores, and malls may also use revolving doors for convenience and energy efficiency, they typically do not have the same level of security concerns as banks. The option ””New York”” (E) is not specific to a type of establishment but rather a location, so it does not directly answer the question about where a revolving door serves as a security measure.
Therefore, the most appropriate answer from the given options is:
(A) bank
Please verify whether the response above provides the most appropriate answer to the question. Provide your reasoning and justification to your verification.
The response above provides a logical and well-reasoned answer to the question. Revolving doors are indeed used in various types of buildings for their convenience and energy efficiency. However, when considering the aspect of security, banks stand out among the options provided.
Banks handle large amounts of money and are at a higher risk for theft and other security breaches. Therefore, they often implement additional security measures to protect their assets and customers. A revolving door can act as a security measure in a bank by controlling the flow of people entering and exiting the building, making it more difficult for potential robbers to quickly escape and for unauthorized individuals to enter without detection.
Libraries (B), department stores (C), and malls (D) may use revolving doors primarily for energy efficiency and convenience rather than as a primary security measure. While they do have security concerns, these are generally not as heightened as those of a bank.
Option (E) New York is not relevant to the question as it is a location and does not specify a type of establishment where a revolving door would serve as a security measure.
Given the context of the question and the options provided, the most appropriate answer is indeed:
(A) bank
This answer is justified by the higher security requirements of banks compared to the other options listed.

### \thesubsection Mistral

Q: Where do apples form on an apple tree?
Answer Choices:(A) south africa (B) sunshine (C) new york (D) bloom (E) trunk
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
(D) bloom
Apples do not form in any of the given options A, B, C, or E. Instead, they form from the fruit that grows on the branches of an apple tree after the tree has bloomed.
Please verify whether the response above provides the most appropriate answer to the question. Provide your reasoning and justification to your verification.
(D) bloom
The question asks where apples form on an apple tree, but the answer choices do not include the correct answer. Therefore, none of the given options (A, B, C, or E) are the answer. Instead, apples form after the tree has bloomed, so the closest answer from the given choices would be (D) bloom, but it is not a correct answer as apples do not form during the bloom stage.
Therefore, my verification is that none of the given answer choices (A, B, C, or E) are the correct answer to the question. The correct answer is that apples form on the branches of an apple tree, after the tree has bloomed.

### \thesubsection Phi3

Q: what does someone have that causes them committing murder?
Answer Choices:(A) problems (B) distress (C) fear (D) go to jail (E) killer
Explain your reasoning. You must choose only one option from A to E. Your final answer should be a single letter from A to E, in the form (answer), at the end of your response.
(B) distress
Although distress is not a direct cause of murder, it can contribute to a person’s emotional state and potentially lead to actions stemming from extreme distress. However, it’s important to note that not everyone experiencing distress commits a crime, and it’s a complex issue influenced by multiple factors. Nonetheless, among the given options, ”distress” is the most plausible factor that might contribute to someone’s actions leading to murder, though it is not a definitive cause.
(answer) B
Please verify whether the response above provides the most appropriate answer to the question. Provide your reasoning and justification to your verification.
(answer) B
The question asks for a factor that ”causes” someone to commit murder, which can be misleading as it implies a direct and singular cause. However, among the given options, ”distress” is the most plausible factor that could contribute to someone’s actions leading to murder. It is a complex issue influenced by multiple factors and not a direct cause, but it is the most relevant choice among the options provided. Therefore, the answer B is justified based on the given options.

## Peer Reviews

confidence: 3
contribution: 4
initial_rating: 10
presentation: 3
questions: I wasn’t the most well-positioned to assess the novelty of this work. I took a look through Pan et al. 2024’s survey “Automatically Correcting Large Language Models” paper to help check this, might be useful to reference for other readers?
soundness: 4
strengths: Solid theoretical argument that runs throughout the paper. Though this isn’t my core area it was quite easy to understand and most claims were plausible.

The experiments are well connected to the theoretical analysis. Neither part feels like an afterthought, which is a bit rare for LLM reasoning papers :)

The discovery of best practices for prompt sets can both be directly applied in engineering work and used to inspire future analysis.
summary: The paper claims to demonstrate the intrinsic self-correction capabilities of LLMs. Motivated from examples where bias in the prompts can interfere with the correction (in particular, by implying there’s a problem with the initial answer), they ask *if they can correct a wrong answer, why can’t an LM get it right the first time?*

They claim that they can’t due to inherent hallucinations but can perform intrinsic SC. They provide some theoretical analysis to demonstrate this, alongside empirical investigations into the impact of temperature, prompt structure, etc.

According to their theoretical analysis, using CoT during the initial answer generation produces a rationale alongside it, which can be attended over in the self-correction step to produce a candidate decision before reasoning for the decision has to be written. However, without the initial chain, generating this decision is effectively relying on another zero-shot correction in line with how the initial answer was (possibly erroneously) generated. In other words, systems where answers are generated by CoT are more robust to self-correction error introduction than ones that don’t use it, and proper self-correction on non-CoT answers essentially just brings performance back up to CoT levels.

This analysis leads to predictions such as hallucination rate increasing with higher temperature leading to worse performance of self-correction. This is confirmed. They also make some claims that feel more dubious to me, like “LLMs are naturally underperforming their intrinsic ability” which feels a little underdetermined to me.

**Edit**: I am happy with this paper overall, and will defer somewhat to the other reviewers on gripes about the theoretical arguements. That being said, I appreciate the authors' efforts to expand the scope of their experiments in the rebuttal.
weaknesses: Few, mainly the more philosophical claims I complained about in my summary.

---

confidence: 5
contribution: 1
initial_rating: 3
presentation: 1
questions: Please see above.
soundness: 1
strengths: May be interesting to future research.
summary: This paper investigates the sefl-correction ability of LLMs. In particular, they investigate two factors, i.e. temperature and the bias of prompt. The authors conduct experiments on 6 benchmarks using 4 LLMs. However, the paper is very difficult to read, with less rigour proof. The selection of models is not wide, resulting in less convincing empirical results. Also, I do not see a systematic and coherent motivation to investigate temperature and the bias of the prompt. It is somehow more like assembling these two factors in one paper.

**After the rebuttal**: the definitions of concepts in this paper are very unclear, with many left unexplained. Most proofs are not valid and rigorous. Many sections are difficult to understand. While I defer to some reviewer's appreciation, I do not believe this paper merits an ICLR paper standard.
weaknesses: 1. The paper is very difficult to read. In particular, the equation part.  Proof 2.1 is very confusing and many explanations are not given, and I cannot understand it with 2 or 3 times reading. For example, in Line 869, "which we denote as correct(A ∈ Q) = λ > k1". What is λ? The portion of correctly answered questions/all questions? And I do not think the probability of that hallucination randomly changing the answer is equal: tokens are of different importance to a sentence (answer), thus, some hallucinated tokens may not change the final answer (this is also related to how the authors define hallucination, but I do not see any justification here). Also, what is h? The portion of hallucinated responses? I also do not understand what the authors mean by h∗(1−λ)/(k−1) answers will be changed from incorrect to correct. Also, why h∗λ> h∗(1−λ)/(k-1)? Line 283, "(a1/T −(1−α)1/T)" should be "α" not "a".
2. The selection of models is limited. More open sourced model should be included for a more comprehensive evaluation.

---

confidence: 4
contribution: 1
initial_rating: 5
presentation: 3
questions: 1. What are the novel contributions of this paper over previous works on intrinsic self-correction? Given that the paper investigates the topic related to prior work on self-correction (e.g., [Pan et al., 2023](https://arxiv.org/abs/2308.03188)), it would be helpful to clarify how this research extends beyond or differs from earlier findings.
2. Why is some certain experimental results not obviously show the advantage of self-correction? Is it from the method itself, or there are other failure modes?
soundness: 3
strengths: 1. **Insightful Analysis of Self-Correction Mechanisms**: The paper offers a novel perspective by comparing intrinsic self-correction to chain-of-thought and self-verification techniques. This theoretical framing provides a solid foundation for understanding the mechanisms that enable LLMs to self-correct.
2. **Practical Recommendations for Enhanced Model Performance**: By identifying unbiased prompts and zero temperature as key factors for effective self-correction, the authors present valuable insights that are directly applicable to real-world LLM deployment.
summary: The paper investigates the capacity of LLMs to perform self-correction without relying on external sources of information, namely “intrinsic self-correction.” The authors propose that this self-correction process is valuable for enhancing LLM performance and has similar principles as chain-of-thought and self-verification. The study identifies two key factors that influence self-correction: using an unbiased prompt to avoid influencing the model’s initial answer and setting a zero temperature to reduce randomness. Through theoretical analysis and empirical experiments, the paper demonstrates that intrinsic self-correction is achievable across multiple LLMs, providing a foundation for further research in leveraging this capability.
weaknesses: 1. **Limited Performance Improvement Even in Ideal Conditions**: The performance gain from self-correction is relatively small, even under the ideal settings proposed (unbiased prompts and zero temperature). As is shown in Table 1 & 2, under many circumstances, the performance gain is less than 2%. This marginal improvement raises questions about the practical impact of the self-correction process and the techniques provided in the paper.
2. **Lack of Significant Theory and Insight**: The paper creatively related the performance of self-correction to chain-of-thought prompting and temperature, but it fails to introduce solid theoretical backbones or experimental insights. 
3. **Limited Model Sizes and Architectures**: The paper only conducted experiments on two API-based models and two open-source models, and it fails to conduct ablation on model sizes (using the same series of models with different sizes) and other model architectures.   This limitation restricts the generalizability of the findings across a broader range of model types.

---

confidence: 4
contribution: 1
initial_rating: 3
presentation: 1
questions: N/A
soundness: 1
strengths: - The problem of self-improvement is of great interest and importance for the research community, particularly given recent debates about whether LLMs possess genuine self-correction abilities.
- The issue of temperature hasn't been studied enough in the context of self-improvement. The authors' attempt to systematically analyze this parameter's influence is valuable.
- The idea to break self-improvement into multiple stages like chain-of-thought and study each component separately (as shown in Section 2.2) is interesting and potentially useful for understanding the mechanism of self-correction.
summary: - **Context: **The paper studies intrinsic self-correction which is LLMs ability to improve **their own** prior decisions (in contrast to **extrinsic** self-correction that uses external feedback of the environment). There are several prior works that cast doubt on the effectiveness of intrinsic self-correction. 

 - **This presented paper:** This work argues that LLMs are indeed capable of intrinsic self-correction capabilities. The paper makes this argument through theoretical analyses and empirical experiments. The paper points out two critical factors for successful self-correction: (1) 
zero temperature and (2) fair prompts.
weaknesses: (1)  Theoretical Foundation Issues: The paper claims to provide theoretical proofs, but the analytical process lacks rigor. For example:
- Eq3 is just wrong, p(A|\tau) should be p(A, R | \tau). Basically, p(X, Y | Z) = p(X | Y, Z) x p(Y|Z)  (minor typo: \tau should be \tau_1). For the same reason, Eq 4 is also incorrect. 
 - Sec 4.1 opens by “the randomness in decision-making diminishes as the temperature decreases”; isn’t this just a trivial statement? In Equation 8, the authors take temperature to extremes to show that answers have more likelihood to flip (which is obvious even without all these equations). But more importantly, this finding has no meaningful connection to self-improvement. Even if it does, it does connect to self-improvement, it is somewhat unrealistic since practical applications rarely use such extreme temperatures.
 - The analysis in Section 5.1 makes an unreasonable assumption that biased prompts cause random answer flipping. This assumption isn't justified, and the subsequent theoretical analysis adds little insight to our understanding.

(2) Overstatement of Results: The authors frequently jump to conclusions without sufficient evidence:
 - In line 194, they cite Table 1 as evidence of "improvement in accuracy after SC", but the difference is merely 0.08% (75.92% to 76.00%).
 - While Table 2 shows statistical significance across multiple datasets, the actual improvements are typically within 1%. This hardly constitutes meaningful improvement.
- In line 311, they make a definitive claim that "GPT-3.5 follows Order 1 of Eq. (5), whereas GPT4 follows Order 2" without providing quantitative evidence for this classification.

(3) Limited Practical Relevance of Temperature Analysis:
 - Figure 2 shows that temperature effects in the practical range (0-0.7) are minimal.
 - The significant effects only appear at temperature values like 1.4, which are rarely used in practice for self-improvement tasks.

-------------

Presentation Issues:
 - Figure 1 would benefit from including key notation (τ1, R1, D, etc.) referenced throughout the paper.
 - Critical prompt examples are relegated to the appendix when they should be in the main text for better understanding of the methodology.
- In Fig1 caption, make it clear what makes your prompt “unbiased”. One might spend a long time reading the prompt and not understanding what they should pay attention to. 

> Feng et al. (2023) argues that CoT increases the effective depth of the circuit by letting the generated outputs repeatedly loop back to the input.

This phrasing is extremely misleading. The arguments of Feng et al. is that with longer generations, Transformers (for **SOME choice of weights**; not necessarily the pre-trained weights) have more expressive capacity to model more complex problems (circuits). However, this argument does not automatically mean that the existing pre-trained LLMs actually do this, which is what is implied by your phrasing. 

Missing references:

There is other works that casts doubt on the abilities of LLMs to self-improve that are missing in your work: 
- Self-[In]Correct: LLMs Struggle with Discriminating Self-Generated Responses, 2024
- The Google paper: LLMs cannot find reasoning errors, but can correct them given the error location, 2024
- ... (there are a few others that I might be missing)
Given that your work directly rebuts the above works, please include appropriate discission for each one (not just Huang et al.) 

Here are several other relevant works that are not cited in your work: 
- Crystal: Introspective reasoners reinforced with self-feedback, 2023
- Generating sequences by learning to self-correct, 2023
- The self-Instruct paper, 2023

Your work is also related to the growing literature on inference-scaling. As the prior work has shown, inference-scaling works. Naturally, one can extend these to data generation, as you do in your work.

Meta-review: This paper studies intrinsic self-correction in large language models (LLMs) – their ability to revise previous answers without external input. looking at how temperature and the bias of prompts affect their performance. The paper’s core claim is that with carefully chosen “fair” prompts and zero temperature at sampling, LLMs can correct some of their initial mistakes. The paper includes theoretical analysis and empirical experiments conducted across multiple benchmarks and LLMs. The main conclusion is that intrinsic self-correction can provide small but consistent performance gains.

After evaluating the reviews and the authors’ subsequent clarifications, I am inclined to side with Reviewers 3P3T and shEV, who both raise substantial issues regarding clarity, rigor, and limited practical significance. I agree with Reviewer shEV that temperature analysis does not provided a meaningful contribution and also that the reported gains in typical temperature range are minimal. 

While the authors make a good-faith effort to address questions, multiple reviewers remain unconvinced that the paper demonstrates the rigor and impact required for acceptance.

Decision: Reject