Self-Reflection Outcome is Sensitive to Prompt Construction
Authors: Fengyuan Michael Liu, Nouar AlDahoul, Gregory Eady, Yasir Zaki, Bedoor AlShebli, Talal Rahwan


Reviewer paper summaries: This paper studies the effectiveness of self-reflection prompting for LLMs, where previous works have identified inconsistent performance improvement. he authors observed that prior research often relied on a single prompt with explicit terms (e.g., "problem," "mistake"), potentially biasing the model's self-reflection behaviors and leading to overcorrection of already correct answers ("false positives"). By removing these biases, self-reflection prompting performance improved across different prompts.  Therefore, the authors argue that the outcome of self-reflection is sensitive to prompt wording. To mitigate this issue, they propose a mixture-of-prompt (MoP) method that utilizes prompts with both over-correction and under-correction biases to elicit a balanced set of responses and use the aggregated results as the final prediction. Experimental results demonstrate that the proposed method effectively improves MMLU performance.
Summary of strengths: 1. This study is well-motivated based on a meta-analysis of previous works and combined verification experiments. 
2. The experiment results partially support the main argument on model sensitivity over prompt wording. 
3. The proposed mixture-of-prompt method is easy-to-understand and it directly follows the findings in this paper.
Summary of weaknesses: 1. **Contribution is limited and motivation is arguable**: It is well-known that certain wording may trigger better performance (e.g., "Let's think step by step" [1]), but better performance may not mean the model truly follows the instruction to do desired reasoning [5]. Self-reflection prompting is one of the prompting strategies, and the findings in this paper may be a re-discovery of this general prompting sensitivity problem in a narrow domain. Previous works [2, 3] differ in that they adopt minimal design in prompting and focus on evaluating model self-reflection behavior. It is arguable whether certain desired model behaviors needing specific design to be triggered still count as intrinsic model capabilities, or if human users are adding such capabilities through additional design. From these viewpoints, the motivation of this paper is arguable and its contribution might be limited.

2. **The main experiment setting is not well-explained**: By comparing the most relevant previous works [2, 3], a couple of issues arise. First, in [2, 3], no self-consistency prompting is adopted when directly examining the effectiveness of self-reflection prompting, but in this paper, self-consistency prompting is adopted without further explanation. Considering that the temperature $T>0$ may introduce stochasticity in the sampled outputs and self-consistency can enforce additional bias by aggregating multiple outputs [4], it is hard to measure the sole influence of prompt wording. More rigorous statistical significance verification is needed to support the findings. Second, the temperature setting and other decoding parameter settings also appear uncommon. In the self-consistency paper [4], it has been observed that lower temperatures lead to worse performance as the model converges to greedy decoding.

3. **Metric design is not reliable**: The prompt false positive rate (Section 4) proposed in this paper is computed using the same model verifying its own outputs. Following the same finding as in the "critique"/"review" stage of self-reflection prompting, this part may also be sensitive to prompt wording and not reliable. It is confusing to use the model to compute such a false positive rate since the accuracy numbers should be a function only of the question, the ground truth answer, and the model responses. There should not be any model involvement. 

4. **Experiment analysis for the proposed method is insufficient**: The proposed MoP prompting does not bring significant performance over under-correction prompting (Table 3, only 0.2% improvement in accuracy, and even incurs a bit higher false positive rate). So the performance gain is still sensitive to wording as the author found, but the proposed balanced prompting does not work. Also, it seems that compared with over-correction prompting used in some previous works, the under-correction rate increased by a lot (30%) under MoP prompting, so it is confusing where the performance gain comes from considering only 16% mitigation on the over-correction rate. There should be more analysis explaining how to interpret these results, and why certain variants (e.g., MoP (array) ) may not work well. 


References:
[1] Kojima, Takeshi, et al. "Large language models are zero-shot reasoners." Advances in neural information processing systems 35 (2022): 22199-22213.

[2] Huang, Jie, et al. "Large Language Models Cannot Self-Correct Reasoning Yet." ICLR 2024. 

[3] Li, Yanhong, Chenghao Yang, and Allyson Ettinger. "When Hindsight is Not 20/20: Testing Limits on Reflective Thinking in Large Language Models." Findings of the Association for Computational Linguistics: NAACL 2024.

[4] Wang, Xuezhi, et al. "Self-Consistency Improves Chain of Thought Reasoning in Language Models." ICLR 2023. 

[5] Wang, Boshi, et al. "Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters." ACL 2023.
Comments, suggestions and typos: 1. Minor question: The analysis of prompts used in previous works might need further examination. In [2], "either correct or incorrect" is used as a way to balance the model responses, similar to the proposed method MoP (unbiased). In [3], according to the Merriam-Webster dictionary, "Criticism is most often used broadly to refer to the act of negatively criticizing someone or something (“I’m more interested in encouragement right now than criticism”) or a remark or comment that expresses disapproval (“She shared a minor criticism about the design”), while critique is a more formal word for a carefully expressed judgment, opinion, or evaluation of both the good and bad qualities of something—for example, books or movies.", so it should not hint overcorrection behaviors. The prompt modification over prompt in [2] also does not bring a significant difference. So the argument for previous works using explicit terms to hint at overcorrection behaviors might need to be tuned down. 

2. Table 2 should be noted as "MMLU results" as it is not clear what this set of results belongs to what dataset in the main text.
Overall assessment scores: 2.5