Entrospect: Information-Theoretic Self-Reflection Elicits Better Response Refinement of Small Language Models
Authors: Tianqiang Yan, Ziqiao Lin, Zhenglong Sun, Lin Zhang, Yuan Gao
Accepted at: acl25

Entrospect: Information-Theoretic Self-Reflection Elicits Better Response Refinement of Small Language Models

Tianqiang Yan
Department of Data Science and AI
Faculty of Information Technology
Monash University
Melbourne, Victoria, Australia
tianqiang.yan@monash.edu

Ziqiao Lin
School of Science and Engineering
The Chinese University of Hong Kong
Shenzhen, Guangdong Province, China
ziqiaolin@link.cuhk.edu.cn

Lin Zhang
Shenzhen Technology University
Symbiosis-X Technology (Shenzhen) Inc.
Shenzhen, Guangdong Province, China
linzhang0529@gmail.com

Zhenglong Sun
School of Science and Engineering
The Chinese University of Hong Kong
Shenzhen, Guangdong Province, China
sunzhenglong@cuhk.edu.cn

Yuan Gao*
Shenzhen Institute of Artificial Intelligence and Robotics for Society
Shenzhen, Guangdong Province, China
gaoyuan@cuhk.edu.cn

Abstract
Self-reflection helps de-hallucinate Large Language Models (LLMs). However, the effectiveness of self-reflection remains insufficiently validated in the context of Small Language Models (SLMs), which exhibit limited semantic capacities. In particular, we demonstrate that the conventional self-reflection paradigm, such as Self-Refine, fails to deliver robust response refinement for models with parameter sizes of 10 billion or smaller, even when compared to generations elicited through Chain-of-Thought (CoT) prompting. To improve SLMs’ self-reflection, we redesign Self-Refine and introduce Entrospect (Entropy-aware Introspection), an information-theoretic framework based on prompt engineering.

We evaluated Entrospect using accuracy and average time consumption metrics to comprehensively assess its precision and computational efficiency. Experiments conducted across four distinct SLMs and four baseline methods demonstrate that Entrospect achieves the highest performance on validation tasks. Notably, under identical model and data settings, Entrospect delivers a remarkable improvement of up to 36.2% in reasoning accuracy while enhancing computational efficiency by as much as 10 times compared to its predecessor, Self-Refine.

*Corresponding Author

![A flowchart comparing Self-Refine and Entrospect for refining a mathematical problem. The Self-Refine approach fails, while Entrospect successfully solves the problem.](page_1047_1042_388_388.png)

Figure 1: The single-round refinement of an initial response for the same query, comparing Self-Refine and our proposed Entrospect. Self-Refine fully relies on the model’s self-reflected feedback, where any biases introduced during reflect are directly carried over into refine, hindering constructive improvements. On the other hand, our Entrospect identifies the optimal revision suggestion from an itemized output of the self-reflection, enabling Entrospect to achieve more robust and reliable response refinement.
1 Introduction

Large Language Models have advanced rapidly, impacting many fields with improved natural language generation (Brown et al., 2020; Chang et al., 2024). However, their tendency to produce hallucinations—especially counterfactual ones—poses a critical challenge to reliability (Zhang et al., 2023; Huang et al., 2023). Hallucinations occur when models generate factually incorrect or nonsensical outputs, undermining their trustworthiness and hindering real-world adoption. Addressing this issue is essential for improving their practical utility and acceptance (Weidinger et al., 2021, 2022).

To address these challenges, self-reflection has been proposed as a solution to counterfactual hallucinations, particularly for black-box models with inaccessible parameters (Madaan et al., 2024). However, its effectiveness is limited in Small Language Models (SLMs), which often lack sufficient semantic capabilities, inducing frequent occurrences of imperfect feedback, encompassing the self-reflected revision suggestions. Given the widespread use of SLMs in resource-constrained environments (Li et al., 2024; Wang et al., 2024), this limitation is particularly significant. In such cases, self-reflection may fail to consistently assist in the corrections of outputs, highlighting the need for more robust and scalable approaches.

Given the challenges of applying self-reflection to SLMs, a key question arises: how might we construct a framework that effectively integrates self-reflection to improve the precision of SLM outputs, all while preserving the computational efficiency? In response to this challenge, we propose Entrospect\footnote{https://github.com/henryyantq/Entrospect}, an information-theoretic framework predicated on Self-Refine that lessens the dependency on explicit semantic outputs from the model. Contrary to Self-Refine’s equal consideration of all revision suggestions, Entrospect employs an unsupervised mechanism to identify the most effective revision candidate, minimizing the impact of inferior ones, as illustrated in Figure 1.

Specifically, Entrospect is implemented with an Optimal Revision Suggestion Selector (ORSS) Inspired by (Wu et al., 2024) and (Yang et al., 2024b), the ORSS intervenes between the “reflect” and the “refine” stages that are tightly connected in the Self-Refine’s pipeline. It evaluates revision suggestions generated through self-reflection and identifies the one that minimizes the semantic uncertainty in the model’s refinement of the prior response, where low-quality suggestions conceivably ruining the successive procedures are ruled out. This selective approach distinguishes Entrospect from its predecessors, enhancing both the quality and reliability of the refined responses.

Architecturally, Entrospect retains the simplicity and efficiency of Self-Refine, operating as a parameter-free, recurrent finite-state machine (FSM) where modules are interconnected through purpose-specific prompts. This design ensures computational efficiency while maintaining the flexibility to adapt to diverse conversational AI tasks. Figure 2 summarizes the multifaceted contributions of Entrospect, the central focus of this study.

We evaluated Entrospect on natural language reasoning tasks, including the MATH dataset (Hendrycks et al., 2021) for math reasoning and HaluEval (Li et al., 2023) for hallucination detection. The results show Entrospect outperforms baselines like zero-shot, few-shot, Chain-of-Thought (CoT), and Self-Refine. These findings underscore two critical advances:

1. Selective Use of Self-Reflection: We highlight that the outcomes of a model’s self-reflection should not be directly or entirely relied upon as guidance for the response refinement.

2. ORSS-Driven Optimization: Our proposed Entrospect improves Self-Refine by introducing ORSS, an information-theoretic mechanism that unsupervisedly identifies the opti-

![A radar chart comparing Zero-Shot CoT, Self-Refine, and Entrospect across Computational Efficiency, Quality of Reasoning Outcomes, and Task Adaptability.](page_768_153_484_484.png)

Figure 2: Entrospect contributes furtherance to the response refinement of SLMs particularly over its predecessor, Self-Refine, across three major aspects.
mal revision from multiple candidates. Combined with our semantic similarity-based stopping condition, Entrospect allows a more robust and systematic approach to self-reflection for response refinement. Compared to its precursor, Self-Refine, Entrospect accomplishes a remarkable performance boost, delivering up to 36.2% improvement in accuracy under identical dataset and model conditions, while elevating computational efficiency by as much as 10 times.

2 Related Work

2.1 Self-Reflection of Language Models
The empirical foundation of self-reflection is that given some queries, language models may not be able to provide proper answers every time (Yan and Xu, 2023). Self-reflection assists in alleviating such problems by explicitly instructing a language model to review its generated response, providing a feedback on potential deficiencies within the current response and how they could be eliminated. The feedback is subsequently used for guiding the refinement of the previous answer. This procedure can be fully automated through a prompt-driven framework, by which a language model iteratively reflects and refines the answer to a query on its own (Lee et al., 2024).

Techniques like Self-Refine introducing mechanisms for models to improve their own responses (Madaan et al., 2024), especially in question-answering (QA) scenarios, to enhance generation quality. This approach has been further advanced in research such as Reflexion and Agent-Pro (Shinn et al., 2024; Zhang et al., 2024b), which extend self-reflection to agentic scenarios, increasing the efficiency and success rate of task execution during scenario exploration and trajectory execution. However, there remains significant room for improvement in its performance, particularly when it comes to SLMs.

Through extensive review, we found lack of report on the effectiveness of self-reflection applied to models which possess fewer than 10 billion parameters. Its success relies heavily on the context generated during the self-reflection process (Cheng et al., 2024) and is prone to overconfidence in its generated content (Zhang et al., 2024a), including biases.

We assessed the self-reflective capabilities of several SLMs across a variety of tasks, with Self-Refine chosen as a baseline approach. Our findings reveal that reflective thinking of these models fails to produce meaningful improvements in their generative performance. Entrospect is specifically designed to enhance the performance of SLMs by leveraging information theory to assist in the self-reflection process.

2.2 Enhancing the Reasoning Capabilities of Small Language Models
Recent studies have made significant strides in enhancing the reasoning capabilities of SLMs. Bi et al. introduced Solution-Guidance Fine-Tuning (Bi et al., 2024), focusing on problem understanding and decomposition to improve SLMs’ generalization and reasoning abilities. Wang and Lu explored continual pre-training on a synthetic dataset to inject multi-step reasoning abilities into moderate-sized models (Wang and Lu, 2023). Fu et al. specialized small models towards multi-step reasoning through knowledge distillation from large models (Fu et al., 2023). Yu et al. developed TRIPOST, an algorithm enabling small models to self-improve via interaction with large ones (Yu et al., 2023).

However, these methods often necessitate a substantial amount of additional data, whether it is synthetically created or derived from larger models, which may not be readily accessible or easy to produce. They entail a certain degree of computational overhead, be it in data generation, pre-training, or iterative training processes. Differently, Entrospect does not require any additional data or specialized training, thus drastically reducing both overhead and resource demands, allowing broader applicability across diverse domains and use cases.

3 Methodology

3.1 Problem Definition
While frameworks like Self-Refine aim to automate response refinement in language models through self-reflection, they do not inherently ensure that such refinements are beneficial. This limitation is particularly pronounced in SLMs, where constrained semantic capabilities lead to unreliable self-reflections, resulting in reflective contamination. Reflective contamination occurs when the model’s self-generated feedback contains biases, which can degrade rather than improve the refined response.

To formalize this problem, consider the \( t \)-th refinement round, where the model \( \mathcal{M}_\theta \) gener-
ates feedback \( F_t \) based on the query \( Q \), reflection prompt \( P_{\text{reflect}} \), and current response \( A_t \). This feedback, represented as \( \mathcal{M}_\theta (Q \| A_t \| P_{\text{reflect}}) \), consists of two components: 1) A valid portion \( S_t = \rho_t F_t \), which supports effective refinement. 2) Reflective contamination \( N_t = (1 - \rho_t) F_t \), which introduces biases. Here, \( \rho_t \in (0, 1) \) represents the proportion of valid feedback in \( F_t \). The refined response \( A_{t+1} \) is then generated using \( F_t, Q \), and the refinement prompt \( P_{\text{refine}} \), expressed as:

\[
A_{t+1} = \mathcal{M}_\theta (Q \| A_t \| F_t \| P_{\text{refine}})
= A_t + \alpha_t^S S_t - \alpha_t^N N_t
= A_t + \alpha_t^S \rho_t F_t - \alpha_t^N (1 - \rho_t) F_t
= A_t + \left[ (\alpha_t^S + \alpha_t^N) \rho_t - \alpha_t^N \right] F_t,
\]

where \( \alpha_t^S \) and \( \alpha_t^N \) are partial attention factors (\( \alpha \in (0, 1) \)) applied to the valid and contaminated portions of \( F_t \), respectively.

The Core Problem:

1. A successful refinement requires \( A_{t+1} \geq A_t \), but this is not guaranteed. When \( \rho_t \) is low (i.e., the feedback contains more contamination), the refined response may degrade, as described by the condition:

\[
\rho_t < \frac{\alpha_t^N}{\alpha_t^S + \alpha_t^N}.
\]

2. SLMs, with their limited semantic competence, often exhibit low \( \rho_t \) and high \( \alpha_t^N \) (or low \( \alpha_t^S \)), making them prone to degradation during the refinement phase of the response.

Objective: Within the realm of black-box models, \( \alpha_t^S, \alpha_t^N \) and \( \rho_t \) are inaccessible. This presents a significant obstacle in accurately differentiating between \( S_t \) and \( N_t \). An alternative perspective involves concentrating exclusively on the optimal component of \( F_t \). Entrospect proposes an unsupervised mechanism driven by information theory, providing a systematic solution to this complication.

3.2 Optimal Revision Suggestion Selector

By employing a formatting prompt, we can steer the model’s self-reflective output towards a systematic arrangement of multiple revision suggestions. In this way, \( F_t \) is characterized as an ensemble of strings \( \{ f_t^0, f_t^1, \ldots, f_t^k \} \), framing our goal as “discerning an optimal revision suggestion from this set”. However, in the absence of supervision, defining what constitutes optimal becomes a fundamental issue.

To address this, we propose a solution called the Optimal Revision Suggestion Selector (ORSS), which uses heuristic information-theoretic approaches for prompt selection (Wu et al., 2024; Yang et al., 2024b). These studies suggest that an optimal prompt should minimize the semantic uncertainty of a language model when processing a query, which is equivalent to maximizing the conditional mutual information (CMI) between the input and the output. Unlike recent work which assumes a manually constructed prompt pool, \( F_t \) as the candidate set in our case is constructed in an automatic fashion, where revision suggestions become prompt candidates, and the one to be selected renders the maximum CMI following Equation 3:
\[
f_t^* = \arg\max_{f_t \in F_t} I\left(A_{t+1}; f_t \mid Q \| A_t\right),
\]
where
\[
I = H\left(A_{t+1} \mid Q \| A_t\right) \\
- H\left(A_{t+1} \mid f_t, Q \| A_t\right).
\]
(3)

In Equation 3, \(Q \| A_t\) stands for the prompt “Please provide a refined solution of <Q> given <A_t>”, and \((f_t, Q \| A_t)\) signifies a slightly different prompt “Please provide a refined solution of <Q> given <A_t>. <f_t>”. The two \(H\)'s characterize the marginal entropy and the conditional entropy in classical information theory, respectively. The value of CMI \(I\) stands for the extent to which a revision suggestion \(f_t\) enhances the model’s confidence in the refinement applied to the current answer \(A_t\).

3.3 Eliciting the Convergence of Entrospect

We established a universal mechanism to enable Entrospect to automatically terminate its iterations. The core principle is that, at the semantic level, \(A_t\) and \(A_{t+1}\) are essentially equivalent. Consequently, when a language model employs greedy search (\(temperature = 0\)) for output sampling, subsequent outputs naturally converge toward consistency, rendering the increments from reflection and refinement negligible. Given these circumstances, the framework no longer introduces meaningful improvements to the response, a state we defined as “convergence”. More precisely, we leverage the cosine similarity \(S(\cdot, \cdot)\) to quantify the degree of semantic resemblance between two answers, modeled as

\[
S(A_1, A_2) = \frac{\mathbf{v}_1 \cdot \mathbf{v}_2}{\|\mathbf{v}_1\| \|\mathbf{v}_2\|}
= \frac{\sum_{i=1}^m (v_{1i} \cdot v_{2i})}{\sqrt{\sum_{i=1}^m v_{1i}^2} \cdot \sqrt{\sum_{i=1}^m v_{2i}^2}},
\]
(4)

where \(\mathbf{v} = [v_1 \ v_2 \ \ldots \ v_m]^T\) indicates the \(A\)'s tokenized vector in a continuous, \(m\)-dimensional semantic space. The range of \(S\) is \([-1, 1]\), with a higher value referring to a stronger semantic similarity between the two entities compared. Leveraging semantic similarity as a stopping condition for the iterative refinement procedure guarantees an appropriate termination juncture, thus optimizing performance results.

3.4 Framework of Entrospect

Slightly different from the three-step process of respond \(\rightarrow\) reflect \(\rightarrow\) refine adopted by Self-Refine, Entrospect follows an extended four-step strategy: respond \(\rightarrow\) reflect \(\rightarrow\) select \(\rightarrow\) refine. In the following, we detail each step sequentially; see Figure 3 for an intuitive illustration of the pipeline and Algorithm 1 for implementation guidance.

Respond: The iterations begin with the language model generating an initial answer \(A_0\) for the input query \(Q\).

Reflect and Select: During iteration \(t\), the model \(\mathcal{M}_\theta\), guided by the prompt \(P_{\text{reflect}}\), the original query \(Q\), and the current answer \(A_t\), generates a set of candidate revision suggestions denoted as \(F_t = \{f_t^0, f_t^1, \ldots, f_t^n\}\). The prompt \(P_{\text{reflect}}\) serves as a directive that instructs the model on how to evaluate potential deficiencies in the current answer and construct appropriate \(F_t\) accordingly. Thereafter, the ORSS selects the optimal \(f_t^*\) that maximizes the CMI between the input and the output of the model. In practical implementation, the Cross-Entropy Loss \(L_{CE}\) output by the model for a given input can be used to calculate the marginal entropy and the conditional entropy, allowing for the straightforward computation of the CMI.

Refine: Leveraging the \(f_t^*\) as the key instruction to the refinement, the model \(\mathcal{M}_\theta\) utilizes the prompt \(P_{\text{refine}}\), in conjunction with the original query \(Q\) and the current answer \(A_t\), to generate an updated answer \(A_{t+1}\).

Stop Condition: Subsequent to the generation of the \(A_{t+1}\), we exert the semantic textual similarity measure to check whether the iterative process should be terminated. When \(A_t\) and \(A_{t+1}\) exhibit a high degree of semantic resemblance, this suggests that Entrospect has entered a state of convergence from the current iteration onward. Following that, \(A_{t+1}\) is designated as the final output. To meet the requirements of long-text encoding with high representational fidelity, we opted for the Jina Embeddings V3 (Sturua et al., 2024) with a dedicated LoRA adapter for text-matching tasks, an encoder-based model which natively supports an input sequence length of up to 8192 tokens. In our experiments, \(S \geq 0.9\) is adopted as the threshold for considering \(A_t\) and \(A_{t+1}\) semantically equivalent.

We detailed the instructions involved in the operation process of Entrospect in Figure 6.
Algorithm 1 The algorithm pipeline of Entrospect
Require: query \( Q \), model \( \mathcal{M}_\theta \), prompt \( P_{\text{reflect}} \) (:= \( P_t \)), prompt \( P_{\text{refine}} \) (:= \( P_t \)), semantic similarity threshold \( S_{\text{th}} \)
1: \( A_0 \leftarrow \mathcal{M}_\theta(Q) \) \hspace{0.5em} ▷ Respond
2: \( A_t \leftarrow A_0 \)
3: while True do
4:   \( F_t \leftarrow \mathcal{M}_\theta(P_t||Q||A_t) \) \hspace{0.5em} ▷ Reflect
5:   \( \{f_t^0, f_t^1, \ldots, f_t^n\} \leftarrow \text{list}(F_t) \) \hspace{0.5em} ▷ Itemize
6:   \( I_{\max} \leftarrow 0 \)
7:   for \( f_t \) in list(\( F_t \)) do \hspace{0.5em} ▷ Select (ORSS)
8:     \( H_t^{\text{marg}} \leftarrow \mathcal{L}_{\text{CE}}\left(\mathcal{M}_\theta(P_t||Q||A_t)\right) \)
9:     \( H_t^{\text{cond}} \leftarrow \mathcal{L}_{\text{CE}}\left(\mathcal{M}_\theta(P_t||Q||A_t||f_t^*)\right) \)
10:    \( I_t \leftarrow H_t^{\text{marg}} - H_t^{\text{cond}} \)
11:    if \( I_t > I_{\max} \) then
12:      \( f_t^* \leftarrow f_t \)
13:    end if
14:   end for
15:   \( A_{t+1} \leftarrow \mathcal{M}_\theta(P_t||Q||A_t||f_t^*) \) \hspace{0.5em} ▷ Refine
16:   if \( S(A_t, A_{t+1}) \geq S_{\text{th}} \) then
17:     break
18:   end if
19:   \( A_t \leftarrow A_{t+1} \)
20: end while
21: return \( A_{t+1} \)

Table 1: Accuracies (%) of various methods equipped by four of the latest SLMs on reasoning tasks MATH (The average accuracies of level 1 to level 5) and HaluEval. We highlight the best results in bold.

<table>
  <tr>
    <th>Model Name</th>
    <th>Method</th>
    <th>MATH</th>
    <th>HaluEval</th>
  </tr>
  <tr>
    <td rowspan="2">DeepSeek-R1-Distilled</td>
    <td>Zero-Shot</td>
    <td>94.2</td>
    <td>80.5</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>90.2</td>
    <td>29.5</td>
  </tr>
  <tr>
    <td rowspan="2">Qwen 2.5</td>
    <td>Zero-Shot CoT</td>
    <td>91.3</td>
    <td>91.0</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>88.5</td>
    <td>80.0</td>
  </tr>
  <tr>
    <td>Instruct 1.5B</td>
    <td><b>Entrospect</b></td>
    <td><b>98.4</b></td>
    <td><b>95.5</b></td>
  </tr>
  <tr>
    <td rowspan="2">Qwen 2.5</td>
    <td>Zero-Shot</td>
    <td>78.2</td>
    <td>94.5</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>72.8</td>
    <td>91.0</td>
  </tr>
  <tr>
    <td rowspan="2">Instruct 7B</td>
    <td>Zero-Shot CoT</td>
    <td>83.8</td>
    <td>98.0</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>73.0</td>
    <td>97.5</td>
  </tr>
  <tr>
    <td><b>Entrospect</b></td>
    <td><b>86.0</b></td>
    <td><b>100.0</b></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">Llama 3.1</td>
    <td>Zero-Shot</td>
    <td>61.7</td>
    <td>94.5</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>56.5</td>
    <td>94.0</td>
  </tr>
  <tr>
    <td rowspan="2">Instruct 8B</td>
    <td>Zero-Shot CoT</td>
    <td>73.7</td>
    <td>94.5</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>44.3</td>
    <td>95.0</td>
  </tr>
  <tr>
    <td><b>Entrospect</b></td>
    <td><b>80.5</b></td>
    <td><b>99.5</b></td>
    <td></td>
  </tr>
  <tr>
    <td rowspan="2">GLM 4</td>
    <td>Zero-Shot</td>
    <td>55.0</td>
    <td>98.5</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>57.9</td>
    <td>97.5</td>
  </tr>
  <tr>
    <td rowspan="2">Instruct 9B</td>
    <td>Zero-Shot CoT</td>
    <td>65.8</td>
    <td>97.5</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>56.8</td>
    <td>97.5</td>
  </tr>
  <tr>
    <td><b>Entrospect</b></td>
    <td><b>69.7</b></td>
    <td><b>100.0</b></td>
    <td></td>
  </tr>
</table>

4 Experiments and Results

4.1 Experimental Settings
We evaluated Entrospect equipped by four of the latest SLMs, including DeepSeek-R1-distilled, Qwen 2.5 1.5B (Yang et al., 2024a; Guo et al., 2025), Qwen 2.5 7B (Yang et al., 2024a), Llama 3.1 8B (AI, 2024), and GLM-4 9B (GLM et al., 2024), as compared to the baselines (see Section 4.4) on a math reasoning dataset and a hallucination detection dataset, namely MATH (Hendrycks et al., 2021) and HaluEval (Li et al., 2023). Each SLM was quantized to INT4 precision with either Auto-GPTQ or BitsAndBytes (Pan, 2023; Dettmers et al., 2022).

4.2 Datasets
To comprehensively assess whether Entrospect heightens the ubiquitous reasoning performance of SLMs, we sourced our validation data from two representative datasets, MATH and HaluEval, with illustrative examples provided in Table 4.

MATH (Hendrycks et al., 2021): a dataset designed to measure the mathematical reasoning capabilities of language models, consisting of problems sourced from high school math competitions, tagged with difficulty levels from 1 to 5 and covering a wide range of topics including algebra, geometry, number theory, and combinatorics. MATH is notable for its complexity compared to the other datasets of the same category (Frieder et al., 2024), e.g. GSM8K (Cobbe et al., 2021). Besides, the latest findings have unveiled that MATH suffers less leakage than GSM8K does from the worsening cheating on model training (Xu et al., 2024), underlining its fairness. We randomly chose 120 samples from each difficulty level to serve as our experimental dataset.

HaluEval (Li et al., 2023): a dataset that gauges the performance of language models in recognizing hallucinations, featuring general user queries and task-specific examples across question answering, dialogue, and text summarization. We randomly sampled 200 pairs from this dataset, providing a robust evaluation platform for analyzing the effectiveness of our framework in detecting and reducing hallucinations.

4.3 Evaluation Metrics
We selected two evaluation metrics, i.e. Accuracy and Average Time Consumption (Han et al., 2023;
Xu et al., 2023; Xiao et al., 2024), to provide both qualitative and quantitative insights into the effectiveness of Entrospect.

Accuracy: a pivotal evaluation metric, is delineated as the proportion of problems correctly resolved relative to the total number of problems the model attempts, computed via \( A_{\text{correct}} / (A_{\text{correct}} + A_{\text{wrong}}) \times 100\% \). A higher accuracy signifies that a prompting scheme is more effective in lifting the model’s reasoning outcomes.

Average Time Consumption: We measured the Average Time Consumption (ATC) of the selected prompting schemes, spanning from the moment the input is supplied to the generation of the final output. Given the sample size \( N \) of the validation set, ATC is calculated by \( \frac{1}{N} \sum_k^N (t_{k_o} - t_{k_i}) \), where \( t_{k_o} - t_{k_i} \) denotes the duration, counted in seconds, from the moment the \( k \)-th input is supplied to the time the \( k \)-th output is generated. A smaller ATC embodies better computational efficiency of a prompting method, which is vital for industrial implementation, notably on edge computing devices running local SLMs. In our assessments, both of the above metrics are considered for more comprehensive analysis.

4.4 Baseline Selection

We compared Entrospect against the following well-established prompting methods as well as its ablated version, functioning as robust benchmarks for appraising the performance uplift in SLMs achieved with Entrospect.

Zero-Shot and Few-Shot Prompting (Brown et al., 2020): Zero-shot prompting directs a language model to perform tasks with only high-level instructions, often sacrificing accuracy for complex inputs. Conversely, few-shot prompting supplies demonstrations to improve context awareness and performance, yet its success hinges on the quality of examples, which may not fully capture task complexity and may be labor-intensive to gather in practice.

Chain-of-Thought Prompting (Wei et al., 2022): An approach that guides language models to generate a structured reasoning path before arriving at the final answer, encouraging more systematic and transparent problem solving. A key downside is the increased potential for longer outputs, as irrelevant, inaccurate, and repetitive steps may appear in the generated thought chain, especially concerning SLMs, impairing the overall outcome.

![The Accuracy-ATC results derived from evaluating Entrospect and Self-Refine across four models and two tasks. The dividing lines in the chart correspond to the decision boundaries determined by linear SVMs fit on the data points of Entrospect and Self-Refine. Data points positioned closer to the top-left corner signify a more favorable trade-off between computational efficiency and reasoning accuracy, indicating superior overall performance.](page_1012_186_579_410.png)

Figure 4: The Accuracy-ATC results derived from evaluating Entrospect and Self-Refine across four models and two tasks. The dividing lines in the chart correspond to the decision boundaries determined by linear SVMs fit on the data points of Entrospect and Self-Refine. Data points positioned closer to the top-left corner signify a more favorable trade-off between computational efficiency and reasoning accuracy, indicating superior overall performance.

Self-Refine (Madaan et al., 2024): The framework allows a model to iteratively revise its own outputs with identified errors from the self-reflection’s feedback. Despite its potential, such a strategy may introduce unnecessary or incorrect changes during the refinement cycles, especially for SLMs, as mentioned in Section 1.

Ablated Entrospect: The variant of Entrospect without the semantic similarity-based stopping condition. Instead, a manual setting of 5 fixed iterations is assigned. This baseline serves as the ablation study that verifies the efficacy of our nominated convergence policy.

4.5 Results

We report the Entrospect’s competitive competences versus the baseline prompting approaches, especially Self-Refine, in augmenting the SLMs’ semantic reasoning across two validation tasks.

Entrospect improves reasoning accuracies: Displayed in Table 1 and 3, SLMs armed with En-
Figure 5: (A higher Number of Correct Outputs is better) Constrained on a fixed 5 rounds of refinement rather than the stopping condition, the ablated Entrospect falls into suboptimal performance in contrast to the complete version across both tasks and all involved models. This highlights the significance and efficacy of importing semantic similarity comparison as the stopping condition for our framework.

trospect outshines all other baselines pertaining to the reasoning accuracies across both MATH and HaluEval validation sets. In contrast specifically to Self-Refine, Entrospect yields a maximum improvement of 36.2% with Llama 3.1 Instruct 8B (44.3% → 80.5%) on the MATH dataset and 15.5% with DeepSeek-R1-Distilled Qwen 2.5 Instruct 1.5B (80.0% → 95.5%) on the HaluEval dataset. Moreover, Figure 7 highlights Entrospect’s robustness beyond handling math problems with a fixed complexity. When set against Self-Refine, Entrospect consistently offers more substantial mitigation against the overall degradation of reasoning accuracy as the problem difficulty rises, securing a reduced decay rate as much as 52.8%.

The exceptional computational efficiency: As depicted in Figure 4, Entrospect reaches convergence faster than Self-Refine across most instances. On the MATH dataset, Entrospect reduces runtime by an average factor of up to 2.8 (e.g., Llama 3.1 8B + Entrospect), meanwhile demonstrating even more pronounced efficiency gains on the HaluEval dataset, with runtime reductions reaching up to 10-fold (e.g., DeepSeek R1-Distill Qwen 2.5 1.5B + Entrospect). Beyond its efficiency advantages, Figure 4 highlights Entrospect’s ability to strike a superior balance between computational efficiency and accuracy, driving substantial overall performance enhancements in SLMs.

To investigate potential correlations between model parameter sizes and the ATC outcomes achieved by Entrospect, we employed Spearman’s rank correlation coefficient alongside corresponding p-values (Spearman, 2010). However, no statistically significant relationship was observed within the scope of our experiments (MATH: corr = −0.600, \( p = 0.400 \); HaluEval: corr = −0.200, \( p = 0.800 \)).

Ablation study: To validate whether the semantic similarity-based stopping condition is crucial for propelling a higher reasoning accuracy of Entrospect, we conducted an ablation study by removing this mechanism and fixing the number of refinement cycles to 5. Figure 5 illustrates that the ablated Entrospect constantly underperforms compared to the complete implementation, witnessing performance deficits of 1.8 → 8.9% on the MATH dataset and 1.5% → 3.5% on the HaluEval dataset across all tested SLMs. The results solidify the role of the semantic similarity-guided stopping condition as a cornerstone for enhancing Entrospect’s overall performance.

5 Conclusion

This paper introduces Entrospect, an optimized Self-Refine framework that leverages an information-theoretic Optimal Revision Suggestion Selector to provide optimal revision suggestions during the self-reflection stage while eliminating ineffective ones for efficient refinement of initial responses from SLMs. Besides, the convergence of Entrospect is made possible with a dedicated semantic similarity-determined stopping condition. Through our holistic evaluations, Entrospect claimed superior performance relative to the baseline methods on both of our reasoning tasks across four SLMs of diverse parameter sizes, obtaining a maximum 36.2% reasoning accuracy uplift and at most 10 times the computational efficiency exclusively over its antecedent, Self-Refine.

We aspire for this study to inspire further advancements in small language models research and furnishes new perspectives for information-
theoretic prompt engineering.

Limitations

There remains much room for promoting Entrospect, and our future studies shall prioritize the following key limitations:

More solid definition of an optimal revision suggestion: The ORSS of Entrospect, grounded in maximizing the conditional mutual information, operates as an approximate selection technique in unsupervised settings. This approach gauges the quality of a revision suggestion by leveraging the model’s intrinsic output uncertainty as a pivotal determinant. However, its reliability is compromised when the model demonstrates undue confidence in erroneous outputs. As a result, it is imperative to pursue a more precise and theoretically grounded definition of what constitutes an optimal revision suggestion in our future studies.

Beyond semantic similarity comparison as the stopping condition: A high semantic similarity between consecutive refinement iterations as a sign of convergence is logically aligned with language models adopting greedy search sampling. In conversational situations, however, sampling methods such as Top-K and nucleus sampling are more regularly used to ensure generative variability. Our future work will seek to modify the current convergence mechanism tailored to these sampling configurations.

Ethics Statement

This study strictly adheres to the Ethical Policy of the Association for Computational Linguistics. We conducted a thorough assessment of the potential impacts of our research and did not identify any evident ethical concerns. All datasets utilized in this study were sourced from publicly available resources and were handled strictly in accordance with their respective terms of use. Nevertheless, we acknowledge the possibility of unforeseen impacts in any research and invite readers to share feedback on any potential ethical concerns they may identify.

Acknowledgements

We appreciate the anonymous reviewers for their time and insightful comments. In addition, we would like to express our special gratitude to Dr. Lizhen Qu2 from the Department of Data Science and AI, Faculty of Information Technology, Monash University, for his invaluable guidance in establishing the theoretical foundation of our methodology.

References

Meta AI. 2024. Llama 3.1: The most capable open foundation models. https://ai.meta.com/blog/meta-llama-3-1/.

Jing Bi, Yuting Wu, Weiwei Xing, and Zhenjie Wei. 2024. Enhancing the reasoning capabilities of small language models via solution guidance fine-tuning. arXiv preprint arXiv:2412.09906.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.

Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1–45.

Ruxo Cheng, Haoxuan Ma, and Shuirong Cao. 2024. Deceiving to enlighten: Coaxing llms to self-reflection for enhanced bias detection and mitigation. arXiv preprint arXiv:2404.10160.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.

Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3. int8 ( ): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318–30332.

Simon Frieder, Mirek Olsák, Julius Berner, and Thomas Lukasiewicz. 2024. The imo small challenge: Not-too-hard olympiad math datasets for llms. In Tiny Papers@ ICLR.

Yao Fu, Hao Peng, Litou Ou, Ashish Sabharwal, and Tushar Khot. 2023. Specializing smaller language models towards multi-step reasoning. In International Conference on Machine Learning, pages 10421–10430. PMLR.

Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.

Chengcheng Han, Xiaowei Du, Che Zhang, Yixin Lian, Xiang Li, Ming Gao, and Baoyuan Wang. 2023. Dialcot meets ppo: Decomposing and exploring reasoning paths in smaller language models. arXiv preprint arXiv:2310.05074.

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).

Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2023. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems.

Dongyub Lee, Eunhwan Park, Hodong Lee, and Heui-Seok Lim. 2024. Ask, assess, and refine: Rectifying factual consistency and hallucination in llms with metric-guided feedback learning. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2422–2433.

Beibin Li, Yi Zhang, Sébastien Bubeck, Jeevan Pathuri, and Ishai Menache. 2024. Small language models for application interactions: A case study. arXiv preprint arXiv:2405.20347.

Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. Halueval: A large-scale hallucination evaluation benchmark for large language models.

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegrefe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36.

Qiwei Pan. 2023. Autogptq: An easy-to-use llms quantization package with user-friendly apis, based on gptq algorithm.

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36.

C Spearman. 2010. The proof and measurement of association between two things. International Journal of Epidemiology, 39(5):1137–1150.

Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastropas, Andreas Koukounas, Nan Wang, et al. 2024. jina-embeddings-v3: Multilingual embeddings with task lora. arXiv preprint arXiv:2409.10173.

Fali Wang, Zhiwei Zhang, Xianren Zhang, Zongyu Wu, Tuzhao Mo, Qiuqiao Lu, Wanjing Wang, Rui Li, Junjie Xu, Xianfeng Tang, et al. 2024. A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trustworthiness. arXiv preprint arXiv:2411.03350.

Tianduo Wang and Wei Lu. 2023. Learning multi-step reasoning by solving arithmetic tasks. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1229–1238.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359.

Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, et al. 2022. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 214–229.

Junda Wu, Tong Yu, Rui Wang, Zhao Song, Ruiyi Zhang, Handong Zhao, Chaochao Lu, Shuai Li, and Ricardo Henao. 2024. Infoprompt: Information-theoretic soft prompt tuning for natural language understanding. Advances in Neural Information Processing Systems, 36.

Bin Xiao, Burak Kantarci, Jiawen Kang, Dusit Niyato, and Mohsen Guizani. 2024. Efficient prompting for llm-based generative internet of things. arXiv preprint arXiv:2406.10382.

Ruijie Xu, Zengzhi Wang, Run-Ze Fan, and Pengfei Liu. 2024. Benchmarking benchmark leakage in large language models. arXiv preprint arXiv:2404.18824.

Zhaozhao Xu, Zirui Liu, Beidi Chen, Yuxin Tang, Jue Wang, Kaixiong Zhou, Xia Hu, and Anshumali Shrivastava. 2023. Compress, then prompt: Improving accuracy-efficiency trade-off of llm inference with transferable prompt. arXiv preprint arXiv:2305.11186.

Tianqiang Yan and Tiansheng Xu. 2023. Refining the responses of llms by themselves. arXiv preprint arXiv:2305.04039.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115.

Sohee Yang, Jonghyeon Kim, Joel Jang, Seonghyeon Ye, Hyunji Lee, and Minjoon Seo. 2024b. Improving probability-based prompt selection through unified evaluation and analysis. Transactions of the Association for Computational Linguistics, 12:758–774.

Xiao Yu, Baolin Peng, Michel Galley, Jianfeng Gao, and Zhou Yu. 2023. Teaching language models to self-improve through interactive demonstrations. arXiv preprint arXiv:2310.13522.

Wenqi Zhang, Yongliang Shen, Linjuan Wu, Qiuying Peng, Jun Wang, Yueting Zhuang, and Weiming Lu. 2024a. Self-contrast: Better reflection through inconsistent solving perspectives. arXiv preprint arXiv:2401.02009.

Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, and Weiming Lu. 2024b. Agent-pro: Learning to evolve via policy-level reflection and optimization. arXiv preprint arXiv:2402.17574.

Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023. Siren’s song in the ai ocean: a survey on hallucination in large language models. arXiv preprint arXiv:2309.01219.

A Appendix
Input Q

Respond → \( A_0 \)

\( A_0 \rightarrow A_t \)

Reflect (for Hallucination Detection) → \( F_t \):
[Task] Q
[Assessment] \( A_t \).
Analyze the assessment result of whether the provided answer correctly addresses the query based on the given passage. Identify any potential inaccuracies, logical gaps, or areas where the reasoning could be improved. Provide some optimization suggestions to enhance your judgment, ensuring that your evaluation is thorough and accurate.

Reflect (for Math Reasoning) → \( F_t \):
[Problem] Q
[Solution] \( A_t \).
Please analyze the above solution to the given math problem. Your task is to identify any deficiencies or errors in the solution. Please follow these steps:
1. **Understand the Problem**: Carefully read and comprehend the math problem to grasp what is being asked.
2. **Review the Solution**: Examine the provided solution step by step.
3. **Identify Deficiencies**: Look for errors in calculations, logical reasoning, or assumptions. Note any steps that are missing, incorrect, or insufficiently justified.
4. **Assess Clarity and Completeness**: Evaluate the explanation for clarity and whether it fully addresses the problem.

Itemize → \( \{f_1^t, f_2^t, ..., f_n^t\} \):
Format all independent revision suggestions that can be extracted from \( F_t \) in a **Python List of Strings**.

Select \( f_i \) from \( \{f_1^t, f_2^t, ..., f_n^t\} \)

Refine (for Hallucination Detection) → \( A_{t+1} \):
[Task] Q
[Assessment] \( A_t \).
Please re-check your previous assessment referring to the suggestion \( f_i^t \) and provide your final decision.

Refine (for Math Reasoning) → \( A_{t+1} \):
[Problem] Q
[Previous Solution] \( A_t \).
Please provide a refined solution for the problem given the previous solution referring to the suggestion: \( f_i^t \)

Check if (\( A_t, A_{t+1} \)) Meets Stopping Condition or Not
True → Return \( A_{t+1} \)
False → \( A_{t+1} \rightarrow A_t \)

Figure 6: (Referred in Section 3.4) The detailed instructions used for all prompting nodes (modules) within the Entrospect framework during the evaluation phases. These instructions guide the SLMs through the process of generating an initial response, reflecting on its deficiencies, selecting the optimal revision, and refining the response based on the selected suggestion.
Table 2: (Referred in Section 4.2) Representative data samples from the MATH and HaluEval datasets, demonstrating a mathematical reasoning problem and a reading comprehension task.

<table>
  <tr>
    <th>Dataset</th>
    <th>Query</th>
    <th>Label</th>
  </tr>
  <tr>
    <td>MATH</td>
    <td>What is the simplified numerical value of \( \frac{a+11b}{a-b} \) if \( \frac{4a+3b}{a-2b} = 5 \)?</td>
    <td>Let’s play with the given condition a little. Clearing out the denominator gives \( 4a + 3b = 5(a - 2b) = 5a - 10b \). Selectively combine like terms by adding \( 9b - 4a \) to both sides to get \( 12b = a - b \). This gives \( \frac{12b}{a - b} = 1 \). Now, we want to find \( \frac{a + 11b}{a - b} \). Rewrite this as \( \frac{a - b + 12b}{a - b} = \frac{a - b}{a - b} + \frac{12b}{a - b} = 1 + 1 = [2] \), and we are done.</td>
  </tr>
  <tr>
    <td>HaluEval</td>
    <td>The following is a reading comprehension task, which provides a passage, a question related to the passage, and an answer to the question: [Passage] The ValleyCats play at Joseph L. Bruno Stadium which opened in 2002 on the campus of Hudson Valley Community College located in Troy. Joseph Bruno Stadium is a stadium located on the campus of Hudson Valley Community College in Troy, New York. [Question] The Tri-City ValleyCats play at which stadium located on the campus of Hudson Valley Community College in Troy, New York? [Answer] Troy Community Stadium, located on Hudson Valley Community College campus. Please determine whether the given answer is correct. If it is correct, output ‘PASS’; if it is incorrect, output ‘FAIL’.</td>
    <td>FAIL</td>
  </tr>
</table>
Table 3: (Referred in Section 4.5) The extended table of accuracies(%) on the MATH dataset, providing a detailed breakdown of all results across Level 1 to Level 5, where Entrospect performs the best with all SLMs relative to the baseline prompting methods across all difficulty levels. We highlight the best results in bold.

<table>
  <tr>
    <th>Model Name</th>
    <th>Method</th>
    <th>MATH-L1</th>
    <th>MATH-L2</th>
    <th>MATH-L3</th>
    <th>MATH-L4</th>
    <th>MATH-L5</th>
  </tr>
  <tr>
    <td rowspan="4">DeepSeek-R1-Distilled Qwen 2.5 Instruct 1.5B</td>
    <td>Zero-Shot</td>
    <td>97.5</td>
    <td>95.0</td>
    <td>96.7</td>
    <td>91.7</td>
    <td>90.0</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>96.7</td>
    <td>92.5</td>
    <td>94.2</td>
    <td>91.7</td>
    <td>75.8</td>
  </tr>
  <tr>
    <td>Zero-Shot CoT</td>
    <td>75.0</td>
    <td>98.3</td>
    <td>98.3</td>
    <td>93.3</td>
    <td>91.7</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>90.0</td>
    <td>89.2</td>
    <td>90.8</td>
    <td>88.3</td>
    <td>84.2</td>
  </tr>
  <tr>
    <td>Entrospect</td>
    <td><b>99.2</b></td>
    <td><b>99.2</b></td>
    <td><b>99.2</b></td>
    <td><b>96.7</b></td>
    <td><b>97.5</b></td>
  </tr>
  <tr>
    <td rowspan="4">Qwen 2.5 Instruct 7B</td>
    <td>Zero-Shot</td>
    <td>91.7</td>
    <td>92.5</td>
    <td>85.8</td>
    <td>73.3</td>
    <td>47.5</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>90.8</td>
    <td>92.5</td>
    <td>81.7</td>
    <td>62.5</td>
    <td>36.7</td>
  </tr>
  <tr>
    <td>Zero-Shot CoT</td>
    <td>95.0</td>
    <td>93.3</td>
    <td>91.7</td>
    <td>79.2</td>
    <td>60.0</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>85.0</td>
    <td>89.2</td>
    <td>82.5</td>
    <td>65.8</td>
    <td>42.5</td>
  </tr>
  <tr>
    <td>Entrospect</td>
    <td><b>95.0</b></td>
    <td><b>95.8</b></td>
    <td><b>91.7</b></td>
    <td><b>84.2</b></td>
    <td><b>63.3</b></td>
  </tr>
  <tr>
    <td rowspan="4">Llama 3.1 Instruct 8B</td>
    <td>Zero-Shot</td>
    <td>87.5</td>
    <td>74.2</td>
    <td>60.8</td>
    <td>48.3</td>
    <td>37.5</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>88.3</td>
    <td>70.0</td>
    <td>62.5</td>
    <td>41.7</td>
    <td>20.0</td>
  </tr>
  <tr>
    <td>Zero-Shot CoT</td>
    <td>91.7</td>
    <td>83.3</td>
    <td>77.5</td>
    <td>65.8</td>
    <td>50.0</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>72.5</td>
    <td>54.2</td>
    <td>43.3</td>
    <td>28.3</td>
    <td>23.3</td>
  </tr>
  <tr>
    <td>Entrospect</td>
    <td><b>95.0</b></td>
    <td><b>88.3</b></td>
    <td><b>84.2</b></td>
    <td><b>74.2</b></td>
    <td><b>60.8</b></td>
  </tr>
  <tr>
    <td rowspan="4">GLM 4 Instruct 9B</td>
    <td>Zero-Shot</td>
    <td>82.5</td>
    <td>66.7</td>
    <td>55.0</td>
    <td>46.7</td>
    <td>24.2</td>
  </tr>
  <tr>
    <td>5-Shot</td>
    <td>86.7</td>
    <td>66.7</td>
    <td>66.7</td>
    <td>44.2</td>
    <td>25.0</td>
  </tr>
  <tr>
    <td>Zero-Shot CoT</td>
    <td>90.0</td>
    <td>81.7</td>
    <td>75.8</td>
    <td>51.7</td>
    <td>30.0</td>
  </tr>
  <tr>
    <td>Self-Refine</td>
    <td>85.0</td>
    <td>69.2</td>
    <td>63.3</td>
    <td>45.0</td>
    <td>21.7</td>
  </tr>
  <tr>
    <td>Entrospect</td>
    <td><b>92.5</b></td>
    <td><b>85.8</b></td>
    <td><b>79.2</b></td>
    <td><b>56.7</b></td>
    <td><b>34.2</b></td>
  </tr>
</table>

Table 4: To further demonstrate the effectiveness of Entrospect, an additional evaluation was conducted using the GPQA dataset, which is challenging. A random sample of 100 GPQA pairs was selected for this purpose. The table below presents the number of correct answers obtained by each method, providing a clear comparison of their performance.

<table>
  <tr>
    <th>Model</th>
    <th>Entrospect</th>
    <th>Zero-Shot CoT</th>
    <th>Self-Refine</th>
  </tr>
  <tr>
    <td>DeepSeek-R1-Distilled-Qwen2.5-1.5B</td>
    <td><b>17</b></td>
    <td>15</td>
    <td>0</td>
  </tr>
  <tr>
    <td>Qwen2.5-7B</td>
    <td><b>24</b></td>
    <td>11</td>
    <td>7</td>
  </tr>
  <tr>
    <td>Llama3.1-8B</td>
    <td><b>22</b></td>
    <td>12</td>
    <td>10</td>
  </tr>
  <tr>
    <td>GLM4-9B</td>
    <td><b>14</b></td>
    <td>5</td>
    <td>7</td>
  </tr>
</table>
Figure 7: (Referred in Section 4.5) We employed linear regression to model the decline in reasoning accuracy, as measured by Entrospect and Self-Refine on the MATH validation set with increasing difficulty levels. The four charts correspond to the four distinct SLMs we evaluated, where the decay rate equals the slope of each fitted decay line. A decay rate with a larger absolute value indicates a more rapid deterioration in reasoning accuracy as the difficulty level rises. Across all tested models, observations indicate that as the difficulty level of the test data increases, the performance degradation exhibited by Entrospect is, overall, less pronounced than that of Self-Refine.

Reviewer paper summaries: Hallucination is an important failure case for LLMs. Self-reflection has been identified as one potential good approach to reducing hallucinations, yet it remains ineffective for small language models (<10 billion parameters). The authors propose an improvement to standard self-reflection (Self-Refine) called Entrospect, which outperforms zero-shot, few-shot, zero-shot CoT, and Self-Refine on the MATH and HaluEval dataset across a variety of models and model sizes. The Entrospect framework selects good feedback using the Optimal Revision Suggestion Selection (ORSS). ORSS uses the key observation that we can select good revision options by maximizing the conditional mutual information between the model’s refined response, $A_{t+1}$, and suggested feedback, $f_t$. The reflection process terminates when the model’s newly refined answer is sufficiently similar to its previous answer.

This paper provides an improved method for model refinement by introducing Entrospect. The main focus on the method lies in introducing a selection phase after obtaining potential refined prompts. Compared to previous works such as Self Refine, Entrospect shows an improvement of up to 36.2% on the MATH and HaluEval datasets – demonstrating the effectiveness of the proposed research method. The methods in the paper are well-supported by quantitative thresholds used to distinguish results, minimising subjectivity. However, some concerns linger about technical details and possible findings that could be extrapolated from this experiment.

The authors present Entrospect, a framework designed to enhance self-reflection in Small Language Models by refining their responses through an iterative process. Building on the Self-Refine approach, Entrospect operates in four stages: generating an initial response, identifying weaknesses, selecting the best revision strategy, and refining the output accordingly. A key component of this method is the Optimal Revision Suggestion Selector, which filters revision options to ensure the most effective improvements while minimizing uncertainty. The framework functions without additional parameters, iterating until responses stabilize. To determine when to stop, it uses a semantic similarity measure based on cosine similarity, ending the process when further refinements yield negligible changes. The approach is evaluated using accuracy, which measures the proportion of correctly resolved problems, and average time consumption, which tracks how long the refinement process takes. Their results indicate that Entrospect outperforms existing approaches such as zero-shot, 5-shot, zero-shot CoT, and Self-Refine.
Summary of strengths: - I think the paper does a good job of formally breaking down the problem of prompt selection and identifying the individual sources of error.  
- I think choosing good feedback to focus on by maximizing the mutual information between the feedback and the model’s output is an interesting and clever approach.

- The general thought process behind the Optimal Revision Suggestion Selector is very well supported. The authors walk us through the entire process and clearly define their guidelines for which a prompt must be selected. Through the use of standard methods such as CMI, the ORSS becomes a well-supported component of the framework of Entrospect. Furthermore, the authors recognize its current limitations and next steps to theoretically define what the ORSS should choose as the optimal prompt.
- Throughout the paper, data is well-supported by quantitative thresholds. An example of this can be seen in lines 352-356, where the authors clearly define what is considered to be a high degree of semantic resemblance. This removes subjectivity from the research process, and is highly encouraged.

The authors clearly define the problem of reflective contamination in Small Language Models, explaining how their limited semantic capabilities lead to unreliable self-reflection and degrade response refinement. They formalize this issue mathematically, illustrating how biases in self-generated feedback negatively impact results, providing a solid foundation for their proposed solution. The Entrospect framework offers a novel approach to improving self-reflection in SLMs. By integrating the Optimal Revision Suggestion Selector, it selectively applies the most effective revision candidates rather than treating all suggestions equally, as previous methods like Self-Refine do. This information-theoretic approach leads to notable improvements in reasoning accuracy and efficiency, demonstrating its potential impact. The evaluation of Entrospect is thorough, with rigorous testing across multiple SLMs and datasets, including MATH for mathematical reasoning and HaluEval for hallucination detection. Accuracy and computational efficiency are insightful metrics that capture concerns around the method, and the baseline comparisons and ablation studies further validate the framework’s effectiveness.
Summary of weaknesses: - I believe the introduction or related works section should mention alternative approaches to self-reflection for reducing hallucinations and generally improving performance. For example, automated prompt engineering, retrieval (ex., RAG), ensemble methods (ex., mixture of experts, majority voting), etc.  
- I believe the related works section is missing follow-up work to Self-Refine. For example, other self-reflective methods have been proposed for selecting the best feedback to include, such as debate and majority voting.  
- HaluEval benchmark seems to be too easy. For 3/4 of the tested models, zero-shot prompting achieves \>=94.5% accuracy. Therefore, I believe the results from this benchmark are less significant.  
- In table 1 of Madaan et al., 2023 ([https://arxiv.org/pdf/2303.17651](https://arxiv.org/pdf/2303.17651)), they show that Self-Refine improves performance significantly in all settings other than math reasoning where performance is unchanged. In table 1, of your paper, we see a similar trend that Self-Refine does not seem to improve performance relative to the baseline for the Math dataset. Thus, your results show that Entrospect improves performance in a situation where Self-Refine has known poor performance, which is a good result. However, it does not convince be that Entrospect outperforms Self-Refine in general. Specifically, I think claiming a “state-of-the-art” performance (line 23 and 471\) is not justified without further experimentation.  
- Baseline results don’t seem to match the literature. In Table 1, zero-shot performance seems to be very high. It performs better than 5-shot 7/8 of the tested setups despite few-shot prompting being known to improve performance. One example is Brown et al., 2020 ([https://arxiv.org/pdf/2005.14165](https://arxiv.org/pdf/2005.14165)).  
- LLM results vary significantly depending on which prompt is used Zhou et al., 2022 ([https://arxiv.org/abs/2211.01910](https://arxiv.org/abs/2211.01910)). Therefore, for reproducibility, I request that all prompts for all evaluated approaches be released.  
- The choice of in-context examples can have a significant impact on the performance of few-shot prompting Zhang et al., 2022 ([https://arxiv.org/pdf/2210.03493](https://arxiv.org/pdf/2210.03493)). However, I could not find a section which describes how the 5 in-context examples are selected for the few-shot baseline. I believe this information is important for reproducibility, and I request that it be added.

- In Lines 265-268, the paper mentions a shift from manual prompt pools to feedback being constructed in an automatic manner before the selection process. Is there an evaluation metric that justifies this transition from previous setups? Are there any limitations or trade-offs involved with this shift? More clarity could be added to highlight any changes this approach might create compared to previous baselines.
- In Lines 506-514, the authors mention that no statistically significant relationship was found between model parameter sizes and Entrospect results. However throughout the models, Entrospect’s performance on the MATH dataset drops as the model parameter size increases. This phenomenon could be further acknowledged in the paper, considering the baseline being compared against (Self Refine) shows an improvement in mathematical reasoning between Llama 3.1 Instruct 8B and GLM 4 Instruct 9B, whereas Entrospect shows a distinct decline as the number of model parameters increase.

Entrospect’s semantic similarity-based stopping condition, crucial for optimizing refinement, lacks thorough validation. The authors' ablation study mainly focuses on accuracy and offers limited exploration of the stopping condition’s performance across varying similarity thresholds, text embeddings, tasks, and datasets. A deeper evaluation, including an analysis of convergence behavior, would strengthen claims of its robustness and generalizability. The paper also omits a broader comparison to other self-reflection frameworks beyond prompt-based methods and Self-Refine. While CoT prompting is mentioned, there is no in-depth evaluation against other techniques, making it difficult to assess Entrospect’s novelty and impact. A broader comparison would better contextualize its contributions. Lastly, while Average Time Consumption (ATC) measures efficiency, it does not fully address computational costs, especially in API-based settings where each model call incurs expenses. The number of iterations, a key factor for both industrial and academic use, directly impacts these costs.
Comments, suggestions and typos: Comments:
- I found section 3.1 difficult to follow. I think defining the notation a little more clearly could help.  
- I think the strength of the paper can be significantly increased with more thorough experimentation

Questions:
- I don’t understand how $M\_{\\theta}(Q||A\_t||P\_{present})$ is defined. In the text it is written as the current “feedback” provided by the model. However, in equation 1, it is written as $A\_t$ (the model’s current response) \+ $\\alpha\_t^S S\_t$ \- $\\alpha\_t^N N\_t$, where $\\alpha\_t^S, \\alpha\_t^N \\in (0, 1)$, $S\_t$ is the “valid portion,” and $N\_t$ is the “reflective contamination.” From what I understand $A\_t, S\_t, N\_t$ are all strings and $\\alpha\_t^S, \\alpha\_t^N$ are real numbers. How can you multiply and add/subtract these expressions together? Are $A\_t, S\_t, N\_t$ not strings and instead probabilities? Even then, I don’t see how this constitutes a valid probability calculation.  
- I don’t understand your mutual information calculation. In equation 3, you are using $I(A\_{t+1}; f\_t | Q||A\_t)$, but in lines 8-10 of algorithm 1, your calculations do not include $A\_{t+1}$ (because it doesn’t exist yet). So, how were you able to avoid requiring $A\_{t+1}$ in the calculations of algorithm 1?

- The Caption for Figure 3 could have its colours removed, it slightly erodes clarity as is.
- Does the formula for cosine similarity need to be expanded in the paper?
- 4-bit models can lose quite a bit of precision when downgraded, have you considered using 16-bit or 32-bit when evaluating Entrospect on SLMs?

- It would be beneficial to introduce Self-Refine for readers before discussing how your work builds on it in the introduction.
- Include a discussion on the alignment (or lack thereof) between the number of iterations, the task, and the ATC.
- Note when figures/tables are in the appendix.
- On line 494, remove the period.
Overall assessment scores: 3.5

3

4