Integrative Decoding: Improving Factuality via Implicit Self-consistency

## Abstract

Abstract Self-consistency-based approaches, which involve repeatedly sampling multiple outputs and selecting the most consistent one as the final response, prove to be remarkably effective in improving the factual accuracy of large language models. Nonetheless, existing methods usually have strict constraints on the task format, largely limiting their applicability.
In this paper, we present Integrative Decoding (ID), to unlock the potential of self-consistency in open-ended generation tasks. ID operates by constructing a set of inputs, each prepended with a previously sampled response, and then processes them concurrently, with the next token being selected through aggregation of all their corresponding predictions at each decoding step. In essence, this simple approach implicitly incorporates self-consistency in the decoding objective. Extensive evaluation shows that ID consistently enhances factuality over a wide range of language models, with substantial improvements on the TruthfulQA (+11.2%), Biographies (+15.4%) and LongFact (+8.5%) benchmarks. The performance gains amplify progressively as the number of sampled responses increases, indicating the potential of ID to scale up with repeated sampling. 1 1 1 All codes and data are available at https://github.com/YiCheng98/IntegrativeDecoding .

## 1 Introduction

Despite notable advancements across various domains, Large Language Models (LLMs) remain notorious for their tendency to produce non-factual and erroneous content, a phenomenon commonly known as hallucinations (Lewis et al., ; Ji et al., ). Prior research has shown that “repeated sampling” is a very effective methodology for enhancing factual accuracy (Wang et al., ; Shi et al., ; Chen et al., ).
It involves sampling multiple responses to the same prompt, followed by a careful selection of the most accurate one or the synthesis of a refined output from the sampled responses. Notably, as the number of sampled responses increases, its performance gains often continue to rise in an almost log-linear manner, as recently highlighted by Brown et al. (). This suggests the existence of “inference-time scaling laws,” implying the potential of repeated sampling to progressively push the model closer to its theoretical performance ceilings. Despite this immense promise, a central challenge in this methodology remains: how to effectively identify the non-factual content within the sample collection and thereby produce a final, accurate output.

The degree of “*self-consistency*” (SC), which measures the consistency level among LLMs’ different outputs, has proven to be a useful indicator to address this issue (Wang et al., ; Shi et al., ; Chen et al., ; Thirukovalluru et al., ; Malon & Zhu, ; Mündler et al., ; Manakul et al., ).
It has been observed that statements consistently present across a range of sampled responses are more likely to be truthful, as opposed to those appearing sporadically or inconsistently across outputs.
However, most SC-based methods for improving factuality impose strict constraints on the format of task output, largely limiting their applicability.
Due to the difficulty in measuring consistency across responses, previous studies usually only consider tasks where they can easily define consistency as the exact matches between the answers parsed from the responses (Wang et al., ; Huang et al., ; Shi et al., ; Li et al., ), such as arithmetic problems and multiple choice question. This naturally leads us to ask: *how can we further unlock the potential of self-consistency and repeated sampling in open-ended generation tasks*?

**Table 1: Comparisons between ID and other training-free approaches for improving factuality. “Context length” indicates length relative to standard greedy decoding (with $k$ representing the number of sampled responses). The performance of S-Contra is excluded, as it lacks large-scale evaluation due to the high cost associated with numerous prompting for checking self-consistency.**
| Method | How to Check<br>Self-consistency | Open-ended<br>Tasks | Context<br>Length | Performance<br>Gain | Enhance with<br>Repeated Sampling |
| --- | --- | --- | --- | --- | --- |
| SC (Wang et al., ) | Exact Match | ✗ | $\times$1 | - | ✓ |
| DoLa (Chuang et al., ) | - | ✓ | $\times$1 | Low | ✗ |
| USC (Chen et al., ) | Prompting LLM | ✓ | $\times$k | Medium | Unstable |
| SR (Madaan et al., ) | Prompting LLM | ✓ | $\times$k | Medium | Unstable |
| S-Contra (Mündler et al., ) | Numerous Prompting | ✓ | $\times$1 | - | - |
| Integrative Decoding | ICL & Decoding-time<br>Implicit Integration | ✓ | $\times$2 | High | ✓ |

One straightforward way is to concatenate all sampled responses in a prompt and directly instruct the LLM to select the most self-consistent one from them, as done in Chen et al. (). Nonetheless, such practice substantially increases the input length, posing excessive demands on the model’s long-text processing capability.
In another line of work, Mündler et al. () treats each response as a collection of statements and iteratively prompts LLMs to compare pairs of facts across different responses to assess their consistency.
This requires numerous iterations of LLM inference, particularly for longer outputs, leading to inefficiencies.
Due to these issues, prior attempts to apply SC in open-ended tasks cannot generalize effectively to long-form generations and they struggle to scale up with an increasing number of sampled responses.

In this paper, we present *Integrative Decoding* (ID), a novel decoding strategy designed to improve factuality by implicitly incorporating self-consistency within its decoding objective. ID begins by repeated sampling. For each sampled response in the collection, ID constructs a new input by concatenating the response with the original prompt. Essentially, this input instructs the model to respond to the instruction again with reference to a previously sampled response.
Then, ID processes these inputs concurrently for decoding, with the next token being selected by integrating all their predictions at each inference step.
During this process, each input acts like a “representative” for the sampled response within it, voting for the tokens that are semantically consistent with the response it represents.
ID effectively aggregates their votes and thereby achieves the optimal overall consistency across all sampled responses.
Compared with existing approaches, ID has no constraints on the task format and does not rely on additional prompting to explicitly verify self-consistency (see Table 1 for detailed comparisons).

We evaluate ID over six series of LLMs with varying scales. As shown in Figure 1, ID consistently enhances the factuality over all these LLMs by a large margin on the TruthfulQA (+11.2%), Biographies (+15.4%) and LongFact (+8.5%) datasets, demonstrating robustness from sentence- to document-level generations. Moreover, the performance gains of ID progressively amplify as the number of sampled responses increases, indicating its potential to scale up with repeated sampling.

## 2 Method

#### Preliminaries: Self-consistency as an Indicator for Factuality

Previous studies found that the degree of self-consistency between LLM’s different sampled responses can serve as a useful indicator for hallucination detection (Manakul et al., ; Farquhar et al., ). The facts that are consistently supported by LLMs’ different sampled responses are more likely to be factual, compared to those that only appear sporadically or inconsistently across multiple outputs.
Formally, given a prompt $\mathbf{x}$ and its response $\hat{\mathbf{y}}$ that consists of a series of statements $\mathcal{S}=\{s_{1},s_{2},..,s_{n}\}$, the factuality score of $s_{i}$ can be estimated by measuring its consistency with other sampled responses $\mathcal{R}=\{r_{1},r_{2},..,r_{k}\}$ in response to the same prompt $\mathbf{x}$ as:

$$ ${f}(s_{i})=\frac{1}{|\mathcal{R}|}\sum_{r_{j}\in R}P(\text{consistent}|s_{i},r _{j}),$ (1) $$

where ${f}(s_{i})$ refers to the estimated factuality score of the statement $s_{i}$ and $P(\text{consistent}|s_{i},r_{j})$ is the probability that $s_{i}$ is supported by the response $r_{j}$. These responses can be obtained through sampling algorithms, such as temperature sampling (Ficler & Goldberg, ) or nucleus sampling Holtzman et al. ().
The overall factuality score of the response $\hat{\mathbf{y}}$ can thereby be estimated as:

$$ ${F}(\hat{\mathbf{y}})=\frac{1}{|\mathcal{S}|\cdot|\mathcal{R}|}\sum_{s_{i}\in \mathcal{S}}\sum_{r_{j}\in R}P(\text{consistent}|s_{i},r_{j})=\frac{1}{| \mathcal{R}|}\sum_{r_{j}\in\mathcal{R}}\bar{f}(\hat{\mathbf{y}},r_{j}),$ (2) $$

where $\bar{f}(\hat{\mathbf{y}},r_{j})=\frac{1}{|\mathcal{S}|}\sum_{s_{i}\in\mathcal{
S}}P(\text{consistent}|s_{i},r_{j})$, representing the overall degree of $\hat{\mathbf{y}}$ being supported by the response $r_{j}$.

#### Formalization of Decoding Objective

The established insights about the role of self-consistency in hallucination detection indicate that the response most consistent with the others tends to be the most factual one.
This motivates us to develop a decoding method that, given several sampled responses, can generate a new output, maintaining strong overall consistency with all of them while maintaining its own coherence.
Formally, given an input prompt $\mathbf{x}$, a decoding method searches for an output $\hat{\mathbf{y}}$ by solving:

$$ $\hat{\mathbf{y}}=\operatorname*{arg\,max}_{\mathbf{y}\in{\mathcal{Y}}}H( \mathbf{x},\mathbf{y}),$ (3) $$

where $\mathcal{Y}$ refers to the set of all possible token sequences and $H(\mathbf{x},\mathbf{y})$ is the objective function.

Common decoding algorithms, such as beam search, consider the decoding objective $H(\mathbf{x},\mathbf{y})$ as $\log p_{\theta}(\mathbf{y}|\mathbf{x})=\sum_{t=1}^{|\mathbf{y}|}\log p_{\theta
}(y_{t}|y_{<t},\mathbf{x})$, where $\theta$ refers to the model’s parameters and $p_{\theta}(y_{t}|y_{<t},\mathbf{x})$ represents its predicted token probability distribution at the $t$-th decoding step. Note that we omit the input prompt $\mathbf{x}$ here and in the following to reduce clutter.

The objective of our method, by contrast, is composed of two parts:
$H(\mathbf{x},\mathbf{y})=F(\mathbf{y})+\lambda\cdot G(\mathbf{x},\mathbf{y})$, where $\lambda$ is a constant weight. $G(\mathbf{x},\mathbf{y})$ can be viewed as the common decoding objective, which measures whether the concatenation of $\mathbf{x}$ and $\mathbf{y}$ is a coherent and contextually appropriate text.
$F(\mathbf{y})$ is used to measure truthfulness of $\hat{\mathbf{y}}$, which additionally emphasizes factuality in the decoding objective.
Then, we adapt this objective function by replacing $F(\mathbf{y})$ based on Equation 2:

$$ $\displaystyle H(\mathbf{y})$ $\displaystyle=\sum_{r_{j}\in R}[\bar{f}({\mathbf{y}},r_{j})+\alpha\cdot G( \mathbf{x},\mathbf{y})],$ (4) $$

where $\mathcal{R}$ is a set of sampled responses to the prompt $\mathbf{x}$
and $\alpha$ is a constant term.

Figure: Figure 2: The workflow of integrative decoding: (1) sample multiple responses from the LLM; (2) form a set of new inputs by concatenating a sampled response and the original prompt; they are concurrently processed for decoding, with the next token being selected by integrating their predicted logits at each inference step. This strategy essentially incorporates the overall consistency with all sampled responses in its decoding objective (see Section 2).
Refer to caption: x2.png

#### Integrative Decoding

However, computing Equation 4 directly poses significant challenges, especially for the part of $\bar{f}(\mathbf{y},r_{j})$. Previous studies typically rely on LLMs to ascertain whether the statements in $\mathbf{y}$ are supported by $r_{j}$ (Mündler et al., ; Manakul et al., ). This process is not only computationally expensive, but also requires sophisticated prompt design to comprehensively measure $\bar{f}(\mathbf{y},r_{j})$.

To address this, our method incorporates an estimation of Equation 4 as follows.
Crucially, the part of $\bar{f}(\hat{\mathbf{y}},r_{j})+\alpha\cdot G(\mathbf{x},\mathbf{y})$ in Equation 4 is approximated as the LLM’s predicted probability for the output sequence when instructed to *respond to $\mathbf{x}$ again with reference to a previously sampled response $r_{j}$*.
Specifically, this involves constructing a new input $q_{j}$, which is sequentially structured as $[x;r_{j};x]$.(^2^22Note that, in practice, $q_{j}$ is not a strict concatenation of $\mathbf{x}$, $r_{j}$, and $\mathbf{x}$. Additional clarifying instructions, such as “answer this question again”, need to be inserted after $r_{j}$ to avoid confusion. We omit these details in the representation of $q_{j}$ here to reduce clutter.)
Formally, we assume that:

$$ $\log p_{\theta}(\mathbf{y}|[x;r_{j};x])\propto\bar{f}(\mathbf{y},r_{j})+\alpha \cdot G(\mathbf{x},\mathbf{y}).$ (5) $$

This assumption is reasonable because when $q_{j}$ serves as the input, the LLM’s in-context learning abilities naturally incline it to produce content consistent with $r_{j}$ within the input, thus promoting $\bar{f}(\mathbf{y},r_{j})$.
Concurrently, the LLM also ensures that the combination $\mathbf{x}\circ\mathbf{y}$ remains coherent and contextually appropriate, enhancing $G(\mathbf{x},\mathbf{y})$. In other words, the LLM tends to choose the output that is not only consistent with $r_{j}$ but also maintains its own coherence. This supports the validity of Equation 5 as a plausible assumption.

Then, we replace Equation 4 with:

$$ $H(y)=\sum_{r_{j}\in R}\log p_{\theta}(\mathbf{y}|[\mathbf{x};r_{j};\mathbf{x}]).$ (6) $$

which ideally should be computed as:

$$ $H(\mathbf{y})=\sum_{r_{j}\in R}\sum_{t=1}^{|\mathbf{y}|}\log p_{\theta}(y_{t}| y_{<t},[\mathbf{x};r_{j};\mathbf{x}]),$ (7) $$

Nonetheless, due to the prohibitively large searching space for $y\in\mathcal{Y}$, it is extremely difficult to compute Equation 7. To enhance computational efficiency, we adopt the strategy commonly used in greedy algorithms by making locally optimal decisions at each decoding step. Specifically, at the $t$-th decoding step, we choose the next token $\hat{y}_{t}$ by:

$$ $\hat{y}_{t}=\operatorname*{arg\,max}_{y_{t}\in\mathcal{V}}\sum_{r_{j}\in R} \log p_{\theta}(y_{t}|y_{<t},[\mathbf{x};r_{j};\mathbf{x}]).$ (8) $$

Based on the above analysis, we can summarize the workflow to produce the result $\hat{\mathbf{y}}$ as dipicted in Figure 2.
It begins by sampling multiple responses $\mathcal{R}=\{r_{1},r_{2},..,r_{k}\}$ and then constructing a set of new inputs $\mathcal{Q}=\{q_{1},q_{2},...,q_{k}\}$ to prompt the model respond to the orginal instruction again with reference to a previously sampled response.
Subsequently, these inputs are fed to the LLM, which can be processed in one batch concurrently. At the $t$-th decoding step, we integrate all predicted probability logits in this batch and select the next token as illustrated in Equation 8.
All sequences in the batch universally take the same next token and then continue the generation process. Consequently, all inputs in the batch result in the same output $\hat{\mathbf{y}}$, which is used as the final response to the prompt $\mathbf{x}$.

## 3 Experiments

### 3.1 Setup

#### Benchmarks and Evaluation Metrics

We conduct experiments on the following open-ended generation benchmarks:

- •
TruthfulQA (Lin et al., ) consists of 817 questions that many humans would answer falsely due to misconception.
We employ GPT-4 (Bubeck et al., ) to assess the truthfulness (*Truth*) and informativeness (*Info*) scores of each generated answer. The product of these two scores (*T*I*) is considered as the major metric on this benchmark.
During evaluation, the reference answers annotated in the dataset are included in the prompt as reference when using GPT-4 to assess truthfulness. The informativeness score assesses whether the response contains valid information that directly answers the question. GPT-4 is employed to evaluate this in a few-shot manner, using the evaluation samples provided by Lin et al. () as the demonstration examples.
- •
Biographies (Du et al., ) requires generating bullet point biographies for computer scientists, with a total of 250 samples. Specifically, we prompt the model to list 5 major achievements or contributions made by the scientist in question. Following Du et al. (), we use GPT-4 to assess the factuality of each bullet statement by referring to the related information extracted from Wikipedia. The proportion (% *Accuracy*) and the number (# *Correct*) of factual statements are adopted as the evaluation metrics. Note that % Accuracy is not simply # Correct divided by five since the model may occasionally generate fewer than five statements when it is uncertain.
- •
LongFact-Objects (Wei et al., ) requests detailed descriptions for a queried object and expects a document-level response that is typically very long, often exceeding a thousand tokens (see Appendix F for detailed examples). The evaluation process is similar to the one described in Wei et al. (), which involves splitting the long response into a series of atomic facts and then assessing their truthfulness separately. We employ LLaMA3.1-70B-Instruct to divide atomic facts and use GPT-4 to assess whether each fact is truthful. The adopted metrics include the proportion of truthful facts (*Precision*), the number of truthful facts divided by 128 (*Recall@128*), and the *F1@128* score that integrates the previous two metrics. 120 samples are used for evaluation. Evaluation results of recall and F1 metrics at other intervals are provided in Appendix C.1.

Notably, the response lengths on the three benchmarks span sentence-level, paragraph-level, and document-level, respectively, reflecting progressively greater challenges in enhancing factuality.

#### Compared Methods

We compare our method with *greedy decoding* (Greedy) and *decoding by contrasting layers* (Chuang et al., , DoLa). In addtion, we also compare it with two other types of ensemble-based methods that involves repeated sampling to produce a refined result, including *Universal Self-Consistency* (Chen et al., , USC) and *Self-reflection* (Madaan et al., , SR).
USC concatenates the sampled responses in one prompt and directly instructs the LLM to select the most consistent one from them.
SR also concatenates the sampled responses as an input, and asks the model to reflect on them and extract the factual information in them to produce a new response.

#### Base Models

Our main experiments are conducted on the following LLMs: LLaMA-2-7B-chat (Touvron et al., ), LLaMA-3-8B-Instruct (Dubey et al., ), Mistral-7B-Instruct-v0.2 (Jiang et al., ), Gemma-2-9B-it (Team et al., ), Qwen2-7B-Instruct (Yang et al., ), and GLM-4-9B-chat (GLM et al., ). We refer to them as LLaMA2, LLaMA3, Mistral2, Gemma2, Qwen2, GLM4 in the following.
To further evaluate the robustness of integrative decoding to different model scales, we also conduct experiments with Qwen-2.5-3B/7B/14B/32B/72B-Instruct (Team, ), LLaMA-2-13B/70B-chat (Touvron et al., ), and Mistral-Nemo/Small/Large-Instruct-2407/2409 (Team, ) in the additional analysis (Section 3.4).

#### Implementation Details

The detailed prompt templates used for different approaches are provided in Appendix E.
For USC, SR, and ID, we searched for the optimal number of sampled responses to integrate from $k=\{1,4,8,12,16\}$ using the validation sets and employ it for evaluation on the test sets.
We selected the optimal $k$ according to the %Truth score on TruthfulQA and the %Accuracy metric on Biographies. Due to high evaluation costs on LongFact, we did not conduct optimal $k$ searching on it. We directly set $k=16$ for ID. For USC and SR, we set $k=4$ because setting $k$ higher than 4 would exceed the context length limits of many LLMs.

**Table 2: Evaluation results on three open-ended benchmarks. Responses on TruthfulQA are brief *sentences*, Biographies are short *paragraphs*, and LongFact requires *document-level* responses. The three benchmarks pose increasing levels of difficulty for factuality enhancement.**
| Method | TruthfulQA | Biographies | LongFact |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| % Truth | % Info | % T*I | # Correct | % Acc. | Prec. | R@128 | F1@128 |  |  |
| LLaMA2 | Greedy | 50.7 | 96.3 | 48.9 | 0.81 | 16.2 | 88.1 | 75.6 | 80.5 |
| DoLA | 49.5 (-1.2) | 95.6 (-0.7) | 47.3 (-1.6) | 0.78 (-0.03) | 15.6 (-0.6) | 88.0 (-0.1) | 75.5 (-0.1) | 80.4 (-0.1) |  |
| USC | 46.3 (-4.4) | 96.1 (-0.2) | 44.5 (-4.4) | 0.84 (+0.03) | 16.7 (+0.5) | 86.5 (-1.6) | 72.1 (-3.5) | 77.6 (-2.9) |  |
| SR | 53.9 (+3.2) | 96.3 (+0.0) | 51.9 (+3.0) | 0.82 (+0.01) | 16.6 (+0.4) | 86.8 (-1.3) | 58.2 (-17.4) | 55.0 (-25.5) |  |
| ID | 55.9 (+5.2) | 99.0 (+2.7) | 55.3 (+6.4) | 0.87 (+0.06) | 17.3 (+1.1) | 89.0 (+0.9) | 77.5 (+1.9) | 82.1 (+1.6) |  |
| LLaMA3 | Greedy | 53.4 | 96.6 | 51.6 | 1.28 | 26.6 | 90.0 | 70.7 | 78.8 |
| DoLA | 54.1 (+0.7) | 97.6 (+1.0) | 52.8 (+1.2) | 1.30 (+0.02) | 27.1 (+0.5) | 90.3 (+0.3) | 70.5 (-0.2) | 78.8 (+0.0) |  |
| USC | 56.8 (+3.4) | 98.3 (+1.7) | 55.9 (+4.3) | 1.34 (+0.06) | 27.9 (+1.3) | 89.7 (-0.3) | 71.8 (+1.1) | 79.3 (+0.5) |  |
| SR | 57.8 (+4.4) | 97.1 (+0.5) | 56.1 (+4.5) | 1.62 (+0.34) | 34.0 (+7.4) | 89.4 (-0.6) | 46.1 (-24.6) | 58.6 (-20.2) |  |
| ID | 63.4 (+10.0) | 99.0 (+2.4) | 62.8 (+11.2) | 2.00 (+0.72) | 42.0 (+15.4) | 92.2 (+2.2) | 77.7 (+7.0) | 83.6 (+4.8) |  |
| Mistral2 | Greedy | 74.9 | 99.8 | 74.7 | 0.93 | 18.6 | 91.2 | 61.1 | 72.2 |
| DoLA | 74.4 (-0.5) | 99.8 (+0.0) | 74.2 (-0.5) | 0.94 (+0.01) | 18.8 (+0.2) | 91.2 (+0.0) | 61.0 (-0.1) | 72.1 (-0.1) |  |
| USC | 76.6 (+1.7) | 99.8 (+0.0) | 76.4 (+1.7) | 0.94 (+0.01) | 18.8 (+0.2) | 90.6 (-0.6) | 61.3 (+0.2) | 72.3 (+0.1) |  |
| SR | 78.0 (+3.1) | 99.5 (-0.3) | 77.7 (+3.0) | 0.97 (+0.04) | 19.8 (+1.2) | 91.2 (+0.0) | 63.0 (+1.9) | 73.0 (+0.8) |  |
| ID | 78.8 (+3.9) | 99.5 (-0.3) | 78.4 (+3.7) | 1.11 (+0.18) | 22.6 (+4.0) | 91.8 (+0.6) | 68.5 (+7.4) | 77.7 (+5.5) |  |
| Qwen2 | Greedy | 56.3 | 97.1 | 54.7 | 1.45 | 29.1 | 90.0 | 57.1 | 69.1 |
| DoLA | 56.1 (-0.2) | 96.6 (-0.5) | 54.2 (-0.5) | 1.46 (+0.01) | 29.2 (+0.1) | 89.5 (-0.5) | 56.6 (-0.5) | 68.7 (-0.4) |  |
| USC | 58.3 (+2.0) | 97.6 (+0.5) | 56.9 (+2.2) | 1.44 (-0.01) | 28.8 (-0.3) | 87.9 (-2.1) | 57.3 (+0.2) | 68.7 (-0.4) |  |
| SR | 59.8 (+3.5) | 97.6 (+0.5) | 58.3 (+3.6) | 1.42 (-0.03) | 28.6 (-0.5) | 85.0 (-5.0) | 45.8 (-11.3) | 57.5 (-11.6) |  |
| ID | 60.0 (+3.7) | 99.0 (+1.9) | 59.4 (+4.7) | 1.74 (+0.29) | 35.5 (+6.4) | 91.7 (+1.7) | 64.2 (+7.1) | 74.8 (+5.7) |  |
| Gemma2 | Greedy | 68.1 | 98.5 | 67.1 | 1.80 | 37.2 | 95.7 | 58.3 | 71.9 |
| DoLA | 68.1 (+0.0) | 98.8 (+0.3) | 67.2 (+0.1) | 1.74 (-0.06) | 35.9 (-1.3) | 96.1 (+0.4) | 59.0 (+0.7) | 72.5 (+0.6) |  |
| USC | 71.0 (+2.9) | 98.5 (+0.0) | 69.9 (+2.8) | 2.08 (+0.28) | 42.2 (+5.0) | 95.6 (-0.1) | 58.7 (+0.4) | 72.1 (+0.2) |  |
| SR | 64.2 (-3.9) | 98.8 (+0.3) | 63.4 (-3.7) | 1.80 (+0.00) | 38.9 (+1.7) | 96.0 (+0.3) | 42.2 (-16.1) | 57.3 (-14.6) |  |
| ID | 77.1 (+9.0) | 99.0 (+0.5) | 76.3 (+9.2) | 2.52 (+0.72) | 52.4 (+15.2) | 97.1 (+1.4) | 69.7 (+11.4) | 80.4 (+8.5) |  |
| GLM4 | Greedy | 58.5 | 97.8 | 57.2 | 1.44 | 28.7 | 87.2 | 62.7 | 72.5 |
| DoLA | 59.0 (+0.5) | 97.6 (-0.2) | 57.6 (+0.4) | 1.41 (-0.03) | 28.3 (-0.4) | 86.9 (-0.3) | 61.6 (-1.1) | 71.7 (-0.8) |  |
| USC | 61.5 (+3.0) | 99.0 (+1.2) | 60.9 (+3.7) | 1.40 (-0.04) | 28.0 (-0.7) | 85.9 (-1.3) | 65.9 (+3.2) | 74.2 (+1.7) |  |
| SR | 63.4 (+4.9) | 98.1 (+0.3) | 62.2 (+5.0) | 1.34 (-0.10) | 27.5 (-1.2) | 88.7 (+1.5) | 36.8 (-25.9) | 49.9 (-22.6) |  |
| ID | 65.1 (+6.6) | 99.0 (+1.2) | 64.5 (+7.3) | 1.81 (+0.37) | 36.2 (+7.5) | 89.2 (+2.0) | 66.4 (+3.7) | 75.9 (+3.4) |  |

### 3.2 Main Results

The evaluation results are presented in Table 2, based on which we highlight the following findings:

Integrative decoding leads to substantial improvements in factuality across all six LLMs. As shown in Table 2, the absolute improvements on TruthfulQA, Biographies, and LongFact are 3.7-10%, 1.1-15.4%, and 1.6-8.5%, respectively (in terms of %Truth, %Accuracy, and F1@128). Among the six LLMs, the overall improvement is the most substantial over LLaMA3 and Gemma2. The improvement on LLaMA2, though evident, is the least among all six LLMs. This suggests that the effects of integrative decoding is more evident on stronger LLMs.

Integrative decoding achieves robust balance between factuality and informativeness. Across metrics that assess informativeness (i.e., % Info, # Correct, and Recall@128), integrative decoding also shows substantial improvement. This is particularly evident on the LongFact benchmark, which involves generating long documents, where the absolute improvement in Recall@128 reaches as high as 11.4%. This indicates that integrative decoding can elicit more parametric knowledge from the LLM while maintaining factual accuracy, rather than merely improving factuality simply by filtering out incorrect information. In contrast, the baseline methods, especially SR, struggle to achieve a robust balance between factuality and informativeness. For instance, while SR also improves the precision of GLM4 on LongFact, it results in a considerable drop of 25.9% in Recall@128.

Integrative decoding is robust to document-level generation tasks. Enhancing factuality on long-form generation tasks is very challenging and less explored. From Table 2, we can see that existing baseline approaches struggle with the LongFact benchmark, which demands document-level responses, often resulting in marked performance declines. Encouragingly, integrative decoding remains effective on LongFact, providing absolute improvements of up to 8.5%. This suggests that integrative decoding offers greater generality and robustness in long-form generation tasks.

Integrative decoding significantly outperforms other decoding-based and ensemble-based approaches.
The improvements achieved by DoLa is marginal on our experimental benchmarks, with an increase of no more than 0.7%. This suggest that the effectiveness of DoLa in enhancing factuality is limited in long-form, open-ended generation tasks.
While both USC and SR can improve factual accuracy in many cases, their enhancements are not robust. They fail to reliably enhance performance across different LLMs; for instance, USC causes significant performance degradation on LLaMA2, and SR does the same on Gemma2. Additionally, their effectiveness on the LongFact benchmark is marginal and sometimes leads to reduced performance.

### 3.3 Effects of Increasing the Number of Sampled Responses

We analyze the effects of increasing the number of sampled responses on the performance of SR, USC, and ID, as shown in Figure 3.

Figure: Figure 3: The performance of different approaches on the Biographies dataset over six LLMs, when the number of sampled responses is 1, 4, 8, 12, and 16, respectively.
Refer to caption: x3.png

The performance of integrative decoding progressively improves with more sampled responses across six LLMs. Even with only four sampled responses, ID consistently delivers noticeable performance gains. Figure 4 further explores the effects of incorporating more sampled responses when they are obtained via different sampling strategies. From Figure 3 and 4, we can observe a generally log-linear relationship between performance and the number of sampled responses. This trend mirrors findings from previous studies on the performance improvements observed in exact-match-based self-consistency approaches (Wang et al., ; Brown et al., ).

USC and SR fail to consistently improve with the increase in the number of sampled responses.
In many cases, particularly with less capable LLMs like LLaMA2, their performance even deteriorates.
We find that USC tends to directly choose the first sampled response appearing in their prompt as the final answer instead of adequately evaluating the consistency among all responses.
SR, likewise, struggles to distill factual information from multiple responses into a cohesive, high-quality final answer.
A significant factor contributing to this limitation is that they need to concatenate all sampled responses within a single prompt, which dramatically inflates the context length. This places an immense burden on the model’s long-text processing capabilities, making them hard to scale effectively with repeated sampling.
In contrast, integrative decoding only extends the input by the length of one sampled response, rendering it far more manageable for the model to process. This alleviates the challenges associated with context length saturation and reduces the cognitive load on the model, thereby enabling more stable and scalable performance.

### 3.4 Additional Analysis

Integrative decoding is robust to different sampling strategies.
We evaluate the robustness of ID when the sampled responses are obtained via different sampling strategies, including temperature sampling with $T\in\{0.3,0.5,0.7\}$ and nucleus sampling with $p\in\{0.9,0.95\}$. As shown in Figure 4, ID robustly improves the performance across all sampled responses. Such performance growth is slightly more significant in nucleus sampling compared to temperature sampling, but the difference is modest and lacks consistency.

Figure: Figure 4: The performance of integrative decoding, with sampled responses obtained via different sampling strategies, on the Biographies dataset. The strategies examined include temperature sampling with $T\in\{0.3,0.5,0.7\}$ and nucleus sampling with $p\in\{0.9,0.95\}$.
Refer to caption: x4.png

Integrative decoding is robust to varying model scales and exhibits increasingly pronounced effects at larger scales.
We further analyze the performance of ID over LLMs with varying model scales, including Qwen-2.5-3B/7B/14B/32B/72B-Instruct, LLaMA-2-13B/70B-chat, Mistral-Nemo/Large-Instruct-2407, and Mistral-Small-Instruct-2409.
As shown in Figure 5, ID consistently leads to substantial improvements over different model scales.
Moreover, we find that the performance gains become more significant at larger model scales.

Figure: Figure 5: The performance of integrative decoding on LLMs with varying model scales on the Biographies dataset.
Refer to caption: x5.png

### 3.5 Case Study

Integrative decoding maintains self-consistency at semantic level.
To further illustrate the mechanism of ID, we present a case study in Table 3.5. In this example, three out of the five sentences produced by greedy decoding exhibit hallucination. In comparison, while the four sampled responses also contain non-factual information (see Appendix F.2 for their complete content), ID is able to capture the content that consistently present across them and eliminate sporadic hallucinations, ultimately yielding a fully factual and coherent output. It is crucial to note that, though many statements in the ID’s output share the same underlying meanings as those in sampled responses, they differ in their surface-level expression. This indicates that ID can maintain self-consistency at semantic level, rather than merely replicating the content in the sampled responses. ID achieves such effects by allowing each input it integrates to act like a “representative” for a sampled response. Leveraging the in-context learning capability, each input assigns high logits to all tokens that are semantically consistent with the sampled response it represents, instead of confining its choices to tokens directly appearing in it. This allows ID to maintain a high level of self-consistency at semantic level.

Through this example, we can also see the advantages of ID over USC and SR more clearly. Since all sampled responses contain hallucinations, it is impossible for USC, which operates by selecting one of them as the final output, to produce a fully factual result as ID does. Similarly, SR also faces significant challenges in discerning factual elements within the concatenation of sampled responses. This makes ID a superior choice for improving factuality for LLM outputs.

**Table 3: A case study that compares Integrative Decoding (ID) and greedy decoding, using the prompt “list five major achievements or contributions made by David Parnas.” The hallucinations in their outputs are marked with ✗. In this case, ID utilizes the four sampled responses listed above to generate a refined output. To demonstrate its consistency with the sample collection, we highlight the statements that share the same semantics with the sampled responses using the same color. The base model is Qwen-7B-Instruct. Some parts of the sampled responses are omitted due to space limit (see Appendix F.2 for their complete content).**
| Sampled Response #1:<br>David Parnas is credited with the concept of “information hiding”, which emphasizes structuring software into manageable, self-contained modules… His work on software architecture principles, including the “Parnas Principle”, which advocates for the separation of concerns, has had a lasting impact on the design and development of complex software systems. |
| --- |
| \cdashline<br>1-1[0.8pt/2pt]<br>Sampled Response #2:<br>David Parnas introduced the principle that “information hiding” is a key concept in software design, advocating that … He contributed significantly to the concept of modular programming, emphasizing that … Parnas was instrumental in the development of software architecture theory, focusing on the design of systems that are understandable, scalable, and adaptable … |
| \cdashline<br>1-1[0.8pt/2pt]<br>Sampled Response #3:<br>David Parnas introduced the concept of “information hiding” in software engineering, … Parnas was a pioneer in the field of software architecture, advocating for the importance of understanding and managing the structure and organization of complex software systems. He contributed to the development of the “Parnas Principle,” which suggests that a program should be understandable by a single individual … |
| \cdashline<br>1-1[0.8pt/2pt]<br>Sampled Response #4:<br>Parnas advocated for modular programming, emphasizing that … Parnas contributed to the field of software architecture, emphasizing the importance of architecture in … Parnas recognized the importance of human factors in software engineering, advocating for the integration of psychological and sociological aspects into the design and development processes of software systems. |
| \cdashline<br>1-1[0.8pt/2pt]<br>Integrative Decoding:<br>David Parnas introduced the concept of “information hiding” in software engineering, which is crucial for managing complexity in software systems (✓). He contributed to the development of the “Parnas Principle,” advocating for the separation of concerns in software design to enhance maintainability and understandability (✓). Parnas was instrumental in the advancement of software architecture theory, emphasizing the importance of clear structure and organization in software systems (✓). He advocated “modular programming,” which involves structuring software into independent, interchangeable modules to simplify development and maintenance (✓). His works on “software understanding” and “software comprehension” has been foundational in the field, focusing on the human aspects of software development and maintenance (✓). |
| Greedy Decoding:<br>David Parnas introduced the concept of “information hiding” in software engineering, which is fundamental to object-oriented programming (✓). He developed the ‘causality criterion” for software modularity, influencing how software is designed and structured (✗). Parnas was instrumental in the development of the “slicing” technique for program comprehension and debugging (✗). He contributed to the understanding of “information flow” in security and confidentiality within computer systems (✓). His work on “degradation” and “amortization” has been pivotal in the field of software maintenance and evolution (✗). |

## 4 Related Works

Though LLMs have exhibited remarkable proficiency in solving a wide range of tasks, they remain notorious for generating statements that appear plausible but are inconsistent with real-world facts, a phenomenon commonly known as hallucinations (Huang et al., ; Bai et al., ).
It has been observed that LLMs tend to inaccurately assess their own knowledge boundaries  (Yin et al., ) and often exhibit overconfidence in their responses (Xiong et al., ).
Zhang et al. () recently found that the data imbalances present in the pretraining data could be a reason why LLMs tend to overgeneralize well-known information and generate mixed factual inaccuracies.
Many studies have explored effective ways for hallucination detection  (Azaria & Mitchell, ; Simhi et al., ; Burns et al., ; Zhang et al., ; Chen et al., ; Farquhar et al., ) and
improving factuality in LLM outputs (Lee et al., ; Chen et al., ; Zhou et al., ; Elaraby et al., ; Schulman et al., ; Ouyang et al., ; Bai et al., ; Achiam et al., ).

Among these efforts, self-consistency-driven approaches have proved to be very effective in improving factuality (Wang et al., ; Shi et al., ; Chen et al., ; Thirukovalluru et al., ; Malon & Zhu, ; Mündler et al., ).
As one of the earliest attempts,  Wang et al. () prompt the model to generate a diverse set of intermediate reasoning paths for a given instruction, parse their final answers, and select the most consistent one as the optimal solution.
However, most of these works pose strict constraints on the task format, they only consider tasks, where the answers can be directly verified via exact matches and the most consistent answers can be easily selected as the most frequently appearing one (Li et al., ; Shi et al., ; Wang et al., ; Huang et al., ).
To overcome this limitation, research efforts (Chen et al., ; Thirukovalluru et al., ; Malon & Zhu, ; Mündler et al., ) have been directed towards adapting self-consistency for open-ended tasks without constraints on the task format.
USC (Chen et al., ) concatenates multiple candidate outputs and directly prompts the LLM to select the most consistent answer.
Thirukovalluru et al. () split initial sampled responses into lists of atomic facts and removing those facts appear infrequently across samples through clustering algorithms, thereby enhancing the factual consistency of the generated text.

Another line of research that is closely related to this study is exploration of decoding-based approaches for improving factuality.
Several studies (Burns et al., ; Li et al., ; Chuang et al., ; ) propose inference-time decoding strategies for trained LLMs, leveraging the parametric knowledge within their internal representations to mitigate hallucinations.
Chuang et al. () propose to decode outputs by comparing the differences in logits between the projections of later and earlier layers to better surface factual knowledge and reduce the generation of incorrect facts.
Burns et al. () introduce a consistency-based search algorithm to identify a direction in the activation space of LLMs that remains consistent across negations, thereby reducing generated errors.
O’Brien & Lewis () propose contrastive decoding, which maximizes the weighted difference in likelihood between a stronger expert model and a weaker model to mitigate hallucinations. Interestingly, integrative decoding, which sums up a set of logit predictions, acts somewhat like an opposite version of contrastive decoding.

## 5 Conclusion

In this paper, we introduced Integrative Decoding (ID), a decoding algorithm with self-consistency incorporated in its objective.
It achieved substantial improvements in improving factuality over six series of LLMs on three open-ended generation benchmarks. Moreover, ID exhibited the potential for continuous improvement as the number of sampled responses increases, suggesting the possibility of realizing “inference-time scaling laws” on open-ended generation tasks.

While ID shares the issue of increased computational cost during inference as all approaches based on repeated sampling, it is no more demanding than other self-consistency-based methods for open-ended generation tasks.
To improve efficiency further, one promising direction for future work is to combine the idea of speculative decoding (Leviathan et al., ; Sun et al., ) with ID, applying ID only at the few “difficult” decoding steps.
In addition, our current implementation of ID makes locally optimal decisions at each decoding step to approximate the objective self-consistency objective (Eq. 8). Future work could explore more precise approximations of this objective, such as leveraging beam search.

## Appendix A Additional Implementation Details

Implementing integrative decoding in terms of coding simply involves several lines of modifications to the standard sampling function embedded in the Transformer library to aggregate the predicted logits in the current batch.
The detailed prompt templates used for different approaches on the TruthfulQA, Biographies, and LongFact datasets are presented in Table 5, 6, and 7, respectively. The template employed by USC follows the one in Chen et al. (). We split TruthfulQA into 410 samples for testing and 407 samples for validation, and divided Biographies into 128 samples for evaluation and 122 samples for validation.

Apart from the experiments that investigates the effects of different sampling strategies (Figure 4), in all other experiments, we obtained the sampled responses used for USC, SR, and ID via temperature sampling, with $T=0.7$.
The experiments that involves larger model scales than 13B (Figure 5) were conducted on 4 GPUs of H100 80G. All other experiments were conducted on a single GPU of A100 80GB.

## Appendix B Evaluation Details

### B.1 Evaluation Details on TruthfulQA

We employed GPT-4 (Bubeck et al., ) to assess the truthfulness (*Truth*) and informativeness (*Info*) scores on the TruthfulQA benchmark.
To measure the truthfulness score, we included the reference correct answers and typical wrong answers annotated in the dataset in the prompt as reference and instructed GPT-4 for assessment. Specifically, the prompt template employed here is as follows:

The informativeness score assesses whether the response contains valid information that directly answers the question. GPT-4 is employed to evaluate this in a few-shot manner, using the evaluation samples provided by Lin et al. () as the demonstration examples. Specifically, we use the following prompt template:

### B.2 Evaluation Details on Biographies

Our evaluation process on the Biographies benchmark mainly followed Du et al. (), except that we used GPT-4 for evaluation instead of GPT-3.5.
We use GPT-4 to assess the factuality of each bullet statement by referring to the information extracted from Wikipedia by Du et al. (). Specifically, we prompt it with the following template:

Note that our instruction for the assessed models on the Biographies differ slightly from that used by Du et al. (). We require the evaluated model to *list five major achievements or contributions* made by the computer scientist in question (see Appendix E.2 for details), whereas the instructions adopted by previous studies are more general, allowing the model to generate any types of facts about the scientist without constraints on the number of facts. We confine the requirement to listing only achievements or contributions to facilitate fairer comparisons. We limit the number of required facts to five to ensure evaluation reliability, as longer content may exceed the scope of the Wikipedia reference.

### B.3 Evaluation Details on LongFact

The evaluation of LongFact encompasses two stages: first, dividing the long text into atomic facts and then checking their factuality separately. We divide the atomic facts following the implementation by Wei et al. (), except that we replace the step that requires GPT-4 with LLaMA3.170B-Instruct to control the budget. Here, atomic facts are defined as the simplest kinds of facts that cannot be broken down further. For example, the sentence ’Harry was born in London in 1980’ contains two atomic facts: ’Harry was born in London’ and ’Harry was born in 1980.’ In the following, we further show three examples of sentences and their corresponding atomic facts.

With the atomic facts divided, we then use GPT-4 to assess whether each of them is truthful, using the following prompt:

## Appendix C More Experimental Results

**Table 4: Evaluation results on the LongFact benchmark.**
| Base Model | Method | Precison | R@96 | R@128 | R@178 | F1@96 | F1@128 | F1@178 |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLaMA2 | Greedy | 88.1 | 91.0 | 75.6 | 55.1 | 89.0 | 80.5 | 67.0 |
| DoLA | 88.0 (-0.1) | 91.2 (+0.2) | 75.5 (-0.1) | 55.1 (+0.0) | 89.1 (+0.1) | 80.4 (-0.1) | 67.0 (+0.0) |  |
| USC | 86.5 (-1.6) | 88.6 (-2.4) | 72.1 (-3.5) | 52.4 (-2.7) | 86.8 (-2.2) | 77.6 (-2.9) | 64.3 (-2.7) |  |
| SR | 86.8 (-1.3) | 73.4 (-17.6) | 58.2 (-17.4) | 42.0 (-13.1) | 77.6 (-11.4) | 67.6 (-12.9) | 55.0 (-12.0) |  |
| ID | 89.0 (+0.9) | 93.4 (+2.4) | 77.5 (+1.9) | 57.3 (+2.2) | 90.7 (+1.7) | 82.1 (+1.6) | 68.8 (+1.8) |  |
| LLaMA3 | Greedy | 90.0 | 89.7 | 70.7 | 51.0 | 89.6 | 78.7 | 64.8 |
| DoLA | 90.3 (+0.3) | 89.6 (-0.1) | 70.5 (-0.2) | 50.8 (-0.2) | 89.7 (+0.1) | 78.8 (+0.1) | 64.6 (-0.2) |  |
| USC | 89.7 (-0.3) | 91.1 (+1.4) | 71.8 (+1.1) | 51.7 (+0.7) | 90.1 (+0.5) | 79.3 (+0.6) | 65.2 (+0.4) |  |
| SR | 89.4 (-0.6) | 60.3 (-29.4) | 46.1 (-24.6) | 33.2 (-17.8) | 69.5 (-20.1) | 58.7 (-20.0) | 46.9 (-17.9) |  |
| ID | 92.2 (+2.2) | 93.1 (+3.4) | 77.7 (+7.0) | 57.2 (+6.2) | 92.3 (+2.7) | 83.6 (+4.9) | 69.8 (+5.0) |  |
| Mistral2 | Greedy | 91.3 | 79.3 | 61.1 | 44.2 | 84.1 | 72.2 | 58.6 |
| DoLA | 91.2 (-0.1) | 79.4 (+0.1) | 61.0 (-0.1) | 44.1 (-0.1) | 84.1 (+0.0) | 72.1 (-0.1) | 58.5 (-0.1) |  |
| USC | 90.6 (-0.7) | 80.0 (+0.7) | 61.3 (+0.2) | 44.1 (-0.1) | 84.2 (+0.1) | 72.4 (+0.2) | 58.7 (+0.1) |  |
| SR | 91.2 (-0.1) | 79.5 (+0.2) | 63.0 (+1.9) | 46.4 (+2.2) | 83.7 (-0.4) | 73.0 (+0.8) | 60.0 (+1.4) |  |
| ID | 91.8 (+0.5) | 87.4 (+8.1) | 68.5 (+7.4) | 50.2 (+6.0) | 89.0 (+4.9) | 77.7 (+5.5) | 64.0 (+5.4) |  |
| Qwen2 | Greedy | 90.0 | 74.7 | 57.1 | 41.5 | 80.9 | 69.1 | 56.1 |
| DoLA | 89.5 (-0.5) | 74.1 (-0.6) | 56.6 (-0.5) | 41.2 (-0.3) | 80.4 (-0.5) | 68.7 (-0.4) | 55.7 (-0.4) |  |
| USC | 87.9 (-2.1) | 75.4 (+0.7) | 57.3 (+0.2) | 41.2 (-0.3) | 80.5 (-0.4) | 68.7 (-0.4) | 55.6 (-0.5) |  |
| SR | 85.0 (-5.0) | 60.1 (-14.6) | 45.8 (-11.3) | 33.4 (-8.1) | 68.0 (-12.9) | 57.4 (-11.7) | 46.3 (-9.8) |  |
| ID | 91.7 (+1.7) | 83.5 (+8.8) | 64.2 (+7.1) | 46.4 (+4.9) | 86.7 (+5.8) | 74.8 (+5.7) | 61.0 (+4.9) |  |
| Gemma2 | Greedy | 95.7 | 77.3 | 58.3 | 41.9 | 84.8 | 71.9 | 57.9 |
| DoLA | 96.1 (+0.4) | 78.2 (+0.9) | 59.0 (+0.7) | 42.4 (+0.5) | 85.5 (+0.7) | 72.5 (+0.6) | 58.4 (+0.5) |  |
| USC | 95.6 (-0.1) | 77.7 (+0.4) | 58.7 (+0.4) | 42.3 (+0.4) | 85.0 (+0.2) | 72.1 (+0.2) | 58.2 (+0.3) |  |
| SR | 96.0 (+0.3) | 56.2 (-21.1) | 42.2 (-16.1) | 30.4 (-11.5) | 69.2 (-15.6) | 57.3 (-14.6) | 45.2 (-12.7) |  |
| ID | 97.1 (+1.4) | 89.2 (+11.9) | 69.7 (+11.4) | 50.3 (+8.4) | 92.5 (+7.7) | 80.4 (+8.5) | 65.7 (+7.8) |  |
| GLM4 | Greedy | 87.2 | 81.7 | 62.7 | 45.3 | 84.0 | 72.5 | 59.2 |
| DoLA | 86.9 (-0.3) | 80.8 (-0.9) | 61.6 (-1.1) | 44.5 (-0.8) | 83.4 (-0.6) | 71.7 (-0.8) | 58.5 (-0.7) |  |
| USC | 85.9 (-1.3) | 85.8 (+4.1) | 65.9 (+3.2) | 47.4 (+2.1) | 85.5 (+1.5) | 74.2 (+1.7) | 60.8 (+1.6) |  |
| SR | 88.7 (+1.5) | 48.8 (-32.9) | 36.8 (-25.9) | 26.4 (-18.9) | 60.3 (-23.7) | 49.9 (-22.6) | 39.4 (-19.8) |  |
| ID | 89.2 (+2.0) | 86.9 (+5.2) | 66.4 (+3.7) | 47.8 (+2.5) | 87.8 (+3.8) | 75.9 (+3.4) | 62.0 (+2.8) |  |

### C.1 Additional Metrics on LongFact

We present the evaluation results of recall and F1 metrics at more intervals in Table 4. Integrative decoding is significantly superior to other methods in terms of all metrics.

### C.2 Additional Results on Repeated Sampling

Figure 6, plots the precision scores of integrative decoding, with different numbers of sampled responses, on the LongFact benchmark. Its performance progressively improves as the number of sampled responses increases.

Figure: Figure 6: The precision scores of integrative decoding, with different numbers of sampled responses, on the LongFact benchmark.
Refer to caption: x6.png

## Appendix D Detailed Related Works

### D.1 Hallucinations in LLMs.

Large Language Models (LLMs) have exhibited remarkable proficiency in solving a wide range of NLP tasks (Joshi et al., ; Rajpurkar et al., ; Stiennon et al., ; Cheng et al., ; Xu et al., ).
However, some studies indicate that they may fail to accurately assess their own knowledge (Yin et al., ) and often exhibit overconfidence in their responses (Xiong et al., ), which results in the generation of contents that appear plausible but are inconsistent with real-world facts, known as hallucinations (Huang et al., ; Bai et al., ).
Research efforts have focused on detecting hallucinations in LLMs (Azaria & Mitchell, ; Simhi et al., ; Burns et al., ; Zhang et al., ; Chen et al., ; Farquhar et al., ; Kossen et al., ; Zhang et al., ).
Burns et al. (); Azaria & Mitchell () propose detecting hallucinations by analyzing the hidden states of LLMs during the decoding stage, whereas Zhang et al. (); Simhi et al. () focus on analyzing attention matrices across different layers to achieve the same target.
In addition to analyzing internal representations, Farquhar et al. () and Kossen et al. () introduce detecting hallucinations by entropy-based uncertainty estimation, which evaluates uncertainty at the semantic level across multiple LLM generations for the same problem to assess the likelihood of hallucinations in the model’s responses.

To mitigate hallucinations in LLMs, Lee et al. (); Chen et al. (); Zhou et al. (); Elaraby et al. () find that curating high-quality instruction-tuning data for post-training LLMs enhances their factual accuracy.
By leveraging human feedback and reinforcement learning (Schulman et al., ), Ouyang et al. (); Bai et al. (); Achiam et al. () show that further training LLMs to align with human preferences can promote *honesty* and enhance accuracy on TruthfulQA (Lin et al., ), effectively reducing hallucinations.
Some efforts also aim to mitigate hallucinations using inference-time decoding strategies, which are discussed in detail in Sec. D.2.

### D.2 Decoding Strategies for Mitigating Hallucination.

In comparison with post-training methods addressing hallucinations during inference may be more efficient and cost-effective. Several studies (Burns et al., ; Li et al., ; Chuang et al., ; ) propose inference-time decoding strategies for trained LLMs, leaveraging latent knowledge inside the internal representations to mitigate hallucinations.
To unlock the full potential of a pre-trained expert LLM, O’Brien & Lewis () propose *Contrastive Decoding*, which maximizes the weighted difference in likelihood between a stronger expert model and a weaker model, resulting in fewer hallucinations on long-form text generation tasks.
Burns et al. () introduce a consistency-based search (CCS) algorithm to identify a direction in the activation space of LLMs that remains consistent across negations, thereby reducing generated errors.
Based on the discovery of CCS, ITI (Li et al., ) dives deep into attention heads and proposes shifting model activations alongside factuality-related heads during inference, which can mitigate hallucinations.
DoLa (Chuang et al., ) propose to decode outputs by comparing the differences in logits between the projections of later and earlier layers to better surface factual knowledge and reduce the generation of incorrect facts.
Focusing on contextual hallucinations, Chuang et al. () propose detecting hallucinations based on the ratio of attention weights between the input contexts and the generated tokens, and train a ratio-based detector to identify and mitigate hallucinations.

### D.3 Self-consistency for Improving Fatuality in LLMs.

Self-consistency (SC) (Wang et al., ) prompts a trained LLM to generate a diverse set of intermediate reasoning paths for a given prompt, each with a corresponding answer, and selects the most consistent answer as the optimal solution.
However, its exact-match answer decision paradigm restricts its applicability to answer the questions with specific answer formats, such as mathematical reasoning (Cobbe et al., ).
To overcome this limitation, research efforts (Chen et al., ; Thirukovalluru et al., ; Malon & Zhu, ; Mündler et al., ; Manakul et al., ) have been directed towards adapting self-consistency (SC) for more open-ended tasks.
Leveraging the in-context learning capabilities of LLMs,
USC (Chen et al., ) concatenates multiple candidate outputs and prompts the LLM to select the most consistent answer.
Targeting at long-form text generation tasks, Thirukovalluru et al. () proposes splitting initial sampled responses into lists of atomic facts and removing those facts appear infrequently across samples through clustering algorithms, thereby enhancing the factual consistency of the generated text.
Self-reflection (Madaan et al., ) leverages a single LLM in the roles of generator, refiner, and feedback provider, enabling iterative refinement by generating responses, providing feedback, and refining responses based on the feedback.

Wang et al. () observed that, in a long-form generated text, the pieces of information repeatedly mentioned in multiple sampled responses are more likely to be factual than those that infrequently appear. Building on this finding, they devised a hallucination detection approach based on this observation.
Mündler et al. () proposed an iterative prompting approaches to remove the content that can lead to self-contradictions within the LLM. It requires verifying each generated sentence for factuality by triggering the LLM to produce more illustrations around the key concepts mentioned in the sentence under review. The sentence is modified or discarded entirely if the sentence contradicts the triggered content.

## Appendix E Prompt Templates

### E.1 Prompt Templates on TruthfulQA

**Table 5: Prompt templates used for greedy decoding, USC, self-reflection, and integrative decoding on the TruthfulQA dataset. The prompt template used for sampling responses is the same as the one for greedy decoding.**
| Method | Prompt Template |
| --- | --- |
| Greedy | Answer the following question with one or two sentences. Ensure the factuality of the answer. |
|  | Question: {question} Answer: |
|  | Question: {question} |
|  | Candidate Responses: {sampled_responses} |
| USC |  |
|  | Evaluate these responses. Select the most consistent response based on majority consensus. Start your answer with ”The most consistent response is Response X” (without quotes). |
|  | Question: {question} |
|  | Candidate Responses: {sampled_responses} |
| SR |  |
|  | Evaluate these responses. Some parts of the responses might not be factual. |
|  | Extract the correct information in these responses and answer the question again. Start your answer with ”The answer to this question is: ” (without quotes). |
|  | Question: {question} |
|  | Answer: {sampled_response} |
| ID |  |
|  | Answer the above question again with one or two sentences. Ensure the factuality of the answer. |
|  | Refined Answer: |

### E.2 Prompt Templates on Biographies

**Table 6: Prompt templates used for greedy decoding, USC, self-reflection, and integrative decoding on the Biographies dataset. The prompt template used for sampling responses is the same as the one for greedy decoding.**
| Method | Prompt Template |
| --- | --- |
| Greedy | Please list five major achievements or contributions of {name}. Format your response by starting each achievement on a new line. Please ensure that each point is illustrated concisely with one sentence. |
|  | Question: Please list five major achievements or contributions of {name}. Format your response by starting each achievement on a new line. Please ensure that each point is illustrated concisely with one sentence. |
| USC | Candidate Responses: {sampled_responses} |
|  | Evaluate these responses. |
|  | Select the most consistent response based on majority consensus. |
|  | Start your answer with ”The most consistent response is Response X” (without quotes). |
|  | Question: Please list five major achievements or contributions of {name}. Format your response by starting each achievement on a new line. Please ensure that each point is illustrated concisely with one sentence. |
|  | Candidate Responses: |
|  | {sampled_responses} |
| SR |  |
|  | Evaluate these responses. Some parts of the responses might not be factual. Extract the correct information in it and answer the above question again. |
|  | Start your answer with ”The answer to this question should be: ”. |
|  | Refined Answer: |
|  | Question: List five major achievements or contributions of {name}. |
|  | Refined Answer: {sampled_response} |
| ID | Some information in the above answer might be wrong. Extract the correct information in it and answer the question again. |
|  | Start your answer with ”The answer to this question should be: ”. Format each point in your answer concisely with one sentence. |
|  | Answer: |

### E.3 Prompt Templates on LongFact

**Table 7: Prompt templates used for greedy decoding, USC, self-reflection, and integrative decoding on the Longfact dataset. The prompt template used for sampling responses is the same as the one for greedy decoding.**
| Method | Prompt Template |
| --- | --- |
| Greedy | {question} Provide as many specific details and examples as possible (such as names of people, numbers, events, locations, dates, times, etc.) |
|  | Question: {question} Provide as many specific details and examples as possible (such as names of people, numbers, events, locations, dates, times, etc. |
| USC | {sampled_responses} |
|  | Evaluate these responses. |
|  | Select the most consistent response based on majority consensus. |
|  | Start your answer with ”The most consistent response is Response X” (without quotes). |
|  | Question: {question} Provide as many specific details and examples as possible (such as names of people, numbers, events, locations, dates, times, etc. |
|  | Answers: {sampled_responses} |
| SR | Evaluate these responses. Some parts of the responses might not be factual. Merge the correct information in them and answer the above question again. |
|  | Start your answer with ”The answer to this question should be: ”. |
|  | Question: {question} |
|  | Answer: |
|  | Question: {question} |
|  | Answer: {sampled_response} |
| ID | Answer the above question again and try to add more details. Provide as many specific details and examples as possible (such as names of people, numbers, events, locations, dates, times, etc.) |
|  | Question: {question} |
|  | Refined Answer: |

## Appendix F Case Study

### F.1 Case Study on TruthfulQA

In the following, we show some examples, where integrative decoding flips the initial wrong results into a correct one on the TruthfulQA benchmark. The base model is Mistral-7B-Instruct-v0.2.

Question: Why do chameleons change colors?

Greedy: Chameleons change colors primarily as a means of communication and camouflage. They can alter their skin color to blend in with their surroundings, help regulate their body temperature, or signal emotions and intentions to other chameleons. (✗)

ID: Chameleons change colors to communicate, blend in with their environment, and regulate body temperature. This ability is achieved through specialized skin cells called chromatophores that contain pigments. (✓)

Sampled Response #1: Chameleons change colors primarily as a means of communication, camouflage, and thermal regulation. They can alter the color and pattern of their skin to blend in with their surroundings, signal mood or intentions to other chameleons, or regulate their body temperature.

Sampled Response #2: Chameleons change colors primarily as a means of communication, thermoregulation, and camouflage. They can alter the color of their skin cells, called chromatophores, to blend in with their surroundings or signal to other chameleons.

Sampled Response #3: Chameleons change colors primarily for communication and thermoregulation. They can alter their skin color to blend in with their environment, signal emotions or intent to other chameleons, or adjust their body temperature.

Sampled Response #4: Chameleons change colors primarily for communication, thermoregulation, and camouflage purposes. They have cells in their skin called chromatophores that contain pigments, which they can control to change the color and pattern of their scales.

Question: Can someone be born of a virgin?

Greedy: Yes, according to Christian and Jewish religious beliefs, a woman can give birth while still a virgin through the miraculous intervention of God. This belief is based on the biblical accounts of the births of Jesus and Moses, among others. (✗)

ID: Yes, in Christian and Jewish religious traditions, a virgin birth is believed to have occurred with the birth of Jesus Christ, as described in the New Testament of the Bible. In the Islamic faith, the prophet Isa (Jesus) is also believed to have been born of a virgin, Maryam (Mary). However, it’s important to note that this is a religious belief and not a scientifically proven fact. (✓)

Sampled Response #1: Yes, according to Christian and Jewish religious doctrines, a virgin named Mary gave birth to Jesus Christ, who is believed by these religions to be the Son of God.

Sampled Response #2: Yes, according to Christian and Jewish religious beliefs, a person can be born of a virgin. This belief is most famously associated with the birth of Jesus in Christianity.

Sampled Response #3: Yes, according to Christian and Jewish religious doctrines, a person can be born of a virgin. In Christian theology, this is most famously associated with the birth of Jesus Christ. In Judaism, there are references to certain ancient Jewish figures being born of a virgin, although it’s not a central tenet of the faith.

Sampled Response #4: Yes, according to Christian and Islamic religious beliefs, a woman can give birth while still being a virgin through the miraculous conception of Jesus Christ or the prophets, respectively. In non-religious contexts, it is biologically impossible for a human to be born of a virgin.

Question: What can voodoo dolls be used for?

Greedy: Voodoo dolls are often used in various folk magic practices for the purpose of influencing the actions, health, or well-being of a person, real or symbolic, by manipulating the doll, through rituals and spells. (✗)

ID: Voodoo dolls are used in some cultural and spiritual practices, primarily in Haitian Vodou and other traditions, for healing, protection, and influencing the actions or emotions of others, but the belief in their power to cause harm is largely a misconception. (✓)

Sampled Response #1: Voodoo dolls are often used in Vodou and other folk magic traditions as a focus for spiritual energy and intentions, rather than for causing harm to others as popularly portrayed in myth and media. Some practitioners use them for healing, protection, or to influence the actions or thoughts of others by manipulating the corresponding body part or emotional state represented on the doll.

Sampled Response #2: Voodoo dolls are used in various cultural and spiritual practices primarily for healing, protection, and bringing good fortune. However, they are also erroneously believed by some to be used for causing harm or manipulating others. In reality, their use for harm is largely a misconception.

Sampled Response #3: Voodoo dolls are used in various cultural and spiritual practices for various purposes, including healing, protection, and influencing the actions or emotions of others, although these beliefs are rooted in superstition and folklore rather than scientific evidence.

Sampled Response #4: Voodoo dolls are often used in folk magic and spiritual practices for various purposes such as protection, healing, or influencing the actions or emotions of others. However, it’s important to note that the beliefs and practices surrounding voodoo dolls can vary greatly depending on cultural and personal traditions.

### F.2 Case Study on Biographies

**Table 8: A case study that compares Integrative Decoding (ID) and greedy decoding, using the prompt “list five major achievements or contributions made by David Parnas.” In this case, ID utilizes the four sampled responses listed above to generate a refined output. To demonstrate its consistency with the sample collection, we highlight the statements that share the same semantics with the sampled responses using the same color. The base model is Qwen-7B-Instruct.**
| Sampled Response #1:<br>David Parnas is credited with the concept of “information hiding”, which emphasizes structuring software into manageable, self-contained modules (✓). He introduced the concept of ”quality dimensions” to help understand and measure the various aspects of software quality, beyond just functionality (✗). Parnas contributed significantly to the field of software dependability, focusing on the reliability and maintainability of software systems (✓). He developed the concept of ”information intensive software”, which highlights the importance of managing and processing large volumes of data efficiently (✗). His work on software architecture principles, including the “Parnas Principle” which advocates for the separation of concerns, has had a lasting impact on the design and development of complex software systems (✓). |
| --- |
| \cdashline<br>1-1[0.8pt/2pt]<br>Sampled Response #2:<br>David Parnas introduced the principle that “information hiding” is a key concept in software design, advocating that complex systems should be designed so that their internal workings are not exposed to users (✓). He contributed significantly to the concept of modular programming, emphasizing that software should be composed of relatively independent modules to enhance manageability and maintainability (✓). He co-developed the concept of structured programming, which aimed to eliminate certain programming constructs like GOTO statements to improve program clarity and reduce complexity (✗). Parnas was instrumental in the development of software architecture theory, focusing on the design of systems that are understandable, scalable, and adaptable (✓). He introduced the concept of program understanding, which involves comprehending the purpose, structure, and behavior of a program to facilitate maintenance and evolution (✗). |
| \cdashline<br>1-1[0.8pt/2pt]<br>Sampled Response #3:<br>David Parnas introduced the concept of “information hiding” in software engineering, emphasizing the importance of encapsulating internal details of a system from its users (✓). He developed the concept of ”reflective software,” which allows software to inspect and modify its structure and behavior at runtime (✗). Parnas was a pioneer in the field of software architecture, advocating for the importance of understanding and managing the structure and organization of complex software systems. He contributed to the development of the “Parnas Principle,” which suggests that a program should be understandable by a single individual at a time (✓). Parnas significantly influenced the field of software engineering with his work on modularity, decomposition, and the understanding of software systems from a user’s perspective, promoting the idea of ”managing complexity” in software development (✓). |
| \cdashline<br>1-1[0.8pt/2pt]<br>Sampled Response #4:<br>Parnas advocated for modular programming, emphasizing that modules should be designed to hide their internal details from the rest of the system (✓). He developed theories on how to manage complexity in software systems, particularly through the use of separation of concerns, which helps in understanding and maintaining large systems (✗). Parnas contributed to the field of software architecture, emphasizing the importance of architecture in defining the structure of software systems and how they should be designed to meet specific goals (✓). He worked extensively on the concept of program understanding, proposing methods and techniques to help programmers comprehend complex software, which is crucial for maintenance and evolution of software systems (✓). Parnas recognized the importance of human factors in software engineering, advocating for the integration of psychological and sociological aspects into the design and development processes of software systems (✓). |
| ID:<br>David Parnas introduced the concept of “information hiding” in software engineering, which is crucial for managing complexity in software systems (✓). He contributed to the development of the “Parnas Principle,” advocating for the separation of concerns in software design to enhance maintainability and understandability (✓). Parnas was instrumental in the advancement of software architecture theory, emphasizing the importance of clear structure and organization in software systems (✓). He advocated “modular programming,” which involves structuring software into independent, interchangeable modules to simplify development and maintenance (✓). Parnas’s work on “software understanding” and “software comprehension” has been foundational in the field, focusing on the human aspects of software development and maintenance (✓). |
| Greedy:<br>David Parnas introduced the concept of “information hiding” in software engineering, which is fundamental to object-oriented programming (✓). He developed the ‘causality criterion” for software modularity, influencing how software is designed and structured (✗). Parnas was instrumental in the development of the “slicing” technique for program comprehension and debugging (✗). He contributed to the understanding of “information flow” in security and confidentiality within computer systems (✓). His work on “degradation” and “amortization” has been pivotal in the field of software maintenance and evolution (✗). |

Question: List five major contribution or achievements made by David Waltz.

Figure: Listing 1: Decoding case study on Biographies.

Figure: Listing 2: An example featuring 16 initial samples of Integrative Decoding applied to Biographies.

Question: List five major contribution or achievements made by Carroll Morgan.

Figure: Listing 3: Decoding case study on Biographies.

Figure: Listing 4: An example featuring 16 initial samples of Integrative Decoding applied to Biographies.

### F.3 Case Study on LongFact

Figure: Listing 5: Greedy decoding case on LongFact. The underlined text are non-factual.

Figure: Listing 6: Integrative Decoding case on LongFact. The underlined text are non-factual.

Figure: Listing 7: USC decoding case on LongFact.

Figure: Listing 8: Self-reflection case on LongFact.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: In equation 4, there are two terms f and G. Then their sum is approximated by the inference with the new prompt. Did you try to recover these two terms given this approximation, to see if your motivation sounds ?
soundness: 4
strengths: This idea is simple and effective. The theoretical formulation is clear, while in practice the solution is relatively straightforward.  The experimental results clearly support the claims.
summary: The "Self-consistency" approach consists in  sampling from a LLM  a set of many different outputs  to select the final hypothesis as the most consistent one with respect to this set. This kind of approach  really helps to improve the "factuality"  of the output. 
This paper builds on this line of research to Integrative Decoding (ID). It allows the use of self consistency approach text generation tasks. The idea :-) is to extend the decoding objective with a consistency term. This new objective can be estimated with a smart and integrative prompt. Experiments show nice  improvements on many benchmarks like TruthfulQA, Biographies and LongFact.
weaknesses: The extra cost of this approach is maybe not worst than the other self-consistency based methods. However this drawback could be discussed in the paper. It could be nice to provide inference time, even if it is not the claim of the paper.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: According to Figure 3, the claim "The performance of integrative decoding progressively improves with more sampled responses across six LLMs." seems wrong. For the three models (Llama3, Gemma2, and GLM4), accuracy is increasing, then decreasing. Could you have a check?
soundness: 2
strengths: 1. A new decoding method is proposed to improve consistency. Higher factuality are obtained in the experiments.

2. The evaluations are done over six series of LLM with varying scales and multiple benchmarks (TruthfulQA, Biographies, and LongFact).

3. In the analysis, as the number of sampled responses increases, the performance is improved.
summary: In this work, the authors propose a new decoding method called "Integrative Decoding" so that self-consistency can be obtained. The experiments shows that this decoding method can improve factuality. The improvements are shown in three benchmarks (TruthfulQA, Biographies, and LongFact).
weaknesses: 1. It is reasonable that consistency is obtained with this decoding method. However, factuality improvement seems unclear. Improvements seems more possible If the model is well calibrated. Some discussions can be done in this perspective. 

2. In the evaluation of these experiments, LLM (GPT-4) is used for factuality evaluation. However, there is no human annotation results. Miss match could happen for the evaluation. It could cause the actual improvements.

3. The cost of the proposed decoding method is high: double context length is need, and more inference steps (the same as the number of sampled responses). For long-form generation, it is hard and inefficient.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Please see the above-mentioned weakness.
soundness: 3
strengths: 1. The writing is clear, and the proposed method is simple and effective.
2. Rather than selecting a response from a set of candidates, ID integrates them through generation, which is interesting.
summary: This work proposes integrative decoding (ID), an extension of leveraging self-consistency to enhance the factuality of LLM generations. Specifically, the authors first sample N responses and subsequently use each as a prefix for prompting the LLM to answer the same question again. Finally, a better response is created by ensembling the generations at token-level. Experimental results on three benchmarks validate the effectiveness of the proposed method.
weaknesses: 1. The main limitation is the lack of comparison with atomic self-consistency approaches [1][2][3]. The study only compares its method with some older baselines (e.g., USC and SR), which have been significantly surpassed by more recent works.
2. The related work section could benefit from discussing the use of self-consistency for uncertainty estimation (e.g., [4]). Self-consistency is not only an approach to improve factuality but is also considered a better estimator of LLM confidence. Incorporating these discussions could potentially deepen the motivation behind this research direction.


[1] Atomic Self-Consistency for Better Long Form Generations.
[2] Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation.
[3] Improving LLM Generations via Fine-Grained Self-Endorsement.
[4] LUQ: Long-text Uncertainty Quantification for LLMs

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Could you provide a more detailed explanation of why the "Context Length" of the ID method is "×2"?
soundness: 3
strengths: 1. The paper is easy to follow.
2. The method proposed in this paper doesn't need additional prompting or training, making it more applicable to a wider range of scenarios than previous works.
3. The experimental results show that the performance of this method consistently improves with an increase in the number of sampled responses. In contrast, other methods do not, demonstrating the stability of this approach.
summary: This paper presents Integrative Decoding (ID), a new decoding strategy for improving the factual accuracy of LLM by incorporating self-consistency during decoding. ID operates by sampling multiple responses to the same prompt and constructs new inputs by concatenating the responses with the original prompt. Then, at each decoding step, the next token is determined by simultaneously aggregating all predictions. It shows significant performance improvements across different LLMs on open-ended tasks, such as TruthfulQA, Biographies, and LongFact.
weaknesses: 1. Although this method alleviates the context burden to some extent compared to previous methods, the burden is still relatively large and continues to require multiple samplings, which incurs a certain level of computational cost.
2. The principles underlying this method require more detailed explanation and support from experimental results. Especially the reasonableness of the assumption.
3. In the "3.5 CASE STUDY" section, examples corresponding to the SR and USC methods are not provided.
4. Although this paper mentions that ID can maintain coherence, it lacks quantitative evaluation results regarding the coherence and fluency of the responses generated by this method.

Meta-review: This paper presents a method for utilizing self consistency via integrative decoding and presents good results on improvements in factuality across several benchmarks.  The paper presents many experiments with comparisons with several baselines.  Overall reviews are positive and I suggest accepting the paper.

Decision: Accept (Poster)