Venue: NAACL 2025

# CORD: Balancing Consistency and Rank Distillation

for Robust Retrieval-Augmented Generation

Youngwon Lee1Seung-won Hwang\({}^{*}\) Daniel Campos

Filip Gralinski Zhewei Yao Yuxiong He

Snowflake AI Research & Seoul National University

Footnote 1: For presentation brevity, we reveal in later section.

###### Abstract

With the adoption of retrieval-augmented generation (RAG), large language models (LLMs) are expected to ground their generation to the retrieved contexts. Yet, this is hindered by position bias of LLMs, failing to evenly attend to all contexts. Previous work has addressed this by synthesizing contexts with perturbed positions of gold segment, creating a position-diversified train set. We extend this intuition to propose consistency regularization with augmentation and distillation. First, we augment each training instance with its position perturbation to encourage consistent predictions, regardless of ordering. We also distill behaviors of this pair, although it can be counterproductive in certain RAG scenarios where the given order from the retriever is crucial for generation quality. We thus propose CORD, balancing COnsistency and Rank Distillation: CORD adaptively samples noise-controlled perturbations from an interpolation space, ensuring both consistency and respect for the rank prior. Empirical results show this balance enables CORD to outperform consistently in diverse RAG benchmarks.

+
Footnote †: _Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)_, pages 787–796

## 1 Introduction

Recently, large language models (LLMs) have incorporated retrievers to augment input contexts for more grounded generation. However, during retrieval-augmented generation (RAG), LLMs reportedly suffer from position bias where they pay disproportionate attention to different parts, worsened as the input becomes longer Liu et al. (2024). An existing solution has synthesized a training set by randomizing the position of gold segment An et al. (2024). It allows LLMs to implicitly learn that relevant information can appear at any position, mitigating position bias.

Our distinction is to pursue dual goals of (1) **CO**nsistency for mitigating position bias and (2) **Rank**Distillation, learning to utilize meaningful signals in the given order from the retriever and also to denoise it, for robust RAG.

For CO, we extend the position-perturbing training intuition, by augmenting the retriever-given order of contexts \(\mathbf{c}\) with its perturbation \(\mathbf{c}^{\prime}\), sharing the same ground truth \(\hat{y}\). Green arrows in Figure 1 visualize how this augmentation indirectly enforces consistency by guiding predictions \(y\) from \(\mathbf{c}\) and \(y^{\prime}\) from \(\mathbf{c}^{\prime}\), to converge to the ground-truth \(\hat{y}\).

To further enforce consistency, a distillation loss can be added to directly penalize the distributional divergence in all outputs. The blue arrow in Figure 1 visualizes this loss further incentivizing consistent internal representation, by distilling 'dark knowledge' Hinton et al. (2015); Sadowski et al. (2015); Furlanello et al. (2018) from one to another.

However, pursuing CO objective alone, without balancing it with the RD objective, is counterproductive in some scenarios as illustrated in Table 1. It contrasts two representative real-life RAG scenarios A and B:1 In A, retriver provides a reliable rank prior, such that distilling predictions from a

Figure 1: Enforcing consistency with (1) augmentation (green) and (2) distillation (blue).

\begin{table}
\begin{tabular}{l c c} \hline Method & (A) & (B) \\ \hline Given order & 41.34 & 56.52 \\  + consistency & 36.87 (↓) & 57.87 (↑) \\ CORD (ours) & **44.74** (↑) & **58.71** (↑) \\ \hline \end{tabular}
\end{table}
Table 1: Generation quality with different methods in representative RAG scenarios A and B, where distillation may hinder or enhance, respectively.



randomized ordering can unlearn this helpful prior, as evidenced by the degradation in generation quality after consistency regularization. Meanwhile, in B, where generation is not sensitive to the given order, CO objective enhances performance.

Our technical contribution is to adapt \({\bf c}^{\prime}\) to the given scenario, by controlling the degree of perturbation, in place of \({\bf c}^{\prime}\) with a fixed randomization. We define an interpolated space of perturbations and dynamically sample an appropriate level of perturbation from it. Table 1 shows CORD outperforms in both scenarios, by sampling smaller perturbations in scenario A, where rank prior is important, and larger perturbations in scenario B, where robustness to position bias is crucial.

Our contribution can be summarized as follows: (1) We propose CORD, balancing consnistency and rank distillation in RAG. (2) We show distilling with a controlled perturbation, sampled from an interpolated space of teachers, is effective across 5 diverse RAG scenarios, whereas existing consistency methods fall short.

## 2 Related Work

### Position Bias in Long Context LLMs

Liu et al. (2024) and similar works have shown that LLMs favor input contexts placed at the beginning or end of the input, a tendency that benchmarks such as needle-in-a-haystack2 aim to assess by testing their ability to locate relevant information (_needle_) within long, potentially irrelevant contexts (_hayastack_). An et al. (2024) extended this understanding by training models on synthetic data, intentionally perturbing a position of gold segment and adding random noises. Similarly, Fu et al. (2024) examined continual pretraining of LLMs on long-context data to expand their context window size for retrieving information.

Footnote 2: github.com/gkamradt/LLMTest_NeedleInAHystack

Our distinction is to use position perturbation for a different objective of data augmentation for consistency training.

### Data Augmentation for Consistency

Pairing a datapoint with a counterfactual applying a small perturbation has been mainly studied for robust training on simpler tasks such as classification (Xie et al., 2020). To our knowledge, we are the first to augment a position-perturbed retriever during training and enforce consistency for RAG.

Another related line of work is interpolating two training instances Chuang and Mroueh (2021), which we extend to define a space of controlled perturbations for dynamic adaptation in Section 3.2.

## 3 Method

### CO: Consistency Regularization

We propose to mitigate position bias by regularizing output consistency over possible perturbations, through (1) augmentation and (2) distillation.

First, we explain how augmenting position-perturbed examples contributes to consistency. We first formalize RAG as generating an answer \(y\) given an input \(x\),

\[y\sim p(\cdot\,|\,x,{\bf c}), \tag{1}\]

along with the sequence of \(n\) retrieved contexts \({\bf c}=[c_{1};c_{2};\cdots;c_{n}]\). Then, for a training triplet \((x,{\bf c},\hat{y})\) the negative log-likelihood (NLL) loss for maximum likelihood estimation training is

\[\mathcal{L}_{\text{n}}=-\sum_{t}\log p(\hat{y}_{t}\,|\,x,{\bf c},\hat{y}_{< t}), \tag{2}\]

which encourages the model to produce the correct answer \(\hat{y}\) given the input \(x\) and retrieved contexts.

Inspired by An et al. (2024), referred to as In2, we employ position perturbation to augment \({\bf c}\) from the corpus \(\mathcal{C}\) with \({\bf c}^{\prime}\). For comparison, In2 synthesized question and context \((q,{\bf c})\) pairs where the gold passage \(s\) for generating the gold answer \(\hat{y}\) appears in various positions. As Figure 2 shows, we retain both the original \((q,{\bf c},\hat{y})\) and the perturbed examples \((q,{\bf c}^{\prime},\hat{y})\): Unlike In2's using \({\bf c}^{\prime}\) only for training (orange arrows), we train over the augmented dataset \(\mathcal{C}^{\prime}\) which includes both \({\bf c}\) and \({\bf c}^{\prime}\) (blue arrows). Predictions for both are supervised to converge to the same ground-truth \(\hat{y}\) using the loss in Eq. 2.

Second, by adding a distillation loss, we can further match token-level output probability distributions for \({\bf c}\) and \({\bf c}^{\prime}\). We use the sum of Jensen-Shannon Divergence (JSD) between output proba

Figure 2: (Left) In2 only uses \({\bf c}^{\prime}\). (Right) We augment the given order \({\bf c}\) (top) with perturbed ranking \({\bf c}^{\prime}\) (bottom) and use both.

 

bility distributions at each time step \(t\) for this purpose:3

Footnote 3: While we default to summing all terms, the number of time steps \(t\) to aggregate in Eq 3 can be adjusted for efficiency, as detailed in Appendix B.

\[{\cal L}_{\text{d}}=\sum_{t}\operatorname*{JSD}\left(f_{t}(\mathbf{c})\parallel f _{t}(\mathbf{c^{\prime}})\right), \tag{3}\]

where \(f_{t}(\mathbf{c})=p(\hat{y}_{t}\mid x,\mathbf{c},\hat{y}_{<t})\). This encourages the model to align its internal representations of input and association with the output, encoded in the 'dark knowledge' Hinton et al. (2015); Sadowski et al. (2015); Furlanello et al. (2018) across different perturbations.

Finally, the two types of loss in Eq. 2 and 3 can be combined to obtain our training objective:

\[{\cal L}={\cal L}_{\text{n}}+\lambda\cdot{\cal L}_{\text{d}}, \tag{4}\]

where the hyperparameter \(\lambda\) determines the relative strength of the two terms.

### RD: Adaptive Teacher Selection for Rank Distillation

However, as previously outlined in Table 1(A), distill loss on a random perturbation \(\mathbf{c^{\prime}}\) may interfere with the RD objective in an RAG scenario where retriever provides a meaningful ranking \(\mathbf{c}\) with valuable prior: In this work, we consider MS MARCO Bajaj et al. (2018) as a representative example, where an industry-scale complex retrieval system provides the ranking.

Figure 3(A) depicts such unlearning of ranker prior, when distilled from a random perturbation in scenario A. The \(y\)-axis in the plot represents the probability the LLM assigns to the ground-truth answer, \(p(\hat{y}\mid x,\mathbf{c})\) for the given order \(\mathbf{c}\) (solid circle) and \(p(\hat{y}\mid x,\mathbf{c^{\prime}})\) for random perturbation \(\mathbf{c^{\prime}}\) (empty circle). In MS MARCO, the given order \(\mathbf{c}\) carries a useful prior, resulting in high probability of the ground-truth \(p(\hat{y}\mid x,\mathbf{c})\). Randomizing this order would greatly lower the probability \(p(\hat{y}\mid x,\mathbf{c^{\prime}})\), such that enforcing consistency between the two would unlearn the benefit of rank prior.

To tackle this, instead of fixing \(\mathbf{c^{\prime}}\) as a random perturbation, we define a sample space and strategy for adaptive teacher selection, to control the degree of perturbation for distillation. We introduce an interpolation of \(\mathbf{c}\) and \(\mathbf{c^{\prime}}\) with a controlled noise degree of \(\alpha\), denoted as \(\mathbf{c^{\prime}}_{\alpha}\): Here, the lower ranked \(\alpha\) proportion of the retrieved contexts is randomized while the remaining retains the given order. In Figure 3, such interpolated sample is shown as a shaded circle on a dotted line, the interpolated path connecting \(\mathbf{c}\) and \(\mathbf{c^{\prime}}\), as the noise degree \(\alpha\) varies from 0 to 1. For brevity, we assume a desirable single value of \(\alpha\) for the given task is known a priori, and later discuss how to find it in Section 3.3.

This interpolation allows to select a better teacher between \(\mathbf{c^{\prime}}_{\alpha}\) and \(\mathbf{c^{\prime}}\) by choosing the one with a higher probability of predicting the ground truth. As shown in Figure 3A, small perturbations tend to yield higher \(y\) values in scenario A as they retain the given order in part, leading to \(\mathbf{c^{\prime}}_{\alpha}\) chosen for distillation. This corresponds to ensembling two retrievers, which agree on top-ranked documents but diversify the ranks of the rest.

An added advantage is, the same approach seamlessly supports scenario B, where there is no conflict between CO and RD. As illustrated in Figure 3(B), the \(y\)-axis score remains relatively stable across different orderings, and moreover, the score is no longer sensitive to ordering. Thus, pairing the given order with the one that has a higher \(y\) score essentially serves the goal of pursuing CO.

### Score-Aware Teacher Sampling

So far, we have mainly focused on utilizing _rank_ prior from the retriever; however, the retriever may provide varying level of information in different RAG scenarios, such as score for each item as well. We describe how to incorporate such additional signals into adaptive teacher sampling.

When no prior knowledge of the distribution of the probability of ground-truth \(p(\hat{y}\mid x,\mathbf{c^{\prime}}_{\alpha})\) over the interpolated path is known, we follow the principle of maximum entropy Jaynes (1957) to assume uniform distribution. That is, we choose to sample \(\alpha=0.5\) from the interpolated space defined in Section 3.2, where \(\alpha\) varies in the range of \((0,1)\).

Alternatively, we utilize retriever scores as a

Figure 3: Interpolated sample space for scenario A and B from Table 1, where (A, left) perturbation leads to a large drop in probability of ground-truth \(\hat{y}\), and (B, right) with no such drop.

 

proxy for the unknown distribution of \(p(\hat{y}\,|\,x,\mathbf{c}^{\prime}_{\alpha})\), from which the optimal noise level \(\alpha\) can be determined. Specifically, we aim to extract the most confident top-ranked contexts identified by the retriever, by preserving the contexts ranked above the largest discontinuity in scores and perturbing the rest. Given scores \(s_{i}\) for each retrieved context \(c_{i}\in\mathbf{c}\), which are sorted in descending order of score, i.e., \(s_{1}>s_{2}>\cdots>s_{n}\), we locate the adjacent pair of passages with the largest gap in retriever score \(\hat{i}=\operatorname*{argmax}_{i}(s_{i}-s_{i+1})\) and perturb the passages ranked lower than \(\hat{i}\). In other words, we choose \(\alpha=1-\hat{i}/n\) for this example. Intuitively, this approximates finding the largest acceptable degree of noise that would still result in sufficiently high \(p(\hat{y}\,|\,x,\mathbf{c}^{\prime}_{\alpha})\).

## 4 Results

We design evaluations to answer these research questions:

* [noitemsep]
* (RQ1) Does CORD pursue dual goals of CO and RD effectively?
* (RQ2) Does CORD adaptively choose \((\mathbf{c},\mathbf{c}^{\prime})\) pair in different scenarios?
* (RQ3) How can the noise degree \(\alpha\) for interpolation be tuned per task or example?

### Experimental settings

We have evaluated our proposed method on several QA benchmarks: MS MARCO Bajaj et al. (2018), HotpotQA Yang et al. (2018), NaturalQuestions Kwiatkowski et al. (2019); NQ as reorganized by Liu et al. (2024). We further consider multi-needle (MN) dataset, which is built following An et al. (2024), as a scenario where irrelevant contexts are prevalent and retriever prior is not meaningful.4

Footnote 4: This corresponds to scenario B in Table 1 and Figure 3.

For evaluation, we used widely reported metrics for each benchmark, namely ROUGE-L for MS MARCO, exact match (EM) for HotpotQA, and span-based exact match, or 'accuracy' for NQ. We also adopted the evaluation protocol from Yang et al. (2024) using GPT-4, allowing more flexibility in answers. For MN where answers typically contain a few sentences, we report sentence-level F\({}_{1}\), and for MN-IDK, an unanswerable split of MN, we report accuracy. Further details can be found in Appendix A.

### Results

Bias mitigation and rank distillationTable 2 shows that our proposed method outperforms the baselines across all benchmarks, validating its effectiveness in pursuing dual goals of CO and RD.

In addition, Table 3 shows the importance of denoising through consistency in rank distillation. There is a clear performance gap between the model trained on the given order **c** without augmentation (second row), and those augmented (third and fourth) on MS MARCO. This suggests that even with a strong rank prior, consistency across slight perturbation positively contributes to RD, by mitigating potential bias from retriever or generator.

Adaptive pair selectionCORD indeed selects the proper teacher for enforcing consistency, while

\begin{table}
\begin{tabular}{l c c} \hline \hline  & MN & MN-IDK \\ Finetuning Method & F1 & Acc \\ \hline CORD & 58.71 & 98.83 \\ + Adaptive \(\alpha\) & 59.16 & 98.83 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Effect of dynamically adjusting \(\alpha\) based on retriever score.

\begin{table}
\begin{tabular}{l l c c c c c c c} \hline \hline  & \multicolumn{2}{c}{MS MARCO} & \multicolumn{2}{c}{HotpotQA} & \multicolumn{2}{c}{NQ} & MN & MN-IDK \\ \cline{2-9} Finetuning Objective & R-L & GPT-4 & EM & GPT-4 & Acc & GPT-4 & F\({}_{1}\) & Acc \\ \hline No finetuning & 41.34 & 51.94 & 42.86 & 66.50 & 52.18 & 62.46 & 56.52 & 54.82 \\ \(\mathcal{L}_{\text{all}}\) on \(\mathcal{C}^{\prime}\) & 44.52 & **57.28** & 58.62 & 83.75 & 55.60 & 63.51 & 56.25 & 95.78 \\ CORD & **44.74** & **57.28** & **63.55** & **85.72** & **58.55** & **63.72** & **58.71** & **98.83** \\ \hline \hline \end{tabular}
\end{table}
Table 2: RAG performance with Phi-3 3B as the generator and different finetuning strategies applied.

\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{MS MARCO} & \multicolumn{2}{c}{HotpotQA} & \multicolumn{2}{c}{NQ} & MN & MN-IDK \\ \cline{2-9} Finetuning Objective & R-L & GPT-4 & EM & GPT-4 & Acc & GPT-4 & F\({}_{1}\) & Acc \\ \hline No finetuning & 41.34 & 51.94 & 42.86 & 66.50 & 52.18 & 62.46 & 56.52 & 54.82 \\ \(\mathcal{L}_{\text{all}}\) on \(\mathcal{C}^{\prime}\) & 44.52 & **57.28** & 58.62 & 83.75 & 55.60 & 63.51 & 56.25 & 95.78 \\ CORD & **44.74** & **57.28** & **63.55** & **85.72** & **58.55** & **63.72** & **58.71** & **98.83** \\ \hline \hline \end{tabular}
\end{table}
Table 2: RAG performance with Phi-3 3B as the generator and different finetuning strategies applied.

 

the tendency in choices exhibit clear difference per different RAG scenario, as shown in Figure 4. The ratio of \({\bf c}^{\prime}_{\alpha}\) paired with **c** is shown with dark blue, while the ratio of \({\bf c}^{\prime}\) paired with **c** is presented by light blue bar. Comparing MS MARCO (top) and MN (bottom), it is clearly shown that \({\bf c}^{\prime}_{\alpha}\) is much more likely to be paired with **c** in the former, where the RD objective is more prominent. This supports our rationale behind designing adaptive teacher selection in Section 3.2.

Score-aware teacher samplingTable 4 shows that score-aware dynamic adjustment of \(\alpha\), described in Section 3.3 brings further gain; the effective mean value of \(\alpha\) throughout the train set was 0.8, suggesting a larger portion of the ranking was allowed to be perturbed.

## 5 Conclusion

We have presented CORD, to balance the tension between CO (consistency) and RD (rank distillation) objectives in RAG. For the former, we augment order-perturbed contexts and add distillation loss for explicit consistency regularization. For the latter, CORD adaptively chooses desirable degree of perturbation to prevent unlearning valuable prior from the retriever. CORD consistently outperforms existing methods in diverse RAG scenarios.

### Limitations

Whether our findings generalize over diverse models can be further explored. In addition, the pros and cons of diverse mixing strategies for an interpolated sample space, such as employing another retriever for mix, can be explored; we leave it as future work.

## References

* An et al. (2024) Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, and Jian-Guang Lou. 2024. Make your llm fully utilize the context. _Preprint_, arXiv:2404.16811.
* Bajaj et al. (2018) Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. Ms marco: A human generated machine reading comprehension dataset. _Preprint_, arXiv:1611.09268.
* Chuang and Mroueh (2021) Ching-Yao Chuang and Youssef Mroueh. 2021. Fair mixup: Fairness via interpolation. In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
* Fu et al. (2024) Yao Fu, Ramsewar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. 2024. Data engineering for scaling language models to 128k context. In _Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024_. OpenReview.net.
* Furlanello et al. (2018) Tommaso Furlanello, Zachary Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. 2018. Born again neural networks. In _Proceedings of the 35th International Conference on Machine Learning_, volume 80 of _Proceedings of Machine Learning Research_, pages 1607-1616. PMLR.
* Hinton et al. (2015) Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. In _NIPS Deep Learning and Representation Learning Workshop_.
* Jaynes (1957) E. T. Jaynes. 1957. Information theory and statistical mechanics. _Phys. Rev._, 106:620-630.
* Jha et al. (2024) Rohan Jha, Bo Wang, Michael Gunther, Saba Sturua, Mohammad Kalim Akram, and Han Xiao. 2024. Jina-colbert-v2: A general-purpose multilingual late interaction retriever. _CoRR_, abs/2408.16672.
* Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answering research. _Transactions of the Association for Computational Linguistics_, 7:452-466.
* Lin et al. (2024) Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. 2024. The unlocking spell on base llms: Rethinking alignment via in-context learning. In _International Conference on Learning Representations_.
* Liu et al. (2018) Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paramjape, Michele Bevilacqua, Fabio Petroni, and Percy

Figure 4: (Top) On MS MARCO, the interpolated noise-controlled perturbation \({\bf c}^{\prime}_{\alpha}\) (dark blue) is much more likely to be paired with the given order **c**, than \({\bf c}^{\prime}\) (light blue). (Bottom) The gap is much smaller on MN.

 

Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, pp. 157-173. Cited by: SS6.
* P. Sadowski, J. Collado, D. Whiteson, and P. Baldi (2015)Deep learning, dark knowledge, and dark matter. In Proceedings of the NIPS 2014 Workshop on High-energy Physics and Machine Learning, Vol. 42, Montreal, Canada, pp. 81-87. External Links: ISBN 978-3-319-50356-X, Link, Document Cited by: SS6.
* Q. Xie, Z. Dai, E. H. Hovy, T. Luong, and Q. Le (2020)Unsupervised data augmentation for consistency training. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Cited by: SS6.
* comprehensive rag benchmark. CoRRabs/2406.04744. Cited by: SS6.
* Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. Cohen, R. Salakhutdinov, and C. D. Manning (2018)HotpotQA: a dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 2369-2380. External Links: Link, Document Cited by: SS6.

 

Implementation Details

MN constructionFor MN data construction, we generally followed the recipe from An et al. (2024), with the subtle difference that Mixtral was used for question and answer generation. When preparing the MN dataset following An et al. (2024), we generally abide by their practices, while using Mixtral as the LLM for question and answer extraction, and employed GPT-4 to verify it. For the seed corpus, we utilized the same realnewslike subset from the C4 corpus as \({\cal C}\). We refer the reader to their original paper for more details.

In addition, to study how LLMs can be trained to refuse to answer when there are insufficient evidence provided, rather than to hallucinate, we split the test set into two settings, answerable and unanswerable: In the latter, dubbed MN-IDK, the gold segment \(s\) that provides the evidence to answer the given question is omitted. Thus, the model is expected to answer it does not have enough evidence in the contexts to provide the correct answer, or, 'I don't know.'

MetricsThe evaluation protocol involving GPT-4 as the judge from Yang et al. (2024) evaluates the correctness of the answer with greater flexibility, compared to the canonical lexical match based metrics, and is known to align better with human judgment. Also, it penalizes hallucinated response more than simply abstaining.

While other benchmarks considered in this work require shorter answers, expected answers in MN and MN-IDK typically comprise of a few sentences: thus, we report sentence-level F1 score for MN, where GPT-4 was used as a judge in the same manner as the method described above to decide each sentence in the generated answer is supported by the ground-truth (precision), and vice versa (recall). For MN-IDK, GPT-4 determined whether the model response successfully refused to provide the answer or not, and we reported the accuracy.

Prompts provided to LLM for both type of evaluation can be found in Appendix C.

TrainingFor MS MARCO, HotpotQA and MN, we finetuned Phi-3 3B model on their respective train data: for MS MARCO, we used 20k examples held out from v2.1 dev set for training, and used non-overlapping subset for testing.

For training with CORD on MN, as described in Section 3.2, we generated an artificial ranking over the passages by reranking them with a ColBERT variant model from Jina AI,56 which also provided scores for each passage. This artificial ranking serves as the opposite extreme of the interpolated perturbation space, \(\mathbf{c^{\prime}}\).

Footnote 5: huggingface.co/jinaai/jina-colbert-v2

Footnote 6: While our work is completely orthogonal to the choice of retriever, we chose this lightweight model that reportedly perform well across several IR benchmarks (Jha et al., 2024).

The base model, Phi-3 3B, was trained with LoRA at bf16 precision. The relevant hyperparameter configuration was as follows: for LoRA related settings, we used rank of \(r=8\), \(\alpha=32\), and dropout rate of 0.1. For general configuration, we used linear decay for scheduling with initial learning rate of 1e-4 and effective batch size of 4; we trained the model for 5 epochs with weight decay of 0.01 applied. For CORD-specific configuration, we set coefficient for consistency loss strength \(\lambda\) as 10 and the noise degree for interpolating contexts \(\alpha\) as 0.5 throughout our experiments. We leave it as future efforts to search for optimal configuration for these values per different scenarios.

## Appendix B Design of Consistency Loss

Using the loss from the first token of the answer only also worked reasonably. We attribute this to that contribution of the consistency loss terms from earlier time steps, i.e., those from the beginning of the ground-truth, are larger than that of those from later time steps. The model output probability distribution for time step \(t\) defined previously in Section 3.1 is indeed conditioned on the shared prefix of the ground-truth answer \(y_{<t}\): as more tokens in the prefix are conditioned in both sides as \(t\) increases, the distribution over the token to be immediately followed \(f_{t}\) would converge, as less and less options would be part of a plausible continuation of the answer. This results in terms from later \(t\) contributing smaller to the total loss \(\mathcal{L}_{\text{con}}\), which is why dropping all of them but some at the beginning, just one in the extreme case, suffices to regularize the model output. It is consistent with the findings from previous papers showed that token-level distributional shift between the base and finetuned LLM decreases over time step during decoding (Lin et al., 2024).

While the benchmarks we have considered generally require rather short responses, it remains to see if this mechanism of using the first time step only for consistency loss computation also work well for long-form answer generation tasks.

 

* [25] A. K. Krizhevsky, "Theoretical Computer Science: The Art of the Philosophy of Human Language and its Applications", _Journal of Artificial Intelligence_**106**(4), 379-484.
* [26] M. L. B. McIntyre, "A New Approach to Human Language Processing in Natural Language Processing", _IEEE Transactions on Knowledge Discovery and Data Engineering_**10**(1), 1-10.
* [27] R. C. Myers, "An Introduction to Human Language Processing", _Proceedings of the National Academy of Sciences_**106**(4), 1111-1118.

[MISSING_PAGE_POST]

 

* [10] A. K. Krizhevsky, "Theoretical Computer Science Handbook", vol. 32 of _Lecture Notes in Physics_, vol. 497 of _Springer Series in Mathematics and Statistics_, pp. 56-80, Springer Verlag, Berlin, 1996.

[MISSING_PAGE_POST]

 

LLM Prompts

We provide prompts used for LLM-as-a-judge evaluation of accuracy (Figure 5) and sentence-level F\({}_{1}\) score (Figure 6).