Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness
Authors: Baolong Bi
Shenghua Liu
Yiwei Wang
Lingrui Mei
Junfeng Fang
Hongcheng Gao
Shiyu Ni
Xueqi Cheng
Venue: ICLR 2025 Poster (2025)
Keywords: Large Language Models
In-Context Faithfulness
Factuality Enhancement
Primary area: interpretability and explainable AI

Abstract: As the modern tools of choice for text understanding and generation, large language models (LLMs) are expected to accurately output answers by leveraging the input context.
This requires LLMs to possess both context-faithfulness and factual accuracy.
While extensive efforts aim to reduce hallucinations through factuality enhancement methods, they also pose risks of hindering context-faithfulness, as factuality enhancement can lead LLMs to become overly confident in their parametric knowledge, causing them to overlook the relevant input context.
In this work, we argue that current factuality enhancement methods can significantly undermine the context-faithfulness of LLMs.
We first revisit the current factuality enhancement methods and evaluate their effectiveness in enhancing factual accuracy.
Next, we evaluate their performance on knowledge editing tasks to assess the potential impact on context-faithfulness.
The experimental results reveal that while these methods may yield inconsistent improvements in factual accuracy, they also cause a more severe decline in context-faithfulness, with the largest decrease reaching a striking 69.7\%.
To explain these declines, we analyze the hidden states and logit distributions for the tokens representing new knowledge and parametric knowledge respectively, highlighting the limitations of current approaches.
Our finding highlights the complex trade-offs inherent in enhancing LLMs.
Therefore, we recommend that more research on LLMs' factuality enhancement make efforts to reduce the sacrifice of context-faithfulness.

Published as a conference paper at ICLR 2025
Is FACTUALITY ENHANCEMENT A FREE LUNCH FOR LLMs? BETTER FACTUALITY CAN LEAD TO WORSE CONTEXI-FAITHFULNSS
Baolong Bit.2 Shenghua Liu Yiwei Wang' Lingrui Mei
Junfeng Fang* Hongcheng Gao? Shiyu Ni Xueqi Cheng 1.2
"CAS Key Laboratory of AI Safety, Institute of Computing Technology, CAS
University of Chinese Academy of Sciences UUniversity California, Merced *University of Science and Technology of China (USTC), Hefei, China /bibaolong23z, liushenghua, nishiyu23z, exq] @ict.ac.en, wangyw.eve evan @ @ewii.lmmmmom.mm meiiinggmi22, gaohongcheng23) ) mails. wul.l..cas.ac.. ac.cn, fjf@ jifemmilluttc.duu...
ABSTRACT
As the modern tools of choice for text understanding and generation, large lan- guage models (LLMs) are expected to accurately output answers by leveraging the input context. This requires LLMs to possess both context-faithfulnes: and factual accuracy. While extensive efforts aim to reduce hallucinations through factual- ity enhancement methods, they also pose risks of hindering context-faithfulme as factuality enhancement can lead LLMS to become overly confident in their parametric knowledge, causing them to overlook the relevant input context. In this work, we argue that current factuality enhancement methods can significantly undermine the context-faithfullesss of LLMs. We first revisit the current factuality enhancement methods and evaluate their effectiveness in enhancing factual accu- racy. Next, we evaluate their performance on knowledge editing tasks assess the potential impact on context- context-faithfulne faithfulness. The experimental results reveal that while these methods may yield inconsistent improvements factual accuracy, they also cause more severe decline in fontextsfaittfullnsss, with the largest decrease reaching a striking 69.7%. To explain these declines, we analyze the hidden states and logit distributions for the tokens representing new knowledge and parametric knowledge respectively, highlighting the limitations of current approaches. Our finding highlights the complex trade-offs inherent in enhancing LLMs. Therefore, we recommend that more research on LLMs' factuality enhancement make efforts reduce the sacrifice of context-faithfulie
INTRODUCTION
Leveraging their powerful capabilities in text understanding and generation, large language models (LLMs) can effectively integrate contextual information to infer and generate truthful responses (OpenAl 2023; Touvron et al.] 2023a b) Song et al.] 2024). However, hallucinations (Huang et al.) al. 2024) significantly undermine the LLMs' reliability, primarily due inadequate adherence to contextual instructions and failure to produce truthful responses (Zhang et al. 2023b).
Generation (RAG) (Fan al. 2024 San- thanam et al. [021] Schick et al.) [024]] Qin et al., (202)), the study of context-faithflnes, in LLMs
With the widespread adoption of Retrieval-
becoming increasingly important (Chen et al., 2022 Li et al.) 2024c] Bi et al., 2024a). This is because they depend on the effective integration of retrieved external information into their generated responses ensure contextual relevance. Therefore, exceptional LLMs must not only produce factual outputs based on their parametric knowledge but also comply with contextual requirements. Context-faithfulet enhances LLMs' compliance with user instructions and improves performance, particularly when the parametric knowledge is insufficient or outdated.
Corresponding author.

---

Published as a conference paper at ICLR 2025
Original Output:
Original Ourput:
Donald Trump
Donald Trump Biden
Biden
other
Input Question:
Injected Context:
Who the president of United States?
Fatuality Enhancement
Joe Biden currently serves the president of the United States.
Enhunced Output:
Enhanced Output:
Donald
Donold Trunp Jon Bidan
Joe
other
other
Comparison of responses from native LLMs and factuality-enhanced LLMs before and after context injection. The enhanced LLMs, due to overconfidence in their parametric knowledge, struggle to integrate information from the context, resulting in incorrect answers.
Figure I:
Prior work (Li et al. 2024a Zhang et al. 2024c) suggests that the cause of factual hallucinations is LLMSs inability to accurately "convey" knowledge during inference, even when it has been effectively "learned" during pretraining. Some emerging works focuses narrowing the gap between "knowing" and "telling" in LLMs to improve LLMs' factuality. These approaches can be divided into two categories: one involves editing internal representations. such attention mechanisms (Li et al. 2024b] Chen et al., 2024) or hidden states (Zhang et al. 2024b). and the other modifies the logits of tokens during decoding (Li al.) 2023b] Chuang et al. 2023 Zhang al. (023a). These methods enable LLMs to output their knowledge more accurately. reducing hallucinations, without altering parameters, as they do not require introducing new factual knowledge through supervised fine-tuning (SFT) OOadia et al. (023) or RLHF (Bai et al. 2022).
However, is this factuality enhancement free lunch for LLMs? We argue that while factuality enhancement can reduce factual hallucinations in LLM outputs, it may lead to excessive confidence in their parametric knowledge, making it difficult for them to comply with contextual information. especially when external knowledge contradicts their parametric knowledge (Petroni et al.) 20200 Si al. 2023: Xie al.) 2024). Figure iillustrates a simple example of the scenario we envisioned. In this work, we first revisit several popular methods for factuality enhancement and evaluate their performance on TruthfulQA (Lin et al., 202T). The results show that while most methods provide some improvement in factuality, some approaches exhibit minimal inconsistent gains. Therefore, suspect that current factuality enhancement methods, despite improving factuality, might negatively greater extent.
impact nantext-fatthullness
To test our hypothesis, we evaluate these factuality enhancement methods on knowledge editing (KE) (Sinitsin et al.) 22200) De Cao et al., [021; Mitchell et al.] [022] Yao al.) (023b) bench- marks, comparing them against the baseline performance of LLAMA2-CHAY
Specifically, we conduct experiments with in-context editing (ICE) (Zhong et al. 2023 Zheng et al., 2023 Bi
et al., 2024c) methods on the MQUAKE datasets. The accuracy of ICE effectively reflects the context- faithulless of LLMs. as it provides outdated or counterfactual contextual knowledge that may contradict the models' parametric knowledge. Experimental results indicate that, compared to the baseline, current popular factuality enhancement methods lead to a significant decline in LLM performance in context-faithfulne 2024b) on 2---MB-2---BBAAA enhancement aids LLMs in accurately conveying factual information, it also introduces the risk of overconfidence, resulting in a substantial setback for the
For example, the editing accuracy of TruthX (Zhang et al.) decreased by as much as 60.3%. This suggests that while factuality
of LLMs.
We explain the decline in context-faithfullet tions of LLMs' outputs, based on knowledge tokens captured by our specially designed algorithm. Compared to the baseline, the hidden states become more concentrated and closely align with the original parametric outputs after inserting contextual information. Additionally, the logits for tokens representing new knowledge decrease, while the logits for parametric knowledge increase.
by analyzing the hidden states and loken-wise distribu-
Altogether, our study highlights the significant risks posed by current factuality enhancement methods, validate and explain their notable decline in context context-faithfulnes faithfulness. This demonstrates the
2

---

Published as a conference paper at ICLR 2025
importance of ensuring that LLMs not only enhance factual accuracy but also maintain strong alignment with the provided context to minimize the occurrence of hallucinations. We strongly advocate for the development of LLMs that prioritize both faithful adherence to context and the accurate conveyance of factual information. Therefore, we recommend future research explore strategies balance these two critical aspects.
PRELIMINARY
This section revisits the core concepts of language modeling (Section 2.1 and knowledge editing
(Section 2.2 of LLMs, with the aim of conducting more in-depth exploration of existing factuality
enhancement strategies (Section B) and their interpretability (Section 5 as well as the use of knowledge editing to assess context- faithfulness (Section
2.1 LANGUAGE MODELING
The current language modeling of LLMs aims to generate text based on the probability distribution of token sequences. Formally, given a sequence of tokens T, {,11, 22,.,, the goal is to predict the next token x by modeling the conditional probability PP
To achieve this, LLMs process the input token sequence through layers of attention (Attn) and feed-forward networks (FFN). The hidden state is computed as follows:
H FFN(Att(Xx:.)
Here, Xet represents the embedded representation of the input tokens Iet, which is passed through Attn to capture contextual relationships, followed by FFN to produce the hidden state.
Then, the model can predict the probability distribution of the next token 2; over the vocabulary set V by applying linear transformation <i-) followed by softmax function:
(( (T<ce) softmax ( (o (HH,), It EV
The softmax function converts the output logits from o o(H;) into probability distribution over the vocabulary V, allowing the model to assign a likelihood to each possible next token. At decoding time, various decoding strategies can be applied each step select the token 2;, with the given predicted distribution of the next token (o (x).. This iterative process continues until the sequence generation reaches designated end token or meets predefined stopping condition.
2.2 KNOWLEDGE EDITING
requires LLMs to effectively adhere to external context. This means they must fully accept new knowledge provided by that context, even if it conflicts with their parametric knowledge acquired during pretraining, which may be insufficient or outdated. Therefore, this paper uses the success rate of KE to assess the context-faithinn faithullls of LLMs, as KE provides counterfactual new knowledge that contradicts the LLMs' own parametric knowledge.
KE aims to efficiently adjust the behavior of the original model foase into post-edit model fe with specific editing descriptors ze. The editing descriptors =e describe a desired change in model behavior and can be represented as 2 = (Iee Ve), where (Ie, Te, Ue) represents triplet such as (US, President, Joe Biden) meaning Joe Biden is the president of US. The ultimate objective of KE is to edit the L.M output that fc (xe,, re) ye while [Base(Ke, re) * ye-
thorough edit should not only modify the relevant knowledge but also update all knowledge affected by this edit in multi-hop relationships. For instance, consider the two-hop fact triple (WWE Velocity, created by, Vince McMahon) and (Vince McMahon, spouse, Linda McMahon). With a fact edit (WWE Velocity, created by, Stan Lee) and additional fact (Stan spouse, Joan Lee), the correctly updated answer should be Joan Lee. We adopt a multi-hop editing task to measure context-faithfllie of LLMs in Section evaluating the accuracy of multi-hop question answering with fact edits. This poses challenges for LLMs in adhering to the editing context due to potential conflicts between new knowledge and their parametric knowledge during the reasoning process.
3

---

Published as a conference paper at ICLR 2025
TruthfulQA
FACTSCORE
Model
Method
MC1 (%)
MC2 (%)
MC3 (%)
Resp. (%)
Facts (ff)
Score (%)
Baseline
37.6
54.6
28.1
37.5
45.7
63.8
DoLa
32.9
60.8
29.5
40.7
48.7
61.3
LLAMA2-
ICD*
46.3
69.1
41.2
36.1
46.6
66.3
7B-CHAT
ITI
37.0
54.7
27.8
41.9
40.8
62.4
TruthX
54.2
73.9
44.3
40.6
43.7
65.3
Baseline
37.7
55.6
28.2
77.0
37.6
52.5
DoLa
37.3
61.8
32.7
55.8
46.7
59.8
LLAMA2-
CD*
28.2
54.9
29.8
74.2
39.8
53.5
13B-CHAT
ICD
45.6
67.5
42.3
46.9
43.4
59.5
ITI
38.9
53.4
29.6
78.9
34.5
55.3
Table I: Experimental results of factuality evaluation on TruthfulQA and FACTSCORF. We conduct experiments with various factuality enhancement methods, including both contrastive decoding and representation editing, on LLAMA2-CHAT Methods marked with belong to contrastive decoding. while those marked with O belong to representation editing. FACTSCORE, % Resp. indicates response ratio of LLMs, while # Facts represents the number of extracted atomic facts per response. Here, red highlights a decrease in factuality compared to the Baseline.
FACTUALITY ENHANCEMENT FOR LLMS
3.1
CURRENT APPROACHEES TO FACTUALITY ENHANCEMENT
SFT RLHF methods Yang et al. 2023a Ovadia et al. 2023 Ouyang al. 2022 Bai al. 2022) enhance the factuality of LLMs s by injecting external training data. However, this approach significantly increases costs and introduces unreliability, potentially jeopardizing the integrity of the model's inherent knowledge. Current methods for enhancing factuality do not modify the underlying structure parameters LLMs; instead, they focus on improving how LLMs convey the information they have already learned to reduce hallucinations. These approaches primarily involve modifying the outputs I.LMs, including the hidden states obtained through Attn and FFNs (eq. (T). as well as
the logits distribution used to predict the next token during the decoding phase (eq. (2). The current
factuality enhancement methods in generation can be classified into the following two categories based on their different approaches to modifying LLM outputs:
Representation Editing Representation editing methods (Burns et al. 2022 et al. al. 2024b Chen al. 2024) enhance factuality by editing the internal representations of LLM outputs. They learn a direction within attention heads or a potential truthful space to modify the attention patterns of the LLM, thereby adjusting the hidden states.
2024b Zhang
Contrastive Decoding Contrastive decoding methods (Li et al. 2023b Chuang et al. 2023 Zhang ct al. 2023a Kai et al. 22044] Bi et al., (202bb) enhance the factuality of LLMs by comparing the output probabilities of expert and amateur models, different layers within the transformer, various tokens, and the outputs of normal and mallucination-injecaa models.
3.2 EVALUATION OF FACTUALITY
We evaluate the performance of popular factuality enhancement methods on the LLAMA2-7B-CUIE and LLAMA2-11 1---111--CA-Y models in terms of factuality of LLMs using the TruthfulQA (Lin et al. 202T) and FACTSCORE (Min et al., 2023 benchmarks. Evaluation on both benchmarks adheres to the settings of previous studies (Chuang et al., 2023 Li et al., 2024b Zhang al.) 2023a). Using TruthfulQA, we employ a multipl-choice task where the LLM selects an answer from various correct and incorrect options, evaluated through multiple-choice accuracy (MC1, MC2, and
4

---

Published as a conference paper at ICLR 2025
Model
Method
3-shot
5-shot
10-shot
COT (10-shot)
Bascline
68.5
78.7
81.3
82.5
DoLLa
52.3 () (2.4)
63.7 (L 180)
66.5 (U 1480
62.5 (4 20.0)
LLAMA2-
ICD
Zhang ct al.
2023a
50.1 269)
61.7 ( 17.0
64.9 1644
60.5 22,0)
7B-CHAT
ITI
Li al.
2024b
51.9
62.2 (1
67.5 1339
65.7
TruthX
Zhang
2024b
26.7 (1.01.1)
32.5 (1 58.)
32.7 (0 (M.9)
26.5 (4 67.99
Baseline
78.3
84.5
86.1 (+)
86.3(
DoLa
Chuang et al.
2023
58.3 20,0
67.7
70.2 15.9)
70.3 16.0)
LLAMA2-
ICD
Zhang al.
2023a
59.5 (1 232)
66.2 (1 1833
72.4 13.7)
67.6 1877
13B-CHAT
ITi
Li al.
2024b
64.3 () 140)
69.8 (L 14.)
74.2 (U 11:99
65.7 20,6)
CD
Li ct al.
2023b
55.2 29.5)
62.6 () 21.9)
67.1 19,00
64.9 (1.4)
Table 2: Experimental results of context-faithfuneet evaluation on MQUAKE dataset. k-shot represents the number of context demonstrations provided, which is detailed in Figure allong with the use of COT. In the subscript (L A), A (%) represents the decrease compared to the baseline.
MC3). For FACTSCORE, the assessment conducted through retrieve+chatGPT methodology. The implementation details of the factuality enhancement methods are in Appendix A
The experimental results of LLM factulity on LLAMA2 CHAT are presented in Table A notable observation is that, compared to the baseline, factuality enhancement methods show improvements across most metrics. Without altering the structure or parameters of LLMss, these methods reduce the gap between "know' and 'tell" knowledge to mitigate hallucinations, making such improvements highly valuable. However, these methods still have significant limitations. The improvement in factuality is modest and quite unstable. The red-marked sections in Table Mindicate that the evaluated methods do not show consistent enhancements across all metrics; in fact, some metrics exhibit declines compared to the baseline. This suggest that these methods for modifying LLM outputs may negatively impact the model's natural generation capabilities.
A contextual demonstrations abbreviated
Question: What is the official language of the country where the creator of WWE Velocity's spouse is a citizen?
Edit: The creator WWE Velocity Stan Lee. The official language of US German.
Thoughts (if COT): WWE Velocity was created by Stan Lee, Joan Lee the spouse of Stanlee, Joan Lee is citizen of US, the official language US German,
Answer: German.
Figure 2: An illustration of the ICE task to evaluate LLMs' context- faitexl-aaithfullnsss. It involves multi-hop hop question answering with corresponding edits, utilizing & contextual demonstrations to guide editing and align output format. The highlighted text represents the expected output from the LLMs, while Thoughts indicates the additional step taken when using Chain of Thought (COT) reasoning.
EVALUATION OF CONTEXT-FATTHFULNER
The factuality enhancement methods increase LLMs' confidence in their parametric knowledge to produce more accurate outputs. However, we argue that this can lead to overconfidence, which reduces their faithfulness to context and consequently introduces new risks of hallucination. validate this suspicion, we evaluates the impact of these factuality enhancement methods on context-faithfulen
4.1 EXPERIMENTAL SETUP
Task We meticulously design the In-Context Editing (ICE) (Zhngg et al. 2023 Zhong et al. 2023 Wang et al. 2024 Bi et al.) 2024d c) task to evaluate the context-fathtu of LLMs. As the most effective KE method currently, ICE prompts L.LMs to reason and generate answers that faithfully
5

---

Published as a conference paper at ICLR 2025
incorporate new knowledge retrieved from edit memory in the form of context. We configure the edit memory to include only question-relevan editing instances, ensuring that all provided editing context influences the LLMs' outputs. We assess question-answering with edits, as this reflects whether LLMSs remain faithful to external context when it confficts with their internal knowledge. Figure 2 provides simple illustration of our task, with additional contextual demonstrations listed in the Appendix D.2
based on the accuracy of
Models and Datasets We use I.LAM2-7B--HA.. and LLAMA 1--MA-2-1-- as baseline models to conduct experiments MQUAKE (Zhong et al. 2023 comparing various factuality en- hancement methods against the natural LLAMA2-
models. MQUAKE provides multi-hop knowledge questions to evaluate knowledge editing counterfactual edits, as introduced in Section 2.2 This verifies whether knowledge has been properly edited and effectively demonstrates LLMs' CD compares the decoding logits of the 13B version of LLAMA2-CHAT with those of the 7B version, using LLAMA2-1 -LAMA2-13B-CIL B-CCIA as the baseline for comparison. Implementation details of the factuality enhancement methods be found in the Appendix A
context-faithfulne
Model
Baseline
DoLa
CD*
ICD
ITI
TruthX
LLAMA2-7B-CHAT
54.5 (-)
39.4 (7.7)
29.8 () 45.3)
31.5 (1.4221
22.6 (1.58)
LLAMA2-13B-CHAT
63.8 (-)
35.6 ( 44.2)
38.6 39.5)
39.5 () 52.2)
34.5 (1.489)
Table 3: Experimental results of context-faithfunnes evaluation using MeLLo method.
4.2
MAIN RESULTS
Based on the above experimental setup, we eval- uate the faithulesss of current factu- ality enhancement methods. The experimen- tal results of ICE on LLAMA2-7B-CHAN and LLAMA2- 1---CHATT are shown in Table As observed, all factuality enhancement methods exhibit a marked decline compared to the base- line performance on the ICE task under any setting. Notably, this decline is not minor, with an average decrease exceeding 20%, and occurs without exception across all methods. Particu- larly in the state-of-the-art TruthX for factuality, the decline reaches as high as 67.9%. This suggests that the potential risks to notext--aithfullesss could be significant.
Dela
Calidee Paeually
CD
m
TrubX
Figure 3: Changes (%) in Fontext-Faithfulness and Factuality of Factuality-Enhancee LLMs.
An interesting observation that the accuracy of all methods, including the baseline, improves as the number of context demonstrations increases. This aligns with the findings of Zheng al. (2023).. suggesting that more faithful examples can enhance the context-faithfulee
of LLMs. Additionally, we find that introducing Chain of Thought (COT) (Wei al., 2022) reasoning into the baseline improves its accuracy. However, the performance of COT in factuality enhancement methods is unstable, with most methods showing a decline compared to those without COT. This may be due to the fact that during the additional COT reasoning process, LLMs with factuality enhancements tend to recall parametric knowledge more, thereby negating the new knowledge from the context. This can be observed in the cases discussed in Section
Figure visualizes the changes in both factuality and context-faithfulnes of these methods compared to the baseline, based on the results of MC2 in T and 3-shot in Table 2 It illustrates that while these factuality enhancement methods provide unstable improvements in factuality, they also lead to more significant decline in the context- faithfulfttsssss of LLMs.
In addition to the aforementioned basic ICE setup, we also employ the advanced KE method MeL.o (Zhong al. 2023) assess LLMs' context-faithullnes. MeLLo can decompose complex multi-hop questions into sequential sub-questions, checking for conflicts with relevant new knowledge
'ee specifically use the released TQUAKF-CF-33K data version for our experiments, which addresses the internal leoge conflicts within the dataset.

---

Published as a conference paper at ICLR 2025
retrieved from edit memory to determine if edits necessary. Therefore, we further validate context- faithfulness using McLLo, as it performs one-hop edits, thereby avoiding hallucinations introduced by multi-hop reasoning. The prompt template and implementation details of MeLLo are provided in the Appendix [.I] The results in Table 3) show that, when using MeI.Lo, which is more suited for realistic editing scenarios, the accuracy of factuality-enhanced L.LMs exhibits more significant decline compared to the previous simple ICE evaluation. Most declines exceed 40% relative to the baseline, with larger models experiencing even more pronounced decreases. All of the editing results that these factuality enhancement methods significantly reduce the accuracy of knowledge editing tasks, highlighting that the current approaches severely impact the faithulles of LLMs.
IN-DEPTH EXPLORATION OF CONTEXT-FAITHFULNNT
The evaluation of context-faithfulness in Section confirms the earlier suspicion that improved factuality leads to a significant decline in context-fattffuness. This section aims to further explain this decline from model interpretability perspective. We investigate the logit distributions (Section (5.2) and hidden state representations (Section 5.3) of relevant knowledge in LLM outputs captured by our algrithm (Section (.T].. and conduct case study (Section 5.4) based on the generated text, to reveal how these factuality enhancement methods impact context-fathfulness.
Algorithm 1 Knowledge Token Capturing
Require: The LLM generates a token sequence of length n. V: vocabulary of LLM, P. in (P1, Ps. Pa): logits distribution of tokens, H, in (H,, H2, .... H,): hidden states of tokens, Snew: string of new knowledge, Span:: string of parametric knowledge.
Ensure: Captured knowledge logits and hidden states Pnew, Ppeca Hnews Hpam
1: Initialize Paew. P Paraa -Noee
2: Scom = -COM((Seee Spara)
Identify common substrings
3: for PP in (P1, P2. .... Pn)) do
4: for token I; in V do
>. Sort by P; in descending order
5:
2; - x;
Decode Hy to string a'
6:
if w/, in Samm and Pnew P para None: break
r'j indistinguishable
7:
if x in Snow and P new None: Phew t Pe.ji Hnow + Hi
Capture new knowledge
8:
if x in Spara and par None: P P'paa Pi.j H pam t- H;
Capture para knowledge
9:
end for end for
11: return Pnew. P para+ Hnews H para
5.1 DISTINGUISHING NEW AND PARAMETRIC KNOWLEDGE IN LLM OUTPUTS
Our objective is to identify the parts of LLM outputs that distinguish between the knowledge acquired from context and the parametric knowledge embedded within the LLMs, rather than analyzing repetitive meaningless outputs. For example, consider expected L.MM output with injected context, such as "Answer: United States", " compared to the original parametric output without the injected context, which would be "Answer: United Kingdom", In this case, we should not capture " "Answer:" as it holds no factual meaning, nor should we focus on "United" since is repetitive and does not reflect the difference. Instead, our focus should be on capturing tokens with clear factual significance those that can differentiate between the new knowledge introduced by context and the parametric knowledge inherent to the model. this example, tokens like "Kingdom" serve the key markers of distinction, effectively highlighting the critical divergence between the contextual information and the model's existing knowledge.
To achieve this, we design novel knowledge token capturing algorithm, with details presented in Algorithm]] [ The algorithm captures the nighest-probabilit tokens that differentiate new and parametric knowledge by matching the decoded tokens with the respective knowledge strings. We conduct experiments on the MQUAKE MQUAKE-STUBBORN dataset provided by Bi et al. (2024b), which effectively highlights the distinction between new knowledge and parametric knowledge. Specifically, we collect 2,000 question-answ instances, each consisting of the output from LLMs prior to injecting context containing new knowledge, as well as the output after the injection.
7

---

Published as a conference paper at ICLR 2025
New Knowledge
Parametric Knowledge
Avenug Lagits
40-
32.42
30
19AT
14.15
Baseline
DoLa
111
KD
ThraX
Boselize
DoLa
11
KD
CD
1laMA-2-Chat-7
1LaMA-2-Cha-134
Figure 4: Average logits of tokens representing and parametric knowledge after context injection.
5.2 ANALYSIS LOGITS DISTRIBUTION
To further investigate L.MM context-faitfullnes from the perspective of logits distribution (eq. 2 we capture the tokens corresponding to both the in-context new knowledge and the parametric knowledge after context injection. The average probabilities of these tokens for the factuality enhancement methods and baselines are shown in Figure We observe that, compared factuality enhancement methods show a decrease in the probability of new knowledge tokens, while the probability of parametric knowledge tokens increases. This provides clear explanation for the reduction in context-fails context-faithuulnes: discussed in Section These methods effectively boost the retention of parametric knowledge but at the cost diminished faithfulness external context. Furthermore, the difference in probabilities between new and parametric knowledge correlates well with the editing accuracy in Table 2 where larger differences lead better editing performance, while TruthX's s poor editing is explained by the dominance of parametric knowledge new knowledge.
the baseline, all
2.5
Dematy
Donsity
ThunX
1.5
0-
0.0
0.0
0.4
0.6
0.8
1.0
0.4
0.5
0.8
(78)
I. Logits Distribution of New Knowledge
(138)
Beseloe Jola
Budiae
0.30
DoLa
Density
0.25
0.20
0.10
0.15
0.10-
0.05
0.05
0.00
0.00
15
Distribution New Knowledge
(138)
Figure 5: Statisties of new knowledge distribution for both logits and rank on LLAMA2-CHAT (left) and -LLMA2--3B--HHA (right). Rank is recorded by capturing the position of the new knowledge token within the vocabulary V. based on its logits ranking.
Results in figure 5 provides a deeper statistical analysis of the logits associated with new knowledge. The baselines of the LLAMA2-CHAI models consistently exhibit higher concentration in the probability range of 0.8 to 1.0, with top token ranks to 5) clearly dominating. In contrast, factuality enhancement methods shift the distribution of new knowledge tokens towards lower probability ranges, indicating a diminished confidence in-context knowledge compared to the baselines.
5.3 ANALYSIS HIDDEN STATES
Section 3 shows that all factuality enhancement methods impact the logits distribution of the output, resulting a decline in context- faithfulness. context-faithfulness Some methods also aim to improve factuality by editing the internal representations (eq. (T)) of L.Mss, which requires analysis based hidden states. We
8

---

Published as a conference paper at ICLR 2025
Hidden States before Injection
Hidden States after Injection
40
40
20
0
-30
-20
-10
-20
-00
60
-30
-10
-30
10
-10
LLaMa2-7B-Chat
LLaMa2-7B-Chhi w/ ITI
LLaMa2-7B-Chat TruthX
Figure 6: Distribution of reduced hidden states for new knowledge before and after context injection. The dashed circles represent the 95% confidence intervals, showing that factuality enhancement methods lead post-injection convergence towards the pre-injection distribution.
similarly use theknowledge token capturing algorithm to collect these crucial hidden states. Since the contrastive decoding method does not alter internal representations, our evaluation focuses the ITT (Li et al. 2024b) and TruthX (Zhang et al., (024b) methods. We used t-SNE to reduce the dimensionality of the hidden states representing new knowledge in LLAMA2-7B-CIAA from the original 4096 dimensions to 2 dimensions, with the perplexity set to 15.
Figure 6 presents the distribution of hidden states before and after context injection for these repre- sentation editing methods. We find that the distribution of the reduced hidden states after context injection broader and more dispersed compared to before the injection. This indicates that context injection alters the distribution of knowledge representations, leading them to deviate from their original factuality, where the hidden states prior to injection represent the parametric factuality inherent in the LLMs. Notably, compared to the significant difference observed between the pre- and post-context injection hidden states in the baseline, the differences for ITI and TruthX are much smaller. The hidden states of new knowledge after injection are more confined and closer to those before the injection. Especially, TruthX exhibits a distribution that closely resembles the original range, which explains its significant decline in context-faithfulne the FFN and Attn representations, making them more resistant to incorporating new knowledge and more firmly tied to the model's parametric knowledge. Overall, these representation editing methods tend to limit the natural dispersion of hidden states after context injection. As a result, the output decoded from the constrained hidden states remains more aligned with the original parametric knowledge, rather than incorporating the new contextual knowledge effectively.
This is because it modifies both
5.4
CASE STUDY
We conduct a case study on the outputs factuality enhancement methods in the ICE task (Section 4) We classify the context-unfaithful outputs into three categories: ignoring context, denying context, and misformatting context, with illustrative examples for each type provided in Table 4 Beyond knowledge conflicts causing the context to be overlooked contradicted, failing properly follow Additional examples
the context demonstration format also indicates a lack of context-faithfulne of real context-unfaithuu outputs are provided in Appendix E for further examination.
RELATED WORK
Hallucinations (Kaddour et al. 2023 Tonmoy al., 2024 Wang et al.. 2023, Mei et al. 2024b ch have garnered significant attention due to their pronounced side effects,
Hallucinations in LLMs
leading to the generation unreliable content (Gunjal et al., 2024 Huang et al.] 2023 Liu et al.. 2024 Zhang et al.) 2024a). These hallucinations can arise from various sources different stages of the LLM lifecycle. The primary reasons include conflicts between the context provided and the
LLMs' inherent memory, well as the models' inability to accurately generate knowledge that they have already acquired during pretraining (Zhang et al. 2023b] Chen et al. 2023). This paper primarily explores the interplay between these two aspects.

---

Published as a conference paper at ICLR 2025
Question:
What the official longuage of the country where the creator of WWE Velocity's 's spouse is a citkzen?
Injected Context: (k context demonstrations) Edit: The creator WWE Velocity is Ston Lee. The official language of US Germon.
Caithul output
Ignoring context
Denying context
Misformatting context
Thoughts: WWE Velocity was cre-
Thoughts: WWE Velocity was cre- ated by Vince McMalon. Vince McMahon''s spouse is McMa-
Thoughts: W WWE Velocity was
The othicial language is German, bused the provided context. How WWE created by Vince McMdahon and the official language of the U.S. is Ein glish. (Do follow the contect demontttation format)
aled Stan Lee. Joan Lee the
ated Stun Lec. Joan Lee the
Lee izen of US, the official language of US is German. Answer: German.
sponse Stumlee
spouse SuntLee. Joan
hon, Linda McMahon is citzzen of US. the official language of US is English. Answer: English.
isen of US, the official language of the US not German: it primanlly English Amower: English.
Table 4: Case study of context-unfaithul outputs from factuality-ehaaneed LLMs. The main categories include ignoring, denying new knowledge from the context, and not following the context format in responses. Red text highlights the unfaithful parts or reasons.
Knowledge Confficts Various tools (Nakano et al. 2022 Yao et al. 2023a Qin al. 2024) and
reurieval-augmented methods (Guu et al. 2020 Izacard & Grave 2021 Huang et al. 2025). such as ChatGPT Plugins and New Bing. have been introduced as effective strategies supply external knowledge evidence. However, the integration of external knowledge is not without challenges, as it can sometimes clash with the LLMs' parametrie knowledge (Petroni et al., 2020; Xie al.) 2024 Mei et al. 2024a Ni et al. 2025), resulting in inconsistencies or unreliable outputs, especially when LLMs exhibit overconfidence in their inherent parametric knowledge. This conflict between external sources and the internal knowledge stored in LLMs continues to pose significant challenges for ensuring reliable model performance. Therefore, we argue that current factuality enhancement methods may actually reinforce the model's reliance on its own parametric knowledge, exacerbating knowledge conflicts and making it more difficult for LLMs to adapt to or follow the injected context.
Knowledge Editing Knowledge editing (KE) Yao et al. 2023b) has been proposed to update outdated model knowledge, ensuring accurate responses to current queries. Generally, Current KE approaches can be divided into following two categories. Model editing (Meng et al.) 2022a b
Mitchell et al. 20222 Yao et al. 2023b] Zhang et al.; Li et al., 2025) focuses on identifying and
modifying internal components, such as neurons or parameter matrices. In-context editing (Madaan
al. 2022: Zhong et al. 2023 Zheng et al., 2023 Wang et al., 2024 Bi et al., 2024c) retrieves
relevant knowledge from external memory during inference, allowing LLMs to utilize updated
information dynamically.
7 CONCLUSION AND DISCUSSION
In this paper, we conduct an in-depth exploration of popular factuality enhancement methods and their implications for LLMs. We argue that existing approaches to enhancing factuality can significantly undermine the context-faithfuner tasks and knowledge editing scenarios, we demonstrated that while these methods may yield unstable improvements in factual accuracy, they also lead to substantial decrease in editing accuracy.
of LLMs. Through evaluations factual question-answeniie answering
By examining the logits distribution and hidden states, we gained insights into the underlying causes of this decline. Our experimental results further validate our concerns: improved factuality achieved through these methods can indeed result in diminished context-faithfunnes of LLMs. This finding highlights the complex trade-offs inherent in enhancing LLMs.
Ultimately, high- perfforeff LLM should possess both factual accuracy and context- context-fathhfunnes Therefore. we recommend that future research should focus on developing strategies to controllably and effectively balance these two critical aspects. By addressing this duality, we can work towards creating L.LMs that not only provide accurate information but also maintain a robust adherence to contextual cues, enhancing their overall reliability and usefulness.
ETHICS STATEMENT
Ethical considerations are of utmost importance in our research endeavors. In this paper, we consci- entiously adhere to ethical principles by exclusively utilizing open-source datasets and employing
10

---

Published as a conference paper at ICLR 2025
models that are either open-souree widely recognized in the community, Moreover, our proposed method is designed to ensure that the model does not produce any harmful or misleading informa- tion. We are committed to upholding ethical standards throughout the research process, prioritizing transparency, and promoting the responsible use of technology for the betterment of society.
ACKNOWLEDGEMENTS
This paper is partially supported by National KeyR&D Program of China (No. NN..2023YFC33353003 and the National Science Foundation of China under CrantNo.U21B22. and 62377043.
REFERENCES
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.0586) 2022.
Baolong Bi, Shaohan Huang, Yiwei Wang. Tianchi Yang, Zihan Zhang, Haizhen Huang, Lingrui Mei. Junfeng Fang, Zehao Li, Furu Wei, et al. Context-dpo: Aligning language models for fontext-faithfulness.. arviv preprint arXiv:2412.15280, 2024a.
Baolong Bi, Shenghua Liu, Lingrui Mei, Yiwei Wang, Pengliang Ji, and Xueqi Cheng. Decoding by contrasting knowledge: Enhancing Ilms' confidence on edited facts, 2024b.
Baolong Bi. Shenghua Liu, Yiwei Wang, Lingrui Mei, Hongcheng Gao, Junfeng Fang, and Xueqi Cheng. Struedit: Structured outputs enable the fast and accurate knowledge editing for large language models. arXiv preprint arXiv:2409. 10132, 2024c.
Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, Hongcheng Gao, Yilong Xu, and Xueqi Cheng. Adaptive token biaser: Knowledge editing via biasing key entities. arXiv preprint arXiv:2406. 12468, 2024d.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827, 2022.
Fei-Long Chen, Du-Zhen Zhang, Ming-Lun Han, Xiu- Chen, Jing Shi. Shuang Xu, and Bo Xu. Vlp: A survey on vision-language pre-training. Machine Intelligence Research, 20(1):38-56, 2023.
Hung-Ting Chen, Michael JQ Zhang, and Eunsol Choi. Rich knowledge sources bring com- plex knowledge conflicts: Recalibrating models to reflect conflicting evidence. arXiv preprint arXiv:2210.13777 137001, 2022.
Zhongzhi Chen, Xingwu Sun, Xianfeng Jiao, Fengzong Lian, Zhanhui Kang, Di Wang, and Cheng- Zhong Xu. Truth forest: Toward multi-scale truthfulness in large language models through intervention without tuning, 2024. BRLpattp://acxiv..oom
abs wta.a///3/.... 2312.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309.03883, 2023.
Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models, arXiv preprint arXiv:2104.08166 2021.
Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting Ilms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6491-6501, 2024.
Anisha Gunjal, Jihan Yin, and Erhan Bas. Detecting and preventing hallucinations in large vision language models. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 18135- 18 1814, 2024.
11

---

Published as a conference paper at ICLR 2025
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming- Wei Chang. Realm: Retrieval- augmented language model pre-training. 2020. URL attps://arxxi www.iiioor//aa.... 2002
08909
Lei Huang. Weijiang Yu. Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions, 2023.
Pengcheng Huang. Zhenghao Liu, Yukun Yan, Xiaoyuan Yi, Hao Chen, Zhiyuan Liu, Maosong Sun, Tong Xiao, Ge Yu, and Chenyan Xiong. Pip-kag: Mitigating knowledge conflicts in knowledge- augmented generation via parametric pruning, 2025. URL
15543
Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. 2021. URL
01282
Jean Kaddour, Joshua Harris, Maximilian Mozes, Herbie Bradley, Roberta Raileanu, and Robert McHardy. Challenges and applications of large language models. arXiv preprint arXiv:2307.100 101669 2023.
Jushi Kai, Tianhang Zhang, Hai Hu, and Zhouhan Lin. Sh2: Self- eeff-highlighted hesitation helps you decode more truthfully. arXiv preprint #rXiv:2401.05000 2024.
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian- Nie, and Ji-Rong Wen. Halueval: A large- scale hallucination evaluation benchmark for large language models, 2023a. URL https
Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Wayne Xin Zhao, Jian- Yun Nie, and Ji-Rong Wen.
The dawn after the dark: An empirical study on factuality hallucination in large language models. arXiv preprint #xXv:2401.00300. 2024a.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from language model, 2024b. URL https
/arxiv.org/abs////
03341
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Fisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp. 12286- 12312, Toronto, Canada, July 2023b. Association for Computational Linguistics. doi: 10.18653/ v1/2023.acl-long long-oog....88.. URL /Rt/aatsss//acaaaatooooaaao
org 2023 ac1-long. 687
Yuepei Li, Kang Zhou, Qiao Qiao, Bach Nguyen, Qing Wang, and Qi Li. Investigating context- faithfulness in large language models: The roles of memory strength and evidence style. arXiv preprint 2rXvv:2409.10955 2024c.
Zherui Li, Houcheng Jiang, Hao Chen, Baolong Bi, Zhenhong Zhou, Fei Sun, Junfeng Fang, and Xiang Wang. Reinforced lifelong editing for language models. arXiv preprint arXiv:2502.05759, 2025.
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.0795 2021.
Fang Liu, Yang Liu, Lin Shi. Houkun Huang, Ruifeng Wang, Zhen Yang. and Li Zhang. Exploring and evaluating hallucinations in Ilm-powered code generation. arXiv preprint arXiv:2404.00971. 2024.
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West. Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. Generated knowledge prompting for commonsense reasoning. arXiv preprint arXiv:21 10.00387, 2021.
Aman Madaan, Niket Tandon, Peter Clark, and Yiming Yang. Mesory-assisted prompt editing to improve gpt-3 after deployment. arXiv preprint arXiv:2201.066 206099 2022.
12

---

Published as a conference paper at ICLR 2025
Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, and Xueqi Cheng. Slang: New concept comprehension large language models, 2024a.
Lingrui Mei, Shenghua Liu, Yiwei Wang, Baolong Bi, Jiayi Mao, and Xueqi Cheng. not aligned" is not" malicious": Being careful about hallucinations of large language models' jailbreak. arXiv preprint arXiv:2406.1 1668, 2024b.
Lingrui Mci, Shenghua Liu, Yiwei Wang, Baolong Bi, Ruibin Yuan, and Xueqi Cheng. Hiddenguard: Fine-grained safe generation with specialized representation router, 2024c. URL https://
wexivvoor//bao 2410 02684
Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 08:17399-17777 2022a.
Kevin Meng, Arnb Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229, 2022b.
Sewon Min, Kalpesh Krishma, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit lyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. arXiv preprint arXiv:2305. 14251, 2023.
Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. Memory- based model editing at scale. In International Conference Machine Learning, pp. 15817-15831. PMLR, 2022.
Reiichiro
Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Floundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman, Webgpt: Browser-assisted guestion-answering with human feedback, 2022. URL https:://axxt
09332
Shiyu Ni, Keping Bi, Jiafeng Guo, Lulu Yu, Baolong Bi, and Xueqi Cheng. Towards fully exploiting Ilm internal states to enhance knowledge boundary perception, 2025. URL httts:://arx:
org abs//ab/// 2502. 11677
OpenAl. Gpt-4 technical report, 2023.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Oded Ovadia, Menachem Brief, Moshik Mishaeli, and Oren Elisha. Fine-tuning or retrieval? comparing knowledge injection in llms. arXiv preprint arXiv:2312.05934 2023.
Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel. Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. How context affects language models' factual predictions, 2020. URL
org/ alss 2005. 04611
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng. Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang. Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li, Ziwei Tang, Jing Yi, Yuzhang Zhu, Zhenning Dai, Lan Yan, Xin Cong. Yaxi Lu, Weilin Zhao, Yuxiang Huang. Junxi Yan, Xu Han, Xian Sun, Dahai Li. Jason Phang, Cheng Yang, Tongshuang Wu, Heng Ji. Zhiyuan Liu, and Maosong Sun. Tool learning with foundation models, 2024. URL https: satps://aat /tt::v/axxvvvv / abs 2304 08354
Keshav Santhanam, Omar Khattab, Jon Sead-Falcon, Christopher Potts, and Matei Zaharia. Colberty2: Effective and efficient retrieval via lightweight late interaction. arXiv preprint urXiv:2112.0144
2021.
Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Published as a conference paper at ICLR 2025
Chenglei Si. Zhe Gan, Zhengyuan Yang. Shuohang Wang. Jianfeng Wang, Jordan Boyd- and Lijuan Wang. Prompting gpt-3 to be reliable, 2023. URL https::/axxxi....oommmm wttpaatwwww.aaaaaaaaaoaaaaaatttt wrgpabs..rraaaaae.aeeaoeo abs 2210 09150
Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitriy Pyrkin, Sergei Popov, and Artem Babenko. Editable neural networks. arXiv preprint arXiv:2004.00345 2020.
Zezheng Song, Jiaxin Yuan, and Haizhao Yang. Fmint: Bridging human designed and data pretrained models for differential equation foundation model. arXiv preprint arXiv:2404. 146888 2024.
Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. Recitation-augmente language models. arXiv preprint ar xiiv:2210.0029900 2022.
Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher D Manning, and Chelsea Finn. Fine-tuning language models for factuality. arXiv preprint arXiv:2311.088 1.08401, 2023.
S. M. Towhidul Islam Tonmoy, S M Mehedi Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. comprehensive survey hallucination mitigation techniques in large language models, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models. CoRR, abs/2302. 13971, 2023a. doi: 10445500/RXXV..002..333 139711 //doi.org /00 48550/arXXis 2302 13971
URL https
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cris- tian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Aune Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuming Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023b.
Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Cheng Jiayang, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, et al. Survey on factuality in large language models: Knowledge, retrieval and domain-speciffccty. arXiv preprint arXiv:2310.0752,, 2023.
Yiwei Wang, Muhao Chen, Nanyun Peng, and Kai-Wei Chang. Deepedit: Knowledge editing as decoding with constraints. arXiv preprint arXiv:2401.10471, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et Chain-of-thought prompting elicits reasoning large language models. Advances in neural information processing systems, 15:24824-24887 2022.
Zhengxuan Wu, Attieus Geiger, Aryaman Arora, Jing Huang, Zheng Wang, Noah D. Goodman, Christopher D. Manning, and Christopher Potts. pyvene: A library for understanding and improving pytorch models via interventions, 2024. URL
wtt.o/ggg/b///..o/ee/ee/ee 07809
Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts, 2024. URL
https://aat /ttp:://aaxivvvi wwt.///.a.......oo abs 2ee.///33121010001 13300
Linyao Yang, Hongyang Chen, Zhao Li, Xiao Ding, and Xindong Wu. Chatgpt is not enough: Enhancing large language models with knowledge graphs for fact-aware language modeling. arXiv preprint arxiv:200.. 1489, 2023a.
14

---

Published as a conference paper at ICLR 2025
Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu. Alignment for honesty. arXiv preprint arXiv:2312.07000, 2023b.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.
React: Synergizing reasoning and acting in language models, 2023a. URL httpt:://axxiiiiiii.
org/aba/ 22 10. 03629
Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. arXiv preprint arXiv:2305. 2023b.
Duzhen Zhang, Yahan Yu, Chenxing Li, Jiahua Dong, Dan Su, Chenhui Chu, and Dong Yu. Mm-llms: Recent advances in multimodal large language models. arXiv preprint arXiv:2401.13601 2024a.
Shaolci Zhang. Tian Yu, and Yang Feng. Truthx: Alleviating hallucinations by editing large language models in truthful space, 2024b. URL hitps:
220024422222 178111
Tianyu Zhang, Junfeng Fang, Houcheng Jiang. Baolong Bi, Xiang Wang. and Xiangnan He. Explain- able and efficient editing for large language models. In THE WEB CONFERENCE 2025.
Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Lifeng Jin, Linfeng Song, Haitao Mi, and Helen Meng. Self-alignment for factuality: Mitigating hallucinations llms via self-evaluation. arXiv preprint arXiv:2402.09267, 2024c.
Yue Zhang, Leyang Cui, Wei Bi, and Shuming Shi. Alleviating hallucinations of large language models through induced hallucinations. arXiv preprint 1rXiv:2312.15710, 2023a.
Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. Siren's 's song in the ocean: A survey on hallucination in large language models, 2023b.
Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. Can we edit factual knowledge by in-context learning? arXiv preprint arXiv:2305.12744 2023.
Zexuan Zhong, Zhengxuan Wu, Christopher D Manning, Christopher Potts. and Danqi Chen. Mquake: Assessing knowledge editing language models via multi-hop questions. arXiv preprint arXiv:2305. 14795, 2023.
15

---

Published as a conference paper at ICLR 2025
A DETAILS OF THE FACTUALITY ENHANCEMENT
In this paper, categorize factuality enhancement methods into four key groups: Representation Editing, Contrastive Decoding, Factuality Prompting, and Factuality Training, with brief overview of each below.
Representation Editing Representation editing methods (Burss et al. 2022 et al. 2024b Zhang
et al. 2024b Chen al., 2024) enhance factuality by editing the internal representations of LLM outputs. They learn a direction within attention heads or a potential truthful space to modify the attention patterns of the LLM, thereby adjusting the hidden states.
Contrastive Decoding Contrastive decoding methods (Li et al. 2023b Chuang et al. 2023 Zhang
al. 2023a; Kai al. [2024] Bi al. (024b) enhance the factuality of LLMs by comparing the output probabilities of expert and amateur models, different layers within the transformer, various tokens, and the outputs of normal and hallucination-innected models.
Factuality Prompting Factuality prompting methods enhance the factuality of LLMs' final outputs without modifying the model or its representations. Instead, they rely carefully designed external prompts (Sun et al. 2022) or multi-step reasoning processes (Liu et al. 202T).
Factuality Training Factuality training methods improve the factuality of LLMs by leveraging tech-
niques such as supervised fine-tuning (SFT) (Yang et al., 2023a Ovadia et al. 2023), reinforcement learning with human feedback (RLHF) (Ouyang al. [2022] Bai et al., 2022), or direct preference This process involves injecting new
optimization (DPO) (Tian et al. 2023 Zhang et al. (024c).
knowledge and modifying the model's 's inherent parameters.
This paper focuses on evaluating and analyzing popular factuality enhancement methods. We provide detailed overview of these approaches, along with the implementation details used in our experiments.
CD. Contrastive decoding (CD) (Li al.) 2023b) leverages the distinctions between expert and amateur LMs of varying sizes by selecting tokens that maximize the difference in their lieeiiiisss.sds. Consequently, factual knowledge that remains unlearned by the weaker amateur model is highlighted by contrastive decoding in the stronger expert model to enhance factuality.
DoLa. DoLa (Chuang et al. 2023 leverages modular encoding of knowledge to magnify factual knowledge within an LM through contrastive decoding approach. In this method, the next-word probability output is derived from the disparity in logits between a higher layer and lower layer. By accentuating the knowledge from higher layers and diminishing that from lower layers, LoRa aims to reduce factual hallucinations.
ICD. Zhang et al. 202.3a) first constructs factually weak LLM by inducing hallucinations from original L.Ms,, and then penalizes these induced hallucinations during decoding to enhance the factuality of the generated content. Specifically, ICD determines the final next-token predictions by amplifying the predictions from the original model and downplaying the induced untruthful predictions via contrastive decoding.
ITL Inference- Time Intervention (ITI) Li et al. 2024b) first identifies a sparse set of attention heads with high linear probing accuracy for truthfulness, as defined by the TruthfulQA benchmark. Then, during inference, it shifts activations along these truth-correlated directions. This process is repeated autoregressively until the entire answer generated.
TruthX TruthX (Zhang et al. 2024b) uses an autoencoder to map the representations of LLMs into both semantic and factual latent spaces, and applies contrastive learning identify the correct editing direction within the factual space. During inference, TruthX enhances the factuality of LLMs by editing their internal representations in the factual space.
16

---

Published as a conference paper at ICLR 2025
RECITE RECITE (Sun et al. 2022) enhances factual accuracy by sampling and reciting relevant passages from LLMs' own memory before generating final answers, without relying on external retrieval. This paradigm proves effective for showledge-intensing NLP tasks.
Conf-Num/Verb Honesty alignment (Sun et al. 2022) is an SFT-based method that aligns LLMs with honesty, ensuring they refuse answer when lacking knowledge while avoiding excessive conservatism. This approach defines honesty inspired by the Analects of Confucius, develops metries to measure alignment progress, and implements a flexible training framework with efficient fine-tuning techniques.
Self-Eval Self-Eval (Sun et al. 2022) introduces Self-Eval, prompting the LLM to validate its out- puts using internal knowledge, and Gelf-Knowledge Tuning (SK-Tuning), which improves confidence estimation and calibration. Fine-tuning with self-annotated responses via DPO significantly boosts factual accuracy
We employ two different sizes of LLMs. LLAMA2-7B-CUME and LLAMA2-13B-CHE
using the unchanged decoding strategy the baseline. Consistent with prior studies, we apply robust factuality enhancement strategies. For DoLa, we configure early-exit layers for both sizes of the LLAMA2-CHAT model according to the guidelines provided in P For CD, we utilize the 13B and 7B LLAMA2-CIIA models as the expert and amateur models, respectively. For ICD, we utilize hallucination-inneed finetuned LLAM A 7B-BCHHTT model. We load this model, which was finetuned on selected samples from HaluEval (Li et al. (023a) and is provided by as the amateur and
model, with LLAMA2-7B utilize the Boanest-llama2-chee model with activation differences, implemented via pyvene (Wu et al. 2024). available at along with the open-source models available at 5] For TruthX, we download its ALAMA2-788 7B-CHAT version from T and deploy it following the methodology outlined by Zhhang et al. (2024b). During inference, we align all factuality enhancement methods with the baseline settings, including temperature=0.9, top-x=0.95, and others. For RECITE, we evaluate integrating counter-face editing context into the original QA task. For Conf-Num/Verb.. We evaluate using the open-source fine-tuned enhanced models, Conf-Num and Conf- Verb, available at [ and respectively. For Self-Eval, we dpo the self-alignment version by collecting data based on the provided pipeline
1-MMMA2-1BB-HHEAE serving as expert counterparts. For ITI.
its performance on the ICE task using the designed prompt templates and pipeline 7
B COMPLETE EXPERIMENTAL RESULTS
The complete experimental results for the ICE task, evaluating the contcxt-laithfulness of LLMs, are presented in Table A These results encompass evaluations of the natural LLMs (baseline) well as fact-enhancement methods, including contrastive decoding, representation editing, prompt-based approaches, and training- based approaches.
C FUTURE CONSIDERING
Addressing the decline in context faithfulness, our experiments reveal that all current factuality enhancement methods--hether training-based, inference-based, or compromise context faithfulness. We strongly advocate for future research focus on balaneing factuality and context faithfulness to ensure the reliable development of large language models. We provide recommendations for future directions to address the loss context faithfulness caused by factuality enhancement, based on the interpretive analysis in Section
https:
17

---

Published as a conference paper at ICLR 2025
Model
Method
3-shot
5-shot
10-shot
COT (10-shot)
Baseline 6
68.5 (-)
78.7 e
81.3 6
82.5 (-)
DoLa
Chuang et
2023
52.3 (1:22.4
63.7 ( 15.0)
66.5 (1 14.8)
62.5 (0,00))
ICD Zhang et al. 2023a
50.1 (L,66)
61.7 ()
64.9
60.5
IT1 Li et al.
2024b
51.9 (1 21.80
62.2 ( 16.5)
67.5 ( 13,5)
65.7 (3 168)
LLAMA2-
TruthX x
Zhang al..
2024b
26.7 (L 61.1
32.5 (1 58.0
32.7 (4
26.5 (1 67.9)
7B 7B-CHAT
RECITE
(Sun el al.
2022
48.8 (L 288)
60.7 () 22,9)
62.5 23.1)
63.2 (3 233)
Conf-Num
YYng
202.3b
40.6 (L 4877)
46.2 (1 41.2)
48.9 () 39.9)
50.2 (1.3)
Conf- Verb
Yang ct al.
2023b
47.9 (L 30.0
53.7 ( 51.7)
56.2
57.3 30.5
Sclf-Eval
Zhang ct al.
2024cc
50.2 (L 267)
56.3 () 28.4)
57.8 () 28.99
59.3 (4 28.1)
Baseline ())
78.3 (4)
84.5 (4)
86. (4)
86.3 (
DoLa Chuang al..
2023
58.3 (L 20.0)
67.7 () 16.8)
70.2 (1 18.9)
70.3 (4 16.0)
ICD Zhang et al. 2023a
59.5 2321
66.2 (1 18.3)
72.4
67.6 (87)
ITT
Li al.
2024b
64.3 (L 14m
69.8 () 14.7)
74.2 () 11.99
65.7 (2 200))
LLAMA2-
CD
Li et al.
202.b
55.2 29.5
62.6 (L 21,9)
67.1
64.9
13B-CHAT
RECITE
(Sun et al.
2022
45.3 (1 42.1
52,9 () 51.6)
55.6 ( 35.4)
57.2 (377
Conf-Num
Yang al.
2023b
42.5 (L 457)
44. 1 (L 46.3 (1
47.5 44.9
Conf-Verb*
Yang et al.
2023b
50. (L mmm
51.6 () 38.9)
52.7 38.7)
53.8 (3 (7.77
Sclf-Eval Zhang ct al.
2024c
48.5 (L 38.1)
53.4 () 96.8)
55.8 (1 85.3)
56.2 (1 34.9)
Table 5: Experimental results of context-faithfulness evaluation on MQUAKE dataset. k-shot represents the number of context demonstrations provided, which is detailed in Figure 2 along with the use of COT. In the subscript (1 A), A (%) represents the decrease compared to the baseline. Methods marked with belong to contrastive decoding, 9 denote representation editing, v indicate prompt-based approaches, and represent training- approaches.
Specifically, since factuality enhancement methods amplify the probabilities of parametric knowledge tokens, making it more difficult for contextual knowledge tokens to surpass them, future work could focus on capturing and boosting the probabilities of context-relevant tokens improve context faithfulness. Additionally, as factuality enhancement methods restrict the dispersion of hidden state distributions, thereby limiting the model's ability to integrate new contextual knowledge, future research could explore controllable representation editing techniques regulate this limitation and enable a broader and more flexible hidden state distribution even after factuality enhancement. We believe these insights offer valuable guidance for addressing the trade-offs between factuality enhancement and context faithfulness in future research.
D CONTEXT PROMPTS IN KE TASK
D.1 TEMPLATE FOR MELLO
In Section 4 we use MeLLo (Zhong et al. 2023) to conduct our experiments for context- faitext--aithuulness. with the results presented T MeLLo first decomposes multi-hop question into subquestions during LLMs inference, and then prompts the L.LMs provide tentative answers to these subquestions. Next, it self-checks their compatibility with edited facts by retrieving edit demonstrations from the knowledge base, thereby maintaining adjusting them accordingly. Table 6 provides the prompt templates used for McLLo.
D.2 CONTEXT DEMONSTRATIONS USED IN ICE
We provide the context demonstrations used in our ICE experiments (Table which were omitted in Figure 2 for brevity. Here, the green text represents the expected context-faithfu output.
Question: What is the capital city of the country of citizenship of Ivanka Trump's 's spouse?
Edit: Jared Kushner is a citizen of Canada. Answer: Ottawa.
18

---

Published as a conference paper at ICLR 2025
Question: What is the capital city of the country of citizenship of Ivanka Trump's spouse? Subquestion: Who is Ivanka Trump's spouse?
Generated answer: Ivanka Trump's spouse Jared Kushner.
Retrieved fact: David Cameron married to Samantha Cameron.
Retrieved fact does contradict generated answer. Intermediate answer: Jared Kushner
Subquestion: What is the country of citizenship of Jared Kushner? Generated answer: The country of citizenship of Jared Kushner is United States. Retrieved fact: Jared Kushner a cilizen of Canada.
Retrieved fact contradicts to generated answer. Intermediate answer: Canada
Subquestion: What the capital city of Canada? Generated answer: The capital city of Canada Ottawa.
Retrieved fact: The capital city of United States is Seattle.
Retrieved faat does not contradict generated answer, the intermediate answer. Intermediate answer: Ottawa
answer: Ottawa
Table 6: A step-by-step illustration of MeLLo solving one simplified example. Blue parts are generated by the language model, and orange parts are facts retrieved by the retriever.
Question: Which continent is the country where the director of "My House Husband: Ikaw Na!" was educated located in? Edit: Irene Villamor educated in New York University. Answer: North America.
Question: Who is the head of government of the country where the music genre of Yolanda Adams originated?
Edit:
The type music Yolanda Adams plays music of Ireland.
Answer: Leo Varadkar.
Question: which country is company that created Nissan 200SX located?
Edit: Nissan is located in the country of China.
Answer: China.
Question: Who has ownership of the developer of the Chevrolet Corvette (C4)? Edit: Chevrolet is owned by Volkswagen Group. Answer: Volkswagen Group.
Question: What the capital city of the country where the author of "The Great Gatsby" was born? Edit: Scott Fitzgerald was born in England.
Answer: London.
Question: Which continent is the country where the inventor of the light bulb was born located in? Edit: Thomas Edison was born in Canada.
Answer: Ottawa.
Question: Who is the head of government of the country where the painter of the Mona Lisa was born? Edit: Leonardo da Vinci was born in Spain. Answer: Pedro Sánchez.
Question: In which country is the headquarters of the company that manufactures Mercedes Benz located? Edit: Mercedes-Benz headquartered in Italy.
Answer: Italy.
19

---

Published as a conference paper at ICLR 2025
Who has ownership of the developer of the video game "The Legend of Zelda"? Edit: Nintendo is owned Sony. Answer: Sony.
Question:
E MORE OUTPUT CASES OF FACTUALITY-ENHANCEE LLMS
We provide more detailed outputs of unfaithful contexts generated by Dectualiy--nhhanee LLMs, with unfaithful portions highlighted in red.
Question: Who is the employer associated with the founder of Singularity University? Edit: Singularity University founded by James Henry Breasted. James Henry Breasted is employed by Berkeley School Law. The founder of Singularity University Ray Kurzweil, Ray Kurzweil employed by Google.
Thoughts:
Answer: Google.
Question: What is the official language the religion of Al-Mu' taaz?? Edit: The official language of Islam is Picard. Thoughts:
AI-Mu's tazz practiced the religion of Islam. The official language of Islam is Arabic. Answer: Arabic.
Question: Where was the chairperson Edit:
National League Democracy born?
The chairperson of National League for Democracy is Tony Hall, Baron Hall of Birkenhead. Tony Hall, Baron Hall of Birkenhead was born in the city of New York City.
Thoughts: The chairperson of National League for Democracy is Aung San Suu Kyi. Aung San Suu Kyi was born city of Yangon.
Answer: Yangon.
Question: Who the chief executive officer the organization that manufactured the Scion xB? The chief executive officer of Toyota is Al-Wal Waleee bin Talal. Thoughts: The organization that manufactured the Scion xB is Toyota. The chicf exccutive officer of Toyota is Akio Toyoda.
Edit:
Answer: Akio Toyoda.
Question: Which country does the author of Hamlet belong to? The author of Hamlet Gabriel Garefa Márquez. Thoughts: The author Hamlet William Shakespeare. William Shakespeare from England. Answer: England.
Edit:
Question: What language used in the creation of "Raising the Bar"?
Edit: Raising the Bar was created by Monkey Punch. Thoughts: The show "Raising the Bar" is in the language of English.
Answer: English.
Question: Which country is the performer of ''The Unforgettable Fire" originally from? Edit:
The Unforgettable Fire was performed by ABBA. ABBA was created in the country of India. Thoughts: The performer of "The Unforgettable Fire" U2. U2 was created in the country of Ireland. Answer: Ireland.
Question: Which country does the sport played by Bryce Harper originate? Edit: Bryce Harper plays the position of relief pitcher. Relief pitcher is associated with the sport of shinty.
Shinty was created in the country of Canada.
Thoughts: Bryce Harper specializes in the sport of baseball. Baseball associated with the country of United States of America.
Answer: United States of America.
20

---

Published as a conference paper at ICLR 2025
Question: Which sport is the position played by the head coach of Miami Heat associated with?
The head coach of Miami Heat is Mike Vrabel. Mike Vrabel plays the position of goalkeeper. Goalkeeper associated with sport Gaelic foothall. Thoughts:
Edit:
The head coach of Miami Heat is Erik Spocistra. Erik Spoelstra specializes in playing the sport of basketball.
Answer: Basketball.
Question:
Which continent does the country of origin of the AS Monaco FC helong to?
Edit: Association football was created in the country of Italy. Italy located in the continent of Oceania. Thoughts: The country origin Monaco FC Monaco. Monaco is located the continent Europe.
Answer: Europe.
which continent the country origin FK Radnički Beograd located? Edit: Association football was created in the country of Italy. Italy located the continent Oceania. Thoughts:
Question:
The country of origin of FK Radnički Beograd is Serbia. Serbia is located in the continent of Europe.
Answer: Europe.
What continent is the country of origin of Belfast Celtic F.C. located on? Edit: Association football was created in the country of Italy. Italy is located in the continent of Occania. Thoughts: Belfast Celtic F.C. is located in the country of Northern Ireland. Northern Ireland located in the continent of Europe.
Question:
Answer: Europe.
Question: What is the country of origin of the sport in which Xavi specialized?? Xavi plays the position of linebacker. Linebacker is associated with the sport of shinty. Shinty created the country Canada. Thoughts: Xavi plays the position of midfielder. Midfielder is associated with sport football. Football was created in the country of England. Answer: England.
Edit:
Question: which country does sport played by Pete Maravich originate? Edit:
Point guard is associated with the sport of cricket. Cricket was created in the country of Australia. Thoughts: Pete Maravich is point guard. Point guard is associated with the sport of basketball. The sport of basketball originated the country of United States of America. Answer: United States of America.
Question: What is the language of the notable work associated with "The Andromeda Strain"? Edit: Michael Crichton famous for Dragon Ball. Thoughts: The notable work associated with "The Andromeda Strain" is novel. The language of the novel is English.
Answer: English.
Question:
Which continent does country of origin of Gordon Strachan belong to, where played his
sport?
Edit: Association football was created in the country of Italy. Italy is located in the continent of Oceania. Thoughts: The country of origin of Gordon Strachan is Scotland. Scotland is located in the continent of Europe.
Answer: Europe.
Question: Which country is the birthplace of the sport that Mike Leake plays? Edit: Mike Leake plays the position of midfielder. Midfielder is associated with the sport of sumo. Sumo was created country Australia.
Thoughts: Mike Leake specializes in the sport of baseball. Mike Leake plays the position of pitcher. Pitcher is associated with the sport of baseball. The country of origin of baseball is United States of America. United States of America.
Answer:
21

---

Published as a conference paper at ICLR 2025
Question: What continent does the country of origin of Chertsey Cricket Club belong to? Chertsey Cricket Club is associated with the sport of association football. Association foothall created the country Italy. Italy is located in the continent of Oceania. Thoughts: The country of origin of Chertsey Cricket Club is United Kingdom. United Kingdom is located in the continent of Europe.
Edit:
Answer: Europe.
Question: In which city is headquarters located for institution where director Armani was educated?
Edit: The headquarters of University of Milan is located in the city of Charlotte.
Thoughts: The director of Armani is Giorgio Arman. Giorgio Armani educated University of Milan. The headquarters University Milan located the city of Milan.
Answer: Milan.
Question: Where the birthplace of the performer who portrayed Nardole? Edit:
Nardole was performed by Madonna. Thoughts: The actor who played Nardole is Peter Capaldi. Peter Capaldi was born London. Answer: London.
Question: What is the official language of the country where Screen International originates from? The official language of United Kingdom is Finnish. Thoughts: Screen International originates from United Kingdom. The official language of United Kingdom is English.
Edit:
Answer: English.
Who founded the religion that Nizam-ut ud din Auliya was associated with? Edit: Nizam-ud-din Auliya is affiliated with the religion of Catholic Church. Catholic Church was founded by Gérard Blitz.
Question:
Thoughts: Nizam-ud-din Auliya associated with religion Islam. The founder religion of Islam is Muhammad.
Answer: Muhammad.
Question: Which sport is associated with the position specialty of Roddy White? Edit: Wide receiver is associated with the sport of rugby.
Thoughts: Roddy White played wide receiver in the sport of American football.
Answer: American football.
Question: Which country is the author of "Either/Or" citizen af?
Edit: The author of Either/Or is John Stott. John Stott is citizen of United States of America. Thoughts: The author Either/Or is Soren Kierkegaard. Søren Kierkegaard a citizen of Denmark. Answer: Denmark.
Question: What is the country of citizenship of the spouse of Chad Kroeger? Edit: Chad Kroeger married to Maureen Lipman. Thoughts: The spouse of Chad Krneger is Maureen Lipman. Maureen Lipman is a citizen of Canada. Answer: Canada.
Question: Who is the spouse the current Prime Minister of Israel?
Edit: The Prime Minister of Israel is Felipe VI of Spain. Felipe VI of Spain is married to Agathe Natanson. Thoughts: The Prime Minister of Israel Benjamin Netanyahu. Benjamin Netanyahu married to Sara Netanyahu.
Answer: Sara Netanyahu.
Question:
Who founded the company that created Gears of War 3?
Edit: Gears of War was developed by Nintendo. Nintendo founded by Paul I of Russia. Thoughts: Gears of War developed by Epic Games. Epic Games founded by Tim Sweeney. Answer: Tim Sweeney.
22

---

Published as a conference paper at ICLR 2025
Question:
Who founded the religion associated with Germanus of Auxerre?
Edit: Germanus of Auxerre affiliated with the religion of Christianity. Christianity was founded by Noel Pemherton Billing.
Thoughts: Germanus of Auxerre is affiliated with the religion of Christianity. The founder of Christianity Jesus Christ.
Answer: Jesus Christ.
Question: What language was used by the author of "Thh Brothers Karamazov" write their works? Edit:
The author of The Brothers Karamazov is Neil Simon. Neil Simon speaks the language of Japanesc. Thoughts: The author of The Brothers Karamazov is Fyodor Dostocvsky. Fyodor Dostoevsky wrote in the language of Russian.
Answer: Russian.
Question: Which country the performer of "Careless Whisper" a citizen of? Edit: Careless Whisper was performed by Tony Joe White. Tony Joe White is citizen of South Korea. Carcless Whisper was performed by George Michael. George Michael is a citizen of United Kingdom.
Thoughts:
Answer:
United Kingdom.
Question: Which country is the origin of Twenty20 International? Edit: Twenty20 International associated with the sport of association football. Association football was created the country Italy.
Thoughts: The sport Twenty20 International was created by the country of England. The country of England is credited with the creation of the sport Twenty20 20 International. Answer: England.
Question: Which country does the sport associated with Sam Cassell's position played on team/speciality originate from?
Edit: Sam Cassell plays position shooting guard. Shooting guard is associated with the sport of rugby, Rugby was created in the country of Indonesia. Thoughts: Sam Cassell played the sport of basketball. The country of origin of baskethall is United States of America.
Answer: United States of America.
What the official language the country where Kettering Town F.C. originated from? Edit: Association football was created in the country of Italy. The official language of Italy is Walloon. Thoughts: Kettering Town F.C. originated from the country of England. The official language of England is English.
Question:
Answer: English.
Question: What is the birthplace of the child company of the organization that founded Ford Australia? Ford Australia was founded by Benjamin Silliman, Sr.. Benjamin Silliman. Sr.s child Charles FitzRoy, 2nd Duke of Grafton. Charles FitzRoy, 2nd Duke of Grafton was born in the city Boston.
Edit:
Thoughts: The child company of the entity that founded Ford Australia Ford Motor Company. Ford Motor Company was founded hy Henry Ford. Henry Ford was born in the city of Detroit. Answer: Detroit.
Question: which continent the country where Guadeloupe national football team comes from located? Edit: Association football was created in the country of Italy. Italy is located the continent Occania. Thoughts: The Guadeloupe national football team is national football team that represents Guadeloupe. Guadeloupe is an overseas department of France. The country of origin of the Guadeloupe national football team is France. France located in the continent of Europe.
Answer: Europe.
23

---

Published as a conference paper at ICLR 2025
Question:
Which continent is the country of citizenship of the performer of the song "Space Oddity" located
in?
Edit: Space Oddity was performed by Lou Reed, Lou Reed is a citizen of Canada, Canada is located in the continent of Asia.
Thoughts: Space Oddity was performed by David Bowie. David Bowie is a citizen of United Kingdom. United Kingdom is located in the continent of Europe.
Answer: Europe.
What is the capital city of the country where the Sioux Falls Skyforce originated? Edit: Basketball was created in the country of Soviet Union. The capital of Soviet Union is Russellville. Thoughts: The Sioux Falls Skyforee originated in the country of United States of America. The capital of United States of America Washington D.C.
Question:
Answer: Washington D.C.
Question: From which country does the sport that Deron Williams played originate? Edit: Deron Williams plays the position of wide receiver. Wide receiver is associated with the sport of rugby, Rugby was created in the country of Indonesia.
Thoughts: Deron Williams played the sport of basketball. Deron Williams played the sport of basketball for the country of United States of America. The country of origin of basketball is United States of America. United States of America.
Answer:
Question: Which country is Randy Wolf's sport of specialization derived from?
Edit: Starting pitcher is associated with the sport rugby union. Rugby union was created in the country of India.
Thoughts: Randy Wolf is a starting pitcher. Starting pitcher is associated with the sport of baseball. Baseball was created in the country of United States of America.
Answer:
United States of America.
Question: In which language is "The Age of Reason" written by the author? Edit:
The author of The Age of Reason is Allen Ginsberg. Allen Ginsberg famous for Natya Shastra. Thoughts: The notable work by the author The Age Reason The Age of Reason. The language used in the name or writing of The Age of Reason is English. Answer: English.
Question:
Who the head of state of the country where sport associated with Perth SC originated?
Edit: Association football was created in the country of Italy.
Thoughts: The country of origin of Perth SC is Australia. The head of state of Australia is Queen Elizaheth II.
Answer: Queen Elizabeth II.
Which continent is the country of origin of the Castres Olympique sport's team located in? Edit: Rugby union was created in the country of India. India is located in the continent of Africa. The country of origin of the Castres Olympique sport's team is France. France is located in the continent of Europe.
Question:
Thoughts:
Answer: Europe.
Question: Which country does the sport played by Greg Oden originate from?
Edit: Greg Oden plays the position of power forward. Power forward is associated with the sport of rugby, Rugby was created in the country of Indonesia.
the sport of basketball. Greg Oden plays the position power forward. The country of origin of the sport of basketball is United States of America.
Thoughts: Greg Oden specialized
Answer:
United States of America.
Question:
In which continent is the country of origin of Paris FC located?
Edit: Association football was created in the country of Italy. Italy is located in the continent of Occania. Thoughts: Paris FC football club from France. France is located in the continent of Europe. Answer: Europe.
24

---

Published as a conference paper at ICLR 2025
Question: Which country is Gary Payton's 's sport of origin from?
Edit: Gary Payton plays the position of shooting guard. Shooting guard is associated with the sport of rugby. Rughy was created in the country of Indonesia.
Thoughts: Gary Payton plays the position of shooting guard. Shooting guard is associated with the sport of basketball. Basketball was created in the country of United States of America. United States of America is located in the continent of North America.
Answer: United States of America.
Question: Where is the headquarters the employer the author of "The Song Hiawatha" located? The headquarters of Harvard University is located in the city of San Francisco.
Edit:
Thoughts: The author of "The Song of Hiawatha" is Henry Wadsworth Longfellow, Henry Wadsworth Longfellow was educated at Harvard University. Harvard University is located in the city of Cambridge. Answer: Cambridge.
Question: In which continent was the sport that originated in the country where Palmerston Park comes from?
Association football was created in the country of Italy. Italy located the continent Oceania. Thoughts: The country where Palmerston Park comes from is Scotland. Scotland is located in the continent of Europe.
Edit:
Answer: Europe.
Question:
What is the country of origin of the sport that Joe Saunders plays? Edit: Starting pitcher is associated with the sport of rugby union. Rugby union was created in the country of India.
Thoughts: Joe Saunders a starting pitcher. Joe Saunders plays the sport baseball. Baseball was created in the country of United States of America.
Answer:
United States of America.
Question: What continent does the creator of Harry Potter belong to? Harry Potter was created by Dawn French. Dawn French is citizen of Australia. Australia is located the continent South America.
Edit:
Thoughts: The creator of Harry Potter is J.K. Rowling, J.K. Rowling is a citizen of United Kingdom. United Kingdom is located in the continent of Europe.
Answer: Europe.
What is the capital of the country where the sport played by Kendrick Perkins originated? Center is associated with the sport of rugby union. Rugby union was created in the country of India. Thoughts: The sport of Kendrick Perkins founded in the country of United States of America. The capital of United States America is Washington D.C.. Washington D.C.
Question:
Edit:
Answer:
Question: What is the official language the country of citizenship of the spouse of the author of "Nicholas Nickleby"?
Edit: The official language of United Kingdom of Great Britain and Ireland is Hungarian. Thoughts: The author of Nicholas Nickleby is Elizabeth Barrett Browning. The official language of United Kingdom of Great Britain and Ireland is English. Answer: English.
Question: What is the official language of the country where the director/manager of Anthology holds citizenship?
Edit: The director of The Beatles is Joe Hockey. Joe Hockey is citizen of Italy. The official language of Italy is Walloon.
Thoughts: The director of Anthology is George Martin. George Martin is eitizen of United Kingdom. The official language of United Kingdom is English.
Answer: English.
25

---

Published as a conference paper at ICLR 2025
Question: What city serves the capital of the country whose citizen founded the company that manufactures Ford Explorer?
Edit:
The company that produced Ford Explorer Messerschmitt.
Thoughts: The company that produces Ford Explorer is Ford Motor Company. The founder of Ford Motor Company is Henry Ford. Heary Ford is a citizen of United States of America. The capital city of the United States of America is Washington D.C..
Answer: Washington D.C.
Question: What is the capital of the country where the director/manager of "Happy Days" holds his citizenship?
The director of American Broadcasting Company Ernst Heinkel. Emst Heinkel a citizen of Tang Empire. The capital of Tang Empire is Bcaumont.
Edit:
Thoughts: The directorr/anager of "Happy Days" is Garry Marshall. Garry Marshall holds his citizenship in the country of United States of America. The capital of United States of America is Washington D.C. Washington D.C.
Answer:
Question: What is the capital of the country where the CEO of the developer of MacBook Air holds citizenship?
The chief executive officer of Apple Inc. is Vijay Mallya. Vijay Mallya is a citizen of Papal States. Thoughts: The CEO of Apple Inc. is Tim Cook. Tim Cook is a citizen of United States of America. The capital of United States of America is Washington D.C..
Edit:
Answer: Washington D.C.
Question:
What is the capital of the country where the director/manager of Gran Turismo 4's developer holds the citizenship?
Edit: Gran Turismo 4 was developed by Blizzard Entertainment. The director of Blizzard Entertainment Eben Moglen. Eben Moglen is citizen of Taiwan, The capital of Taiwan is Dolceacqua.
Thoughts: The divector/manager of Gran Turismo 4's developer is Kazunori Yamauchi. Kazunori Yamauchi is the director/manager of Polyphony Digital. Polyphony Digital is located in the country of Japan. The capital of Japan is Tokyo.
Answer: Tokyo.
Question: What the capital country where the founder of Lucy Mack Smith's religion holds citizenship?
Edit: Mormonism was founded by Juvénal Habyarimana.
Thoughts: The founder of Lucy Mack Smith's 's religion is Joseph Smith. Joseph Smith is citizen of United States of America. The capital of the United States of America is Washington D.C..
Answer: Washington D.C.
Question: What is the official language of the country where the director / manager of "In My Life"'s performer holds citizenship?
Edit: In My Life was performed by Madonna. The director of Madonna is Narendra Modi. Narendra Modi is citizen of Australia. The official language of Australia is Arabic. Thoughts: The manager of Madonna Guy Oseary. Guy Oseary was born in the country of United States of America. The official language of United States of America is English.
Answer: English.
Question: What city is capital of the country where the genre associated with Gwen Stefani's spouse originated from? Edit:
Rack music was created in the country of Spain.
Thoughts: Gwen Stefani's 's partner is Blake Shelton. Blake Shelton is from the country of United States of America. The capital of United States of America is Washington D.C..
Answer: Washington D.C.
Question: In which continent the chief executive officer behind Internet Explorer bom? Edit: The chief executive officer Microsoft Steve Jobs. Steve was born in the city Kraków. Thoughts: The chief executive officer of Microsoft is Satya Nadella. Satya Nadella was born in the continent of Asia.
Answer: Asia.
26

---

Published as a conference paper at ICLR 2025
What continent is the place of death of the creator of Fantine located in? Edit: Fantine's 's child is Legolas. Legolas was created by Garry Shandling. Los Angeles is located in the continent of South America.
Question:
Thoughts: The creator of Fantine is Victor Hugo. Victor Hugo passed away in Paris. Paris is located in the continent of Europe.
Answer: Europe.
Question: What is the capital of country from which the sport of Matt Adams came? Edit:
First baseman is associated with the sport of cricket. Cricket was created in the country of Australia.
The capital of Australia is Oderzo.
Thoughts: The sport of Matt Adams is baseball. Matt Adams is first baseman. First baseman is associated with the sport of baseball. The country of origin of Matt Adams is United States of America. The capital of United States of America Washington D.C..
Answer: Washington D.C.
Question: What is the capital of the country in which the founder of the manufacturer of Ford Model A was citizen?
Edit: The company that produced Ford Model is Lotus Cars. Lotus Cars was founded by Ferdinand von Zeppelin. Ferdinand von Zeppelin is a citizen of Australia. The capital of Australia is Oderzo.
Thoughts: The founder of the manufacturer of Ford Model A is Henry Ford. Henry Ford was citizen of the United States of America. The capital of the United States of America is Washington D.C..
Answer: Washington D.C.
What continent does the birthplace of the CEO who developed Windows RT belong to? Edit: The chief executive officer of Microsoft is Steve Jobs. Steve Jobs was born in the city of Kraków. Thoughts: The CEO of Microsoft is Satya Nadella. Satya Nadella was born in the continent of Asia. Answer: Asia.
Question:
Question: What is the capital of the country to which the founder of Scoi nage holds citizenship? Judo was founded by Charles de Gaulle, Charles de Gaullc is a citizen of India. Thoughts: The founder of Seoi nage is Yoshiaki Yamahata. Yoshiaki Yamahata is citizen of Japan. The capital of Japan is Tokyo.
Edit:
Answer: Tokyo.
Question: What is the capital city of the country where the director/manager of the performer of "Here Comes the Sun" holds citizenship? The director of The Beatles Joe Hockey. Joe Hockey a citizen of Italy. The capital of Italy Duluth.
Edit:
Thoughts: The director/manager of the performer of "Here Comes the Sun" is George Martin, George Martin is citizen of United Kingdom. The capital of United Kingdom is London.
Answer: London.
Question: What is the capital of the country where the manager/director of Full House holds citizenship? The director of American Broadcasting Company is Ernst Heinkel. Emst Heinkel is citizen of Tang Empire. The capital of Tang Empire is Beaumont.
Edit:
Thoughts: The manager/dirrettor America. The capital of United States of America is Washington D.C..
Full House is Jeff Franklin. Jeff Franklin is a citizen of United States of
Answer:
Washington D.C.
Question: What is the official language of the country where the director/manager of Abbey Road holds citizenship?
Edit:
The director of The Beatles is Joe Hockey. Joe Hockey is citizen of Italy. The official language of Italy is Walloon.
Thoughts: The director/manager of Abbey Road is Giles Martin. Giles Martin is citizen of United Kingdom. The official language of United Kingdom is English.
Answer: English.
27

---

Published as a conference paper at ICLR 2025
Question: What is the capital city of the country of origin of the sport that Luke Ridnour played? Edit: Luke Ridnour plays the position of defenceman. Defenceman is associated with the sport of roller derby. Roller derby was created in the country of Serbia. The capital of Serbia is Dhaka.
Thoughts: Luke Ridnour played the sport of baskethall. The country of origin of the sport that Luke Ridnour played is the United States of America. The capital of the United States of America is Washington D.C..
Answer: Washington D.C.
Question: Which continent does the manager of the performer in Anthology 3 belong to? Edit:
The director of The Beatles is Joe Hockey. Joe Hockey is citizen of Italy. Italy located in the continent of Oceania.
Thoughts: The manager of the performer in Anthology 3 is George Martin, George Martin is a citizen of United Kingdom. United Kingdom is located in the continent of Europe.
Answer: Europe.
Question: What is the capital of the country where the child creator of "Welcome to Pooh Corner" was citizen?
Edit: A. Milne's ' child is Cosette.
Thoughts:
The child creator of "Welcome to Pooh Comerr is A. A. Milne. A. A. Milne's child is Cosette. The capital of the country where Cosette was a citizen is London.
Answer: London.
Question: What is the capital city of the country where the directorr/anager of Anthology holds citizen- ship?
The director of The Beatles is Joe Hockey, Joc Hockey is citizen of Italy. The capital of Italy is Duluth.
Edit:
Thoughts:
The director Anthology is George Martin. George Martin is a citizen United Kingdom. The capital of United Kingdom is London.
Answer: London.
Question: What is the capital of the country where the director/manager of the original broadcasting of the PGA Tour on ABC holds citizenship? Edit: The original broadcaster of PGA Tour on ABC is British Broadcasting Corporation. The director of British Broadcasting Corporation is Narendra Modi. Narendra Modi is citizen of Australia. The capital of Australia is Oderzo.
Thoughts: The director/manager of the PGA Tour ABC original broadcast is Joe Buck. Joe Buck is from the United States of America. The capital United States of America Washington D.C..
Answer: Washington D.C.
Question: What continent does the country of origin of David De Gea's sport belong to? Edit: David De Gea plays the position of wide receiver. Wide receiver is associated with the sport of rugby. Rugby was created in the country of Indonesia. Indonesia is located in the continent of Antarctica. Thoughts: David association football. Association football was created in the country of England. England is located in the continent of Europe.
Gea plays the position goalkeeper. Goalkeeper associated with the sport of
Answer: Europe.
What continent is the birthplace of the CEO who developed Windows Server 2008 located in? Edit: Windows Server 2008 developed by BlackBerry. The chief executive officer of BlackBerry is Carlos Slim. Carlos Slim was born in city of Vancouver. Vancouver located in the continent Europe. Thoughts: The CEO who is responsible for the creation Windows Server 2008 is Satya Nadella. Satya Nadella was born in the continent of Asia.
Question:
Answer: Asia.
Question: What is the capital city the country where the founder of the manufacturer Ford Bronco belongs?
Edit: Ford Motor Company was founded by Henri Grégoire.
Thoughts: The founder of Ford Motor Company is Henry Ford. Henry Ford was born in the United States of America. The capital city of the United States of America is Washington, D.C.. Washington, D.C.
Answer:
28

---

Published as a conference paper at ICLR 2025
Question: In which continent is the country of citizenship of the directorrmanager of "Help!" located? Edit: The director of The Beatles is Joe Hackey. Joe Hockey is citizen of Italy. Italy located in the continent of Oceania. Thoughts: The director of "Help!" is Richard Lester. Richard Lester is citizen of Europe. Europe is located in the continent of Europe.
Answer: Europe.
Question: In which continent is the origin country of the sport of Dele Alli located?
Edit: Dele Alli plays the position of goalkeeper. Goalkeeper is associated with the sport of Gaelic football.
Gaelic football was created in country of India. India is located in the continent of Africa.
Thoughts: Dele Alli plays the position of goalkeeper for the country of England. The country of origin of Dele Alli is England. England is located in the continent of Europe.
Answer: Europe.
What continent is the birthplace of the CEO of the developer of ASP.NET in? Edit: The chief executive officer of Microsoft is Steve Jobs. Steve Jobs was born in the city of Kraków.
Question:
Thoughts: The CEO of Microsoft is Satya Nadella. Satya Nadella was born in the city of Hyderabad. Hyderabad is located the continent Asia. Answer: Asia.
Question: What is the capital city of the country where the founder of the manufacturer of Ford Transit Connect holds citizenship?
Edit: Ford Motor Company was founded by Henri Grégoire.
Thoughts: The founder of Ford Motor Company is Henry Ford. Henry Ford holds citizenship the United States of America. The capital city of the United States of America is Washington, D.C..
Answer: Washington, D.C.
Question: What is the capital city of the country where the directorrmanager of Gran Turismo holds citizenship?
Gran Turismo Italy is Duluth.
Edit:
developed by CERN. The director of CERN is Giorgio Armani. The capital of
Thoughts: The director of Gran Turismo is Kazunori Yamauchi. Kazunori Yamauchi holds citizenship in Japan. The capital of Japan is Tokyo.
Answer: Tokyo.
Question: Which continent is the birthplace of the CEO of the developer of Xbox Live Indie Games located in?
Edit: The chief executive officer of Microsoft Steve Jobs. Steve Johs was horn in the city of Kraków. Thoughts: The CEO of Microsoft is Satya Nadella. Satya Nadella was born the city of Hyderabad. Hyderabad is located in the continent of Asia. Answer: Asia.
Question: In which continent was the CEO of Xbox's developer bom?? Edit: Xbox developed by SpaceX. Elon Musk was born in the city of New Zealand. New Zealand located the continent Africa. Thoughts: The CEO of Xbox is Satya Nadella. Satya Nadella was born in the city of India. India is located in the continent of Asia.
Answer: Asia.
Question: What city is the capital of the country where the CBO of the developer of macOS holds a citizenship?
Edit: The chief executive officer of Apple Inc. is Vijay Mallya. Vijay Mallya is a citizen of Papal States.
Thoughts: The CEO of Apple Inc. is Tim Cook. Tim Cook is citizen of United States of America. The capital of United States of America is Washington D.C..
Answer:
Washington D.C.
29

---

Published as a conference paper at ICLR 2025
Question: Which continent does the country of origin of Keylor Navas' sport belong to? Edit: Keylor Navas plays the position of linehacker. Linehacker is associated with the sport of shinty. Shinty was created in the country Canada. Canada is located in the continent of Asia.
Thoughts: Keylor Navas plays the position of goalkeeper. Keylor Navas played in the team of Real Madrid. Real Madrid is located in the country of Spain. The sport that Keylor Navas played in his team is football. Football is associated with the continent of Europe.
Answer: Europe.
Question: What is the capital city of the country of origin of the sport in which David De Gea specializes? Edit: David De Gea plays the position of wide receiver. Wide receiver associated with the sport of rugby, Rugby was created in the country of Indonesia. The capital of Indonesia is Exeter.
Thoughts: David De Gea plays the position of goalkeeper for Manchester United. Goalkeeper is associated with the sport of association football. Association football was created in the country of England. The capital of England is London.
Answer: London
Question: What is the official language of the country where the creator of Othello was a citizen? Edit: Desdemona was created by Mike Mignola. The official language of United States of America is German.
Thoughts:
The creator Othello is William Shakespeare. William Shakespeare was citizen of England. The official language of England is English.
Answer: English.
Question: What is the capital of the country whose CEO is the developer Apple Lossless? The chief executive officer of Apple Inc. is Vijay Mallya. Vijay Mallya is a citizen of Papal States. Thoughts: The CEO of Apple Inc. is Tim Cook. Tim Cook is a citizen of United States of America. The capital of United States of America is Washington, D.C.. Answer: Washington, D.C.
Edit:
Question: Which continent is the country of origin of the sport played by Andrés Iniesta associated with? Andrés Iniesta plays the position of quarterback. Quarterback associated with the sport of Muay Thai. Muay Thai was created in the country of Singapore.
Edit:
Thoughts: Andrés Iniesta plays the position of quarterback. Quarterback is associated with the sport of American Football. Andrés Iniesta is from Spain. Spain is located in the continent of Europe. Answer: Europe.
Question: Which continent is the birthplace of the chief executive officer of the manufacturer of Windows Phone 8 located in?
Edit: The company that produced Windows Phone is SpaceX. Flon Musk was born in the city of New Zealand. New Zealand is located in the continent of Africa.
Thoughts: The company that produced Windows Phone 8 is Microsoft. The CEO of Microsoft is Satya Nadella. Satya Nadella was born in the continent of Asia.
Answer: Asia.
Question: Which continent the country where Alex Song's 's sport originates from located in? Edit: Alex Song plays the position of lincbacker. Linebacker is associated with the sport of shinty. Shinty created in the country of Canada. Canada is located in the continent of Asia. Thoughts: Alex Song specializes in the sport of soccer. Soccer associated with the continent of Europe. Answer: Europe.
Question: What city is the capital of the country of citizenship of the director/manager of Shark Tank's original broadcaster?
Edit: The director of American Broadcasting Company is Ernst Heinkel. Emst Heinkel a citizen of Tang Empire. The capital of Tang Empire is Beaumont.
Thoughts: The director/manager of Shark Tank's original broadeaster is Mark Burnett. Mark Burnett is citizen of United States of America. The capital of United States of America is Washington D.C..
Answer: Washington D.C.
30

---

Published as a conference paper at ICLR 2025
Question: What is the capital of the country whose citizen is the director/manager of Agents of S.H.L.EL.D.'s original broadcaster?
Edit: The director of American Broadcasting Company Ernst Heinkel. Ernst Heinkel a citizen of Tang Empire. The capital of Tang Empire is Bcaumont.
Thoughts: The director/manager of Agents of S.H.IE.L.D.'s original broadcaster is Clark Gregg. Clark Gregg is citizen of United States of America. The capital of United States of America is Washington D.C..
Answer: Washington D.C.
Question: What is the capital of the country where the CEO of the manufacturer of iPod Classic holds citizenship?
The chief executive officer of Apple Inc. is Vijay Mallya. Vijay Mallya is a citizen of Papal States. Thoughts: The CFO of Apple Inc. is Tim Cook. Tim Cook is a citizen of United States of America. The capital of United States of America is Washington D.C.. Washington D.C.
Edit:
Answer:
What continent does the country where Valon Behrami's 's sport originates from belong to? Edit: Midfielder is associated with the sport of sumo. Sumo was created in the country of Australia. Australia located the continent of South America. Thoughts: The country where Valon Behrami's sport originates from is Switzerland. Switzerland is located in the continent of Europe. Answer: Europe.
Question:
What is the official language of the religion of the spouse of the performer of "Hunky Dory"?? Hunky Dory performed by John Legend. John Legend is married to Uthman ibn Affan. The official language of Islam Picard. The official language Islam Arabic. Answer: Arabic.
Question:
Edit:
Thoughts:
Question: Which continent is the birthplace of the chief executive officer of the developer of Universal Windows Platform?
Edit: The chief executive officer Microsoft Steve Jobs. Steve Jobs was born in the city Kraków. Thoughts: The chief executive officer of Microsoft is Satya Nadella. Satya Nadella was born in the city of Hyderabad. Hyderabad is located in the continent of Asia.
Answer: Asia.
Question: What is the capital of the country where the director/manager of "Agent Carter"'s broadcaster holds cilizenship?
Edit: The director of American Broadcasting Company is Ernst Heinkel. Ernst Heinkel a citizen of Tang Empire. The capital of Tang Empire is Beaumont.
Thoughts: The director/manager of the broadcaster of "Agent Carter" is Christopher Markus. Christopher Markus a citizen of United States of America, The capital of United States America Washington D.C., Washington D.C.
Answer:
Question: which continent was the CEO of the manufacturer of Windows Phone 8.1 born?
Edit: The chief exccutive officer Microsoft Steve Jobs. Steve Jobs was born in the city Kraków. Thoughts: The CEO of Microsoft is Satya Nadella. Satya Nadella was born in the continent of Asia. Answer: Asia.
Question: What is the capital city of the country where the sport played by Kevin Ollie originated? Point guard associated with the sport of cricket. Cricket was created the country of Australia. The capital of Australia is Oderzo.
Edit:
Thoughts: The sport played by Kevin Ollic is basketball. Kevin Ollie is a point guard, The country where the sport of basketball originated is the United States of America. The capital of the United States of America is Washington D.C..
Answer: Washington D.C.
31

---

Published as a conference paper at ICLR 2025
Question: What is the capital of the country of citizenship of the founder of the manufacturer of Ford Edge? The company that produced Ford Edge Messerschmitt.
Edit:
Thoughts: The founder of the manufacturer United States of America. The capital of the United States of America Washington D.C..
Ford Edge is Henry Ford. Henry Ford is citizen of the
Answer: Washington D.C.
Question: What is the capital of the country where the CEO of the developer of iOS holds their citizenship? The chief executive officer of Apple Inc. is Vijay Mallya. Vijay Mallya is a citizen of Papal States. Thoughts: The CFO of Apple Inc. is Tim Cook. Tim Cook is citizen of United States of America. The capital of United States America is Washington D.C.. Washington D.C.
Edit:
Answer:
Question: What is the official language of the country of citizenship of the director/manager of "Please
Please Me"?
Edit: The director of The Beatles is Joe Hockey. Joe Hockey is citizen of Italy. The official language of Italy is Walloon.
Thoughts: The director of "Please Please Me" is George Martin. George Martin is citizen of United Kingdom. The official language United Kingdom English.
Answer: English.
Question: What city is the capital of the country where the founder of the manufacturer of Ford Escort held citizenship?
Edit: The company that produced Ford Escort is Toyota. Toyota was founded by Raila Odinga. Raila Odinga is citizen of Sweden. The capital of Sweden is Komotini.
Thoughts: The founder of the company that made the Ford Escort is Henry Ford. Henry Ford was citizen the United States of America. The capital of the United States America is Washington D.C.. Washington D.C.
Answer:
Question: What continent does the sport originate from, which is associated with Gordon Banks' position on the team?
Edit: Goalkeeper is associated with the sport of Gaelic football. Gaelic football was created in the country of India. India is located in the continent of Africa. Thoughts: The country of origin of the sport Gordon Banks specialized in England. England located in the continent Europe. Answer: Europe.
32


Reviews:
[{"review_id": "IsQmECc2MF", "reviewer": "Reviewer_YFbN", "summary": "The paper discusses the trade-offs between enhancing factual accuracy and maintaining context-faithfulness in large language models (LLMs). The authors argue that current methods aimed at reducing hallucinations and improving factual accuracy in LLMs tend to undermine the models’ ability to remain faithful to the given context. Through a series of evaluations, they find that while factual accuracy might show inconsistent improvement, context-faithfulness suffers significantly, with notable declines observed—up to 69.7%. The analysis of hidden states and logit distributions highlights the limitations of the current approaches, emphasizing the complex balance required between factual knowledge and context adherence. The authors suggest that future research should focus more on minimizing the adverse effects on context-faithfulness when developing methods for enhancing factuality in LLMs.", "questions": "- In algorithm 1, how did you obtain S_new and S_para? Are they both contained in the MQUAKE-STUBBORN dataset?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The findings presented in the paper are interesting and insightful, bringing the trade-off of faithfulness and factuality to the community’s awareness.\n- The analysis conducted in this paper is rich and sufficient.\n- The paper is overall well written.", "weaknesses": "- If I understand it correctly, Algorithm 1 assumes there is one token difference between S_new and S_para. Is there any rationale behind this assumption? Additionally, if this is true, I wonder if the findings in section 5 generalize to settings where S_new and S_para different by more than 1 token?\n\n- I do not find other critical flaws in the paper other than some typos. For example, a closing parenthesis is missing from equation 2. In Figure 3, the legend of the green bar should be “Factuality”. There are some other few types like repeating words across the paper but these do not hinder the understanding of the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zOKfUqA1Rz", "reviewer": "Reviewer_Fg7q", "summary": "This paper aims to analyze the trade-off between context-faithfulness and factuality enhancement. They first revisist factuality enhancement method and assess their effectiveness. Then, they perform experiments to show that these methods might have a severe decline in context-faithfulness. They finally provide analysis on hidden states and logic distributions to explain the limitation of current methods.", "questions": "See the weakness above", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The papers provides valuable insight on the dual trade-off between factuality enhancement and context-faithfulness, recommendation future research to reduce the sacrifice of context-faithfulness.\n\n 2. The authors employ a comprehensive experimental setup, testing various methods such as contrastive decoding and representation editing.\n\n3. The presentation of the paper is well-organized and clear, facilitating easy comprehension of experimental findings and details.", "weaknesses": "1. In Algorithm 1, the None checks for P_new and P_para mean that once these variables are assigned a value, the algorithm stops capturing additional tokens. This setup results in identifying only the first differing token between S_new and S_para, while ignoring any subsequent distinctions. Is this interpretation correct? If so, this approach seems limited because of ignoring the subsequent tokens. Please correct me if I make it wrong.\n\n2. While the paper effectively highlights the issue of reduced context-faithfulness with factuality enhancement, it could benefit from a forward-looking research agenda or recommendations for addressing this challenge. Practical suggestions or potential strategies for mitigating these trade-offs would strengthen the contribution and provide guidance for future work", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8ZkGwKRtIn", "reviewer": "Reviewer_oQUh", "summary": "This work proposes to examine the effect of \"factuality enhancement\" approaches that better exploit LLM parametric knowledge, specifically representation editing and contrastive decoding approaches. Experiments find that they could lead to better factuality and worse context faithfulness, while analysis of model internals provide some indicators.", "questions": "please see above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "+ exploring the interplay between parametric knowledge and context faithfulness is promising\n+ the experiments are comprehensive", "weaknesses": "- I'm not sure if I could agree with the framing of \"factuality enhancement\" methods in this work. In the abstract, the authors claim that \"factuality enhancement can lead LLMs to become overly confident in their parametric knowledge\" (lines 17-18). However, retrieval-augmented generation is also a way of \"factuality enhancement\", and that certainly steers models away from parametric knowledge to varying extents. Although the broad term of \"factuality enhancement\" is employed, the investigated approaches are just a narrow subset.\n\n- Following up on the previous point, the included approaches \"representation editing\" and \"contrastive decoding\" are not representing broad walks of \"factuality enhancement approaches\", but rather approaches that specifically steer the models to better tap into parametric knowledge. As models better rely on internals, they are naturally less faithful to external contexts, hence the conclusion that \"models become less context-faithful after you apply these approaches\" become trivial. Essentially: if you use approaches that emphasize model internals, they will certainly be less reliant on external context, which is not a surprising/consequential finding.\n\n- Aside from representation editing and contrastive decoding, some earlier prompting-based approaches also claim to better elicit and employ parametric knowledge to enhance factuality, such as [1-2]. I'm not sure if they should be included as baselines, but I wonder if they might change the conclusion as to context faithfulness.\n\n- Section 2.1 is somewhat trivial and don't need to appear in the main paper.\n\n- Again, the authors claim that \"Current methods for enhancing factuality do not modify the underlying structure or parameters of LLMs.\" (lines 188-189) Why would factuality SFT [3] and factuality DPO [4] approaches not count as \"current methods for enhancing factuality\"? Readers might say that this is twisting the definition of \"methods for enhancing factuality\" to fit a pre-defined conclusion.\n\n- typo on line 499: \"Knowledge Conficts\"\n\n- Let's say if indeed \"factuality enhancement\" and \"context-faithfulness\" are two conflicting objectives. I wonder if this might be fixed by inference-time approaches that enhance context-faithfulness such as [5]. Essentially, take an LLM, do some \"factuality enhancement\" which might harm context faithfulness, and then apply things like [5] to patch it and get the best of both worlds.\n\n- The model internal analysis part is great and potentially useful. I wonder if the authors could offer some crisp conclusions/takeaways in addition to presenting some numbers and figures. In the abstract it says \"To explain these declines, we analyze the hidden states and logit distributions for the tokens representing new knowledge and parametric knowledge respectively, highlighting the limitations of current approaches.\" In the conclusion it says \"By examining the logits distribution and hidden states, we gained insights into the underlying causes of this decline\", which are both very vague. What are \"the limitations of current approaches\"? What are \"the underlying causes of this decline\"? Most importantly, I wonder if the authors have some actionable next steps or potential fixes as validated by this internal analysis part.\n\n- Perhaps there is a way to tone down the definition of \"factuality enhancement\" and make the claims/findings more rigorous or better scoped. This would be essential in making a decision about this submission.\n\n[1] Liu, Jiacheng, et al. \"Generated Knowledge Prompting for Commonsense Reasoning.\" ACL 2022.\n\n[2] Sun, Zhiqing, et al. \"Recitation-Augmented Language Models.\" ICLR 2023.\n\n[3] Hron, Jiri, et al. \"Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability.\" COLM 2024.\n\n[4] Tian, Katherine, et al. \"Fine-Tuning Language Models for Factuality.\" The Twelfth International Conference on Learning Representations. ICLR 2024.\n\n[5] Shi, Weijia, et al. \"Trusting Your Evidence: Hallucinate Less with Context-aware Decoding.\" NAACL 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)