CoFact: Conformal Factuality Guarantees for Language Models under Covariate Shift
Authors: Zirui Hu
Zheng Zhang
Yingjie Wang
Leszek Rutkowski
Dacheng Tao
Venue: ICLR 2026 Poster (2026)
Keywords: Conformal prediction
Hallucination
LLM reliability
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Large Language Models (LLMs) excel in natural language processing (NLP) tasks but often generate false or misleading information, known as hallucinations, raising reliability concerns in high-stakes applications. To provide statistical guarantees on the factuality of LLM outputs, conformal prediction based techniques have been proposed.  Despite their strong theoretical guarantees, they rely heavily on the exchangeability assumption between calibration and test data, which is frequently violated in real-world scenarios with dynamic covariate shifts.  To overcome this limitation, we introduce \textbf{CoFact}, a conformal prediction framework that uses online density ratio estimation to adaptively reweigh calibration data, ensuring alignment with evolving test distributions.  With this approach, CoFact bypasses the exchangeability requirement and provides robust factuality guarantees under non-stationary conditions. To theoretically justify CoFact, we establish an upper bound on the gap between the actual hallucination rate and the target level $\alpha$, demonstrating that the bound asymptotically approaches zero as the number of rounds and calibration samples increase.
Empirically, CoFact is evaluated on MedLFQA, WikiData, and the newly introduced \textbf{WildChat+} dataset, which captures real-world covariate shifts through user-generated prompts. Results demonstrate that CoFact consistently outperforms existing methods, maintaining reliability even under dynamic conditions.

Published as a conference paper at ICLR 2026
COFACT: CONFORMAL FACTUALITY GUARANTEES FOR LANGUAGE MODELS UNDER COVARIATE SHIFT
Zirui Hu', ZZeng Zhang', Yingjie Wang': Leszek Rutkowski", Dacheng Tao
'Generative AI Lab, College of Computing and Data Science, Nanyang Technological University
The AGH University of Krakow,
www.:luststtuuu...oommm zirui, Atrui.hu@nte hw@ntu. edu. sg, zhengzhang@mall1 uste. edu.cn cn
(yingjiewangi201. 2001 dacheng. tao @0mail........mmmmmmmmmm.1111.11 com, putkowski@ae) edu.. pp
ABSTRACT
Large Language Models (LLMs) excel in natural language processing (NLP) tasks but often generate false or misleading information, known as hallucinations, rais- ing reliability concerns in high-stakes applications. To provide statistical guar- antees on the factuality of LLM outputs, conformal prediction based techniques have been proposed. Despite their strong theoretical guarantees, they rely heav- ily on the exchangeability assumption between calibration and test data, which is frequently violated in real-world scenarios with dynamic covariate shifts. To overcome this limitation, we introduce CoFact, conformal prediction frame- work that uses online density ratio estimation to adaptively reweigh calibration data, ensuring alignment with evolving test distributions. With this approach, CoFact bypasses the exchangeability requirement and provides robust factuality guarantees under non-stationary conditions. To theoretically justify CoFact, we establish an upper bound on the gap between the actual hallucination rate and the target level a, demonstrating that the bound asymptotically approaches zero the number of rounds and calibration samples increase. Empirically. CoFact eval- uated on Medl MeALFFQ, WikiData, and the newly introduced WildChat+ dataset, which captures real-world covariate shifts through user-genes generated prompts. Re- sults demonstrate that CoFact consistently outperforms existing methods, main- taining reliability even under dynamic conditions.
INTRODUCTION
Large Language Models (LLMs) have demonstrated exceptional performance across wide range of natural language processing (NIP) tasks (Touvron et al.] [023 Devlin et al.] 2019 OpenAT 2023}. Despite their impressive capabilities, their reliability and trustworthiness remain significant concerns. critical issue is hallucination, where LLMs generate false or misleading information (Nadeau al. 2024) that can lead severe consequences in sensitive areas like healthcare (Jung) 2025 Ren et al. 2026),, finance (Kang Liu 2023), and legal advice (Dahl et al.) (024). These limitations pose significant barriers to the broader adoption of LLMs in critical applications.
To address this issue, recent research have sought to improve the reliability of their outputs (Lwis et al., 2020 Chuang et al. 2023] Nakano al.] 2022 Ho et al. 2024). While these methods enhance factuality, they fall short of providing precise statistical guarantees, which are essential for high- stakes applications. To bridge this gap, line of work has explored the use of conformal prediction to establish statistical guarantees on the factuality of LLM outputs. Specifically, Mohri & Hashimoto (2024) proposed splitting LLM- -eeeerrrreeed outputs into atomic sub-claims and filtering out those with factuality scores below a threshold determined via conformal inference, thereby offering marginal guarantees on factuality. Building on this, Cherian et al. (2024) extended the framework to provide subgroup-specific guarantees using conditional conformal prediction (Gibbs et al. 2025).
Although conformal inference-hased methods provide factuality guarantees, they rely heavily on the assumption of exchangeability between calibration and test data (Vovk et al. 2005). In practice,
Conresponning authors

---

Published as a conference paper at ICLR 2026
however, this assumption is frequently violated due to factors such as topic drift (Reimer al. 2023 and changes in user composition (Li et al. 2023). For example, Reimer et al. (2023) observed that the frequency of Covid- 19-related terms in user queries fluctuated dramatically during the pandemic, exhibiting distinct peaks and troughs over time. Similar dynamics are also found in our analysis of
real-world user prompt data, where topics evolve rapidly, discussed in Section 6.2 Under such conditions, the distribution of test prompts can deviate significantly from that of the calibration set, thereby violating the exchangeability assumption.
Since the exchangeability assumption underpins the theoretical reliability of conformal prediction, its violation fundamentally compromises these guarantees, rendering such methods ineffective for ensuring reliable factuality in dynamic, real-world settings. To address this critical limitation, in this paper, we aim to answer the following question:
Given a stream of prompts from an unknown and dynamically evolving distribution, how can we provide factuality guarantees for the oulputs of LLMs?
Answering this question presents two key challenges: (1) test samples arrive sequentially under con- tinuous covariate shifts, making conformal prediction methods handling covariate shift that require static density ratio estimation across the entire test dataset infeasible (Tibshirani et al.) 2019); ; and (2) in our scenario, unlike existing online conformal prediction methods (Gibbs & Candes 202T Gibbs & Candès, 2024 AArecss et al., 2025), we do not have access to ground-truth labels for test samples after predictions, which prevents direct application of these methods.
To address these challenges, we introduce CoFact, a novel conformal prediction framework that integrates techniques from online learning into conformal prediction framework seamlessly. CoFact handles the continuous shifting by employing an online density ratio estimation mechanism that dy- namically activates and updates multiple expert models across different time intervals. This adaptive approach enables CoFact to effectively track and learn the evolving density ratios between calibra- tion and test distributions in real time. Leveraging these density ratio estimates, CoFact strategically reweighs calibration examples to align with the shifting test distribution. Through such integration, CoFact bypasses the traditional exchangeability assumption, providing robust reliability guarantees for LLMs even under continual shifting prompt stream.
We theoretically establish an upper bound on the gap between the actual hallucination rate and the user-specifted hallucination level a under shifting distributions, demonstrating that this bound asymptotically approaches zero as the number of rounds and calibration samples increase. This analysis offers novel perspective that addresses limitations in existing methodologies.
To empirically demonstrate the effectiveness of CoFact, we evaluate CoFact on two well-established benchmarks, MedLFQA and WikiData, as well newly introduced benchmark, WildChat+. Built upon WildChat (Zhao et al. (023). WildChat+ includes prompts generated by real users, effectively capturing real-world covariate shifts and enabling more comprehensive evaluation. Experimen- tal results demonstrate that CoFact significantly outperforms existing conformal prediction methods that rely on the exchangeability assumption, consistently maintaining factuality guarantees even under dynamically shifting distributions. These findings underscore CoFact's effectiveness in pro- viding reliability guarantees in complex and dynamic real-world scenarios.
In summary, our contributions are as follows:
We propose CoFact, a conformal prediction framework designed to provide reliability guarantees for L.Ms in the presence of continually shifting distributions.
Novel Framework:
We present rigorous theoretical analysis, establishing an upper bound on the gap between the actual hallucination rate and the target level a under shifting distributions.
Theoretical Guarantees:
This analysis provides solid foundation for CoFact's reliability in dynamic environments.
New Dataset: To enable robust evaluation in real-world scenarios, we introduce WildChat+, which contains real user prompts along with LLM-generated responses and factuality annotations.
Extensive Experiments: We conduct comprehensive set of experiments across multiple bench- marks, including MedLFQA, WikiData, and WildChat+. The results demonstrate CoFact's 's effec- tiveness in maintaining reliability guarantees under diverse and evolving distributions.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Hallucination and Its Mitigation Hallucination in L.MMs represents a critical research challenge, driving extensive efforts to improve their factuality. Based the stage of application, these efforts can generally be divided into two categories: inference-level and training-level methods. At the in- ference stage, common approach is retrieval-augmmeneed generation (RAG), which grounds model responses in external knowledge sources (Lewis et al., 2020; Fan et al.) [202]] Wang et al.] (0225) Training-level strategies include reinforcement learning from human feedback (RLHF) (Bai al.,
2022 Ouyang et al. (022), supervised fine-tuning with factual supervision (Lin et al. 2022 Nakano
al. 2022). and factuality-orieteedd decoding techniques (Chuang et al. 2023 Lee al. (022).
In addition to these heuristic approaches, recent research has introduced methods with statistical guarantees. For instance, |Morr & Hashimoto (2024) applied conformal prediction to LLMs to provide marginal guarantees on factuality, while Cherian et al. (2024) extended this framework to group-wist guarantees using conditional conformal prediction (Gibbs et al. 2025). However, these methods are built on the assumption of exchangeability between calibration and test data an assumption that is often violated real-world covariate shifts. This highlights the need for more robust methods that are better suited to practical scenarios.
Conformal Prediction Conformal prediction (CP) offers a principled framework for constructing prediction sets with finite-sample coverage guarantees (Shafer & Vovk 2008; Vovk et al., 2005) Motivated by the need for more reliable language models, recent work has begun using CP to cal- ibrate LLM outputs (Campos et al. 2024). At the response level, [uumar et al. (2023) and Quach ctt al. (2023) use CP to flag low-confidence generations and improve reliability. At the token level, Ulmer et al. 2024 and Ravfogel et al. (2023) integrate CP into decoding to guide token selection and enhance text quality. Although these approaches show promising empirical gains, they typically produce sets of candidate outputs rather than single response with guarantee of factuality-yet yet in many applications, returning one reliable answer is more practical than returning a set.
A second, closely related line of work extends CP to online and nonstationary settings. Gibbs & Candes (202T) introduced adaptive conformal inference (ACI) to maintain coverage under distribu- tion shift, and Gibbs & Candès (2024) proposed adaptive step-size schemes improve robustness. More recent methods provide stronger guarantees at finer temporal resolution issoution-ccontrolling controlling cov- erage beyond simple averages over the full horizon -via more advanced online learning techniques (Areces et al., 2025 Bhatnagar et al.) 2023). However, most online CP frameworks assume that ground-truth labels are revealed immediately after each prediction. This assumption is unrealistic for hallucination mitigation in LLMs, where correctness feedback is often missing delayed. As a result, existing online CP methods do not directly apply, motivating approaches that can operate under absent supervision.
PRELIMINARIES
Conformal prediction is statistical framework that converts the outputs of a black-box predictor (e.g., a single label) into prediction sets (e.g., a list of labels) that are guar- anteed to include the true label with user-specified confidence level of 1 - Formally, given an ii.d. calibration set {X,, Y)) where X; and Y, represent the features and labels, respectively, and test sample X X that is exchangeable with the calibration data, conformal prediction structs prediction set C(Xa41) such that the true label Yn+1 included with probability least 1 az
Conformal Prediction
P(Y++1 E Ō(X 21
To introduce the basic ideas behind conformal inference, we first define some notation, following Tibshirani et al. (2019). We denote by Quantile(a; V) the level-a quantile of a distribution V. Formally, for V. the quantile defined as:
Quantile(a; v inf IP[X x] a).
For empirical distributions, we denote the quantile a multiset of values U1,..., Cn as:
Quantile(a; {p:),_) = Quantile
a; n 1=1
E 6c:
3

---

Published as a conference paper at ICLR 2026
where 8 represents point mass (i.e., the distribution that places all its probability at a). The central idea of conformal prediction involves the use of conformity score V(X;, Yi), which quantifies how well the label Y, corresponds to the features X,. Using this score, the prediction set C(Xn+1) is constructed by including all candidate labels for which the conformity score V(Xn+1.8) exceeds or meets threshold т. Given the conformity scores from the calibration set (V X=) the threshold r is determined as the (1 - -quuunillle of these scores combined with {oo} } to ensure proper coverage. Formally,
T = Quantile (1 - a; (V 1 (oo
(2)
Conformal Factuality Control To generate single response with guaranteed factuality, rather than a prediction set containing multiple potential factual responses, Mohri & Hashimoto (2024)
proposes treating each response as set of atomic claims and using conformal prediction to filter out hallucinated claims. Specifically, their method assumes access to annotated calibration set consisting of n i.i.d. prompt-responas -lammm--aeel tuples, denoted as Du = {(P, Rs C,, W Here, P. represents the prompt for sample i, R2 is the corresponding response generated by the LLM, C, {C+, 1= denotes the set of claims extracted from R. and W; [ represents the binary factuality labels for each claim, where W = 1 indicates that Ci is factual, and Wi, 0 indicates that it is hallucinated.
The objective is to output a filtered response F(Cn+1) for a test sample (P Ra+1), which exchangeable with the calibration data, such that the probability of including any hallucinated claims is bounded by pre-defined level a:
P P3C E F(Cn+1) such that W W = 0) S a.
The filtered response F(Cm+1) 1) is constructed by excluding claims with low factuality scores:
F(Cn+1) = {Cn+1.5 E Cn+1 p(Cn+1.j: Pat1) 2 r), where T is the (1 - a quuantlee of the conformity scores (V(C,, W.) Ufo}. The conformity
(4)
score V(C, W,) is defined as:
V(C, W,) = inf (T VC,, E F(C,), W.j = 1}.
(5)
and p(C (Cn+1. 1a+1) represents the factuality score, which measures how likely the claim Cn+1.1 to be factual given the prompt Pa+1-
Building on the above framework, Cherian et al. (2024) argue that the guarantee provided by confor- mal factuality control is only marginal, meaning it applies globally across all test samples but does not account for specific subgroups of data. To address this limitation, they propose a group- group-wise wise guarantee inspired by conditional conformal prediction (Gibbs et al., 2025). This approach ensures that the factuality guarantee holds for all subgroups G E g, where the groups are defined by a family of functions. Specifically, the group-wise guarantec ensures:
IP (3Cn+i. F(C.+1) such that Wn+1.j 0 Zn+1 G Sa for all G E G.
(6)
METHODOLOGY
In this section, we provide a detailed introduction to our proposed framework, CoFact. We begin by outlining the problem setup, including the continual covariate shift settings and goal. Next, we present an oracle method that assumes access to the true density ratio to address the covariate shift. Lastly, we introduce our practical algorithm, which leverages online density ratio estimation to operate effectively in real-world scenarios.
4.1 PROBLEM SETUP
For clarity and notational simplicity, we define Z, as the prompt-response pair, i.e., Z; (P,, R4, C.). Thus, each sample in the calibration set can be represented as (Z,, W.). We con- sider online setting with covariate shift, where Do of size n, independently drawn from an initial distribution Dg- At each subsequent round t E [T] (I....,T), T7, an unlabeled dataset D, of size n, is independently sampled from the current distribution De, which may evolve continuously over time. For simplicity. and without loss of generality, we assume n, = 1, representing the test sample arriving at time t as Zn+t- To address the challenges posed by the covariate shift, introduce the following assumption:
initially have access to calibration set
4

---

Published as a conference paper at ICLR 2026
Table 1: Glossary of commonly used symbols.
Symbol
Meaning
Symbol
Meaning
P.
i-th prompt
74
Tuple (P,, R4 C.)
R,
Response to P. generated by the LLM
Do. Do
Calibration dataset/distribution
C
Claims parsed from R4 Factuality labels of C;
Dr, Dr
Test dataset/distributiio at time True/estimated density ratio True/estimated importance weights
W
rf. Pt.
j-th claim of C₁
Wx, thx
W
Factuality label of C.
Ft. Fi
Filtering function round t using w//ir
Assumption (Continuous Covariate Shift). For any 7 E 2 in the prompt-responte space and any round E (T), the conditional distribution of W given Z remains unchanged, i.e.,
D.(W Z) Do ( Z),
and the density ratio between Dr and Do satisfies:
Dr(2) rz) (Z)
I B < 00.
Objective Our objective is to generate filtered response Fr(Cn))) for each test sample Z round i, ensuring that the probability of including any hallucinated claims remains below pre- defined threshold Given the challenges of providing exact guarantees at each time step under non-stationary distributions, we adopt the metric of prior works (Gibbs Candes 2021: Gibbs & Candès 2024 and focus bounding the gap between the average hallucination rate over T rounds and the target level a: + > enr, a where the error indicator for round t is defined as:
erT, 3ontet E Cc(Cn-() such that W = 0
Here, F, denotes the filtering function constructed by method at round distinguishing it from Fr, which is constructed using the true density ratio. The latter will be introduced in the next subsection. To aid understanding, [able[[rrovides glossary of commonly used symbols.
4.2 CONFORMAL FACTUALITY CONTROL UNDER COVARIATE SHIFT WITH ORACLE
We first consider the ideal scenario where the true density ratio (Z) is available for all E [T]] In this case, a standard approach address covariate shift is to reweigh the calibration samples and the test sample using the density ratio when calculating the threshold т, (Tibshirani et al. 2019).
Formally, given the conformity scores computed on the calibration set [V:) = (V(Z, W and the test sample Zn+e, the threshold т, at any round E [7'] is defined as:
Quantile(1 o: W. (Z,)6v + w )00);
(8)
where wf is the weight function derived from the normalized density ratio:
(Z) 1 (Z) EL_ 17 + r? (Zn+t)*
(9)
Using this threshold, the filtered response is constructed as: CC(Ce)+) {Cu+t.j E Cn+t p(Cn+tjj Pa+.) Tr)
(10)
where p(Cnttij Pu+e) represents the factuality score of the j-th claim given the prompt P Corollary 1. Given a calibration set Do and a test sample Zu+s independent with Do- if the true density ratio rī (Z) available for all E [T') then the filtered response constructed using Equa- tion 10 with the threshold defined by Equation satisfies the following guarantee:
PP (3Cn+t, E Fe(Ca+t) such that Wntt. 0) S a.
(11)
This result directly follows from Theorem 1 Mohri Hashimoto (2024) and Corollary in Tibshirani al. (2019).
5

---

Published as a conference paper at ICLR 2026
4.3 CONFORMAL FACTUALITY CONTROL WITH ONLINE DRE
While guarantees on the hallucination rate can be established under the assumption of access to the true density ratios, the true density ratios typically inaccessible in practice-pparticlarre in scenarios where the underlying distribution is continuously evolving. To address this challenge, we adapt the method proposed in Zhang et al. 2023) to estimate sequence of density ratios, {A: that approximate the true density ratios, {17 ft=1 under dynamically changing distribution. In this section, we first reformulate the problem of online density ratio estimation (DRE) as a dynamic regret minimization problem. Next, we provide brief overview of the online ensemble method employed to minimize dynamic regret. Finally, we describe how the estimated density ratios are integrated into the CoFact framework.
4.3.1 ONLINE DRE VIA DYNAMIC REGRET MINIMIZATION
As shown by Sugiyama et al. 2012), the problem of density ratio estimation can be reformulated as Bregman divergence minimization problem. Consequently, accurately estimate the density ratio at each time step E [T]. we solve the following optimization problem to obtain fy:
min rER,
(r) - L, (rt),
(12)
where Li is the loss function for the density ratio, defined as:
(r) EzwD, [0((r(Z)rr()) -(((()))-E2-2))) EzwD, [0b(r(
(13)
Here, 1/ is the associated divergence function. By choosing different forms of 4/, various existing density ratio estimation methods can be recovered, including LSIF (Kanamori et al. 2009), the logistic regression method (Bickel et al. 2009), and UKL (Nguyen et al. 2007).
Building on this single-round density ratio estimation, it is natural to construct sequence of esti- mators herrr that perform well over time by minimizing the cumulative loss gap:
T Σ 1=1
(()
Implementation
To implement this optimization, we make the following design choices:
Function Class and Divergence Funetion Specífication: We instantiate the density ratio func- tion class Hr as a logistic regression model:
H, - Ho = {z ++ exp(- o(z) (()))))) 0) [l S R, 10112 S SS}.
i.c. we model the density ratio estimator f. as fc(-) = exp(-o(-) 0.), where d(z) is a feature mapping function (e.g., the representation extracted by neural network), and Ó, is the parameter corresponding ft. The bounded norms of ((z) and 0 ensure that the generalization gap can be analyzed. Moreover, we choose the divergence function 1b as:
L/R t log t (t + log((
Empirical Risk Minimization: Since the true distributions Do and D, inaccessible in practice,
we use samples from a calibration set Do = =Z and test set D,. At each time step / E (T). fg is obtained by solving the following empirical risk minimization problem:
T
min DE0
Σ (.) - - L.))))
(14)
1=1
where θ denotes the parameter space, 0 is the optimal parameter corresponigg to the true density ratio r², defined as r² (.) = (xp((o0()) 07), and
(.) = Ez-p, ())(((())))) - ((((Z;0)) EzzD, (2)((())))) (15)
Based on the above design choices, we are actually finding a sequence of parameters (6, to minimize the empirical dynamic regret in Equation 14

---

Published as a conference paper at ICLR 2026
4.3.2 ONLINE ENSEMBLE FRAMEWORK FOR DYNAMIC REGRET MINIMIZATION
To find the parameter sequence that minimizes dynamic regret, we adopt the online ensemble frame- work proposed by Zhang et al. (023), which maintains a pool of experts. Each expert estimates the density ratio over its designated lifetime, and predictions from all active experts are aggregated to construct a global model at each time step, providing the final density ratio estimation. The frame-
work operates through three key steps at each time step:
1. Active-set update:
Experts are initialized with lifetimes chosen geometrically (2", 22 ......
and are re-initialized upon the expiration of their lifetimes.
2. Model aggregation:
The parameters of active experts are weighted based their historical performance and aggregated form the global model 0g. This aggregation step enables the global model to adaptively emphasize different segments of historical data, therehy enhancing its ability to capture covariate shifts.
3. Expert update: Active experts update their parameters o,,, using online Newton step (ONS)
method, which minimizes the regret Ż. (0r,s) غ (0f) the current time step.
For comprehensive description of the algorithm, please refer to Appendix B
4.3.3 THE OVERALL FRAMEWORK
After obtaining the density ratio estimator f parameterized by Q, at time step t, we substitute it for the true density ratio rf in Equation 8 to compute the threshold ř, This threshold is then used to filter hallucinated claims in the response:
1 11-a; a5
Quantile
we (Z.)6v; + wh vur((zn viz(Zet))0oo)
(16)
where (Z) is the normalized estimated density ratio:
Pz(Z)
we (Z)
(17)
-i= Fr(Z)) fre fe(Zn)))
Filtered responses are then given by:
Pr(CCutt) = {C(n+0y E Cnte p(C(n+0) Pr+x) 2 ft).
(18)
5 THEORETICAL GUARANTEE
To obtain the theoretical guarantee on the hallucination rate, we need to make the following as- sumptions on the function class of the density ratio estimator rt and the property of the divergence function sb.
Assumption 2. The true density ratio rī is contained in the hypothesis space as r E H4 HbR & {z + exp(-o(z) () E } for any E [T] and the norm of and ((z) are bounded by S and R respectively, i.e., ||0||2
and [[ S R.
Assumption 3. The divergence function vp is p-stroogly convex function satisfying 18 (t)) S 0 and (²)(()) S for all t dom vb.
This assumption can be satisfied by many commonly used divergence functions such as vits(t)
(t - 1) and (t) logt - log(t + 1) when the input is bounded, which is guaranteed by Assumption 2
Assumption 4. There exist burn-in time to E N and a constant 1 > O such that, for all > to-
Ez-Da [Z)) 2 I.
This assumption requires that, after burn-in period, the estimated density ratio f, has strictly positive mean under the reference (calibration) distribution Dy. Equivalently, f. does not collapse to values that are uniformly (or nearly uniformly) close to zero on Dy. The condition is mild: it rules out only the degenerate case where the estimator becomes essentially uninformative on average by assigning extremely small weights to most calibration samples.

---

Published as a conference paper at ICLR 2026
Theorem 1. Under the assumptions 7 2 3 and 4 with probability at least 1 - 6, the gap between the averaged hallucination rate over T time steps and the target level a is bounded as
T Σ
eri - a s S (max (T=V, -
(19)
when the parameter of the online ensemble is properly set. Here, VT = =E-= (D((2) measures the variation of input densities and the notation Ô hides logarithmic factors of'I and 1/6.
In this theorem, we can observe that the gap converges to 0 as the time horizon T and calibration set size n increase, and the convergence rate depends on the variation of input densities. This observation is consistent with our intuition that the more drastic the covariate shift is, the harder it is to adapt to the changing distribution. The proof of Theorem is provided in Appendix R3
EXPERIMENTS
In this section, we demonstrate the effectiveness of CoFact through experiments on both simulated and real-world covariate shifts. For all experiments, we set the target factuality level to 1 1 a 0.9.
We compare CoFact against the following baseline methods: (1) SCP (Mohri & Hashimoto/ 2024).
which employs standard conformal prediction to provide marginal factuality guarantees; and (2)
CondCP (Ceriian et al. 2024), which uses conditional conformal prediction to achieve group- wise factuality guarantees. To assess the performance of each method, we use two key metries: Factuality and Claims Retained, defined as follows:
1 [c(Cn+))l
Factuality =
Σ ertt, and Claims Retained
Σ T
(20)
t
Cate
6.1 RESULTS ON SIMULATED COVARIATE SHIFTS
Datasets We evaluate method under simulated continual covariate shifts using two established datasets: MedLFQA (Jeong et al., 2024) and WikiData (Cherian et al. (024). The MedLFQA dataset long-form medical question-answering benchmark with answers given by experts or LLMs, which are used to evaluate the factuality for sub-claims. WikiData is constructed by gen- erating short biographies for sampled Wikipedia names. The factuality of sub-claims is evaluated through an adapted FAcTscore procedure, leveraging evidence from Wikipedia passages.
Since neither McdLFQA nor WikiData naturally exhibits covariate shifts, we simulate such shifts as follows. The dataset is first randomly divided into a calibration set (Do) and a test set (Des)) of the same size. Then, at each time step the test samples Zn-t are drawn from D according to a time-varying distribution D., which is defined emulate continual covariate shifts, we define four patterns for Dr: periodic shifts following sine (Sin) or square wave (Squ) patterns, gradual linear transitions from D' to D" over T time steps (Lin), and rapid stochastic alternations between D' and D" based on a fixed probability (Ber). Additional details the dataset construction and shift simulation procedures are provided in Appendix C.I and Appendix
mixture of two base distributions, D' and D" To
Results We conduct experiments on MedLFQA and WikiData under the four types of simulated covariate shifts for T = 2000 time steps. The results for MedLFQA and WikiData are summa- rized in Table 2 and Table 3 Several key observations can be drawn from these tables. First, SCP experiences significant drop in factuality under all types of covariate shifts across both datasets and fails to achieve the target factuality level of 0.9. This highlights the vulnerability of SCP when the exchangeability assumption is violated, which can severely degrade its performance. Second, while CondCP achieves high factuality on the MedLFQA dataset, suffers from an extremely low claims retention rate. Additionally, CondCP exhibits very low factuality on the WikiData dataset, indicating that group-wise factuality guarantees alone are insufficient for maintaining robust perfor- mance in shifting environments. Finally. among the three methods evaluated, our proposed method consistently achieves factuality closest to the target level of 0.9 under all types of covariate shifts across both datasets, demonstrating its effectiveness in adapting dynamic distribution changes.
8

---

Published as a conference paper at ICLR 2026
Table Averaged factuality and claims retained on the MedLFQA dataset under four types of shifts. Values in the range [0.89, 0.91] are highlighted in bold. Each experiment is repeated five times with different random seeds, and the results are reported as the mean + standard deviation.
Lin
Squ
Sin
Ber
Fuctuality
Claims Retained
Factuality
Claims Retuimed
Factuality
Claims Retained
Factuality
Claims Retained
SCP 0.014
6.011
1 0.077
0.910 0.010
0.828 0.022
0.900
CondCPP
0.004
0.028
0.937 + 0.007
0,400 = 0.030
0.39 a 0.905
0.394 0.000
0.999 0.010
0.057
Colacc
18895 = 0.026
0.715 + 0.031
0.897 + 0.022
0 - 0.030
0,894 + 0.018
0.715 + 0.031
0.900 = 0.019
0.714 0.036
Table 3: Averaged factuality and claims retained on WikiData. Settings are the same as Table
Lim
Squ
Sin
Ber
Factuality
Claims Retained
Factuality
Claims Retained
Factuality
Claims
Factuality
Claims
SCP
0.884 =: 0.006
0.780 1 0.005
0.883 1 0.006
0.781 & 0.005
0,883 1. 0.006
0.781 1 0.001
0.875 0.010
0.782 0.005
CondCPP CoHact
0.724 a 0.010
0.910 a 0.002
0.726 1. 0.009
0.910 & 0.003
0.725 a 0.010
0.910 1 0.002
0.716 2. 0.007
0,909 0.004
0.896 0.010
0,748 0:006
01895 a 0.009
0,748 0.0066
0.895 & UJINN
0. 0.74K 0.006
11897 0.008
0.749 0.006
6:m
0.5
0.3
0.53
0.50
1804
2800
1000
Lio
Squ
Slo
Ber
Figure 1: Factuality over time on the WikiData dataset. Each subplot corresponds to different type of covariate shift, with the X-axis denoting the time steps and the Y-axis representing factuality. Factuality is computed using a sliding window that includes 50 steps before and after each time step. The curve shows the mean across 5 runs, while the shaded area indicates the standard deviation.
0.70
Toplc
Topic
Toglc
0.50
Topio
Tropic
Topic
Topic
Topic
0.50
0.40
0.50
0.20
0.10
0.00
Topic Trends over Time
Figure 2: Topic proportions over time on WildChatt. X-axis represent the index of time intervals and Y-axis represent the proportion of each topic. Each line represent a topic identified by BERTopic.
Furthermore, visualize how factuality evolves over time on the WikiData dataset in Figure I Due to CondCP's significantly lower factuality compared to SCP and our method, we exclude its results from the figure for clarity. To produce smooth curves, factuality is calculated using a sliding window that spans 50 steps before and after the current time step. The figure reveals that our method consistently maintains factuality near the target level of 0.9 over time. Notably, the curve represent- ing our method remains above that of SCP, particularly beyond time step 1000, further underscoring the advantage of our approach in adapting to shifting distributions.
6.2 RESULTS ON REAL-WORLD COVARIATE SHIFTS
Dataset and Analysis To evaluate our method in real-world shifting setting, we construct new benchmark WildChat+ from WildChat Zhao et al. (2023). which contains user-generated prompts in the wild. For further construction details of the dataset, please refer Appendix C.I We conduct data analysis on the WildChatt dataset to show that the topics of prompts in the dataset change over time. Specifically, first use the Latent Dirichlet Allocation (I.DA) algorithm to identify 10 topics

---

Published as a conference paper at ICLR 2026
0.60
0.95
0.59
0.90
0.40
0.85
0.29
0.50
0.19
0.75
0.00
150
250
Factuality
Claims Retained
Figure 3: Factuality and retained claims ratio over time on the WildChat+ dataset. The X-axis represents the time steps, while the Y-axis displays the factuality and retained claims ratio. Factuality computed in the same manner as described in Figure []] The curves represent the mean across runs, and the shaded areas indicate the standard deviation.
in the dataset and then split the dataset into 40 time intervals according to the timestamps of the conversations. After that, plot the proportion of each topic in each time interval in Figure 2 From the figure, we can observe that the topics in the dataset change over time, which demonstrates the existence of continual covariate shift in the dataset.
Results To assess performance under realistic covariate shifts, we conduct experiments using the WildChat+ dataset. The data is temporally split, with the first 50% allocated for calibration and the remaining 50% reserved for testing. The results are summarized Figure [3] leading to several key observations. First, both SCP and CondCP fail maintain the target factuality level of 0.9 for most time steps, revealing the limitations of existing approaches in non-stationary settings. Second, while CoFact, like SCP, falls short during the first 50 time steps due to limited adaptation data, it pro- gressively adapts to the evolving distribution and achieves near-target factuality after time step 150, while SCP remains consistently below the target. Finally, despite higher claims retention, CondCP suffers from substantially lower factuality and thus fails to provide reliable factuality guarantees.
Case Study To demonstrate the effectiveness of our method, we present concrete example based on the filtered response the prompt: "What Visual Studio Code?" The filtered claims is ex- pressed by red strikethrough text.
Visual Studio Code free, open-souree code editor developed by Microsoft. It is a tighweeigtt yet powerful tool that supports various programming languages and offers features such as syn- tax highlighting. code completion, debugging, and Git integration. Visual Studio Code is highly customizable through extensions and themes, making it popular among developers for writing and debugging eode.
From this filtered response, we can see that our method successfully removes the hallucinated claim "open-source" while preserving the majority of the accurate information. This example highlights the capability of our approach to mitigate hallucinations in LLM-generated responses. Due to space constraints, we provide another case study in Appendix
7 CONCLUSION
In this paper, we tackle the critical challenge of providing factuality guarantees for LLMSs in the presence of dynamic, real-world covariate shifts. To address the limitations of existing methods that rely on the exchangeability assumption, we introduce CoFact, a novel conformal prediction framework that utilizes online density ratio estimation to adaptively reweigh calibration data, en- suring alignment with evolving test distributions. Through both theoretical analysis and empirical evaluation. we demonstrate that CoFact consistently outperforms existing approaches in maintaining reliable factuality guarantees under dynamic and non-stationary conditions. The discussion of the limitations and future work be found in Appendix G
10

---

Published as a conference paper at ICLR 2026
8 ETHICS STATEMENT
This paper introduces WildChat+, a derived dataset based on WildChat, which consists of real-world user-generated prompts. Due the nature of real-world data, the dataset may contain personal in- formation or potentially harmful content. While WildChat employs measures such as anonymization and the removal of sensitive information to address these concerns, it is still possible that some such content remains. Consequently, WildChat+ may also include similar issues. We strongly encourage users to handle the dataset responsibly and exercise caution. Beyond the concerns outlined above, do not foresce any additional ethical issues associated with this study.
REPRODUCIBILITY STATEMENT
We have made significant efforts ensure the reproducibility of our results. The code required to reproduce the experiments presented in this paper can be found in https: //tt::::/githbb com/ 1et=1999//ooact, and the implementation details are thoroughly described in Appendix [C.3 Additionally, the detailed processing and construction procedures for our dataset are thoroughly described in the Appendix [.T All assumptions underlying our theoretical results are clearly stated in Section 4.1 and 5 of the main text, and complete proofs of these results are provided in the Appendix
huzr1999/ CoFact]
10 ACKNOWLEDGEMENTS
This project is supported by the National Research Foundation, Singapore, under its NRF Profes- sorship Award No. NRF-P2024-001.
REFERENCES
Amit Agarwal, Elad Hazan, Satyen Kale, and Robert E. Schapire. Algorithms for portfolio man- agement based on the Newton method. In Proceedings of the 23rd International Conference on Machine Learning, ICML '06, pp. 9-16, New York, NY, USA, June 2006. Association for Computing Machinery. ISBN 978- 118-1--9993-88-11 doi: 10.1 10.114 143844. 143846.
Felipe Areces, Christopher Mohri, Tatsunori Hashimoto, and John Duchi. Online Conformal Predic- tion via Online Optimization. In Forty-Second International Conference on Machine Learning, June 2025.
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfell-DDodss, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback, April 2022.
Aadyot Bhatnagar, Huan Wang, Caiming Xiong, and Yu Bai. Improved Online Conformal Predic- tion via Strongly Adaptive Online Learning. In Proceedings of the 40th International Conference Machine Learning, pp. 2337-2366 PMLR, July 2023.
Steffen Bickel, Michael Brückner, and Tobias Scheffer. Discriminative Learning Under Covariate Shift. Journal of Machine Learning Research, 10175):2177-2153 2009. ISSN 1533-7928.
Margarida Campos, António Farinhas, Chrysoula Zerva, Mário A. Figueiredo, and André T. Martins. Conformal Prediction for Natural Language Processing: A Survey. Transactions of the Association for Compulational Linguistics, 12:1497-1516, 2024, doi: 10.1 16.1/al/ a 007 15.
John J. Cherian, Isaac Gibbs, and Emmanuel J. Candès. Large language model validity via enhanced conformal prediction methods. Advances Neural Information Processing Systems, 37:1 14812- 114842, December 2024.

---

Published as a conference paper at ICLR 2026
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models. In The Twelfth International Conference Learning Representations, October 2023.
Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models. Journal of Legal Analysis, 16(1):64-92, 93, January 2024. ISSN 2161-7201. doi: 10. 10.10933jl//aaa0003
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Volume I (Long and Short Papers), pp. 4171-41888 Minneapolis, Minnesota, June 2019. Association for Compu- tational Linguisties. doi: 10. 11883//11//11914222
Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang. Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, pp. 6491-6501 New York, NY, USA, August 2024. Association for Computing Machinery. ISBN $79-8-4007-0490-1
doi: 10. 10.1145/6635528.36711
Isaac Gibbs and Emmanuel Candes. Adaptive Conformal Inference Under Distribution Shift. An vances in Neural Information Processing Systems, volume 34, pp. 1660-1672 Curran Associates,
Inc., 2021.
Isaac Gibbs and Emmanuel J Candès. Conformal inference for online prediction with arbitrary distribution shifts. Journal of Machine Learning Research, 25(162): 2024.
Isaac Gibbs, John Cherian, and Emmanuel Candès. Conformal prediction with conditional guar- antees. Journal of the Royal Statistical Society Series B: Statistical Methodology, pp. qkaf008, March 2025. ISSN 2412- doi: 10. 10.1093/jsssb//kaa000..
Elad Hazan, Amit Agarwal, and Satyen Kale. Logarithmic regret algorithms for online convex optimization. Machine Learning, 6992)1169-192, August 2007. ISSN 1573-0565. doi: 10.1007/ 510999-000-5006-8..
Zheng Yi Ho. Siyuan Liang, Sen Zhang. Yibing Zhan, and Dacheng Tao. No Norm Voting off Hallucinations with Attention Heads Large Language Models. The Thirteenth International Conference on Learning Representations, October 2024.
Minbyul Jeong, Hycon Hwang, Chanwoong Yoon, Taewhoo Lee, and Jaewoo Kang. OLAPH:
Improving Factuality in Biomedical Long-form Question Answering, October 2024.
Kyu-Hwan Jung. Large Language Models in Medicine: Clinical Applications, Technical Chal- lenges, and Ethical Considerations. Healthcare Informatics Research, 3142):114-12, April 2025. ISSN 2093-3681. doi: 10. 1044258hhi.2222331.2.111
Takafumi Kanamori, Shohei Hido, and Masashi Sugiyama. A Least-squares Approach to Direct Importance Estimation. Journal of Machine Learning Research, 13941): 2009. ISSN 1533-7928.
Haoqiang Kang and Xiao Xiao-Yang Liu. Deficiency of Large Language Models in Finance: An Empiri- cal Examination of Hallucination, November 2023.
Bhawesh Kumar, Charlie Lu, Gauri Gupta, Anil Palepu, David Bellamy, Ramesh Raskar, and An- drew Beam. Conformal Prediction with Large Language Models for Multi-Choice Question An- swering, July 2023.
Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale N. Fung, Mohammad Shocybi, and Bryan Catanzaro. Factuality Enhanced Language Models for Open-Ende Text Generation. Ad- vances in Neural Information Processing Systems, 35:34586-34599 December 2022.
12

---

Published as a conference paper at ICLR 2026
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-Augmented Generation for Knowledge-Intense NLP Tasks. In Advances in Neural Information Processing Systems, volume 33, pp. 9459-9474. Curran Associates, Inc., 2020.
Moxin Li, Wenjie Wang, Fuli Feng, Yixin Cao, Jizhi Zhang, and Tat-Seng Chua. Robust Prompt Optimization for Large Language Models Against Distribution Shifts. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.). Proceedings of the 2023 Conference on Empirical Methods in Natu- ral Language Processing, pp. 1539- 1554, Singapore, December 2023. Association for Computa- tional Linguistics. doi: 10.18653/v 1/2023.emnlpppppp main.95.
Stephanie Lin, Jacob Hilton, and Owain Evans, TruthfulQA: Measuring How Models Mimic Human Falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214-32112 3252, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 1/211663//11222.ac 1/00://.0122....ooooo2222222 lon..ll....2.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. FActScore: Fine dinengr Atomic Evaluation of Factual Precision in Long Form Text Generation. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12076-12100, Singapore, December 2023, Association for Computational Linguistics. doi: 10.18653/w1/20233emml emnlp-main.745
Christopher Mohri and Tatsunori Hashimoto. Language Models with Conformal Factuality Guar- antees. In Proceedings of the 41st International Conference on Machine Learning, pp. 36029- 36047. PMLR, July 2024.
David Nadeau, Mike Kroutikov, Karen McNeil, and Simon Baribeau. Benchmarking Llama2, Mis- tral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations, April 2024.
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloun- dou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted question-answering with human feedback, June 2022.
XuanLong Nguyen, Martin Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by penalized convex risk minimization. In Advances in Neural Informa- tion Processing Systems, volume 20. Curran Associates, Inc., 2007.
OpenAl. Gpt-4 technical report.
20233.
Long Ouyang. Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Training language models follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, December 2022.
Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi S. Jaakkola, and Regina Barzilay. Conformal Language Modeling. In The Twelfth International Conference on Learning Representations, October 2023.
Shauli Ravfogel, Yoav Goldberg, and Jacob Goldberger. Conformal Nucleus Sampling. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Compu- fational Linguistics: ACL 2023, pp. 27-34, Toronto, Canada, July 2023. Association for Compu- tational Linguistics. doi: 10. 186518663/vl1 http//2222...n2...../..
Jan Heinrich Reimer, Sebastian Schmidt, Maik Fröbe, Lukas Gienapp, Harrisen Scells, Benno Stein, Matthias Hagen, and Martin Potthast. The Archive Query Log: Mining Millions of Search Re- sult Pages of Hundreds of Search Engines from 25 Years of Web Archives. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Re- trieval, SIGIR '23, pp. 2848-2860, New York, NY, USA. July 2023. Association for Computing Machinery. ISBN 978- 1-4503-94008-
doi: 10. 00.1145/353961..300 33591990.
13

---

Published as a conference paper at ICLR 2026
Zhiyao Ren, Yibing Zhan, Siyuan Liang, Guozheng Ma, Baosheng Yu, and Dacheng Tao. To- wards reliable medical llms: Benchmarking and enhancing confidence estimation large lan- guage models in medical consultation. arXiv preprint arXiv:2601.15645, 2026.
Glenn Shafer and Vladimir Vovk. A Tutorial Conformal Prediction. Journal of Machine Learning Research, 9(2))371-421 2008. ISSN 1533-7928.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density-ratio matching under the Bregman divergence: unified framework of density-ratio estimation. Annals of the stitute of Statistical Mathematics, 64(5):100-10044, October 2012. ISSN 1572-9052. doi:
1211077/11466--11-0044333
Ryan Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal Predic- tion Under Covariate Shift. In Advances in Neural Information Processing Systems, volume 32.
Curran Associates, Inc., 2019.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Ar- mand Joulin, Edouard Grave, and Guillaume Lample. LLaMA: Open and Efficient Foundation Language Models, February 2023.
Dennis Ulmer, Chrysoula Zerva, and Andre Martins. Non- Non-Exchangeable Conformal Language Gen- eration with Nearest Neighbors. In Yvette Graham and Matthew Purver (eds.), Findings of the As- sociation for Computational Linguistics: EACL 2024, pp. 1929, St. Julian's, Malta, March 2024. Association for Computational Linguisties.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. Springer, 2005.
Wenbin Wang, Yongcheng Jing, Liang Ding, Yingjie Wang, Li Shen, Yong Luo, Bo Du, and Dacheng Tao. Retrieval-Augmented Perception: High-resolution Image Perception Meets Visual RAG. In Proceedings of the 42nd International Conference on Machine Learning, pp. 63290- 63307. PMLR, October 2025.
HuaJun Xi, Kangdao Liu, Hao Zeng, Wenguang Sun, and Hongxin Wei. Exploring the Noise Ro- bustness of Online Conformal Prediction. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, October 2025.
Yu-Jie Zhang, Zhen-Yo Yu Zhang, Peng Zhao, and Masashi Sugiyama. Adapting to Continuous Co- variate Shift via Online Density Ratio Estimation. Advances Neural Information Processing Systems, 36:29074-291
December 2023.
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. WildChat: IM ChatGPT Interaction Logs in the Wild. In The Twelfth International Conference on Learning Representations, October 2023.
14

---

Published as a conference paper at ICLR 2026
Algorithm 1 Cofact's Online DRE Framework, adapted from Zhang et al. (2023)
Require: Calibration data Do (2: 1-=) number of time steps T
1: Initialize the set of lifetime length list C = [1, 2, 4, ......lo,,, 71]
2: Initialize the active set of experts A with [C] initialized experts
3: for /.= -....., ,,"' do
4:
for L do
5: if = 0 mod L then
6:
Reinitialize the expert (its 0, E and v) corresponding to the lifetime length L, i.e.
Allog2 L]
7:
for & do
8:
Update Pt, using εx-1, and 1z-
9:
Aggregate the global model 0,
10:
for &, E do
11:
Update the parameters of & i.e., 0.+1, Update the potential Vt,i and step size εr,i
A THE USE OF LLMS
In this work, LLMs were solely used for text polishing to enhance the clarity and readability of the manuscript. LLMs played no role in generating research ideas, problem formulations, proofs, theorems, algorithms, experiments, results, figures, evaluations. All content produced refined by LLMs was meticulously reviewed and validated by the authors to ensure accuracy and consis- tency with the intended meaning. The technical contributions and intellectual work presented in this study are entirely the authors' own.
B OMITTED ALGORITHM DETAILS
In this section, we outline the approach to minimizing dynamic regret, as defined in Equation 14 us- ing the online ensemble framework proposed by /Zhan et al. (003)). At a high level, the framework maintains pool of experts, where each expert models density ratio estimator over its designated lifetime. each time step, the predictions from all active experts are aggregated form global model, which provides the final density ratio estimation.
The overall algorithm detailed Algorithm which consists of three main steps: active-set update (lines 3-6), model aggregation (lines 7-9), and expert update (lines 10-12). Below, we provide an overview of each step.
Active-set update. The algorithm maintains set of experts, each assigned lifetime length cho- sen geometrically as 20, 20,2", 22 ..... At each time step f. the algorithm checks if any expert's lifetime has expired. If so, the expired expert is re-initialized with updated parameters, including the model parameter d,,,, the potential u, and the step size 6,,- Specifically, the initialization procedure is as follows:
Model parameter initialization. The model parameter Ú,, is initialized to the current global model: 0,, = 0,.
Potential initialization. The potential v.a is initialized U2 1/T.
Step size initialization. The step size 6,, initialized as <,, = min (1/2, vnn
Model aggregation. After updating the active experts, the algorithm aggregates their predictions to form global model. This global model is then used to make predictions for the current test sample 7 For each expert, a "potential" vr,; is maintained reflect its historical performance, while step size ε,, controls the update of this potential. The weights and the global model are computed as follows:
0-1,(01-11
and 0, = Σ Er.de.. iEA
Ptçi Eiea 1-1,,01111
15

---

Published as a conference paper at ICLR 2026
Expert update. Once the global model 0g is obtained, each active expert &, is updated using the newly arrived test sample Zn+s- The expert update consists of two components: model parameter update and updates to the potential and step size.
1. Model parameter update. The model parameter d.,: is updated using the online Newton step (ONS) method (Hazan et al. 2007 Agarwal et al., (006), which incorporates second-order in- formation achieve efficient and adaptive updates. The update rule is given by:
0.+1,1 = A [0r. -
A V (O..s))
where A is the accumulated second-order matrix defined as:
Ar.: = + Σ vL. VL. T=8i
and 8; denotes the last initialization time step expert &,. The term 11 represents the pro- jection of the updated parameter onto the feasible set θ, with the projection performed under the norm induced by the matrix At,i- This ensures that the updated parameter remains within the allowable parameter space.
Potential and step size update. The potential vr,, and step size ε,, are updated to reflect ex- pert's s performance. First, we define the term m,,i which captures the performance gap between the expert 0r. and the global model 0g over the linearized loss:
(L). O, 0r.s)
m4,
SR
Using ma,s the updates performed as follows: Potential update. The potential vt; updated as:
Ut = 0--11 (1 + m-1,
Step size update. The step size Ce is updated as:
In T
Et min
2 2'1
1+ Et= m
C OMITTED EXPERIMENTAL DETAILS
C.1 OMITTED DATASET DETAILS
MedLFQA (Jeong et al. 2024g) MedLFQA is a long-form medical question-answering dataset that integrates several previously established benchmarks. Each prompt in the dataset is paired with responses generated by either an LLM or human. To obtain sub-claims annotated with factuality labels, Cherian labels,Cherian al. (2024) first GPT-3.5- to generate responses for the prompts and GPT- 4o to parse these responses into self-contained sub-claims. Then, the factuality each sub-claim assessed by querying GPT-335-Turbo, which evaluates the claims based the LLM or human- generated responses provided for the prompts.
WikiData (Cherian et al. (024)) WikiData is constructed by first sampling names from Wikipedia and then querying GPT-3.5- Turbo with the prompt: "Write me a short biography of [NAME].". After that, the generated biographies are parsed into self-contained sub-claims using GPT-4o. Factuality labels for these sub-claims are assigned using a variant the FAcTScore procedure developed by Min et al. (2023). This process involves identifying relevant Wikipedia passages using the BM25 ranking function and incorporating them into the LLM prompt to determine whether the claims are supported.
16

---

Published as a conference paper at ICLR 2026
WildChat+ To evaluate our method in real-world dynamic setting, construct new dataset from WildChat Zhao et al. (2023). which features user-generated prompts in natural, uncontrolled environments. Since not all responses elicited by prompts in the dataset suitable for the hallu- cination mitigation task, we first filter them by using GPT--4-mini to identify whether the prompts be answered using knowledge available Wikipedia. For the filtered prompts, GPT-40 assigns relevant Wikipedia titles, and we retrieve the corresponding passages using the Wikipedia API. Fi- nally, we apply the FAcTSScre procedure, following Cherian al. (2024), to annotate the factuality labels of the claims in the responses. Due to the high cost of annotation, we randomly sample 1450 prompts from the filtered prompts for annotation.
C.2 OMITTED DETAILS OF SHIFT SIMULATION
Here, we describe the procedure for sampling Za+z from Diest to simulate covariate shifts. Each prompt is associated with metadata, which is used to define feature vector x. Prompts Zazz sampled with probabilities proportional to w(x) = exp(x" ((1 ((1-(e) ft)v/ / +(v")), where v' and y" are predefined weight vectors, and & E [0, 1] is varyyng factor. Since both Do and Diest originate from the same underlying distribution, this resampling strategy effectively simulates shift between the initial distribution Do and the deeedeeendent test distribution D,.
For the MedLFQA dataset, the feature vector x is defined using five attributes, as detailed in Cherian al.] (2024): the length of the prompt, the length of the response, the mean log prg--bbabilltty of the response given the prompt, the standard error of the log- log-probability of the response given the prompt, and the dataset from which the prompt originates. / and v" are configured such that v" assigns higher weights to prompts with longer responses that the sampling favors prompts with longer responses over time.
For the WikiData dataset, the feature vector x is constructed using the first, second, and third powers of the number of views received by the Wikipedia pages corresponding to each prompt, following Cherian et al. (2024). In this case, 2/ and y/ are chosen such that n assigns higher weights to prompts associated with less-viewed Wikipedia pages.
The time-varying factor & introduced model four types of covariate shifts: Linear Shift (Lin), Square Shift (Squ), Sine Shift (Sin), and Bernoulli Shift (Ber). Each type captures distinct real- world scenarios in which data distributions evolve over time:
Linear Shift (Lin): & is defined as & = t/T, representing a gradual and continuous change in the environment over T time steps.
Square Shift (Squ): & alternates between and every M steps, where 2M defines the period length. To model rapidly changing environment with periodic behavior, we set M O ((VT).
Sine Shift (Sin): & follows a sinusoidal pattern, defined as & = sin(Tt/M) where M represents the period length. Similar to the Square Shift, we set = ((VT).
Bernoulli Shift (Ber): & retains the value of & with probability p E [0, 1]. and switches to
1 - with probability 1-p. pp. To simulate rapidly changing environment, we set ((IVTT).
C.3 OMITTED IMPLEMENTATION DETAILS
We preprocess the MedLFQA and WikiData datasets in accordance with the methodology described by Cherian et al. 2024). For the baseline methods, SCP and CondConf, we utilize the original code provided by Cherian et al. (2024). To implement our approach, we adapt the code from Zhang et al. (023) to perform the online density ratio estimation procedure, using the same hyperparameter configuration as specified in their original implementation. All experiments are conducted on machine equipped with a 16-core Ultra 9 285H CPU.
D ADDITIONAL EXPERIMENTAL RESULTS
In this section, we present additional experimental results including comparisons with several oracle baselines and ablation studies of various components of our method.
17

---

Published as a conference paper at ICLR 2026
Table 4: Comparison with the method handling covariate shift on MedLFQA dataset.
Lin
Sqg
Sin
Factuality
Chains Retainol
Resained
Factuality
Clains Retained
Claima Retained
Factuality
Clainss Retained
Total Partial
08556000: 015
0.8400..211
0112
0.348.0.022
0.858.0.015
0.53.0.015
6.5346.011
3.9990.011
0 0.262..012
8000000:00 0
0.825an1226
0 0:806.0.012
0.822.00025
0.92200112
0. 0.566100166
6.99500.008
054220.012
0.881000111
Colaett
0189560026
0.78840000
0.8945000088
0.71500031
0:90500011
0.71340.077
0500
0.715600127
Table 5: Comparison with the method handling covariate shift on WikiData dataset.
Squ
Fartuality
Ohinne Rotained
Factuality
Cliins Retained
Pactsollit
Clairas Retainnd
Pactrality
Claime Rerailned
Fartuality
Claime Rebained
Total
( 0.955.0010
766 004
0.895an.00 D09
0. 0.26kan pos
0.895.00006
0.76880.0004
0. 0.887a0:012
6. 6.77300.004
0.58530.00 000
1.77 004
Partial
740tet 1077
0.806-000111
0.780:40 It1
0 0.875400014
0/70440 2017
78560
CoFact
0.896e9.001
0.895an.000
0.248a000000
0.895a0.008
0.748a0.000
0,897500000
6.749a0.006
0.59650.010
0.753x0.007
D.1 COMPARISON WITH METHOD IN CONFORMAL PREDICTION UNDER COVARIATE SHIFT
To provide a more comprehensive evaluation of CoFact, we compared it against existing approaches for handling covariate shifts by implementing the method proposed by Tibshirani et al. (2119), which specifically addresses covariate shift in conformal prediction. This method requires access to the entire set of test samples to train fixed density ratio estimator between the training and test distributions. To thoroughly assess its performance, we implemented two variants of this method: (I) Total: This variant uses all test samples to estimate the density ratio and filter potentially hal- lucinated sub-claims; (2) Partial: This variant uses only the test samples observed during the first 500 time steps to estimate the density ratio and filter potentially hallucinated sub-claims for all time steps. It is important to emphasize that both variants are impractical in our online setting, as they require access to either the entire set of test samples or a substantial portion of them in advance, which violates the constraints of online learning. But we include them as oracle baselines to pro- vide insights into the effectiveness of density ratio estimation in improving factuality control under covariate shifts.
In addition to the four shifting scenarios introduced the manuscript, we also evaluated these variants under an additional scenario, Fixed Shift (Fix), where the test distribution remains constant but differs from the training distribution setting similar to that adopted by Tibshirani et al. (0199).
The results of these experiments on the McdLFQA and WikiData datasets are presented in Table 4 and Table [5] respectively. From these results, we can make the following observations. First, The Total variant performs better than the Partial variant in terms of factuality control, indicating that access to all test samples improves density ratio estimation and enables the construction of better factuality control. Second, CoFact consistently outperforms both Total and Partial variants across nearly all shift types and datasets in terms of factuality control. Even under fixed shifts, CoFact achieves superior or comparable performance the baselines, demonstrating its robustness in handling both stable and dynamic covariate shifts.
D.2 COMPARISON WITH METHOD IN ONLINE CONFORMAL PREDICTION
We also compare our method with the approach proposed by Gibbs & Candes 2021 which is de- veloped for online conformal prediction under potential distribution shifts over time. However, this method is not directly applicable to our setting, as it assumes immediate access to true labels after each prediction. Despite this mismatch, we include as a baseline to provide more comprehensive evaluation of CoFact.
We evaluated two update strategies from Gibbs & Candes 2021) ACI-Simple, defined by Equa- tion (2) in the original paper, and ACI-Momentum, defined by Equation (3). Additionally, we varied the step size parameter ? over (0.0005, 0.001, 0.005, 0.01) to analyze its impact on performance. The results on the MedQA and Wiki datasets are presented in Tande T respectively.
From the results, we can draw the following observations. First, the ability of ACI to precisely control the factuality level is highly sensitive to the choice of the step size 7, A step size that is too small results in poor control of factuality, while a step size that is too large produces overly conservative prediction sets with low number of retained claims. Although the original paper recommends setting 7 = 0.005, our results indicate that this choice leads to overly conservative filtering in our scenario. This suggests that ACI requires extensive tuming of y to achieve reasonable performance across different contexts. Secondly, although CoFact does not rely on access to true labels during the online learning process, it still achieves comparable factuality control performance
18

---

Published as a conference paper at ICLR 2026
Table 6: Comparison with the method handling online conformal prediction on MedLFQA dataset.
Lin
Sqn
Sin
Ber
Factuality
Clainss Retained
Factuality
Claims Retalmed
Factuality
Claims Retalined
Factuality
Claims Retained
ACT- ACT-Simple-0.000
0.832:0.000 0.0
0.747 00.05
0.854:0003
0.023:0.055
0.8851:0.002 0.00
0.736:10.05
0.884:0000
0.751 00,08
ACI-M ACI-Monentumm-
0,0005
0.882
3247200005
080010000
0.725.00.055
0.885110.003
0.735
0.885.0000 001
0.747110037
ACI- SCI-Simple00.01
0.0091
10.01
0.91210003
0.617_00073
0.912110.002
0,508.10.07
0.11 0000
0.65310.770
(11--onnettum--..00
0.910=::1004
0633=000511
0:913+00004
8612-0.073
0.912+400002
0.606++0.078
0.113 000
0,639+00000
ACI-Simple-U1.05
0,975
0.207=0.150
0
0.194:00.855
0.97362000005
0.385:+0.041
0572::11000 0006
0.209:0008
ACI-Momentuman
0,976-+00005
0.206-0.063
0.980+00004
D. 0.153-0.043
0.999)-0.002
0.167+0.053
0977-0 000
0227-00.99
ACI-Simple--0.01
0.956-03003
0.131-0.020
0.957+0005
0.114-0.041
0.968+00.002
0.115+0.054
0.85-00000 0.000
8.161+00 -0.61-0.90
ACI-M ACI-Momnnntum-O.0
0.967.10.004
0.121
0.99010:002
0.088_00.030
0.989100:005
0.102.10,054
0.991_0.005
0.00910.026
CoFact
0.,051
0.715.00.031
0.891110.022
0.718_00.030
0.894110.0018
0,715:10.031
0.900.0.019
0,71410.006
Table 7: Comparison with the method handling online conformal prediction on WikiData dataset.
Lin
Squ
Sin
Ber
Factuality
Clainss Retsined
Faciuality
Claims Retained
Factuality
Claims Retained
Factuality
Claims Retained
ACT--Simpl--n.00s
0.804-01004
0.766--0.00 0004
0.895-000005
D. 0.765-00.005
0.894+000004
0.764+00003
0 0.892-0.000 0.006
0.760 0.760-+0.01
ACI-Momentum--l.oom
0.834+01004
0.765-0.004
0.896++00004
0.765-0.000
0L805--00000
0.764+0.0034
0 0.93-00000 000
0.740-+0.012
ACI- ACI-SSmple--.001
0 100000
0.75500.007
0.902 0.00100000
0.251 000000
0.902 1.000210:005
0.750.10.006
0.9031 0..05
0.746 10.015
ACT-Momentum--0.00
20002:0.002
0.753:0.010
0.903:20.000
0.750:00.000 0.0
0.900:2:0:02
0,750:10 007
0:003:0000 0.005
0.748100.01
ACI- ACT-Simpl--0.005
0.940-00.006 0.6
0.603-0.081
09402:00007
0.612-0.079
0.94):=00006
0,586:10.00 00.097
09:5=0.015
0.598++01154 134
ACI-Moasentum--Dee 0,005
0.9441250077
0.586.0.000
0.945100005
0.578.00.089
0.964110.007
0,577 191
0.9:8_0.0015
0.50810.148
ACI-Simples 0.01
0.95210.011
0.504 11?
0.95210.011
1.509_10.123
0.951110.015
0.514110:11 1.7
0.9611.0.011
0.44010.1244
MCI-Menecttum--0.0
0:966-000012
0.367=00110
00968=000055
0.339=0.1154
0.968:-11004
0.335==0:78
0.971=11.066
0.297=0.11 116
CoFact
0.896200010
0.748200:006
0.895:00009
00748:00.006
0.895
0,748:20:106
0.897:011000
0,749:008805
Table 8: Comparison with method handling robust online conformal prediction on McdLFQA
dataset.
Liu
Squ
Sin Claims Retained
Ber
Factuality
Claims Retained
Factuality
C Claims Retained
Factuality
Factuality
Claims Retained
FACI-LUS
0.893=00.022
0 0.343-30.081
0.894:000072
0.322:10.078
0.894:000012
0.321
0.804:10.001
0.358-0.058
eACI-0.10
0.89110.001
0.2851000755
0.89111.0000
0.294
0.296.10.06
0.89210.003
0.203_0.064
rAC1-0.15
0.88810.006
0.271
0.887:10.007
0.263-10.079
0.887:00.006
0.271
0.88910.005
0.000
rAC1-0.20
0.88-+000099
0.249:00.180
0.886::0.005
0.253=+0.050
0.886-=000000
0.236:+0.057
0.66611.006
0.226-00062
CoFact
0.895+00026
0.715-00:031
0.897+00.22
0.718+00.030
0.894-0.018
0.715+0.031
0.900+0.019
0.714-0.056
Table 9: Comparison with method handling robust online conformal prediction on WikiData dataset.
IJnn
Squ
Sin
Ber
Factuality
Cinims Retained
Factuality
Claims Retained
Factuality
Claims Retoined
Factuality
Claims Retnlined
rACI-0.05
0.89510.001
0.7321
0.89510.000
0.73710.018
0.000100001
0.73210.019
0.895
0.738_0.015
rAC1-0.10
0.80110.001
0.741110.01
0.890.10.001
0.742.1.0016
0.891 10.001
0.7:0010.016
0.390110.001
0.728_0.00
rACI-0.15
0.88610.001
0.71710.027
0.886:10.001
0.71740.028
0.886 0.00
0.71910.030
0,8861100002
0.713=0.029
rACI-0.20
0.88010.001
0.719100:05
0.8011000001
0.712110027
1.00110.001
0.715100002
0.87910.002
0.731_0.015
CoFact
0.89610.010
0.748.10.006
0.895:00.009
0.748.10.006
0.89510.0088
0.7481100066
0.89710.008
0.749-00006
to ACI with well-tuned y values, further underscoring its effectiveness in our online setting. It is important to note that while ACI can achieve controlled factuality and, in some cases, retain slightly more claims than CoFact, this does not imply that ACI superior. ACI relies on access to true labels after making predictions at each time step, requirement that is impractical in our online setting,
D.3 COMPARISON WITH ROBUST ONLINE CONFORMAL PREDICTION
In this section, we relax the assumption of inaccessible true labels and instead assume access to noisy labels in the online conformal prediction setting. Under this relaxed setting, compare our method with rACI, an extension of ACI proposed by Xi et al. (2025). which is designed to handle label noise.
To simulate a weak supervision scenario, we introduced uniform random noise to the true factual- ity labels in our experiments and compared the performance of CoFact with rACI. The results are presented in Table 8 and Table 9 for the MedLFQA and WikiData datasets, respectively. The noise ratio was varied from 5% to 20% to evaluate the robustness of both methods under different noise conditions.
From these results, we observe that while rACI exhibits some robustness to label noise, its perfor- mance deteriorates significantly as the noise level increases. Notably, on the MedQA dataset, the Claims Retained metric for rACI is extremely low, making it impractical for real-world applica- tions. Furthermore, rACI relies strong assumptions about the nature of noise, specifically that label-flipping probabilities are consistent across different sub-claims and samples, and that the true noise rates are known. These assumptions are rarely realistic in practical scenarios. Given these lim- itations, CoFact demonstrates clear advantages in terms of practicality and robustness for factuality control in online settings.
19

---

Published as a conference paper at ICLR 2026
Table 10: Experiments with varying calibration fractions on MedLFQA dataset.
Iin
Squ
Sin
Ber
Factuality
Claine Retained
Factuality
Clains Retained
Factuality
Clains Retained
Fartuality
Claims Retained
calibration frac=0.05
0.920-0.038
3613-00000 094
0.921 +0.034
0612-0.09
0.933-0003
0.62--0.09 0.099
0.91 040004
0:62100 180
calibrationn_aaaec Frac=1.11
0.902-0.015
0.669 0000
6.900+0.019
0674-0003
0.94+00018
0.83-00.00
0.91 +-0,009
0.633-0.12 136
calibratioon 1rac-0,33
0,89010.014
0.722.0.057
0.8:910.006
0.22110.0566
0.80110.018
0.753_0.057
10.015
0.666.0:017
Table 11: Experiments with varying calibration fractions on WikiData dataset.
Squ
Sin
Factuality
Clainns Retained
Factuality
Claine Retaline
Factuality
Claims Retained
Factuality
Clains Retained
calibration Trac=fi55
0.899-0.012
0.742-0019 0.011100013
0-808+00012
0744-00199
0.80000011
0.343-0.010
0.750-0:016
calibration frac=0.1
0.905 0.003
076110011 0750=00013
0.261_0.013 0.750=0.011
0.897.00.015
calibration, Trac=t.33
08922000012
0.21
0.992200013
0.83380013
0.894220.01
0.754=0:009
Table 12: Experiments with varying expert number base on MedLFQA dataset.
Lin
Squa
Sin
Ber
Factuality
C Claims Retained
Factuality
Claims Retained
Factuulity
Claims Retained
Factuality
Claims Retained
expert Laum, 3use=3
10.011
0.0022
0.858.1.0019
0.712.00000
0.887 00.02
0,714100022
0.0011
0,713L0.032
expert aum basew5
0.508 622
0.20000022
0.87700.002
0.71950225 0:4400024
0
0.723=00028
0.99:20.000
expert www..auuusseeeeeee 100
0.802+0.00 620
0 0715-0.025
0. 0.896+0.000
0.900-0.02
0,714+0022
0.84-00014
0.713-0:229
Table 13: Experiments with varying expert number base on WikiData dataset.
Squ
Factuality
Retaiind
Factuality
Retained
Factuulliy
Claims Retuined
Factuality
Retuined
expert num basew3
0100
0.899800011 894+000007
0.74220008
0899 0.894-0.007
0.745 0.747+00000 0.746.10.088
0.750=
expert_num, Iusess expert num. buse=10
0.66-0.010 0.74510.00s
0800-0013
0.001100011
0,90010.0055
0.746.10.07
0,903100115
0,901_0.012
0,705
D.4 ABLATION STUDY ON CALIBRATION SAMPLE SIZE
We conducted experiments using different calibration set ratios, specifically {0.05, 0.1, 0.2), on the MedLFQA dataset (corresponding to approximately 240, 480, and 1440 samples) and the WikiData dataset (approximately 425, 850, and 2550 samples) under four shift types. These experiments were designed to assess the impact of calibration set size on the performance of our method. The results for the two datasets are summarized in Table 10 and Table IT respectively.
The results indicate that our method exhibits strong robustness to variations in calibration set size. Even with a relatively small calibration set (approximately 240 samples for MedQA and 425 samples for Wiki), our method achieves satisfactory performance in terms of both Factuality and Claims Retained. This underscores the practicality of our approach, particularly in scenarios where only limited calibration data is available.
D.5 ABLATION STUDY ON VARYING EXPERT NUMBER
To investigate the impact of the number of experts, adjusted the expert lifetime schedule by modifying the lifetime scaling factor from 2 to 3', 5', and 10'. This change alters the number of active experts each time step log, logs L, and log respectively. The experimental results on the MedLFQA and WikiData datasets, evaluated under four shift types, are summarized in Table T2 and Table 13
The results show that our method consistently achieves reliable performance in terms of both Fac- tuality and Claims Retained across different expert base settings. This indicates that the method is robust to variations in the number of active experts, further highlighting its adaptability to different configurations.
D.6 ABLATION STUDY ON VARYING STEPS PER UPDATE
To further evaluate the applicability of our method in an online batch learning setup, we performed
experiments using different batch sizes ((1, 5, 10, 20)) on the MedLFOA and WikiData datasets across four shift types. The results are summarized in Table 14 and Table 15]
The findings indicate that our method consistently maintains strong performance in terms of both Factuality and Claims Retained when processing incoming samples in batches. This demonstrates the suitability of approach for online hatch learning scenarios.
20

---

Published as a conference paper at ICLR 2026
Table 14: Experiments with varying steps per update on MedLFQA dataset.
Lin
Squ
Sin
Ber
Factuality
Claims Retained
Factuality
Claims Retained
Factuality
Claims Retuined
Factuality
Claims Retained
per step num-5
0.89310.024
0.7
0.5
0.722-0.027
0.887.0.029
0.726:0.023
0.896
0.71200.00
per.Nlep.mum=10
0.89710.024
0.895 0005
0.701 0,028
0.89710.025 0.999000225
0.709 0.0980000000
0.00210.012
0.696100027
per step. numm20
0,899200024
0.6962200011
0:00=00025
0.003=000322
0.04200001
0.684-20.002
Table 15: Experiments with varying steps per update on WikiData dataset.
Squ
Sin
Factuality
Retnined
Factuallity
Claims Retained
Factuality
Claims Retained
Factuality
Clains Retained
per step num-5
0.900100.011
0.750100088
0.81
0.050
0.9011:0001
0.7472:0.008
0.89210.0166
0.04010.0111
per step. 10mm100
0,903+00002
00748
0:905-0.009
0.749-0.000 0.747.0.000
0.901+00.000
0.748-0.007 0.748.0.007
0397++0.008
0.747+0.01
per per.step.num-a
0.899100.07
0.708.10.066
0.90 10000
0.90010.007
0.900.0.0033
0.751 10000
Table 16: Experiments with varying noise levels on MedLFQA Dataset.
Squ
Factualiry
Retained
Factualltty
Claims Retained
Factuality
Claims Retained
Factuality
Claims Retained
0.806100:011
0.89
0.220:1:002 0,74240.04 0.776:000047
0.99
0.725:1.002
0.713
0.892:0.0044
0.33+000441
0.891
0.03-00025
1.746++0039 0.781=00046
0.902-0.021
modse level=0.15
0.884200.08
0.776:000088
0.779
0.32=000028
0.893
0.708=00041
Table 17: Experiments with varying noise levels on the WikiData Dataset.
Lin
Squ
Sin
Ber
Factuality
Cluims Retained
Factuality
Cluims Retained
Factuality
Claims Retained
Factuality
Claims Retained
mober level=11.15
1.89-+0.000
0.755+0.008
0.897
0,255+00.00 0,76110.010 0,76550.010
0.005
0.755-60.00 007
0.752-n 006
moise 1ese=.1ese:00110
0.761 0.103:00.000
0.895100
0,758 1..0000.000
Jevele0.15
0.89260.099
0.365=0.010
0.89
0.992=00008
0.944:0.010
0.76
D.7 EXPERIMENTS ON VARYING NOISE LEVELS IN FEATURE REPRESENTATION
We conducted experiments to evaluate the sensitivity of our method to feature representations by introducing Gaussian noise with varying levels of variance to the feature representations on the MedLFQA and WikiData datasets. The results are summarized in 16 Table 17
To ensure a meaningful analysis, the feature representations were normalized prior to adding noise, resulting in median feature value of approximately 0.15. As such, introducing noise level of 0.15 constitutes a substantial perturbation to the features.
The results show that even under relatively high noise levels, our method consistently maintains robust performance in terms of both Factuality and Claims Retained. This highlights the resilience of our approach to variations and noise in feature representations.
E MORE CASE STUDIES
Prompt: What is MVC in software engineering?
Filtered Response: MVC stands for Model- Viadll-iewwCCotrolleer, which is design pattern commonly used in software engineering.
Model: Represents the data and behavior of the application. It a bridge between the View and Controfler, handling the business logic and data manipulation.
View: Represents the user interface of the application. It displays the data to the user and sends user inputs to the Controller.
Controller: Acts an intermediary between the Model and the View. It processes user input from the View, interacts with the Model to retrieve or update data, updetts the View-aceordingly.
MVC helps to separate concerns and facilitate maintainability, reusability, and scalability of the codebase. is commonly used in web development frameworks like Ruby on Rails, ASP.NET MVC, and Spring MVC.
In this case, CoFact effectively removes the hallucinated claims "It acts as a bridge between the View and the Controller" while retaining most of the correct claims.
21

---

Published as a conference paper at ICLR 2026
F THEOREM AND PROOF
F.1 USEFUL LEMMA
Lemma 1 (Hoeffding's Inequality). Let X1,..., Xa be independent random variables with X; Vlow how + L, almost surely. Define the sample mean Xn = Ei= X2 and μ = E[Xn]. Then for any 8 O, with probability least 6.
log (2/5)
- - L.
(21)
2n
Lemma 2 (Azuma-Hoeffding Inequality). Let (Ma)i= be martingale difference sequence with respect to the filtration [F i.e.
B[M Ft-1)= 0, 4t= I.....
Suppose the differences bounded almost surely by
[M S G,
V1 1, ,,...,
Define the partial average B,, Σ Mc. Then for any with probability at least 1 6, we have
BA S cV log (2/5).
(22)
(023)), Suppose Assumptions and 3 hold. Denote [e]+ = m 0}. Let d be the dimension of the parameter space O Then, for any density ratio estimator (2) = h(;0)) E Hr. the empirical estimation error is bounded by
Lemma (Theorem in Zhang et al.
1
Ex (a) [x) (x(x)) s
Σ (A.)
T Σ (x))t)
+0
Vdlog(T/0) HVn
T
μT
(23)
provided that h(Z,0) 0) is bounded for any Z E and 0 θ and Lipschitz continuous.
Lemma 4 (Theorem 2 in Zhang et al. 2023 ). Suppose Assumptions X and 3 hold. Then, with
probability least 1-8 8, the dynamic regret of the density ratio estimator sequence {fr]T= learned from Algorithm 7 bounded by
Σ (T()) S
max { 1)
(24)
when the parameters are set as 7 3(1 + 3) and = 1. In the above, V1 = -EE D (x)]] 1 measures the variation of input densities. B exp(SR) represents the maximum value of the estimated density ratio ft.
(D,
Corollary 2. Suppose Assumption 2 and hold. Then, with probability at least 1- the dynamic regret of the density ratio estimator fx(x) = exp(-o(z) (0,) is bounded by
T
Σ
Ex-D, -x < 2 (n- max & t=1
(25)
T
when the parameters are as 7 3(1 + 9) and X 1. In the above, VT ( 1x-1(x)11 measures the variation of inpul densities.
F.2 NOTATIONS AND PRELIMINARIES FOR MAIN THEOREM PROOF
For clarity, here we define some notations used in the proof.
Let & denote the event that there exists any hallucinated claim in the prediction set obtained by true density ratio using at time step i.e..
e, = 3C E Fr(Cn+t) such that W = 0).
(26)
22

---

Published as a conference paper at ICLR 2026
Similarly, denote the event that there exists any hallucinated claim in the prediction set obtained by estimated density ratio time step &,, i.e.,
& Cn+i, Pr(Cnte) such that W Wa+e, =0]
(27)
As result, we have err, [ and err, [ Recall that the initial calibration dataset
Do {(Pt, R,, C,,, W.)
At time step I, the algorithm observes the partially labeled data point
Dj Zn+e (Pr+s, Rutt, Cn+t):
while the label W ++ is not observed at decision time. For analytical convenience, we define the corresponding label-augmentdd data point
D = (Zn+e, Wn+e) = (Prte, Ru-t, Cu+t: Wn+t),
which contains the unobserved label Wn+*.
We define single filtration that captures all information available prior making the decision time t:
Fi-1:= (Do, (D.) D.), t t=
By construction, the threshold it is -s-F---mmauurable, since it is determined by the initial calibration dataset Do and the previously observed data points (D,) The event & is F--measurable, as depends 9t and the label W.+*.
Moreover, conditional on the observed covariates Cn+t (equivalently, on D,), the label Wa+t is independent of the past history encoded in Ft-1- Although W is not observed by the algorithm time f, is well-defined on the underlying probability space. This distinction between observability and measurability is sufficient for the martingale- based analysis used in subsequent proofs.
F.3 PROOF OF THEOREM
Lemma 5. For any > 0, with probability at least 1 - 6.
1
1
Σ
P & Fi-1)
S/1 $V7
2
log
2
(28)
T
T
Proof. Define
Y :- err, PPEE F.-1) 1[ F(-1); 1= I.
Since 0 S 1[ < 1, we have Y < almost surely. Moreover, by construction, E|Y F 1 [ F [[1 Fi-1] = 0.
Thus (Y,)², is martingale difference sequence with respect to (F.) Applying the Azuma-Hogffding inequality yields that, with probability at least
Σ Y
<
2T log
2 8
Dividing both sides by T completes the proof.
Lemma 6. Suppose Assumption 7 holds. Let ft and rī denote the estimated and true density ratios time step t, respectively. and let Zn+t be test sample drawn independently at time step t. Then, the following inequality holds with probability at least 1 - 8:
T
B
2
2 5
T (re(Ze+e r2 (Zn+t))
<
Σ
EZWD. [1((Z) - (i()) +
E
log
8, (29)
t=1
where B' is bound the differences
(Fx(2) rf for all
23

---

Published as a conference paper at ICLR 2026
Proof.
Here, we define filtration facilitate the analysis. Let
Fi-1= (Do- (D.) t 1=1...., T.
For the purpose of the martingale argument, we consider the filtration containing only information available strictly before the test sample at time t. This filtration may differ from those used else- where; however, all probability statements are taken over the same underlying probability space and can therefore be combined via union bounds
For each time step t,
U+ = (Pr(Z)))) - ri (Z(24))) EzzD, - ( (Z)1
(30)
By construction, ft is measurable with respect to F/-1. and Zn+t is drawn independently from D,. Thus, we have E[U, Fi-1] = O. Therefore, (U,)=1 forms a martingale difference sequence with respect to the filtration {Fi) Additionally, since [f((z) (2) is bounded by 8 for all 2, we have |U&] 3 almost surely.
Applying the Azuma-Hoefiding inequality (Lemma 2 we obtain that with probability at least 1 Ó-
1
(e) r r (Zn+e)l T
Σ
Bz (2) rf (2)1)
T
(31)
2 T
( .B',
2 5
log
V
Using the change-of-measune technique, we have
EzzD, (1A (2) = EznDA (2) - r
(32)
BBEzwD [Z) (Z)))
where the bound on rī (Z). Averaging over / ,,...,T T gives
1 Ez Ez-D, (Z)
B T
r;
E Ez-D, (Z) A
(33)
T
Combining Equation 31 and Equation 33 we prove the lemma.
Lemma 7. Given the hypothesis space H, satisfying Assumption 2 let 3' max,eh,, Ir(2) rf (2() and G maxzez,0e3 V 8) 12. For any sequence of density ratio estimators (ft 1T
and corresponding true density ratio {rī )T under distribution Do. the following bound holds with probability at least 1 -8:
T
dlog (68G, T 8
2
E Ez-D. (2()) 1=1
T E Bz-Do - - r2(2)]] $ 2,35' t=1
Σ
+
T
2n
T
(34)
Proof. We aim to bound the discrepancy between the population and empirical absolute errors for sequence of density ratio estimators. The main difficulty is that the estimators f are data-dependent, which prevents direct application of concentration inequalities such as Hoeffding's inequality. To address this issue, employ standard covering number argument: we first control the deviation uniformly over an e-net the hypothesis space H,, and then extend the bound to the entire space by considering approximation error.
Fix an arbitrary hypothesis r E Hy. Let be an selement from an c-net of 7, such that - S Define the function
= (r()) (2)
(35)
24

---

Published as a conference paper at ICLR 2026
We decompose the population-empirice discrepancy follows:
E EzzD, (2)]
71 Σ 9r (Zi)
1
I BZNDD (Z)] Ez lo (Z)] + ED 9r - Σ rr (z)
Term(a)
(36)
Term(b)
2
n
+
Σ 1=1
9r (3.) Σ (Z,)
Term(c)
We analyze each term separately:
Term (a) This term captures the approximation error due to using r' instead of r. By the properties of the e-net, we have:
(37)
(12-),1lr(Z)) r' s.
Term (b) This represents the estimation error for the approximating model r'.
For a fixed r', the random variables 9r (Z:) are i.i.d. and bounded by [0, By applying Hoeffd- ing's inequality, have with probability least 1 8:
log(2/8)
Ez EzzDo lyr (2))
Σ gr (Z,)
S B
(38)
2n
Applying the union bound over all elements in the e-net e-net.W(H.. ((Hr,,
el E, lloo). we obtain:
log ellllae)))))))
sup
E EZwD,lar(2))
Σ gr (Z:)
S 8'
(39)
n
2n
with probability at least 8.
Term (c) This term is also bounded by e, similar to term (a), due to the e-closeness of r and r'.
Adding these terms, the total bound for any model r H,p is:
log
Ez-Do (Z)]
Σ gr (Z,)
< 2c + 3
(40)
2n
By setting and summing over all e [T]. the final aggregated bound for the entire sequence of estimators under consideration becomes:
1/T.I.1
log
2 T
EZzDA 19,. (Z))
fF. (7+)
S
88
(41)
n
2n
Next, we focus bounding the covering number of the hypothesis space H,. Since parameterize the density ratio functions in H. using a parametric model (xxx, 0) with parameters in a bounded set O we relate the covering number of H, to that of θ
25

---

Published as a conference paper at ICLR 2026
Let 0, 0' E è be the parameters corresponding to the two density ratio functions r' E Ho. We can show that for any |0 - 81/12 $ e, the following inequality holds:
|| r' oo
ZE2 (r, ((2,)))((2,0))
- 12
where 1 maxzez,8Ee [i(Z, (T(2,0)] 0)ll 2 is the Lipschitz continuity constant of h.
As result, we can bound the covering number of H. in terms of || Ilso by the covering number of O in terms of ||2. Specifically, have:
WH. 1/T, |I [1~) ( (/(A, 112)
Given that the parameter space @ is essentially L2-ball with radius S, its covering number is bounded by (3S/e) Therefore, choosing e = 1/(G,,"), we obtain:
N(O,1/(G,,T) (12) S (3SG,T)
Combining these results, we conclude:
N(Ho,1/T, |1 < (3S
Substituting this bound into our earlier expression, we complete the proof.
Lemma 8. Suppose Assumption and 3 hold, then for any sequence of density ratio estimators (Fe)T= and corresponding true density ratios {r?]T=n the following bound holds with probability least ò:
T
(( sō BV
(42)
T
Proof. The proof straightforward by combining the results from Lemma 6 Lemma and Corol- lary 2
Lemma 9. Let V Z + R be any measurable score and define the population CDF under D3 by
Vr(v) := P=PZD.. (V S b).
Define the self-normalized weighted empirical CDF that includes the test point:
(Z,))
V((1) = Σ
wf,; ((V(Z)) S v),
WT,;
E Ese(n]u(A+H) (Z,)
iE(m)
Then for any E (0, 1), with probability at least - 5, simultaneously for all t E [T).
4T log
2B
sup
(V()) - [x()) s 8B
(43)
VEX
n
n
provided 2 2Ba log
27
Proof. Fix f E [T]. Define
go(z): = 1(V(=) 5 u).
Let
S,
7 (Z))
R₂ = ri E [0, B).
Define the standard self- nelf-normalized importance sampling CDF based only on Dy samples:
ri (1;) g. (7:)
(v)
Sn
26

---

Published as a conference paper at ICLR 2026
We begin by decomposing the error:
sup
vR (V(()) - v.((v) s sup
v((v)
V (v)
+ suu T (v)) vc(w))
vE
Term
Term
We bound the two terms separately.
Bounding Term (a). This term captures the perturbation effect of including the test point Zn- in the self-normalized IS CDF. We first express (v) in terms of V (i,((v) and gu (Zn+e). Specifically, using S, + R2 Eielniu(nt)) rī (2,), we can write
Sn
Re
v((0)
L en
+
gu (Zn+t).
Sn + Rs
Sn + Rs
Hence
Rs
Rr
B
[V(()) 10, (v)] =
9e (nn-)))) - V Vi,((u)]
S
Sn + R
Sn R
Sn
and therefore
B
sup V((e)) (e,n(u)] S Sn
(44)
Then, we lower bound Sn using concentration. Note that S ri (Z:) B and ED, [ri(] Hoeffding's 's inequality gives
1,
P
5 =P
Sn
1 n E - 15
(Z)))
exp 2B
Thus, if 2 2T log( 2T), then by union bound over E
[7]. with probability least 6/2
have S, 2 n/2 simultaneously for all t. On this event,
B 2B S n
sup (1(
V (v)
S
(45)
Bounding Term (b) This term captures the standard nelf-normalized importance sampling CDF approximation error based on n samples from Do. Recall that
(v) ED, (Z)) Ep, [r (Z) (Z)) and Ep.lri()) = 1.
Let
An(u): 1
(Zz)
B = - Σ (Z())
Sn B, 1|
n1
i=1
so that V (w) = An(v)/ Bn- For any U.
An(v) We(w) s
(An(v) V.(v)]
M [1)I BB, 1
An((v)
B,
Bn
BN
Bn
B,
since (.(v) E 10, 1]. Taking the supremum over u gives
Sup/ v.(v)) sup, (x v.(o)
BB, -
(46)
Bn
Bn
Because 0 < rI (Z,) go (Z.) I and the class {g. O E R} has VC dimension 1, standard VC maximal inequalities (e.g., uniform Hoeffding bounds for VC classes) imply that there exists universal constant c > such that, with probability at least - 8',
log(1/8")
sup (v) v,(v) S co BV/
In addition, Hoeffding gives with probability at least 8'.
log (2/8))
B, 1/ B
2n
27

---

Published as a conference paper at ICLR 2026
Choose 8' (2T) and union bound over t E [T] to make these hold simultaneously for all t with probability at least 1 - 5/2.
On the event Bn 2 1/2, combining with equation 46 gives
log
sup V em (v) Wy(w)l & 2 (n)(v)
- -11- 8B
71
after absorbing constants.
Combining the bounds Combining the bounds for Term (a) and Term (b), and using the high-
probability events from both steps, we proves equation 43
Lemma 10. Suppose Assumptions
2
and 4 hold. Then with probability at least 8,
P(E.)
E P(E.
F(-1)
< n1-/2
1/2 max {
1/3 , T- 1/2) 1/23)
(47)
Proof. Let (v(v)) denote the true cumulative distribution function (CDF) of V (Cn+t, Wn+1) and the estimated weighted CDF using the oracle density ratio estimator Ê as
(v)
Σ
1 (Z: S v),
(E(m)u(n+e)
where w (2,)
(Z,)
the true weights derived from the true density ratio r7, and
Ene ri (Z,) 2/ denotes the value of ( W.).
Similarly, the estimated weighted CDF at time step t is:
1 (V())
Σ
(x( S s.,,
(E(m)U(n+r)
where wi (Z:)
f.(2)) [e( (2,) are the estimated weights based on the estimated density ratio
fz.
Due to the nested property, i.e., the size of F3 non-decreasing w.r.t ft. the following equation holds: & (V(C,++e> Wn+e) > Th), (V(Cne Wn+1) > f.).
where and 6t are (l-a)-tu of vr and V,, respectively. Consequently, we have:
P[E = (.(r)) PÊE Fi-1l (ft).
Using these definitions, express the difference in average probabilities as:
T
T
T
P
Σ
pl Fz
v.(ft) - (T()
=
T
v.(F)) - V.(Ta) + -..(F) (,
+
(.(+)) (.((F)
v.(+)) Vi(Tt) V((F)) v.((t)
T
v.(F)) - V((+))
+
V (T()) - v..(F))
T
+ sup (.(v)- v((0)
1
Σ
V(((t) - (.(F)
torm (a)
torm (b)
28

---

Published as a conference paper at ICLR 2026
The forth to last equality holds because (.(T)) and fz.
(e((f) a, by definition of the thresholds 7r
Bounding term (a)
By Lemma 9 have with probability least 1 8:
T
log
47 8
23 Ô (n=2)
sup (v) (v)) - 8,1
T
Bounding term (b)
Expanding (fr( and V (ft), we have:
T (Fte
V (ft)
1
Σ
1/s S 6z) (wi - 1x (2i))
T
T
Am (()
Substituting wi (Z,) and we (Z,), we obtain:
サ Σ
(Z)))
P((2))
1l. sm)
Eje(m)u(ntt) ri (2,)
Eje(aju(nt)) Pz(Z,)
t= (e(n]U(++t)
(2() Σ EjEnjUNtt A((Z)) Po( E EsElmuen+e rt (Z()
Σ
L fil
Σjemnu((+t ri(Z;) Eyemmuunttoy fe(z))
t= ic|n)u(n+()
Simplifying the numerator of the fraction, let:
Term; = =2,
Σ
Σ
(Z))
jEin)u(n+e)
jE(n]U(n+(}
Expanding Term; have:
Term, = (Z,)
Σ
(17()) - f((2))
( +
Σ
(Z2))
jE(n (n++)
jE(n)u(n+e)
Summing over i E [n]
U {n + + we bound [Term as:
Σ
Term:
S
Σ rZ,z)
Σ
-i
(E(n]u(n-t)
JE(m]UAn+()
Σ
(Z)( (Z.))
Σ
(Z,)
jE(m|U(2+1)
22 (Z)))
Σ
Σ
(f( - (Zz))
sEln|u(n+()
Combining terms and simplifying, we find:
T
V
(tt)
+
Σ t=1
(fte
t=1
T
Σ
1/ fil
Term;
Σje(n) rZ)) E (())))
t= ic (E(n]u[n4))
Term,
Σ
Σ
T
EseWnu(nHe ri(),) Eje(n)u(n4))) Pz(Z,)
t=1 (E|nju(nte)
1
Σ
2
Σ
-;
T
Eeemnuunat f((Z)))
1=1
29

---

Published as a conference paper at ICLR 2026
Invoking Assumption 4 and taking to 1, obtain
T
2
term (b) v nl T
fz(2,) +2(2 +
(e(zee)) - - (Z
JEIn]
If instead to > 1, splitting the time average into & S to and & > to yields additional additive term at most oo (using (1(()) (x)) S 1), which is negligible and does not change the final rate.
Finally, bounding the terms using Corollary and Lemma 8 and combine with the bound on term (a), conclude the proof.
Theorem 1. Under the assumptions and 4 with probability at least 1 - 6, the gap between the averaged hallucination rate over T time steps and the target level bounded as
Σ cn, S Ô (max (VV -m
(19)
when the parameter of the online ensemble is properly set. Here, Vr= EI=>>>11D(2)) ( D
measures the variation of input densities and the notation Ô hides logarithmic factors of T and 1/6.
Proof. We begin with the following decomposition:
T Σ
1
Σ
P(E F(-1)
T
CTt a
T
cIT
T
concentration
T Σ
p(
Fi-1)
P(E))
T
T t=1
(48)
estimation bias
(E))
T
(c) oracle error
Bounding term (a). By Lemma have
5 which establishes concentration around the conditional mean,
term (a) - 1(2-1
with probability at least
Bounding term (b). By Lemma 10, which controls the discrepancy between the conditional error probability and its oracle counterpart, obtain
term (b) < Ö0 (nnn11/2 + max ( 1/31/811// 1-1/3)
with probability at least
Bounding term (c). By construction of the oracle filtering rule, P(E.) = a for all t, and hence
term (c)
Conclusion. Combining the bounds on terms (a) (c) and applying a union bound completes the proof.
30

---

Published as a conference paper at ICLR 2026
G LIMITATIONS AND FUTURE WORK
CoFact's theoretical guarantees apply to the entire time horizon and may not hold for smaller time intervals. Developing methods that provide finer-grained factuality guarantees is an important di- rection for future work. Additionally, while CoFact focuses on ensuring the factuality of filtered claims, other response qualities, such as informativeness and diversity, could also be required in certain scenarios. Extending CoFact to incorporate these aspects presents another promising avenue for future research.
31


Reviews:
[{"review_id": "Y3NuW6gEiu", "reviewer": "Reviewer_G3xv", "summary": "This paper addresses the critical challenge of providing statistical guarantees on the factuality of Large Language Model (LLM) outputs under distribution shift. The authors propose CoFact, a conformal prediction framework that employs online density ratio estimation (DRE) to adaptively reweigh calibration data, thereby maintaining factuality guarantees even when the exchangeability assumption is violated.  key contributions include:\nA novel framework combining conformal prediction with online DRE to handle continuous distribution shifts; Theoretical analysis establishing an upper bound on the gap between actual and target hallucination rates; WildChat+, a new dataset capturing real-world distribution shifts; Empirical validation on MedLFQA, WikiData, and WildChat+ demonstrating superior performance over baseline methods", "questions": "How many calibration samples n are needed in practice to achieve reasonable performance? The method requires know about feature representations $\\phi(z)$. How should these be chosen for different applications? Statistical significance testing would strengthen the claims on table 2, 3. There seems no ablation on key design choices (e.g., number of experts, expert lifetime schedule, choice of divergence function. How sensitive is performance to the feature representation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "The paper tackles a significant limitation of existing conformal prediction methods for LLMs, they rely on the exchangeability assumption, which is frequently violated in real-world applications. The integration of online DRE with conformal prediction is creative and well-motivated. The use of an ensemble of experts with geometric lifetimes to track evolving distributions is technically sound. THeoretical part is solid: Theorem 1 provides a rigorous bound showing that the hallucination rate gap converges to zero as O(max{T^{-2/3}V_T^{2/3}, T^{-1/2}} + 1/n). It also has detailed experiments on the evaluation in simulated shifts (4 types) and real-world shifts (WildChat+).", "weaknesses": "Assumption 1 requires that the conditional distribution P(W|Z) remains unchanged while only the marginal P(Z) shifts. This is quite strong and may not hold in many real scenarios (eg, if model quality degrades over time or if certain types of prompts systematically elicit more hallucinations). The paper only compares against SCP and CondCP. What about other methods for handling covariate shift in conformal prediction. Figure 2, Legend is difficult to read. The paper doesn't discuss how to set T in advance or what happens when the time horizon is unknown.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TiWF0Z3jaX", "reviewer": "Reviewer_XA7a", "summary": "To theoretically control the rate of hallucination of large language models (LLMs) for their trustworthy use on safety-critical applications under dynamic environments where the test distribution may be different from the calibration data for training, authors propose the online confomal prediction method using an online density estimation technique.\n\nSpecifically, they assume that the test data distribution continuously changes over time, not in a too extreme manner (Assumption 1).\n\nBesides, for the definition of hallucination of the generated answer on a given prompt, following Mohri et al. (2024), authors define the term as whether a filtered answer contain any hallucinated facts.\n\nIn terms of the technical contribution when compared to existing works on online conformal prediction under the distribution shift scenario, authors assume more challenging problem set-ups.\n\n(1) Specifically, the authors assume continuous distribution shift scenario on the test data distribution, unlike a single distribution shift scenario (Tibshirani et al., 2019).\n(2) Furthermore, they assume the correctness labels on the test data (i.e., whether a hallucinated sub-claim is contained is the filtered sub-claims) remain unrevealed even after the online evaluation.\n\nThis is a more challenging scenario compared to existing online conformal prediction methods (Gibbs and Candes, 2021; Gibbs and Candes, 2024; Areces et al., 2025) in which correct labels are revealed, enabling them to be used for training in subsequent time steps. \n\nIn addition to the theoretical guarantee on the upper bound of the gap between the target coverage level and the average hallucination rate, empirical results on existing and newly proposed benchmarks show that CoFact controls the hallucination rate to the desired degree.", "questions": "[Question 1] Shouldn't the \"T\" in Eq. (14) be substituted to \"t\"? Additionally, the term \\theta_t^\\ast is not formally defined.\n\n[Question 2] How the Eq. (8) would look like if you consider a general setting, where a batch of samples may be provided for each time step? If you can propose one, does it also enjoy the theoretical guarantee that Eq. (8) has? Please refer to [Weakness 1] in Weaknesses Section for detail.\n\n[Question 3] Could you compare CoFact with existing online conformal prediction methods under the scenario where existing methods assume? Please refer to [Weakness 2] in Weaknesses Section for detail.\n\n[Queston 4]  The followings are typos to be addressed.\n\n(Line 35) Despite of => Despite\n\n(Line 77) CoFact bypass => CoFact bypasses\n\n(Line 106-107) The following expression seems awkward to me: \"... that transforms the outputs of a black-box predictor into prediction sets...\"\n\n(Line 115-118) \\alpha => \\beta or \\beta => \\alpha\n\n(Line 314-315) \\leq => \\geq\n\n(Line 269, Line 328) The same divergence function \\psi is defined differently.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- It is a well-written paper which is easy-to-follow.\n\n- They tackle the online conformal prediction problem for the theoretical control of hallucination of LLMs for its trustworthy use in safety-critical applications, which is one of primary interests from LLM users these days. \nSpecifically, they propose the method which is valid under the more challenging set-up compared to existing works, which most resembles the dynamic real world problems.", "weaknesses": "[Weakness 1] Generalizability to online batch setting: I have understood that you are assuming a problem set-up, where single sample is provided for each time step for simplicity (Line 185-186). Then, how the Eq. (8) would look like if you consider a general setting, where a batch of samples may be provided for each time step? If not, the proposed threshold estimator may not be used in an online batch learning set-up. \n\n[Weakness 2] While existing online conformal prediction methods require additional information in terms of training which is not possible in the current problem set-up, it would be more informative to provide results of these methods equipped with the necessary information for training, since baselines in the Experiment section all assume i.i.d. data generating process under the batch learning set-up. As an concrete example, you may consider a problem set-up with (1) a single distribution shift scenario as Tibshirani et al. (2019) and (2) an accessibility to the ground truth labels in an online manner. While existing methods are expected to show more superior performance comapred to CoFact since they utilize additional information in terms of online threshold selection, I think it would be much more informative than just comparing with baselines assuming i.i.d. data generating process under the batch learning set-up.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TKUpIgmn2r", "reviewer": "Reviewer_H5QG", "summary": "This paper addresses the challenge of maintaining the factuality of large language model (LLM) outputs in real-world scenarios where the distribution of user prompts evolves. The authors correctly point out that existing conformal prediction (CP) methods for providing factuality guarantees fall short in this setting. This is because they depend on the exchangeability assumption, which holds when the calibration and test data come from the same distribution. However, this assumption no longer holds when the prompt distribution changes.\n\nTo address this limitation, the paper proposes CoFact. This is a new conformal prediction framework designed to maintain factual reliability under distribution shift. The key idea behind CoFact is to employ online density ratio estimation to adaptively reweight the static calibration data. This way, it aligns with the evolving test distribution at each time step. This enables the computation of an adaptive conformal threshold that effectively filters out hallucinated claims. Moreover, the system no longer relies on the exchangeability assumption or the unrealistic requirement of having ground-truth labels for test instances.", "questions": "The CoFact methodology relies on an online ensemble framework where each expert is updated using an Online Newton Step (ONS). I wonder if this method is computationally more expensive. If it is more costly, this is a potential limitation for real-world, low-latency deployment.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "This paper addresses the problem of LLM factuality guarantees under the realistic condition of distribution shift. Moreover, the experimental validation is comprehensive. A key strength is the inclusion of the new challenging real-world benchmark (WildChat+). The results show that CoFact outperforms baselines and achieves its stated goal.", "weaknesses": "The CoFact methodology relies on an online ensemble framework where each expert is updated using an Online Newton Step (ONS). I wonder if this method is computationally more expensive. If it is more costly, this is a potential limitation for real-world, low-latency deployment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l6FYAUikQU", "reviewer": "Reviewer_C6P4", "summary": "The paper introduces CoFact, a novel conformal prediction framework designed to ensure factuality guarantees for large language models (LLMs) under dynamic, real-world distribution shifts. Specifically, CoFact incorporates online density ratio estimation (DRE), enabling adaptive reweighting of calibration data to align with the changing test distribution. CoFact is validated through a theoretical analysis and empirical experiments.", "questions": "Including case studies and error analysis can significantly enhance the clarity and impact of the paper. \n\n1. Case studies can illustrate how the proposed method performs under distribution shifts, offering concrete examples of its effectiveness. \n\n2. Meanwhile, an in-depth error analysis can help readers understand why and how CoFact outperforms previous approaches, shedding light on its strengths and limitations in handling challenging real-world scenarios.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The paper addresses a critical limitation of existing conformal prediction methods by introducing online density ratio estimation, which allows for adaptation to dynamic and non-stationary distributions.\n\n2. The authors provide a solid mathematical foundation for CoFact by establishing a theoretical upper bound on the hallucination rate under shifting distributions.\n\n3. A new dataset, WildChat+, is proposed to evaluate the approach, featuring real-world user-generated prompts that effectively capture distribution shifts.\n\n4. CoFact is rigorously evaluated across multiple experimental settings, demonstrating its robustness and effectiveness.", "weaknesses": "1. The writing in the paper is difficult to follow, as many key terms are introduced without proper explanation. For instance, the concept of calibration is presented without clarifying its meaning or role within the framework, making it harder for readers to grasp its significance.\n\n2. The framework heavily depends on accurate online density ratio estimation, which can be computationally intensive and challenging to implement efficiently, particularly for high-dimensional or complex data distributions.\n\n3. While the inclusion of WildChat+ is a notable contribution, the paper could benefit from exploring additional domain-specific real-world applications, such as legal, financial, or healthcare contexts, to further demonstrate its practical impact in high-stake tasks.\n\n4. CoFact's emphasis on factuality overlooks other crucial response qualities, such as informativeness. For example, the framework might encourage the model to produce overly short or simplistic responses that sacrifice depth or detail in order to meet factuality requirements. This potential trade-off warrants further investigation.\n\n5. The paper lacks an extensive discussion on distribution shifts, such as distinguishing between in-distribution and out-of-distribution user prompts. Providing examples or a detailed analysis of these distinctions would help clarify how the framework handles varying prompt distributions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)