ConfHit: Conformal Generative Design with Oracle-Free Guarantees
Authors: Siddhartha Laghuvarapu
Ying Jin
Jimeng Sun
Venue: ICLR 2026 Poster (2026)
Keywords: conformal prediction
generative modeling
risk control
molecule generation
applications to drug discovery
Primary area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
TL;DR: A framework for providing conformal guarentees on generative models, without requiring any oracle access during calibration. Experiments are demonstrated on molecule generation such as structure based drug discovery.

Abstract: The success of deep generative models in scientific discovery requires not only the ability to generate novel candidates but also reliable guarantees that these candidates indeed satisfy desired properties.  Recent conformal-prediction methods offer a path to such guarantees, but its application to generative modeling in drug discovery is limited by budget constraints, lack of oracle access, and distribution shift. To this end, we introduce ConfHit, a distribution-free framework that provides validity guarantees under these conditions. ConfHit formalizes two central questions: (i) Certification: whether a generated batch can be guaranteed to contain at least one hit with a user-specified confidence level, and (ii) Design: whether the generation can be refined to a compact set without weakening this guarantee. ConfHit leverages weighted exchangeability between historical and generated samples to eliminate the need for an experimental oracle, constructs multiple-sample density-ratio weighted conformal p-value to quantify statistical confidence in hits, and proposes a nested testing procedure to certify and refine candidate sets of multiple generated samples while maintaining statistical guarantees. Across representative generative molecule design tasks and a broad range of methods, ConfHit consistently delivers valid coverage guarantees at multiple confidence levels while maintaining compact certified sets, establishing a principled and reliable framework for generative modeling.

Published as a conference paper at ICLR 2026
CONFHIT: CONFORMAL GENERATIVE DESIGN WITH ORACLE FREE GUARANTEES
Siddhartha Laghuvarapu's Ying Jin Jimeng Sun
1 Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, II. USA 2 Department of Statistics and Data Science, Wharton School, University of Pennysylvania, PA, USA (s1160, ,0, j immen) ) @1 iinois.................m... linois.edu,, yjinstat@wharton upenn. edu
ABSTRACT
The success of deep generative models in scientific discovery requires not only the ability to generate novel candidates but also reliable guarantees that these candi- dates indeed satisfy desired properties. Recent conformal conformal-prediction methods offer a path to such guarantees, but its application to generative modeling in drug discov- ery is limited by budget constraints, lack of oracle access, and distribution shift. To address these challenges, we introduce CONFHIT, a model-agnostic framework that provides validity guarantees under these conditions. CONFHIT formalizes two central questions: (i) Certification: whether generated batch can be guaranteed to contain at least one hit with a user-specified confidence level, and (ii) Design: be refined to compact set without weakening this
whether the generation guarantee. CONFHIT leverages weighted exchangeability between historical and generated samples to eliminate the need for an experimental oracle, constructs multiple-sampa density-ratio weighted conformal p- p-value to quantify statistical confidence in hits, and proposes a nested testing procedure to certify and refine candidate sets of multiple generated samples while maintaining statistical guaran- tees. Across representative generative molecule design tasks and broad range of methods, CONFHIT consistently delivers valid coverage guarantees at multiple confidence levels while maintaining compact certified sets, thereby establishing a principled and reliable framework for generative modeling.
INTRODUCTION
Deep generative modeling has demonstrated remarkable ability to explore high-dimensional spaces, driving advances in various applications like text generation (Radford et al., 2018), image synthe- sis (Ho et al., 2022), protein engineering (Madani et al., 2020), and molecular discovery (Gómez- Bombarelli et al., 2018). In critical domains such as drug discovery, however, the success requires more than generation: While powerful generative models have been developed to accelerate early- stage discovery (Madani al., 2021; Hoogeboom et al., 2022; Yim et al., 2023), their practical utility depends on whether the generated candidates indeed satisfy key biochemical properties. Since these properties can only be verified through costly wet-lab or in-vivo experiments, it crucial to assess and guarantee, in advance, the viability of the generated samples (hits), leading to a central question:
Given a generative model, how to construct batches of generated samples that can, with high statistical confidence, be guaranteed to contain at least one valid hit?
Conformal prediction provides a monel-agnostic framework for establishing statistical guarantees for black-box prediction models (Vovk et al., 2005), and is recently extended to calibrate any generative model to produce sets of generated samples that contain at least one high-quality instance with high probability (Quach et al., 2023; Ulmer et al., 2024; Shahrokhi et al., 2025). While such guarantees can be profoundly useful, direct application of the existing methods in resource-restricee like drug discovery is limited by several challenges. (i) Certification: With limited generation budget and no assumptions on the generative model, it is not always feasible to produce a valid hit - therefore be provided and when it cannot. On the technical side, (ii) Lack of oracle access: Existing methods rely on an oracle that evaluates newly generated samples for existing inputs (such as by comparing to a gold-standard output). In drug discovery, this means one needs to synthesize and experimentally validate the generated samples, which is infeasible in resource- -imited settings alike (Kladny al., 2024). (iii) Distribution shift: The generated candidates may follow a distribution different from the calibration data, violating the exchangeability assumption.
problems
important to state clearly when a guarantee
'These authors jointly supervised this work.

---

Published as a conference paper at ICLR 2026
Test input
X
Geneatee ougguts
Colbration
Generative
Confomity score
Density estimator
Joint waighted conformalp-aee p-volees
Input
Sempled outgutt
2
2
Cur iitiiiation::
P no in C)(K)en)) S
Sa?
$
Contormal testing
Design:
P( no hit (xs))) S
Generative
Shortlist
Figure 1: (a) Prohlem setup: given input, certify and generate set of candidates that contains at least one "hit" (green) with probability at least - a. (b) CONFHIT workflow. Given nested sequence candidate batches, we estimate the density ratio between labeled data and generated samples, compute a conformal p-value for each batch to quantify the confidence in it containing hit, and return the smallest batch whose p-value falls below a.
We introduce CONFHIT, model-agnostic framework that provides reliability guarantees for gen- erative modeling in rasource-constraine settings. To tackle the budget limitation, we expand the generation problem into two connected fundamental questions:
1. Certificattin.. Given an input Xiest a batch Co (ees) of generated samples and an unknown property indicator A(-) E {0. 1}. can we guarentee, at a given confidence level 1-a E (0, 1), that the batch C( X (ss) contains at least hit?
2. Design. cases with strong confidence, can we design a compact candidate set C(Xieu) while preserving the guarantee that it contains a valid hit with probability at least 1 a??
Here A(-) {0, 1} is oracle that returns 1 only when the sample satisfies desired property. In molecule design, it may indicate whether a molecule x improves upon a seed molecule Xiest in terms of activity, and these questions ensure the experimental validation on C( ((xies) is unlikely to be wasted effort (Figure la). We remark that, in resource-abundan settings, certification and refinement can be combined: the generation budget may be enlarged until certification is achieved, after which the set pruned to a compact certified subset, leading to the same validity guarantees by existing methods.
To eliminate oracle access and handle the distribution shift, CONFHIT leverages the exchangeability structure between historical (labeled) and generated samples, rather than among generated samples for both existing and new model inputs as in earlier methods, and estimates their density ratio, thereby inducing weighted exchangeability (Tibshirani et al., 2019; Jin & Candès, 2023a). For the certification problem, given any viability-scoring function, we construct a weighted permutation p-value that quantifies the evidence against the global null that none of the samples in the generated batch are viable. For the design problem, CONFHIT then examines the certification confidence of a nested sequence of candidate batches, and returns the smallest batch that can be certified at given confidence level a E (0, 1). We show that this procedure bounds the probability of returning a batch with no viable sample below E (0, 1), regardless of the scoring function and the generative model.
We demonstrate the robust guarantees of CONFHIT on two representative generative design tasks: (i) Constrained molecule optimisation, which seeks new molecule that satisfies target property while remaining similar to a given scaffold; (ii) Structure-based drug discovery, which aims to generate molecules that active against given protein. We summarize our contributions below:
We formalise the task of generative modeling in resource-constralle validity guarantee: given an input context (such as a lead molecule or protein pocket), certify and produce a set of candidates that contains at least one hit at a pre-specified confidence level a.
settings with a conformal
We introduce class of density-ratio-weiggee certification problem, and show their validity for certifying the existence of at least one hit in given batch of generated samples under distribution shift.
multiple-test-samph) conformal p-values for the
We propose general nested testing framework that achieves the validity guarantees for the design problem. In specific, we show that given a sequence of our p-values, stopping as soon as the p-value drops below a achieves finite-sample error control.
We develop practical strategies for two key elements in the method: score modelling and density ratio estimation, and demonstrate the robust performance on two standard molecule design tasks: constrained molecule optimisation and structure-based based drug discovery.
2

---

Published as a conference paper at ICLR 2026
Related Work. Conformal prediction (CP) (Vovk et al., 2005) offers distribution-free free uncertainty quantification for any black-box model by constructing prediction sets from exchangeable calibration data (Papadopoulos al., 2002). Recent advances extend CP to generative tasks (Angclopoulos et al., 2021; Quach et al., 2023; Shahrokhi et al., 2025; Kladny et al., 2024). However, these methods require oracle access, which is impractical for resource-constrained scenarios like drug discovery.
In scientific/drug discovery tasks, conformal prediction has so far primarily been used in predictive inference for regression and classification problems in property prediction (Sun et al., 2017: Svensson et al., 20;8; Cortés-Ciriano & Bender, 2018; 2019; Zhang et al., 2021) and their extension to covariate-shift settings via importance-weight estimation (Fannjiang al., 2022; Laghuvarapu et al., 2023; Prinster et al.. 2023; Fannjiang & Park, 2025). Beyond prediction-set construction, conformal selection employs p-values to prioritize compounds with selective error control (Bai et al., 2024; Jin & Candès, 2023b; Bai & Jin, 2024). As we address a distinct generative design problem, our formulation, inference scheme, and methodology differ sharply from prior work.
Meanwhile, there is a growing literature on generative modelling in scientific design tasks, including goal-directed small-molecule optimization based ligand design (Corso et al., 2022; Guan et al., 2023), large-scale protein generators (Madani et al., 2020; Ingraham et al., 2019; Strokach & Kim, 2022), and materials discovery (Xie et al., 2022; Zeni et al., 2023). Our contributions are orthogonal to this literature as CONFHIT is model agnostic and offers guarantees for building property-satisfyiin samples from these generative models.
et al., 2018; Jin et al., 2020), structure-
To summarize, compared with existing works, CONFHIT is the first framework that (i) eliminates the need for oracle access, (ii) corrects for covariate shift between historical and generated samples, and (iii) provides finite-sample guarantees for both certification and compact design in generative design.
2 PROBLEM FORMULATION
For expositional convenience, we slightly override the notations in Section 1.
Data and distributions. We assume access to of i.i.d. labeled (calibration) samples Dealib ((X,, Y,)D from an unknown distribution P, e.g., molecules from past campaigns with known properties. Here X; X is the feature, e.g., the chemical structure of a molecule, and Y; E {0, 13 is binary label, e.g., oracle-judged Y; = A(X.). A generative model M produces i.i.d. sam- ples (Xn+)122 from another distribution Q with unknown labels {Yn+51121 (the properties Yn+j = A(xn+)) should they be judged by the oracle). Although Q's density is, in principle, computable, doing so is often costly. Instead, we assume a covariate shift between the two distri- butions: dQ/ dd((r,)) = w(x) for some function w: X' - RR. It posits that X captures all the information for predicting Y. which is especially sensible and standard in drug discovery contexts where the biological property is determined by the structure (though in a highly complex fashion).
Guarantees. We now formalize our guarantees for (i) certifying the presence of hit in set of generated candidates, and (ii) constructing a set of samples with at least a hit. For goal (i), given a pre-specified confidence level a (0,1) 1) and generated set Crow == {Xn+s) LV of budget N N aim to propose test (b CBow) E (0, 1) such that
(Certification) that is, the probability of falsely certifying low-quality candidate set is upper bounded by a. For goal (ii), we would like to find a (random) stopping point Ñ s N such that
IP (E (D calib: CDew) 1 ((X+)))= = 0, AI S < N) < a
(1)
(Design)
IP (PA(Xn+))) 0, v1 S j S Ñ) < a,
outputing the list of samples Enew = (X X_ ) which contains a hit while controlling the error of vacuous declaration. We define ñ 0 to accommodate cases where we are not confident enough to declare any positive sample within the generation budget (failure of certification).
n
METHOD
We introduce our conformal nested testing framework to address the certification problem (Section 3.1) and the design problem (Section 3.2), and prove their model-agnostic, finite-sample guarantees.
"Tii is consistent with existing works on uncertainty quantification for generative models (Quach et al., 2023), where "null" option (i.e., the method fails to confidently generate good samples) required.
3

---

Published as a conference paper at ICLR 2026
3.1 CERTIFICATION: JOINT WEIGHTED CONFORMAL P-VALUE
We begin by addressing the certification problem: Given a of generated samples, how to quantify the confidence in it containing a hit? Our key strategy is to construct a p p-value based on conformal inference ideas that test the presence of any hit in (X++j 1 j=1- Recall that there is covariate shift w(x) dQ/ Pa: y) for the labeled calibration data and the new test samples. To fix ideas, assume the density ratio w-)) is known for now, and discuss estimated density ratio in Remark 3.3.
Our p-values leverage the inactive calibration data {X;: i Io} where Io == {i E [n] Y; = 0}. We let X = (Xi,..., X Xa+1...., X denote the vector of (inactive) calibration and test covariates. Following split conformal prediction (Vovk et al., 2005), we let V x - IR be any function whose training process is independent of {[X,, 12) which is then viewed as fixed. We call V the conformity score function, whose choice discussed near the end of this subsection. Without loss of generality, we assume larger value of the conformity score indicates stronger confidence in the presence of positive value in [Yn+)]}
To address multiple test points without introducing too much computational overhead, we leverage randomization (see Appendix A.1 for the non-randomized version). Let N be the collection of all permutations of {L..... Ro., n + l..... n2 + NN), and denote the identity mapping as T(0,, i.c. T (i) i for any i. Formally, fixing any B E N+ we draw x x(1), ..... ((B) Ltd. Unif(Ily), the uniform distribution over the permutation space. Define
Ebzo ((x((); X) [<V X) S ((T(0) x))
PN
EEE (() X)
(3)
Here, for any permutation 1I,, we define the conformity score from the permuted data (ix; x) = V(Xx(I)..... X((na); xx(++))--- xx())))))))))(x))) and w(;; X) II;= xx(xx(++)); is the joint likelihood ratio after permutation 9, where w(x) = dQ/ dP(x, y) is the density ratio.
The construction of our p-vuuue relies inactive calibration data [X,: i E ] and the test samples {Xm++s) \N under distribution shift, detailed in Appendix A.2 due to limited space. It extends the conformal p-values in the literature via an extended weighted exchangeability structure (Tibshirani et al., 2019) for multiple test samples (Hu & Lei, 2024; Jin & Candès, 2023a); these connections are discussed in Appendix A.3.
analysis of the exchangeability structure between the
The following theorem establishes the finite-sample validity of the randomization p-value (3). Notably, this holds regardless of the number of sampled permutations. Its proof is in Appendix B.4.
Theorem 3.1. Under the covariate shift assumption, it holds for any fixed t E [0, 1] that
P( PN rand < maxi = 0) S t.
Therefore, the certification test function (Dealit. [X = 1 (ppndd S a achieves (1).
Inheriting the model-free nature of conformal inference, the validity of pk and yf holds regardless of the score function V. as long as its training process is independent of the calibration and test data. Remark 3.2 (Outlier detection). Our p-values can also be viewed as testing for the presence of at least one outlier in [Xn+j when (Xi)iEz, are viewed as calibration inliers. We refer the readers to discussion on the connection to outlier detection under covariate shifts (Bates et al., 2023; Jin & Candès, 2023a) in Appendix A.4.
Remark 3.3 (Fstimated density ratio). In many practical scenarios, the density ratio w(() is unknown or difficult to evaluate. In such cases, we estimate the density ratio by wd()) and use w(() instead of w(-) in the construction of our p-values {pk}. For instance, we can train the density ratio over a random subset of all the labeled data and independently generated test samples. We do not write the exact formulas for such plug-in in p-values for brevity. Recognizing that the success of CONFHIT hinges on accurate density ratio estimation, we provide a thorough discussion on the robusiness of CONTHIT with estimated weights and several practical diagostics in Section 3.3.
Conformity score function. Because we certify the candidate set when py is small, V must grow when Xn+In+N contains positive to indicate strong confidence. To this end, we assume access to a pre-trained model fe: X + RR that predicts the unknown property Ya+; based on Xn-j. which will be used in the score V. We suggest several natural choices: (i) Max-pooling: V (x Int lo+
4

---

Published as a conference paper at ICLR 2026
maxig/sN ((xn+z), (ii) Sum-of Sum-offpredde prediction: V( Bn-im+NN) Σ}= (zn (iii) Rank- N
1= Rm where Ra+j is the rank among all scores, (iv) Likelihood 21=1 j==1 log( T- We explain these scores in Appendix A.5. We note that CONFHIT is model-agnostic: practitioners can choose any suitable model to build the score V. The quality of the score/model affects the power, but not the error control of CONFHIT, Finally, practitioners may want to avoid sample splitting when labeled data is limited; we provide such variant in Appendix A.6 where the labeled data are used for both training f and p-value construction.
sum: V (Z 4o+
Im+ +onnnn+++nn 2
ratio: (Z Un++in+
A(x++s))
3.2 DESIGN: CONFORMAL NESTED TESTING
Upon the valid p-value and certification test for a given candidate set, we proceed address the design problem: How to propose a compact set of generated samples that contains at least one hir with high probability? To this end, we connect (2) the problem of constructing smaller subset of candidates that can be certified by our procedure, and establish its statistical guarantee.
To be specific, for every 1 S k S N, we consider the null hypothesis
Hy: Yn+j 0, A 1 S k.
(4)
That is, H& posits that none of the first generated instances obeys the desired property. From a hypothesis testing perspective, rejecting H₃ thus suggests sufficient confidence in declaring a hit in
(Xn+s) The certification strategy in Section 3. is readily applicable to obtain a p-value P& for certifying each subset
[x++ indexed by k that obeys P(px S t H2 is true) < t for 1 E [0, 1].
Our solution to the design problem appears simple: determine an index Ñ such that Hş can be rejected (i.e., {X,+j)Ø= can be confidently declared as containing a hit). Specifically, suppose we can construct a decreasing sequence of p p-values E [0, 1] for each fixed HA. Then, we set
Ñ == inf {k: pk S a}.
(5)
That is, we take the first p-vuuue that passes the significance level a E (0, 1). Theorem 3.4 confirms the validity of this nested testing strategy, whose proof in Appendix B.1.
Theorem 3.4. Suppose the p-values pk [0, 1] obey: (i) Monotonicity: P1 2 P2 2 .Pp (ii)
Validity: For any fixed & 2 1 and E [0. 11, it holds that P(px I <t| Hg is true) S t. Then, we have
P(max P(maxisish Yn+j 0) S
for the generation threshold Ñ computed as in (5). Here H₃ is defined in (4).
We note that any p-values } obeying condition (ii) in Theorem 3.4, such as those constructed in the certification problem, can be turned to p = maxxsjsN Pj which satisfy both conditions in Theorem 3.4. Perhaps surprisingly, even though we are simultaneously examining multiple batches of candidates, it turns out that using a monotone sequence of individually valid p-values suffices to achieve our goal, and no adjustment for multiplicity is needed. We remark that the key to our theoretical guarantee the nested nature the hypotheses: if H& is true, then "earlier" hypotheses (He)esk must also be true. We thus call our method "conformal nested testing".
Putting everything together. So far, we have completed all the elements of CONFHIT. We summarize the entire procedure in Algorithm 1, including both certification (for every nested subset) and design. Note that the input w(-) denotes estimated density ratio function, which coincides with w(-) when the density ratio is known. When it fails to certify even the largest batch, CONFHIT flags "not confident enough" to clearly communicate the difficulty of the generation task.
CONFHIT:
3.3 ROBUSTNESS AND DIAGNOSTICS FOR DENSITY RATIO ESTIMATION
The density ratio often needs to be estimated in practice and its quality naturally affects the perfor- mance of CIT The difference between historical assay data and newly generated molecules is persistent issue in drug discovery and needs to be addressed in nearly all conformal prediction methods this domain (Krstajic, 2021; Laghuvarapu et al., 2023; Jin & Candès, 2023a; Fannjiang & Park, 2025). Estimating the density ratio is a statistically standard problem used in many fields including machine learning, statistics, and causal inference with rich existing toolboxes such as kernel or dassificationnbbsel methods beyond the kernel density estimation in our experiments (Horvitz. &
5

---

Published as a conference paper at ICLR 2026
Algorithm 1 CONFHIT
Input: Calibration data {X+)= for which Y; O, test data {Xn+sly= 1y=1, conformity score V: x R, (estimated) weight function w: X' + R confidence level E (0, 1), budget N E N+. Monte Carlo size B N+
1: for li 1,...,NO do
2: Set IT = { all permutations of (1, [1,.... no, n +1,++1..... 1,..., ++}
I/ Sequential conformal p-values
3: Draw (0() de UI) for b = I,.... ...,. B.
4: Compute p-value as in (3) with w(x) == 10(2).
5: end for
6: Set pt: max Pe iteratively for k = N 1,....1
I/ Monotonize p-values
7: Compute Ñ a E [N]] Ph I a), with Ñ if a
Output: Confident shortlist C = {X++i)}= or "not confident enough" when N 0.
Thompson, 1952; Shimodaira, 2000; Sugiyama et al., 2007; Nguyen et al., 2010; Sugiyama et al., 2012; Shapiro, 2003). Below, we elaborate on (i) a theoretical analysis of CONFHIT's 's robustness to the estimation error, followed by (ii) three diagnostic approaches for the estimation quality.
Theorem 3.5 shows that with estimated density ratio, the error inflation depends on how aggressive (resp. conservative) the weights are for small (resp. large) p-values; its proof is in Appendix B.5.
Theorem 3.5 (Robustness to estimation enror). Let po be the p-value (3) using any estimated weights v.()). Define the positive/negative parts of the estimation error 8* (T;; X)= [ύ{ X) w(TT; X) +.
with (T:; X)
1 á (((+)))). Then, for any fixed E (0, 1), we have
N
PP prand
max Yn+s = =0) + I9SNN
maxISIEN Ynts =
In (((0)
where, setting (N( = Y ((x () we define = argmax ( (V X): E
Enzo ((T()))))
11, PN (V X)) S tt as the score cutoff. and the corresponding p-value as é == ( and
Å = E2 3+ (1; X) ((V(( X) 6}, Á = - (T((1; X) ((V((()))))))))))) X) 2 i}.
While Theorem 3.5 offers insights on how the guarantees degrade with the (unknown) estimation error, it often useful to perform robustness diagnostics in practice. The three approaches below demonstrated in our experiments in "Robustness check" Section 4.5.
(1) Balance check. Following the balancing idea in causal inference (Ben-Michael et al., 2021), reweighting via the correct density ratio will bring the empirical mean of any function of the covariates in the calibration data close to that of the test data (under the null with no hits). A natural diagnostic
is to check the imbalance I. Σe ú (X.))((X.) - - Σソー - [(Xn+z)] for some f X' + IR
(2) Validation shift. A second approach is to examine the reliability of estimation via synthetic shift. One may create artificial distribution shifts in the labeled data, such as by scaffold splitting: with one fold as the calibration data and the other the test data,
use the known labels to examine the uniformity of p-values and the quality of the density ratio estimation algorithm.
(3) Sensitivity analysis. Finally, one may conduct sensitivity analysis to probe the robustness of CONFHIT to perturbations of estimated weights. Similar to causal inference (Rosenbaum, 2005), we may assume the correct weights are within a distance from the estimated weights, and examine the worst-case p-values and testing results while varying the weights within the assumed range.
While generative models do not naturally provide a built-in notion of reliability, CONFHIT augments them with new capabilities for practical deployment; it is model-agnostie and only relies scalar quantity, the density ratio, that can be validated and stress-tested thereby making the uncertainty measurable and more robust than trusting the raw model alone (Section 4.5, "Additional baselines").
4 EXPERIMENTS
4.1 TASKS, MODELS, AND DATASETS
We evaluate our framework on two representative conditional poolecule-generatiie tasks and highlight the model-agnostic nature of our approach by employing five generative models spanning VAE,

---

Published as a conference paper at ICLR 2026
autoregressive transformers, diffusion models, and Bayesian Flow Networks. For evaluation purposes, we use computational oracles to judge the property of generated samples since wet-lab evaluation is infeasible (Jin et al., 2020); the oracle is not used in running CONFHIT.
App. 1: Constrained Molecule Optimisation (CMO). Given seed molecule X that fails target property, the goal is to generate a molecule X that is close in Tanimoto distance to X and satisfies the property. This task reflects lead-optimisation workflows where experimental assays provide ground-truth. Following the standard practice of using a computational oracle, in our experiments, we use the property DRD2 receptor binding (DRD2), with success (Y 1) if DRD2(z) 0.5. Additional comphrehensive results on Quantitative Estimate of Drug-likeness improvement (QED) reported in the Appendix E.2.
To demonstrate the compatibility of CONFHIT with general generative models, we employ two state-of-the-a CMO models: HGRAPHHGRAPHH a hierarchical VAE editing scaffold-level motifs while preserving validity (Jin et al., 2020); 2. SELF-EDIT, autoregressive transformer generating SELFIES strings to optimise the property while maintaining similarity to the seed (Jiao et al., 2023).
We take subset of the training split in Chemprop (Yang et al., 2019) of ChEMBL data (Gaulton et al., 2012) as the labeled calibration data (X,, } (seed input, sample from past campaigns, and oracle property), following Jin et al. (2020). Inactive molecules in the test split serve test inputs (X(0) | For each ((), the model generates (x(4) 1m 1=1
App. 2: Structure- Based Drug Discovery (SBDD). Here the input X is 3D protein binding pocket, and the task is to generate ligands X that bind to it, so the label Y depends on (X, Following standard practice, for evaluation purposes only, we use AutoDock Vina (Trott & Olson, 2010) as computation oracle and label a ligand as a hit if its score is below -7.5 kcal mol -! such a threshold captures about 75% of known active samples in CrossDock (Francoeur al., 2020a).
In this task. we apply CONFHIT with three state-of-the thtt--r SBDD models: Targe:Diff (Guan et al., 2023), an SE())-cquivarian diffusion model conditioned on pocket meshes; DecompDifff (Guan et al., 2024), using decomposed priors to separate structural components; MolCRAFT (Qu et al., 2024), Bayesian Field Network operating fully continuous parameter space.
The calibration data are pootein-ligand pairs {(X;, (x)) from CrossDocked (Francoeur et al., 2020b). Following the same split in Guan et al. (2023), we use the proteins in the test split as inputs (X(0) I= and the model generates candidate ligands (Xn++Jf= given each input (().
4.2 IMPLEMENTATION DETAILS
Conformity Score Function. We define V using property prediction model pa-)):
CMO. For both properties (QED and DRD2) we train a binary classifier (a()) with molecular graph inputs using the Chemprop library (Yang et al., 2019) and ChEMBL data splits in Jin et al. (2020).
SBDD, We train an EGNN (a-)) (Satorras et al., 2021) on PDBBind crystal structure data (Wang et al., 2005) to predict binding affinity, using TARGETDIFF (Guan al., 2023) codebase.
In both tasks, î is trained on data independent of calibration and test sets. Training details, hyperpa- rameters, and compute for K and the generative models are given in Appendix C. For conciseness, we present most results with the Max-pooling score V V( In+1s+ Comparisons other score functions appear in Figure
maxisisk ((x++)).
Density Ratio Estimation. We leverage kernel density estimation (KDE) for constructing the weights in our p-values. Following the discussion in Lemma A.2 in Appendix A.2, the desired weight w(-) equals the marginal density ratio of generated samples, regardless of hit status. Also, plugging the densities for the joint distribution of input/generation pairs, denoted as (( .²) and P(T., æ:), yields the same p-value due to normalization. Thus, we directly estimate p(-) and 7(-) to ensure sufficient data for estimation. Following CODRUG (Laghuvarapu et al., 2023), we extract latent features g x) from the penultimate layer of the property model, and fit Gaussian KDEs p(ž. x) and ((g(ū, x)) on calibration and test samples, forming weights w 2) ((0( (((
4.3 RESULTS: CERTIFICATION
In this section, we present results of the certification experiment, where the goal (1) is to certify the presence of hit in a batch fixed size N at level a. Our evaluation focuses on (i) error control. and (ii) power, i.e,, the frequency of successfully certifying hits. Results for both CMO and SBDD tasks
7

---

Published as a conference paper at ICLR 2026
Structure Based Drug Discovery (SBDD)
Error Rate ve Alpha (N=10)
Ernor Rate vs Alpha (All NI
Fraction Certifted (AII NJ
Frection Certiflee (5)
hautke Cantifiol
Eree Rota 050
100cey
102zmy
Alpha tsignificance level)
Alpha Isignificace level)
Alpha Dignsffiance level)
Constrained Molecule Optimization (DRD2)
Rate Alpha (N-7)
Rate Alpha
Fraction Certified N)
Highghh
bror tate ved
Praction Core Pood Pho
hutton Certlied Med
Seledt
BreE mate Pho
Free late
Tnerton Cartiied
103xmy
broe sulo Yee/
hoLLinn De ulo 2
100xer
(dgetfficncc
inignificance
(significace
Figure 2: Certification results. Left: realized error rates at fixed N for different models and error levels in SBDD (upper) and CMO (lower). Middle: average error rates while varying budget N. Right: power, i.e., the fraction of actives certified at various error level a and budget N values. The dashed line denotes the ideal y = x error bound. Our method consistently achieves valid coverage across scenarios. Results are averaged over 5 random runs; error bars and additional results for other values of N are in Appendix F.2
Structure Based Drug Discovery (SBDD)
Error Rate Alpha (N-10)
Moan Set Size Alpha (vs N)
Empty Set Purcentage Alpha (vs N)
Mi contre
Mean Set Sieee
Corfiit 1e5
Corfil te10
Confritte
Erer Ratt
Corfiit N-15
Contiet
Bostomee Nag]
Contratt NH-5
Banferon
sonti nont N-5
Bonterow No15
sonteeoon 10
Sentlenzal N
Alpha Isignifiance levell
Alpha (significance levell
Alpha Calgnfficacce level)
Constrained Molecule Optimization (DRD2)
Rate va Alpha (N-5)
Mean Set Alpha (va
Enoott Sell waaaenatggee (%6
Emppy Set Percentage va Alpha (va N)
Ded7 Contiz
Dri2 Selleck: Corliit
Dastr)
Dvd2 Hgraph corforin
Cost
Corfili N-5
Dedt Sallect Derienoni
Optti) Nw
Corlit fe-
100xey...
Bonferoni M-3
Borieeroni ko
Borfarent R-5
Borterani
Alpha Digniffcance tevel)
Alpha orignificance level)
Alpha csigniffiance level)
Figure 3: Design results. Error rate at fixed N for different methods (left), mean set sizes averaged across methods at different values of N (middle), and empty set percentage at different values of N (right) across target levels The top row shows results for SBDD and the bottom for CMO. Dashed black line in the error plots indicates the ideal y = bound. CONFHIT achieves tight error control while producing substantially smaller sets. Results are averaged over random runs; additional results are provided in the Appendix E.2.
across different generative models are shown in Figure 2. rate, i.c. the frequency of certifying sets that contain no hit. Across all settings, observe tight error control. The middle panel shows the error rates (averaged across methods for visualization) for different budgets N, and observe that the error stays below the target error levels. Finally, the right panel plots the fraction of sets containing an active that are certified as a function of a at different, where CONFHIT is able detect true hits with satisfactory power. As expected, this fraction increases with the significance level and set size N.
The left panel reports the empirical error
4.4 RESULTS: DESIGN
In this section, we present experiments for the design problem in (2), where we prune a generated batch to obtain compact subsets while preserving tight error control. As summarized in Figure 3, our nested procedure refines candidate sets while preserving statistical guarantees.
As we address novel problem under unique constraints, there are no directly comparable baselines (existing methods in conformal generative modeling (Quach et al., 2023; Kladny et al., 2024) are not comparable due to the unavailable oracle). Instead, we consider two baselines to demonstrate the benefits of CONFHIT: (i) Bonferroni correction. Given N test samples, we threshold the mne-test-sample p- vavuu (set each candidate as the test set in (3)) at a/N. It provides a stronger guarantee of no false certification for any test sample, but can be conservative. (ii) Certification-only. The certification procedure from Section 4.3, which serves as baseline without pruning.
8

---

Published as a conference paper at ICLR 2026
Compared with Bonferroni, our approach substantially outperforms in both CMO and SBDD tasks. At stringent levels (a = 0.1). Bonferroni yields empty sets for nearly 100% of SBDD inputs, while CONFHrTleads to empty set frequency low as 16% for N = 15'. Moreover, in the rare cases where Bonferroni is able to make discoveries, it usually exhaust the full budget, whereas CONFHIT consistently prunes 2-5 molecules (30-50% of the full budget). Due to conservativeness, Bonferroni has low or near zero error rates, whereas CONFHIT achieves adaptive and tight coverage.
Compared with certification-only whose results are in Figure the nested procedure successfully prunes promising initial large sets to smaller ones. For example, in SBDD at a = 0.1 with N = 15, the nested procedure reduces the mean set size to about molecules without compromising the error control. Similar improvements hold in CMO, where nested pruning typically halves the certified set size while delivering tighter agreement between realized and nominal error.
In summary, CONFHIT delivers compact certified sets with strict error control. It largely avoids vacuous results and yields smaller, more actionable shortlists with higher coverage for subsequent experimental validation. This makes our framework particularly valuable in scientific discovery workflows, where experimental budgets require both statistical confidence and practical tractability.
4.5 ADDITIONAL INVESTIGATIONS
Choice of test statistic. While we report the results with the Max-pool conformity score, Figure evaluate CONFHIT with five choices of the conformity score (..) min, mean, likelihood-ratio (LR), rank-sum, and max-on the SBDD task with TARGETDIFF at a budget of = 5; results for other tasks and budgets show similar patterns and provided in the Appendix E.2. Across the board, CONFHIT consistently keeps the realised error rate close to the target a, confirming the model-agnostic validity. However, the choice of the score does affect power: max certifies most often (e.g., 73 % of seeds at a= = 0. while rank-sum is the most conservative.
Rejection Rate
Erioor Rate Alpha
relh
41.4%
FSBB
Borlivant
rork s.m
2
FRDC, sure
19.33.%
Appha (Sigrificance LeveD
Figure 4: Comparison of score statistics on TargetDiff (SBDD, N=5). Left: rejection (power) a 0.1,0.3. 0.3. Right: error vs. a. All remain valid; the max statistic shows the highest power.
Choice of property prediction model. As we discussed in the end of Section 3.1, the choice of the property prediction model only affects the power. We performed an ablation study with weak property prediction models by adding noninformative perturbations to the predictions. CONFHIT maintains robust error rate control despite that the weaker predictor decreases the power; see Appendix F.
Distribution shift adjustment. The guarantees of CONFHIT in both certification and design rely on valid p-values, which further hinges on distribution shift adjustment. As an ablation study, we compare CONFHIT with the version without distribution shift adjustment, i.e., taking w(.) = 1. We observe that running CONFHIT without distribution shift adjustment can violate the coverage guarantee especially for stringent target error rates, for example, using the TARGETDIFF model in the SBDD task, and the SELFEDIT model in the CMO task. The differences are highlighted in Figure 5a. In addition, we plot the distribution of p-values computed with negative samples across different datasets and observe that they are approximately uniform; see Appendix E.3.
Robustness check. We exemplify the robustness diagnostics in Section 3.3, with a sensitivity in Appendix H. 1, synthetic validation with a realistic scaffold split in Appendix H.2 and a balance check based on key features in Appendix H.3. First, by perturbing the weights via power law, we showcase how to examine the robustness of CONFHIT's output, with additional evaluation to show the mild degradation of error control. Meanwhile, CONFHIT achieves tight error control under synthetic scaffold splits, and the estimated density ratio improves the balance in key features.
Returning empty sets means enough confidence under the generation hudget N, which is inevitable without strong assumptions the generative power.

---

Published as a conference paper at ICLR 2026
Comparisen,
07
CMO(DRD2) Budget Analysis
1.
SADO Budget Analysis
bink follioa cendlly Taryslin sal0 bot
Us-
Ackal ansiive
Obarree
CP9
lrqedenn mncoss pcrili
Drcanpart Fooktted pindice
1.2% 1.4c
3.26 3.5
2.5
Alyha Jogrsficasee lesel)
Budget
Budget
Figure 5: (a) Distribution shift adjustment (left). Coverage violations when CONFHIT is run without density correction. (h) Budget analysis (middle and right). Fraction inputs with at least one under increasing generation budget. Solid lines: fraction of actual hits; dashed lines: predicted fraction of hits.
Additional baselines. We additionally evaluate two baselines. (i) A heuristic baseline which uses an estimated probability to calibrate the generation batch size (Appendix G. It violates error control when the estimation quality moderately degrades; in contrast, CONFHIT model-agnostic and maintains validity under the same degradation (Appendix F), showing that principled uncertainty quantification provides robust guarantees. (ii) An oracle baseline which (unrealistically) labels generated samples (Quach et al., 2023) yet without covariate shift adjustment (Appendix G.2). Even with oracle access, ignoring the distribution shift leads to both high errors and high frequency of empty sets (perhaps since it relies on exchangeability between inputs, which reduces the sample size).
4.6 WORKING UNDER AN OVERALL BUDGET
Finally, we demonstrate the utility of CONFHIT for allocating a fixed budget across multiple tasks. In drug discovery campaigns, scientists often work with multiple generation inputs but can only validate a limited number of samples experimentally. In this setting, the confidence in valid hits provided by CONFHIT serves as a practical hcuristic for budget synthesize per task. Given K inputs (tasks) and total budget B, we fix maximum size N for each input; then, CONFHIT suggests the smallest subset which contains a hit for each input at a confidence level a. We then vary the value of a until the budget is exhausted. Finally, we estimate the fraction of tasks whose output set contains a hit via (1 - at) minus the fraction of empty sets (i.e., when CONFHIT fails to certify a hit at level a). The exact procedure is described in Appendix D.
how many candidates to
In Figure 5b, we compare predicted and realized positives under different budget levels (measured as multiples of the number K of inputs). Across both CMO and SBDD, the realized fraction of actives consistently exceeds our estimates, showing the reliability and practical utility of CONFHIT. Complementing the certification and design results, this budge allocation perspective confirms the robust empirical performance of CONFHIT when coupled with more complex, albeit heuristic, applications. Of course, we acknowledge that this is only a heuristic approach without formal theoretical guarantees, and the application of CONFHIT in such tasks warrants careful future study.
CONCLUSION
In this work, we introduced CONFHIT, model-agnostic framework that delivers fiite-ssmple guarantees for conditional generative models. By re-weighting calibration data with density ratios, CONFHIT corrects the distributional shift from historical compounds model-generated candidates, thereby removing the need for an external experimental oracle, an assumption required by all existing conformal-prediction approaches in generative modcling. Building on these adjusted weights, we derive nested conformal p-value that certifies the probability of sampling at least one viable molecule. Across two standard benchmarks for optimization of constrained molecules and structure- based design, and across wide range of methods and budget regimes, CONFHIT consistently provides valid coverage guarantees while maintaining compact certified sets. Our results establish CONFHIT principled and practical framework for both certification and design, enabling reliable generative modeling under stringent resource constraints.
Limitations The coverage guarantee relies on density-ratio estimates, which can be noisy when the calibration set is small or the feature extractor is poorly aligned with the target domain. Our experiments focus on small molecules; extending the approach to proteins or other macromolecules with larger, more structured generative space will require additional work. Validation currently relies on in-silico oracles; without wet-lab confirmation, transferability remains to be seen. Demonstrating robustness across broader chemical and experimental settings is a key direction for future research.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Because the framework is model-agnostic and targets conditional molecule generation, we expect benefit diverse discovery pipelines in chemistry, biology, and materials science. The work builds on long-standing practices in molecular modeling, so we do not anticipate negative societal consequences. However, acknowledge potential dual use, such as designing harmful compounds, and responsible use will require oversight and access controls.
LLM Usage The use of LLMs in this work is limited to polishing the writing and assisting with code-related tasks.
REPRODUCIBILITY STATEMENT
All experimental settings are described in Section 4.1, with extended discussion of implementation details in Appendix C. Information on compute resources and runtime provided in Section C.4. The code, datasets along with the config files used for experiments in this work are included at htttps:/ggithin com/ si adharthal/ Sew.foom---eeeeeiii---eeeeeeee and are further detailed in Appendix C.5.
REFERENCES
Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. Learn then test: Calibrating predictive algorithms achieve risk control. arXiv preprint arXiv:2110.01052, 2021.
Tian Bai and Ying Jin. Optimized conformal selection: Powerful selective inference after conformity score optimization. arXiv preprint arXiv:2411.17983, 2024.
Tianyu Bai, Puchen Tang, Yifan Xu, Vladimir Svetnik, Abbass Khalili, Xusheng Yu, and Andrew Yang. Conformal selection for efficient and accurate compound screening in drug discovery. arXiv preprint arXiv:2402. /2345, 2024.
Stephen Bates, Emmanuel Candès, Lihua Lei, Yaniv Romano, and Matteo Sesia. Testing for outliers with conformal p-values. The Annals of Statistics, 51(1):149-177 2023.
Eli Ben-Michael, Avi Feller, David A Hirshberg, and José Zubizarreta. The balaneing act causal inference. arXiv preprint arXiv:2110. 14.21110...4,,,, 14831, 2021.
Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking. arXiv preprint arXiv:2210.01776, 2022.
Isidro Cortés-Ciriano and Andreas Bender. Deep confidence: A computationally efficient framework for calculating reliable prediction errors for deep neural networks. Journal of Chemical Information and Modeling, 59(3): 269-1281, 2018.
Isidro Cortés-Ciriano and Andreas Bender. Concepts and applications of conformal prediction in computational drug discovery. arXiv preprint (tto:::90.0055533 2019.
Clara Fannjiang and Ji Won Park. Reliable algorithm selection for machine learning-guide design.
arXiv preprint arXiv:2503.20767 2025.
Clara Fannjiang, Stephen Bates, Anastasios N. Angelopoulos, Jennifer Listgarten, and Michael I. Jordan. Conformal prediction under feedback covariate shift for biomolecular design. Proceedings of the National Academy of Sciences, 119(4))= 2022.
Ronald Aylmer Fisher and Ronald A Fisher. The design of experiments. Springer, 1971.
Patrick Francoeur, Ruben J. Gomez-Bombarell Jianfeng Pei, and Rohit Singh.
convolutional neural networks and a crossdocked dataset for structure-bac based drug design. Journal of Chemical Information and Modeling, 4200)):000-4215 2020a. doi: 10.1021/ace 0021/acs.ciim.0000411
11

---

Published as a conference paper at ICLR 2026
Paul G Francoeur, Tomohide Masuda, Jocelyn Sunseri, Andrew Jia, Richard B lovanisci, Ian Snyder, and David R Koes. direessimnni convolutional neural networks and a cross-docked data set for structure-based drug design. Journal of chemical information and modeling, 6009):4200-2215, 2020b.
Anna Gaulton, Louisa J Bellis, Patricia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, Bissan Al-Lazikani, et al. Chembl: a large-scale bioactivity database for drug discovery. Nucleic acids research, (00D): 107, 2012.
Jiaqi Guan, Wesley Wei Qian, Xingang Peng, Yufeng Su, Jian Peng, and Jianzhu Ma. 3d equivariant diffusion for target-aware molecule generation and affinity prediction. International Conference Learning Representations (ICLR), 2023. URL https: petttt:://ppereeeeee /owpnrpeeeeeee.ee//e//e net hwt.://ettoouuut/////e forum?id= SUqXEPXMsEC..
Jiaqi Guan, Xiangxin Zhou, Yuwei Yang, Yu Bao, Jian Peng, Jianzhu Ma, Qiang Liu, Liang Wang. and Quanquan Gu. Decompdiff: diffusion models with decomposed priors for structure- based drug design. arXiv preprint urXiv:2403.07902 2024.
Rafael Gómez-Bombardde Bombarelli, Jennifer N. Wei, David Duvenaud, and et al. Automatic chemical design using a data-driven continuous representation of molecules. ACS Central Science, 4(2):268-277) 2018.
Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1-33, 2022.
Emiel Hoogeboom, Victor Garcia Satorras, and et al. Equivariant diffusion for molecular generation in 3d. In International Conference Machine Learning, 2022.
Daniel G Horvitz and Donovan J Thompson. A generalization of sampling without replacement from finite universe. Journal of the American statistical Association, 47(260):663 685, 1952.
Xiaoyu Hu and Jing Lei. A two-sample conditional distribution test using conformal prediction and 136-1144, 2024.
weighted rank sum. Journal of the American Statistical Association,
John Ingraham, Vikas K. Garg, Regina Barzilay, and Tommi Jaakkola. Generative models for graph-based protein design. In Advances in Neural Information Processing Systems, 2019.
Miao Jiao, Yiqing Wang, Xiufeng Chen, and Tien Huynh. Self-edit: Structure molecular optimisation using selfies editing transformer. Applied Intelligence, 53(18):21455-2147) 2023. doi: (0000077/11449-2------0
Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Hierarchical generation of molecular graphs using structural motifs. In Proceedings of the 37th International Conference on Machine Learn- ing (ICML), pp. 4839-4848, 2020. URL https::////o /ttp:s///rrcceeee ngs m1 wpp.ss/vvvivi jin2 Da. html.
Ying Jin and Emmanuel Candès. Model-free selective inference under covariate shift via weighted conformal p-values. arXiv preprint arXiv:2307.09299, 2023a.
Ying Jin and Emmanuel J Candès. Selection by prediction with conformal p-values. Journal of Machine Learning Research, 24(244):1 2022b
Klaus-Rudolf Kladny, Bernhard Schölkopf, and Michael Muchlebach. Conformal generative modeling with improved sample efficiency through sequential greedy filtering. arXiv preprint arXiv:2410.016o 6660 2024.
Damjan Krstajic. Critical assessment of conformal prediction methods applied in binary classification settings. Journal of Chemical Information and Modeling, 61(10):4823 4826, 2021. doi: 10. 102 waw.saccc....000000 1c00549. URL https: /tttaa:///ooooooeeooeooo 10. 1021 acss joim. 1c00549, PMID: 34550693.
12

---

Published as a conference paper at ICLR 2026
Siddhartha Laghuvarapu, Zhen Lin, and Jimeng Sun. Codrug: Conformal drug property prediction with density estimation under covariate shift. Advances in Neural Information Processing Systems, 36:37728-3777 37747, 2023.
Ali Madani, Bryan McCann, Nikhil Naik, Nitish Shirish Keskar, Namrata Anand, Raphael R. Eguchi, Po-Ssu Huang, and Richard Socher. Progen: Language modeling for protein generation. arXiv preprint arXiv:2004.03497, 2020.
Ali Madani, Bryan McCann, Nikhil Naik, and et al. Progen: Language modeling for protein generation. arXiv preprint arXiv:2004.03499 2021.
XuanLonng Nguyen, Martin J Wainwright, and Michael Jordan. Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Transactions on Information Theory, 56(11))5847-5861, 2010.
Harris Papadopoulos, Kostas Proedrou, Vassilis Vovk, and Alex Gammerman. Inductive confidence machines for regression. In Proceedings of the 13th European Conference on Machine Learning (ECML), pp. 345-356, 2002.
Drew Prinster, Suchi Saria, and Anqi Liu. Jaws-x: Addressing efficiency bottlenecks of conformal prediction under standard and feedback covariate shift. In Proceedings of the 40th International Conference on Machine Learning (ICML), 2023.
Yanru Qu, Keyue Qiu, Yuxuan Song, Jingjing Gong, Jiawei Han, Mingyue Zheng, Hao Zhou, and Wei- Ving Ma. Molcraft: structure-based drug design in continuous parameter space, arXiv preprint arXiv:2404. 12141, 2024.
Victor Quach, Adam Fisch, Tal Schuster, Adam Yala, Jae Ho Sohn, Tommi $ Jaakkola, and Regina Barzilay. Conformal language modeling. arXiv preprint 2rXiv:2306.10193 2023.
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-trate trainini 2018.
Paul Rosenbaum. Sensitivity analysis in observational studies. Encyclopedia of statistics in behavioral science, 4:1809-1814, 2005.
Victor Garcia Satorras, Emiel Hoogeboom, and Max Welling. E (n) equivariant graph neural networks. In International conference on machine learning, pp. 9323-9332. PMLR, 2021.
Hooman Shahrokhi, Devjeet Raj Roy, Yan Yan, Venera Arnaoudova, and Janaradhan Rao Doppa. Conformal prediction sets for deep generative models via reduction conformal regression. arXiv preprint 0rXiv:2503.10511 2025.
Alexander Shapiro. Monte carlo sampling methods. Handbooks in operations research and manage- ment science, 10:353-425 2003.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of statistical planning and inference, 90(2):227-244, 2000.
Alexey Strokach and Philip M. Kim. Deep generative modeling protein design. Current Opinion in Structural Biology, 72:226-236, 2022. doi: 10. sa..00666.sbi.20211111 2021..1.00.0
Masashi Sugiyama, Shinichi Nakajima, Hisashi Kashima, Paul Buenau, and Motoaki Kawanabe. Direct importance estimation with model selection and its application to covariate shift adaptation. Advances in neural information processing systems, 20, 2007.
Masashi Sugiyama, Taiji Suzuki, and Takafumi Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012.
Jiangming Sun, Lars Carlsson, Ernst Ahlberg, Ulf Norinder, Ola Engkvist, and Hongming Chen. Ap- plying mondrian cross cross-conformal prediction to estimate prediction confidence on large imbalanced bioactivity data sets. Journal of Chemical Information and Modeling, 57(7):1511-1598. 2017.
13

---

Published as a conference paper at ICLR 2026
Fredrik Svensson, Natalia Aniceto, Ulf Norinder, Isidro Cortés-Ciriano, Ola Spjuth, Lars Carlsson, and Andreas Bender. Conformal regression for quantitative structure activity relationship model- ing quantifying prediction uncertainty. Journal Chemical Information and Modeling, 58(5): 1132-1140) 2018.
Ryan| J Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. Advances in neural information processing systems, 32, 2019.
Oleg Trott and Arthur J Olson. Autodock vina: improving the speed and accuracy of docking with new scoring function, efficient optimization, and multithreading. Journal of computational chemistry, 31(2):555-4611 2010.
Dennis Ulmer, Chrysoula Zerva, and André F. T. Martins. Non-exchangeable conformal language 00707.
generation with nearest neighbors. 2024. URL arXiv:2402.00707.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learnnng in random world, volume 29, Springer, 2005.
Renxiao Wang, Xueliang Fang. Yipin Lu, Chao- Yie Yang, and Shaomeng Wang. The pdbbind database: methodologies and updates. Journal of medicinal chemistry, 48(12):411-4119, 2005,
Tian Xie, Xiang Fu, Octavian -ueiannn--eenn-n Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal diffusion variational autoencoder for periodic material generation. In International Conference on Learning Representations, 2022.
Kevin Yang. Kyle Swanson, Wenhao Jin, Connor Coley, and Philipp Eiden et al. Analyzing learned molecular representations for property prediction. Journal of Chemical Information and Modeling. 59(8):3370- 3388, 2019.
doi: (am.:///cc..iimmm000033333
Jason Yim, Andrew Campbell, Andrew YK Foong, Michael Gastegger, José Jiménez-Luna, Sarah Lewis, Victor Garcia Satorras, Bastiaan S Veeling, Regina Barzilay, Tommi Jaakkola, et al. Fast protein backbone generation with se (3) flow matching, 2023.
Claudio Zeni, Robert Pinsler, Daniel Zügner, Andrew Fowler, Matthew Horton, et al. Mattergen: generative model for inorganic materials design. arXiv preprint arXiv:23 12.006877 2023.
Jin Zhang, Ulf Norinder, and Fredrik Svensson. Deep learning- based conformal prediction of toxicity. Journal of Chemical Information and Modeling, 61(66):648-266) 2021.
A ADDITIONAL DISCUSSION
A.1 NON-RANDOMIZED CONFORMAL. P-VALUE
Let II be the collection of all permutations of (1...., To.
......,, 1,..., + k}. For any permutation
II, denote the obtained by permuting the observed features by
V(:; X) = V(Xx(1), Xa(ma)= X (x(++)):-... X (x(++)))
We additionally denote the identity mapping mu. i.e., (1) i for any i. Finally, define
Ph Exen ((;;; X) (V mo: X)= S ((T;
(6)
ΣTEn W[:: X)
Here, w(w;; X)
11 wxXx (X)(n+))) the joint likelihood ratio after permutation æ, where w((x) is the density ratio. In nutshell, p& considers all permutations of the inactive calibration samples and the test samples, and compute the weighted rank of the score computed with observed data, V(Tu: X), among all scores with permuted data.
The validity of the deterministic p-value is established in the following theorem, whose proof is in Appendix B.3.
Theorem A.I. Under the covariate shift assumption, it holds for any fixed & E [0, 1] that
P(Pk Ppk 5 MuxisjjkYYan 0) < t.
14

---

Published as a conference paper at ICLR 2026
A.2 JOINT WEIGHTED EXCHANGEABILITY
In this part, we characterize the exchangeability structure of the inactive calibration data (Xi) and the test samples (Xn+j])= under the null case
Ha:
max Ya+j ISISA
Here for generality, we use the index to indicate that (Xx+j) } might be a subset of the original proposal. The following lemma characterizes the distribution of these data conditional on the labels. Its proof simply applies the Bayes' rule, whose proof is deferred to Appendix B.2 for completeness.
Lemma A.2. Conditional on { ) {Zn-shi21 well as maxisisk Yn+i 0, the features (Xi)iETo are i.i.d. from Po Y=u whereas (Xn+j)5=1 are i.i.d. from Qo = Qx IY=0- Furthermore, dQo/ (2) ) w(z). where w(x) dQ (x. dPx (x) is the density ratio.
aQx
Lemma A.2 states that, conditional on all labels, on the "null" event that H3 is true, the "marginal" density ratio wxx suffices to adjust for distribution shifts between the inactive calibration data in To and the newly generated data. We discuss its implications on density ratio estimation below.
Remark A.3. When w(-) is unknown, Lemma A.2 justifies using a random subset of the labeled data and an independent set of generated samples (no matter whether they are active) estimate w(-)) An alternative approach is to directly estimate the density ratio between {X; JiEz and {Xu+j] }=1 However, this approach leads to quite limited sample sizes when k: small.
Lemma A.2 allows to characterize the exchangeability structure among calibration and test points, extending the weighted exchangeability notion (Tibshirani et al., 2019) to multiple test points. is consequence of conditional permutation-style result which extends foundational results in conformal prediction (Vovk et al., 2005; Tibshirani et al., 2019).
Let X [X,,.... N no: X [x]] = [.,,...: Fng: &ng++ &ng }+]- Then, given [X] [a=]. the only randomness that remains lies which value æ corresponds each X,, Proposition A.4 is direct implication of Lemma A.2, extending the weighted exchangeability notion in Tibshirani et al. (2019) to multiple test points.
X&. be the unordered set of the features, and fix any value
Proposition A.4. Let II be the collection of all permutations of [no + ks). Then, for any fixed values -1,.... Fang Fag +1y ynn and any permutation æ (no k:] [no + k..
10 1ing
peond X1 =xx()) X = (xo+++)
[ [al) Ex'en W( +k)
where we define क([;; #lmooth = I1 (xxx(no+))) and peond (.) denotes the distribution after conditioning on )2 U (Zn+s as well as maxisisk Yn++ 0.
Proposition A.4 describes the probability of Xi,...,Xa X taking on the value of any permutation of æ conditional on the unordered set of their realized values. Such conditional permutation-style result is foundational for many results in conformal inference, thereby connecting our p-value to existing conformal p-values. Specifically, when w(-) = and m 1, it reduces to the standard exchangeability condition, and the p-value (6) coincides with the classical conformal p-value. In general with m 1, it reduces to the weighted exchangeablity studied in (Tibshirani al., 2019), and our p-value (6) coincides with the weighted conformal p-value (Tibshirani et al., 2019; Jin & Candès, 2023a). discuss the connections the next subsection.
A.3 CONFORMAL P-VALUE FOR ONE TEST POINT
Our p-values build on conformal prediction ideas (Vovk et al., 2005). To provide more contexts, briefly discuss the widely used conformal p-value for one test point, the key concept in deriving conformal prediction sets. Given any function s: X' x y RR (often referred as "nonconformity score") whose training process is independent of the calibration data {(2,, Yo) and a new test point X Xn+1, the conformal p v-value defined as
1 + Σ (( Y)) s(Xn+1,
p(y)
n +
15

---

Published as a conference paper at ICLR 2026
When ((X,, Y= exchangeable (such as i..dd) across i [n + 1]. one show that p(Yn+1) dominated by Unif[o, 1]. This property is key constructing prediction sets for the unknown Yn+1- Recently, conformal p-values based on exchangeability are used in the context of multiple testing for detecting outliers (Bates et al., 2023) and identifying test instances with desirable label values (Jin & Candès, 2023b). Extensions covariate shift settings also studied for individual p-value (Tibshirani et al., 2019) and multiple testing (Jin & Candès, 2023a).
Here briefly discuss Remark 3.2. Note that when m 1, n no and w(x) in (7), setting
(zi...... Ena 2n+1) = ((x++1, 0):
where we use = as a placeholder, our p-value (6) reduces to p(0) in (7).
A.4 CONNECTION TO OUTLIER DETECTION UNDER COVARIATE SHIFT
more explicit connection is that of our p-value the outlier detection p-value. Formally, for test point xx+1 independent of calibration "inliers" {X,) iid. ~ P, Bates et al. (2023) proposes using
the conformal p-value (the signs are flipped be consistent with the current notations)
1+ Σ (x, s s(x+11))
n n+1
test the null hypothesis Ho: Xn+1 ~ P, where 8: X > a score indicating how likely a value is an outlier compared with the normal values under P. Under the null hypothesis Ho, it can be shown that the p-value p is dominated by Unif(0, 1].
Jin & Candès (2023a) extend this problem to outlier detection under covariate shift, where Ho: Xn+1 ~ Q for some distribution Q obeying dQ/ dP(x) = w(x) for a known density ra-
tio w(.). The corresponding weighted conformal p-value is
w(Xn+1) + E w(X, (x(,)) S s(xn+1))
Pw
(8)
EE w(X))
In this case, our p-value (6) reduces (8) by setting
Va Tnn #n+1) 8 8(Fn+1)
That is, the score function in our setting can be viewed as an "outlier" score for the new test point.
Again for generality, use the index k (instead of N) indicate that {Xn+s); 1 might be a subset of the original proposal. Finally, we remark that by Lemma A.2, our null hypotheses iid. iid. Qo where dQu/ dPu(=) = w(x) and (Xihezz
Ha: maxisiss Yn+j implies Xntj
Po. Thus, our p-value pı can be viewed as testing the global null hypotheses
M1: (Xn+s)
Lid.
Qo
using calibration "inliers" (Xi Hizo iid. Po. Therefore, pk can be viewed extending P.. in (8) to simultancous inference for multiple test points.
A.5 CHOICE OF SCORE FUNCTION
Drawing inspirations from classical permutation tests, suggest several natural choices:
Max-pooling. Since we aim to judge whether any one of the test samples is promising enough, natural choice is V (2ri:gg #o+1s++)) = max1 ((xn+z), the maximum predicted test score.
Sum-of--rediction. Inspired by Fisher's randomization test in causal inference (Fisher & Fisher, 1971), we can compare the average predicted values between test and calibration data. Up to permutations, is equivalent setting V(x (gimm An+1ln4s Σ ((x++)))
Rank-sum statistic. Inspired by Wilcoxon's 's rank-sum test, we compare the sum of ranks of test points. Up to permutations, we set V (Iimos In+1m+*) = Σラ Ra+j. where Ratj is the rank of ju( (X+z)) among (2 U (f(Xn+z))& (in ascending order).
16

---

Published as a conference paper at ICLR 2026
Likelihood ratio statistic. Inspired by the likelihood ratio test, observe that (á(z) can be viewed as an estimator P(Y = 1 || = z). thus ((z)/(1 (((n+z)) serves as an estimator for the likelihood ratio for testing + = given Xn+j- Accordingly, we consider the joint likelihood ratio for the test points and set V( Tntintk) Σド= log(A(Ent))/(1 ((xn+s)).
A.6 NON-SPLIT VERSION OF CONFHIT
In the main text, we discuss CONFHIT when the conformity score is based on pre-trained model j: X + IR, as standard in split conformal prediction (Vovk al., 2005). This makes our framework naturally compatible with any pre-trained prediction models in the literature (such as those in our experiments). However, practitioners may want to involve all the labeled data in both model training and calibration of p-values. In this part, we present an extension of CONFHIT where the labeled inactive data are used in both training and calibration, yet still maintaining type-I error control. Notably, this feature is unique to our problem (since we condition on the labels on the null event).
Formally, we let A be any generic training algorithm that produces a prediction model A(Z Zx) based on labeled input data (Zi,... Zx) with Z; = (X,, Y:). We assume that the training process of A permutation invariant to its input data, which is satisfied by many commonly used algorithms.
We first consider the certification problem. Recall that the inactive labeled data is {(X,, })} 21 the active labeled data, without loss of generality, is {X,,, 1) f==no+1 and the test data is {X f=1" We consider the imputed data Zn+; = (Xn+j, + 0) for 1 S j S k;, and
1
D = (Z)...., Zn 1. Zn+k).
Then, we let (a-) A(D), i.e., the trained model with the imputed data. Finally, define the conformity scores V as well as the p-value in the same way as in Section 3.1. These p-values for each & E N+ can then be combined in the same way in Section 3.2 to address the design problem.
is straightforward to see that our p-value remains valid conditional on maxisisA [X=+r] = Since the trained model j is invariant to the permutation of data in D. under any permutation over {I...., Ro. + 1..... n kc), the trained model remains the same, and the weighted exchangeability
between the conformity scores follows exactly the same arguments.
B TECHNICAL PROOF
B. PROOF OF THEOREM 3.4
Proof of Theorem 3.4. We define Kg = max(k: & N: maxisis* Yntj= = =0] i.e., the last instance where all earlier molecules non-positive. We also let Ko when Zn+1 so that Ko is well-defined. Note that Ko is measurable with respect to (Yn+sh21. Conditional on {Yn+sl521. Ko thus becomes deterministic. and by the definition of Ñ, we have
P( max Yn+j = 0 (K4tsh21) = Ñ > 0, max Zntj 0 (Yntib221) JSN
ISiEN
=P 0 N Kg
{Zntyl221) 1 [(Ko 0}.
Since the p-values are monotone, we know Ñ S Ko is equivalent to PKo S a, hence
= P PKo <a (2n+sh21) 1 1(Ko > 0}
= =p(pk, S a (Zu+sb22 max Znts = 0) 1(Ko > 03 S a. J5Ko
Here the second line uses the fact that maxjςKo Znts is measurable with respect to (Zn+s)21- as well as the validity condition (ii). Finally, by the tower property, we obtain the desired result. 0
B.2 PROOF OF LEMMA A.2
Proof of Lemma A.2. Conditional on all calibration and test labels well max sjck Ynnj it is clear that the features in In are i.i.d. from Po = Px io= whereas the features {X++j)}
17

---

Proof of Theorem A.I. Throughout we condition on (2:)221 and maxjsk Zn+s = O, and let peoed denote the distribution conditional these information. For any fixed vector [æ)), we let (T;;;;)) x) be the p-value obtained when the data is realized We assume Published as a conference paper at ICLR 2026
Li.d. from Qu Qx Y By the Bayes' rule, the two distributions related by
dQo Q(Y = = dQx w()),
dP (z)
P(Y 0 x) dPx
where Px. Qx are the distribution of X under P and Q, respectively. Here, we used the fact that Q(Y = 0 x = P(Y 0 X x) due to the covariate shift assumption.
B.3 PROOF OF THEOREM A.
X
conditional on X
as (X1...... X ng xn-1..... xn+)) (Iz(1)..... Im(ng) ".tn:t))))....((((:.....
(IT;; X)'s have ties almost surely for simplicity, but all the arguments go through in general.
Under the conditions of Lemma A.2, Proposition A.4 implies that
peondd (ps = (T((T;;))) a) [X] [a=))
w w(T; w)
ppondd X
[X] = [x=]
Eren ((():))
Therefore, noting that by definition
pa(T;; æ)- Ezen σi o n; x) ([V [(T=))} Eren ((+xx) () 1(V ((T;))
Σ Eeen ((T;
Σxen w( (T;
and letting Te be the permutation with the largest (TT;; x) such that (m x) S t, we have
poodd S t |X] = [z=)
Σ (T; s t prand (px = (T;
[X] = [x=])
w(T; x))
Σ
(ox (T; S
E'E W(
well
W(T (T)
Σ ([V(T;;)) 2 V(Ta=)) Σven w(x'x=) (m(m)))))) S t.
SEN
Here the first equality uses the conditional distribution described above, the third equality uses the monotonicity of pa (TT; æ) in V VIT; æ) and the definition of ,., and the last equality holds by definition.
Marginalizing over [X] (but still conditional on {2, 2a+ and maxi Yn+5 = 0) then leads to the desired result.
L
B.4 PROOF OF THEOREM 3.1
Proof of Theorem 3.1. As usual, throughout, we condition on {Yi)21 and maXisjs* Yn+5 = 0, and use Ipound to denote the distribution given these information. We condition on the event that the unordered set [x] [x]] for any fixed vector æ (ZI, Ing In.1..... ..++k)). The labels
{Z,)== are also conditioned on usual. The randomness then comes from (i) which values in [x] correspond to each data point, and (ii) the randomly sampled permutations.
Let Å be the unique permutation of [1...., Ro. Ri. n+1, k) such that X (I+()).... #+(++t))
Then for any permutation æ E II, we have
V(T: X) = V( ( (0); X) = V(T o ,t;)),
and similarly for ((;; X) क( n; ..). By Proposition A.4, we know that TP(#
[
(z)) xx w(T;;)), where we recall the definition of ((; = II w ((( addition, we have
w(+:))
+ b= D w( o ( 1V < (
rad
Pk
w( xt) + EE w (1);
18

---

Published as a conference paper at ICLR 2026
where z is the fixed vector, and the randomness comes from ã and { 1b=1
For notational simplicity, denote =00) â., and (()) â (b)) for b = I...., B. We now study
the joint distribution of (# ( ..... ( For any fixed permutations no. .o...... BB II, we note
peond [10) = To, (1))
T1
# (B)
TB
X = [x)]
poond क = To: x = = T1, ((B) = = o TB [X] = [x]
1 |
peond
i
no
[X] (x)) w( no: m))
where To is the inverse permutation of Tn. Above, the last equality uses the fact that n T(1), ..... (B)
are i.i.d. uniformly sampled from II and independent of everything. Therefore, when we further
conditional on the unordered set of realized values of the permutations [#(0), =(1), ..... *(B} =
[Mo, ..0....,........................ TB]
] for any (fixed) value of To...... TB. we know that
poomd (+ (1) (1),
(B)) = (T) Fo(B)) (()),(0))
#(1)
[(B)] = [mo.... .... [X] = [a))
w (TT(()))
Eoeï 10 (T)
where IT is the collection of all permutations of {0, 1,.,,,, B}. This also means that for every b E {0, (O......,B) B},
peodd ((()= Tb (0) (() ( [To..... ..l..,,
1 Tb:
E8= W(TT; x))
On the other hand, for any E II, when
(B1) (To ................................ (o(B)), we have
((T x:) + E 87 (b): x)) 1 (Ta(o): 2)) V (TTor())
cand
Pk
((F(())))) >m= t6 (T)())))))
B
E w(m; x)) (T)) V (m);))
(10)
Ee (Tb; a))
That is, the value of pk poodd only depends which value among (To. ........ Tn) the data permutation (()) takes on. Thus,
poond
rand
< (0) (1) ,..., [B]] [mo. TR], [ =(xi)
Pk
Σ (#(0) (1)
,..., ( = (To {B) [0(1), (1) ..... (B)) = [TO. [
señ
2 0 oe (Tb x) 1 VVV (
(To 5 (TTOi x)}
1
St
Σ xi (hi x))
B
B
Σ
(0) = Tb.
[+(10] (10))
(B) I ..., TB
6-=0
Σ ur (p(); 1 (V(TT=;;)) < V (m); 2r <
x
Σh (m);;))))) ac)
w Tb= xr)
b (i x [(V (TT+ :)) < (To: S st St.
1
ー w0 (V;
Σn w( Th; x)
Here, the first equality follows from (10), the third equality follows from (9), and the last inequality by definition. Finally, marginalizing over [X] and [+(0) ,..., (B)) yields the desired result. n
19

---

Published as a conference paper at ICLR 2026
B.5 PROOF OF THEOREM 3.5
In this part, we prove the robustness of joint weighted conformal p-value with estimated density ratio. For generality, we use a general index k instead of the total budget N. to indicate that {Xn+i) 3=1 f==1 can be a subset of the original samples (as in Section 3.2). For conceptual simplicity, we present the detailed proof for deterministic p-values (Appendix A.I), and the result for randomized p-values in the main text naturally follows with the same ideas.
Proof of Theurem 3.5. Let (.) denote the distribution conditional on all the labels (Y:)21 and the event maxisjsk Yn+ fixed value æ, we let ph (T;; (a x) be the p-value with the correct weights w(.) obtained when the data is realized as (X .
O. Following the arguments in the proof of Theorem A.1, for any
(I+(I), ................................ In(o))) (x(++))........ (xxx)))))))))))) and similarly we define (T; Then we have pu (V(VV (T; where Pu (t) is defined in Theorem 3.5, and by definition, we have
-X.t - .x+k))
Σ Li S0 TEN
(0) sisi.
E Eve ((T)())
In addition, by the definition of f, we have (T; S f if and only if (s t for any e II. As a result, we have
pound (ix S <
= =0) = - 1l [6x (T; (T) S S.
W [T; x
Σn eee en ;(T';
mEll
Σ
(1; Si}
[T; x)
r'Ell ((<';z)
nEll
((T:))
@( x)) En ((T1:)))
< 1 Σ (p<(TT;)) S 1)
Er' (TT:=))
tell
((T;; ar)
w ar)
< f Σ ((V (T;x) 2 0}
E'' (((':())
Even ((T';
TEll
St+ Σren ((;:)) (T; 2 0)
Eren Ú(T;; m)) 11 [[V(T::)) 0) =.
Even x(
Σven w(T1;
Rearranging the terms, we have
Å Emen "(T;;))) (V
81) E TEl ((T:)) <
Ere ((< Σ
Erell ((T(;=))
6))
TEn LVV 0))
T'l ((T(;))
Enenlu + 8- (T: (T: 6)
Emen ((
(E ((T: E E ((T';)))
EEeenL0 [ύ(T;)) x) - [);))))) 1(V 6)) (E Ú(;)) 1[(T)) 20)
Eren ò (T;I) ([V 2 0) Σ Ten u (T;z) <))
(Eren W(T;: x)) Exee ((T';z))
(Eren (t;())) (VV m) < i)) (Eren ((T;)) x) 2))
(Eren ((<)==)) (Erem (();;))
where we used the non-negativity of the weights and the definition of 8 (T) Further, noting that
f - Exen ((x;)) L(8V we thus have
Even ((x)))
Å (1-1) (Eren ò- (τ;)) 1[V x) 2 0)) +8. (Eren )+;);)) ( x) < <))
Σven x)));
20

---

Published as a conference paper at ICLR 2026
which completes our proof for the results of pk-
Finally, all the arguments still go through (following some arguments in the proof of Theorem 3.1) for Pk after replacing by (a [=o and additionally conditioning on the unordered set of { ( ,..., π. ). where स is the permutation such that X (#*(1)) (x(((())))) We omit the details here for brevity.
C EXPERIMENTAL SETTING DETAILS
C.I CONSTRAINED MOLECULE OPTIMIZATION
For training the generator, used the publicly available training code from the HGRAPH2GRAPH (Jin et al., 2020) repository. We retained the default training/test splits and randomly held out 20% of the training set for calibration. The test set contains 800 samples and is also sourced from the same repository.
For the SELF-EDIT (Jiao et al., 2023) baseline, used the same data splits and obtained the training code from its official repository. Both models were trained with their default configurations, without modification.
In all cases, we sampled up to 50 candidate molecules per input until sufficient number of eligible candidates were obtained. We defined ineligible candidates those that were invalid (e.g.. chemically invalid), duplicates, or out-of-distributte those test samples for which at least N {5, 10) valid candidates could be generated.
(OOD) with respect to the training data. We retained only
The property oracles were sourced from RDKit for QED and from the Therapeutics Data Commons library for DRD2.
Density Estimation and Scoring Function. We trained a molecular property predictor using the Chemprop( Yang et al., 2019) library for both QED and DRD2 tasks. The model used 3-layer Message passing Neural Network with final hidden dimension of for feature extrac- tion and was trained for 5 epochs using the default Chemprop trainer settings. We used the i wrtoomoonenteessageeee
in module without any modification.
For density estimation, we followed the CODRUG approach by extracting penultimate layer features and fitting a Gaussian kernel density estimator (KDE) using the scipy library with bandwidth of 1. To filter OOD samples, we computed the 95th percentile density the calibration set and removed test-time generated molecules falling below this threshold. Additional samples were drawn until the target number N of valid candidates met.
For sequential p-vuuue computation, we used 2000 Monte Carlo samples per test instance.
C.2 HYPERPARAMETER CHOICES AND DESIGN JUSTIFICATIONS
Density ratio model and feature selection. For density estimation, follow the CoDRUG proce- dure, which has been shown to be robust for small-molecut density estimation under covariate shift. We the penultimate-layer representation of the property predictor, which yields 4-dimensional feature vector that is expressive yet oww-dimesioona enough to enable stable kernel density esti- mation. For bandwidth selection, we adopt the same strategy as CODRUG: we perform 5-fold cross-validation over bandwidths 10.1, 1, 10), compute the KDE likelihood on each held-out split, and choose the bandwidth with the highest average likelihood. This avoids post-hoc tuning and ensures that the KDE matches the calibration distribution as closely as possible.
Predictive model selection. Conformal validity does not depend on the complexity of the predictive model, but the power of the method improves when better predictors and more discriminative score functions are used. To avoid unnecessary tuning, we select models with strong reported performance and publicly available implementations. For the CMO tasks (QED/DRD2). we train standard Chemprop MPNN. For the SBDD task, we use the default EGNN predictor from the TARGET DIFF implementation. These choices are simple, stable, and sufficient for producing informative conformity scores.
21

---

Published as a conference paper at ICLR 2026
Choice of test statistic. Since the procedure operates ser of generated molecules, the choice of score statistic affects the power but not the validity. We use the maximum score across the set, which is a natural heuristic in this context: if at least one strong candidate is present, the max-score captures this evidence immediately, whereas statistics such as the mean or min dilute the signal with weaker candidates. Empirically, found the max statistic to produce higher power and lower rejection rates than alternatives.
Number of permutations (B), We set B = 2000 Monte Carlo permutations, which we found sufficient for stable performance while maintaining modest computational cost. Running the full CONFHIT procedure with B = 2000 takes only 4 minutes for 800 samples (see Fig. 6). We also evaluated B E (500, 1000, 2000, 5000} and observed negligible variance in empirical coverage across these values, confirming that the Monte Carlo approximation converges quickly. Runtime scales linearly with B, and 2000 provides an effective balance between statistical stability and efficiency.
DRD2 Hgraph, N
(%)
0=0 (feen Std
0-0 3 (feeen a Std
35
a 5 (Meon a 5kd)
32
Enter Roto 80
23
0.:0.50
p=0.39
10
Number Permuratiion
Figure 6: Error rate function of the number of Monte Carlo permutations B for DRD2 with HGraph2Graph (N = 5). The empirical error rate remains stable with minimal variance.
C.3 STRUCTURE-BASEE DRUG DISCOVERY
For this task, used pre-trained generative model checkpoint obtained from the official TARGET- DIFF (Guan et al., 2023) repository, along with their corresponding test set.
To estimate scores and extract features, we trained an EGNN- model using their provided codebase. We modified the final hidden layer to output an 8-dimensional embedding. For validation data, used ligand complexes from the CrossDocked2020 (Francoeur et al., 2020a) dataset, excluding all training and test set entries to avoid data leakage.
Density estimation was performed as above, using Gaussian KDE with bandwidth of 1.
C.4 COMPUTE RESOURCES
All experiments were conducted on single NVIDIA A100 GPU. The approximate runtime per component is as follows:
Property predictor training (QED/DRD2): 20 minutes
HGRAPH2GRADE APH generator training: 10 hours
SELF-EDIT training: 48 hours
Density estimator training for SBDD: 24 hours
Inference using TARGETDIFF: 30 minutes
CONFHIT procedure per test batch: 30 minutes (over 800 samples; 2=2.5 seconds/sample)
Candidate generation (CMO task): 14 minutes (2:1 seconds/sample)
22

---

Published as a conference paper at ICLR 2026
Predictor feature extraction: 6 minutes (==0.45 seconds/sample)
KDE computation: minutes (zv0.23 seconds/sample)
Running the conformal procedure: minutes (=0030 seconds/sample)
C.5 CODE AND DATA AVAILABILITY
We include with the code the following:
Model checkpoints.
Extracted features and CrossDocked identifiers used for calibration SBDD.
Calibration and test splits.
Configuration files for reproducing results across datasets and models.
Installation instructions.
The code is publicly available at CONFHIT-Con f rormal. - -Generat Destiiieeeeeegneeeee fre and is distributed under MIT license.
D WORKING UNDER AN OVERALL BUDGET
In Section 4.6, we discussed the application of CONFHIT in providing practically useful heuristics while constructing predicting sets when fixed total budget B is available. The detailed algorithm is provided in Algorithm 2.
Algorithm 2 Working under an overall budget
Input:
Calibration data {e with Y; = 0, test batches (X(t)) of size N, conformity score V, weight function ù. grid of confidence levels A C (0, 1), total budget N
I: Initialize Pmax +- C + 0, a* 0. do 3: For each test batch = 1,..., T, run Algorithm with level a to obtain prunded prediction sets c(e)
2: for each
Compute total cost Σi= [c!))
5: Record number empty sets = Ht c.a) = 0H|-
6:
if C' > B then
7:
Sort C((a in descending order of size.
8:
Iteratively delete until total cost B. Let be the number deleted sets.
10:
else
11:
Set D + O.
12:
end if
13:
Estimate positives P(a) (1-a)
14: if P(a) Pmax then
15:
Update Pmaxx P(a), (C..u)).
end if
17:
end for
Output: Best estimated positives Pmax with corresponding sets C* at significance level a*
E ADDITIONAL EXPERIMENTS
E. DETAILED RESULTS
In this section, we provide error bars for the plots in Sections 4.3 and 4.4. The tables include Error Rate, fraction certified, empty set fraction and mean set size depicted in 4.3 and 4.4 along with standard deviation computed over 5 random runs, across different methods. Further, report metrics on the Constrained Molecule Optimization (CMO) task, using the Qualitative Estimate of LikgeLLieeeess (QED) property with success (Y 1) if QFD(x:) 0.9, following Jin al. (2020).
23

---

Published as a conference paper at ICLR 2026
E.1.1
CERTIFICATION RESULTS
Table I: DRD2 HEDDPH Cortification Results
N
Q
Error Rate (%)
Certified Fraction (%)
3
0.1
6.5 + 1.1
22.6 = 2.4
0.2
9.6 1.4
33.2 = 4.1
3
0.3
18.5 + 1.9
39.8 = 4.7
3
0.4
26.8 2.1
47.6 = 3.5
3
0.5
26.0 2.0
57.5 + 6.8
0.1
2.0
43.0 + 4.3
5
0.2
9.3 1.2
53.0 4.6
0.3
1.2
6.2
5
0.4
21.8 2.6
4.6
5
0.5
21.6 - 2.2
70.2 6.1
7
0.1
5.5
51.5 5.1
7
0.2
8.1
2.5
7
0.3
11.8 3.1
66.2
0.4
19.7 3.2
70.4 = 3.6
7
0.5
20.6 + 2.2
74.5 + 6.2
Table 2: ORD2_SHLFEDIT Certification Results
N
a
Error Rate (%)
Certified Fraction (%)
3
0.1
7.1 + 0.4
19.1 + 2.1
0.2
8.7 1.7
21.5 = 3.1
3
0.3
19.8 + 1.2
28.2 = 3,2
3
0.4
19.8 1
39.3 4.1
3
0.5
+.3
42.5 +.2 5.2
0.1
1.0
22.8 3.1
5
0.2
1.3
25.9 3
0.3
16.4
30.2 4.1
5
0.4
1..4
4.6
5
0.5
26.9
5.1
7
0.1
6.3 0.9
29.0 2.2
7
0.2
1.6
3.1
7
0.3
4.7
0.4
17.1
47.6 4.9
7
0.5
2.24
51.6 5.6
24

---

Published as a conference paper at ICLR 2026
Table 3:
Certification Results
N
a
Error Rate (%)
Certified Fraction (%)
3
0.1
7.8 1.2
49.3 = 2.0
3
0.2
13.4 + 1.6
47.8 + 1.8
3
0.3
16.3 2.1
54.0 = 1.0
3
0.4
19.4 2.7
56.8 + 1.0
3
0.5
21.8 3.2
60.8 3.4
7
0.1
7.9 0.5
0.2
14.6
56.5 2.0
7
0.3
17.9 2.1
5.2
0.4
20.3
7
0.5
22.5 3.4
72.0 2.7
Table 4: QED_SELFEDIT Certification Results
N
a
Error Rate (%)
Certified Fraction (%)
3
0.1
7.6 + 1.2
42.6 + 6.1
3
0.2
10.4
5.9
3
0.3
1.3
5.7
3
0.4
1.17
5..0
3
0.5
17.2
62.7
7
0.1
1.4
52.2
7
0.2
21.6 + 2.1
62.3 1 4.0
7
0.3
29.7
61.2
7
0.4
30.8 1.2
77.9 + 4.3
0.5
32.9 3.2
78.4 = 1.6
Table 5: TARGETDIFF Certification Results
N
Error Rate (%)
Certified Fraction (%)
5
0.1
8.4 = 2.6
42.9 + 2.5
0.2
10.8 3.0
52.1 2.9
5
0.3
11.4 2.3
57.7 3.8
0.4
14.9 4.4
64.3 3.6
5
0.5
17.1
68.5 3.4
10
0.1
2.5
57.8
10
0.2
8.2
33.
10
0.3
9.0
08.8
10
0.4
1
72,4
10
0.5
12.8
3.9
76.6 3.7
15
0.1
10.1 1.1
69.5 3.3
15
0.2
7.6
70.0 4.6
15
0.3
1.5
73.1 5.1
15
0.4
9.8 2.0
74.5
15
0.5
12.0 + 1.8
79.7 2.3
25

---

Published as a conference paper at ICLR 2026
Table 6: MOICRAFT Certification Results
N
a
Error Rate (%)
Certified Fraction (%)
5
0.1
7.0 1.2
40.1 + 2.7
5
0.2
11.0 1.8
47.9 3.5
5
0.3
12.7 1.0
55.5 + 4.9
0.4
15.1 1.2
63.1 4.9
5
0.5
16.6
68.1 3.8
10
0.1
5.2
58.3 5.8
10
0.2
6.7
5.8
10
0.3
7.5
4.9
10
0.4
1.6
7.11
10
0.5
9.0
7.5
15
0.1
7.3
4.14
15
0.2
9.1
5.8
15
0.3
9.1
5.9
15
0.4
9.5
7.21
15
0.5
11.5
7.8
Table 7: DECOMPDIFF Certification Results
N
Error Rate (%)
Certified Fraction (%)
5
0.1
3.2 2.2
56.6 + 1.4
5
0.2
3.1 = 1.9
63.5 + 1.2
5
0.3
4.7
69.0 + 2.4
0.4
2.4
73.4
5
0.5
2.6
1.2
10
0.1
2.1
7.04
10
0.2
3.1
3.5
10
0.3
3.1
79.4
10
0.4
2,2
82.7
10
0.5
3.0
82.7
15
0.1
2.
8.72
15
0.2
2.5
5.09
15
0.3
1.7
5.3
15
0.4
1.8
6.2
15
0.5
2.5 3.1
87.1 5.2
26

---

Published as a conference paper at ICLR 2026
E.1.2
DESIGN RESULTS
Table 8: DRD2_HGRAPH Design Results
N
Error Rate (%)
Mean Set Size
Empty Set (%)
3
0.1
4.2 = 0.3
1.9 + 0.0
68.8 + 1.3
3
0.2
6.6 = 0.5
1.8 0.0
46.1 = 1.3
3
0.3
9.1 0.6
1.8 0.0
25.3 + 1.8
3
0.4
0.8
1.6
15.0 1.3
3
0.5
13.0
+.0
9.9 0.8
5
0.1
5.4 1.6
2.4
47.9 + 2.6
0.2
2.7
2..7
5
0.3
3.3
0.8
15.8
0.4
4.0
1.5
1
5
0.5
1.4
1.5
8.8 1.5
7
0.1
3.6 0.8
3.1
53.0 1.2
7
0.2
6.7
2.1
30.0 1.4
7
0.3
9.0
2.1
1.7
7
0.4
1..2+111
1.8
9.2
0.5
12.9
1.7
0.6
DRD2_SELFE SELFEDIT
Table 9:
Design Results
Error Rate (%)
Mean Set Size
Empty Set (%)
3
0.1
9.8 = 1.3
2.0 0.0
64.5 + 0.5
3
0.2
14.6 + 2.2
1.9 0.1
42.3 0.2
3
0.3
19.6 3.2
1.0
24.0 + 0.9
3
2.2
1.6
12.3 + 0.1
3
0.5
24.5 4.0
1.5
7.3 0.6
5
0.1
9.5 + 1.2
2.1
53.0 + 1.5
5
0.2
2.4
2.1
33.3 + 2.6
5
0.3
20.7
2.0
14.9
5
0.4
2.6
1.7
1.1
0.5
28.8 3.1
1.6
4.2
7
0.1
11.8
3.1
46.6
7
0.2
19.5 3.3
2.0
22.933.5
7
0.3
24.8 6.3
2.1
12.2 4 1.9
7
0.4
28.6 7.1
1.8
7.7
7
0.5
6.3
1.7
0.6
HED_HGRAPH
27

---

Published as a conference paper at ICLR 2026
Table 10: QED HGRAPH Design Results
N
Error Rate (%)
Mean Set Size
Empty Set (%)
3
0.1
10.1 + 0.7
1.2 + 0.0
70.7 + 1.9
3
0.2
17.0 + 0.6
1.3 + 0.0
45.4 + 1.6
3
0.3
21.2
1.4
1.3
26.8 + 0.9
3
0.4
25.2 1.5
1.0
13.2 - 1.2
0.5
28.2
1.2
6.9 0.6
5
0.1
12.2 0.7
2.0
27.2 + 0.2
5
0.2
18.8 1.1
2.3
7.1
5
0.3
24.4
1.8
0.0
5
0.4
27.7 1.3
1.6
0.0
5
0.5
29.9 + 1.0
1.5
0.0
7
0.1
10.4+112
2.5
56.6 0.8
7
0.2
1
2.0
3.87
7
0.3
1.
1.7
1.6
0.4
1.5
1.4
9.5
7
0.5
1.
1.0
4.1
QED_SELFEDIT
Table 11: QED_SELFEDDI Design Results
Error Rate (%)
Mean Set Size
Empty Set (%)
3
0.1
10.1 + 0.4
1.2 0.0
64.8 + 1.8
3
0.2
14.4 + 0.9
1.3 0.1
45.2 1.5
3
0.3
18.8 1.4
1.3
27.1 + 1.8
3
21.5
1.2
13.3 1.0
3
0.5
+.94
1.2
7.0 + 1.0
5
0.1
11.9
2.7
26.0 2.4
5
0.2
20.1
2.1
0.7
5
0.3
2.48
1.9
0.0
5
0.4
2.5
1.6
0.0
5
0.5
2.1
1.2
0.0
7
0.1
18.2 2.7
2.1
56.7
7
0.2
3.8
1.9
2.9
7
0.3
34.5
3.0
3.0
1.6
18.3 + 2.2
7
0.4
39.2
1.4
10.2 + 2.5
7
0.5
42.
1.3
5.7 1.6
TARGETDIFF
28

---

Published as a conference paper at ICLR 2026
Table 12: TARGETDIFF Design Results
N
Error Rate (%)
Mean Set Size
Empty Set (%)
5
0.1
10.0 + 1.5
2.5 + 0.1
37.9 + 2.6
5
0.2
17.4 + 1.5
2.2 + 0.1
21.4 + 3.1
5
0.3
27.2 + 3.5
2.0 0.1
8.6 + 0.0
5
0.4
34.2 + 2.9
1.7
2.1 1.4
0.5
37.0
1.2
1.6
0.4
0.1
9.4 2.7
3.1
2.0
10
0.2
3.5
2.9
3.5
0.3
29.4 0.9
0.0
1.0
10
0.4
35.6 0.9
1.8
0.0
10
0.5
37.5 + 3.5
1.6
0.0
15
0.1
11.2 1.8
4.7
15.0
15
0.2
21.9 2.7
0.0
3.8
15
0.3
31.9
2.4
1.8
15
0.4
$
1.8
0.6
15
0.5
38.1 2.7
1.6
0.0
MOLCRAFT
Table 13: MOLCRAFT Design Results
N
Error Rate (%)
Mean Set Size
Empty Set (%)
5
0.1
9.8 + 1.2
2.6 + 0.1
40.7 1.9
5
0.2
18.7 + 1.4
2.3 0.2
20.7 5.6
5
0.3
26.0 3.1
2.0
7.7 0.7
5
0.4
29.3 + 5.3
5.3
1.7
2.4
5
0.5
32.9 4.2
1.6
1.2112
10
0.1
1.3
0.3
22.2
0.2
18.2
3.2
9.1
10
0.3
25.1 + 0.7
2.2
2.9
10
0.4
1.4
1.8
2.1
10
0.5
29.6 2.5
1.7
0.4
15
0.1
1..1
4.3
16.0
15
0.2
1.0
3.2
1.2
2.0
2.2
1.2
15
0.4
2.0
1.8
1.2
15
0.5
2.0
1.6
1.0
DECOMPDIFF
29

---

Published as a conference paper at ICLR 2026
Table 14: DECOMPDIFF Design Results
N
Error Rate (%)
Mean Set Size
Empty Set (%)
5
0.1
9.6 + 2.6
2.6 + 0.3
25.0 + 2.2
5
0.2
16.7 + 4.7
2.3 + 0.2
11.7 0.7
5
0.3
24.2 0.7
2.0 0.0
2.5.2.2
5
0.4
26.7 0.7
1.7
1.2 0.0
0.5
28.8 + 1.2
1.6
0.8
0.1
11.2 0.5
3.1
1.9
10
0.2
19.2
2.7
2.6
0.3
24.4 3.6
2.1
0.0
10
0.4
26.3 + 0.9
1.7
0.0
10
0.5
27.6 0.9
1.6
0.0
15
0.1
+.9
4.4
2.3
15
0.2
3.6
2.9
1
15
0.3
2.5
2.1
0.0
15
0.4
3.6
1.8
0.0
15
0.5
27.6
1.7
0.0
30

---

Published as a conference paper at ICLR 2026
E.2 COMPARING DIFFERENT TEST STATISTICS
In this section, we include additional results for Section 4.5 comparing different test statistics. While the power varies based on the choice of the test statistic, CONFHIT attains valid coverage across statistics.
Comparison of Score Functions for DRD2 (Hgraph) N=3 Rejection Rate at = 0.1 and 0.3
Error Rate vs. Alpha
min
mih
50-
Rejection Ralt (56)
50
Hos hoodd ratto
Hoeehoodd_
ymuttaU (tarpet)
11.0%
20
10
4.1% 9.1%
0.2
Alpha (Signiffcance Levell
Figure Comparison of score statisties on DRD2 (HGRAPH2GRAPH, N=3). Left: rejection (power) at a 0.1, 0.3. Right: error vs. a
Comparison of Score Functions for DRD2 (Hgraph) at N
Rejaction
vs. Alpha
60
min
mcbn
mean
Hoood rallo
Reellroo reliu
axjecction Rate (%6
rark_sum
rani_sum
60
Earoa Rate
43.7%
36.7%
11.242125
20
Alpha (Signiffiance Level)
Figure 8: Comparison of score statisties on DRD2 (HGRAPH2GRAPH, N=5). Left: rejection (power)
0.1, 0.3. Right: error vs.
Rejection Rate Comparison Functions (Hgraph)
Error Rate
100
Rnjection Rake 183)
ratio
30.5
$4.25
rank sum
max
A0
maax
rank, sum
100 Strrget)
40
33.7%
16.7%
10
0.1
0.3
0.2
0.6
0.6
0.8
1.0
Alpha (Significance Level)
Figure 9: Comparison of score statistics on DRD2 (HGRAPH2GRAPII, N=7). Left: rejection (power) at
0.1, 0.3. Right: error vs.
31

---

Published as a conference paper at ICLR 2026
Rejaction Comparison Score Functions TARGETDIFF (selfedit) at N
100
50
Rejection Rate 0%1
Ihol satio
ratio
rark, Sum
13.0%
-0
max
rani, sui
00
Bate
y-er 100 (tarpet)
40
23.156
0.2
0.8
Alpha (Signiffcance Levell
Figure 10: Comparison of score statistics on SBDD (TARGETDIFF, N=5). Left: rejection (power) at 0.1, 0.3.
Right: error
Rejection Comparison Score Functions (selfedit) Rate
100
$5.0%
50
Rejection Rate (N)
ratlo
rato
rank
40
rani sum
max
y=o Starpet
40
15.84
12.09
0.3
an
0.2
0.4
066
0.5
Alpha (Signiftrance Level)
Figure 11: Comparison of score statistics on SBDD (TARGETDIFF, N=10). Left: rejection (power) at a
0.1, 0.3. Right: eтror vs.
32

---

Published as a conference paper at ICLR 2026
E.3 P-VALUES
In this section, we show the validity of conformal p-value plots described in section 4.5, across tasks and datasets. In each panel, the distribution remains close the uniform (dashed line; also the reported KL divergence from uniformity), confirming approximate validity of our p-values (albeit slightly conservative).
XL-divergence 0.0061
KL-divergence 0.0066
KL-divergence 0.0080
Denaity
0.0
0.0
0.4
0.6
0.0
0.6
0.8
1.0
p-value
p-value
p-value
Figure 12: Density of conformal nested p-values for QED using SELFEDIT at set sizes k = 3,5,
k 1, KL-divergence 0.0002
k 5, KL-diverrennce 0.0006
k 10, KL-divergence 0.0007
0.75
Density
Denaity
0.50
0.25
0.25
0.0
0.2
0.4
1.0
0.0
0.2
0.4
0.6
0.8
3.0
0.5
0.8
1.0
p-valuee
p-value
p-value
Figure 13: Density of conformal nested p-values for DRD2 using HGRAPH at set sizes h: 3,
divergeeee
Density
Damaity
0.5
0.5
0.0
0.0
0.0
p-value
Figure 14: Density of conformal nested p-values for SBDD using TARGETDIFF sizes k 1,5, 10.
F EFFECT PREDICTOR QUALITY
In this section, we investigate how the accuracy of the predictor affects the performance of CONFHIT. consider three scenarios:
1. Normal predictor where the calibrated classifier outputs p Pr(active xz).
2. Noisy predictor obtained by adding uniform Gaussian noise, giving
p ((0,1
3. Inverse predictor obtained by replacing p with 1 - p.
Figure 15 illustrates how CONPHIT behaves under these perturbation settings. Even when the predictor is corrupted or inverted, the method maintains valid error control, with empirical error rates remaining below the target line y 100a. This demonstrates that the statistical validity of CONFHIT does not rely on predictor accuracy.
Predictor quality affects only the power. As the predictor becomes less informative. the conformal sets become less efficient, which leads to a higher frequency of empty sets or larger sets depending on the task. Despite this expected degradation, the method consistently preserves its coverage guarantees.
33

---

Published as a conference paper at ICLR 2026
Adjection Kaba at and
Errrr Rade Alphaa
parpt
Aghn (ightfkaace Level!
OED ligmphh at
Retection Rate 6.3 and and
Error Alpha
poattt Sareet
Rxpocton
0-
Figure Effect of predictor quality on CONFHrT's error rate the CMO task. both DRD2 (top) and QED (bottom) using the Hgraph model, noisy and inverse predictors exhibit reduced power but maintain valid error control across all values of a.
2R07 Raselie Pirer Kate vs Alphs
DED Rapeline Encer Pate Aigha
Falo do
Figure 16: Results for heuristic baseline with predicted probabilities (while varying predictor quality), showing rejection rates and error rates for QED with N using the Hgraph model. As predictor quality deteriorates, the performance quickly drops with severe violation error control.
G ADDITIONAL BASELINES
G.1 HEURISTIC BASELINE
We evaluate a simple confidence based heuristic. Given a predicted probability ş success, heuristic selects the smallest such that (1 -p) S a. This provides natural baseline that depends only on classifier scores without any uncertainty calibration.
The results in Figures 16 and show that this heuristic does not reliably control the error rate. When the predictor is accurate, the error rates sometimes follow the target line, but under noisy or inverted predictors the increases sharply and often surpasses the desired level. The degradation is visible across both DRD2 and QED: the error curves drift upward as the predictor worsens, and the rejection rates fluctuate significantly, indicating unstable behavior.
In contrast, ConfHit remains below the target line in all cases. The predictor quality affects only the power, whereas the heuristic baseline loses emor control as the predictor becomes unreliable.
34

---

Published as a conference paper at ICLR 2026
Error Rate Alpha
Set Percentage Alpha
CREZ Souine
103
CRD2 briginal
0100 Original
OBB Ohopnd
DED Oripeel
Emotyy Set Percentaue (N)
Error
Alpha
Aigha
Conformal LM baseline for molecule optimization. Error rate (left) and empty-set percentage (right) versus for QED and DRD2 N 5 using the Hgraph model. The method breaks coverage and underperforms CONFHIT despite relying on an oracle that is unavailable real-world settings.
Figure 17:
The empirical curves highlight that the heuristic transitions from reasonable to poor performance, while ConfHit maintains stable guarantees across all predictor settings.
G.2 COMPARISON TO ORACLE BASED METHODS
We adapt an oracle-based CLM (Quach al., 2023) baseline as depicted in Algorithm use the sum of prediction scores as the total confidence of candidate set, without performing any density correction or filtering of low-quality samples. As shown in Figure 17, this baseline fails to maintain valid guarantees: the empirical error rate systematically exceeds the target level, and the empty-set percentages deviate sharply from expected behavior across all a. Despite relying on an idealized oracle, this approach also exhibits lower power than CONFHIT, whereas CONFHIT attains both higher detection power and valid coverage due its calibrated scoring rule and density-aware correction. Finally, we emphasize that this bascline is not directly comparable CONFHIT, since assumes access to perfect oracle predictor an unrealistic requirement in practical molccular design workflows.
Algorithm 3 Conformal Sampling for Constrained Molecular Optimization
Input: Input molecule x; generator p- x); scoring function F: calibrated threshold X; sampling budget kinax
Output: Candidate set CA certified to contain at least one improved molecule
1: C2 + Ø (Initialize output set)
2: for k= kmax do
3: Sample y/ p-(- x)) (Generate candidate)
4: C₃ + C+ U { {Add to set)
5: if F(x, Cx)
then
6: break (Set certified)
7: end if
8: end for
9: return Cx
H ROBUSTNESS AND DIAGNOSTICS FOR DENSITY RATIO ESTIMATION
H.1 SENSITIVITY ANALYSIS
We study the effect of misspecifying the likelihood ratios by applying power transformations ? across range of exponents 7. Figure 18 shows how such transformations affect the KL divergence of the p- p-vuuu of the negative samples, together with the resulting error rate and rejection (empty set) rate. In practice, such results can be used to judge whether the analysis robust enough. Here,
35

---

Published as a conference paper at ICLR 2026
QED KL Divergence va Weight Power
QED Accuracy Metrica Weight Power
Naster Joo
2.3
33
06
Divergence Weight Power
Accurncy Metrics Weight Power
73
Weight Power
Weight Poweer
Figure 18: Effect of power transformations w of likelihood-ratio weights QED and DRD2 using the HGraph2Graph model at N different a. Across transformations, CONFHIT maintains graceful degradation of error control with major changes in terms of the rejection rate.
additionally evaluate the error control performance under such perturbation to also show the robust performance of CONFHIT.
The KL divergence here measures the deviation of the empirical p-value distribution from the Uniform(0.1) distribution. Smaller values indicate that the transformed likelihood ratios still produce p-values close to uniform, while larger values reflect increasing distortion away from the ideal uniform shape. In both the datasets, we see that the KL divergence is only moderately affected under moderate misspecification. Only amplifying the weights (7 >> 3) causes the KL divergence rise sharply in QED case we notice substantial deviation from uniformity.
The етrоr rate consistently stays near the target level across all transformations for = 0.1 and below the target level for a 0.3. Only the rejection rate is affected, with lower rejection occurring when the KL divergence is small and higher rejection when the p-values deviate more strongly from uniform. This shows that mild misspecification primarily adjusts the conservativeness of the method without affecting its validity.
similar pattern appears for DRD2. Flattened weights reduce the KL divergence, while large values o inflate it. Error control remains stable throughout, whereas the rejection rate increases in the high-KL. KL. regime corresponding to more severe distortions of the p-value distribution. Overall, these results demonstrate that ConfHit is robust moderate incorrect specification of the likelihood ratios.
H.2 SCAFFOLD SPLITTING CALIBRATION SET
To assess the robustness of the calibration procedure, we perform scaffold split the calibration negatives. We select the top 30 scaffolds that also appear in the test set, producing a nontrivial distribution shift that reflects realistic drug discovery scenarios where scaffold diversity plays an important role. This setting provides a diagnostic test: if the calibration procedure effective, the resulting p-values this shifted split should remain close uniform.
36

---

Published as a conference paper at ICLR 2026
QLD Pusaus Distributiion by Scatfold Spitt
DR02 - Proauus destributiion by Scarfald Spltt
S-nalime
Cole
(a) QED Hgraph: Calibration significantly reduces deviation of p-value distribution from uniformity.
(b) DRD2 Hgraph: Calibrated p-values closely match the uniform distribution.
P-value distributions on scaffold-spli calibration negatives for QED (left) and DRD2 (right). In both cases, calibration produces p-values that closely follow the uniform distribution, indicating that the density correction remains reliable under scaffold-hasd distribution shift.
Figure 19:
Figures 19a and 19b show the p-value distributions before and after applying the density ratio correction. In both QED and DRD2, the uncalibrated p-values deviate substantially from the ideal uniform distribution, as indicated by large KL divergences. After calibration, the p-values become significantly closer to uniform, with much lower KI. divergences and visibly more homogeneous density across the interval.
The close match between the calibrated p-values and the uniform reference suggests that the cali- bration procedure successfully corrects for distributional differences tied to scaffold variation. This indicates that the method is likely to remain well calibrated the test set, even when the chemical space differs from the training distribution. In practical applications, having partial knowledge of the test set's chemical space enables simple sanity check: if the p-values on the corresponding scaffolds appear close to uniform, one can be confident that the method will behave reliably on the actual test compounds.
H.3 BALANCE CHECKS
Finally, To diagnose the effect of density correction, compare the feature means of the calibration set with the test set before and after reweighting as discussed in Section 3.3. Figures 20 and 21 show the mean and standard deviation of each feature dimension, where post-calibration test features weighted by the likelihood ratio Poal (2)/ (tte)/Pos))))
For QED (Figure 20), the pre-calibration test features differ substantially from the calibration distribution, especially in the first dimension. After applying density correction, the weighted test features align much more closely with the calibration means across dimensions. The cosine distance between the mean feature vectors decreases from 0.0623 to 0.0105, indicating strong improvement distributional matching.
A similar trend appears for DRD2 (Figure 21), where large discrepancies in several dimensions are substantially reduced after reweighting. The cosine distance drops from 0.0491 to 0.0075, confirming that the corrected test distribution becomes much closer to the calibration distribution.
These balance checks provide a practical diagnostic: given an unlabeled test set, one can evaluate whether density correction brings its feature distribution closer to the calibration set, indicating that the calibration procedure likely to remain valid.
37

---

Published as a conference paper at ICLR 2026
QED Feature means
bss (he Calbration)
Calbration
Tedt (hoe-Calbraten
+827
4.41
-.12
0.66
4.34
Carine (KGEEEnG) Roadin Hott EdLOS
Figure 20: Feature means for QED using the Hgraph model at N 5. Density correction aligns the test features more closely with the calibration distribution.
DRD2 Feature means
Be (Pe CellCeibrntoon
Calbratin
les. th abar)
3,07
0.29
-2,40
4.22
Codhe B Beteese
witn Maad-Cads
Figure 21: Feature means for DRD2 using the Hgraph model at N Density correction reduces the discrepancy between calibration and test distributions.
38


Reviews:
[{"review_id": "iUYp1Vc33G", "reviewer": "Reviewer_VPnq", "summary": "The paper introduces CONFHIT, a framework that brings formal reliability guarantees to conditional generative design under tight experimental budgets. Instead of assuming an oracle that can instantly validate new samples, the method leverages historical labeled data and corrects distribution shift between past and newly generated candidates through density ratio weighting. It builds joint conformal p values over batches to certify that a generated set contains at least one viable hit and extends this to a nested testing strategy that prunes a large batch to a compact shortlist while retaining finite sample error control. The authors ground the approach with theory and evaluate it on constrained molecule optimization and structure based drug discovery, showing tight error control across target levels and models, practical reductions in certified set size, and sensible behavior in ablations that remove density correction.", "questions": "-", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The problem is well framed around the realities of discovery work where assay budgets are limited and oracle access is infeasible, and the split between certification and design clarifies when guarantees are possible and how to act when they are. The use of weighted exchangeability to address covariate shift is principled and connects cleanly to modern conformal theory, and the nested testing idea is simple to implement yet surprisingly powerful, needing only monotone individually valid p values to avoid multiplicity penalties. The empirical study spans multiple model classes and tasks and reports both coverage and power with clear plots, while the ablation on removing density correction provides convincing evidence that the weighting is not cosmetic. The paper is mostly model agnostic in how it defines conformity scores and offers several reasonable choices, which improves the portability of the framework. The budget allocation vignette is a useful bridge from statistics to program level decision making.", "weaknesses": "The guarantees depend critically on accurate density ratio estimates built on learned features, and the robustness analysis, while helpful, still leaves open how to diagnose and mitigate misspecification in practice when calibration data are scarce or shift is large. The approach relies on a property predictor to define the conformity score, so end to end performance will be sensitive to that model and feature extractor, yet guidance on model selection, calibration, and failure modes is limited. Comparisons to prior conformal generative methods are discussed as not directly applicable because of oracle assumptions, but the experimental section would be stronger with alternative practical baselines beyond Bonferroni, for example heuristic pruning driven by predictor uncertainty, or variants that reuse unlabeled generated data for semi supervised calibration. Computational cost may be nontrivial due to permutation sampling and kernel density estimation, and the paper does not provide a thorough accounting of wall clock cost versus gains in certified set size. The empirical scope focuses on small molecule scenarios with in silico oracles rather than wet lab measurements, so external validity to other domains such as proteins or materials remains uncertain. Finally, some design choices are only briefly justified, such as training the density model on a subset and the specific feature layers used, and readers may benefit from clearer guidance on hyperparameters like the number of permutations, batch budgets, and how to pick among score statistics without post hoc tuning. \n\n\n---\n\n\nItemized weaknesses for rebuttal and discussion:\n\n1. Heavy reliance on accurate density ratio estimates with limited guidance for diagnosing misspecification\n2. Sensitivity to the property predictor and feature extractor with sparse advice on model selection and calibration\n3. Limited baselines beyond Bonferroni; lacks comparisons to stronger practical heuristics or semi supervised variants\n4. Computational overhead from permutation testing and density modeling without clear accounting of wall clock cost\n5. Evaluation focused on in silico small molecule settings, leaving external validity to other domains uncertain\n6. Several design and hyperparameter choices are under explained, reducing reproducibility and practitioner guidance", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GgvM6RKTQX", "reviewer": "Reviewer_SYBt", "summary": "The article describes variant of conformal prediction for guaranteeing that a batch of samples generated from a generative model satisfies a predefined property. The use case in mind is molecular generation, where such a predefined property could be a valid base-scaffold or a certain amount of improvement over an existing seed molecule.", "questions": "- ConfHit is based on a permutation test, where permutations are subsampled to tame the computational effort. However, the subsampling introduces variance that only decreases with 1/B. Hence estimation of p-values might be difficult, in particular if these are small and might require large B and significant computational effort. Have the authors experienced issues along these lines? In order words, would one be able to obtain similar results if computational time is reduced from 30min (App. C3) to 5min.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The article is overall well-written and the concepts are presented in a clear and coherent manner. Numerical examples indicate that the method can indeed be applied to molecular generation problems.", "weaknesses": "- The core contribution is rather incremental from a statistical/mathematical perspective. The ConfHit algorithm (Alg. 1) is a variant of a permutation test that has been known in the literature for a long time. The idea of incorporating nested testing is a relatively straightforward addition (and I would be surprised if similar techniques have not been used before, maybe in a slightly different context).\n\n- The significance of the numerical results are difficult to interpret (I am also not an expert in molecular generation). However, baseline comparisions against earlier methods are omitted with the argument that a completely new set-up is studied, where there is a distribution shift between the calibration data and the distribution of the generative model. Nonetheless, in the absence of such distribution shift a comparison to the relevant baselines Quach et al., 2023, and Kladny et al., 2024 seems doable and provides an indication of the size and the quality of the generated set.\n\n- The method requires the estimation of a covariate shift w between the distribution used for calibration and the distribution of the generated samples. It might be difficult to reliably estimate w in practice.\n\n- The latex-style file has been modified to ommit the label Figure 5, possibly to deal with page restrictions. I recommend the authors not to do this in future submissions, as it is an unfair practice and seems to be against the submission rules.\n\n- The guarantees hold marginal and not hold conditional on the calibration data. This could be emphasized and explained more as it might otherwise mislead and misguide the interpretation of the statistical guarantee. Moreover, for getting a guarantee conditional on the calibration data, one would potentially need a large calibration dataset, exacerbating the computational challenge (permutation testing) of the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zKIqKUKISS", "reviewer": "Reviewer_PH3i", "summary": "CONFHIT is a wrapper you put around any generator so that a small shortlist of molecules comes with a statistical promise: with confidence , at least one on this list is a hit. It does this without calling an oracle and while correcting for distribution shift between old data  new generations made from the generator.", "questions": "1. How robust is coverage when the density-ratio w(x) is misspecified—can you bound degradation or provide diagnostics a practitioner can run before trusting certification?\n\n2. Can you offer principled guidance for choosing N and \\alpha (or adaptive rules) that minimize empty-set rates while preserving guarantees?\n\n3. How sensitive is CONFHIT to mild leakage or correlated training (e.g., feature reuse) between mu hat and calibration/test data, and can cross-fitting or data-splitting strategies be recommended to preserve coverage without sacrificing too much power?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Main strength is that this paper tries to tackle one of the critical problems in drug discovery. The fact that it is model-agnostic and no oracle is required is another strength. This makes the method actually usable in resource-constrained settings. THe Nested Testing Framework seems useful because it doesn't just certify N samples but returns the smallest certified subset", "weaknesses": "1. Coverage hinges on density-ratio correction (and the covariate-shift assumption). \n\n2. May return an empty shortlist under tight α or small N. The paper notes that empty sets are inevitable without strong assumptions when the generation budget is limited, and empirically Bonferroni is almost always empty while CONFHIT still has ~16% empty sets in SBDD—improving on baselines but still a practical failure mode.\n\n3. Needs an independently trained scoring model; power hinges on that predictor. The method seems to be only valid if the conformity/score function V is trained independently of the calibration/test data", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Xu48xH92FL", "reviewer": "Reviewer_ZnoF", "summary": "The authors present ConfHit, a conformal prediction technique that can be used for two applications:\n\n(i) Certification: Given a set of examples, assert whether at least one example fullfills some property (called *hit*), with the guarantee that incorrectly detecting a hit comes at probability $\\leq \\alpha$.\n\n(ii) Design: Generate a minimal set of examples using a generative model, such that there exists at least one hit, with probability $\\geq 1- \\alpha$ (built on top of the method for (i)).\n\nImportantly, and in contrast to prior work, ConfHit does not assume access to an oracle for performing calibration, but only assumes access to a set of examples without a hit.\n\nThe method is evaluated on constrained molecule optimisation and structure-based drug discovery and performs favorably in comparison to naive baselines.", "questions": "* l.10: *\"Recent advances extend CP to provide guarantees in language generation (Angelopoulos et al., 2021; Quach et al., 2023)\"* I believe that [1, 2] should also be cited here.\n\n* l.169: That means we discard all data that fulfills the property of interest? I wonder whether the statistical efficiency of the method could be improved by incorporating these examples somehow.\n\n* Algorithm 1: Maybe I missed it, but how large should we choose the \"original\" set $\\lbrace X_{n+j} \\rbrace_{j=1}^N$? Obviously, if $N$ is too small, the set $\\lbrace k \\in [N]: p_k \\leq \\alpha \\rbrace$ will be empty. What do we do in this case? Reject returning a set? I believe that this corner case is missing in the algorithm.\n\n[1] Kladny, Klaus-Rudolf, Bernhard Schölkopf, and Michael Muehlebach. \"Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering.\" International Conference on Learning Representations (2025).\n\n[2] Shahrokhi, Hooman, et al. \"Conformal prediction sets for deep generative models via reduction to conformal regression.\" Uncertainty in Artificial Intelligence (2025).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 4, "strengths": "Overall, I believe that this paper is strong and should be accepted.\n\n1. The writing is phenomenal. Everything is clear, the notation is wisely chosen. Also, Figure 1 is excellent.\n\n2. The motivation is well-justified. The authors tackle a relevant limitation of existing approaches.\n\n3. Rigorous theoretical analysis is present.\n\n4. The method is relatively simple.\n\n5. Strong experimental evaluation.", "weaknesses": "A great concern I have is the fact that ConfHit crucially hinges on a perfect estimate of the density ratio $w$. While Theorem 3.4 nicely quantifies the effect of estimation error of $w$ on the guarantee, the corrected bound is obviously not computable in practice. The authors motivate their technique via the need for guarantees for erroneous generative models. However, the technique itself relies on a erroneous density estimate, for which the guarantee does no longer hold. Thus, it seems that ConfHit shifts the problem from one ML model to another ML model, but it does not actually solve the problem. The authors write that they use kernel density estimation on top of a neural network representation to estimate $w$. Is there anything that suggests that trusting this density ratio estimate is more reasonable than simply trusting the outputs of the generative model? Density estimation is a harder problem than generative modeling (especially in high dimensions), so I have doubts.\n\nThe authors briefly discuss this point in their limitations paragraph of the conclusion, but I think it would be important to stress this crucial issue more (ideally, even in the manuscript's abstract) and go into why estimating $w$ should be more reliable than estimating the generative model. I am willing to increase my score if the authors convincingly address this point.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)