Integral Imprecise Probability Metrics
Authors: Siu Lun Chau
Michele Caprio
Krikamol Muandet
Venue: NeurIPS 2025 poster ()
Keywords: Imprecise Probability
Integral Probability Metric
Uncertainty Quantification
Primary area: probabilistic_methods
TL;DR: We propose a novel choquet integral-based framework called Integral Imprecise Probability Metrics to quantify differences between imprecise probabilities.

Abstract: Quantifying differences between probability distributions is fundamental to statistics and machine learning, primarily for comparing statistical uncertainty. In contrast, epistemic uncertainty---due to incomplete knowledge---requires richer representations than those offered by classical probability. Imprecise probability (IP) theory offers such models, capturing ambiguity and partial belief. This has driven growing interest in imprecise probabilistic machine learning (IPML), where inference and decision-making rely on broader uncertainty models---highlighting the need for metrics beyond classical probability. This work introduces the Integral Imprecise Probability Metric (IIPM) framework, a Choquet integral-based generalisation of classical Integral Probability Metric to the setting of capacities---a broad class of IP models encompassing many existing ones, including lower probabilities, probability intervals, belief functions, and more. Theoretically, we establish conditions under which IIPM serves as a valid metric and metrises a form of weak convergence of capacities. Practically, IIPM not only enables comparison across different IP models but also supports the quantification of epistemic uncertainty~(EU) within a single IP model. In particular, by comparing an IP model with its conjugate, IIPM gives rise to a new class of epistemic uncertainty measures---Maximum Mean Imprecision (MMI) ---which satisfy key axiomatic properties proposed in the Uncertainty Quantification literature. We validate MMI through selective classification experiments, demonstrating strong empirical performance against established EU measures, and outperforming them when classical methods struggle to scale to a large number of classes. Our work advances both theory and practice in Imprecise Probabilistic Machine Learning, offering a principled framework for comparing and quantifying epistemic uncertainty under imprecision.

Integral Imprecise Probability Metrics
Siu Lun Chau
Michele Caprio University of Manchester United Kingdom
Krikamol Muandet RI Lab, CISPA Saarbrüccen,, Germany
Nanyang Technological University Singapore
Abstract
Quantifying differences between probability distributions is fundamental to statis- ties and machine learning, primarily for comparing statistical uncertainty. contrast, epistemic uncertainty-duut due to incomplete representations than those offered by classical probability. Imprecise probability (IP) theory offers such models, capturing ambiguity and partial belief. This has driven growing interest in imprecise probabilistic machine learning (IPML). where inference and decision-making rely on broader uncertainty Noddels-highlightine the need for metrics beyond classical probability. This work introduces the Inte- gral Imprecise Probability Metric (IIPM) framework, Choquet integral-based generalisation of classical Integral Probability Metrics (IPMs) to the setting of capacities-a broad class of IP models encompassing many existing ones, including lower probabilities, probability intervals, belief functions, and more. Theoretically, we establish conditions under which IIPM serves valid metric and metrises a form of weak convergence of capacities. Practically, IIPM not only enables com- parison across different IP models but also supports the quantification of epistemic uncertainty (EU) within single IP model. In particular, by comparing an IP model with its conjugate, IIPM gives rise to a new class of EU measures- Lessuure--Mximmmn Mean Imprecisions (MMIs) (MMis)-which satisfy key axiomatic properties proposed in the uncertainty quantification literature. We validate MMI through selective classification experiments, demonstrating strong empirical performance against established EU measures, and outperforming them when classical methods struggle to scale to large number of classes. Our work advances both theory and practice in IPML, offering a principled framework for comparing and quantifying epistemic uncertainty under imprecision.
richer
1 Introduction
Inductive reasoning underpins artificial intelligence, sciences, and human judgment by enabling generalisation from observed evidence unobserved phenomena. This requires navigating effectively the boundary between certain and uncertain information. Measure-theoretie probability theory, formalised by Kolmogorov [1], provides the mathematical foundation widely adopted represent and manage such uncertainties. In machine learning (ML), its use is pervasive-from modelling data-generating processes guiding predictions and decision-making. Probability theory primarily addresses uncertainties described as 'risk', "first-order uncertainty', 'statistical uncertainty', or 'known unknowns' [2], commonly referred as aleatoric uncertainty in ML [3, 4].
A growing focus in ML concerns another form of uncertainty arising from ignorance or lack of knowledge- termed 'Knightian uncertainty' [5], "second-order uncertainty', 'unknown unknowns', commonly known as epistemic uncertainty (EU). While probability theory adequately models aleatoric uncertainty, scholars [6-8] argue that classical precise probability, relying single probability measure, falls short in representing EU. It struggles to formally model missing, uncertain, qualitative data [9, Section 1.3] and fails to distinguish between indifference (equal belief) and
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

genuine ignorance (absence of knowledge) [8, Section 1.1.4]; see also Hajek's [10] discussion on Laplace's and Bertrand's paradox [11]. At a more fundamental level, when probability is used to encode subjective belief or confidence, the additivity axiom of Kolmogorov implies that high uncertainty (low confidence) about an event necessarily entails high certainty (high confidence) about its complement. Yer under limited information, this may not hold-we we might reasonably remain ambiguous about both. These limitations contribute to practical issues in ML, including overconfidence [12], poor generalisation [13, 14), and inadequate handling of partial ignorance [15].
See Manchingal and Cuzzolin [16] for a recent discussion on the need to equip machine learning systems with epistemic uncertainty awareness.
Encertainty-centrre ML paradigms have emerged address the limitations of classical approaches by explicitly modelling EU. Examples include Bayesian ML [17] and evidential learning [18] -each grounded in distinct philosophies with respective strengths and limitations. ternative, known as Imprecise Probabilistic Machine Learning (IPML), focuses on inference and decision-making using generalisations of classical probability, commonly known as imprecise proba- bilities (IPs) [8. 19, 20J. Examples of IP models include interval probabilities [6], random sets [21], fuzzy measures [22], belief functions [23], higher-order probabilities [24, 25J, possibility theory [26], credal sets 127, 28], and lower/upper probabilities [8, 29]. Many of these fall under the broader class of non-additive set functions called Choquet capacities [30]; see Definition 2 and Figure in Appendix A for their generality. IPML provides principled tools to handle imprecision arising from model misspecification and data uncertainty, with growing applications in classification [31- 33]. hypothesis testing [34, 35], scoring rules [36, 37], conformal prediction [38, 391, computer vision (40, 41 probabilistic programming [42], explainability [43-46), neural networks [31, 47, 48), learning theory [49], causal inference [50, 51], active and continual learning [52, 53], fixed point theory [54], and many more.
growing al-
Motivated by these advances, this work contributes to the core development of IPML by introducing a first-of-itskkndd family of statistical distances for imprecise probabilities, termed Integral Imprecise Probability Metric (IIPM). IIPMs strictly generalise the classical Integral Probability Metric (IPM)
framework [55] by leveraging Choquet integration [30] to compare capacities, thereby recovering IPMs as special case. With appropriate choices of function classes, IIPMs extend well-known metrics-sut as the Dudley metric and total variation distance- to the setting of IPs. Theoretically, establish conditions under which IIPMs serve as valid metrics and metrise the weak convergence of capacities. As an illustration, we show how IIPMs recover key theoretical results in a recent optimal transport under c-contamination [56] problem. Beyond these theoretical contributions, IIPMs naturally induce a new class of EU measures by quantifying the discrepancy between a capacity and its conjugate (see Section 2.1). This yields the Maximum Mean Imprecision (MMI), a family of epistemic uncertainty quantification (UQ) measures that satisfy several foundational axioms from the literature, along with an efficiently computable upper bound. We empirically validate MMI and its linear- time upper bound in selective classification tasks [57], demonstrating strong empirical performance and improved scalability over existing epistemic UQ measures particularly in settings with large number of classes, where conventional methods often fail.
The paper is structured as follows. Section 2 reviews IPMs and IP, followed by IIPM and MMI in Section 3 and 4, respectively. Section 5 discusses related work, Section 6 presents empirical results, and Section 7 concludes with discussion. All proofs and derivations can be found in Appendix B.
2 Background
Notations. Let (X, Ex) be our measurable space, where X is a non-empty set and Σx a o-algebra. We denote Ca(X') as the space of continuous bounded real-valued measurable functions on X. We denote (2x) the space of capacities on (X, Ex) (see Definition 2), while P(X), P(X) represent the spaces of lower and upper probability measures P. P on (X, Σx) (see Definition 3) respectively, and P()') the space of probability measures P on (x,Ex)
Integral Probability Metric. Quantifying the difference between two probability measures P, Q E P(X)) is a ubiquitous task in statistics and machine learning. Popular classes of divergence include the d--ivergence [58, 59], Bregman divergence [60, 61], and a-divergence [62]. Another widely used family of divergence measures, central to our paper, is the Integral Probability Metric (IPM) [55, 63). also known probability metrics with a Ç-structure [64]. Given set of continuous bounded
2

---

real-valued measurable functions F _ Co(2), and probability measures P. E P(X), the IPM associated to JF is defined as
(/ fdp jap-Jsaaat
IPMx = sup [EF
where f SdP is the Lebesgue integral of f with respect to P. With suitable choices of F, we recover popular probability distances, including the Dudley metric [65], Kantorovich metric and Wasserstein distance [66], total variation distance and Kolmogorov distance [1], and kernel distance commonly known maximum mean discrepency (MMD) [67]. IPMs have also found numerous theoretical apppicatioossaappariinn in proof's of central limit theorems [68], empirical process theory [69], and the metrisation of weak topology on P(X) [70, Chapter -1]-as as well practical applications in hypothesis testing [71 and generative model training [72].
2.1 Beyond Precise Probability and Expectations: Capacities and Choquet Integration
While powerful, the IPM framework is limited to probability measures. We extend to capacities broad class encompassing many IP models. This section introduces key concepts: probabilities, capacities, lower probabilities, and Choquet integration.
Probabilities. We include here the formal definition of probability measure for completeness,
Definition 1 (Kolmogorov [1]) A probability measure P E P(X) measurable space (X' Ex) is a set function P: Exx + [0, 1] such that (i) P(A) 2 0 for any A E Ex (ii) P(x) 1, and (iii) for any sequence of disjoint sets (Ai)221 with each A; E Ex, P(Us21A,) Eiz1 P(As).
We adopt countable additivity for generality, though some scholars prefer finite additivity. Kol- mogorov's probability theory elegantly applies measure theory to the measurable space (X' Ex), where the additive measure P assigns credence (degrees of belief) events in Σx.
Capacities. Nonetheless, various scholars have argued that the additivity axiom is too restrictive when dealing with uncertainty involving partial ignorance and ambiguity [73]. This leads to an interest in studying more general non-additive measures, starting with capacities:
V(X) on measurable space Ex) is set function v Ex + J0, 1) such that (i) v(0) = 0, v(x)= 1, (ii any A, B E Ex with A C B, ((A) < BB), and (ii) v is continuous from above and below.
Definition 2 (Choquet [30]), A capacity
The last condition can be omitted when working with finite spaces X' [74, Section 6.8]. Furthermore, say capacity is convex, or 2-monotonic, if for any A, B Σx. v(A U B) + v(An B) 2 u(A) + (B). Probability measures P are always capacities (and 2-monotonic), but not vice versa. 2-monotonicity equips capacities with more structure make them computationally easier to work with (see e.g. Lemma 5), while still including as particular cases many of the IP models, such as e-contamination models [75]. belief functions [76], and probability intervals [6].
Lower Probabilities. Given capacity 1 I-v(A), for all A E Ex. The conjugate capacity serves as a dual representation of the original capacity, meaning one fully determines the other. We clarify its significance and interpretation after introducing lower probability, which lies between capacities and probability measures in generality.
define its conjugate V² Ex + 10, 1J. as v"
Definition 3. [Cerreia- Vioglio et al. 77, Section 2. 2.i.v A set function P. Σx - [0, 1] is a lower probability if there exists a compact set C C P(X) of probability measures such that P(A) minpec (AA) for all A E Exx
The compact set C of measures is called the credal set and the compactness here has be understood in the weak* topology [77, P.2 Remark 1]. Lower probabilities are always capacities but not vice versa [77, Section 2. The conjugate of a lower probability P. is the upper probability P. defined by P(A) = - (())) = maxpec P(A) for E Σx. In IPML, lower probabilities are typically constructed from set of probabilistic predictors, e.g.. C= = (Y X )) m [78-80, 48), or via basic probability assignments [81], as in Dempster-Shan Shafer theory [76), or by constructing predictive
intervals [82, 83].
3

---

Lower and upper probabilities offer two complementary ways of expressing credence in an event's s truthfulness. Specifically,, the lower probability P(4) reflects the minimal credence assigned directly to event A, while the upper probability P(4) captures the maximal credence, computed as one minus the lower probability of the complement, i.e., P(44). These represent, respectively, pessimistic and optimistic assessments of A's objective likelihood. For subjectivist interpretation of lower and upper probabilities grounded in de Finetti's s framework of probability coherent betting rates, see Walley [8, Section 2.3.5]. classical probability, these bounds coincide, since P(A) - P(A9) for any event A E Σx. In other words, P is self-conjugate.
Choquet Integration. Another concept central to our contribution is the use of Choquet integrals to measure the discrepancies between capacities, akin to how Lebesgue integrals play a key role for IPM. Choquet integration [30] generalises Lebesgue integration for probability measures to capacities: Definition 4 (Choquet Integral). Given a capacity v E ((2) and real-valued function on X, the Choquet integral of f with respect to is
f <Vlff* 2fddt 1 rlf 2jdt
(1)
provided the difference on the right-hand side is well defined, where f+= max(0, -min(0, f), and [ 2 1)== {x & f(x) 2 1).
==x(0 f), f
When both terms on the right-hand side of Equation (1) are finite, then we say f Choquet integrable with respect to v. When v is a probability measure, the Choquet integral becomes the standard Lebesgue integral. If is bounded, then by Troffaes and De Cooman [19, Proposition C.3.ii), is Choquet integrable with respect to and the integral can be written as,
f fdv= f
1" v(U 2 2)) dt,
where /= infzeæ (xx and := SUPzEX (xx. The Choquet integral provides a generalised notion of "expectation" that accommodates non-additive uncertainty assignments, commonly encountered in frameworks such as imprecise probabilities, belief functions, and fuzzy measures. For example, the Choquet integral of a lower probability provides a lower bound to the worst-case expectation across the entire credal set, as demonstrated in the following Lemma.
Lemma 5. For lower probability P. associated to credal set C, we have f fdP S inf S fdP for any f C.()). When P is 2- 2-monotonic, the inequality becomes equality.
3 The Integral Imprecise Probability Metric (UPM) framework
The motivation of [PM follows from a classical result [70, Lemma 9.3.2], stating that two probability measures are equal if and only if their Lehesgue integrals agree for all f E Cz()). Before introducing IIPM, we examine whether an analogous result holds for capacities and Choquet integrals. We follow the conditions Dudley [70, Lemma 9.3.2] and examine metric spaces (X, (X,d) for some metric d. Theorem 6. Let (X, d) be a metric space. For any capacities V, p E ((X), we have fdv =f fd for all f E (x)) if and only if y μ.
Narukawa [84, Proposition 7] presents similar result but for bounded non-negative continuous functions under stricter conditions on the capacities (regularity) and different conditions on X' (locally compact Hausdorff). While Theorem 6 characterises equality of capacities via Choquet integrals, the full class (2') often too rich for practical use. Following the spirit of IPMs, we define the Integral Imprecise Probability Metric with respect to function class F Cx()) follows.
Definition 7 (Integral Imprecise Probability Metric (IIPM).. For function class F C C,(X) and capacities v. μ E (X)), the integral imprecise probability metric associated with F between v, μ
{ff fdv fdp feF
TIPM)(v,
=sup
As working with bounded real-valued functions, the Choquet integrals are well-defined.

---

Corollary 8. For any P.Q P(X) and F S CC()), IIPMs(P.Q) -PM,(PPQ))
This illustrates that IIPM generalises IPM, but not vice versa, since Lebesgue integrals are defined for probability measures, not capacities. We verify IIPM, serves as a pseudometric on ((2)).
Proposition 9. For any F S. Ca(2), IIPM a pseudometric on V(X); it is non-negative, symmetric, and satisfies the triangle inequality.
When F rich enough for IIPMF to be point-separating-foo instance, when F= Cihe) IPMM defines proper metric on F(X). More desirably, albeit stronger, is for convergence in IPM to imply and necessitate convergence in another established sense, i.e., II, v) + should ideally entail that v/ "truly" converges to μr, where "truly" refers convergence under some other notion of interest independent to the choice of 11 If this implication holds, then 1 is said to metrise that notion of convergence important for analysing the reliability of inference procedures that depend on them [85]. In our case, we follow Feng and Nguyen [86] and consider the Choquet weak convergence of capacities: a sequence V, E V(X) is said to converge to V in this sense i fdv, +f f for all f E Ca()). This leads to the following natural question: What conditions F are sufficient for IIPM, to metrise Choquet weak convergence on V(X)? Our initial analysis gives the following answer:
(()). Studying the metrisation properties of divergences
Theorem 10. Let F S (x) be dense in (x) with respect to the || ol norm. Then, IIPM, metrises the Choquet weak comergencee of (X²).
To build intuition for Theorem 10, recall from Theorem 6 that C& (x) is point-separatinn on ((X). If F is dense in (x( under the || Dc norm, then this allows us to "import" the separating ability of (L) to F. Our analysis follows from Gretton et al. [71, Theorem 18] for IPMs. However, more general conditions on F (and X) for metrising weak convergence in P(X)) have been studied see, e.g., Sriperumbudur et al. [87], Sriperumbudur [88]. Extending such analysis to the case of capacities and understanding the precise conditions under which F and X' ensure metrisation of Choquet weak convergence on (x) is definitely an interesting direction for future work.
3.1 Example use cases of the IIPM framework
While general, capacities are often too complex for practical use without additional structure. We therefore focus on lower probabilities (cf. Definition 3) for the remainder of the paper. Nonetheless, our framework also applies to other IP models such as probability intervals, belief functions, and possibility measures all of which are structured subclasses of capacities. We show how the IIPM framework gives rise to new divergences and can be used to prove existing results in IPMI. differently.
The Lower Dudley Metric. We start with a lower probability version of the Dudley metric.
Definition 11 (Lower Dudley Metric). For metric space (X, d). let Fa == ( E C,(X): |l S 1}. where the Lipschitz norm || := olo + Ilf l| and Iffll == sup((/(ee- x y E X}, we define IIPMf, as the Lower Dudley metric.
Proposition 12. The Lower Dudley metric metrises Choquet weak convergence on P(x').
This result immediately follows from Theorem 10, and showcases that the lower Dudley metric can be applied in settings where analysing weak convergence of conservative uncertainty assessment is essential, e.g., in robust statistics and safety-critical applications.
Lower Total Variation. We can define the lower total variation distance as follows,
Definition 13 (Lower Total Variation Distances). Let P, Q (P()) be two lower probabilities on (X, Ex) and Frv == (1A: : A E Ex}, then we define IIPM Fry (P.Q) sup [(AA)-((A)) as the Lower Total Variation Distances (LTV) between P.Q.
This equality follows from Troffaes and De Cooman [19, Proposition C.5(i)).. In fact, the LTV distance was also considered in Levin and Peres [89, Proposition 4.2] but was not derived from any integral-based metric framework. is also important to highlight that many results that hold for IPMs do not carry over to IIPMs due to the loss of additivity. Specifically, while expectations under IPMs are linear, expectations under IyPMs-typicall modelled using the Choquet integral-are generally non-linear and, at best, super or sublinear [90]. The following is one such example:
5

---

Remark 14. Let X' be finite. For any P.Q E P(X), we have IPM (P.Q) - sup IP(A)
Q(A)] = 1/ Exex (P((x)) Q((z))) In contrast, in the imprecise case. there exists P, Q E P(x)
such that IIPMfry (P.Q) == supaeEx P((A) Q(A) * Eosx (P((z))- -((x)))
In subsequent applications of IIPM to epistemic UQ, we adopt the LTV distance for its simplicity.
Lower Kantorovich problem with e-contamination sets. To illustrate the theoretical utility of IIPM, we consider the well-known e-contamination model from robust statistics [75]. We show how IIPM rederives a key result from Caprio [56, Theorem 11], namely that the solution to the restricted lower probability Kantorovich problem (RLPK) [56, Definition 10] coincides with the classical Kantorovich problem under e-contamination. For completeness, a detailed treatment of the RLPK problem is provided in Appendix D.1. Recall the e-contamination set Pe, of probability measure & ((2) is given by,
P.. = (P P(X) 3R E P(X) S.t. P(A) (1 (1-c) +c((A) Ex).
(2)
with E [0, 1] the contamination rate of the model. The lower probability corresponding to Pe, is derived in Walley [8, Section 2.9.2].
Lemma 15. Let P..p be an e-contaminated model defined in Equation (2). Then, the associated lower probability P. is given by
[ (1 c)(()) for all A E Ex for A X
P.(A)
inf P(A) PEP.,
1,
Now we can recover the results from Caprio [56] using the IIPM framework.
Theorem 16. Let Fw {f Ca(X): : llflle S 1} where llfll sep (11())-F(olfetaaag and e the transportation cost in a restricted lower probability Kantorovich (RLPK) problem [56, Definition 10]. Let P.. Q be lower probabilities of the c-contaminated models Pe, and Pe. Then, 11PM (P.Q) Q) coincides with the objective of the RLPK problem, and thus coincides with the classical Kantorovich' 's optimal transport problem involving P and Q.
We include this result illustrate the utility and flexibility of the IIPM framework for theoretical anal- ysis. We also highlight that, in general, a version of the Kantorovich- Ruiiiihh--nuintteiinnn duality theorem for RLPK does not (yet) exist, therefore, we cannot simply associate 11P as the lower Wasserstein-1 distance. In Appendix D.2, we continue our analysis using kernels, following the derivation of kernel distances in Gretton et al. [71]. Specifically, we consider F& {f HA Il S 1}, where & is kernel and H4 the corresponding reproducing kernel Hilbert space (RKHS). Mon-parametric estimator of IIPM (P Q,) (different contamination level is allowed) is also derived. We defer this result due to space constraints.
e-contamination models with IIPMs, but
4 Measuring epistemic uncertainty with IIM:: Maximum Mean Imprecision
Beyond its appeal for theoretical analyses, the IIM framework can find practical application in epistemic UQ for ML. By quantifying the gap between capacities and their conjugates, IIPM captures the divergence between optimistic and pessimistic assessments -iiiingiinnggg rise new class of EU measures, which we call Maximum Mean Imprecision (MMI). As in Section 3, we illustrate MMI using lower probabilities, though the framework remains broadly applicable to other IP models.
Definition 17 (Maximum Mean Imprecision). Let C Cx(X). The Maximum Mean Imprecision with respect to F is the function MMI5 PP(X) + R defined as,
{ sdP sde).
MMI=(P) = IPMM((P.P) sup fEF
Note that we have omitted the absolute value sign between the Choquet integrals. This is intentional: since the upper probability P setwise-dominate the lower probability P, the difference is always non- negative, rendering the absolute value sign unnecessary. To explain MMI's underlying mechanism, we present alternative formulation of MMIF.

---

Proposition 18. The definition of MMI is equivalent to
J 1 (P(L) < <)) P(U 2 14)
MMIx(P) = sup SEF /
First, notice that for any E [f.]]. (f < uf
Equation (3) shows that MMIy
quantifies EU as the largest possible accumulation of disagreement between = P(()) and the sum of the lower probabilities assigned to the complementary events {f < <t} and {f 2 t) under P
"slide" the threshold I. As an illustrative example, we now demonstrate how to quantify the EU for the e-contamination model, previously introduced in Section 3.
Proposition 19 (MMI e-contamination set.). Lei P. be the lower probability associated with Pe,r and F [ (2)) Then MMI,
(sup,ez 2 f(x) f(y))). For the LTV distance with Frv = [1A A E Σx}. we have MMI (P.) = supAes, ( (. -.A =c.
This aligns with the intuition that contaminating a probability assessment by e amount should proportionally increase the EU by e. The "unit" of this uncertainty is determined by the variability of the chosen function class. For TV distance, the unit precisely 1. This result is useful later when we derive an upper bound for the MMI of any lower probability in Proposition 21.
Desirable axiomatic properties for epistemic UQ measure. While MMI appears to be an intuitive measure of uncertainty, is important examine whether satisfies key axiomatic properties proposed in the literature on credal UQ [91]. where the EU modelled using IPs is measured. This is especially relevant given the abundance of UQ measures [4, 92, 93], many of which lack rigorous axiomatic support. The axioms we present below were originally formulated for credal sets and are adapted here for lower probabilities, as both representations mostly convey interchangeable semantics. In particular, Abellán and Klir [94], Jiroušek and Shenoy [95], Hüllermeier et al. [96]. and Sale et al. [97] propose that a valid credal uncertainty measure P(X) + IR should satisfy:
A1 Non-negativity and boundedness: (a) U(P) 2 0, for all P E P(x);; (b) there exists u E R such that U(P)= < u, for all P P(x)..
A2 Continuity: U is continuous functional.
A3 Monotonicity: for 2.Q E R(x), P(A) S QAA) for all A E Ex, then U(P) 2 U(Q).
A4 Probability consistency: for all P P(X) we have C(P)
A5 Sub-additivity: Let (X,, Ex, and (X2, Σx,;) be measurable spaces and such that X' X2 x X2. Let (.(-) = P(, X2) and P2(-) P( be the corresponding marginal lower probabilities on 21 and X2 respectively. Then, we have U(P) S U(P,) U(22).
A6 Additivity: Following from A5, if P1 and P2 are independent, then U(P) - U(P.) + U(P.)
Note Axiom A6 assumes a notion of independence for IPs, for which multiple, non-equivalent definitions exist; see Couso et [98], Cozman [99] for a review. Axioms A1 and A2 set out the basic requirements for U to be "sensible". Axiom A3 states that if a representation P is uniformly more conservative than Q assigning lower values all events then P should be deemed more epistemically uncertain. Axiom A4 ensures that precise probabilities correspond to zero EU. Some work [100] instead prefers the measure to reflect pure aleatoric uncertainty when EU is absent. Axiom A5 captures the intuition that joint reasoning reduces uncertainty, as dependencies between events provide additional structure. Axiom A6 complements this by stating that, when no such dependencies present, joint or separate consideration should not affect the uncertainty measure. A5 and A6 are less relevant when quantifying predictive uncertainty in classification or real-valued regression tasks, since the output spaces are generally not product spaces.
At this point, one may naturally ask whether MMI, satisfies these axioms or under what conditions on F this can be ensured. Particular care is required for Axioms A5 and A6, which involve reasoning over product spaces. To meaningfully evaluate these axioms, we must clarify the function classes defined over the marginal domains X; and X2. Suppose X' = Xi x X2 as specified in axiom A5. Let Fi S (x))),, F2 S Co(2)), and F12 = {s E Cx( s.t. f( x2) = (1 + f2(22), for some fi Fi, f2 E F2).
7

---

Theorem 20. For any F S CL(2), MMIy satisfies axioms A1-A4. If P is 2-monotonic and with F F12 defined above, then MMIji, satisfies axioms AI-A5. For A5, the subadditivity becomes M S MMI,, + M, If the notion of independence in A6 is taken to be strong independence in the sense of Cozman [99], A6 also holds, and MM MMT,, + MM
Theorem 20 establishes MMI; as a principled measure of EU for lower probabilities. Ax- ioms A1-A4 hold for any function class F C Ca(2), while A5 and A6 require additional 2- iaattonniity-sttiifieed, for instance, by belief functions and suitably chosen product function spaces in multivariate settings. Examples of credal EU measures that satisfy some of the axioms include: the difference between maximal and minimal entropy within credal set [100], which fails A3; the generalised Hartley (GH) measure [94]. which satisfies all axioms; and the volume of the credal set, which fails A6. Nonetheless, Sale et al. [97] demonstrated that credal set volume fails be a good EU measure in K-class classification problems when K > 2 due to the lack of robustness in computation as the dimension K increases. We, therefore, exclude it from our experimental comparisons. We also note that, when | = 2 as in binary classification, MMIFrv corresponds to the credal epistemic measure, P([x}) - P([x}) for x E X, considered in Hüllermeier et al. [101].
linear-time upper bound. Computing MMI supAgæ (P(A) P(A)) in K-class classification is computationally expensive, as it requires evaluation over 2 subsets-akee to computing the generalised Hartley measure. To address this bottleneck, Proposition 21 introduces a practical linear-time upper bound. The key idea is to approximate any P with the closest and uniformly more conservative contamination model P.. i.e. (A) S. P(A) for all events A, ensuring no additional certainty is introduced for this approximation. Leveraging Axiom A3 in Theorem 20 and Proposition 19, we derive computationally efficient upper bound for MMI (P). relaxing computation to O(K).
Proposition 21. Let X' be finite. For any P P(X), MMIF (P) S ExEx P((x)).
Related work
Distances and Divergences for IPs. Several metrics and divergences have been proposed for credal sets [102-106], and were recently categorised by Chau et al. [34] into inclusion, equality, and intersection measures. Given its connection with lower probabilities, our IIPM belongs to the equality class. For fuzzy sets and fuzzy measures, Couso al. [107], Montes al. [108]. Specifically for lower probabilities, Hable [109] studied minimum distance estimation between a precise parametric model and lower probability model, using special case of TIPM; with bounded continuous functions over a discretised comain-thhuuee without theoretical analysis. In the context of belief functions, divergences such as those in [110-1 12] operate on the möbius transform of belief functions, while the Minkowski measure [113] compares belief functions via summation of all assessments. More recently. Xiao al. [114] introduced generalised f-divergence for belief functions. Closest in spirit to our work is Catalano and Lavenant [115], who proposed an integral- integral-based based metric for higher-order probabilities.
Uncertainty Quantification for IPs. To quantify epistemic-and and occasionally in credal-set-based models, several measures have been proposed, in- cluding maximal entropy [116], entropy difference [96], the generalised Hartley measure [94], and credal set volume [117]. In hicrarchical or second-order models such Bayesian and evidential frameworks, entropy-based measures-includee
entropy, conditional entropy, and mutual information- have been widely used to capture total, epistemic, and alcatoric uncertainty [118, 92]; see Wimmer et al. [119], Smith et al. [120] for critiques. Alternatively, Sale et al. [121, 122] advocate variance and probability distances as general-purposs EU measures for second-order models. A recent review is provided by Hoarau et al. [123].
Overall, to our knowledge, our work is the first to introduce a Choquet-integral-bnnt metric frame- work for comparing capacities-a foundational class for many uncertainty models maakkin a novel use of integral-bor based distances for quantifying credal uncertainty.
8

---

OBESITY Dalasel (K
DIGITS Datasel (K 10)
CIFAR10 Datasel 10)
CIFAR100 Dalasel 100)
1.0
140
E90
1.0
E:O JAUC 95.220.7)
E-DIT (AUC 95.5=0.2)
(AUC
(AUC 96.20001)
E06
GAUC 96.3102
MAUC 90.340.1)
NM (AUC 95.2+0.5
MMI (AUC
MMI (AUC 36.3=0.1)
E-DE (AUC 90.500))
632
MM-LLin JAUC 96.3:0.3)
MMHLI (AUG 06.3x
MMHUs (AUC 95.320.1)
MMB-L I
(AUE S1 21.5))))
Rejecion Parcantagge
Rojectin Percentiage
Rejacton Persintage
Rejection Percenlage
Figure 1: Accuracy-Rejectionn (AR) curves on four classification tasks. The area under the curve (AUC) is reported for numerical comparison. We consistently outperform entropy difference (E-Diff) and match the performance of Generalised Hartley (GH). On large-scale problems, our efficient upper bound (MMI-Lin) remains tractable and continues to outperform E-Diff.
6 Empirical validation of MMI with selective prediction experiments
This section demonstrates the practicality of the proposed MMI measure. Additional ablation stud- provided in Appendix C. The code to reproduce
ies and detailed experiment descriptions experiments here [124]. We evaluate MMI by its ability to capture informative EU in K K-class classification. As ground-trum truth uncertainty is unavailable, the informativeness of epistemic UQ is typi- cally assessed by its utility in improving prediction or decision-making [125, 126]. Following Shaker and Hüllermeier [80], Hüllermeier al. [96] while extending their binary classification experiments to the multiclass case, we evaluate our performance by plotting the accuracy-rejection (AR) curves in selective classification problems. An AR curve plots accuracy against rejection rate: model that abstains on 5% of inputs and predicts on the most certain (1 - p) p)% should show increasing accuracy with p. An informative EU measure yields a monotonic AR curve, approaching the top-left corner analoonna to ROC curves in standard classification.
Setup. As we are working with discrete label spaces, we consider Fry as our function class for MMI. Its linear-time upper bound is denoted as MMI-Lin. We compare them against two popular epistemic UQ measures: the entropy difference (E-Diff) [100] and the generalised Hartley measure (CH) [127]. To construct the credal set and corresponding lower probabilities, we utilise m = 10 probabilistic predictors trained with different hyperparameter configurations-rann random forests for tabular data and neural networks for image data Lata-and make predictions using the centroid of the credal set, equivalent to averaging over all predictors. Specifically, for an instance x and predictors {P) X = -)} the lower probability is constructed as P(A) = minje P E A X x= for every possible subset of classes, and predictions are made using my Σド= P3 X = x). While more advanced aggregation methods exist [14, 48], our aim is assess UQ measures, not optimise prediction. The lower probability here encodes the EU information we wish to recover using The. lower probability captures the epistemic uncertainty that MMI seeks to quantify. We evaluate on two UCI datasets [128, 129] and CIFAR-10/101 [130]. We perform train-test splits on the datasets and produce AR curves the latter. All experiments repeated 10 times, and the mean and standard deviation of the accuracies are plotted. The full experimental set-up is discussed in Appendix C.
Results. Figure presents AR curves for our classification problems with 7, 10, and 100 classes. The plots share the same message: MMI and MMI-Lin consistently outperform E-Diff and perform almost identical to GH, which is often considered the most informative measure for epistemic UQ [123]. This equivalence in performance may be because both measures satisfy all desirable axioms for epistemic UQ measures. Notably, Hoarau et al. [123] showed that GH and the credal epistemic measure (i.e., MMI when K 2) exhibit perfect correlation, suggesting they encode identical information. Our results extend this insight, indicating that this equivalence may hold for arbitrary K. For large K, where exact computation MMI and GH becomes intractable (0(2*), )), the upper bound MMI-Lin remains efficient (O(K), ) and continues to outperform E-Diff, offering a practical and reliable alternative to GH. These results clearly demonstrate that our proposed IIPM framework and its application as an epistemic UQ measure not only theoretically justified, but also practical.

---

7 Discussion, limitations, and future directions
We introduced Integral Imprecise Probability Metries (IIPMs) as generalisation of classical Integral Probability Metrics (IPMs) to the setting of imprecise probabilities. This framework enables the comparison of IP models through Choquet integration. We provided initial theoretical analyses and illustrated their practicality through epistemic UQ. Notably, the framework gives rise to new class of epistemic UQ measures- -trrmee Maximum Mean Imprecisions- which outperforms several popular alternatives in empirical evaluations on selective classification problems.
While this work opens new avenues for IPML research, a key limitation is the lack of sampling- based estimation method for IIPMs, unlike in classical IPMs. This arises from the absence of canonical sampling procedure for imprecise models, which capture subjective beliefss rather than event frequencies. Nonetheless, advancing this direction could, for example, enable the integration of EU into generative models based on IPMs [131, 72]. On the theoretical side, it would be valuable to connect and extend other divergence classes, such as f-divergences and Bregman divergences, to the context of capacities and more general IPs. Another open question is whether weaker conditions on the function class F or space X' can ensure that IIPMF metrises capacities. From computational perspective, developing efficient nonparametric estimation methods for IIPMs is key priority. A fur- ther application-oriented direction is to quantify credal epistemic uncertainty in regression problems using the IIPM framework, particularly light of recent proposals for uncertainty quantification axioms in regression [132].
The authors would like to thank Anurag Singh, Shireen Kudukkil Manchingal, and Fabio Cuzzolin for insightful discussions. Special thanks Masha Naslidnyk for her detailed proofreading and thoughtful feedback. The authors also gratefully acknowledge the University of Manchester and CISPA Helmholtz Center for Information Security for supporting Siu Lun's 's research visit to Manchester, without which this project would not have been possible.
Acknowledgement.
References
[1] Kolmogorov. Sulla determinazione empirica di una legge didistribuzione. Giorn Dell inst Ital Degli Arr, 1189-91, 1933.
Donald Rumsfeld. Known and unknown: a memoir. Penguin, 2011.
[3] Stephen C Hora. Aleatory and epistemic uncertainty probability elicitation with an example from hazardous waste management. Reliability Engineering & System Safety, 54(2-3):217-233, 1996.
[4] Eyke Hüllermeier and Willem Waegeman. Aleatoric and Epistemic Uncertainty in Ma- chine Learning: An Introduction to Concepts and Methods. Machine Learning, 1 110(3): 457-506, March 2021. ISSN 0885-6125, 1573-0565. doi: 10. 00.10077/10999-01--9996666 arXiv:1910.09457 [es, stat].
[5] Frank Hyneman Knight. Risk, uncertainty and profit, volume 31. Houghton Mifflin, 1921.
[6] John Maynard Keynes. A treatise on probability. 1921.
[7] David Lewis. A subjectivist's guide to objective chance. In IFS: Conditionals, Belief, Decision, Chance and Time, pages 267-297. Springer,
26rnner. 1980.
[8] Peter Walley, Statistical reasoning with imprecise probabilities, volume 42. Springer, 1991.
[9] Fabio Cuzzolin. The geometry of uncertainty; the geometry of imprecise probabilities. Springer Nature, 2020.
[10] Alan Hájek. Interpretations of Probability. In Edward N. Zalta, editor. The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, fall 2019 edition, 2019. URL https://plato: stanford, htth///raiieee/aaa111///mmaa bebbablityy-ntteppee.. interpret/.
probability-
[11] Joseph Bertrand. Calcul des probabilités. Gauthier-Ville 1889.
10

---

[12] Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang. Qiang- long Chen, Weihua Peng. Xiaocheng Feng. Bing Qin, et survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Information Systems, 43(2):1-55, 2025.
[13] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain Generalization via Invariant Feature Representation, January 2013. URL attp://arxiv org/ 11/111111.2110 2115. AEKiv:1301.2111 [cs, stat].
[14] Anurag Singh, Siu Lan Chau, Shahine Bouabid, and Krikamol Muandet. Domain generalisa- tion via imprecise learning. arXiv preprint arXiv:2404.04660 2024.
[15] Siu Lun Chau, Antonin Schrab, Arthur Gretton, Dino Sejdinovic, and Krikamol Muandet. Credal two-sample tests of epistemic uncertainty. In International Conference on Artificial Intelligence and Statistics, pages 127-135. PMLR, 2025.
[16] Shireen Kudukkil Manchingal and Fabio Cuzzolin. Position: Epistemic artificial intelligence is essential for machine learning models to know when they do not know. arXiv preprint arXiv:2505.049955 2025.
[17] Vincent Fortuin, Mohammad Emtiyaz. Khan, Mark van der Wilk, Zoubin Ghahramani, and Katharine Fisher. Rethinking the role of bayesianism the age of modern ai (dagstuhl seminar 24461). Dagstuhl Reports, 14(1 40:40-59, 2025.
[18] Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. Advances in neural information processing systems, 31, 2018.
[19] Matthias CM Troffaes and Gert De Cooman. Lower previsions. John Wiley & Sons, 2014.
[20] Thomas Augustin, Frank A. Coolen, Gert De Cooman, and Matthias C. M. Troffaes, editors. Introduction imprecise probabilities. Wiley series in probability and statistics. Wiley, Hoboken, NJ, 2014. ISBN 978-0-470-97331-..
[21] JFC Kingman. G. matheron, random sets and integral geometry. 1975.
[22] Michio Sugeno. Theory of fuzzy integrals and its applications. Doctoral Thesis, Tokyo Institute of Technology. 1974.
[23] Glenn Shafer. A mathematical theory of evidence, volume 42. Princeton university press, 1976.
[24] Jonathan Baron. Second-order probabilities and belief functions. Theory and Decision, 23(1): 25-36, 1987.
[25] Haim Gaifman. A theory of higher order probabilities. In Theoretical aspects of reasoning about knowledge, pages 275-292. Elsevier, 1986.
[26] Didier Dubois and Henri Prade. Théorie des possibilités, 1985.
[27] Isaac Levi. The enterprise of knowledge: An essay on knowledge, credal probability, and chance. MIT press, 1980.
[28] Michele Caprio and Teddy Seidenfeld. Constriction for sets of probabilities. In Enrique Miranda, Ignacio Montes, Frik Quaeghebeur, and Barbara Vantaggi, editors, Proceedings of the Thirteenth International Symposium Imprecise Probability: Theories and Applications, volume 215 of Proceedings of Machine Learning Research, pages 84-95. PMLR, 11-14 Jul 2023. URL https: httpes///occceeeeeeeee caaiings.mmr...... mlr. pptppp/ee1//eeepeeeeeeeee http///211/aaaa222/ html.
[29] Michele Caprio, Yusuf Sale, Eyke Hüllermeier, and Insup Lec. A Novel Bayes' Theorem for Upper Probabilities. In Fabio Cuzzolin and Maryam Sultana, editors, Epistemic Uncertainty in Artificial Intelligence, pages 1-12, Cham, 2024. Springer Nature Switzerland.
[30] G. Choquet. Théorie des capacités. Ann. Inst. Fourier 5 (1953/1954) 131-292., 1953.
11

---

[31] Thierry Denoeux. A neural network classifier based on dempster-shafer theory. IEEE Trans- actions on Systems, Man, and Cybernetics-Pen Part A: Systems and Humans, 30(2):31-150. 2000.
[32] Marco Zaffalon. Exact credal treatment of missing data. Journal of statistical planning and inference, 105(1):105- 122, 2002.
[33] Michele Caprio, David Stutz, Shuo Li, and Arnaud Doucet. Conformalized credal regions for 1.04852, classificatton with ambiguous ground truth. arXiv preprint arXiv:241 1.04452,, 2024.
[34] Siu Lun Chau, Antonin Schrab, Arthur Gretton, Dino Sejdinovic, and Krikamol Muandet. Credal two-sample tests of epistemic ignorance. arXiv preprint arXiv:2410. /2921, 2024.
[35] Mira Jürgens, Thomas Mortier, Eyke Hüllermeier, Viktor Bengs, and Willem Waegeman. calibration test for evaluating set-based epistemic uncertainty representations. arXiv preprint urXiv:2502. 16299, 2025.
[36] Christian Fröhlich and Robert C Williamson. Scoring rules and calibration for imprecise probabilities. arXiv preprint arXiv:2410.23001, 2024.
[37] Anurag Singh, Siu Lun Chau, and Krikamol Muandet. Truthful elicitation of imprecise forecasts. arXiv preprint arXiv:2503. 16395, 2025.
[38] David Stutz, Abhijit Guha Roy, Tatiana Matejovicova, Patricia Strachan, Ali Taylan Cemgil, and Arnaud Doucet. Conformal prediction under ambiguous ground truth. arXiv preprint (XXiv23377099000 2023.
[39] Michele Caprio, Yusuf Sale, and Eyke Hüllermeier. Conformal prediction regions are imprecise highest density regions. arXiv preprint arXiv:2502.06333 2025.
[40] Fabio Cuzzolin and Ruggero Frezza. Evidential reasoning framework for object tracking. In Telemanipulator and Telepresence Technologies VI, volume 3840, pages 13-24. SPIE, 1999.
[41] Eleonora Giunchiglia, Mihaela Cätälina Stoian, Salman Khan, Fabio Cuzzolin, and Thomas Lukasiewicz. Road-r: the autonomous driving dataset with logical requirements. Machine Learning, 3261)3326133 2023.
[42] Jack Liell-Cock and Sam Staton. Compositional imprecise probability: A solution from graded monads and markov categories. 9(POPL), 2025. doi: 10.1 10.1145/3704890. URL 1145/3704890.
[43] Siu Lun Chau, Krikamol Muandet, and Dino Sejdinovic. Explaining the uncertain: Stochastic shapley values for gaussian process models. Advances in Neural Information Processing Systems, 36:50769-50795, 2023.
[44] Lev Utkin, Andrei Konstantinov, Kirill Vishniakov, and Igor Ilin. Imprecise shap as tool for explaining the class probability distributions under limited training data. In Digital Systems and Information Technologies in the Energy Sector, pages 369-389, Springer, 2025.
[45] Majid Mohammadi, Krikamol Muandet, Ilaria Tiddi, Annette Ten Teije, and Siu Lun Chau. Exact shapley attributions in quadratic-time for fanova gaussian processes. arXiv preprint arXiv:2508. 14499, 2025.
[46] Majid Mohammadi, Krikamol Muandet, Ilaria Tiddi, Annette Ten Teije, and Siu Lun Chau. Exact shapley attributions in quadratic-time for fanova gaussian processes. AAAI, 2026.
[47] Michele Caprio, Souradeep Dutta, Kuk Jin Jang, Vivian Lin, Radoslav Ivanov, Oleg Sokolsky, and Insup Lee. Credal bayesian deep learning. arXiv e-prints, pages arXiv-2302, 2023.
[48] Kaizheng Wang. Fabio Cuzzolin, Keivan Shariatmadar, David Moens, Hans Hallez, al. Credal deep ensembles for uncertainty quantification. Advances in Neural Information Pro- cessing Systems, 37:79540-79572. 2024.
[49] Michele Caprio, Maryam Sultana, Fleni Elia, and Fabio Cuzzolin, Credal Learning Theory, February 2024. 0rXiv:2402.00957 [cs, stat].
12

---

[50] Fabio G Cozman. Credal networks. Artificial intelligence, 120(2):199 (191))199933,, 2000.
[51] Marco Zaffalon, Alessandro Antomucci, Rafael Cabañas, and David Huber. Approximat- ing counterfactual bounds while fusing observational, biased and randomised data sources. International Journal of Approximate Reasoning, 162:109023, 2023.
[52] Souradeep Dutta, Michele Caprio, Vivian Lin, Matthew Cleaveland, Kuk Jin Jang, Ivan Ruchkin, Oleg Sokolsky, and Insup Lee. Distributionally Robust Statistical Verification with Imprecise Neural Networks. PMLR (Accepted HSCC 2025), 2024.
[53] Pengyuan Lu, Michele Caprio, Eric Eaton, and Insup Lee. IBCL: Zero-shot Model Generation for Task Trade-off's in Continual Learning. arXiv preprint arXiv:2305.1478. 2024.
[54] Michele Caprio, Siu Lun Chau, and Krikamol Muandet. When do credal stabilize? fixed-point theorems for credal set updates. arXiv preprint 1rxXi:2510.004662 2025.
[55] Alfred Müller. Integral probability metrics and their generating classes of functions. Advances in applied probability. 29(2):429-443, 1997.
[56] Michele Caprio. Optimal transport for e-contaminated credal sets. PMLR (accepted to ISIPTA 2025), 2025. URL https://aaxxiv.orr//aaaaaa
[57] Mohammad Hossein Shaker and Eyke Hüllermeier. Aleatoric and Epistemic Uncer- tainty with Random Forests, January 2020. URL http://arxiv,, http:///xxxivv.r//aaaa/ org/ooa///////e// http/abaa200..... 00893. arXiv:2001.0089) [cs, stat].
[58] Syed Mumtaz Ali and Samuel D Silvey. A general class of coefficients of divergence of one distribution from another. Journal of the Royal Statistical Society: Series B (Methodological), 28(1):131-142, 1966.
[59] Imre Csiszár. On information-type measure of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar., 2:299-318. 1967.
[60] Lev M Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR computational mathematics and mathematical physics, 7(3):200-22) 1967.
[61] Arindam Banerjee, Srujana Merugu, Inderjit S Dhillon, and Joydeep Ghosh. Clustering with bregman divergences. Journal of machine learning research, 6(Oct): 1705-1997 2005.
[62] Alfréd Rényi. On measures of entropy and information. In Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, volume I: contributions to the theory of statisties, volume 4, pages 547-562. University of California Press, 1961.
[63] Bharath K Sriperumbudur, Kenji Fukumizu, Arthur Gretton, Bernhard Schölkopf, and Gert RG Lanckriet. On the empirical estimation of integral probability metries. 2012.
[64] Vladimir Mikhailovich Zolotarev. Probability metrics. Teoriya Veroyatmostei i ee Primeneniya, 28(2):264-28) 1983.
[65] Richard M Dudley. The sizes of compact subsets of hilbert space and continuity of gaussian processes. Journal of Functional Analysis, 1(3):290-333 -300000, 1967.
[66] Leonid V Kantorovich. Mathematical methods of organizing and planning production. Man- agement science, 644):366442,, 1960.
[67] Arthur Gretton, Karsten Borgwardt, Malte Rasch, Bernhard Schölkopf, and Alex Smola. A kernel method for the wwo-sample-problem. Advances in Neural Information Processing Systems, 19:513-520, 2006.
[68] Charles Stein. A bound for the error in the normal approximation to the distribution of sum of dependent random variables. In Proceedings of the sixth Berkeley symposium mathematical statistics and probability, volume Probability theory, volume 6, pages 583-603. University of California Press, 1972.
13

---

[69] Aad W Van Der Vaart, Jon A Wellner, Aad W van der Vaart, and Jon A Wellner. Weak convergence. Springer, 1996.
[70] Richard Dudley. Real analysis and probability. 2002.
[71] Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(1):72--773, 2012.
[72] Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. Advances in Neural Information Processing Systems, 2019.
[73] Henry E Kyburg Jr. Bayesian and non-bayesian evidential updating. Artificial intelligence, 31 (3):271-293, 1987.
[74] Vladimir Vovk and Glenn Shafer. Game-theoretic probability. Introduction to Imprecise Probabilities, pages 114-134, 2014.
[75] Peter J Huber. Robust statistics, volume 523. John Wiley & Sons, 2004.
[76] Glenn Shafer. Dempster-shafer theory. Encyclopedia of artificial intelligence, 1:330-331,
1992.
[77] Simone Cerreia-Vioglio, Fabio Maccheroni, and Massimo Marinacci. Ergodic theorems for lower probabilities. Proceedings of the American Mathematical Society, 144(8):3381-3399 2016.
[78] Francesca Mangili. A prior near-ignorance gaussian process model for nonparametric regres-
sion. International Journal of Approximate Reasoning, 78:153-171, 2016.
[79] Giorgio Corani and Andrea Mignatti. Credal model averaging for classification: representing prior ignorance and expert opinions. International Journal of Approximate Reasoning, 56: 264 277, 2015.
[80] Mohammad Hossein Shaker and Eyke Hüllermeier. Ensemble-based uncertainty quantification: Bayesian versus credal inference. In PROCEEDINGS 31. WORKSHOP COMPUTATIONAL INTELLIGENCE, volume 25, page 63, 2021.
[81] Shireen Kudukkil Manchingal, Muhammad Mubashar, Kaizheng Wang, Keivan Shariatmadar, and Fabio Cuzzolin. Random-set convolutional neural network (rs-cnn) for epistemic deep learning. 2023.
[82] Jonathan Sadeghi, Marco De Angelis, and Edoardo Patelli, Efficient training of interval neural networks for imprecise training data. Neural Networks, 1 118:338-33 3518,,51,2201 2019.
[83] Kaizheng Wang, Keivan Shariatmadar, Shireen Kudukkil Manchingal, Fabio Cuzzolin, David Moens, and Hans Hallez, Creinns: Credal-set interval neural networks for uncertainty estima- tion in classification tasks. Neural Networks, page 107198, 2025.
[84] Yasuo Narukawa. Inner and outer representation by choquet integral. Fuzzy Sets and Systems, 158(9):963-77,, 2007.
[85] Carl-Johann Sibon-Gabriel, Alessandro Barp, Bernhard Schölkopf, and Lester Mackey. Metriz- ing weak convergence with maximum mean discrepancies. Journal of Machine Learning Research, 24(184):1 2023.
[86] Ding Feng and Hung T Nguyen. Choquet weak convergence of capacity functionals of random sets. Information Sciences, 177(16):3239 3250, 2007.
[87] Bharath K Sriperumbudur, Kenji Fukumizu, and Gert RG Lanckriet. Universality, characteristic kernels and RKHS embedding of measures. Journal of Machine Learning Research, 12(7), 2011.
[88] Bharath Sriperumbudur. On the optimal estimation probability measures in weak and strong topologies. 2016.
14

---

[89] David A Levin and Yuval Peres. Markov chains and mixing times, volume 107. American Mathematical Soc., 2017.
[90] Shige Peng. Nonlinear expectation theory and stochastic calculus under knightian uncertainty. In Real Options, Ambiguity, Risk and Insurance, pages 144-184. IOS Press, 2013.
[91] Nikhil R Pal, James C Bezdek, and Rohan Hemasinha. Uncertainty measures for evidential reasoning i: A review. International Journal of Approximate Reasoning, 7(3-4):165-183, 1992.
[92] Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
[93] Fabio Cuzzolin. Uncertainty measures: A critical survey. Information Fusion, page 102609, August 2024. doi: 10.1016/..nfffus22224... 102609. URL URL.tttss://iiniigguubb. elsevier. com/
[94] Joaquín Abellán and George Klir. Additivity of uncertainty measures on credal sets. Interna- tional Journal of General Systems, 34(6):691-713, 2005.
[95] Radim Jiroušek and Prakash P Shenoy. new definition of entropy of belief functions in the dempster-shafer theory. International Journal of Approximate Reasoning, 92:49-65, 2018.
[96] Eyke Hüllermeier, Sébastien Destercke, and Mohammad Hossein Shaker. Quantification of Credal Uncertainty in Machine Learning: A Critical Analysis and Empirical Comparison. In Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence, pages 548-557. PMLR, August 2022. URL hullermeier22 html. ISSN: 2640-3498.
mlr press/v180
[97] Yusuf Sale, Michele Caprio, and Eyke Hüllermeier. Is the Volume of a Credal Set a Good Mea- sure for Epistemic Uncertainty?, June 2023. URL http:://///aat arg/abs/2306. 09586. arXiv:2306.09586 [cs, stat].
[98] Inés Couso, Serafín Moral, and Peter Walley. A survey of concepts of independence for imprecise probabilities. Risk, Decision and Policy, 5(2):165-181, 2000.
[99] Fabio G Cozman. Sets of probability distributions and independence. SIPTA Summer School Tutorials, July 2-8, Montpellier, France, 2008
[100] Joaquín Abellán, George Klir, and Serafín Moral. Disaggregated total uncertainty measure for credal sets. International Journal of General Systems, 35(1):29-44 44, 2006.
[101] Eyke Hüllermeier, Sébastien Destercke, and Mohammad Hossein Shaker. Quantification of credal uncertainty in machine learning: A critical analysis and empirical comparison. In Uncertainty in Artificial Intelligence, pages 548-557. PMLR, 2022.
[102] Peter J. Veazie. When to Combine Hypotheses and Adjust for Multiple Tests. Health Services Research, (1(3p1):804888,, June 2006. ISSN 0017-9124, 1475-6773. doi: 10.1111/ j.1475-6773.20 200...0000000000020 x.x. URL (475-67733 2006 00512.x.
wiley. /ot///....///................../
[103] Sébastien Destercke. Handling bipolar knowledge with imprecise probabilities. International Journal of Intelligent Systems, 26(5):426 443, March 2012. doi: 10.1002/in.20445. Publisher: Wiley.
[104] Andrey G. Bronevich and Natalia S. Spiridenkova. Some Characteristics of Credal Sets and Their Application to Analysis of Polls Results. Procedia Computer Science, 122:572-578, 2017. ISSN 18770509. doi: 10. procs//.rocc.201....
[105] Michele Caprio and Ruobin Gong. Dynamic precise and imprecise probability kinematics. In Proceedings of the Thirteenth International Symposium Imprecise Probability: Theories and Applications, pages 72-83. PMLR, July 2023. URL https: /ttp///eeeeennssseee/ee/ep/e/sse mlr, press/
v215/ 0./5///ppr223a. html. ISSN: 2640-3498.
15

---

[106] Michele Caprio and Sayan Mukherjee. Ergodic theorems for dynamic imprecise probability kinematics. International Journal of Approximate Reasoning, 152:325-343, 2023.
[107] Inés Couso, Laura Garrido, and Luciano Sánchez. Similarity and dissimilarity measures between fuzzy sets: a formal relational study. Information Sciences, 199:122-14,, 2013.
[108] Susana Montes, Susana Díaz, and Davide Martinetti. Divergence measures: from uncertainty to imprecision. The Mathematics of the Uncertain: A Tribute to Pedro Gil, pages 675-684, 2018.
[109] Robert Hable. minimum distance estimator in an imprecise probability codel-computational aspects and applications. International journal of approximate reasoning, 51(9):1114-11) 2010.
[110] Walter I. Perry and Harry Stephanou. Belief function divergence as a classifier. In Proceed- ings of the 1991 IEEE International Symposium Intelligent Control, pages 280-285, IEEE, 1991.
[111] Samuel S Blackman and Robert Popoli. Design and analysis of modern tracking systems. (No Title), 1999.
[112] Anne-Laure Jousselme, Dominic Grenier, and Éloi Bossé. A new distance between two bodies of evidence. Information fusion, 2(2):91-101, 2001.
[113] George Klir and Mark Wierman. Uncertainty-based information: elements of generalized information theory, volume 15. Springer Science Business Media, 1999.
[114] Fuyuan Xiao, Weiping Ding, and Witold Pedrycz. A generalized f-divergence with appli- cations in pattern classification. IEEE Transactions on Knowledge and Data Engineering, 2025.
[115] Marta Catalano and Hugo Lavenant. Hierarchical integral probability metrics: A distance on random probability measures with low sample complexity. arXiv preprint arXiv:2402.004233 2024.
[116] Joaquin Abellan and Serafin Moral. Maximum of entropy for credal sets. International journal of uncertainty, fuzziness and knowledge-based systems, 11(05):587-597 2003.
[117] Yusuf Sale, Michele Caprio, and Eyke Höllermeier. Is the volume of a credal set a good measure for epistemic uncertainty? In Uncertainty in Artificial Intelligence, pages 1795-1804. PMLR, 2023.
[118] Yarin Gal et al. Uncertainty in deep learning. 2016.
[119] Lisa Wimmer, Yusuf Sale, Paul Hofman, Bernd Bischl, and Eyke Hüllermeier. Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? In Robin J. Evans and Ilya Shpitser, editors, Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence, volume 216 of Pro- ceedings of Machine Learning Research, pages https: http:ss//poccoeinge.. mlr, wwwpppeesssveee wttess//21///iimeer/// html.
PMLR, 31 Jul-04 Aug 2023. URL
[120] Freddie Bickford Smith, Jannik Kossen, Eleanor Trollope, Mark van der Wilk, Adam Fos- ter, and Tom Rainforth. Rethinking aleatoric and epistemic uncertainty. arXiv preprint arXiv:2412.20892, 2024.
[121] Yusuf Sale, Paul Hofman, Lisa Wimmer, Eyke Hüllermeier, and Thomas Nagler. Second-order uncertainty quantification: Variance-based measures. arXiv preprint arXiv:2401.00276, 2023.
[122] Yusuf Sale, Viktor Bengs, Michele Caprio, and Eyke Hüllermeier. Second-Order Uncertainty Quantification: A Eestance-Based Approach, December 2023. URL http://arxiv org/ abs/2312 00995. arXiv:2312 Ttt:2331..00995 [cs, stat].
[123] Arthur Hoarau, Sébastien Destercke, Yusuf Sale, Paul Hofman, and Eyke Hüllermeier. Mea- sures of uncertainty: A quantitative analysis, 2025.
16

---

[124] lipm: Reproducibility code for neurips 2025 submission. https://anonymont 4open. Sciencce/rIIPMMNeor1PPP53 B5E1, 2025.
[125] Siu Lun Chau, Jean-Francois Ton, Javier González, Yee Teh, and Dino Sejdinovic. Bayesimp: Uncertainty quantification for causal data fusion. Advances in Neural Information Processing Systems, 0133466637777 2021.
[126] Siu Lun Chau, Shahine Bouabid, and Dino Sejdinovic. Deconditional downscaling with gaussian processes. Advances in Neural Information Processing Systems, 34:17813-17825, 2021.
[127] Joaquín Abellán and Manuel Gómez. Measures of divergence on credal sets, Fuzzy Sets and Systems, 157(11):1514 June 2006. ISSN 01650114, 14. doi: 10.1016/j. 11.021.
[128] Estimation of obesity levels based on eating habits and physical condition [dataset], 2019. URL https://doi. org/1o///// 24432/C5H312. UCI Machine Learning Repository.
[129] E. Alpaydin and C. Kaynak. Optical recognition of handwritten digits [dataset], 1998. URL https://doi.orge 24432/C50P49.
[130]
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[131] Chun-Liang Li, Wei- Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnabás Póezos. Mmd gan: Towards deeper understanding of moment matching network. Advances in neural information processing systems, 30, 2017.
[132] Christopher Bülte, Yusuf Sale, Timo Löhr, Paul Hofman, Gitta Kutyniok, and Eyke Hüller- meier. An axiomatic assessment of entropy-and variance-based uncertainty quantification in regression. arXiv preprint 250i:::204.144333 2025.
[133] Sébastien Destercke, Didier Dubois, and Eric Chojnacki. Unifying practical uncertainty representations-i: Generalized p-boxes. International Journal of Approximate Reasoning, 49 (3):649-663, 2008.
[134] F. J. Giron and S. Rios. Quasi-Bayesian Behaviour: A more realistic approach to decision making? Trabajos de Estadistica Y de Investigacion Operativa, 31(1):17-38, February 1980. ISSN 0041-0241. doi: 10. 10.10077BF028888445 URL http: springer..... com/
10. 1007 N0.1007/BF028880 /BF02888355.
[135] Leonard Savage. The Foundations of Statistics. Wiley, New York, 1954.
[136] Jon Williamson. In Defence of Objective Bayesianism. Oxford University Press, April 2010. ISBN
doi: 10. 00l.09///apmfooo0007777999999900 1000000000000.
[137] Hamed Rahimian and Sanjay Mehrotra. Distributionally Robust Optimization: A Review, Open Journal of Mathematical Optimization, 3:1-85, July 2022. ISSN 2777-5860. doi: 10.5802/ojo..15 URL http:/ http://axxvvvvv stat].
wttpaawwwwww..aaaa/a/r/aaaaaaaat wwwp/org////aeoeeeeooeeeeee 05659. arXiv:1908.0566 [cs, math,
[138] Yishay Mansour, Mehryar Mohri, and Afshin Rostamizadeh. Multiple Source Adaptation and the Renyi Divergence, May 2012. arXiv:1205.266: [cs, stat].
[139] Simon Föll, Alina Dubatovka, Eugen Ersst Siu Lun Chau, Martin Maritsch, Patrik Okanovic, Gudrun Thaeter, Joachim Buhmann, Felix Wortmann, and Krikamol Muandet. Gated domain units for multi-source domain generalization. Transactions on Machine Learning Research, 2023.
[140] Kaizheng Wang, Fabio Cuzzolin, Keivan Shariatmadar, David Moens, and Hans Hallez. Credal wrapper of model averaging for uncertainty estimation in classification. In International Conference on Learning Representations, 2025.
[141] Fabio Cuzzolin. Uncertainty measures: The big picture, April 2021. URL http: http://arrii org/ abs/2104. 06839, arXiv:2104.06839 [cs, math, stat].
17

---

[142] Ibrahim Ghafir, Konstantinos G Kyriakopoulos, Francisco Aparicio-Navarro, Sangarapillai Lambotharan, Basil AsSadhan, and Hamad BinSalleeh. A basic probability assigmment methodology for unsupervised wireless intrusion detection. IEEE Access, 6:40008-40023, 2018.
[143] Shireen Kudukkil Manchingal, Muhammad Mubashar, Kaizheng Wang, Keivan Shariatmadar, and Fabio Cuzzolin. Random-set neural networks. In The Thirteenth International Conference on Learning Representations, 2025.
[144] Muhammad Mubashar, Shireen Kudukkil Manchingal, and Fabio Cuzzolin. Random-set large language models. arXiv preprint arXiv:2504. 18085, 2025.
[145] Freddy Delbaen. Convex games and extreme points. Journal of Mathematical Analysis and Applications, 45(1):210- 1974.
[146] Ignacio Montes, Enrique Miranda, and Paolo Vicig. 2-monotone outer approximations of coherent lower probabilities. International Journal of Approximate Reasoning, 101:181-205, 2018.
[147] Luigi Ambrosio, Elia Brué, Daniele Semola, al. Lectures optimal transport, volume 130. Springer, 2021.
[148] Stefan Aeberhard and Michele Forina. Wine [dataset]. fttps:///di..rg/10...o C5PC7J, 1992. UCI Machine Learning Repository.
[149] Alex Smola, Arthur Gretton, Le Song, and Bernhard Schölkopf. hilbert space embedding for distributions. In International conference on algorithmic learning theory, pages 13-31. Springer, 2007.
[150] Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, Bernhard Schölkopf, et al. Kernel mean embedding of distributions: A review and beyond. Foundations and Trends@ in Muchine Learning, 10(11)) 2017.
[151] Francois-Xavier Briol, Alessandro Barp, Andrew B Duncan, and Mark Girolami. Statistical inference for generative models with maximum mean discrepancy. 3xXiv:/906.00 05944, 2019.
[152] B-F, Chérief-Ahdellatif and P. Alquier. MMD-Bayes: Robust Bayesian estimation via maxi- mum mean discrepancy. In Proceedings of The 2nd Symposium on Advances in Approximate Bayesian Inference (AABI), pages 1-21, 2020.
[153] Krikamol Muandet, Motonobu Kanagawa, Sorawit Saengkyongam, and Sanparith Marukatat. Counterfactual mean embeddings. Journal of Machine Learning Research, 22(162):1-71, 2021.
[154] D. Sejdinovic. An overview of causal inference using kernel embeddings. arXiv:2410.22754, 2024.
[155] Siu Lun Chau, Robert Hu, Javier Gonzalez, and Dino Sejdinovic. Rkhs-shap: Shapley values for kemel methods. Advances in neural information processing systems, 35:13050-13063, 2022.
[156] Krikamol Muandet, Kenji Fukumizu, Francesco Dinuzzo, and Bernhard Schölkopf. Learn- ing from distributions via support measure machines. In Advances in Neural Information Processing Systems 25, pages 10-18. 2012.
[157] Zoltán Szabó, Bharath K Sriperumbudur, Barnabás Póczos, and Arthur Gretton. Learning theory for distribution regression. Journal Machine Learning Research, 17(1):57225331 2016.
[158] Antonin Schrab and Imun Kim. Robust kernel hypothesis testing under data corruption. arXiv preprint arXiv:2405. 199/2, 2024.
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Section introduces the IIPM framework and how can be used for analyses, Section 4 introduces MMI and how it can be used for epistemic uncertainty quantification and Section presents experments validating our claims on MMI.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: In Section 3 we discussed one limitation of IIPM, which is that, because capacities are more general than probability measures, meaning is less structured, many "nice" properties in expectation, e.g. linearity, does not directly carry over to expectation with imprecise probability. In Section we also listed out another limitation, which the lack of sampling mechanism for imprecise probabilities. leading to the lack of sample-based estimation for IIPM compared to standard IPM.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
19

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: Yes
Justification: We try define everything and also provide the full set of assumptions in the main text. We also provide proof sketches and strategies the main text.
Guidelines:
The answer NA means that the paper does include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide code to reproduce experiments. In the appendix, also provide more detailed experimental set-up for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
also be provided via detailed
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility.
20

---

In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: Yes
Justification:
link the codebase reproduce our experiments given.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nig) cc/
htapii/gguidess/deeubniiiii onolsicyy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ /nips. cccpublic/guu wwwcc///wuee//eeeeeeeeeee httpgguudeee/oeeeumiiieeeeee www.SSummmssoooool..io
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: Yes
Justification:
The full details provided with the code, with explanations in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
We repeated our experiments ten times and produces error bars for our results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
21

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We included this information in the appendix where we discussed the detailed experimental setup.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: :tap:::///eeue http///eerriis.peee http::pppw.pppppiicttieiccceeo/e
Answer: [Yes]
Justification: We followed the instructions and cited the appropriate reference when using the datasets. We do not have to experiment human subjects. Annoynamous code base is also provided.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This paper presents work whose goal is advance the field of core Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.
22

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We use standard datasets in the ML literature so it doesn't It seem applicable here.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We provide references to all datasets we have used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
23

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
We do not involde any crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work do not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
24

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does not involve LLMS as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https: / //neruurpp.... wwc/onneeeenncs///O00e for what should or should not be described.
25

---

Appendix of Integral Imprecise Probability Metrics
Some more introduction Imprecise Probabilistic Machine Learning (IPML)
27
A.1 What is Imprecise Probabilistic Machine Learning actually doing?
27
A.2 Credal sets and their use in IPML
28
A.3 Belief function/random and their use IPML
28
Proofs and derivations
29
B.1 Proof of Lemma 5
29
B.2 Proof of Theorem
30
B.3 Proof of Corollary 8
30
B.4 Proof of Proposition 9.
30
B.5 Proof of Theorem 10
B.6 Proof of Proposition
B.7 Proof of Remark 14
31
B.8 Proof of Lemma 15
32
B.9 Proof of Theorem 16
32
B.10 Proof of Proposition 18
32
B.11 Proof of Proposition 19
33
................................
B.12 Proof for Theorem 20
33
B.13 Proof of Proposition 21
36
C Further experimental details
37
Ablation study: Correlation with Generalised Hartley and Entropy Difference.
C.2 Overview Generalised Hartley measures and Entropy Differences
37
C,3 Implementation details
39
IIPM with epsilon-contamination set
D.I Lower Probability Kantorovich Problem
40
D.2 Nonparametric Estimator of IIPM with e-contamination set using kernel distance.
40
26

---

capacities
credal sets
lower/upper probabilities
2-monotone capacities
clouds
probability intervals
random sets/belief function
comonotonic clouds
p-boxes
possibilities
probabilities
Figure 2: A skeleton demonstrating the connection between various uncertainty calculi. "A - B" means A generalises B, meaning that B is specific instance of A. The figure is adopted from Destereke et al. [133] and Hüllermeier and Waegeman [4]. Most these frameworks generalise classical probability theory. In the main text, have discussed capacities, lower and upper probabilities, and standard probabilities in detail, with a brief mention of 2-monotone capacities. Additional discussion of credal sets and belief functions is provided in Appendix A. We do not elaborate on the methods shown in grey; for those, we refer readers to existing surveys and review articles.
A Some more introduction to Imprecise Probabilistic Machine Learning (IPML)
In the main text, we demonstrate the theoretical appeal and practical utility of IIPM primarily through capacities and lower probabilities. While these models are already quite general, we complement this focus by discussing two other mainstream approaches in IPML- credal sets and belief functions that, although mathematically related, are conceptually motivated in distinct ways. As illustrated in the hierarchy in Figure 2, these models are closely connected to the core ideas of the paper and further support the relevance and broad applicability the proposed IIPM and MMI framework.
A.1 What is Imprecise Probabilistic Machine Learning actually doing?
At its core, probabilistic machine Icarning seeks to construct mathematical models that, through data-driven learning procedures, capture underlying physical or real- real-world phenomena. For instance, in generative modelling, the objective is to learn the marginal probability distribution that governs the gettaagnerating process. In predictive tasks, instead, the goal is estimate the conditional distribution of a target variable Y given an input x z-or or its expectation in the case of regression. We often refer to these kinds of natural variation and randomness as aleatoric uncertainty.
Imprecise probabilistic machine learning (IPML) extends this foundation by moving beyond the exclusive use of precise probability models. While classical probability excels at modelling aleatoric uncertainty, IPML incorporates imprecise probability models to account not only for inherent ran- domness but also for epistemic piceetainty-alloviinn be explicitly represented within the model. For readers interested deeper treatments on IP, we recommend Cuzzolin [9] for comprehensive introduction, Hüllermeier and Waegeman [4, Appendix A], for a concise overview, and Caprio et al. [47, Appendix A], for a discussion on why we should care about imprecision.
ambiguity, partial knowledge, and doubt to
In the following, we provide an overview of two other mainstream modelling approaches in IPML: credal set and belief function approaches. We outline the motivations behind these methods and provide some examples of how they are integrated into MI. to improve uncertainty quantification and predictive performance.
27

---

A.2 Credal sets and their use in IPML
Credal sets sit at the top of the hierarchy shown in Figure making them one of the most general constructs in imprecise probability. Many other models can be seen as special cases of credal sets endowed with additional structure. Consequently, there are numerous ways to construct credal set, depending on the modelling assumptions and information available.
Credal sets are generally understood some convex of probability measures P Convex- ity can be justified in different ways. In Quasi-Bayesian decision theory [134], it can be shown that rationality axioms proposed for partial binary preference naturally leads convex set of probability measures, akin to how Savage [135] showed rational binary preference leads to the existence of unique single probability distribution (paired along with an utility function). Alternatively, in formal epistemology. Williamson [136] put forward the notion of Chance Calibration, which is also closely related to Lewis [7]'s Principal Principle, which puts into the words of statistician means, when, according to the current observations, the actual distribution of interest encoding physical phenomena lies within some set {P ................................ p. } of distributions, but the modeller is indifferent as to which distribution, then rationally, the modeller's ' belief, also represented as distribution, should lie within ConvexHull({P,
Pm This means any distribution the convex hull is considered a rational belief, thus, the set itself captures the set of rational beliefs, encompassing our imprecision.
In IPML, credal sets are often used represent either
1. Dataset/Distribttion--eed uncertainty, or
2. Predictive uncertainty.
Case 1. In the first case, examples include distributionally robust optimisation[137]. where a learning algorithm is optimised against the worst-case empirical risk over a credal set-a set of distributions within an e-distance from the observed empirical distribution. In the oft-of-domain generalisation literature, given observations from multiple source distributions F...... Pm, commonly assumed that the test-time distribution lies within their convex hull [138, 139]. which effectively forms credal set. Singh et al. [14] made this connection explicit and proposed an algorithm that allows the test-time ambiguity to be resolved without additional training. Caprio et al. [49] developed learning-theoretic theoretic framework for supervised learning under credal sets, while Chau et al. [15] introduced a hypothesis testing procedure for statistically comparing credal sets.
Case 2. In the second case, credal are used to model epistemic uncertainty in prediction. In credal Bayesian deep learning (CBDL) [47], finitely generated credal sets over priors and likelihoods are combined, by considering all combinatorial applications of Bayes' rule. to yield a posterior credal set. Wang et al. [83] introduced credal-set interval neural networks, which predict credal sets from probability intervals derived from deterministic interval neural network outputs. Similarly, for classification tasks, Wang et al. [140] proposed defining a predictive credal set as the collection of probability vectors within the simplex that satisfy lower and upper bounds on class probabilities, derived from set of probabilistic predictors.
A.3 Belief function/random set and their use in IPML
Moving down the hicrarchy-and and skipping lower probabilities and 2-monotone capacities, which are already discussed in the main paper-we we briefly describe the roles of random set theory and belief functions.
We focus on finite instance spaces X, where random set theory and belief functions coincide. Our exposition follows the terminology of Shafer's seminal work on The Theory of Evidence [23] and the overview in Cuzzolin [141, Chapter 2.2]. The core philosophy behind belief function theory is its emphasis on representing degrees of support-capturing epistemic uncertainty-ratte specifying how these values are generated, which relates more aleatoric uncertainty. Perhaps more importantly are the tools developed to combine multiple evidence in coherent manner.
than
So, how are belief functions defined? We first introduce fundamental concept, called basic probability assignments. Let X' be finite.
28

---

Definition 22. A basic probability assignment over X' is a set function m 22 - [0, 1] such that
m( = 0, Σ AÇA
m(A) 1.
The subsets that have non-zero mass are known the focal elements within 2 Basic probability assignment happens in practice when, e.g., sensors have limited precision and can only give results of the type "A or B" [142].
Given function m, which intuitively represents the degree of belief that the true outcome lies exactly within the subset A E 2², m((A) quantifies the support assigned to the set A and no more specific subset. From here, we can derive the belief function.
Definition 23. The belief function associated with a basic probability assignment m 2 - [0, 1] is the sel function Bel 2x + [0, 1] defined as,
(el(A)
Σ m(B.
BCA
Its conjugate, known as the plausibility function Pl, measures the amount of evidence not against an event by measuring the following,
PI(A) = 1 Bel(
The theory of evidence is rich and conceptually deep. and a full treatment lies beyond the scope of this appendix. However, the context provided should suffice to understand recent integration of belief functions into machine learning. Specifically, Manchingal al. [143] introduced new class of neural networks, called Random-Set Neural Networks (RS-NNs), for K-class classification. Instead of producing a probability vector with K outputs- as standard Peessffers-RS-Me are designed to output a basic probability assignment over the K classes, requiring 2 output nodes to represent belief mass on all subsets of classes. Since this computationally infeasible for large K, preprocessing step is introduced to select subset of focal elements, including the original singleton classes and additional relevant subsets. Given the resulting mass function, belief and plausibility functions can then be derived for prediction and uncertainty quantification. This principle is further extended to convolutional neural networks in Manchingal et al. [81]. As perhaps the first of its kind, this random set-based learning paradigm has also been recently adapted to large language models in Mubashar et al. [144], offering an explicit mechanism for modelling epistemic uncertainty in LLMs.
B Proofs and derivations
This section presents the proofs and derivation in the main text.
B.1 Proof of Lemma
Lemma 5. For lower probability P. associated to credal set C, we have f fdP < inf inf J fdP for any f E (x) When P. is 2-monotonic, the inequality becomes an equality.
Proof. Let P be the lower probability associated to the credal set C. For f E Cx(A), we can write the Choquet integral as,
f sap 1 1' P(S 13)dt
=[+ /'
inf PEC
P(U 2 f))dt
(' P(U 2 ())dt
f inf Pee
/
inf PEC
fdP.
For 2-monotone P, the results follow from Delbaen [145, Lemma 2].
29

---

B.2 Proof of Theorem 6
Theorem Let (X,d) be a metric space. For any capacities v, v E V(X), we have f fdv = fdp for all f E Co(2), if and only if μ.
Proof. (=>) Let U be any open set in and the complement. Consider the distance d(x, F) = minyer d(x, y). For .... let (x) min(1, nd(x, F)). Then, supxes fn fl where f = ly is the indicating function. Now, as Choquet integration continuous with respect to the topology of uniform convergence [19, Proposition C.5(ix)), we have
f fadv= = of Lu = o Ludu.
lim 11-00
This implies v(U) = p(U). Now, since (X, a metric space, for any E Σx. we can find an increasing sequence of open subsets A C A2. such that An A. Since v, 1r are continuous from below, we have lim v( ((A.)= = ((A,, but since for any open set v(A,) = μu{ (A,), we conclude v(A) {(A), thus μz.
If = μu then it is trivial seef fdv =f fu y for any f Cx()..
B.3 Proof of Corollary 8
Corollary 8. For any P,Q P(X) and F S C.()),
Proof. For any P,Q E ((X) and F FC CC()), we have
{
f sdp - sdo
IIMM((P,Q) suu
FEF
/ saa)
sdp
sup fer
IPM (P,Q),
since the Choquet integral for additive probability is the Lebesgue integral.
B.4 Proof of Proposition
Proposition 9, For any F C. Co(X). IIPMy a pseudometric on V(X); it is non-negative, symmetric, and satisfies the triangle inequality.
Proof. To prove it is pseudometric, we need non-negativity, symmetry, and to show triangle inequality.
Non-negative: It obvious that IIMM v2) 2 0 for any pair of vn, u2 E (
Symmetric: Symmetry is also apparent.
Triangle inequality: Pick v1, 42, va from V(X), then
f fdu - f ídv2
II1M sup FEF
f fdv f [dvus + o fdvs f save)
sup feF
f fdu - f [dvs + sup SEF
f sdus - ff sauzs)
S sup
fEF
IIPM (M1, + 1I v2).
This concludes the proof. We note that in some literature, pseudometric also requires IIPM,(u,)) 0, and this also trivially holds in our case.
30

---

B.5 Proof of Theorem 10
Theorem 10. Let S Co(X) be dense in Ci(2) with respect to the || llso norm. Then, 1 metrises the Choquet weak convergence of (X)).
Proof. To show that 11PMy metrises the Choquet weak convergence of V(X)), need show for Va, V E V(2",, whenever TI v) + 0 then converges to v in the Choquet weak sense.
Now pick
Ca(2),, since by assumption FF is dense in Ca(X), there exists 9 F satisfying Ilf - Olooo < c. By assumption, IIPM)(Vn, - means If gdv, - -f gdvl - O since 9 E F.
Thus, have the bound
f fdvm f fdvl
fdol=
S { fdvn - gavn + + gdvn - -f gdv + If fdu f fdv
odv
2c
for all f Co(X) and 0, which implies Vn converges to in the Choquet weak sense. Proving the other direction is a direct application the result from Theorem since F C C&(A).
B.6 Proof of Proposition
Proposition 12. The Lower Dudley metric metrises Choquet weak convergence on P(X').
Proof. The core idea show that Fa dense in (x)) and then apply Theorem 10. To show denseness, pick any f E (x()), consider the sequence
fn (x)
inf (f(() + nd(xx))) y)}-
yE
Pick 0 such that [f(x) S for all E X. thus If(x) f(v)) S If(x) [ + S for any I, E X. It's clear that -a S fn $ f is bounded and n-Lisschitz continuous. Now we prove fa f uniformly. Fix e 0, there 8 > 0 such that d(x,, y) 5 implies [f(x:) f(w) since is continuous. Then for all x E X, we have
0 S f()) - fx(2) f(x) - inf (f( + nd(xx v)}
DEA
sup - f(y) nd y))
yEX
sup
{f(x) - f(y))= nd(x, y)}
VEX (EX,d(a.)))2uf
If n is such that 2a 8, then
f(x) - fa(r) S supfc v) y E X' s.t. d(x,
v) I 20/n1 <
which follows that [lf fall < e. meaning Fu is dense in Co(X)) in the uniform norm.
B.7 Proof of Remark 14
Remark 14. Let X be finite. For any P.Q P(X), have IPM (P,Q) SuPACE, IP(A)
Q(A)) 1/ Exex (P((x)) Q((x))) In contrast, in the imprecise case, there exists P, P,Q- E 2(x)
such that IIPM (P.Q) = supAeEx [((A) - Q(4) + E (P((z)) Q((x)))
Proof. We provide an example based Montes al. [146, Example 4] for completeness. Consider X {21, 22, 23} and lower probabilities P,, Pa given by
0
[x1
[x2]]
[xg]
{x1. 22
[21, Fzl
[2g, Fz
X
P,
0
0
0
0
1/2
1/2
1/2
I
P
0
0
0
1/3
1/3
1/3
Then, know d1 SupACA LP, (A) P2(A)] = 1/6, whereas de Exex Ri((x)) (a((z)) = 0, therefore d1 # d2.
31

---

B.8 Proof of Lemma 15
Lemma 15. Let P..p be e-contaminated model defined in Equation (2). Then, the associated lower probability P. is given by
P.(A) = inf P(A) P(A))
for all E Ex 1123 for X
PEP.P
1,
Proof. This lemma is proved in Walley [8, Section 2.9.2].
B.9 Proof of Theorem 16
Theorem 16. Let Fw {f E Co(X): Ilflll S 13 where |l supa ((f()) (1f(=))((o)]e((x)))) and e the transportation cost in restricted lower probability Kantorovich (RLPK) problem [56, Definition 10). Let P.. Q be lower probabilities of the e-contaminated models P.. and P.Qq. Then, 11PM ().Q) coincides with the objective of the RLPK problem, and thus coincides with the classical Kantorovich's 's optimal transport problem involving P and Q.
Proof. Our goal is to show that 11PMFw (Le, Q,) probability Kantorovich problem. Note that have,
coincides with the objective of the restricted lower
A (P.
sup feFw
fdP. - fde.
sup feTw
P.lff 2()) 2.f
sup /EFw
P.lff x - ())
1
sup fefw
[(1 P)U 14) - -0-0 > 1))]dt
/ sap - fSQQ)
S
(1 c) sup fEFW
inf TEI((..) / c(x, V) dy)
c)
where I(P, Q) is the set of joint probability with marginals being P and Q. We replaced the inequality with strict inequality in V in Troffaes and De Cooman [19, Proposition C.3.ii]. In & we used the fact that Choquet integration returns the standard Lebesgue integral when the capacity is a probability used Mantorovich-Rubinseee
measure, and in duality between the Kantorovich problem and an IPM formulation using function class Fw. This recovered the result from Caprio [56] through the use of IIPM framework.
theorem [147, Lecture 3), which established the
B.10 Proof of Proposition 18
Proposition 18. The definition of MMI is equivalent
' 1 (P(If< 2t))
MMIx(P) sup JEF
(3)
32

---

Proof. To show the result, it follows from the definition that
f fdP
MMIx(P)
sup
fdP
fEF
sup fer
P([f t}) P(ff 2 tt)]dt
sup fEF
[] (P(US + 2 d)))]
1
which completes the proof.
B.11 Proof of Proposition 19
Proposition 19 (MMI on c-contamination set.). Let P. be the lower probability associated with Pe. and F _ Cs()). Then MMI((P,) = c (sup)Ez sup x.xx f(z) f(y))). For the LTV distance with FTV == [1A A E }x), we have MMIFrv (P.) SupAEE, - =
Proof. First, it can be shown readily the upper probability model for contamination set:
(1 ()-) P(A)
+e for A Σx /0 for = 0
10
Next, we have
fdP -f fdP
MMI = sup fer
P(lf 2 - 2 t))]dt
sup feF
|e ++(1 - 14))
sup fEF 1
sup fEf 1
c dt
=e =e(supl/ supls - -11 - fEF
feF a.Ex (sup sup (f(x) - -(w)))
which shows the result. For Frv, it straightforward to see that the maximum value the terms the bracket of the last equation attain is 1. Therefore,
MMI (P.)
This completes the proof.
B.12 Proof for Theorem 20
To prove our uncertainty measure satisfies the axioms, we need the following useful lemma.
Lemma 24 (Marginalisation preserves 2-monotonicty.). If P is a 2-monotone capacity defined on the joint measurable space (X' x )/ Σx x Ey), and let (,(.) = P((,)) be the marginal capacity on (X, xx) and P,(-) = P(X..) be the marginal capacity on (y,Ey) then P1 and P., are also 2-monotone. Marginalisation also preserves 2-alternating.
Proof. Recall the definition of 2-monotonicty, for any A, E Ex Ey, we have
P(AUB) + + B) 2 P(A) + P(B).
33

---

Now pick A1, B1 E Σx, then consider,
li U Pi (A B1) P((A1 UB) x 2 x =[(AA (B) +((A) x x)) x x
2 L(i x3) + LB, x
(P(A))++ (
This shows that 2-monotonicity holds for P1 and by symmetry, it also holds for Pz- Therefore, marginalisation preserves 2-monotonicity. The steps to show marginalisation preserves 2-alternating analogous.
Theorem 20. For any F S C2(2)), MMI; satisfies axioms A1-A4. If P. is 2-monotonic and with JF F12 defined above, then MM satisfies axioms A1-A5. For A5, the subadditivity becomes MMIF I MMIF, + MM If the notion of independence in A6 is taken to be strong independence in the sense of Cozman [99], A6 also holds, and MMI MMIF, MMIF..
Proof. Axiom A1. Starting from Axiom that MMT non-negative and bounded. First, recall that lower probabilities are super-additive, meaning, for A, B E Ex with A in B = 0, we have
P(AUB) Z P(A) + P(B).
Now let A = (f 1 t) and B = (f < t), then we have
12 P(U 2 ()) +(U <
This means the integrand in Equation (3) is always non-negative, thus, MMI is always non-negative. To show boundedness, notice that,
I' [1 (P((f <t))
[MMI((P)
sup SEF
< sup
1 (P(U +)) P(f < dt
feF
/' ldl
< sup JEF
sup sup [f(x) - f(y)
JEF a.EEN
< 2 sup sup |f(z)] feF Zex
As f E Cb(X), by definition, is a bounded function, thus sup suprex (f(x) is bounded.
Axiom A2. For continuity, our goal is show that given a sequence of lower probabilities P. converges to P in the Choquet weak convergence sense, then MMIx((2,) - MMIx((). Pick e 0, we know there exists n. e N such that for all n > ne.
fap <E
for e Co(X). An immediate result that follows from Troffaes and De Cooman [19, Proposition C.5.iv] for upper probabilities is,
f SaP, n f sdP
f fdP -f -fdP <e
ffap
34

---

since (MM) (MM
-f (.()). Now with this n, we know
supts set
f fdPn f sdent-supf gdP gddp
NET
ret smppf of suPm f fupp+ sap o saR+ f saP f sdR.) supt
-upt f gdP gef
of gdP)
suus let
fdPn f
supt of fdP. - f sdP) + tupp o gdl f gdl)
f gdl -uppl
gdP)
|EF
get
get
S supll fer
fdPn. f + supll f fdRn f fdPl} <
sdP)
jeF
Thus, we have proven continuity of MMIF.
Axiom A3. To prove monotonicity, notice if P setwise dominated by Q, then we have for any t and any f F.F,
P(U 2 2)) + P(U < & Q(U 2 Q(1 <))
(P( 2 +)) + P(( < ()) 2 1 (Q(1s 2 Q( < f)))-
Since the integrand of one integral is always at least large as the other one, by Troffaes and De Cooman [19, Proposition C.5.vi], we have
I / - (PU 2 4) + P(U < <1)) Z /' 1- 1- (QU1 ()+ + QQU <t))ddt 7
MMT,(() MM)
Axiom A4. Showing probability consistency is almost trivial as any P E P(X) is self-conjugate, therefore (M)= O.
Axiom A5. Recall F12 is defined as
F12 = (f Co(X) f(11,02) = (z1) ( + f2(22) for some fi C,()),, E Cg()))) also that for axiom A5 we are working with 2-monotonic lower probabilities P. meaning for any event A, B E Ex,
P(AU B) + P(AnB) 2 P(A) + P(B).
Now with Troffaes and De Cooman [19, Proposition C.7] we know that for 2-monotone capacities, the Choquet integral super-additive, meaning, for F12
f f fadd of fide+ f fzdP= ff fidRi+ + fade.
Notice the last equality holds because P(li 2 4) P((t21 EX X [i(z1) x x2)) ((x1 X; (x1) 2 1)). Similarly, we can show that for upper probabilities of 2-monotone lower probabilities, they are 2-alternating, meaning that
f f f fa)P fD ff hdPP+
fadP= f fidPit o fadP2
Now, combine the two results, we have,
MMI (P)
fdP fdP- Sap]
sup
JeFiz
sup fefia {f fidPi + FadP= f + fodes)
{f SuPi - -f }
sup SEFs {
< sup seFi
MMI; (Pi) + MMMIF
35

---

Axiom A6. Now when P1 and P2 are strongly independent, then the credal set associated to P. denote as is related the credal set associated P1 and P2 denote as C₁ and Cz. as follows:
C : E P(X) P= Pi P3 for some Pi E CI, P2 E C2).
To show that the uncertainty is additive when strong independence holds, we use the following representation of the Choquet integral for 2-monotone lower probabilities,
f f = inf sap
PEC
and similarly for 2-alternating upper probabilities, we have
faP== sup
/ fdp. pec
sup
Now we can show the result, starting with
MM (P)
ffap f sap
sup feF1
(A)
sup PEC Ui++ + 12)dP
/ (/i
/ (Ui + sauae
sup h
inf QEC
(B)
(/ [iddt + [sad.) sup PSE,
inf QrEciRRsEs { fid -1 Fador}}
sup
sup (piec / [ddP++ + sup P.EC2
/ fedPs diet, / fidli inf jitt I sodees
sup
inf
fEFFFEEFF
(C)
{f IuPi
JadP2 f hiPp- - f SadPaa]
sup hEF
{f hidPi f (IdRi) sup hEE
[fdP f s
sup
hE
MMI (P,) + (P2),
In step (A), we used the fact that the Choquet integral of 2-monotone capacities is the lower envelope of the set of expectations with respect to the credal set [145, Lemma 2]. In Step (B), we used the fact that due to strong independence, every element in E C' can be written as a product of some
Pi E C and P2 E C2 and the fact that Lebesgue integration is linear, and f fi (xz) dxz) (() Pi (dæ1). Finally, in step (C), we used Lemma 24, which stated that marginals of 2- monotone capacity remain 2-monotone, therefore, we can write inf I fi dP, back as a Choquet integral.
B.13 Proof of Proposition 21
Finally, prove the result for the upper bound, first recall intermediate result from Montes et al. [146, Proposition 8), which provides a construction for the best pessimistic e-contamination set approximation any lower probability P. E P(X). Understanding this proposition requires clarifying the concept of dominance and outer approximation.
Definition 25 (Outer approximation). Let X be finite and Q P. E PP()) two lower probabilities. We
say Q is outer approximation of P. if for every event A E 2 Q(4) S P(A).
The direction of the inequality might be confusing first, but by realising Q(A) P(A) for every event 2 it means the core of Q i.e.
M(Q) (Q e P(X) : Q(A) 2 Q(A) VAE2),
is at least as large as the core of P.c.e. M(P) M(Q).
36

---

Definition 26 (Undominated outer approximation in C. (X)),, Let P(X) be lower probability Let (x) P(X) be subspace of which the outer approximation Q resides. We say Q is an undominated outer approximation of P in (. if there exists no other lower probabilities Q' in (2C) such that M(P) S M(Q') &. M(Q).
Now have all the language to understand the result from Montes et al. [146]. Let C,())) denote the space all c-contamination models defined on X. Proposition 27. Let P. E P(X) be a lower probability. Define e E (0, 1) and the probability Po by:
P(fx)) Po((=)) = ΣnEX P({x}) for every x E X. 6 6=1- ---E((a)))) ZEX
Denote by P. the e-contaminated model constructed following Lemma 15. Then, P. is the unique undominated outer approximation of P in (X)).
With this result,
now derive the upper bound for MMI Fry (P)- Proposition 21. Let X be finite. For any P(X), MMT Frv (P) 51-ExER P(tx)).
Proof. We know for any P, P. constructed in Proposition 27, is the unique undominated outer approximation of P E (x)) Since it is an outer approximation, meaning that (A() S P(A) for all events E 2*, therefore by the monotonicity axiom (Axiom 3) in theorem 20, we know for any F C (x))), we have
MMI=(P) S (L.)
Now choose F = FrV in Definition 13, then along with Proposition 19, we have
MMI (P) I MMIFr((P)
=6.
Ep((x)).
This concludes the derivation.
C Further experimental details
C.1 Ablation study: Correlation with Generalised Hartley and Entropy Difference.
In the main text, observed that the performance of MMI, its linear-time upper bound MMI-Lin, and the generalised Hartley measure are nearly equivalent in the context of selective classification. This raises the question: are they numerically equivalent? That is, is generalised Hartley and MMI using the total variation function space, i.e. lower TVV, providing the same value? While staring at the equations tells us they are not equivalent, our ablation study suggests that even though they are highly correlated, they are not the same.
To investigate that, use the UCI wine dataset [148], perform round of train-test split, train 10 random forests models as described the main text to construct the lower probabilities, and then compute the epistemic uncertainty measurements for the test data. After that, we plot all possible pairwise comparison plots between the methods, i.e.MMI, MMI-Lin, Generalised Hartley, and Entropy difference. We that MMI and GH are highly correlated but not exactly perfectly correlated. The upper bound is also highly correlated, suggesting empirically (along with experiments in the main text), that this approximation is quite tight. Also, see that MMI and entropy differences do not correlate that well, which explains the difference in downstream performance in the experiments.
C.2 Overview of Generalised Hartley measures and Entropy Differences
We refer the reader to the recent survey by Hoarau et al. [123] on this topic. We hereby provide background on the two methods compared against in the main text.
37

---

VMNUTVI vs Ennney 0 ficrence 6.530
NeVILLTV) vs General sod Hortley (oor 9966)
WW vs www-inninnnnn Foor 0.97)
texd (teetee
5:44
0.10
0.0
6:34-
6:47
9.03
0.22
3.33
664
007
6d6
0.00
663
652
0.3
0.8
335
8.62
000
6.0
035
9,00
007
6.0
NMIEVI
508:Lan Li va Entropy Dfference (Core 0.60)
980--U va Generalaed Farrley (Car 2.00)
Cenera hed Harrtay Entropy Diference (Cor 0.4
6.16-
6:55
6014
0.44
6:04
0.12
662
2:00
0.03
0.32
0.84 0.06
0.22
6.04
0.00
0.00
8.02
0.02
8.03
3.54
0.5
0.00
MMHHUh
Gentera lard Hartley
Figure Comparing the UQ measurements on the withheld test set of the UCI wine dataset [148]. We see that MMI and GH are highly correlated, but not exactly perfectly correlated. The upper bound also highly correlated, suggesting empirically (along with experiments in the main text), that this approximation is quite tight. Finally, we see that MMI and entropy differences do not correlate that well.
Generalised Hartley Measure. Generalised Hartley measure, as the name suggests, is a generali- sation of the classical Hartley measure, which is defined as follows:
Definition 28 (Hartley Measure). Let X be finite. A Hartley measure Un is a function 2 b TR. such that Un (A) = log2 A| for A E x
The Hartley measure can thus be understood as measure of uncertainty over sets, which can also be viewed as function natural numbers. While the expression is simple, Rényi showed that the Hartley measure the only function mapping natural numbers to the reals that satisfies:
1. (m(m)) Un (m))++++)) + (n(n) (additivity)
2. Un(m)) (m) 2 Un(m + 1) (monotonicity)
3. Un = (normalisation)
These are natural conditions for measuring the amount of uncertainty, or equivalently, information, within set. The generalised Hartley measure extends this idea of measuring uncertainty in sets in the following way,
Definition 29 (Generalised Hartley [94]). Let X be finite. Given a lower probability Pe P(X), the generalised Hartley UGh maps P to R as follows:
Uc
Σ mp(A) (ogz())), ACX
where the mass function mp 2x + 10, 1] is the Möbius inverse of P, defined as,
m = (-11)(1A-118)) P(B).
BCA
is well known that the Möbius inverse an equivalent representation lower probability, in the sense that, once the mass function is known, we can recover P. by computing,
P(A) Σ m(B).
BCA
38

---

Entropy Differences. To measure aleatoric uncertainty for a given distribution P. the Shannon entropy an intuitive solution. The Shannon entropy, or simply entropy, measures the amount of information contained or provided by source of information.
Definition 30 (Shannon Entropy). Let X be finite. The Shannon entropy of distribution P is measured by
USA(P) = - ((t)))oog(((((()))
Now, when given set of probabilities, i.e. credal measure the amount of epistemic uncertainty, or in some literature, they call this non-specificity, one natural approach would be to measure the largest difference between the entropies of any two distributions within the set. Formalised below,
Definition 31 (Entropy Difference). Let X be finite and a credal set. The entropy difference is measured by,
max USh PEC
minn Ush (Q). QEC
This measure seems intuitive, but violates the monotonicity axioms that sensible credal UQ measure should satisfy. Consider a credal set C and Pı, P2 the distributions that attained the maximum and minimum of the entropies of the distributions in C. Now, enlarge C to C' by adding distributions that have strictly less entropy than P1 but more entropy than P2, then we have C C C', but the entropy differences will stay the same. This violates the monotonicity axioms that say, when credal strictly larger than the other, then the former should be deemed more epistemically uncertain than the latter.
Nonetheless, the entropy difference is still often used in practice as it is simple to compute, and be used to decompose 'total uncertainty' into 'aleatoric' and 'epistemic' components. See for example in Abellán et al. [100], the author defined,
max USs(P) PEC
min (Q)))) QEC
+ max USn (P) min Usn (Q)
PEL Epistemic Uncertainty
QEE
Uncertainty
Aleatoric Uncertainty
C.3 Implementation details
We provide full experimental details here, which were abbreviated in Section 6 due to space constraints. The experiments were executed on a machine with 8 vCPUs, 30 GB memory, with a NVIDIA V100 GPU.
We evaluate the performance of Maximum Mean Imprecision using a selective classification task, following the setup in Shaker and Hüllermeier [57] and Shaker and Hüllermeier [80]. Each dataset is split into training and test sets. For tabular datasets (the Obesity dataset from UCI and Digits dataset from Sci-kit Iearn), depth) on the same training set, and evaluate them on the test set. For image datasets (CIFARIO and CIFAR 100), use 10 pretrained neural networks per task, available https: thub com/ chenyaofo/ Seanyaofo/yytrrha. cifar- models. These models were trained using PyTorch's 's default CIFAR training sets, and evaluate the standard CIFAR test sets by dividing them into 10 buckets to introduce variability.
train 10 random forests with randomly chosen hyperparameters (c.g.. tree
For both tabular and image data, we use the centroid of the credal set as the predictor, similar to standard ensemble methods. The corresponding lower probability is computed by evaluating the most pessimistic likelihood across the set of predictions for each possible outcome.
D IPMM with epsilon-contamination set
In this section of the appendix, we continue from Section 3 and dive deeper into the connections between IIPM and the epsilon contamination model-a popular class of imprecise models.
39

---

D.1 Lower Probability Kantorovich Problem
In this subsection, we provide the background on a recently considered problem called the lower probability Kantorovich problem, proposed in Caprio [56]. We start by reviewing what a classical Kantorovich problem is.
Classical Kantorovich problem. Let P.Q. E P(t) be two probability measures on X. Let c: x. x X + R4 be a measurable cost function that gives us the cost of moving on unit of probability mass from the first argument to the other. Then the classical Kantorovich problem is the following optimisation problem,
{ c(te 2)da((;;)
arg inf inf
where [P, Q) is the set all joint probability measures whose marginals are P and Q. a is also denote as the transportation plan, and this is also famously known the optimal transport problem.
Lower Probability Kantorovich problem. Caprio [56] consider the following research question,
What does Kanotorovich's 's problem look like, when instead of transporting probability measures, we transport lower probabilities?
As his answer, Caprio [56] provided the following characterisation of the problem.
Definition 32 (Lower Probability Kantorovich's ' OT problem; LPK). Let c: X x + R4 be Borel measurable cost function. Given lower probabilities P and Q X, want to find the joint lower probability alpha on X x X that solves the following optimisation problem
arg inf (er(EQ)) U cxx, 2d)(()
where T(P, Q) is the collection of all joint lower probabilities on X x X whose marginals X are P and Q respectively.
While this might seem like a straightforward extension from the classical formulation, it is important note that for imprecise probability theory, there no unique way perform conditioning, which means extra care has to be taken into defining [[P. Q) In particular, they focus on a subset of the joint lower probabilities constructed from using geometric conditioning, and called the corresponding LPK problem restricted to such conditioning set the restricted LPK problem.
Later in Theorem 11 of [56], they managed to show that the restricted LPK problem coincides exactly with the classical Kantorovich for epsilon-contaminatee sets. We managed to recover this result in our Theorem 16 without needing consider any specific type conditioning. the future, we will investigate how could our result complements to their theory, perhaps allow them to consider other types of conditioning operations in IP.
D.2 Nonparametrie Estimator of TIPM with e-contamination set using kernel distance.
Now consider å E 10 two contamination levels, and distributions P. P(X) which we have i..d samples from. Specifically, let XI...., X, kid P and 71,... 7.m taking values in X. We are interested in quantifying the difference between the e-contaminated model of P. i.e. P.. with respect to the d-contaminated model Q i.e. Q₂
iid Q be random variables
A short overview of kernel distances (MMD). For generic spaces X, with iid samples from P and Q, popular class of non-parametric discrepancy estimator the maximum mean discrep- ancy (MMD) [67, 71]. Specifically, pick kernel function A X' x X' - R and consider the uniform ball in the corresponding reproducing kernel Hilbert space (RKHS), i.e. Fx = {ʃ Hy s.t. ||/ = 1}. where || ||& stands for the RKHS norm. The MMD can be expressed as IPM with respect to
40

---

function class Fk. HIMDx(P.Q) - IPM, (P.Q)
/EH+illfll==1 {1ʃ sap - fooa
sup
M { (s. M(X)))P(X); - Us, / ( (icabp sup seHiil flk==1
/ A(X, ((X))))(())) feHuillfll m1 (5. M(X,) /
sup
(i(x)))())))-)) / 
/ A(X ((xx)) -
(B)
|pp ا |
where in step (A) we first use the reproducing property of RKHS functions, (f(z) = (f,, k(a, 1))) and then use the linearity of the inner product to 'push' the expectation inside. In step (B) we use the fact that inner product is maximised when the two vectors align, so we pick the unit-norm function to be
f J [d(x) ((X, (XQ
((((((X)))d()) f(X, IxQ
Finally, in step (C), we simply write the (Bochner) integral of the feature representation into more familiar-looking expression, called the kernel mean embedding [149, 150]. This simple expression facilitates further simplification, i.e.
(M = ||up- - =
(uP uQQ MP mQ)
DD)
E (x.x'Plk((., X')] 2I, X 2)] + E (k22, Z.
Meaning that don't need parametric assumption how the data is distributed, long have a way to define similarity between instances through a kernel function, we can estimate the difference between the distributions based on their samples by estimating the expectations in step D. Furthermore, for a wide class of kernels, known as charactersitic kernels, the mapping from P P J k(X, .dP(() is injective, thus MMD, is proper metric between probability distributions. Owing to its simplicity, kernel mean embeddings and MMDs have been used to tackle a broad range of statistical tasks, ranging from hypothesis testing [7 to parameter estimation [151, 152], causal inference [153, 154], feature attribution [155, 43], and learning on distributions [156, 157].
Generalising to contamination sets. In general, given lower probability P. the notion of samples from P is ill-defined as lower probability often is used to encode subjective assessment rather than describing the data- generating process. As such, devising sample-based estimator for the Choquet integral, akin to Monte Carlo estimation for the Lebesgue integral, is not yet possible. Nonetheless, in the case of utilising lower probability constructed through an epsilon contamination model, this is possible.
Recall ,, E 10, are two contamination level, with Pe and Q2 the corresponding contaminated models. We are now interested in quantifying the difference between this two lower probabilities using the IIPM framework through the kernel distances. As before, pick Fx = (s E He s.t. ||/l| = 1}. then we have
{ fap, - -f fde,
IPM (P.:
sup feH
11/- (2.(1f> - - E))) dt
sup feHuillfll
(8)
{ (1 (1-c) fdP fdd- (1 6) 5doo
sup fEHaill/la
(18)
||(1-c)pp (1- 18)pplla
41

---

where in step (i) we simply expand our the definition of Choquet integral. In step (ii), we follow Lemma and the proof of Theorem 16 express the Choquet integral now weighted Lebesgue integral. In step (iii), we follow the derivations of standard MMD provided in the previous paragraph. Subsequently, the square of IIPM,, (P.+ Q2) can be expressed as,
IPMMF (L) (1 (1- "
2(1 -
(KX, + (z Z')
This allows then construct non-parametric (unbiased) estimator of (the square of IIPM, (L., Q₂) as follows,
IPM, (P. = -1 c) c)² n(n 1) im j=1.jşi
Xy)
2(1 - c) - 8) mm i=
(X; Zz) + 8)2
Σ Σ
Zz)
Due project scope, did not further investigate the concrete applications such non-nonnaann non-paramctric worst-case probability discrepancy estimator, but in future work, we will explore its use case in robust two-sample testing, akin Schrab and Kim [158], in generative model training, akin Li al.
[131].
42


Reviews:
[{"review_id": "IYdUmbJWJt", "reviewer": "Reviewer_HsWH", "summary": "The paper discusses IIPMs, a generalization of IPMs that operate on capacities (a class that include non-additive measures). This is a theoretical paper that provides for a sound basis for properties such as (a) suitably chosen IIPMs are proper metrics, (b) new/generalized distances such as Lower Dudley and Lower TV, (c) re-derive a optimal transport result, (d) Maximum Mean Imprecision (MMI, a measure of epistemic uncertainty/EU which also is validated axiomatically) etc. Empirical evidence is also provided to compare with traditional uncertainty quantification measures. A linear time upper bound MMI-Lin is also proposed here.", "questions": "Is it possible to get one dataset where EU ground truth is available (semi=synthetic) so that the uncertainty measures can be directly compared?", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "The authors provide a principled investigation and derivation of key results for IIPMs and placing them very nicely relative to classical IPMs.\nThe introduction of MMI as a Epistemic Uncertainty quantification measure is interesting (including its linear-time compute variant)\nThe paper is well motivated.\nIn terms of weaknesses, practicality in real world ML settings is not fully fleshed out (understandably so given the page limits). MMI-Lin is a great starting point, but not addressing these practical gaps (e.g., how to use in regression/structured prediction etc) might limit their value to theoretical interest only.", "was_revised": true, "final_justification": "The responses to concerns raised in the review were satisfactory.", "rebuttal": "We thank the reviewer for their insightful feedback.\n\n**Q1: “In terms of weaknesses, practicality in real-world ML settings is not fully fleshed out (understandably so given the page limits). MMI-Lin is a great starting point, but not addressing these practical gaps (e.g., how to use in regression/structured prediction etc) might limit their value to theoretical interest only.”**\n\nA1: We appreciate the reviewer’s thoughtful comment and fully agree that extending the MMI framework to regression and structured prediction tasks would further broaden its practical impact. Indeed, we are actively investigating these directions beyond the current submission.\n\nLet us emphasise that the theory presented in Section 3 is not limited to discrete spaces—it extends naturally to continuous state spaces, as demonstrated via our use of optimal transport and the epsilon-contaminated MMD framework in Appendix D.2.\n\nSince adapting MMI to regression or structured prediction problems would require the development of additional computational tools and problem-specific modeling choices, we defer it to future work.  Nonetheless, we see this as a natural and exciting avenue for future work, and will clarify the practical use in regression/structured prediction in the camera-ready version. \n\n-----\n**Q2: “Is it possible to get one dataset where EU ground truth is available (semi=synthetic) so that the uncertainty measures can be directly compared?”**\n\nA2: We thank the reviewer for this insightful suggestion. We would like to highlight a fundamental limitation in the evaluation of epistemic uncertainty (EU) that has been established in recent literature. In particular, Bengs et al. [1] show that it is provably impossible to construct a proper scoring rule (i.e., a loss function) that allows for the direct supervision or learning of second-order uncertainty from zeroth-order data (i.e., datasets consisting only of input-output pairs). This result underscores a key epistemological challenge: epistemic uncertainty is inherently about the limitations of knowledge, and not directly observable or labelable from standard supervised data.\n\nAs a result, the epistemic uncertainty quantification (UQ) community has largely moved away from using “ground-truth”-based or accuracy-style metrics, which may be appropriate for aleatoric uncertainty but are not well-justified for epistemic UQ. Instead, the standard approach is to evaluate epistemic UQ methods by their utility in downstream tasks where uncertainty matters—such as selective classification, robust decision-making, or exploration—rather than by ground-truth alignment.\n\nIn line with this principle, our evaluation focuses on selective classification as a practical proxy for assessing the value of uncertainty estimates in real-world scenarios. We believe this reflects a more meaningful and principled evaluation of epistemic UQ methods.\n\nReference:\n[1] Bengs, Viktor, Eyke Hüllermeier, and Willem Waegeman. “On second-order scoring rules for epistemic uncertainty quantification.” International Conference on Machine Learning. PMLR, 2023.\n\n---------\n\nWe hope our responses have added clarity regarding the motivation and applicability of our current work. We kindly ask the reviewer to possibly reconsider their score in light of our discussion. Thank you for your consideration."}, {"review_id": "3hVOjmUbzY", "reviewer": "Reviewer_RsrW", "summary": "This paper highlights the need for a metric to compare imprecise probabilities and introduce a metric called Integral Imprecise Probability Metric (IIPM) framework, a Choquet integral-based generalisation of classical Integral Probability Metrics (IPMs) to the setting of capacities. Based on IIPM, a new epistemic uncertainty measure, Maximum Mean Imprecisions, is also introduced. The theoretical conditions for IIPM are established while MMI is also empirically validated.", "questions": "See weaknesses", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n-\tThe paper comes with strong mathematical setting and presentation of the proposed metric.\n-\tThe metric shows its generalized nature and how it devolves into other popular metrics.\n\nWeaknesses:\n-\tThe paper lacks in experimental evaluation. A widely used way to evaluate epistemic uncertainty is by Out-Of-Distribution (OOD) detection. It might be interesting to see how MMI performs there.\n-\tThe metric requires ensembles of models to compute lower probabilities. This, by default, makes it more resource and time expensive. How does this compare to other available metrics?\n-\tWhat effect does the ensemble size have on metric’s performance?\n-\tIt might be nice to add some visual examples to show the efficacy of the metric and what kind of numbers it produces on different inputs.", "was_revised": true, "final_justification": "Some of my concerns about experimental validation were not addressed but I maintain my score.", "rebuttal": "We thank the reviewer for pointing out the typos and for their valuable suggestions. We agree that exploring the interplay between ensemble size and the quantified uncertainty can provide further insight, and we will include an ablation study on this aspect in the camera-ready version.\n\nFrom a theoretical standpoint, we already know that as we increase the size of the ensemble while keeping the rest of the elements the same, the resulting credal set could only expand or remain unchanged. This implies that the lower probability becomes more conservative (i.e. smaller or unchanged), and by the monotonicity axiom (A3), the quantified EU will increase or stay the same. This property aligns with the intuition that adding diverse hypotheses should reflect greater epistemic caution. \n\nWe believe this ablation will help clarify further. Thank you for your suggestion."}, {"review_id": "vefG1A8QqG", "reviewer": "Reviewer_ph2P", "summary": "In this paper, the authors propose to a family of statistical distances for imprecise probabilities, as well as the framework Integral Imprecise Probability Metrics (IIPM). Specifically, it leverages the Choquet integral to generalize classical integral probability metrics, supporting comparison across different IP models and quantifying epistemic uncertainty within a single IP model. The authors also provide the theoretical basis with a form of weak convergence of capacities. Based on IIPM, a specific uncertainty measure, Maximum Mean Imprecise (MMI), can be derived to quantify epistemic uncertainty in classification tasks. Empirically, the authors validate MMI with experiments on several benchmark datasets and demonstrate that MMI outperforms baseline uncertainty quantification methods.", "questions": "Q1: As mentioned in Section 7, a key limitation is the lack of a sampling-based estimation method for IIPMs. Would there any future plan to address this?\n\nQ2: What is the trade-off and the key of using the efficient upper bound (or `MMI-Lin`)? It would be great to have some elaboration.\n\nQ3: Would the proposed framework be able to be applied to other ML tasks like regression and ranking? If so, it would also be great to have more experiments to validate.\n\nQ4: There are some studies like [a] to leverage uncertainty measures to improve the model quality. Can the proposed framework also do that? It would also be great to have some studies and discussions about this direction.\n\n\n[a] Jiang, J. Y., Chang, W. C., Zhang, J., Hsieh, C. J., & Yu, H. F. (2023, July). Uncertainty Quantification for Extreme Classification. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1649-1659).", "limitations": "In Section 7, the authors have discussed about several limitations of the work, although there is no discussion about negative societal impact.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1. IIPM serves a novel framework to systematically apply Choquet integrals to compare capacities in imprecise probability.\n2. Robust theoretical basis and creditability of IIPM\n3. Empirical experiments on benchmark datasets demonstrate the effectiveness of MMI based on IIPM\n\n### Weaknesses\n1. Lack of sampling-based estimation\n2. Experiments limited on classification tasks and specific datasets\n3. Scalability and efficiency for large-scale problems\n4. Lack of experiments of further utilizing uncertainty measures to improve model peformance", "was_revised": true, "final_justification": "After reading the rebuttal and discussions, I keep my score and recommend borderline accept because of some unresolved limitations, despite there are some contributions in the proved theory and experiments.", "rebuttal": "We thank the reviewer for their insightful and thorough feedback. We will address your questions below:\n\n-----\n\n**Q1: As mentioned in Section 7, a key limitation is the lack of a sampling-based estimation method for IIPMs. Would there be any future plan to address this?**\n\nA1: Yes, we are working on extensions of this work to overcome this problem. One natural direction we are trying out is to utilise IIPM to find the closest precise probability model to a given imprecise probability, and perform sampling from the precise model. Of course, understanding the estimation quality, in this case, is highly non-trivial; therefore, we believe it is outside the scope of this work.\n\n-----\n\n**Q2: What is the trade-off and the key of using the efficient upper bound (or MMI-Lin)? It would be great to have some elaboration.**\n\nA2: In lines 332-336, we provided a sketch proof of Proposition 21. We are happy to clarify it further in the camera-ready version.\nIn terms of trade-off, we have performed an ablation study in the appendix (Figure 3), which empirically looked at the difference between the actual MMI and MMI-Lin. While there are some gaps between these numbers, in our selective classification experiments, we could not find any performance difference. \n\n-----\n\n**Q3: Would the proposed framework be able to be applied to other ML tasks like regression and ranking? If so, it would also be great to have more experiments to validate.**\n\nA3: As mentioned in our limitation section, extending it to problems such as regression would require developing further computational tools to evaluate the Choquet integral in a real line. Given that the goal of the current paper is to establish a strong mathematical groundwork for the metric and to justify its relevance to the ML community through the EU quantification problem, we decided to leave the computational development for the regression problem as future work.\n\n----\n\n**Q4: There are some studies like [a] to leverage uncertainty measures to improve the model quality. Can the proposed framework also do that? It would also be great to have some studies and discussions about this direction.**\n\nA4: We thank the reviewer for pointing this reference out. We will add a brief discussion on this, including the suggested reference, in the camera-ready version.\n\n----\n\nWe hope our responses have added clarity regarding the motivation and applicability of our current work. We kindly ask the reviewer to possibly reconsider their score in light of our discussion. Thank you for your consideration."}, {"review_id": "vBrDQgOFa1", "reviewer": "Reviewer_6KfT", "summary": "This paper introduces imprecise integral probability metrics. Such metrics are a generalization of classical integral probability metrics to compare two distributions, such as the Wasserstein and total variation distance. The imprecise integral probability metrics allow to define a distance over capacities instead of probability measures. As a second contribution, the new metric not only allows to compare two imprecise probability models, but also to quantify epistemic uncertainty in a single imprecise probability model, using the notion of maximum mean imprecision (MMI). In a selective classification experiment that involves computing classification-rejection curves on two UCI and tow image benchmark datasets, the MMI is compared to two baselines: the entropy-based approach to quantify epistemic uncertainty and the generalized Heartley measure.", "questions": "See above.", "limitations": "Limitations are briefly discussed in Section 7.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "My knowledge of imprecise probability theory is very limited, so I find it hard to give constructive feedback on this paper. \n\nDespite my limited knowledge on the topic, I could follow Sections 3 and 4, which constitute the theoretical contributions of this paper, quite well. The authors are able to discuss the novel concepts in a mathematically rigorous manner, which is appreciated by this reviewer. The authors also give evidence of a very good knowledge of the literature on imprecise probability theory (the paper contains 135 references!). To the best of my knowledge, the introduced concepts are also novel. \n\nThe discussion is less clear in Section 6, where the application of the new concepts for ML is showcased on benchmark datasets. That section is very short and some essential information is missing. First of all, how is epistemic uncertainty computed for the baselines in the setting that you consider? I guess that, for the entropy-based method, this is similar to reference [77]. For the generalized Heartley measure, do you consider the volume of the convex hull formed by the ensemble members? For the method presented by the authors, it is also not clear to me how epistemic uncertainty is computed. How do you relate the formulas in Definition 17 or Proposition 18 to the convex hull of the ensemble members? This might be discussed in appendices, but it should be discussed in the main paper (sorry, I have to review 6 papers for Neurips, so I don't have time to read appendices). \n\nIn the experimental setup, it was not so clear to me why the authors only focus on epistemic uncertainty. One can also reject certain instances because of aleatoric uncertainty. That would be a different type of rejection, but it would be interesting to see how this relates to epistemic uncertainty. \n\nFinally, it is a subjective opinion, but I am not sure whether Neurips is a good venue for this type of work. The key contributions are mainly in imprecise probability theory and most references in the paper also come from that field. Furthermore, only the contributions of Section 4 are applicable to ML models and the experiment in Section 6 is quite small.", "was_revised": true, "final_justification": "I thank the authors for a detailed response to my comments. In general I remain on the positive side for this paper.", "rebuttal": "We thank the reviewer for their thoughtful and genuine comments about the clarity of the paper. We address your concerns below:\n\n------\n\n**Q1: Discussion is less clear in Section 6.**\n\nA1: We thank the reviewer for pointing out the relative lack of depth in the discussion of Section 6 compared to the theoretical and methodological content in Sections 3 and 4. As correctly noted, we have provided extensive additional experimental details in Appendix C. Specifically, Section C.1 presents an ablation study comparing the three credal-based uncertainty measures; Section C.2 offers a detailed exposition on the Generalised Hartley and entropy differences; and Section C.3 describes the full experimental setup used in the main text.\n\nWhile we would have liked to include these details in the main body, we made a conscious decision to prioritise the clarity and accessibility of our theoretical contributions—particularly the new theorems and propositions. Given that the experimental framework for evaluating epistemic uncertainty quantification via selective classification is well-established and used in prior work, we opted to place the implementation specifics in the appendix to maintain a focused narrative in the main text.\n\nAs we will have one extra page in the camera ready, we will move some of the details in Appendix C.3 to the main text to make the main text more self-contained. \n\n------\n\n**Q2: “In the experimental setup, it was not so clear to me why the authors only focus on epistemic uncertainty. One can also reject certain instances because of aleatoric uncertainty. That would be a different type of rejection, but it would be interesting to see how this relates to epistemic uncertainty.”**\n\nA2: We agree it is interesting to check whether we should reject instances based on low EU or high AU; However, this experiment is not too relevant to the main purpose of this paper: which is introducing a new metric for IP models and to quantify the amount of imprecision (EU) therein. \n\nAlso, this particular research question has been studied in [2], where the authors empirically demonstrate that it is generally better to consider EU rather than AU in these selective classification problems. We think this is case-specific, as logically if you have a label with very high AU, there is little signal for one to make an accurate prediction anyway, so we should defer making a prediction at this instance. \n\n[2] Nguyen, Vu-Linh, Sébastien Destercke, and Eyke Hüllermeier. \"Epistemic uncertainty sampling.\" International Conference on Discovery Science. Cham: Springer International Publishing, 2019.\n\n------\n\n**Q3: Finally, it is a subjective opinion, but I am not sure whether Neurips is a good venue for this type of work.**\n\nA3: The reviewer’s concern is understandable. While imprecise probabilities (IP) are not yet a mainstream topic in the machine learning community, their foundational framework for modeling ambiguity and imprecision offers principled tools that can be adapted to address challenges in modern machine learning. This is evidenced by a growing number of IP-related papers published at top ML venues, including ICLR, AISTATS, ICML, and NeurIPS. We surveyed several such works from 2023 to 2025, spanning various IP frameworks (e.g., credal sets, belief functions, upper/lower probabilities), and listed them below. Our proposed IIPM and MMI methods are, in principle, compatible with these frameworks. We hope this provides some evidence that our contributions to IP-based metrics and uncertainty quantification are timely and relevant to the broader ML community.\n\n1. Sale, Yusuf, Viktor Bengs, Michele Caprio, and Eyke Hüllermeier. \"Second-order uncertainty quantification: a distance-based approach.\" In Proceedings of the 41st International Conference on Machine Learning, pp. 43060-43076. 2024.\n2. Chau, Siu Lun, Antonin Schrab, Arthur Gretton, Dino Sejdinovic, and Krikamol Muandet. \"Credal Two-Sample Tests of Epistemic Uncertainty.\" In International Conference on Artificial Intelligence and Statistics, pp. 127-135. PMLR, 2025.\n3. Singh, A., Chau, S.L., Bouabid, S.; Muandet, K.. (2024). Domain Generalisation via Imprecise Learning. ICML 2024\n4. Zhou, Zi-Hao, Jun-Jie Wang, Tong Wei, and Min-Ling Zhang. \"Weakly-Supervised Contrastive Learning for Imprecise Class Labels.\" ICML 2025\n5. Javanmardi, A., Stutz, D., & Hüllermeier, E. (2024). Conformalized credal set predictors. Advances in Neural Information Processing Systems, 37, 116987-117014.\n6. Marinescu, Radu, Junkyu Lee, Debarun Bhattacharjya, Fabio Cozman, and Alexander Gray. \"Abductive reasoning in logical credal networks.\" Advances in Neural Information Processing Systems 37 (2024): 67968-67986.\n7. Wang, Kaizheng, Fabio Cuzzolin, Keivan Shariatmadar, David Moens, and Hans Hallez. \"Credal deep ensembles for uncertainty quantification.\" Advances in Neural Information Processing Systems 37 (2024): 79540-79572.\n8. Caprio, Michele, Maryam Sultana, Eleni Elia, and Fabio Cuzzolin. \"Credal learning theory.\" Advances in Neural Information Processing Systems 37 (2024): 38665-38694.\n9. Eberle, Franziska, Felix Hommelsheim, Alexander Lindermayr, Zhenwei Liu, Nicole Megow, and Jens Schlöter. \"Accelerating matroid optimization through fast imprecise oracles.\" Advances in Neural Information Processing Systems 37 (2024): 93790-93825.\n10. Chen, Hao, Ankit Shah, Jindong Wang, Ran Tao, Yidong Wang, Xiang Li, Xing Xie, Masashi Sugiyama, Rita Singh, and Bhiksha Raj. \"Imprecise label learning: A unified framework for learning with various imprecise label configurations.\" Advances in Neural Information Processing Systems 37 (2024): 59621-59654.\n11. Li, Tongxin, et al. \"Safe exploitative play with untrusted type beliefs.\" Advances in Neural Information Processing Systems 37 (2024): 72901-72932.\n12. Shai, Adam, Lucas Teixeira, Alexander Oldenziel, Sarah Marzen, and Paul Riechers. \"Transformers represent belief state geometry in their residual stream.\" Advances in Neural Information Processing Systems 37 (2024): 75012-75034.\n13. Bramblett, Daniel, and Siddharth Srivastava. \"Belief-state query policies for user-aligned POMDPs.\" Advances in Neural Information Processing Systems 37 (2024): 52776-52805.\n14. Wang, Kaizheng, Fabio Cuzzolin, Keivan Shariatmadar, David Moens, and Hans Hallez. \"Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification.\" In The Thirteenth International Conference on Learning Representations.\n15. Hu, E.S., Ahn, K., Liu, Q., Xu, H., Tomar, M., Langford, A., Jayaraman, D., Lamb, A. and Langford, J., The Belief State Transformer. In The Thirteenth International Conference on Learning Representations.\n16. Avalos, Raphaël, Florent Delgrange, Ann Nowe, Guillermo Perez, and Diederik M. Roijers. \"The Wasserstein Believer: Learning Belief Updates for Partially Observable Environments through Reliable Latent Space Models.\" In The Twelfth International Conference on Learning Representations.\n\n--------\n\nWe hope our responses have added clarity regarding the presentation and the relevance of our work to the broader ML community. We kindly ask the reviewer to possibly reconsider their score in light of our discussion. Thank you for your consideration."}]

Author rebuttal:

Meta-review: - Summary: This paper proposes Integral Imprecise Probability Metric framework, which is an extension of integral probability metrics to operate on capacities, instead of probability measures. It establishes the conditions when the proposed metrics are valid. The authors also propose Maximum Mean Imprecisions to enable epistemic uncertainty quantification.
- Strengths:
  - The framework is motivated by robust theoretical analysis.
  - The proposed method is novel and the paper is well written and motivated.
- Weaknesses:
  - The application of the approach to practical settings remains unexplored in addition to its scalability to large scale ML problems.
  - The relevance of the problem to the Neurips community is highlighted as a potential limitation.
  - Empirical evaluations can be stronger. 
    - Reviewer HsWH suggested comparisons with respect to simulated settings where empirical uncertainty is controlled. Authors provided their motivation on why such experiments are not ideal. Even though I acknowledge that there are limitations, I believe the suggested experiments remain useful. Similarly, measuring utility in downstream tasks might be also subject to limitations.
    - Reviewer RsrW suggested OOD detection, which is not addressed during the rebuttal phase by the authors.
- Suggestions:
  - I think focusing on the empirical evaluations and expanding the experiments to cover more downstream tasks as well as simulated settings would make this paper stronger and help with its wider adoption.
- Recommendation:
  - All of the four reviewers recommended acceptance of the paper, two of whom gave borderline acceptance scores.
  - I believe this paper will be a good contribution to Neurips 2025. The paper tackles uncertainty quantification, which is a significant research problem. All of the reviewers acknowledge the strong theoretical foundations of the proposed approach.
Decision: Accept (poster)