Conformalized Credal Set Predictors
Authors: Alireza Javanmardi
David Stutz
Eyke Hüllermeier
Venue: NeurIPS 2024 poster (2024)
Keywords: Conformal Prediction
Credal Sets
Imprecise Probabilities
Uncertainty Representation
Uncertainty Quantification
Primary area: probabilistic_methods
TL;DR: A novel conformal prediction method is introduced to construct credal sets that are able to represent both the aleatoric and epistemic uncertainty in a prediction.

Abstract: Credal sets are sets of probability distributions that are considered as candidates for an imprecisely known ground-truth distribution. In machine learning, they have recently attracted attention as an appealing formalism for uncertainty representation, in particular, due to their ability to represent both the aleatoric and epistemic uncertainty in a prediction. However, the design of methods for learning credal set predictors remains a challenging problem. In this paper, we make use of conformal prediction for this purpose. More specifically, we propose a method for predicting credal sets in the classification task, given training data labeled by probability distributions. Since our method inherits the coverage guarantees of conformal prediction, our conformal credal sets are guaranteed to be valid with high probability (without any assumptions on model or distribution). We demonstrate the applicability of our method on ambiguous classification tasks for uncertainty quantification.

Conformalized Credal Set Predictors
Alireza Javanmardi
David Stutz
LMU Munich, MCML. Munich, Germany
Max Planck Institute for Informatics
Saarbrücken, Germany
al irizzaa. ja javanarddiaii 1mm
david. stutz@mpi- .mp...p......................... de
Eyke Hüllermeier LMU Mumich, MCML Munich, Germany eyke@ifi 1mu de
Abstract
Credal sets are sets of probability distributions that are considered as candidates for an imprecisely known ground-trute distribution. In machine learning, they have recently attracted attention as an appealing formalism for uncertainty representation, in particular, due their ability to represent both the aleatoric and epistemic uncertainty in a prediction. However, the design of methods for learning credal set predictors remains challenging problem. In this paper, we make use of conformal prediction for this purposc. More specifically, for predicting credal sets the classification task, given training data labeled by probability distributions. Since method inherits the coverage guarantees of conformal prediction, our conformal credal sets are guaranteed be valid with high probability (without any assumptions model or distribution). We demonstrate the upplicability of our method on ambiguous classification tasks for uncertainty quantification.
propose method
Introduction
Representing and quantifying uncertainty becoming increasingly important in machine learning (ML), particularly as ML models are employed in safety-critical application domains such as medicine or autonomous driving. such domains, distinction between so-called aleatoric uncertainty (AU) and epistemic uncertainty (EU) is often useful [21]. Broadly speaking, aleatoric uncertainty is due to the inherent randomness of the data-generating process, whereas epistemic uncertainty stems from the learner's 's lack of knowledge about the best predictive model. Thus, while the former is irreducible, the latter can, in principle, be reduced through additional information, e.g., by gathering additional data to learn from.
Representation of aleatoric and epistemic uncertainty requires formalism more expressive than standard probability distributions [22]. One such formalism which prevails in the recent ML literature second-order probability distributions. Essentially, in a classification setting, these are distributions over distributions over classes. Models producing second-order distributions predictions can be learned classical Bayesian way [16, 25] or using more recent approaches such as evidential deep learning [44]. Yet. approaches of that kind not unproblematic and have been subject to criticism [8, 9J. Specifically, such approaches have been shown to misrepresent epistemic uncertainty. Another formalism suitable for representing both types of uncertainty is the concept of credal set, which well-established in the field of imprecise probability theory [58] and meanwhile also attracted attention in ML [23, 46). Credal sets (convex) sets of probability distributions that can be considered as candidates for an imprecisely known ground-trute distribution.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

class
class
class
class 2.
class 3
class 2
class 3
class 2
class 3
Figure I: For the three-class classification setting, the space of probability distributions can be illustrated by a two-dimensional simplex: each point in the simplex corresponds to a probability distribution so that credal sets be depicted as regions. The left case corresponds to the special case singleton (credal) set, i.e., a precise probability distribution, signifying aleatoric but no epistemic uncertainty. The in the middle represents partial knowledge with certain degree of (epistemic) uncertainty about the true distribution, and the right one corresponds to the case of complete ignorance, where nothing is known about the distribution.
Figure shows examples of credal sets in three-class scenario, where the space of distributions can be visualized by the two-dimensional probability simplex. Broadly speaking, the larger the credal set, the higher the epistemic uncertainty, and the more "in the middle" the set located, i.e., the closer is to the uniform distribution, the higher the aleatoric uncertainty.
Learning predict second-order representations, such credal sets aecond-orde distributions, from standard "zero-order" data - training instances together with observed class labels - has been shown to be difficult in that it typically provides biased estimates of epistemic uncertainty [8. 9]. To alleviate this problem, we assume "first-order" training data, i.e., instances associated with probability (frequency) distributions over the class labels. In other words, instances are labeled probabilistically instead of being assigned a deterministic class label. This type of data becoming increasingly available in practice, for example, in the form of aggregations over multiple annotations per data instance [10, 31, 35, 63), and hence increasingly relevant in many applications [52, 53J. Moreover, such data facilitates second-order learning.
In this paper, we leverage conformal prediction (CP), a non-parametric approach for set-valued prediction rooted in classical frequentistt statistics [57], to construct credal sets. With relatively mild assumptions similar to those in CP, this approach inherits the so-called marginal coverage from CP: Predicted sets are guaranteed to cover the true target with high probability. For us, this means that we can use any first-order or second-order predictive model to construct unbiased credal sets equipped with such coverage guarantee. Specifically, we propose various nonconformity functions applicable to first- and second-orde predictors to construct conformal credal sets. We also study the case where only a noisy version of first-order distributions is available, demonstrating that the coverage guarantee holds under a bounded noise assumption. On ChaosNLI [31], an ambiguous natural language inference task with multiple annotations per example, show that our conformal credal sets are valid, i.c., covering the true ground-trute distribution with high probability while comparing the efficiency of different nonconformity functions. Together with experiments on CIFAR 10-H [35]-a variant of the CIFAR 10 test set containing class distributions from human annotations we demonstrate how these credal sets enable practical quantification of aleatoric and epistemic uncertainty. We further complement this study with controlled experiments on synthetic data, specifically investigating the performance of credal set prediction in the presence of noisy data.
2 Background
2.1 Supervised Learning and Predictive Uncertainty
We consider the setting of (polychotomous) classification with label space y = {1,.... K) and an instance space X. As usual, we assume underlying data-generating process in the form of a probability distribution P on X' x V, so that observations (X, Y' are i.i.d. samples from P. We denote by A = (Af.... x E K the conditional probability distribution P(: X = æ). which we also consider as an element of the (K 1)-simplex A Thus, the probability to observe Y outcome for X is given by Ag

---

Since the dependency between instances X and outcomes Y is non-deterministic, the prediction of Y given X x is necessarily afflicted with uncertainty, even if the ground-truth distribution X is known. As already said, this uncertainty is commonly referred to as aleatoric [22]. Intuitively. the closer X _ to the uniform distribution Pusi = (1/K,... 1/ k the higher the uncertainty, and the closer it is to degenerate (Dirac) distribution assigning all probability mass to a single class (a corner point in (), ), the lower the uncertainty.
Instead of assuming A to be known, suppose now that only a prediction Ả of this distribution is available. Epistemic uncertainty refers to the uncertainty about how well the latter approximates the former, and hence to the additional uncertainty in the prediction of outcome Y that is caused by the discrepancy between X and the ground-truth X" We seek to capture this discrepancy by means of credal sets Q QK C A K with the idea that Q è A holds with high probability. Typically, credal sets are assumed to be convex, and further restrictions might be imposed on Qx for practical and computational reasons, for example, a restriction to convex polygons (with a finite number of extreme points).
2.2 Conformal Prediction
Conformal prediction provides a general framework for producing set-valued predictions with certain guarantee of validity. In a supervised setting, consider data points of the form Z = (X, E X' to be equipped X' x Ut and the task is to predict U given = æ. We assume the space with nonconformity measure f: E - TR. that quantifies the "strangeness" of z, i.e., the higher f()), the less normal or expected the data point. Let Dealib C. Z be a (randomly generated) set of data points, called calibration data, and Z another data point that remains unobserved. Under the assumption of exchangeability, i.c., that the calibration data and the query point Z have been generated by an exchangeable process, seek so-called confidence set C C l that guarantees coverage:
PIU E C) 2 1 a.
By simple combinatorial argument [57], the confidence set C' can be constructed
C(x) = u E 4 [(x, u) S {EE
where 8 = (f(x) : 2 E Dealt) } is the set of nonconformity scores, a' = | 1 [)) a, and a() denotes the a' -quantile of E. Importantly, the guarantee (1) holds regardless of the nonconformity function f(-), which, however, has an influence on the efficiency of the prediction: The more appropriate the function, the smaller the prediction set C' tends to be. Normally, f(.) is not predefined but constructed in a data-driven way using training data D For example, common approach is to train a predictor æ : X - U and then define f(x u) in terms of d(u, x (x)),, where .(.,.) is an appropriate distance function on U. Replacing the point-prediction क (x) E U by the prediction set C(x) C U can then be seen as "conformalizing" the predictor
3 Conformal Credal Set Prediction
Our goal is to learn a credal set predictor h X Qk. that is, model that makes predictions in the form of credal sets, thereby representing both alcatoric and epistemic uncertainty. To this end, we assume access to first-order data, i.c., probabilistic training data of the form
D (z A A² ) (NN,, )²x) C X A
The model h should be able to predict the (probabilistic) outcomes for new query instances in reliable way. More specifically, suppose that Nnew is a new query instance (following the same distribution as the training data) for which prediction is sought. To correctly represent epistemic and aleatoric uncertainty, we want the credal prediction Q = h (new) to be valid, meaning Q M X The with high probability, while at the same time being informative such that the (epistemic) uncertainty reflected by Q is as small as possible.
We aim to construct the credal set predictor h by means of conformal prediction. Following the conformalization recipe outlined Section 2.2, we partition into Dirain and D Dealih, using the former for model training and the latter for calibration.
3

---

Algorithm Conformal Credal Set Prediction
Input:
Data D; error rate a; query instance dnew-
Process:
Partition D into Dasin and Peait.
Train first-order (i 1) a second-order 2) predictor using Puain- Choose a nonconformity function fi as in (5) or (8) that suits the trained predictor to obtain the set of scores & Set o'
|&.1_ [(1 (&))(1 a)].
Output:
(TTze)
{ AX A [i A) S q(Ei,a)))
Regarding the training step, we explore two learning strategies connected with two ways of defining a nonconformity function, which is pivotal in the calibration step. The first approach is based on training standard (first-order) probability predictor, i.c., probabilistic classifier g X - A that maps instances to the (first-order) probability distribution on ) This can be achieved, for example, by minimizing the cross cross-entropy loss between the ground truth and the predicted distributions, i.e.,
K
= argmin gEH
Σ
n. log(g(z;))))
k=1
where H is a hypothesis space. Given predictor g(-) of this kind, nonconformity is naturally defined in terms of distance:
(1 x²) d(x", (())),
where .): is a suitable distance function on A K such as total variation, Wasserstein distance, etc.
An alternative approach is motivated by recent work on (epistemic) uncertainty representation via second-order probability distributions [16, 25, 44]. A second-order learner G X - IP( (P(AK) maps each input to distribution over A Given the training data of the form (3), meaningful learning this context can accomplished, for instance, by parameterizing the second-order distributions using Dirichlet distributions. Specifically, distribution characterized by the parameter vector 8 E R.K with the probability density function
assume that each is associated with Dirichlet
K
0
II
1-1 A
(6)
B
where B(-) is the multivariate beta function. This way, A can be thought of as a sample from that distribution, i.e., X" Dir( The model aims to find parameter vectors 0 s that minimize the negative log-likelihood loss
Σ (E)
log log(B(02*)
(0 1) log(A
Given second-order predictor 0 nonconformity can be defined decreasing function of likelihood, e.g., as minus relative likelihood:
P( 8Th
fa A²) = max P(A (0)
(8)
Using the nonconformity function fi(-) (i (1) we obtain the set of nonconformity scores by
: = fi \a)) (æ,, A²)) E Deali))
(9)
Accordingly, the credal set can be defined as
hy (W
A [[ A) S (EE,
(10)
Algorithm outlines summary of the proposed methods. In the following theorem, we state the validity of the predicted set, that is, the restatement the conformal coverage guarantee [57] adjusted to our setting.

---

Theorem 3.1 (Validty Gaurantee), Let P denote the joint probability distribution on (X, A) E X AK If data points in Deulih and (Fuew: A are drawn exchangeably from P, then the conformal credal sets in (10) are valid, i.e.,
P(A FM E h2:42 (Fnew) 21-1a for {1,2}.
is worth mentioning that both predictors can also be trained using zero-order data. A first-order predictor can be achieved through standard training with cross-entropy loss function, while second-order predictor can achieved, for instance, through the means of evidential learning [44]. Hence, it is sufficient have probabilistic calibration data in order have reasonable judgment of the predictors' performances and be able to conformalize them.
Moreover, while these two approaches can be compared in various ways, one immediate observation is that training a second-order predictor poses greater challenges. However, credal sets constructed using the second-order predictor exhibit a natural and superior adaptivity compared to those constructed by the first-order predictor. This is because, with the first-order predictor, once the quantile determined during calibration, all distributions within certain distance are included in the set for any given point at prediction time. On the other hand, with the second-orde predictor, the resulting set depends on the calculated quantile well the skewness of the predicted distribution.
3.1 Generalization to Imprecise First-Order Data
So far, we (implicitly) assumed that ground-truth probability distributions A will be provided as calibration (and training) data. Needless to say. this assumption will rarely hold true in practice. Instead, observations will rather be noisy versions _. of the true probabilities. Notably, such datasets emerge scenarios where each data instance æ is annotated by multiple human experts, which recently have attracted a lot of attention in the context of machine learning [10. 31, 35, 43, 63]
and also conformal prediction [24, 52]. In this context, X denotes the distribution derived from aggregating annotator disagreements concerning the label of instance æ. Of course, conformal
prediction can still be applied noisy data of that kind, but the coverage guarantee will then only hold w.r.t. noisy labeling, i.e., P() FFow
E (((nnww) )) 2 1 11-a.
Practically, one may expect that the guarantees will hold for the ground truth as well, simply because calibration on noisy instead of clean data will tend to make prediction regions larger and hence more conservative. Moreover, since nonconformity is derived from a predictive model g(-) that seeks to recover ground-trutt truth probabilities, the latter should conform least as well as noisy distributions. Of course, this intuition is not formal guarantee. In order to provide such a guarantee for the ground-truth probabilities, one obviously needs to make some assumptions. Concretely, let us make the following bounded noise assumption for the labeling process: The labeling noise is (stochastically) bounded the sense that, given the nonconformity function f and a (small) probability 0, there exists tolerance >> 0 such that the following holds all X:
IP (15(x ((x,")) < 12-2221- 21-
(11)
Theorem 3.2. Let a O be any miscoverage rate, and suppose the bounded noise assumption holds. Let q = (EE ā) be the critical threshold on the noisy calibration data D Deali for miscoverage rate (a 6) 8). Then, for any new query Knew E X,
(f(new ABrr)
+ 1-
The proof is deferred to Appendix B. As consequence of this result, conformal predictor learned on the noisy data with modified miscoverage rate can be turned into a valid predictor (with miscoverage rate a) for the ground-truth data by increasing the learned rejection threshold by e. provided the bounded noise property (11) can be ascertained. Thus, if we denote the corresponding credal set predictor by hr we can guarantee that IP Beew E he (Trew)) 2 1 a. For a comprehensive study of handling noisy data in conformal prediction, we refer to [19].
Experiments
For the sake of comparison, we examine different nonconformity functions in our experiments. When utilizing first-order predictor, besides total variation (TV) and the First Wasserstein (WS)
5

---

GitHub Premise Premiss--Hypathesi Hypothesis puir
TV
KL
WS
Inner
SO
Premise: bar drinks pitcher while a man in a green hat looks glass.
eodace
and black shirt drink from
Hypothesis: women drinking water.
0.24
0.18
0.28
0.36
0.06
Premise: The streets are busy and people contemplate their futures. Hypothesis: People screaming
0.33
0.20
0.33
0.41
0.47
Premise: A photographer snaps naldir of snowbourder.
Hypiothesis: The midair shot snaps at boander of sow.
0.31
0.34
0.08
Figure Various credal sets obtained for three instances from ChaosNLI dataset. The ground truth distributions are denoted by orange squares. Black circles indicate model predictions in cases employing a first-order learner (first four columns). For the last column, utilizing a second- order learner, the predicted second-order distributions are represented through contour plots. The miscoverage rate is a 0.2, and the efficiency of each credal is depicted below it.
distance, also investigate the Kullback-Leible (KT) divergence and minus the inner product (Inner) as nonconformity functions. For the second-order predictor, we consider minus the relative likelihood (SO) defined in (8). Furthermore, quantification and the disentanglement of total uncertainty into epistemic and aleatoric components. We focus measure proposed by Abellán et [2], which regards the upper Shannon entropy H of given credal set Q as total uncertainty, the lower Shannon entropy as aleatoric, and the difference between upper and lower entropy epistemic:
demonstrate how our credal
allow uncertainty
(Q)Q) (Q( (H* (Q)))) with H²(Q) max H(A), H.) min H()). (12) XEQ
XEQ
aleatoric
epistemic
We refer to [23] for other measures, along with discussion of their corresponding strengths and weaknesses. Interestingly enough, the interval (Q), (Q)) [0, 1] can be seen an alternative characterization of the credal set Q, which helps address the challenge of visualization for K > 3.
In this section, focus experiments two real-world datasets. Further information these datasets and details about the learning models can be found in Appendix C. Additional experiments synthetic data, including illustrative example showing how the resulting credal sets change as epistemic uncertainty decreases, and experiments on the impact of imprecise first-order data, provided in Appendix E. All implementations and experiments be found repository.2
ChaosNLI Dataset. We start our experiments with highly ambiguous dataset, ChaosNLI [31], where the task is to classify the textual entailment of Seomisehhypotesse pair into three classes: entailment, contradiction, and neutral. We train both first-order and second-order predictors using this data and construct credal sets with all five nonconformity functions to facilitate comprehen- sive comparison of these methods. In Figure 2, we compare the resulting credal sets of different nonconformity functions for three specific instances. As expected, even though CP should work regardless of the choice of nonconformity score function, this choice affects the size and geometry of the prediction set. To compare the prediction set size, aka efficiency, across different nonconformity functions, we discretize the simplex using a fine grid. The efficiency is gauged by considering the fraction of all distributions that lie within the predicted credal sets. We perform training, calibration, and testing using 10 different random seeds and depict the average coverage and efficiency results of each credal set predictor under different miscoverage rates (a) in Figure 3. Notably, the mean of the
H(A) == - -K As log) with O log = 0 by definition. Thh link to the code: https: //gitt//tttttt com/ com/aliree alireza- wwwrreeajaaaaaarrdee wwwpjaaaaad/aooaoma www.arrddd//oooooooaeeo craaal--eeddaleee..

---

100%
1.0
Average Coverage
Eliicencyy
60%-
40%
Average Coverage
Average Elficiency
0%
0.05
0.1
0.2
0.05
0.1
0.05
0.2 0.05
0.1
0.2
0.05
0.1
0.0
Miscowerage (a)
Figure 3: Coverage and efficiency results of different nonconformity functions applied the
ChaosNLI dataset. The horizontal dashed lines indicate the nominal coverage levels.
1.0
1.0
1.0
1.0
1.0
1.0
0.5-
2 0.5
0.5
0.5
0.0-
0.0
0.0
0.0
0.0
0.5
(a) TV
(b) KT.
(c) WS
(d) Inner
(c) SO
Figure Scatter plots of aleatoric versus epistemic uncertainty using various credal set predictors for the ChaosNLI dataset with = 0.2. The colors indicate the entropy of the ground truth distribution.
The three cases numbered 1 to 3, correspond the first to third rows of instances shown in Figure 5.
average coverage over the test data across various random seeds aligns with or exceeds the nominal value, consistent with the conformal prediction guarantee.
We calculate the uncertainty intervals in (12) for all (test) instances. Figure 4 provides scatter plots of quantified AU vs. EU for different credal set predictors, with colors indicating the entropy of the ground-truth distribution. These plots serve as an evaluation of uncertainty quantification performance. Generally, given access to the first-order distribution, we expect the following: if the EU low, the AU should align closely with the entropy of the ground-truth first-order distribution. However, when the EU is high, the quantified AU may vary, appearing either close to far from this entropy. In contrast, with only standard zero-order data available, such direct evaluation isn' feasible, which is why indirect evaluation methods like OOD detection accuracy-rejection curves are preferred.
Premise
Hypothesis
[H. (Q), H (Q))
The purpose of the Diwan--- K-has is hotly disputed: is necessorily the hall of private audience name implies.
The hall is not know many people.
High
example, Bruce Bar- The Man Nobody Knows, a best vefler in 1925-26, portrays Jesus as the ultimate business-
Bruce Barton's "The Man Nobody Knows", best weller in 1925 26, is known the best ple of Jesus as the ulti-
Low EU, High AU
A woman in long- sleeved shirt checks her phone as a mon leather jocket be- hind her.
Low EU, Low AU
man passing on his to the store.
Figure 5: Different uncertainty situations given the predicted credal sets generated by SO method for the ChaosNLI dataset with = 0.2.

---

1.0
1.0
1.0
0.5
0.5
0.5
AU
0.5
0.5
0.0
0.0
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
(a) TV
(h) KT.
(c) WS
(d) Inner
Figure 6: Scatter plots of aleatoric versus epistemic uncertainty using various credal set predictors for the CIFARIO-H dataset with 0.2. The colors indicate the entropy of the ground truth distribution.
The three cases numbered I to 3, correspond to the first to third rows of instances shown in Figure 7.
(
(Q), H (Q1)
Probubility
Probubility
High
dred dos
sbit muc
plane
co Inc
co
Bog
hand
shiy
Prohubiiity
Prihubility
Low EU, High AT;
co.
tnuc
place ca bim
dug
hond
ahr
imc
Prohudiliyy
Protualiityy
EU,
65
hird
cal dect dlgg
ship fruck
plead
Cu hard
dng frog Thordd ship
Figure 7: Different uncertainty situations given the predicted credal sets generated by KL method for the 1U-ARIO-H dataset with a 0.2.
For the SO method, three uncertainty cases are chosen from the results: high EU, low EU with high AU, and low EU with low AU as shown in Figure 5. When visualizing the uncertainty intervals, we display the entropy of the ground truth distribution in orange and the predicted entropy for the first-order predictor or the mean of the Dirichlet distribution for the second-order predictor in black. However, note that covering the entropy of a distribution with the uncertainty interval does not equate to covering the distribution itself with the credal set.
CIFAR-10H Dataset. We also apply our methods to the CIFAR-IOH dataset [35], which contains the distributions the classes for the CIFAR-10 10 test set derived from human annotations. This time, we simply use a first-order predictor pre-trained on the CIFAR- 10 training set (i.e., standard zero-order training data) and only use CIFAR-1OH for calibration and testing. Since the visualization of credal sets is no longer possible for this dataset, we limit our attention to uncertainty quantification analysis. Figure 6 illustrates scatter plots of quantified AU versus BU for different credal set predictors. Three uncertainty cases are chosen from the results and shown in Figure 7. For instance, in the first row, the model's predicted distribution (g(=)) heavily favors "cat" over "dog", while human- derived distributions assign nearly equal probabilities to both classes. This relatively high epistemic uncertainty is effectively captured by the uncertainty interval.
Further Discussions. Given a model and data point, different methods may exhibit different be- haviors in terms of uncertainty representation, stemming from their distinct approaches to generating the sets. For instance, been observed that KL. tends to limit the set's expansion towards high entropy regions compared other methods. The Inner method, on the other hand, tends to construct

---

credal sets by incorporating corners (degenerate distributions), thereby setting the lower entropy to zero. Consequently, may fail effectively represent different uncertainty components reliable manner. Moreover, this method has the potential to yield empty credal sets, meaning that even the prediction of the first-order predictor may be excluded from the credal set.
From an uncertainty quantification perspective and considering the measure in (12), it becomes apparent that set size may not always accurately reflect epistemic uncertainty. In fact, a credal set of a certain size positioned in the middle of simplex may contain less epistemic uncertainty than the same set located around a corner.
The observed dependency between quantified aleatoric and epistemic uncertainty in Figures 4 and is partly due to the chosen measure in (12): Generally, higher epistemic uncertainty implies larger uncertainty interval, resulting in a lower value for aleatoric uncertainty. Thus, this dependency is inevitable in the high EU region. Another unintended behavior observed in the TV, KI,, and WS approaches is that low epistemic uncertainty coincides solely with high aleatoric uncertainty. This issue arises mainly from the lack of adaptivity in the credal sets constructed by first-order predictors. In Appendix D, we propose a method to enhance the adaptivity of these approaches using the concept of normalized nonconformity functions [33].
Limitations
The methods we propose are promising but still subject to certain limitations. For instance, for our methods to perform effectively, require first-order data, at least for calibration. While such data is becoming increasingly available in practice, it is not accessible for all datasets and domains. Besides, for generalization to the case of imprecise first-order data, practical implementation depends on a meaningful choice of the hyperparameters e and to ensure inference that is both valid and efficient. When labels are based on relative frequencies (as in the case of multiple annotators), classical statistical methods might apply. However, determining an appropriate choice of and 6 for broader practical problems remains an open issue. Another challenge lies in representing credal sets as subsets of the probability simplex. Although the credal sets can always be precisely described by equation (10), for the nonconformity functions used in this work, there for the resulting credal sets. Instead, the sets are represented implicitly through the nonconformity threshold. Numerical approximation is feasible but generally limited to scenarios with small number of classes. The representation issue is also connected to computing uncertainty measures for quantifying epistemic and alcatoric uncertainty in credal set that involves the computation of specific set characteristics [26, 41].
closed-form equations
6 Conclusion and Future Work
Conformal credal set prediction connects machine learning with imprecise probability theory and offers a novel data-driven approach to constructing predictions that effectively capture both aleatoric and epistemic uncertainty. Thereby, it provides the basis of new approach to reliable, uncertainty- aware machine learning. Leveraging the inherent validity of the conformal prediction framework, our conformalized credal sets are assured cover the ground truth distributions with high probability. We have explored different nonconfornity functions within this novel setting and evaluated their performance through numerical experiments.
There are several promising directions for future work. One avenue is to extend our method to standard (zero-order) data. While it has been shown that learning second-order predictor from such data is challenging [8, 9]. whether similar problems apply to credal predictors constructed from such data is not yet fully clear. Additionally, exploring other nonconformity functions that lead to closed-fone solutions for credal sets or enhance efficiency and reduce uncertainty is worth considering. Finally, construeting set of labels from proposed credal sets presents intriguing opportunity for further research, especially as such sets can provide more information compared to standard conformal prediction sets.
The broader impact of our work the advancement of Machine Learning models towards better uncertainty aware predictions, and we do not foresee any negative societal impacts.

---

Acknowledgment
Alireza Javanmardi was supported by the Deutsche Forschungsgemeinschatt (DFG, German Research Foundation): Project number 451737409.
References
[1] J. Abellán and S. Moral. Building classification trees using the total uncertainty criterion. International Journal of Intelligent Systems, 2003.
[2] Abellán, G. Klir, and S. Moral. Disaggregated total uncertainty measure for credal sets. International Journal of General Systems, 35, 2006.
[3] G. Abercrombie, V. Rieser, and D. Hovy. Consistency is key: Disentangling label variation in natural language processing with intra-annotator agreement. arXiv preprini arXiv:2301. 106844 2023.
[4] A. Angelopoulos, S. Bates, J. Malik, and M. I. Jordan. Uncertainty sets for image classifiers using conformal prediction. arXiv preprint woXiv:2009.14192 2020.
[5] L. Aroyo and C. Welty. The three sides of crowdtruth. Human Computation, 1, 2014.
[6] L. Aroyo and C. Welty. Truth is a lie: Crowd truth and the seven myths of human annotation.
Al Magazine, 36, 2015.
[7] R. F. Barber, E. J. Candès, A. Ramdas, and R. J. Tibshirani. Predictive inference with the jackknife+. The Annals of Statistics, 49, 2021.
[8] V. Bengs, E. Hüllermeier, and W. Waegeman. Pitfalls epistemic uncertainty quantification through loss minimisation. In Advances in Neural Information Processing Systems, volume 35, 2022.
[9] V. Bengs, E. Hüllermeier, and W. Waegeman. On second-order scoring rules for epistemic uncertainty quantification. In Proceedingy of the 40th International Conference on Machine Learning, 2023.
[10] L. Beyer, O. J. Hénaff, A. Kolesnikov, X. Zhai, and A. d. Oord. Are we done with imagenet? arXiv preprint arXiv:2006.07159, 2020.
[11] C. Bhagavatula, R. Le Bras, C. Malaviya, Sakaguchi, A. Holtzman, H. Rashkin, D. Downey, W.-t. Yih, and Y. Choi. Abductive commonsense reasoning. In International Conference on Learning Representations, 2019.
[12] S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2015.
[13] L. Cella and R. Martin. Validity, consonant plausibility measures, and conformal prediction. International Journal of Approximate Reasoning, 141, 2022.
[14] G. Corani and M. Zaffalon. Learning reliable classifiers from small or incomplete data sets: The naive credal classifier 2. Journal of Machine Learning Research, 2008.
[15] G. Corani, A. Antonucci, and M. Zaffalon. Bayesian networks with imprecise probabilities: Theory and application to classification. Data Mining: Foundations and Intelligent Paradigms: Volume I: Clustering, Association and Classification, 2012.
[16] S. Depeweg, J.-M. Hernandez-Lobatte
Delei,Velez, and S. Udluft. Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning. In International Conference on Machine Learning. PMLR, 2018.
[17] T. G. Dietterich and J. Hostetler. Conformal prediction intervals for markov decision process trajectories. arXiv preprint arXiv:2206. 04860, 2022.
[18] A. Dumitrache, F. Mediagroep. L. Aroyo, and C. Welty. A crowdsourced frame disambiguation corpus with ambiguity. In Proceedings of NAACI-HILT, 2019.
[19] S. Feldman, B.-S. Einbinder, S. Bates, A. N. Angelopoulos, A. Gendler, and Y. Romano. Con- formal prediction is robust to dispersive label noise. Proceedings of the Twelfth Symposium on Conformal and Probabilistic Prediction with Applications, volume 204. PMLR, 2023.
10

---

[20] A. Fisch, T. Schuster, T. Jaakkola, and R. Barzilay. Conformal prediction sets with limited false positives, In International Conference on Machine Learning. PMLR. 2022.
[21] C. Hora. Aleatory and epistemic uncertainty in probability elicitation with an example from hazardous waste management. Reliability Engineering & System Safery, 54, 1996.
[22] E. Hüllermeier and W. Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110, 2021.
[23] E. Hüllermeier, S. Destercke, and M. H. Shaker. Quantification of credal uncertainty in machine learning: A critical analysis and empirical comparison. In Uncertainty in Artificial Intelligence. PMLR, 2022.
[24] A. Javanmardi, Y. Sale, P. Hofman, and E. Hüllermeier. Conformal prediction with partially labeled data. In Conformal and Probabilistic Prediction with Applications. PMLR, 2023.
[25] A. Kendall and Y. Gal. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017.
[26] G. Klir and M. Wierman. Uncertainty- information: elements of generalized information theory, volume 15. Springer Science & Business Media, 1999.
[27] A. Kovashka, O. Russakovsky, L.. Fei-Fei, K. Grauman, et al. Crowdsourcing in computer vision. Foundations and Trends@) computer graphics and Vision, 10, 2016.
[28] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[29] J. Lienen, C. Demir, and E. Hüllermeier. Conformal credal self-supervised learning. In Conformal and Probabilistic Prediction with Applications. PMLR, 2023.
[30] H. Linusson, U. Johansson, and H. Boström. Efficient conformal predictor ensembles. Neuro- computing, 397, 2020.
[31] Y. Nie, X. Zhou, and M. Bansal. What can we learn from collective human opinions on natural language inference data? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2020.
[32] H. Papadopoulos, K. Proedrou, V. Vovk, and A. Gammerman. Inductive confidence machines for regression. In Machine Learning: ECML 2002: 13th European Conference on Machine Learning. Springer, 2002.
[33] H. Papadopoulos.
Gammerman, and V. Vovk. Normalized nonconformity measures for regression conformal prediction. In Proceedings of the IASTED International Conference on Artificial Intelligence and Applications (ATA 2008), 2008
[34] E. Pavlick and T. Kwiatkowski. Inherent disagreements in human textual inferences. Transac- tions of the Association for Computational Linguistics, 7, 2019.
[35] J. C. Peterson, R. M. Battleday, T. L. Griffiths, and O. Russakovsky. Human uncertainty makes classification more robust. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9617-9626, 2019.
[36] D. Reidsma and R. op den Akker. Exploiting 'subjective'annotttionn annotations. In Coling 2008: Pro- ceedings of the workshop on Human Judgements in Computational Linguistics, pages 8-16, 2008.
[37] Y. Romano, Patterson, and E. Candes. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019.
[38] Y. Romano, M. Sesia, and E. Candes. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems, 2020.
[39] P. Röttger, B. Vidgen, D. Hovy, and J. Pierrehumbert. Two contrasting data annotation paradigms for subjective nlp tasks. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2022.
[40] M. Sadinle, J. Lei, and L. Wasserman. Least ambiguous set-valued classifiers with bounded error levels. Journal of the American Statistical Association, 2019.
[41] Y. Sale, M. Caprio, and E. Hüllermeier. Is the volume of a credal set a good measure for epistemic uncertainty? In Uncertainty in Artificial Intelligence. PMLR, 2023.
11

---

[42] M. Schaekermann. E. Law, A. C. Williams, and W. Callaghan. Resolvable vs. irresolvable ambiguity: new hybrid framework for dealing with uncertain ground truth. In 1st Workshop on Human-Centeree Centered Machine Learning at SIGCHI, volume 2016, 2016.
[43] L.. Schmarje, V. Grossmann, C. Zelenka, S. Dippel, R. Kiko, M. Oszust, M. Pastell, J. Stracke, A. Valros, N. Volkmann, et al. Is one annotation enough?-a data-centric image elassifica- tion benchmark for noisy and ambiguous label estimation. Advances in Neural Information Processing Systems, 35, 2022.
[44] M. Sensoy, L. Kaplan, and M. Kandemir. Evidential deep learning to quantify classification uncertainty, Advances in neural information processing systems, 31, 2018.
[45] M. Sesia and Y. Romano. Conformal prediction using conditional histograms. Advances in Neural Information Processing Systems, 34, 2021.
[46] M. H. Shaker and F. Hüllermeier. Aleatoric and epistemic uncertainty with random forests. In International Symposium on Intelligent Data Analysis. Springer, 2020.
[47] R. Snow, B. O'connor, D. Jurafsky, and A. Y. Ng. Cheap and fast-but is it good? evaluating non-expert annotations for natural language tasks. In Proceedings of the 2008 conference on empirical methods in natural language processing, 2008
[48] A. Sorokin and D. Forsyth. Utility data annotation with amazon mechanical turk. In 2008 IEEE computer society conference on computer vision and pattern recognition workshops. IEEE, 2008.
prediition intervals in linear regression models with [49] I.. Steinherger and H. Leeh. Leave-one-out prediction many variables. arXiv preprint arXiv: 160..00602.05801 2016.
[50] D. Stutz, K. D. Dvijotham, A. T. Cemgil, and A. Doucet. Learning optimal conformal classifiers.
In International Conference on Learning Representations, 2021.
[51] D. Stutz, A. T. Cemgil, A. G. Roy, T. Matejovicova, M. Barshey, P. Strachan, M. Schaekermann,
J. Freyberg, R. Rikhye, B. Freeman, et al. Evaluating ai systems under uncertain ground truth: a case study in dermatology. arXiv preprint arXiv:2307. 021.0,11111000 2023.
[52] D. Stutz, A. G. Roy, T. Matejovicova, P. Strachan, A. T. Cemgil, and A. Doucet. Conformal prediction under ambiguous ground truth. Transactions on Machine Learning Research, 2023.
[53] A. Uma, D. Almanea, and M. Poesio. Scaling and disagreements: Bias, noise, and ambiguity. Frontiers in Artificial Intelligence, 2022.
[54] A. N. Uma, T. Fornaciari, D. Hovy, S. Paun, B. Plank, and M. Poesio. Learning from disagree- ment: A survey. Journal of Artificial Intelligence Research, 72, 2021.
[55] V. Vovk. Cross-confornal predictors. Annals of Mathematics and Artificial Intelligence, 74, 2015.
[56] V. Vovk, I. Nouretdinov, V. Fedorova, I. Petej, and A. Gammerman. Criteria of efficiency for set-valued classification. Annals of Mathematics and Artificial Intelligence, 81, 2017.
[57] V. Vovk, A. Gammerman, and G. Shafer. Algorithmic Learning in a Random World. Springer Nature, 2022.
[58] P. Walley. Statistical reasoning with imprecise probabilitiis, volume 42. Springer, 1991.
[59] P. Walley. Inferences from multinomial data: learning about a bag of marbles. Journal of the Royal Statistical Society Series B: Statistical Methodology, 1996.
[60] A. Williams, N. Nangia, and S. Bowman. A broad-coverage challenge corpus for sentence understanding through inference. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers). Association for Computational Linguistics, 2018
[61] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, Cistac, T. Rault, R. Louf, M. Funtowicz, et al. Huggingface's 's transformers: State-of-the-ar natural language processing. arXiv preprint arXiv: 1910,03771, 2019.
[62] M. Zaffalon. The naive credal classifier. Journal of statistical planning and inference, 2002.
[63] X. X. Zhu, J. Hu, C. Qiu, Y. Shi, Kang, L. Mou, H. Bagheri, M. Haberle, Y. Hua, R. Huang, et al. So2sat lc742: A benchmark data set for the classification of global local climate zones [software and data sets]. IEEE Geoscience and Remote Sensing Magazine, 8, 2020.
12

---

We have Appendix A: Extended Related Work
Credal sets are widely used as models for representing uncertainty, notably within the domain of imprecise probabilities [58]. As already mentioned, they can represent both types of uncertainty. aleatoric and epistemic. In the context of data analysis and statistical inference, credal sets are often used as robust models of prior information, namely for modeling imprecise information about the prior in Bayesian inference [59].
In machine learning, credal sets have been used for generalizing some of the standard methods, including naive Bayes [14, 62], Bayesian networks [15], and decision trees [1]. Typically, these approaches generalize simple frequentist inference to robust Bayesian inference, making use of an imprecise version of the Dirichlet model (a conjugate prior for the multinomial distribution). Com- pared to our approach, these methods are learning on standard (zero-order) training data. Moreover, despite representing uncertainty in predictions, they do not provide any formal guarantees.
Conformal prediction [57], briefly introduced in Section 2.2, has recently gained attention for various applications in machine learning, especially for classification tasks [4, 20, 38, 40, 50J. These methods mostly focus on split conformal prediction using a held-out calibration set [32], overcoming
computational limitations of earlier transductive or bagging approaches [7, 30, 49, 55. 57]. While tackling classification tasks, our method for constructing conformal credal sets has more similarity with conformal regression [37, 45], particularly in multivariate settings [17]. as we essentially conformalize the simplex space of categorical distributions. Our nonconformity scores differ, however, in that they are specific for distributions rather than considering general multivariate spaces. This work also relates to work on appropriate measures of inefficiency [56] as measuring the inefficiency of our conformal credal sets is non-trivial. Most closely related to our work is the recent work by Stutz et al. [52], who consider conformal prediction in settings with high aleatoric uncertainty. However, we explicitly target the construction of conformal credal sets, while Stutz et al. [52] mainly focus on constructing confidence sets of classes. The connection between CP and credal sets has also been explored by Cella and Martin [13] and Lienen et al. [29]. However, their emphasis lies on standard (zero-order) data, which fails to represent uncertainty truthfully. Furthermore, the sets generated by their methods are consistently confined to the simplex corners (around degenerate distributions) and display notably conservative behavior.
First-order data. In settings with high aleatoric uncertainty, labeling each example with single, unique class is clearly insufficient. In practice, this is typically captured by high disagreement among annotators - a problem particularly common natural language tasks [3. 5, 6, 18, 34, 36, 39, 42] or even in computer vision [10, 31, 35, 43, 63]. Handling this disagreement has received considerable attention lately [54] as it offers to go beyond this zero-order information. For example, recent work on evaluation with disagreeing annotators [51] argues the of these annotations to get approximate first-order information for evaluation. This approach is becoming more and more viable with crowdsourcing tools [27, 47, 48] being integral component of the benchmark, making multiple annotations per data instance more accessible. We follow a similar approach in our construction of conformal credal sets.
Appendix B: Proof of Theorem 3.2
Proof. Let denote the event f(xnow, A Mase ) < + e and Á the event f( Xmann <
P(A) 2 P(A A)
= P(A) P(A A)
P(A) (1- (L-P(AA]A)
Since AA means that fX x F 2 q c, the conditional event -4| B implies violation of the closeness condition in (11), wherefore the probability P(-A B) is upper- bounded by 8 according to (11). Therefore, noting that P(A) 2 1 - ā is the standard guarantee by CP,
P(A) 2 P(A)- (1 (1- P(-A A))
2 (1- a) (1-
= =l
13

---

Appendix C: Real-World Datasets and Models Overview
7500
entailment
600
5000
400
0.5
2500
contradiction
200
neartal
0.0
8.0
8.0
0.5 H(**)
1.0
H(A*)
(a) Scatter plot of the ChaosNLI (b) Histogram of the entropies of (c) Histogram of the entropies of CIFARIO-H distributions.
distributions.
ChaosNLI distributions.
Figure 8: General overview of real-world datasets.
ChaosNLI [31] (License: CC BY-NC 4.0 DEED) is an English Natural Language Inference (NLI) dataset that captures the inherent variability in human judgments of textual entailment. Here, the classes are entailment, neutral, and contradiction for each premise-hypothesis pair. Instances in this dataset are selected from the development sets of SNLI [12]. MNLI [60], and AbductiveNLI [11], for which the majority vote was less than three among the five human annotators. These instances were then given 100 independent humans for annotation, given strict annotation guidelines. We combine the chaos-SNLI and chaos-MNLL subsets, resulting in a dataset of 3113 datapoints. For model training. we leverage language model from the Hugging Face transf ormers library [61], initially trained on SNLI and MultiNIII datasets for classification tasks". We utilize the last hidden layer output of this model to embed the premise-by hyyothesss pairs from all 3113 instances into f6s-dimensional vectors, serving inputs for the learning model. To split the data, randomly select 500 instances for calibration, 500 for testing, and the remaining for training.
As for the learner, we employ deep neural network consisting of three hidden layers with 256, 64, and 16 units, utilizing ReLU as the activation function. Prior the output layer, a dropout layer with rate of 0.3 is incorporated. The same model architecture serves both first- and second-ore -odeer predictors, differing only in the activation functions of the output layers. For first-order predictor, softmax is used, while for the second-order predictor, ReL.U is employed. Learning is facilitated using the Adam optimizer with learning rate of 10-1, utilizing cross-entropy as the loss function for the first-order predictor and negative log-likeldoood for the second-order predictor.
CIFARIO-H [35] (License: CC BY-NC-SA 4.0 DEED) is a dataset of soft labels that capture human perceptual uncertainty for the 10000 images of CIFAR-I0 test set [28]. A total of 511, 400 human classifications were gathered via Amazon Mechanical Turk, excluding participants who performed poorly on obvious images. On average, each image received 51 judgments, with the number of judgments per image ranging from to 63. As shown Figure 8, the histogram of the entropies of 10F distributions shows that this dataset is less ambiguous compared to ChaosNLL. For this dataset, we don't perform any model training and instead use a model pre-trained on the CIFAR- 10 training images". We utilize the model's predicted distributions along with the CIFAR- distributions construct credal sets, focusing solely on first-order approaches. We randomly select 1000 instances for testing and the rest for calibration.
Appendix D: Enhancing Adaptivity in First-Order Predictor Credal Sets
As mentioned earlier, with methods based on the first-order predictor, once the quantile is determined during calibration, all distributions within certain distance included in the set for any given point at prediction time. This indicates that these methods lack adaptivity and do not account for the local heterogeneity of the data. Given the uncertainty measure in (12), for given quantile
'The model can be found https://hugginggacee http:////iiiiggeooooo//ooooooooa Apache License
with
'The pre-trained model can be found at httppp////tttaa..o/oo////a/aaaaa license.
with MIT
14

---

0.3
Nonconiomuy Soote
06
Nonoanbmitt Scores
1,00
0.6
64
Noncofoomil
-0
0.75
0.50
0.4
0.5
0.2
0.00
0.50
HIA7)
M(A")
H(29)
H(A))
(a) TV
(b) KL
(c)
WS
(d) Inner
Figure 9: Scatter plots of calibration scores versus the entropy of the first-order predictor's output for different nonconformity scores for the ChaosNLI dataset.
size, when the model's predicted distribution is near the simplex corner, the difference between the highest and lowest entropies is much greater than when the predicted distribution is located in the middle. That is why, in Figure 4, small epistemic uncertainty observed only for high aleatoric cases for methods based on the first-order predictor. To handle this issue, one approach involves utilizing the normalized nonconformity score by dividing the score of each point by measure of) its dispersion at that point [33]. As depicted in Figure 9, we' ve noticed a direct correlation between the score's dispersion and the entropy of the predicted distributions for the ChaosNLI dataset; specifically, higher entropy corresponds to greater variability in the scores. Therefore, we propose the following normalized score:
fxx x= S1 X²) H(X +
(13)
where H1 (²") denotes the entropy of the predicted distribution at point æ and is hyperparameter that prevents division by zero and controls the influence of H(*² on the scores (the higher the т, the lower the influence). The credal set for Bnew will be constructed
{A AK |ffi (xnew A) S q (H " +
where E is the of normalized nonconformity scores. Figure 10 compares the scatter plots of AU versus EU for all first predictor- models with their adaptive counterparts with = 0.1. The adaptive approach decreases the average epistemic uncertainty. For example, the positions of the three instances of Figure 5 moved closer to their corresponding values of the SO method.
15

---

1.0
1.0
1.0
1.0
0.5
0.5
2N 0.5
AU
0.5
13
0.0
0.0
0.0
0.0
0.0
1.0
0.0
0.5
1.0
0.5
0.5
1.0
EU
BU
EU
EU
(a) TV
(b) KL
(c) WS
(d) Inner
1.0
1.0
1.0
1.0
AU
0.5
AU
0.5
AU
0.5
AU
0.5
2
0.0
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
EU
BU
EU
EU
(c) TV-adaptive
(f) KL-aaaptivv
WS-adaptive
(h) Inner-adaptive
0.0
0.5
1.0
Figure 10: Scatter plots of aleatoric versus epistemic uncertainty using various credal set predictors for the ChaosNLI dataset with a = 0.2. The colors indicate the entropy of the ground truth distribution. The three cases numbered I to 3, correspond to the first to third rows of instances shown in Figure 5.
16

---

Appendix E: Synthetic Data
Throughout this paper, we generate synthetic data for a K-class classification task as follows: we consider d-dimensional features X E Rd, where X1, ...... Xa are independent standard normal random variables. Subsequently, we generate random matrix 8 E Rax with its elements drawn independently from the standard normal distribution. To define the ground truth probability over the classes for object X, we use the following formulation:
(x)
X = PYY
: = k: k| = Σ, Zz(z)²
(14)
where Z(x) == exp(x" 3). As for the learner (both first-order and second- -oddrr predictors), in the subsequent experiments, we utilize the same deep neural network described in Appendix for the ChaosNLI dataset, with the input layer adopting the d-dimensional feature vector.
E.1 Illustrative Example
To illustrate the primary concept of credal sets and uncertainty decomposition, we present a straight- forward yet intuitive example. Given the data-generating process (14), we generate 3000 samples with d = 10 and K = 3 solely to facilitate the visualization of the credal sets, splitting them equally between training, calibration, and testing datasets. Initially, we train a model with only 10 points, gradually increasing to 50, 100, 500 and 1000. At each stage, calibrate the model using all the calibration data points and evaluate its performance on the test points. From the test data, we select three examples characterized by high entropy (almost uniform distribution), relatively high entropy (uniform across two classes), and low entropy (almost a Dirac distribution). We then plot their credal sets alongside lower and upper entropies for varying numbers of training data points.
Figure 11 provides intriguing insights! As evident in the first column, where the model was trained with only 10 data points, there exists significant epistemic uncertainty across all three cases. This indicates that none of the predictions are reliable, and it's challenging to predict a single label for any of the given examples. Moving to the third column, observe significant reduction in epistemic uncertainties. In the second example (second row), indicates that the total uncertainty is primarily due to epistemic uncertainty, whereas for the other two cases, it's result of high aleatoric uncertainty with some epistemic uncertainty. This suggests that gathering more data (and reducing the total uncertainty) may not significantly enhance the capability predict single label for the latter cases. Figure 12 illustrates the evolution of scatter plots of AU versus EU for all first prstipredictor methods as the number of training data increases.
E.2 Experiments with Imprecise First-Order Data
We considered another set of experiments with synthetic data to illustrate the impact of imprecise first-order data, particularly to showcase the behavior of the proposed credal sets when only have access to an approximation of the ground truth distributions. Our experiment revolves around a K-class classification task with K (3,4, 4, 6, 8, 10). For each K, generate N 1500 samples with = 10 using (14) to construct the datasets D = { X ) To obtain imprecise versions of D we employ a sampling approach. Specifically, we independently sample each distribution A
=e, m times and utilize relative frequencies to create its noisy counterpart Am We represent the resulting dataset DK {xx, X ) We repeat this process four times with m {1,5, 10, 100).
Given each dataset DKK, we randomly partition data points into 1300 training, 100 calibration, and 100 test instances and perform the proposed methodologies accordingly. Again, we repeat this process ten times with different random seeds for each dataset D.K Due to the computational complexity in calculating efficiency for cases with K > scores as an efficiency metric. In Figure 13, we represent the overall result under different K and m values. It can be observed that the coverage is fulfilledd across almost all scenarios, including m = 1 with degenerate distributions. This observed behavior is somewhat intuitive. The model endeavors learn the underlying probabilistic relationship between X and Y, even given the noisy data 5 Consequently, during calibration with noisy instances, the nonconformity scores of noise-free
utilize the quantile of the calibration nonconformity
Of course, this holds under some reasonable assumptions noise.
17

---

10
50
100
500
1000
class
clees
class
class
class
first
cbos
chock
clas
olasr
cluss
class
closs 3
clad 3
Clecn
class
class 3
class I
class
class
class 1
class
third
clain
closs
clain
Cless
clain
classe
1.5
Figure 11: Credal sets generated by the KL method for three examples of illustrative synthetic data with 0.1. Each row represents of the predicted credal set based on varying numbers of training data points. The lower and upper entropy of each credal set displayed below it.
of three examples, with columns illustrating the evolution
instances are mostly upper-bounded by the scores of their noisy counterparts, resulting in more conservative sets that effectively cover the ground-truth distributions. It can also be seen that the quantile of the nonconformity scores shrinks as increases. In Figure 14, illustrate the evolution of the credal sets as m changes from to 100 for different nonconformity functions when For this case, the full comparison of efficiency and coverage across various nonconformity functions provided in Figure 15.
Appendix F: Experiments Compute Resources
For all experiments, we used an Intel(R) Core(TM) 17-118001 CPU with 16.0 GB of RAM. Model training and calibration steps quite fast and take only few minutes run. In constructing the credal sets, we use high-resolution simplex discretization and determine whether each distribution from the discretized simplex belongs to the credal set via exhaustive search. For the WS method, this process can take up to a few seconds per data instance, while for all other methods, it takes less than second. The uncertainty quantification also completed in less than second.
18

---

[Dinin
TV
KL
WS
Inner
1.0
0.5
ALF
0.5
AU
0.5
10
0.0
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
EU
EU
1.0
1.0
1.0
1.0
0.5
Att
0.5
ATE
0.5
0.5
50
F2
0.0
0.0
0.0
0.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
EU
EU
EU
EU
1.0
1.0
1.0
1.0
0.5
Att
0.5
Att
0.5
100
0.0
0.0
0.0
0.0
0.0
0.5
0.5
1.0
0.0
EU
EU
EU
EU
1.0
1.0
1.0
Atr
0.5
AU
AU
500
0.0
0.0
0.3
0.5
1.0
1.0
0.0
EU
EU
EU
1.0
Atr
0.5
AU
0.5
1000
0.0
0.0
0.0
0.0
0.3
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.3
10
EU
EU
0.0
0.5
1.0
Figure 12: Scatter plots of aleatoric versus epistemic uncertainty using various credal set predictors given different numbers training data for illustrative synthetic data with a 0.1. The colors indicate the entropy of the ground truth distribution. The three cases, numbered I to 3, correspond to the first to third rows instances shown Figure 11.
19

---

Average Coverge
Average Quatille
3
K=4
K =
K K=
= 10
100%
1.0
Averoge Coverage
80%
Quantic
60%
40%
0
+
20%
Average Covenaee Average Qumtile
0%6
100
10
100
10
100
0.0
In (number of samples)
(a) TV
K m
K
K=
10
100%
80%
Averago Quantile
Avezage
40%
20%
Average Coverage Average Quantile
09%
(number samples)
(b) KL
K-3
K 4
K 6-6
K-8
X 10
100%
Aweraoo Quaralic
60%
Averago
40%
Average Coverage Avcrage Quantile
0%
10
100
10
160
10 100
3
10
100
10
160
(number of samples)
(c)
100%
1.0
Avozgee Coverage
80%
0.3
Quanttle
60%
0.6
40%
20%
Coverage
0.2
Average Quantile
0%
1
10
100
8
10
1
100
4
10
160
$
160
0.0
samples)
(d) Inner
Figure 13: Coverage and quantile results for synthetic data with imprecise first-order distributions, where the ground truth distributions are approximated by observing m samples from them. The horizontal dashed lines indicate the nominal coverage level 1 a 0.9.
20

---

m
TV
KL
WS
Inner
SO
(Boc
duni
1.00
1.00
0.70
1.00
1.00
5
0.46
0.67
0.39
0.22
1.00
dow
<bos
10
0.64
0.58
0.68
0.00
0.74
100
0.06
0.09
0.09
0.00
0.43
Figure 14: Credal sets derived using various credal set predictors for example from synthetic data with imprecise first-order distributions. Rows correspond to the number of samples utilized for distribution estimation. The ground truth distribution marked by an orange square, and its noisy versions are denoted by red squares. In cases employing first-order learner (first four columns). model predictions are denoted by black circles. The predicted second-order distributions are illustrated via contour plots in the last column, where second-order learner is employed. The miscoverage rate 0.05, and the efficiency of each credal set is indicated below it.
21

---

Average Coverage
Avenyy Ffficiency
100%
1.0
Average Covarage
Edbcsrcc
60%
40%
0%
0.05
0.1
0.2
0.05
0.1
0.2
0.05
0.1
0.2
0.05
0.1
0.2
0.05
0.1
0.2
0.0
Miseoverage (a)
(a) m
KL
Inner
100%
1.0
80%
Avetage Covenge
40%-
20%
0.2
T
0%
0.0
Miscoverage (ar)
(b) m 5.
TV
KL
WS
Inner
SO
100%
80%
0.88
Average Coverage
60%
40%
20%
0.2
-
0.05
0.2
0.1
0.05
0.2
0.1
0.2
0.05
0.1
Miscoverage rate (a)
(c)
10.
TV
KL
Inner
S0
100%
1.0
80%
10.8
Averago Covetage
Edficooyy
60%
0,6
20%
0%
0.05
0.1
0.2
0.05
0.1
0.2
0.05
0.1
0.2
0.05
0.1
0.2
0.05
0.1
0.2
0.0
Miscoverage rate (Q)
(d)
100.
Figure 15: Coverage and efficiency results of different nonconformity functions applied on the synthetic data with imprecise first-order distributions (K 3). The horizontal dashed lines indicate the nominal coverage levels.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction accurately reflect the paper's contribution: Providing data-driven method to construct credal sets using conformal prediction, assuming access to probabilistically labeled data.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please Section 5.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-tex system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer: [Yes]
Justification: The proof for Theorem 3.1 is not provided, as it is essentially a restatement of the conformal prediction validity theorem adjusted to setting with the reference provided. As for Theorem 3.2, its proof can be found in Appendix B. All assumptions have been stated.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: All experiment details, including models, data, etc., are provided in the paper or its Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
24

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Please
https://ggthub..ccmalaaraaaa
conf canformallceee ceraaal--edddaalseseteeee
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: /euuddde//ooeummiiiiii //t///oooeeeeeeeeeeeeeee wwwisssssssssiioeeeeooooooisssie
"nips..cc ctpps/cccpbbbb//uuuu gwwpc//gggs////eeeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer:
[Yes]
Justification: All details are provided in the paper or its Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification: Where applicable, do so. For instance, see Figure
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
25

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't I make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We provide broader impact statement in the conclusion of our work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
26

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our methods do not pose a risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We provide licenses and citations to all used datasets, models, and libraries.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswi twwode..ooo..... com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
27

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification:
We do not introduce any assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "tJs3X4m8DZ", "reviewer": "Reviewer_gqZt", "summary": "The authors propose a method for predicting credal sets in the classification task, given training data labeled by probability distributions. The resulting conformal credal sets are guaranteed to be valid with high probability (without any assumptions on model or distribution). The applicability of the method on ambiguous classification tasks for uncertainty quantification is demonstrated.", "questions": "1)\tIn lines between 107-109: Could authors please explain more how the proposed method tackle the trade off achieving the high probability of courage (tend to increase the size of credal sets) and being informative?\n\n2)\tIn lines between 124-127: Could authors please explain more why the Dirichlet distribution assumption is a meaningful learning? Would it also an biased estimation of uncertainty due to the lack of the ground-truth values?\n\n4)\tIn lines 141-142: It is stated that both predictors can also be trained using zero-order data. Could the also add some baseline compares like deep ensembles, BNNs or conformal predictions, using zeor-order data to demonstrate the outperformance of the proposal?\n\n5)\tCould the authors indicate what is the potential application for the method?", "limitations": "Some limitations are discussed in the Limitations Section. \nAdditional limitations please refer to Weaknesses and Questions.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1)\tThe methods is a novel and promising proposal.\n\n2)\tExperiments demonstrate interesting results of the proposal.\n\n3)\tThe paper is overall well-structured, some parts are easy to follow.", "weaknesses": "1)\tThere are no baseline comparisons. For instance, seconde-oder distribution methods like Bayesian approaches and some limitations are mentioned  in the Introduction. It is  unclear what are the added gains of the proposed method in terms of uncertainty estimation, compared to the existing tools like Bayesian neural networks, deep ensembles or conformal prediction framework methods. \n\n2)\tFollowing point 1), Appendix F shows that the proposal can take up to a few hours to create the credal set at the test time compared to the training and calibration steps for a few minutes. From the practical point of view, it might be a significant drawback when the added gains are unclear.  In terms of the paper presentation, it would be better that the main body at least indicates that the computational cost is discussed in the Appendix. \n\n3)\tThe paper seems not discuss how to predict the class index from the generated credal sets. Having a clear strategy of making class prediction would be a benefit for the end-users in practice. A recent paper [1] may provide some inspirations.\n\n4)\tUncertainty quantification is a core element in this paper. However, from the results in Figure 4, 5, and 6, it is a bit hard conclude the quality of the uncertainty estimation. Why some down-stream tasks, like Out-Of-Distribution detection or Accuracy Rejection [2], are not used for the evaluation?\n\n[1] Caprio, Michele, et al. \"Credal Bayesian Deep Learning.\" arXiv e-prints (2023): arXiv-2302.\n[2] Shaker, Mohammad Hossein, and Eyke Hüllermeier. \"Ensemble-based uncertainty quantification: Bayesian versus credal inference.\" PROCEEDINGS 31. WORKSHOP COMPUTATIONAL INTELLIGENCE. Vol. 25. 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for their insightful comments and suggestions. We have made every effort to address and clarify their concerns in the following response.\n\n- **W1.** The comparison with other methods was not feasible primarily due to differences in data assumptions. While the methods mentioned in the review rely on zero-order data, our approach requires first-order data. Additionally, methods such as Bayesian Neural Networks (BNNs) or ensembles represent uncertainty using second-order probability distributions, whereas our approach utilizes credal sets. This fundamental difference in representation makes a direct comparison impossible. It's also worth noting that our approach is indeed a variant of conformal prediction specifically adapted to handle first-order data. The key advantage of our proposed conformalized credal sets is that they are designed to be valid in the sense that they cover the ground truth first-order distribution with high probability—a property that, to the best of our knowledge, is unique to our approach. Interestingly, when first-order calibration data is available, our approach can be applied to BNNs or ensembles to construct valid credal sets based on their predicted second-order probability distributions.\n- **W2.** We believe it's important to clarify the details in Appendix F. For CIFAR10-H, we discretized the simplex into 100,000 distributions. For each test data point in CIFAR10-H, creating credal sets—determining whether each of those 100,000 distributions belongs to the credal set across all four methods (TV, KL, WS, and Inner)—takes only 11 seconds. The majority of this time ($\\sim$10.4 seconds) is attributed to the WS method, where we were unable to implement parallelization and had to iterate through all simplex distributions using a for loop. As a result, constructing all credal sets for the entire test set of 1,000 instances can take up to 3 hours. For this dataset, the calibration step and uncertainty quantification were completed in less than a second. We will ensure to clarify this in the paper to avoid any confusion.\n- **W3.** We fully agree with the reviewer that deriving a set of labels from a credal set is indeed a very interesting question, even though it is not the intention of our paper. This topic has been explored in various studies within the field of imprecise probability, such as the very interesting approach mentioned by the reviewer, or even simpler ones, such as the one where a label is excluded from the prediction set if its probability is dominated by other classes across all distributions within the credal set. In addition to these approaches, one could also apply the calibration step of the Monte Carlo Conformal Prediction method proposed by [1] in parallel, allowing for the construction of a prediction set alongside the credal sets.\n- **W4.** Please refer to our global response, specifically the **Evaluation of uncertainty quantification** paragraph.\n- **Q1.** This is indeed the desired property we're aiming for, and the same question applies to standard Conformal Prediction (CP). While CP naturally prevents prediction sets from becoming arbitrarily large by providing an upper bound on coverage probability under reasonable assumptions, the set size also depends on the chosen nonconformity score. In our work, we've explored and compared several scores, as shown in Figure 3 for the ChaosNLI dataset.\n- **Q2.** For second-order predictors, we need to assume a parametric distribution and estimate its parameters from the data. The Dirichlet distribution is a common choice, as it has been used in other second-order predictors like evidential deep networks [2].\nRegarding the second question, if the reviewer is referring to cases where the imprecise first-order distributions are biased estimates of the ground truth and we lack the bounded assumption of Theorem 3.2, our approach would be limited to the observations provided by these imprecise first-order distributions. Consequently, our uncertainty representation would be valid relative to these biased data, which ultimately reflect a bias toward the ground truth.\n- **Q3.** For the reasons why direct comparisons are not feasible, please refer to the response provided for W1. Note that learning models, such as first-order or second-order predictors, can be trained using zero-order data but not the credal predictor. The proposed credal predictor requires at least first-order data for calibration. For example, in our experiments with CIFAR-10H, we conformalize a probabilistic classifier trained on CIFAR-10 (zero-order) training data with CIFAR-10H (first-order) calibration data.\n- **Q4.** Our proposed method enables classifiers to represent uncertainty in a reliable manner through the use of a credal set. With this credal set, we can quantify both aleatoric and epistemic uncertainty, which can then inform decisions on the appropriate course of action. For instance, when epistemic uncertainty is high, it may be addressed by gathering more data. Conversely, if total uncertainty is primarily driven by high aleatoric uncertainty, additional data collection (e.g., consulting more experts) may not be beneficial and could be considered a waste of resources. We also demonstrated this with an illustrative experiment using synthetic data in Appendix E.1.\n\n[1] D. Stutz et al., \"Conformal prediction under ambiguous ground truth.\"\n\n[2] M. Sensoy et al., \"Evidential deep learning to quantify classification uncertainty.\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l73r5AdvZJ", "reviewer": "Reviewer_wGcP", "summary": "The paper shows that if access to the (even noisy) estimates of the conditional categorical label distribution is provided, one can define a confidence region within the simplex of conditional probabilities, that is guaranteed (marginally) that the true aleatoric uncertainty (ground truth label distribution) is within the predicted credal set with $1 - \\beta$ probability. The authors define various score functions based on distances in the probability space and likelihood.", "questions": "1. I believe that the results on the imprecise first-order data (theorem 3.2) can be enhanced leveraging the results from [1]. This study derives the conformal guarantee for the clean ground truth label, from the noisy prediction with a correction over the bias in noise (which in symmetric noise assumption it is 0.5). However, the result I mentioned concerns the prediction and adapting it to classification would be a different approach out of the context of this review. \n\n2. Is there any way that the authors can derive a guarantee over the sampled labels as well? By that I mean deriving the conformal prediction based on labels sampled from the ground truth distribution (in cases that the non-soft label exists like CIFAR10) and an algorithm that returns prediction set from the credal sets.\n\n3. If the answer to (Q2) is yes then what does the conformal quantile threshold look like in the simplex space?\n\n4. Can we directly use some notion of uncertainty quantification (like log-likelihood) through the conformal risk control? I mean in Fig. 3. I see some of the credal sets are far from the ground truth distribution. Can the result enhance if you bound the TV distance in average via risk contol?\n\n5. Are the scores that the authors used always producing symmetric convex halls? By symmetric I mean that the output is always a function of the predicted uncertainty and the computed quantile. If yes is there any way to remedy that?\n\n[1] Guille-Escuret, Charles, and Eugene Ndiaye. \"From Conformal Predictions to Confidence Regions.\" arXiv preprint arXiv:2405.18601 (2024).", "limitations": "I do not see any limitations that are not addressed or mentioned.", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "The problem is really an interesting one. It is still interesting to have guaranteed uncertainty quantification and this paper is a step towards that goal.\n\nThe paper is technically sound. I did not detect flaws or issues in the paper at the end.\n\nThe experimental setup is complete.", "weaknesses": "Please see the questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your kind words and feedback. We found your questions very interesting and did our best to elaborate on them.\n- **Q1.** Thank you for the reference; it is definitely an interesting and relevant work. To the best of our understanding, the main difference between the two papers is that [1] assumes additive noise on the level of labels while we are dealing with noise on the level of first-order distributions. It is not immediately clear to us how to incorporate the results from [1] into our context.\n- **Q2.** and **Q3.** Regarding an algorithm that generates prediction sets from credal sets, as noted in our response to W3 for reviewer gqZt, this topic has been explored in the field of imprecise probability. For instance, one approach involves excluding a label from the prediction set if its probability is dominated by other classes across all distributions within the credal set or the approach mentioned by reviewer gqZt [2]. However, extracting a conformal prediction set (with guarantees) from a conformal credal set is indeed an interesting follow-up research question stemming from our paper, and it is not an easy one to address immediately. More importantly, we believe that such an approach could greatly enrich the prediction sets by providing valuable insights into the uncertainty components (AU and EU).\n- **Q4.** The reason that some predictions do not cover the ground truth within the credal set is due to the specified error rate of Conformal Prediction (CP). While using conformal risk control will change the guarantee from coverage to risk, it's not immediately clear how and in what sense this can improve the results. However, it can definitely be considered an interesting future work. \n- **Q5.** This phenomenon occurs with credal sets constructed by the first-order predictor because, once the quantile is determined during the calibration step, all distributions within a certain distance are included in the set for any given point during test time. This is similar to the issue found in standard Conformal Prediction (CP) for regression when using absolute error as the nonconformity score, which is often criticized for its lack of adaptivity. In the CP for regression literature, one way to address this issue is through normalized nonconformity measures, where the score for each point is divided by a measure of its dispersion at that point [3]. In Appendix D of the paper, we introduced a normalized nonconformity score that divides the distance function by the entropy of the predicted distribution. Figure 10 demonstrates the successful improvement in the adaptivity of our proposed approach. \n\n[1] Guille-Escuret, Charles, and Eugene Ndiaye. \"From Conformal Predictions to Confidence Regions.\"\n\n[2] M. Caprio et al., \"Credal Bayesian Deep Learning.\"\n\n[3] H. Papadopoulos et al., \"Normalized nonconformity measures for regression conformal prediction\".", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oy6agRN4PS", "reviewer": "Reviewer_LLVn", "summary": "Authors propose a new uncertainty estimation method based on credal sets predictors. The new approach is based on the Conformal Prediction framework and shares its finite-sample and distribution-free guarantees. Authors verify the validity of their approach on existing NLP and CV datasets.", "questions": "* Can you provide a theoretical result in noisy case in terms of the standard observations, i.e. one class per data point?\n* Can you fix an issue with the score function?\n* On Figures 4,6,12 the data point collapse to a low-dimensional manifold. It is especially visible on Figure 4, where there is a clear linear dependency between AU and EU at least on a portion of the data. In light of this evidence, how can authors support the claim of part 6: “effectively capture both epistemic and aleatoric uncertainty”?\n* Authors introduce “uncertainty interval” as an additional uncertainty measure and report it in their figures. Part 4 also contains the following passage: “covering the entropy of a distribution with the uncertainty interval does not equate to covering the distribution itself with the credal set”. Could you please further explain the meaning and the role of this additional measure and why covering the (predicted) entropy may be desirable?\n* Conformal prediction can produce sets of different size depending on the non-conformity measure. Figure 3 reports the average set sizes as efficiency for the selected non-conformity measures. How does it affect the downstream task of uncertainty estimation? E.g. conformal prediction literature values smaller sets, is it useful for the UE task?\n* In the literature on conformal prediction, a stronger property than marginal coverage (equation 1) is of interest: conditional coverage [1]. While this might be out of scope for the current work, was this property of the proposed method investigated?", "limitations": "I think that the majority of limitations are well discussed. However, I think that the current theoretical result being written in terms of the noise in score function is a significant limitation.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* theoretical part is based on an established mathematical framework and authors formally prove the validity of their method\n* authors provide full (anonymized) source code of their experiments", "weaknesses": "* The proposed 2nd order score function seems to be not well- defined. For example, if any of $\\theta^x_k$ are small than 1, then maximum in denominator of equation (8) becomes infinite and score function becomes constant. This is undesirable behavior to the best of my understanding.\n* While it is straightforward to assess the influence of the noise in terms of perturbations of score function, the characterization of the noise in such terms seems not natural. The most practical noise is the one that is due to the categorical observations of the label distribution. It seems that there is a gap between these two characterizations of noise.\n* the ability of the method to clearly separate (“disentangle”?) aleatoric and epistemic uncertainty is not fully understood. Judging by Fig.4,6,12, there is a clear functional dependency and in the main text part 4 we read: “approaches with first-order predictors lack adaptivity” with regard to a similar issue. Perhaps a special synthetic experiment can provide more insight whether it is an issue with the base method or the proposed approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for engaging with our work. We tried to clarify two of the mentioned weaknesses/concerns, elaborate on one another, and address their questions.\n\n- **W1.** In our implementations, we ensure that each parameter of the Dirichlet distribution is at least one. We define the Dirichlet parameter as 1+NN(x), where NN(x) represents the non-negative output of the neural network. The intuition behind this is that, in the worst-case scenario where the output of the neural network is zero, we encounter a complete lack of knowledge, and this can be represented by setting the Dirichlet parameter vector to 1, which defines a uniform second-order distribution over all first-order distributions. We'll ensure to clarify this point in the paper.\n- **W2.** We stated the bounded noise assumption at the nonconformity score level to have a universal applicability for all nonconformity functions. For instance, when the Total Variation (TV) distance is used as the nonconformity function, the left-hand side of equation (11) in the paper can be rewritten as:\n    \\begin{align*}\n     \\mathbb{P}\\left(|f(x, {\\lambda}^{x})-f({x}, \\tilde{\\lambda}^{x} ) | < \\epsilon \\right) &= \n     \\mathbb{P}\\left(|d_{TV}(\\hat{\\lambda}^{x}, {\\lambda}^{{x}})-d_{TV}(\\hat{{\\lambda}}^{{x}}, \\tilde{{\\lambda}}^{{x}} ) | < \\epsilon \\right) \\geq \\mathbb{P}\\left(d_{TV}(\\tilde{{\\lambda}}^{{x}}, {\\lambda}^{{x}}) < \\epsilon \\right)\n     \\end{align*} \n     where the inequality holds according to reverse triangle inequality and $\\hat{{\\lambda}}^{{x}}$ denotes the prediction of the model. We can then use the Dvoretzky–Kiefer–Wolfowitz (DKW) inequality  to bound the total variation distance between the true categorical distribution and its empirical estimate from $m$ independent samples:\n    \\begin{align*} \n    \\mathbb{P}\\left( d_{TV}(\\tilde{{\\lambda}}^{{x}}, {{\\lambda}}^{{x}}) \\leq \\epsilon \\right) \\geq 1 - 2K \\exp\\left(-2m\\epsilon^2\\right).\n   \\end{align*} \nHence, when using TV as the nonconformity score for first-order approaches, the bounded noise in the nonconformity score reflects the noise arising from estimating the ground truth first-order distributions with only samples from that distribution. It is not clear how to provide such a bound for arbitrary nonconformity functions, which is why we presented the theoretical results under the assumption of bounded noise on the nonconformity level.\n- **W3.**  Please refer to our global response, in which we explained why we see such dependency and how we handled it.\n- **Q1.** As discussed in the response to W2, the DKW inequality can be used to bound the total variation distance between the true categorical distribution and its empirical estimate based on m independent samples, however, in the special case where m=1, this bound can be quite loose and, therefore, less useful. Interestingly, in Appendix E.2, we conduct experiments with synthetic data where we use the estimation of the ground-truth first-order distribution with varying values of m, including m=1. As illustrated in Figures 13 and 15, coverage is generally achieved in almost all cases. However, for lower values of m, this coverage comes with reduced efficiency, resulting in larger sets.\n- **Q2.** Please refer to the response provided for **W1**.\n- **Q3.** Please refer to our global response.\n- **Q4.** For the uncertainty interval figures, our goal was to present the information similarly to our credal sets, as shown in Figure 2, where we visualize the location of the ground truth distribution, the predicted distribution (either first-order or second-order, depending on the model), and the credal set. Analogously, we display the uncertainty interval together with the entropy of the ground truth distribution and the entropy of the predicted first-order distribution or the entropy of the mean of the Dirichlet distribution for the second-order predictor. \n    However, the entropy function is non-injective, meaning that two different distributions can have the same entropy. Therefore, the fact that the entropy of the ground truth is within the uncertainty interval does not necessarily imply that the credal set covers the distribution itself. \n- **Q5.** It is certainly desirable to achieve the coverage guarantee with the smallest possible set sizes. However, with a credal set, uncertainty depends on both the size and the location of the set. Broadly speaking, set size influences epistemic uncertainty, while location affects aleatoric uncertainty. The least uncertain scenario would be a singleton located at a corner of the simplex.\n- **Q6.** Investigating conditional coverage is definitely an interesting and important direction for future work in our paper. We anticipate this to be an even more challenging problem, as conditional coverage in standard conformal prediction (CP) pertains to the coverage given x or y (the label). In our case, however, it involves coverage given x or the first-order distribution.\n- **Limitations.** We hope our response to W2 has helped clarify the limitation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UnhoeU1Gxa", "reviewer": "Reviewer_ieV3", "summary": "This paper proposes a method for predicting credal sets in classification tasks, generating conformal credal sets that are guaranteed to be valid with high probability without any assumptions on the model or distribution. The method is applicable to both first- and second-order predictors. Experiments on ChaosNLI and CIFAR10-H demonstrate its effectiveness and how these credal sets allow for practical uncertainty quantification in AU and EU.", "questions": "1. Would any model calibration methods provide probabilities that satisfy the bounded noise assumption? \n\n2. While some frequentist methods also built on the first-order probability labeling, they realize it as a deterministic action at every sampling step. Would it be possible to adopt this strategy to avoid the assumption? Would the complexity be acceptable?", "limitations": "The limitations include the implicit representation of the credal set, the intractability on large label sizes, and the assumption of access to reasonable probability labelings. The authors have adequately discussed these limitations.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The paper is well-written, with smooth and clear demonstrations. Despite containing many formulas, it is easy to understand.\n2. The proposed method combines imprecise probability and conformal prediction to leverage the advantages of both, which is interesting.\n3. The experimental design on small label size datasets is comprehensive.\n4. The limitations are discussed and soundly pointed out.", "weaknesses": "1. The technical novelty is moderate: the proposed method is very similar to a previous conformal prediction method on ambiguous labels [1], as both assume reasonable probability labeling and train a standard (first-order) probability predictor (i.e., the score function). Besides, obtaining a probability interval is technically not very novel.\n\n2. The computational complexity limits the general application of the method to large-scale datasets.\n\n3. Access to ground-truth probability labeling seems unreasonable, especially when model calibration itself is already a significant challenge.\n\n4. The key bounded noise assumption for the labeling process may not be very practical, as mentioned in the paper.\n\n\n[1] Stutz, David, et al. \"Conformal prediction under ambiguous ground truth.\" arXiv preprint arXiv:2307.09302 (2023).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your kind words about the paper and for your valuable feedback. We hope our response addresses your comments and questions. \n\n- **W1.** The main similarity between our work and [1] lies in the data assumption we make. However, while the authors in [1] focus on obtaining prediction sets (i.e., sets of labels), our work is centered on the development of credal sets (i.e., sets of probability distributions). The distinction is important because credal sets allow for the disentangling of total uncertainty into aleatoric and epistemic components, something that prediction sets in [1] do not achieve.\n\n    In the context of credal sets, our proposed conformalized credal sets are designed to be valid in the sense that they cover the ground-truth first-order distribution with high probability—a property that, to the best of our knowledge, is unique to our approach.\n\n    Moreover, for model training, we do not limit ourselves to first-order probability predictors. We thoroughly discuss both first-order and second-order probability predictors, providing a family of nonconformity scores tailored to each type of predictor to enhance the flexibility of our proposed methods. We also address the challenge of obtaining a valid credal set when only imprecise first-order data is available. Additionally, we empirically demonstrate the effectiveness of our proposed methods for uncertainty quantification with such datasets.\n- **W2.** It is worth noting that our approach to model training and calibration does not introduce any additional complexity compared to existing models. Our credal sets can always be precisely described by equation (10) in the paper for any arbitrary K. This equation also enables efficient membership checking, allowing us to determine whether a distribution belongs to a set. The only limitation arises when seeking an exact closed-form representation of the credal set or a compact form, such as in the 1-dimensional regression problem where we represent the set with an interval. However, this challenge is essentially unavoidable for Conformal Prediction in complex (infinite) output spaces, such as multi-output regression or structured output prediction. We have also clarified the details of our experiment in Appendix F, as mentioned in our response to W2 from reviewer gqZt.\n- **W3.** We would like to highlight that datasets where multiple human experts annotate each data instance x are indeed becoming increasingly common in practical tasks, including widely used benchmarks like ImageNet [2] and CIFAR-10 [3], and in some cases, have even been shown to be necessary [4]. This is especially crucial in the context of natural language processing, where a long history of research emphasizes its importance [5,6] (Please refer to our extended related work in Appendix A for many more references). Aggregating annotator disagreements regarding the label of an instance x into a distribution over labels can be seen as an effective way to represent aleatoric uncertainty. We believe and demonstrate in this paper that this information can also contribute to advancements in uncertainty representation and quantification research.\n- **W4.** Guarantees rely on assumptions, and we believe that our assumption is not stronger than assumptions made for comparable results in the literature. Intuitively, the assumption seems to be not only sufficient but even necessary: If noise is unbounded, the estimations can deviate arbitrarily from the ground-truth distribution, thereby making any guarantee impossible.\nMoreover, it's important to highlight that in real-world applications, quantifying noise, if present, may not always be feasible. Instead, we are often limited in the observations that are available to us. For instance, in our case, we rely on distributions obtained from aggregating human annotations. It's also worth noting that the guarantee of the proposed method remains always consistent with respect to these observations.\n- **Q1.** If the reviewer is referring to model calibration with zero-order data, to the best of our knowledge, none of the existing calibration methods are instance-wise, i.e., they do not aim to calibrate the predicted probability for each specific input x but rather provide calibration in expectation. It is certainly interesting to explore various calibration methods to see if they can achieve this.\n- **Q2.** We are not entirely certain that we fully understand the method the reviewer is referring to. We believe we could provide a more comprehensive answer with additional context. However, as a general remark, if we aim to remove the bounded noise assumption, we would either need to introduce an alternative assumption (which may come naturally with a certain approach) or reconsider our theorem or guarantee entirely.\n\n[1] D. Stutz et al., \"Conformal prediction under ambiguous ground truth.\"\n\n[2] L. Beyer et al., \"Are we done with imagenet?\"\n\n[3] J. C. Peterson et al., \"Human uncertainty makes classification more robust\"\n\n[4] L. Schmarje, et al. \"Is one annotation enough? a data-centric image classification benchmark for noisy and ambiguous label estimation\"\n\n[5] Y. Nie et al., \"What can we learn from collective human opinions on natural language inference data?\"\n\n[6] D. Stutz et al., \"Evaluating AI systems under uncertain ground truth: a case study in dermatology\"", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We want to express our gratitude to all four reviewers for their valuable feedback. We're encouraged that they found our paper important, interesting, and novel, and we appreciate their insightful comments. We would like to offer some general remarks, particularly concerning the evaluation part.

- **Uncertainty representation vs uncertainty quantification** We would like to emphasize that representing uncertainty through a credal set and decomposing it into aleatoric and epistemic components (i.e., uncertainty quantification) are two distinct tasks. This paper primarily focuses on proposing a novel method for representing uncertainty with a reliable credal set. To disentangle these uncertainties, we use a well-known measure despite some associated criticisms (see [1] for more details). The challenge of identifying an optimal measure for quantifying aleatoric and epistemic uncertainty within a credal set remains an active and important area of research.

- **Evaluation of uncertainty quantification (e.g., Figures 4 and 6)** Methods such as the accuracy-rejection curve and out-of-distribution (OOD) detection (mentioned by reviewer **gqZt**) are valuable for indirectly evaluating uncertainty quantification methods when only zero-order data is available. With first-order data, we can employ more effective methods to assess our approaches. Generally, with access to the ground truth first-order distribution, we should expect the following: When epistemic uncertainty (EU) is low, the quantified aleatoric uncertainty (AU) should be close to the entropy of the ground truth first-order distribution. However, when the EU is high, the quantified AU may vary, either close to or distant from this entropy. This is why we designed AU vs. EU plots, as shown in Figures 4 and 6.

  - The observed functional dependency (mentioned by reviewer **LLVn**) between quantified aleatoric and epistemic uncertainty is partly due to the measure we use. Generally, higher epistemic uncertainty implies a larger uncertainty interval, leading to a lower minimum value, i.e., aleatoric uncertainty. Our experiments with synthetic data in Appendix E.1 illustrate the behavior of the proposed credal sets and the performance of uncertainty quantification. As shown in Figure 12, this dependency disappears as epistemic uncertainty decreases.

  - Another unintended behavior, resulting from the lack of adaptivity in the first-order predictor, is evident in Figure 4, where low EU (below 0.5) is not paired with low AU. To observe such cases, very small credal sets concentrated around the corners would be needed. However, due to the lack of adaptivity, our method includes all distributions within a certain distance of the predicted one in the credal set, regardless of its position. This issue is similar to the problem in standard Conformal Prediction (CP) for regression when using absolute error as the nonconformity score, which is often criticized for its lack of adaptivity. In the CP for regression literature, normalized nonconformity measures address this issue by dividing the score for each point by a measure of its dispersion [2]. In Appendix D of the paper, we introduced a normalized nonconformity score that divides the distance function by the entropy of the predicted distribution. Figure 10 demonstrates the successful improvement in the adaptivity of our proposed approach.

- **New figures** If the current figures are still not as clear as we’d like, we are happy to enhance the readability by including additional figures similar to those attached to this rebuttal. We can plot the absolute difference between the quantified aleatoric uncertainty and the entropy of the ground truth distribution against the quantified epistemic uncertainty. This approach might make it easier to see that we expect a low error in this difference when epistemic uncertainty (EU) is low and a broader range of error when EU is high. We’re flexible and open to making such improvements to ensure the results are as clear and informative as possible.

[1] E. Hüllermeier et al., "Quantification of credal uncertainty in machine learning: A critical analysis and empirical comparison"

[2] H. Papadopoulos et al., "Normalized nonconformity measures for regression conformal prediction"

Meta-review: The paper presents a new method for predicting credal sets in classification tasks, aiming to represent both aleatoric and epistemic uncertainty with high probability guarantees. The method is grounded in solid theory and is validated through experiments on datasets like ChaosNLI and CIFAR10-H. However, the paper received mixed feedback from reviewers. Some praised its novelty and effective uncertainty representation, while others pointed out the lack of comparisons with other uncertainty quantification methods and raised concerns about practical applicability, computational complexity, and the key bounded noise assumption.

In their rebuttal, the authors tried to addressed these concerns by explaining their theoretical contributions and the difficulties of making direct comparisons with other methods. They acknowledged the paper's limitations but defended their approach, emphasizing its unique aspects. 

Overall, although the paper would benefit from further empirical comparisons,  the paper introduces a truly novel approach for uncertainty quantification, which is worth to be explored by the community.
Decision: Accept (poster)