Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation
Abstract
Uncertainty quantification has received increasing attention in machine learning in the recent past.
In particular, a distinction between aleatoric and epistemic uncertainty has been found useful in this regard.
The latter refers to the learner’s (lack of) knowledge and appears to be especially difficult to measure and quantify.
In this paper, we analyse a recent proposal based on the idea of a second-order learner, which yields predictions in the form of distributions over probability distributions.
While standard (first-order) learners can be trained to predict accurate probabilities, namely by minimising suitable loss functions on sample data, we show that loss minimisation does not work for second-order predictors: The loss functions proposed for inducing such predictors do not incentivise the learner to represent its epistemic uncertainty in a faithful way.
Introduction
The notion of uncertainty has received increasing attention in machine learning (ML) research in the last couple of years, especially due to the steadily increasing relevance of ML for practical applications.
In fact, a trustworthy representation of uncertainty should be considered as a key feature of any ML method, all the more in safety-critical domains such as medicine [27, 17] or socio-technical systems [25, 26].
In the literature, two inherently different sources of uncertainty are commonly distinguished, referred to as aleatoric and epistemic [9].
While the former refers to variability due to inherently random effects, the latter is uncertainty caused by a lack of knowledge and hence relates to the epistemic state of an agent.
Thus, epistemic uncertainty can in principle be reduced on the basis of additional information, while aleatoric uncertainty is non-reducible.
The distinction between different types of uncertainty and their quantification has also been adopted in the recent ML literature [21, 13], and various methods for quantifying aleatoric and epistemic uncertainty have been proposed [10].
In the context of supervised learning, the focus is typically on predictive uncertainty , i.e., the learner’s uncertainty in the outcome
y ∈ Y given a query instance x ∈ X for which a prediction is sought.
The aleatoric part of this uncertainty is due to the supposedly stochastic nature of the dependence between instances and outcomes, e.g. due to wrong class annotations or a lack of informative features.
Therefore, the “ground-truth” is a conditional probability distribution p ( · | x ) on Y , i.e., each outcome y has a certain probability p ( y | x ) to occur.
Even with full knowledge about p ( · | x ) , the outcome cannot be predicted with certainty.
Obviously, the learner does not have full knowledge of p ( · | x ) .
Instead, it produces a “guess” ˆ p ( · | x ) on the basis of the sample data provided for training.
Broadly speaking, epistemic uncertainty is uncertainty about the true probability and hence the discrepancy between p and ˆ p .
This (secondorder) uncertainty can be captured and represented in different ways.
In the Bayesian approach, for example, the learner’s uncertainty is represented by the posterior predictive distribution, which results from the posterior on the hypothesis space [7, 6]; in other words, uncertainty about the probabilistic predictor is translated into uncertainty about the prediction in a point x .
Alternatively, the authors of [12] propose to capture the learner’s epistemic uncertainty by means of a kind of meta-learner, which seeks to predict the difference between the total uncertainty (expected loss of the actual predictor) and the aleatoric uncertainty (expected loss of the Bayes predictor); this excess loss is then equated with the learner’s epistemic uncertainty.
Yet another quite popular idea is to estimate uncertainty in a more direct way, and to let the learner itself predict, not only the target variable, but also its own uncertainty about the prediction [22, 18, 19, 20, 3, 11, 15].
For example, instead of predicting a probability distribution ˆ p ( · | x ) , the learner may predict a second-order distribution in the form of a distribution of distributions or a set of distributions [23].
The learner’s epistemic uncertainty is then represented by the “peakedness” of the former and the size of the latter.
Either way, looking at existing methods, it appears that epistemic uncertainty is difficult to quantify in an objective manner.
In fact, most methods dispose of parameters or other means that directly influence the amount of uncertainty, rendering epistemic uncertainty quantification arbitrary to a large extent.
At second glance, this is perhaps not very surprising, because, unlike aleatoric uncertainty, epistemic uncertainty is not a property of the data or the data-generating process, and there is nothing like a “ground truth” epistemic uncertainty.
Instead, the learner’s uncertainty is influenced in various ways, for example by underlying model assumptions and the prior knowledge it is equipped with.
For example, enlarging the learner’s hypothesis space and allowing it to fit the data in a more flexible way will increase its epistemic uncertainty [10].
This is comparable to Bayesian inference, where the informedness of the posterior strongly depends on the informedness of the prior (unless the sample size is very large).
In this paper, we demonstrate the difficulty of epistemic uncertainty quantification for one of the approaches that have recently been proposed in the literature, namely, the direct prediction of secondorder distributions through empirical loss minimisation.
Analysing this approach in a critical way, we isolate problems questioning its practicability (Section 3).
These concerns are substantiated by formal results showing that the approach does not behave as it is supposed to do (Section 4).
These formal results are supported by simulations on synthetic data sets (Section 5)
Setting and Notation
Considering classification as a learning task, we assume a standard setting with instance space X , label space Y =
{ y 1 , . . .
, y K } , and training data D = (cid:8)(cid:0) x
( n ) , y ( n ) (cid:1)(cid:9) N n =1 ⊂ X
× Y .
As usual, we also assume that the data is generated i.i.d.
according to an underlying joint probability measure P on X ×Y , i.e., each z ( n ) =
( x ( n ) , y ( n ) ) is a realisation of Z =
( X, Y ) ∼ P .
Correspondingly, each instance x ∈ X is associated with a conditional distribution p ( · | x ) on Y , such that p ( y | x ) is the probability to observe label y as an outcome given x .
Let P ( Y ) denote the set of probability distributions on Y , which can be identified with the K -simplex
[Equation]
of probability vectors θ , each of which identifies a categorical distribution Cat( θ ) .
Slightly abusing notation
, we shall not distinguish between vectors and distributions (functions); for example, we write θ ( y ) instead of p ( y ) , which means that θ ( y )
=
θ k if y = y k .
A summary of the notation used in this paper is given in Section A.
2.1 Learning Predictive Models
Suppose a hypothesis space H to be given, where a hypothesis h ∈ H is a mapping X −→ ∆ K .
Thus, a hypothesis maps instances x ∈ X to probability distributions on outcomes.
In standard supervised learning, the goal of the learner is, based on a loss function L : ∆ K × Y −→ R , to induce a hypothesis (predictive model) with low risk (expected loss)
[Equation]
The choice of a hypothesis is commonly guided by the empirical risk
[Equation]
i.e., the performance of a hypothesis on the training data.
However, since R emp ( h ) is only an estimation of the true risk R ( h ) , the empirical risk minimiser ˆ h .. =
argmin h ∈H R emp ( h ) (or any other predictor) favored by the learner will normally not coincide with the true risk minimizer (Bayes predictor) h ∗ ..
= argmin h ∈H R ( h ) .
Correspondingly, there remains (epistemic) uncertainty regarding h ∗ as well as the approximation quality of ˆ h (in the sense of its proximity to h ∗ ) and the predictions ˆ p ( · | x ) =
ˆ h ( x ) produced by this hypothesis.
2.2 Learning Level-2 Predictors
Here, motivated by recent methods for uncertainty quantification, we are interested in learning a second-order or level-2 predictor that is able to properly represent its own (epistemic) uncertainty, that is, a hypothesis of the form
[Equation]
where ∆ (2) K = P ( P ( Y )) denotes the set of second-order distributions, i.e., probability distributions on ∆ K .
If H ( x ) = Q ∈ ∆ (2) K , then Q assigns a probability (density) Q ( θ ) to each distribution θ ∈ ∆ K , and the more certain the learner about the true distribution, the more concentrated Q is.
If second-order or level-2 distributions are Dirichlet (cf.
Appendix B), then every Q ∈ ∆ (2) K is identified by a parameter vector α ∈ R K + , and hence ∆ (2) K
with the parameter space R K + .
Thus, hypotheses are of the form H : X −→ R K + , where H ( x ) =
α ( x ) =
( α 1 ( x ) , . . .
, α K ( x )) .
Thus, the Dirichlet parameters α k are expressed as a function of instances, and this dependence is supposedly captured by the underlying hypothesis space H .
2.3 Special Scenarios
For ease of exposition, we shall specifically look at the following scenarios, which are important special cases of the general setting:
• Binary classification: This is the case K = 2 , where Y = { 0 , 1 } consists of only two classes.
The conditional distribution p ( · | x ) is now determined by the probability vector
θ =
( θ 0 , θ 1 )
, i
e., by the two probabilities θ 0 = p (0 | x ) that Y = 0 and θ 1 = p (1 | x ) that Y = 1 .
As these sum up to 1, the learning problem effectively comes down to inference about the ground truth probability θ 1 ( x ) =
p (1 | x ) of the positive class, and hence about the parameter of a Bernoulli distribution.
• Coin tossing:
This is a further simplification of the binary case, which can be seen as learning without instance space.
Or, equivalently, we may assume an instance space X
= { x 0 } consisting of only a single instance, which is observed over and over again (and can therefore be ignored, as it does not carry any information).
Like in the binary case, learning comes down to estimating a ground truth Bernoulli distribution with parameter θ 1 ≡ p (1 | x ) , with the difference that this parameter no longer depends on x .
As an important difference, note that the coin tossing scenario provides several observations pertaining to a single parameter θ 1 , i.e., several realisations of the same Bernoulli random variable, and hence information in the form of relative frequencies .
This yields a solid statistical basis for estimating θ 1 .
In the more general (machine learning) scenario, one can assume that at most a single observation is made in a point x ∈ X , which, in principle, does not allow for estimating a probability [1].
The common way out is to make regularity assumptions , so that outcomes observed for nearby points are also deemed representative for x to some extent.
This becomes especially explicit for local learning methods such as decision trees and nearest neighbours, where the class probabilities are assumed to be constant within a certain region of the instance space — effectively, learning in such a region is thus again reduced to the coin tossing scenario.
Level
Representation
Loss
[Table] [Table]
y
[Table]
Learning Level-2 Predictors
In supervised learning, (level-0) samples drawn from the categorical random variable Y ∼ Cat( θ ) are made available (explicitly or implicitly) as a basis for learning the level-1 distribution θ .
However, corresponding samples are actually not provided for the level-2 distribution Q .
In principle, to estimate Q , for example a Dirichlet parameter α , observations of realisations of that distribution would be needed, i.e., a sample in the form of probability vectors θ (1) , . . .
, θ ( N ) ∼ Dir
( α ) .
Given data of that kind, α could be estimated by means of maximum likelihood maximisation or any other statistical method.
However, such data cannot exist even in principle, because θ (resp.
θ ( x ) ) is supposedly constant.
This suggests that (probabilistic) learning on the epistemic level cannot be frequentist in nature, unlike learning (about θ ) on the aleatoric level.
Instead, it appears that learning on the epistemic level is necessarily Bayesian and requires a prior, which then of course has an influence on the degree of (epistemic) uncertainty.
We shall return to this point in Section 3.2.
In light of this, one may also wonder whether it is possible to learn a level-2 predictor (4) in the “classical” way through loss minimisation, just like a level-1 predictor.
In other words, is it possible to specify a level-2 loss function
[Equation]
comparing level-2 predictions Q ( x ) with level-0 observations y , so that minimising L 2 on the training data D yields a “good” level-2 predictor?
This is the basic idea of direct epistemic uncertainty prediction [22, 18, 19, 20, 3, 11, 15].
Before the above question can be addressed, we need to clarify what we mean by “good” predictor.
For level-1 predictors, this question is answered through the notion of proper scoring rules [8].
These are loss functions L 1 that compare (first-order) probability distributions θ with outcomes
Y and guarantee that the loss minimiser coincides with the ground truth distribution θ ∗ (at least asymptotically).
In other words, the expected loss
[Equation]
is minimised by predicting ˆ θ = θ ∗ .
Consequently, proper scoring rules provide a loss-minimising learner with an incentive to predict the true distribution, e.g. log-loss or quadratic loss [16].
This concept cannot be transferred directly to the case of level-2 predictions, simply because, as already mentioned, there is no ground-truth Q ∗ .
Instead, a level-2 representation is a representation of the learner’s belief about the level-1 ground truth θ ∗ .
So what exactly should be the purpose of a loss L 2 ?
In this regard, one should first of all notice that a (level-1) loss function may in general serve different purposes:
• A loss can be a target loss , which essentially means that it is determined by the application:
L (ˆ θ , y ) or L (ˆ y, y ) is the real cost caused by the level-1 prediction ˆ θ or the level-0 prediction
ˆ y when the ground-truth is y .
• A loss can be a surrogate loss , which means that it serves an auxiliary purpose and is used in a more indirect way: minimising the loss helps to achieve the actual goal, such as probability estimation in the case of proper scoring rules.
Another example is the use of the hinge loss as a surrogate in classification; being convex and continuous, it simplifies training, although the true target is the 0/1 loss.
A level-2 loss
should
arguably be more of the second kind, and its purpose should be
twofold
: It should incentivise the learner to make predictions that are correct in the sense of assigning high probability to the ground-truth θ ∗
, and at the same time faithful in the sense of appropriately expressing the learner’s (epistemic) uncertainty.
The second point appears to be specifically delicate, due to the lack of an objective ground-truth.
In fact, one may wonder how it should be possible to evaluate the faithfulness of a prediction on the basis of empirical data in the form of observed class labels.
Besides, there is of course a risk of imposing the epistemic uncertainty on the learner, i.e., of incentivising a representation of uncertainty only because it appears favourable from a loss minimisation perspective.
3.1 Averaging Level-1 Losses
Several authors have proposed the minimisation of an empirical loss of the form
[Equation]
where Q ( n ) =
H ( x ( n ) ) .
Thus, an individual prediction Q is penalised in terms of the expected level-1 loss, with the expectation taken over the realisations of θ .
For example, in [3] the levelloss is defined in terms of the cross entropy CE
( θ , y ( n ) )
and (8) is called the uncertain cross entropy loss , while in the evidential networks approach [22], L 1 is the quadratic loss (Brier score).
This approach suggests an interpretation in terms of a “matching learner” which samples predictions θ ∼ Q at random according to its current belief Q .
But why should a learner predict in this way?
For a learner seeking to minimise expected loss, wouldn’t it be better to predict the most likely probability θ throughout?
Indeed, if L 1 is a convex loss, then Jensen’s inequality implies that
[Equation]
where ¯ θ = E θ ∼ Q θ is the expected level-1 prediction.
As can be seen, for the learner it is better to predict the expected probability rather than sampling a probability θ at random.
As a consequence, the learner will have a tendency to peak the level-2 distribution, thereby pretending full certainty rather than representing uncertainty in an honest way.
The same happens in the case of a concave loss 1 , although here the tendency is toward extreme predictions.
For illustration, consider the coin tossing scenario (i.e., estimation of a constant Bernoulli parameter θ 1 ), and suppose that N 1 positive and N 0 negative examples have been observed, hence N = N 0 +
N 1 samples in total.
Then, assuming level-2 predictions Q in the form of Dirichlet distributions
Dir( α ) , the loss minimiser of (7) is given by the Dirichlet peaked at θ 1 = N 1 / N , i.e., α = ( c (1 − θ ) , c θ ) for c → ∞ .
So strictly speaking, the loss minimiser is not even well defined.
But perhaps more important than this technical issue is the observation that the learner will always pretend full certainty about θ 1 , regardless of the sample size.
The case of a concave loss L 1 leads to even more questionable predictions.
Here, one obtains the
Dirichlet peaked at θ 1 = 0 resp.
θ 1 = 1 , i.e., α = ( c, 0) resp.
α = (0 , c ) for c → ∞ , in the case where N 0 > N 1 resp.
N 0 < N 1 .
Thus, the learner may even pretend full certainty about a distribution (e.g., θ 1 = 1 ) although that distribution is definitely excluded as the ground truth (e.g., because N 0 > 0 ).
Note that a very similar effect can be observed “one level below” (level-1 loss as expected levelloss): Consider a level-1 learner holding a probability p ∈ ∆ K .
This learner could be assessed by averaging over level-0 losses, i.e.,
[Equation] [Equation]
) where L 0 could be the 0/1 loss.
Again, even if p is a proper expression of the learner’s aleatoric uncertainty, it will not be the minimiser of (10) and hence not be delivered by a loss-minimising learner.
Instead, it will be best to predict the mode y ∗ of p.
3.2 Bayesian Losses and Regularisation
Adopting a Bayesian perspective, a level-2 prediction Q would naturally be seen as the posterior uncertainty about θ given the data, i.e.,
[Equation]
where Q 0 is a prior on ∆ K .
Roughly speaking, (8) only captures the first part on the right-hand side, the likelihood, but not the second part, the prior.
Once again, this explains why putting all mass on a single θ , namely the one with the maximum likelihood, is a plausible strategy.
This will of course be avoided by proper Bayesian inference, because the posterior will then be a compromise between the likelihood and the prior Q 0 , which serves as a regulariser.
Interestingly, a close connection between Bayesian inference and learning through loss minimisation has been established in [2].
There, it is shown that a posterior Q is the minimiser of the loss
[Equation]
where d KL denotes the KL-divergence and L 1 ( θ , y ) is the loss caused by θ on the observation y .
The latter is given by the logarithmic (or self-information) loss − log f ( y | θ ) when the data is known to be generated by the distribution with density f
( y | θ ) , in which case (11) coincides with standard Bayesian inference.
However, L 1 can also be another loss in case the data-generating process is not known.
The authors consider (11) as a Bayesian version of conventional empirical risk minimisation, when the interest is on probability measures Q on ∆ K rather than point estimates θ ∈ ∆ K .
The general solution can be shown to be of the following form:
[Equation]
Using the loss (11) in (7) yields the level-2 empirical loss
[Equation]
where
[Equation] [Equation]
[Equation]
The regularisation parameter λ might be needed in the general case where L 1 is any loss function not necessarily linked to an underlying density f .
In that case, because L 1 could be scaled differently, the “fidelity-to-data” and “fidelity-to-prior” parts might not be calibrated, and hence need to be recalibrated through λ [2].
Assuming that the prior is the same for all data points (hence does not depend on n ) and that levelpredictions Q for instances x are of the form Q = H ϕ
( x ) , where ϕ is indexing hypotheses (i.e., the hypothesis space is of the form H
= { H ϕ | ϕ ∈ Φ } ) and can be thought of as the model parameters fit to the data D , we obtain
[Equation]
[Equation]
Moreover, if Q 0 is the uniform distribution, then the latter is the same as
[Equation]
[Equation]
since the KL-divergence reduces to the (negative) entropy of the posterior.
This is essentially the loss that is also used in the Posterior Network method [3], where L 1 in (15) is given by the cross-entropy, and in the evidential networks approach [22], with L 1 being the Brier score.
Note that, with a level-2 hypothesis H ϕ , we can naturally associate the level-1 hypothesis
[Equation]
(18) which makes point predictions in the form of single probability distributions.
For example, if levelhypotheses H ϕ are Dirichlet, i.e., H ϕ ( x ) =
Dir( α ) , then h ϕ ( x ) =
(cid:0) θ 1 ( x ) , . . .
, θ K ( x ) (cid:1) , where
[Equation]
3.3 Discussion
The deviation from the prior Q 0 obviously serves as a regulariser in (11), but it can also be interpreted from an uncertainty quantification point of view.
Recall that (11) can be seen as a compromise between fidelity to data and fidelity to prior.
Suppose the learner delivers the level-2 prediction Q , knowing that the final (level-1) prediction θ will be sampled from that distribution, i.e., θ ∼ Q .
If the learner has a good guess about the true θ ∗ , it should concentrate Q in the corresponding region in ∆ K , making sure that the sampled θ will be close to θ ∗ .
To this end, however, it has to deviate from the prior Q 0 , which causes a cost d KL ( Q, Q 0 ) .
The latter can thus be seen as a measure of the strength of the learner’s belief, i.e., the cost it is willing to pay for the concentration, and therefore as a measure of its certainty.
This matches quite perfectly with the use of the entropy of Q as a measure of epistemic uncertainty, which is obtained when Q 0 is the uniform distribution.
In this regard, it is also worth mentioning that a level-2 prior Q 0 , i.e., a distribution on ∆ K , is not directly comparable with a level-1 prior θ 0 , i.e., a distribution on Y .
This is mainly because
• a distribution Q 0 represents information about a ground truth θ ∗ , which, even if unknown (and treated as a random variable by the
Bayesian
approach), is assumed to exist and to be
unique
, • whereas a distribution θ 0 represents information about a random outcome Y.
For the purpose of illustration, take again the uniform prior as an example.
Taking the uniform prior θ uni on Y as a representation of complete ignorance is often criticised for the reason that this representation does not allow for distinguishing between a real lack of knowledge about the ground truth θ ∗ and perfectly knowing that θ ∗ =
θ uni ; in the first case, θ uni is meant to represent the learner’s epistemic state, in the latter case it rather refers to an objective reality.
On the epistemic level, however, this confusion is actually not possible, because the uniform distribution Q uni on ∆ K cannot have the second interpretation: Assuming that there is a unique ground truth θ ∗ , Q uni cannot represent an objective reality.
Instead, it is clear that it represents the learner’s knowledge.
Note that the prior Q 0 in (14) should not be confused with a prior on the hypothesis space H either.
The latter is commonly required in Bayesian learning.
Assuming that hypotheses are identified by parameters ϕ ∈ Φ , as we did in our setting, it would be a distribution on the parameter space Φ .
Formally, a prior on H would result in a single regularisation term that is added to the entire (cumulative) empirical loss.
As opposed to this, Q 0 is a pointwise penalty, which is part of the level-2 loss function and applied to every training example ( x ( n ) , y ( n ) ) individually.
The choice of the uniform distribution as a prior Q 0 appears to be natural, especially with the “cost for concentration” interpretation in mind, because the uniform distribution is least concentrated among all distributions.
Restrictively, of course, one has to say that a natural uniform prior may not exist in cases where the parameter space Θ is not bounded.
According to our discussion so far, the loss minimisation approach (14) appears to be quite appealing.
However, this approach is not without problems either.
First, the loss minimiser Q will depend on the loss function L 1 , i.e., different representations of uncertainty will be obtained for different losses.
This is questionable, because, even if a point prediction — the “action” taken by the learner in the end — will clearly be influenced by the loss, one may wonder whether the representation of uncertainty should not be independent.
What this suggests is that (14) incentivises the learner to represent its uncertainty about the best prediction rather than the ground truth .
As another, possibly more severe problem, note that the incentive to concentrate the prediction Q and to deviate from the prior Q 0 will also depend on two other factors: first, the actual ground truth θ ∗ itself, and second, the regularisation parameter λ .
For illustration, take again the example of coin tossing.
As for the ground truth, note that a deviation from the uniform prior will certainly be beneficial if the coin is very biased ( θ ∗ 1 is close to 0 or 1).
In that case, sampling θ 1 uniformly at random will likely end up in a poor prediction.
However, if the coin is fair ( θ ∗ 1 ≈ 1 / 2 ), the learner will gain very little by concentrating Q around θ 1 = 1 / 2 , because predicting θ = (1 / 2 , 1 / 2) with probability 1 will yield roughly the same expected loss E y ∼ B ( θ 1 )
L 1 ( θ , y ) as sampling θ at random.
Thus, there is little incentive for the learner to concentrate in this case, especially considering that a concentration causes a cost.
This brings us to the second factor: the regularisation parameter λ has a direct influence on how costly a concentration is.
Therefore, it determines the optimal compromise between L E and d KL , and thereby the uncertainty represented by the learner.
Again, this is a questionable property, as it renders the uncertainty representation rather arbitrary.
An illustration is shown in Fig.
1, where the expectation of loss L 2 ( Q, y ) in (13) with L 1 being the cross-entropy loss is plotted for different concentrations, regularisation parameters, and ground-truth parameters.
As can be seen by the minima, the optimal concentration depends on both, the regularisation and the ground truth.
Thus, even when the learner perfectly knows the ground truth, it may not predict a Q peaked around this parameter, but instead a much flatter distribution, thereby suggesting to be more uncertain than it actually is.
In this regard, we should also put our previous argument in perspective, namely, that a uniform (or, more generally, non-peaked) level-2 distribution cannot represent the ground truth.
Although this is in principle true, the task of the learner, as incentivised by the loss function, is not to learn the ground truth θ ∗ in the first place, but rather to make loss-minimising predictions, and nothing prevents it from predicting a flat Q if it serves the purpose of incurring low loss.
Also note that the influence of the prior does not diminish with an increasing sample size, like in standard Bayesian learning, because there is one prior per training instance (instead of a single prior for the entire data set).
Coming back to our example of tossing a fair coin, a prediction of Q = Q uni should neither be interpreted as “not knowing the ground-truth θ ∗ ”, nor as “not knowing the best prediction”, but rather as “knowing the prediction does not matter”.
This is of course also related to and essentially caused by the information gap, i.e., learning a level-2 prediction based on level-0 (instead of level-1) feedback.
In fact, if the predictions θ ∼ Q would be compared to observed distributions θ ( n ) instead of observed class labels y ( n ) , this would clearly call for concentrating Q around θ ∗ .
Formal Results
In the beginning of the previous section, we asked for the existence of a level-2 loss function that incentivises the learner to report epistemic uncertainty in an honest way.
Based on our previous discussion, one may wonder whether such a loss can exist.
In this section, we provide negative results for both level-2 loss functions considered above.
We give these results for the simple coin tossing scenario, but they can be easily extended to more complex settings.
What we mean by “appropriate loss” is specified as follows.
Definition 1.
A level-2 loss function L 2 : ∆ (2) K × Y −→ R + , is appropriate if the following holds for the empirical loss minimiser Q ( N ) =
argmin
Q
(cid:80) Nn =1 L
(cid:0)
Q, y (
n ) (cid:1) on
any
i
.i.d.
observational data sequence y (1) , y (2) , . . .
with y ( i ) ∼ θ ∗ :
(A1)
For any sample size N , E y (1: N ) ( U ( Q ( N ) )) ≥ E y (1: N +1)
( U ( Q ( N +1) )) , where U is an uncertainty measure 3 and y (1: N ) abbreviates y (1) , . . .
, y ( N ) .
Moreover, there exist some ˜ N and k such that E y (1: ˜ N ) ( U ( Q ( ˜ N ) )) >
E y (1: ˜ N + k ) ( U ( Q ( ˜ N + k ) )) .
(A2) Q ( N ) P
→ δ θ ∗ as N → ∞ , where δ θ ∗ is the Dirac measure at θ ∗ .
In words, ( A 1) stipulates that the learner’s uncertainty should gradually decrease (in expectation) with increasing sample size N .
In the beginning, the empirical loss minimiser Q ( N ) should represent high uncertainty (and ideally be uniform for N = 0 ), and the larger N , the less uncertain Q ( N ) should be in terms of the uncertainty measure U .
( A 2) states that in the limit, i.e., if the sample size goes to infinity, all epistemic uncertainty should disappear, and consequently the empirical loss minimiser should converge (in probability) to the Dirac measure δ θ ∗ , putting the entire probability mass on the ground-truth θ ∗ .
Both of these assumptions are natural (and minimal) requirements for a suitable level-2 loss function, since honesty is required with respect to the epistemic uncertainty specification on the one side, and consistency of the empirical loss minimiser on the other side.
Averaging Level-1 Losses.
The following theorem shows that a loss minimisation approach using a level-2 loss (8) with commonly used level-1 losses such as the Brier score or the log-loss (or cross-entropy) does not lead to an appropriate level-2 loss (cf.
Section C for the proof).
Theorem 1.
For any level-1 loss function L 1 : ∆ K × Y −→ R + that satisfies L 1 ( E θ ∼ Q θ , · ) ≤ E θ ∼ Q L 1 ( θ , · ) , the level-2 loss in (8) is not appropriate.
Both the Brier score and the log-loss are convex and therefore satisfy the property of Theorem 1.
Moreover, the proof reveals an even worse property, namely that the empirical loss minimiser is always a Dirac measure, regardless of the sample size N .
Consequently, epistemic uncertainty is essentially never reported in a proper way.
Bayesian Losses and Regularisation.
Next, we show that loss minimisation using Bayesian losses as in (14) does not lead to an appropriate level-2 loss either, if instantiated with commonly used level-1 losses.
First, we consider the class of level-1 losses being locally Lipschitz-continuous around the minimiser and strictly proper.
Formally, let d (1) be some appropriate metric on ∆ K , then there exists some constant L > 0 depending on L 1 such that for any θ ∗ ∈ ∆ K and any θ which is close to θ ∗ (in a neighborhood N ( θ ∗ ) in terms of d (1) )
it holds that
[Equation]
The following theorem, the proof of which is deferred to Section D, shows that choosing too large a value for the regularisation parameter λ in (14) leads to a violation of Assumption A2.
Theorem 2.
For any strictly proper level-1 loss function L 1 satisfying (20) for any θ , θ ∗ ∈ ∆ K , and if λ > 0 is such that there exists some ˜ Q ∈ ∆ (2) K with support inside N ( θ ∗ ) , the neighborhood of θ ∗ , and L ˜ Q
[Equation]
N
[Equation]
θ
[Equation] [Equation]
ENT( ˜ Q ) < λ, where ˜ Q ( N ( θ ∗ )) is the probability mass assigned to N ( θ ∗ ) by ˜ Q, then the level-2 loss (14) is not appropriate.
The following theorem shows that choosing the regularisation parameter λ in (14) too low leads to a violation of Assumption A1 for level-1 losses similar to those in Theorem 1 (cf.
Section E).
For this purpose, denote by ˜∆ (2) K the subset of ∆ (2) K consisting of all non-Dirac measures in ∆ (2) K .
Theorem 3.
Let L 1 : ∆ K × Y −→ R + be a level-1 loss function such that for any Q ∈ ˜∆ (2) K there exists ε Q > 0 satisfying E θ ∼ Q L 1 ( θ , · ) − L 1 ( E θ ∼ Q θ , · ) ≥ ε Q .
Then, the level-2 loss in (14) is not appropriate if λ ≤ inf Q ∈ ˜∆ (2) K ε Q ENT( Q ) .
Note that both the Brier score and the log-loss are strictly proper, strictly convex and satisfy the local
Lipschitz property (20).
Discussion.
The above results are general in the sense that Q can be any level-2 distribution, not necessarily restricted to Dirichlet distributions.
Moreover, the results do not depend on the underlying uncertainty measure U in Assumption A1 as long as U is not constant as well as maximal for the uniform distribution and minimal for Dirac measures.
A key problem of the loss minimisation approach, as revealed by the above results, is the following: The quality (and hence loss) of a prediction Q cannot be judged solely in the context of a single observation y .
For example, a very uncertain prediction Q (e.g., close to uniform) is completely fine in the beginning, when N is small, but less desirable when N grows large.
In principle, the loss should also consider the current knowledge about θ ∗ , which, however, is missing from its arguments.
For the loss (14), this concretely means that the penalty term should be higher in the beginning and lower later on, which could be achieved by specifying λ as a decreasing function of N rather than a constant.
Then, however, the learner’s uncertainty is again controlled in an external way.
Due to space limitations, we defer further discussion of the theoretical results to the Appendix F.
Experiments
In the following, we investigate our findings regarding the empirical loss minimiser (ELM) (see
Definition 1) in a simulation study on synthetic data.
We consider two representative scenarios for the binary classification setting (i.e., K = |Y| = 2 ): the scenario with the highest aleatoric uncertainty, where p ( y ) =
B (0 . 5) and a low aleatoric uncertainty scenario, where p ( y ) =
B (0 . 05) .
Note that the latter is representative of an imbalanced learning scenario.
For each scenario, we generate repeatedly observations of different sizes N and compute the corresponding ELM for the Brier score as the underlying level-1 loss function in each run (the results for cross-entropy are similar and hence omitted).
As optimizing over all possible level-2 distributions is computationally expensive, we restrict the optimization to two-component mixtures of Dirichlet distributions.
In the following table, we report the mean entropy (together with the standard deviations) of the ELM’s averaged over 10 runs in dependence on the data set size N for different values of λ for both scenarios:
[Figure]
Note that the differential entropy the level-2 distributions takes values in R − so for ease of comparison we state here the (Shannon) entropy of the quantized version of the level-2 distributions [4], where a value of zero corresponds to a one-point distribution and the uniform distribution has a value of 9 . 967 in this case (i.e., 1000 bins are used).
Furthermore, the last line reports the empirical mean for the corresponding sample size.
We see that for small values of λ the ELM is a point-mass (entropy of zero), while for large values of λ and huge sample sizes the ELM is still far away from being a point-mass, which is in line with our theoretical results.
Moreover, for increasing values of λ , the ELM changes quite abruptly from one extreme (maximum certainty) to the other (maximum uncertainty) when the sample size is small.
Finally, the reported level-2 uncertainty is in both scenarios essentially the same for all different choices of λ and sample sizes N , although the level-1 uncertainties are quite different.
This demonstrates that the influence of λ is in a sense quite arbitrary on the faithful epistemic uncertainty representation, as it simply prevents the learner from being too confident without taking the underlying data-generating distribution into account.
In
Appendix G we also consider a multi-class setting, where similar observations are made.
Conclusion
In machine learning, it is well known that probabilistic classifiers can be trained by empirical loss minimisation: Suitably chosen loss functions, so-called proper scoring rules, incentivise the learner to predict probabilities in an unbiased way, or, stated differently, the learner minimises expected loss if (and only if) it predicts the true (conditional) probability distribution.
In this paper, we investigated the question whether second-order predictors can be trained in a similar way.
This is motivated by recent proposals in the literature, where corresponding methods are used to represent the learner’s epistemic uncertainty.
Obviously, the problem is more challenging, especially due to the lack of an objective ground truth, and because a level-2 representation has to be learned from level-0 data, i.e., data at the observational level where only class labels but no probabilities (level-1 data) are observed.
Indeed, our results are negative in the following sense: Level-2 loss functions do not incentivise the learner to predict its epistemic uncertainty in a faithful way.
Instead, to minimise the loss in expectation, the learner is encouraged to pretend more (or less) confidence than warranted.
Besides, contrary to what one would expect, the uncertainty does not decrease with an increasing sample size.
Thus, we believe that the empirical findings reported in the literature should be reconsidered and carefully analyzed in light of our results.
While it is true that good results are obtained in the majority of existing work, it is also true that the reasons for these results are not always fully transparent.
Our results confirm the difficulty of epistemic uncertainty quantification, which, we believe, is a general problem that also applies to other approaches.
Strictly speaking, however, our formal results only hold for the loss functions that have been proposed in the literature so far, and hence do not completely exclude the existence of other types of losses providing the right incentives for the learner.
As future work, we plan to further elaborate on this problem, either coming up with an appropriate loss function or proving that such a loss cannot exist.
Acknowledgments and Disclosure of Funding
Willem Wageman received funding from the Flemish Government under the “Onderzoeksprogramma Artificiel¨e Intelligentie (AI) Vlaanderen” Programme.
[-Moreover, the larger the parameters α k and hence the sum α 
-]