Decision-Driven Calibration for Cost-Sensitive Uncertainty Quantification

Venue: NeurIPS_cc_2024_Workshop_BDU
Published: 2024-09-06 05:37:42

Abstract: In recent years, the ability of artificial intelligence (AI) systems to quantity their uncertainty has become paramount in building trustworthy AI systems. In standard uncertainty quantification (UQ), AI uncertainty is calibrated such that the confidence of its predictions matches the statistics of the underlying data distribution. However, this method of calibration does not take into consideration the direct influence of UQ on the subsequent actions taken by a downstream decision-maker. Here we demonstrate an alternate, decision-driven method of UQ calibration that explicitly minimizes the incurred costs of downstream decisions. After formulating decision-driven calibration as an optimization program with respect to a known decision-maker, we show in a simulated search-and-rescue scenario how decision-driven temperature scaling can lead to lower incurred decision costs.

Main paper: 
 introduction: 
 Artificial intelligence (AI) has become an integral component in decision-making pipelines in many settings across society, ranging from medicine to scientific discovery. Rather than providing only a single predicted output in these systems, it has become common for AI to also supplement their predicted outputs with quantified notions of uncertainty describing the level of confidence in predictions i.e., perform uncertainty quantification (UQ). For a decision-maker taking actions in downstream tasks, only considering the AI's predicted output in isolation runs the risk of discounting critical information about possible sources of ambiguity in the AI's output, while taking the full UQ distribution into consideration can better enable informed decision-making that trades off the potential costs and benefit of various actions [1] . For instance, a doctor aided by an AI diagnosis system can better weigh the risks of various patient treatment options if the AI provides a confidence level for its diagnosis, only moving ahead with risky treatments if the AI is confident enough.Rather than serving only as a passive aid, in general we expect the specific UQ outputs generated by the AI to have a significant impact on the actions actually taken by downstream decision-makers consuming these outputs. Continuing with the above example, if the doctor only proceeds with a treatment if the AI's confidence exceeds a particular threshold (e.g., 90%), then the difference of only a few confidence percentage points may have a drastic effect on the patient's outcome. This implies that the selection and tuning of the UQ algorithm (e.g., temperature scaling [2] , conformal prediction [3] ) can potentially result in different behaviors by the downstream decision-maker [4] .Based on these observations, we note that common UQ calibration techniques minimizing distributional metrics such as Expected Calibration Error (ECE) do not necessarily improve decision-making performance in the downstream application of interest, as measured by incurred action costs. Recognizing the critical role of UQ in the overall decision-making pipeline, in this work we introduce Decision-Driven Calibration (DDC) as a means of optimizing UQ parameters to explicitly minimize the expected downstream decision cost. Although aspects of decision-aware calibration have been explored in prior work, to our knowledge this work is the first to optimize UQ for minimization of incurred costs with respect to a given decision-cost function and a given (potentially non-ideal) decision-maker profile. We demonstrate in a simulated search-and-rescue task how DDC can reduce decision costs in comparison to standard temperature scaling-based calibration.Related work UQ comprises a vast body of work, providing a multitude of methods for learning uncertainty estimates from data [5, 6] ; here we specifically focus on prior work leveraging knowledge of the downstream decision process to inform UQ selection and calibration. [7] focuses calibration in output probability regions that have the most impact on diagnostic decision making, but considers only the scenario of an unknown decision cost function in narrow scenarios. Another line of work introduces the notion of Decision Calibration Error (DCE), measuring the discrepancy between the estimated decision costs incurred by an output probability distribution in comparison to the true costs incurred under the true data distribution [8, 9, 10] . While optimizing UQ to minimize DCE does ensure that decision costs computed under the UQ distribution are reliable estimates of the true expected cost, this is distinct from our goal of optimizing UQ to minimize the incurred decision costs themselves. [11] does optimize UQ to minimize downstream decision costs while taking into account a given decision-maker model, but is limited to the case of binary decisions. [12] uses a fixed decision-maker model to optimize conformal prediction for minimizing decision-maker classification error; however, this approach is limited to classification actions. [13] explicitly optimizes UQ to minimize incurred decision-making costs, but the decision space is limited to the decision-maker either accepting the AI's recommendation or solving the task themselves. 
methods: 
 Let X denote a data domain (e.g., X = R d ) with individual examples denoted by x. Here we consider the scenario of multi-class classification where each x is associated with a label y ∈ 1 . . . C, but the concepts demonstrated here could be applied to other settings (e.g., regression). Consider an AI model given by the function f θ : X → R C mapping from examples to "logit" vectors z = f θ (x). In standard classification, the AI model's prediction would be taken as y = arg max y ′ f (y ′ ) θ (x), where f (i) denotes the ith entry of f . More generally, the logit output z can be used to generate a UQ output such as a probability distribution (via Softmax) or a conformal set. Adopting notation from [4] , for generality let g ϕ : R C → U denote a generic UQ function parameterized by ϕ and mapping to an uncertainty representation space U. For instance, for Softmax class probabilities we have g : R C → ∆ C-1 given by g(z) = σ(z) where σ is the Softmax function.We suppose that a decision-maker observes g ϕ (f θ (x)) and chooses an action a from some set of K actions A, noting that it need not be the case that A = 1 . . . C or even for K = C. Without loss of generality we model the decision-maker as a known conditional probability distribution δ : U → ∆ K-1 and assume that the action a for data point x is sampled from δ(g ϕ (f θ (x))). We assume that taking action a on example x with ground-truth label y incurs a cost of c(y, a), where c(•, •) is a known cost function. For data x, y distributed according to D X,Y , we define the expecteddecision cost as C(ϕ) = E x,y∼D X,Y E a∼δ(g ϕ (f θ (x))) [c(y, a)].For optimal decision-making (in an average sense), the expected incurred cost C should be as low as possible.Decision-agnostic calibration In typical calibration, the parameters of g ϕ are adjusted such that the UQ output matches the underlying statistics of the data distribution. In practice this is typically achieved by optimizing a scoring rule ℓ over a calibration set S cal sampled from D XY :EQUATIONIn this work we focus on standard Temperature Scaling (TS) applied to a label distribution [2] optimized with Negative Log-likelihood (NLL) as the UQ method and associated calibration criteria.TS adjusts the model's output probabilities by dividing the logits by a scalar "temperature" parameter T which smooths the probability distribution without altering the ranking of predicted classes. In our notation, this corresponds to a UQ function g T (z) = σ( z /T ), where the only UQ parameter ϕ to be optimized is the temperature T . Letting g (i) denote the ith entry of g, the NLL loss is given byℓ NLL (g T (z), y) = -log g (y)T (z) which can be optimized in (1) over the calibration set.Decision-driven calibration While performing standard calibration as in (1) will encourage the UQ outputs g to match the underlying statistics of the data distribution, this method of calibration does not necessarily ensure a reduced decision cost C. Instead, as a loss function in (1) we utilize a decision-driven calibration loss, computed as the expected decision cost incurred with respect to a fixed decision-maker δ (denoting the ith entry of δ by δ (i) ):EQUATIONWe refer to the optimization of (1) with ℓ DDC as decision-driven calibration (DDC). Although DDC assumes that both δ and c are fixed and known, it is possible to learn δ from observed decision-maker behavior ( [11, 12] ), and in many settings numeric costs can be assigned to various action outcomes.We consider two possible decision-maker models for optimization in (2) that accept as input a label distribution and generate a decision:Smooth Bayes' Optimal: It is well-known that the average-case optimal action is the Bayes' optimal action minimizing the expected incurred cost, i.e., a * = arg min a y ′ p (y ′ ) c(y ′ , a), where p is the true conditional probability of an example having label y. Since the decision-maker only has access to g(z) and not p * , they can instead take a Bayes' optimal action with respect to g(z) as a = arg min a y ′ g (y ′ ) (z) c(y ′ , a) (or in vector notation as δ = 1 a= a , i.e., the one-hot vector at a ′ ).To make this decision-maker differentiable, we apply the Softmax approximation 1 i=arg max i z (i) ≈ σ(z) and express the arg min in a as a negated arg max, leading to δ = σ(-L T g(z)) where L is the C × K matrix whose L[i, j] entry is given by c(i, j).Proportional: We model the decision-maker as first selecting a label y ′ with probability g (y ′ ) (z), and then taking the optimal action for that class. This corresponds to a probability of action a given by a y ′ A[y ′ , a] g (y ′ ) (z) where A is the C ×K matrix given by A[i, j] = 1 for j = arg min a c(i, a)and A[i, j] = 0 otherwise, which we can express in vector form as δ = A T g(z). 
simulated decision-making: 
 We aim to demonstrate in a realistic decision-making setting how DDC can result in reduced test-time decision costs when applied to TS, in comparison to standard calibration. To do so, we simulate an AI-assisted search-and-rescue scenario where a drone is tasked with patrolling a section of beach and assisting swimmers in distress. We assume the drone will encounter objects belonging to three classes: boat, swimmer without a life jacket, and swimmer with a life jacket. In this example we assume that all swimmers without a life jacket are in distress, while all swimmers with a life jacket are not in distress. Upon encountering a new object, the drone can take one of three actions: continue its route without intervention (Do Nothing), "mark" the object to return to later (Mark Object), or drop a flotation device and call for help (Rescue). For each class-action pair we assign a nominal numerical "cost" characterizing the quality of each decision, presented in Table 1 and justified as follows: the most severe error is ignoring a swimmer without a life jacket (i.e., in distress)."Marking" these swimmers as objects of interest is also a significant error, since it delays rescue efforts. "Rescuing" or "marking" a boat or a swimmer with a life jacket incurs a moderate cost, to reflect wasted effort.We simulate this decision-making process on the multi-object tracking (MOT) subset of SeaDronesSee [14] , a large-scale dataset of overhead aerial drone footage (see example images in Figure 2 ). This dataset consists of 22 video clips, encompassing a total of ∼50k frames and ∼400k annotations and contains objects in the classes described above. 1 We split SeaDronesSee-MOT as follows: for our "test" set we use the original "valiation" split. To construct a "calibration" set, within each video we partition the original MOT "training" set by allocating the first 75% for model training and the final 25% for calibration. To focus only on the task of object classification, we crop each image to the ground-truth bounding boxes of the objects in each image.This results in ∼120k training instances, ∼40k calibration instances, and ∼50k testing instances.For the AI predictive model f θ mapping from cropped images to logits z, we train a single linear layer as the classification head on a ViT_B_16 backbone [15] . The ViT backbone was pretrained on the SWAG dataset and fine-tuned end-to-end on ImageNet-1k [16] . After freezing the backbone, we fine-tune the classification head on our custom SeaDronesSee-MOT training split (excluding the calibration set). We used a learning rate of 0.003, Cross-Entropy loss with a class re-weighting scheme from [17] , and optimized with the AdamW optimizer [18] . As a UQ output, we then apply TS to the model Softmax outputs σ(z), optimizing over the calibration split using either NLL or DDC calibration loss. Further training details can be found in Appendix A.Our main objective is to compare the average decision cost at test-time incurred by decision-driven calibration TS in comparison to decision-agostic (i.e., standard) TS. For a given T, we measure this incurred cost over a test set S test asC(T ) = 1 |Stest| x,y∈Stest ℓ DDC (g T (f θ (x)), y),where ℓ DDC is computed with respect to a test-time decision-maker δ test not necessarily equal to the one used during decision-driven calibration (when applicable). In particular, we evaluate all TS methods (Decision-agnostic TS, Smooth Bayes TS, and Proportional TS) against both the Smooth Bayes and Proportional decision-makers (Figure 1 , additional analyses in Appendix B). When computing C(T ) for temperatures obtained from Smooth-Bayes DDC, we observe a smaller incurred decision cost than Decision-agnostic TS. However, we find that calibrating according to a Proportional decision-maker incurs worse cost than decision-agnostic TS. Similarly, when evaluating against a Proportional decision-maker at test-time, Proportional TS incurs a smaller cost than Decision-agnostic TS, which itself incurs a lower cost than Smooth Bayes TS. These results indicate that performing decision-driven calibration with temperature scaling does lead to reduced costs at test time, but a mismatch in decision-maker between calibration and test-time can actually increase decision costs. motivates the approach in [11, 12] of accurately estimating a decision-maker before calibration occurs. Important future avenues of work include a theoretical study of the robustness in DDC to decision-maker mismatch or shifts in decision-making behavior, and the utilization of active learning [19] to efficiently estimate a decision-maker model from a minimal number of queries. To simplify our decision-making scenario we discard objects of the empty life jacket class. 
 
 Appendix:  



Reviews: review: title: title: Great idea, but format and presentation can be improved
main_review: main_review: Summary: 
The paper proposes a new decision-driven calibration to directly optimize UQ to minimize incurred costs. 

Strengths:
1. Strong technical explanation of the underlying concept. 
2. The objective is clearly stated and justified. 
3. The examples are appropriate and of high relevance for real-world applications. 

Weaknesses:
1. Many sentences are too long and wordy. It is difficult to follow the logical flow of the idea. My recommendation would be to break any long sentence into smaller ones that comprise only one idea. 
2. The format of the paper is sometimes lacking clarity. For example, there is no clear delineation to where the conclusion starts and where the results end. 
3. The paper is limited in visualisations. A higher number and better visuals might aid the reader in following the paper better. 

Overall, it is a good idea, but a hard to read paper.
title: title: Comment for Decision-Driven Calibration for Cost-Sensitive Uncertainty Quantification
main_review: main_review: **Pros**:
1. The paper introduces a novel and interesting approach by considering the direct influence of Uncertainty Quantification (UQ) on the subsequent actions taken by a downstream decision-maker. This perspective adds valuable insight to the field and worths further exploration.

**Cons**:
1. There are several technical errors in the paper that need revision. For example:
   - On page 2, line 58: The notation $X = ℝ^d$ should be corrected to $X ∈ ℝ^d$.
   - On page 4, line 139: The model name "ViT_B_16" should be revised to "ViT-B/16" in accordance with Dosovitskiy et al. [1].
   
2. On page 3, line 96, the paper uses $σ(z)$ for approximating the $argmax$, but it is strongly recommended to introduce a temperature parameter. This would help control the approximation effect, as a lower temperature results in a closer approximation (and lower gradient flow).

3. The paper lacks a conclusion section, which is crucial for summarizing the key findings and contributions. Additionally, Figure 1 requires significant improvement. For instance, the text on the X-axis could be formatted into multiple rows for better readability.

4. It is strongly suggested that the paper includes a discussion on the differences between this decision-driven approach and policy gradient methods. The expected decision cost formulation on page 2, line 74, shares similarities with policy gradient, and addressing this connection would provide greater clarity and depth to the proposed method.

---

**Reference**:
[1] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in *International Conference on Learning Representations*, 2021.