Title: Decoupling Quantile Representations from Loss Function
Venue: NeurIPS 2023
Accepted: no
Keywords: Quantiles

Duality

Calibration

OOD Detection

simultaneous binary quantile regression (SBQR)

invariant to distortion
Abstract: The simultaneous quantile regression (SQR) technique has been used to estimate 2 uncertainties for deep learning models, but its application is limited by the requirement that the solution at the median quantile $(\tau = 0.5)$ must minimize the mean absolute error (MAE). In this article, we address this limitation by demonstrating a duality between quantiles and estimated probabilities in the case of simultaneous
binary quantile regression (SBQR). This allows us to decouple the construction of quantile representations from the loss function, enabling us to assign an arbitrary classifier $f(x)$ at the median quantile and generate the full spectrum of SBQR quantile representations at different $\tau $values. We validate our approach through two applications: (i) detecting out-of-distribution samples, where we show that
quantile representations outperform standard probability outputs, and (ii) calibrating models, where we demonstrate the robustness of quantile representations to distortions. We conclude with a discussion of several hypotheses arising from these findings.

Abstract

The simultaneous quantile regression (SQR) technique has been used to estimate uncertainties for deep learning models, but its application is limited by the requirement that the solution at the median quantile ( τ = 0 . 5 ) must minimize the mean absolute error (MAE). In this article, we address this limitation by demonstrating a duality between quantiles and estimated probabilities in the case of simultaneous binary quantile regression (SBQR). This allows us to decouple the construction of quantile representations from the loss function, enabling us to assign an arbitrary classifier f ( x ) at the median quantile and generate the full spectrum of SBQR quantile representations at different values of τ . We validate our approach through two applications: (i) detecting out-of-distribution samples, where we show that quantile representations outperform standard probability outputs, and (ii) calibrating models, where we demonstrate the robustness of quantile representations to distortions. We conclude with a discussion of several hypotheses arising from these findings.

1 Introduction

Deep learning models have become ubiquitous across diverse domains, and are increasingly being 16 used for several critical applications. Common questions which arise in practice are - (a) Can this 17 model be used on the given data input? and (b) If so, how much can one trust the probability 18 prediction obtained? The former refers to the problem of Out-of-Distribution (OOD) detection [13, 9] 19 and the latter refers to the problem of Calibration [10, 19, 22]. Understanding the applicability of a 20 given deep learning model is a topic of current research [30, 6, 25, 14]. In this article we consider the 21 quantile regression approach to answer these questions. 22

Quantile regression techniques [16, 17] provide much richer information about the model, allowing 23 for more comprehensive analysis and understanding relationship between different variables. In [32] 24 the authors show how simultaneous quantile regression (SQR) techniques can be used to estimate 25 the uncertainties of the deep learning model in the case of regression problems. However, these 26 techniques aren't widely used in modern deep learning based systems since [5] - (a) The loss function 27 is restricted to be mean absolute error (MAE) or the pinball loss. This might not compatible with 28 domain specific losses. (b) Moreover, it is difficult to optimize the loss function in presence of 29 non-linearity. (c) Adapting the quantile regression approach for classification is also challenging due 30 to piece-wise constant behavior of the loss function, due to discrete labels. 31

32

33

34

35

Decoupling loss function and computing quantile representations: Consider the problem setting where a pre-trained classifier f θ ( x ) (including the dataset on which it is trained) is given and we wish compute the quantile representations for detailed analysis of the pre-trained classifier. Classical approach is to retrain the classifier using the quantile loss (equation 2). However, one runs the risk

of losing important properties while retraining since pinball loss would have different properties 36 compared to loss used for pre-training. Moreover, it is not clear how penalties (used for pre-training) 37 given to attributes such as gender-bias etc. should change with the quantile 1 . Further, aspects like 38 calibration error of the pre-trained network cannot be established by retraining with a different loss. 39 Thus, one requires a more elegant solution than retraining using the pinball loss. 40

41

42

43

44

45

46

47

48

49

50

51

Main Outcomes: In this article we propose an approach to decouple the construction of quantile representations from the loss function . We achieve this by identifying the Duality property between quantiles and probabilities. We leverage the duality to construct the quantile-representations for any pre-trained classifier f θ ( x ) in section 3.1. Such quantile representations are shown to capture the training distributions in section 4.2. We show that these representations outperform the baseline for OOD detection in section 4.4. And further verify that quantile representations can potentially identify OOD samples perfectly. We also show that probabilities arising from quantile-representations are invariant to distortions in section 4.3. Moreover, we see that standard post-hoc calibration techniques such as Platt-scaling fail to preserve invariance to distortions. Proof-of-concept experiments to improve OOD detection and identifying distribution shifts within the data are discussed in the appendix (supplementary material).

52

53

54

55

56

57

58

59

60

61

62

Illustrating the Construction of Quantile Representations: Before diving into the details, we illustrate the construction using a simple toy example and considering the problem of OOD detection. Figure 1a shows a simple toy example with 3 classes 0 , 1 , 2 . Class 0 is taken to be out-of-distribution (OOD), while classes 1 , 2 are taken to in-distribution (ID). To get the quantile representation - (step 1) we first construct a simple classifier to differentiate classes 1 , 2 , (step 2) To get a classifier at quantile τ , construct y + i = I [ p i > τ ] 2 , where p i denotes the predicted probability in (step 1). Construct a classifier using the new labels y + i . Figure 1b illustrates the classifiers obtained at different τ . In (step 3) concatenate the outputs (predictions) of all the classifiers at different τ to get the quantilerepresentations. Figures 1c and 1d demonstrate the advantage of having several classifiers as opposed to one. By aggregating (detailed later) the outputs from different classifiers, we are able to identify OOD vs ID samples (using One-Class-SVM [31]).

(a)

(b)

(c)

(d)

1.25

class

1.50 -

GT-class

1.50 -

GT-class

0.75

1.00

0.75

1.00

1.00 -

ID

OOD

1.00 =

OOD

ID

0.50

0.50

0.50 -

0.50 -

0.25

0.25

0.00

0.00

0.00 -

0.00 -

0.25

0.50

1.5

1.0

0.5

0.0

0.5

X

1.0

1.5

2.0

0.25

0.75

0.50

+

-1.0

00D

ID(class2)

D(class1)

0.5

0.0

0.5

X

1.0

1.5

2.0

-0.50 -

-1.00

0S'T

1.00 -

0.50

I

00'0

00Z

-1.00

-0.50 -

1.50

1.00 -

0.50

I

00'0

0S°0

00

09

Remark: Note that the construction in (step 2) does not depend on the procedure followed in (step 1), 63 but only the output probabilities p i . Thus, one can use any procedure in (step 1) without affecting (step 64 2). This property of quantiles is based on the duality between quantiles and probabilities ( section 2). 65 Intuitively, quantile representations capture the distribution of the training data. Thus, given a pre66 trained classifier, quantile representations can be used to analyze the classifier. In particular, as we 67 shall shortly illustrate, one can perform calibration and OOD-detection. 68

2 Simultaneous Binary Quantile Regression (SBQR) 69

In this section, we review some of the theoretical foundations required for constructing quantile 70 representations. For more details please refer to [16, 17, 32]. 71

Let p data ( X,Y ) , denote the distribution from which the data is generated. X denotes the features 72 and Y denotes the targets (class labels). A classification algorithm predicts the latent variable (a.k.a 73 logits ) Z which are used to make predictions on Y . 74

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

2.1 Review - Quantile Regression and Binary Quantile Regression

Observe that, for binary classification, Z denotes a one dimensional distribution. F Z ( z ) = P ( Z ≤ z ) denotes the cumulative distribution of a random variable Z . The function F -1 Z ( τ ) = inf { z : F Z ( z ) ≥ τ } denotes the quantile distribution of the variable Z , where 0 < τ < 1 . The aim of quantile regression is to predict the τ th quantile of the latent variable Z from the data. That is, we aim to estimate F -1 Z ( τ | X = x ) . Minimizing pinball-loss or check-loss [16],

allows us to learn f θ which estimates the τ th quantile of Y . When τ = 0 . 5 , we obtain the loss to be equivalent to mean absolute error (MAE). For the multi-class case we follow the one-vs-rest procedure to learn quantiles for each class.

Simultaneous Quantile Regression (SQR): Observe that the loss in equation 1 is for a single τ . [32] argues that - minimizing the expected loss over all τ ∈ (0 , 1) ,

is better than optimizing for each separately. Using the loss in equation 2 instead of equation 1

enforces the solution to have monotonicity requires

τ

monotonicity property

. If

Q

(

x

, τ

)

denotes the solution to equation 2,

Observe that for a given x i , the function Q ( x i , τ ) can be interpreted as a (continuous) representation of x i as τ varies over (0 , 1) . The function Q ( x , τ ) is referred to as quantile representation . Q ( x , τ ) is sometimes written as Q ( x , τ ; θ ) , where θ indicates the parameters (such as weights in a neural neural network). For brevity, we do not include the parameters θ in this article unless explicitly required.

Remark on Notation: To differentiate between the latent scores (logits) and probabilities - we use Q ( x , τ ) , f θ ( x ) to denote the probabilities and Q ℓ ( x , τ ) , f ℓ,θ ( x ) to denote the latent scores. Since we have the relation Q ( x , τ ) = σ ( Q ℓ ( x , τ )) and f ℓ ( x ) = σ ( f ℓ,θ ( x )) and σ ( . ) is monotonic, these quantities are related by a monotonic transformation.

Why Quantile Regression? Quantile regression techniques are relatively unknown in the machine learning community, but offers a wide range of advantages over the traditional single point regression. Quantiles give information about the shape of the distribution, in particular if the distribution is skewed. They are robust to outliers, can model extreme events, capture uncertainty in predictions. Quantile regression techniques have been used for pediatric medicine, survival and duration time studies, discrimination and income inequality. (See supplememtary material for a more thorough discussion.)

3 Quantile Representations

As discussed earlier, minimizing equation 2 is not recommended due to unaccountable side-effects. Thus, we require a procedure to construct quantile representations without resorting to minimizing equation 2. In this section we present duality property of the quantile representations, which allows us to do this.

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

Algorithm 1 Generating Quantile Representations.

Let D = { ( x i , y i ) } denote the training dataset. Assume that a pre-trained binary classifier f θ ( x ) is given. The aim is to generate the quantile representations with respect to f ( x ) . We refer to this f θ ( x ) as base-classifier.

Assign Q ( x , 0 . 5) = f θ ( x ) , that is take the median classifier to be the given classifier.

Define y + i,τ = I [ f θ ( x i ) > (1 -τ )] . We refer to this as modified labels at quantile τ .

To obtain Q ( x , τ ) , train the classifier using the dataset D + τ = { ( x i , y + i,τ ) } . Repeating this for all τ allows us to construct the quantile representation Q ( x , τ ) .

3.1 Generating Quantile Representations Using Duality between Quantiles and Probabilities 116

Observe that, for binary classification, equation 1 can be written as 117

Thus the following property holds : 118

We refer to the above property as duality between quantiles and probabilities . Let Q ( x , τ ) denotes a solution to equation 2. It follows from above that, for a given x i and τ 0 , if we have Q ( x i , τ 0 ) = p i , then we should also have Q ( x i , 1 -p i ) = 1 -τ 0 . In words, a solution which predicts the τ th quantile can be interpreted as the quantile at which the probability is 1 -τ . This observation leads to the algorithm 1 for generating the quantile representations.

Why does algorithm 1 return quantile representations? Assume for an arbitrary x i , we have Q ( x i , 0 . 5) = p i . Standard interpretation states - at quantile τ = 0 . 5 , the probability of x i in class 1 is p i . However, thanks to duality in equation 5, this can also be interpreted as - At quantile τ = (1 -p i ) , the probability of x i in class 1 is 0.5.

Thanks to monotonocity property in equation 3, we have for all τ < (1 -p i ) , probability of x i in class 1 is < 0 . 5 , and hence belongs to class 0 . And for all τ > (1 -p i ) , probability of x i in class 1 is > 0 . 5 , and hence belongs to class 1 .

This implies that at a given quantile τ ∗ , x i will belong to class 1 if τ ∗ > (1 -p i ) or if p i > (1 -τ ∗ ) or if f θ ( x i ) > (1 -τ ∗ ) . Defining, y + i,τ ∗ = I [ f θ ( x i ) > (1 -τ ∗ )] , we have that the classifier at quantile τ ∗ fits the data D + τ = { ( x i , y + i,τ ∗ ) } and thus can be used to identify Q ( x , τ ∗ ) . This gives us the algorithm 1 to get the quantile representations for an arbitrary classifier f θ ( x ) .

Remark (Sigmoid vs Indicator function): In theory, we approximate ˆ y i = I [ ˆ z i > 0] (i.e Indicator function) with the sigmoid as ˆ y i = σ ( ˆ z i ) . The algorithm 1 gives a solution up to this approximation. In particular we have the following theorem

Theorem 3.1 Let the base classifier f θ ( x ) = σ ( f ℓ,θ ( x )) be obtained using the MAE loss, i.e by minimizing

Then the solution Q ( x , τ ) obtained by algorithm 1 minimizes the cost in equation 2 over the dataset D , i.e

The proof for the above theorem is discussed in the supplementary material. In simple words, the 142 proof follows from the duality and the fact that we are only interested in the labels (for this theorem) 143 obtained by applying the indicator function. 144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

Importance of duality: Algorithm 1 and theorem 3.1 hinges on the duality property. Recall that pinball loss equation 4 penalizes the positive errors and negative errors differently. In the case of binary classification, since f θ ( x ) ∈ (0 , 1) , positive errors occur for class 1 and negative errors occur for class 0 . Hence, the quantile value implicitly controls the probability of class 1 , giving the duality property.

Thus, using quantile value as an input allows us to control the probabilities and hence confidence of our predictions. This is what we have exploited to construct quantile representations without resorting to optimizing equation 2. This ensures that the properties of the pre-trained model are preserved while still being able to compute quantile representations.

Remark: The other alternate to computing quantile representations are the Bayesian approaches [15]. It is known that computing the full predictive distribution -p ( y |D , x ) = ∫ p ( y | w,x ) p ( w |D ) dw is computationally difficult. Quantile representations approximate the inverse of the c.d.f of the predictive distribution for the binary classification.

To summarize, thanks to the duality in equation 5, one can compute the quantile representations for any arbitrary classifier. This allows for detailed analysis of the classifier and the features learned. In the following section we first discuss the implementation of algorithm 1 in practice and provide both qualitative and quantitative analysis for specific models.

4 Experiments and Analysis

4.1 Generating Quantile Representations in practice

Let f θ ( x ) denote a pre-trained classifier. Given a dataset D = { ( x i , y i ) } i , we construct a quantile dataset -{ ( x i , y + i,τ ) } i,τ as described in algorithm 1 with the following modifications:

Instead of computing y + i,τ = I [ f θ ( x ) > (1 -τ )] , we compute the τ th quantile of logits -{ f ℓ,θ ( x i ) } i . Moreover, as multi-class classification problem gives class imbalance under one-vs-rest paradigm, we compute weighted-quantiles , where weights are inversely proportional to the size of the class. Observe that since f ℓ,θ ( x ) is a monotonic function of f θ ( x ) , this does not make a difference in practice. However, this allows us to circumvent precision issues caused due to the sigmoid function.

We only consider a fixed finite number of quantiles. The n τ quantiles are given by { 1 / n τ +1 , 2 / n τ +1 , · · · , n τ / n τ +1 } .

For the sake of valid experimentation, we model Q ( x , τ ) using the same network as f θ ( x ) , except for the first layer. We concatenate the value of τ to the input, resulting in slightly more number of parameters in the first layer. For efficient optimization we start the training with the weights of the pre-trained classifier f θ ( x ) , except for the first layer.

Loss function to train Q ℓ ( x , τ ) : Recall that Q ℓ ( x , τ ) indicates the latent logits. We use BinaryCrossEntropy loss to train Q ℓ ( x , τ ) where the targets are given by the modified labels { y + i,τ } .

Inference using Q ℓ ( x , τ ) : After training, we compute the probabilities as follows

Remark: For multi-class classification, we follow a one-vs-rest approach. Hence the loss in this case would be sum of losses over all individual classes. The probability, in multi-class case, is taken to be arg max k p i,k . Note that the probabilities p i,k do not add up to 1 over all classes.

Remark: Since the aim is to analyze the pre-trained model, we only consider one specific architecture - Resnet34, and two datasets - CIFAR10 and SVHN to illustrate our results. Other related experiments are included in the appendix (supplementary material).

Training Details and Compute: Training quantile representations was done on a DGX server using 4 GPUs. It takes around 10 hours (40 GPU hours in total) to learn the quantile representations for each

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

model. We use stochastic gradient descent with cyclic learning rate for optimization. The base_lr is taken to be 0 . 02 and max_lr is taken to be 1 . 0 , with exponentially decreasing learning rate using γ = 0 . 99994 . The batch_size is taken to be 1024 for resnet34. The number of steps for the cyclic learning is taken to be 2( size _ dataset / 2( batch _ size ) +1) . The size _ dataset describes the size of the training data. Complete code has been provided with the supplementary material.

4.2 Quantile Representations captures the distribution of the input data

Firstly, we analyze the learned quantile representations Q ℓ ( ., . ) . Broadly, the learned function Q ℓ ( ., . ) captures the 1 dimensional caricature of the input distribution, in the direction where the label changes. We illustrate this using a simple toy example (figure 2). Consider a 1-dimensional dataset in a 2d-space. The labels are assigned by splitting the dataset at the mean of the values on x-axis. We then learn a simple 1 hidden layer neural network with 100 hidden neurons. Using this as a base classifier, we then learn the quantile representation function Q ℓ ( ., . ) as described above.

Then, we reconstruct data in the original space as follows - Assign arbitrary labels at each τ satisfying the monotonicity property equation 3. For each set of labels, keeping the learned function Q ℓ ( ., . ) fixed, learn the input which gives these labels. This is illustrated in figure 2. Observe that the shape of the learned inputs (1-d thread like structure) resembles the shape of the input dataset. This shows that the function Q ℓ ( ., . ) captures how the sample distribution changes in the input space.

4.3 Calibration of ML models

For several applications the confidence of the predictions is important. This is measured by considering how well the output probabilities from the model reflect it's predictive uncertainty. This is referred to as Calibration .

0.5

Data (label=0)

Data (label=1)

0.4

Reconstructed

from

0.3

0.2

0.1

0.0

0.1

-0.2

0.3

-0.8

-0.6

-0.4

-0.2

0.0

0.2

0.4

Several methods [28, 37, 19, 1, 22] are used to improve the calibration of the deep learning models. Most of these methods consider a part of the data (apart from train data) to adjust the probability predictions. However, in [26, 23] it has been shown that most of the calibration approaches fail under distortions. In this section we show that calibration using quantile-representations are invariant to distortions.

Let p i,k denote the predicted probability that the sample x i belongs to class k . A perfectly calibrated model (binary class) will satisfy [10] P ( y i = 1 | p i, 1 = p ∗ ) = p ∗ . For multi-class case this is adapted to P ( y i = arg max k ( p i,k ) | max k ( p i,k ) = p ∗ ) = p ∗ . The degree of miscalibration is usually measured using Expected Calibration Error ( ECE )

This is computed by binning the predictions into m bins B 1 , B 2 , · · · , B m and computing ˆ ECE = ∑ m i =1 ( | B i | / n ) | acc ( B i ) -conf ( B i ) | . where acc ( B i ) = (1 / | B i | ) ∑ j ∈ B i I [ y j = arg max k ( p j,k )] denotes the accuracy of the predictions lying in B i , and conf ( B i ) = ∑ j ∈ B i max k ( p j,k ) indicates the average confidence of the predictions lying in B i .

In the ideal scenario, we have that quantile representations predict perfectly calibrated probabilities as illustrated in the following theorem.

Theorem 4.1 Let f θ ( . ) denote the pre-trained model. Assume that the data is generated using the model y = I [ f θ ( x ) + ϵ > 0] , where ϵ denotes the error distribution. Let Q ( x , τ ) denote the quantile representations obtained on this data. The probabilities obtained using equation 8 are perfectly calibrated, that is,

(a) ECE (Resnet34)

Method

0.30

MSP

0.9

QUANT

0.8

0.25

0.7

0.20

ECE

0.6

0.15

0.5

0.4

0.10

Method

0.3

MSP

0.05

QUANT

0.2

1

2

3

4

5

1

2

3

4

5

Severity

Severity

The proof for theorem 4.1 is given in the supplementary material. The main idea is the notion that 238 Q ( x , τ ) captures P ( f θ ( x ) + ϵ < τ ) . 239

240

241

242

243

Observe that, we can use theorem 4.1 to predict the calibration error of any pre-trained model, given the quantile representations. ( Remark: This is another advantage of computing the quantile representations without retraining the original classifier. If the quantile representations are obtained by minimizing equation 2, then we cannot be sure that calibration error would remain the same.)

244

245

246

247

248

249

250

251

252

253

Experimental Setup In this study, we employ the CIFAR10 dataset and the Resnet34 model to investigate the robustness of classifiers. To evaluate the classifiers' robustness, we use the distorted CIFAR10 dataset introduced in [12], which contains 15 types of common corruptions at five severity levels. This dataset is a standard benchmark for testing the robustness of classifiers. We use quantilerepresentations trained on the CIFAR10 training data to assess the generalization performance of the classifiers on the distorted dataset. We compare the performance with Maximum Softmax Probability ( MSP ) as a baseline and evaluate both accuracy and calibration error. We construct the bins { B i } using 5 equally spaced quantiles within the predicted probabilities. The probabilities of each class are predicted using equation 8. ( Remark: These probabilities do not add upto 1 since we consider a one-vs-rest approach.)

254

255

256

257

258

259

260

261

262

263

264

265

Method

QUANT(Uncalibrated)

0.4

QUANT (Calibrated)

0.3

ECE

0.2

0.1

工

0.0

0

1

2

3

4

5

Severity

266

We present the results in Figure 3. As we increase the severity of the distortions, we observe that the accuracy of both the quantile representations and MSP decreases. However, the probabilities obtained from quantile representations are robust to distortions, as their Expected Calibration Error ( ECE ) does not increase with severity in the same way as MSP 's does. This indicates that quantile representations can more accurately predict calibration error and are more resistant to distortions.

Cannot Correct the Calibration Error Figure 3 shows that calibration error from quantile representations is robust to noise. So, an obvious question which follows is - Can we then correct it using validation data, improve the calibration

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

score without compromising invariance to distortions? It turns out that usual methods fail when trying to correct the calibration error of quantile representations. ( Remark: A similar result is also obtained in Proposition 1 of [5]).

To verify this we perform the same experiment as earlier. Further we use Platt Scaling on validation data and accordingly transform the probability estimates for the corrupted datasets. These results are shown in figure 4. Observe that at severity 0 , the calibration error is 0 for the corrected probabilities as expected. However, as distortion increases, the calibration error increases as well.

Discussion: Consider the following - Given a specific calibration error C (say), let the set of all probability assignments which give the calibration error C be P C . If the network f θ ( x ) has to remain invariant to distortions, it should return one of these possible probability distributions P C . Our explanation for the above result is that - The vanilla neural networks do not have this property. The quantile networks, as illustrated, are evidenced to have this property. However, this also implies that calibration error cannot be corrected post-hoc in a sample independent manner.

4.4 OODDetection using Quantile Representations

An assumption made across all machine learning models is that - Train and test datasets share the same distributions. However, test data can contain samples which are out-of-distribution (OOD) whose labels have not been seen during the training process [25]. Such samples should be ignored during inference. Hence OOD detection is a key component of reliable ML systems. Several methods [13, 20, 2] have been proposed for OOD detection. Here we check how quantile representations compare to the baseline method in [13] for OOD detection.

Quantile representations construct different classifiers at different distances from the base-classifier (illustrated in figure 1b). This allows us to differentiate between OOD samples and ID samples. Intuitively, OOD samples are far from the boundary and result in low softmax probabilities. Thus, one way to assign OOD scores to samples is by considering the maximum softmax probabilities MSP as described in [12]. We compare the OOD detection of quantile representations with this approach.

Experimental Setup For this study, we use the CIFAR10[18] and SVHN[24] datasets as indistribution (ID) datasets and the iSUN[34], LSUN[36], and TinyImagenet[21] datasets as out-ofdistribution (OOD) datasets. Two versions of LSUN and TinyImagenet are considered - resized to 32 × 32 and cropped. We evaluate the quantile representations obtained using ResNet34[11] architecture. For evaluation we use (i) AUROC: The area under the receiver operating characteristic curve of a threshold-based detector. A perfect detector corresponds to an AUROC score of 100%. (ii) TNR at 95% TPR: The probability that an OOD sample is correctly identified (classified as negative) when the true positive rate equals 95%. (iii) Detection accuracy: Measures the maximum possible classification accuracy over all possible thresholds.

Methodology and Results To identify OOD samples with quantile representations, we consider the entire representation Q ℓ ( x i , τ ) as input features. In our experiments this would be of the dimension n τ × n _ classes . To assign an OOD-score we use the One-Class SVM approach. The first rows of Table 1 shows the results. Observe that quantile-representations perform marginally better than than the baseline.

The more interesting result, however, is the fact that quantile representations have all the information required to identify OOD samples. To establish this we perform the following experiment - We use a simple linear classifier (logistic regression) to identify if the ID and OOD datasets are linearly separable or not. We measure the training accuracy to quantify linear separability - If the accuracy is close to 100% , then the datasets are considered to be linearly separable. For comparison we perform the same experiment with the pre-trained logits f ℓ,θ ( x ) . These results are shown in the bottom rows of Table 1. Note that while the baseline scores vary with the dataset, the quantile scores are consistently close to 100%. This provides additional evidence to the hypothesis that quantile representations capture all the 'relevant' information about the train distribution.

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

5 Related Work

[16, 27, 29, 3] provides a comprehensive overview of approaches related to quantile regression and identifying the parameters. [4] extends the quantiles to multi-variate case. [32, 33] use quantile regression based approaches for estimating confidence of neural networks based predictions. [1, 8] uses conformal methods to calibrate probabilities, and is closely related to computing quantiles. [5] proposes a similar algorithm to overcome the restriction to pinball loss for regression problems. [7] generates predictive regions using quantile regression techniques.

6 Conclusion, Limitations and Future work

To summarize, in this article we show the duality between quantiles and probabilities in the case of SBQR. Exploiting the duality, we propose an algorithm to compute quantile representations for any given base classifier. We verify that the quantile representations model the training distribution well both qualitatively (by resconstructing the data in the input space) and quantitatively (using OOD detection baseline). We further show that the probabilities from quantile representations are robust to distortions. Interestingly, we found that traditional approaches cannot be used to correct the calibration error. Further experiments to validate the observations made in this article are discussed in the supplementary material.

The main limitation of the approach is the computation required for algorithm 1 for large scale datasets. Note that algorithm 1 creates n τ copies of the same dataset by assigning different labels. For large scale datasets and large scale networks this requires a lot more computation than using a pre-trained classifier. However, we hypothesize that - we only need to retrain the quantile network only on a small sample size instead of the entire dataset We consider this for future work.

Based on strong convexity and Lipschitzness of loss functions, automatic learning rates can be 339 computed for large networks via the inverse of the Lipschitz constant of the loss function being an 340 ideal learning rate [35]. We conjecture that exploiting the loss functions which inherit some convexity 341 and Lipschitz properties from the known, closed form loss representations would achieve higher 342 learning rates for faster convergence to compute quantile representations. We defer this as future 343 work. 344

345

References

Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua 346 Lei. Learn then test: Calibrating predictive algorithms to achieve risk control. arXiv preprint 347 arXiv:2110.01052 , 2021. 348

Koby Bibas, Meir Feder, and Tal Hassner. Single layer predictive normalized maximum 349 likelihood for out-of-distribution detection. In Neural Inform. Process. Syst. , 2021. 350

Probal Chaudhuri. Generalized regression quantiles: Forming a useful toolkit for robust linear 351 regression. L1 Statistical Analysis and Related Methods, Amsterdam: North-Holland , pages 352 169-185, 1992. 353

Probal Chaudhuri. On a geometric notion of quantiles for multivariate data. Journal of the 354 American Statistical Association , 91(434):862-872, 1996. 355

Youngseog Chung, Willie Neiswanger, Ian Char, and Jeff Schneider. Beyond pinball loss: 356 Quantile methods for calibrated uncertainty quantification. In Neural Inform. Process. Syst. , 357 2021. 358

359

360

Max H. Farrell, Tengyuan Liang, and Sanjog Misra. Deep neural networks for estimation and inference. Econometrica , 89(1):181-213, 2021.

361

362

363

364

Shai Feldman, Stephen Bates, and Yaniv Romano. Calibrated multiple-output quantile regression with representation learning. arXiv preprint arXiv:2110.00816 , 2021.

Shai Feldman, Stephen Bates, and Yaniv Romano. Improving conditional coverage via orthogonal quantile regression. In Neural Inform. Process. Syst. , 2021.

Stanislav Fort, Jie Ren, and Balaji Lakshminarayanan. Exploring the limits of out-of-distribution 365 detection. In Neural Inform. Process. Syst. , 2021. 366

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural 367 networks. In Int. Conf. Mach. Learning , 2017. 368

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image 369 recognition. In Proc. Conf. Comput. Vision Pattern Recognition , 2016. 370

Dan Hendrycks and Thomas G. Dietterich. Benchmarking neural network robustness to common 371 corruptions and perturbations. In Int. Conf. on Learning Representations , 2019. 372

Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution 373 examples in neural networks. In Int. Conf. on Learning Representations , 2017. 374

Heinrich Jiang, Been Kim, Melody Y. Guan, and Maya R. Gupta. To trust or not to trust A 375 classifier. In Neural Inform. Process. Syst. , 2018. 376

Laurent Valentin Jospin, Hamid Laga, Farid Boussaïd, Wray L. Buntine, and Mohammed 377 Bennamoun. Hands-on bayesian neural networks - A tutorial for deep learning users. IEEE 378 Comput. Intell. Mag. , 17(2):29-48, 2022. 379

Roger Koenker. Quantile Regression . Econometric Society Monographs. Cambridge University 380 Press, 2005. 381

Gregory Kordas. Smoothed binary regression quantiles. Journal of Applied Econometrics , 382 21(3):387-407, 2006. 383

Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. The cifar-10 dataset. online: http://www. 384 cs. toronto. edu/kriz/cifar. html , 2014. 385

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable 386 predictive uncertainty estimation using deep ensembles. In Neural Inform. Process. Syst. , 2017. 387

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for 388 detecting out-of-distribution samples and adversarial attacks. In Neural Inform. Process. Syst. , 389 2018. 390

Shiyu Liang, Yixuan Li, and R. Srikant. Enhancing the reliability of out-of-distribution image 391 detection in neural networks. In Int. Conf. on Learning Representations , 2018. 392

Jeremiah Z. Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax-Weiss, and Balaji Lakshmi393 narayanan. Simple and principled uncertainty estimation with deterministic deep learning via 394 distance awareness. In Neural Inform. Process. Syst. , 2020. 395

396

397

398

Matthias Minderer, Josip Djolonga, Rob Romijnders, Frances Hubis, Xiaohua Zhai, Neil Houlsby, Dustin Tran, and Mario Lucic. Revisiting the calibration of modern neural networks. In Neural Inform. Process. Syst. , 2021.

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. Reading digits in natural images with unsupervised feature learning. In Neural Inform. Process. Syst. Workshops , 2011.

Anh Mai Nguyen, Jason Yosinski, and Jeff Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proc. Conf. Comput. Vision Pattern Recognition , 2015.

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua V. Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. arXiv preprint arXiv:1906.02530 , 2019.

Emanuel Parzen. Quantile probability and statistical data modeling. Statistical Science , 19(4):652-662, 2004.

J. Platt. Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In Advances in Large Margin Classifiers , 2000.

Stephen Portnoy and Roger Koenker. Adaptive l-estimation for linear models. The Annals of Statistics , 17(1):362-381, 1989.

Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should I trust you?": Explaining the predictions of any classifier. In Proc. ACM SIGKDD Conf. Knowledge Discovery and Data Minining , 2016.

Bernhard Schölkopf, John C. Platt, John Shawe-Taylor, Alexander J. Smola, and Robert C. Williamson. Estimating the support of a high-dimensional distribution. Neural Comput. , 13(7):1443-1471, 2001.

Natasa Tagasovska and David Lopez-Paz. Single-model uncertainties for deep learning. In Neural Inform. Process. Syst. , 2019.

Anuj Tambwekar, Anirudh Maiya, Soma S. Dhavala, and Snehanshu Saha. Estimation and applications of quantiles in deep binary classification. IEEE Trans. Artif. Intell. , 3(2):275-286, 2022.

Pingmei Xu, Krista A Ehinger, Yinda Zhang, Adam Finkelstein, Sanjeev R Kulkarni, and Jianxiong Xiao. Turkergaze: Crowdsourcing saliency with webcam based eye tracking. arXiv preprint arXiv:1504.06755 , 2015.

Rahul Yedida, Snehanshu Saha, and Tejas Prashanth. Lipschitzlr: Using theoretically computed adaptive learning rates for fast convergence. Applied Intelligence , 51:1460-1478, 2021.

Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 , 2015.

Bianca Zadrozny and Charles Elkan. Transforming classifier scores into accurate multiclass probability estimates. In Proc. ACM SIGKDD Conf. Knowledge Discovery and Data Minining , 2002.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a framework which extends an existing pre-trained classifier to a full quantile representation of the target data. This is done by setting the pre-trained classifier as the median classifier and by optimizing over pinball losses over different quantile values. Experimental results show that the proposed framework is competitive in out-of-distribution detection and calibrating models.\n\nSTRENGTHS:\nThe paper proposes an idea which extends a pre-trained classifier to a whole quantile representation of the data. The framework is based on a simple observation which connects probability with classification quantile levels. Overall, the paper is well-written and easy to follow.\n\nWEAKNESSES:\n- The main algorithm is based on the monotonicity property. However, monotonicity is not guaranteed in general. In particular, the statement \"Using eq.2 enforces the solution to have monotonicity property\" is misleading. This is true only when the global minimum is obtained, which is often not the case in neural networks. Even minimizing with eq.2 can lead to a model that is not monotone, and monotonicity is enforced in model design in real practice. See, e.g. [2,4]\n\n- In the experiments, the proposed framework is only compared with a simple baseline model. Therefore the results are not too convincing.\n\n- I do not agree with the statement that in modern machine learning, loss functions are restricted to pinball loss or MAE. Eq.2, also called continuous ranked probability score is widely applied in quantile regression as well as machine learning. \n\n- In order to perform algorithm 1, we need to minimize multiple pinball losses on different quantile levels (and use the data). The overall costs seem to be similar to minimizing eq.2 with the expectation estimated by sampling (similarly to eq.8) or numerical integration. Therefore I am not sure about the statement that \"pinball loss/eq.2 is difficult to optimize\"\n\n- Overall, I am not too convinced that the proposed framework is more preferable than optimizing an estimated eq.2. I recommend the authors to emphasize the importance of preserving the original property of the pre-trained classifier, and to experimentally show that in some aspects, the proposed framework can compete with a quantile regressor obtained by optimizing over an estimated eq.2.\n\n- The related work section on quantile regression tends to only include classical paper, and does not cover enough recent papers on quantile regression with machine learning. For instance, in probabilistic time series forecasting, there is a collection of work which successfully applies quantile regression with machine learning, for example, [1-3]  \n\n[1] A multi-horizon quantile recurrent forecaster by Wen et al. \n[2] Probabilistic forecasting with spline quantile function RNNs by Gasthaus et al. \n[3] Multi-horizon time series forecasting with temporal attention learning by Fan et al.\n[4] Learning quantile functions without quantile crossing for distribution-free time series forecasting by Park et al.\n\nQUESTIONS:\n- The paper is based on the point that we want to preserve the original property of the pre-trained classifier. Is there an important application to this?\n\n- Are there other similar framework which also extends a pre-trained classifier to compare with?\n\nLIMITATIONS:\nAs mentioned in the weaknesses section, the framework is based on the monotonicity of the model. However, the monotonicity is not guaranteed in real practice.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a method to construct quantile representations of binary classification problems. This construction does not require to use the pinball loss, resolving the difficulty of optimizing the pinball loss. The paper verified its method through experiments.\n\nSTRENGTHS:\nThe proposed approach looks novel and uses an interesting property of pinball loss.\n\nWEAKNESSES:\nThe notations in this paper are confusing, making it hard for me to understand the proposed method and the proof of the theoretical result. \n1. The minimizer of (1) f_\\theta is claimed to be a quantile function. I am confused about which random variable this quantile function belongs to, Y or Z? Y is a binary random variable, and Z is a deterministic function of X, according to line 73-74. That means Z should not have a conditional quantile function (since it is a deterministic function of X), and the quantile function of Y can only take values in {0, 0.5, 1} (since Y is binary). \n2. Eq. (2) and Eq. (7) are different. In Eq. (2), there is no indicator around \\psi, whereas in Eq. (2), there is an indicator around \\psi. So it is confusing whether Q should be a function inside the indicator, or a function outside of the indicator. \n3. In Algorithm 1, the 4th bullet point, it is not clear which loss is used to train the classifier. Note that the loss function should be important for Theorem 3.1 to hold: Theorem 3.1 is claiming the minimizers of two empirical risk minimization problems (not population risk minimization) are equivalent, and it cannot be the case that they are equivalent no matter which loss function is used.\n\nQUESTIONS:\nBasically, I don’t see how this paper properly defines the conditional quantile function of a binary classification distribution. This can neither be the conditional quantile function of Z = f_{\\ell, \\theta}(X, \\theta) given X, since f_\\theta is a deterministic function; nor the conditional quantile function of Y given X, since Y is binary.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

comments: SUMMARY:
This paper proposed an effective approach to estimate the uncertainty for the classification model. By decoupling the quantile representation from the loss function, the proposed method works for arbitrary base classifier.

STRENGTHS:
1. this paper discovered the duality between the probabilities and quantile, and proposed a method to decouple the quantile representations from the loss function for pre-trained classification models;
1. the proposed method demonstrated effectiveness on two problems: out-of-distribution (OOD) discovery and model calibration.

WEAKNESSES:
1. major weakness, as mentioned by the authors, is the scalability of the proposed method wrt large scale datasets and models.

QUESTIONS:
N/A.

LIMITATIONS:
no potential negative societal impact of the work is noted to the best knowledge of the reviewer
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-08

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to address a fundamental question: given a pre-trained classifier $f_{\\theta}(x)$ and its corresponding training dataset, how can we compute quantile representations to conduct a detailed analysis of the pre-trained classifier? The authors investigate alternative approaches to this problem beyond the classical method of retraining the model using the pinball loss.\n\nIn their work, the authors adopt a straightforward approach that involves decoupling the loss function from the computation of quantile representations. By taking this approach, they propose a novel solution to efficiently analyze the pre-trained classifier without losing its important properties.\n\nSTRENGTHS:\nThe primary contribution of the paper is centered around Section 3.1. Within this section, the authors focus on the binary classification problem and highlight the duality that exists between quantiles and probabilities in this specific scenario. They recognize this duality and propose an algorithm that harnesses the power of a pre-trained binary classifier to derive the desired quantile representation\n\nWEAKNESSES:\nThe contribution of this paper is limited for the following reasons:\n\n1. **Specific to binary classification**: The proposed algorithm for generating quantile representations is based on the duality between quantiles and probabilities, which is evident in the context of binary classification. However, this duality may not hold true for general classification problems. As a result, the applicability of the proposed algorithm is restricted to binary classification scenarios, limiting its broader impact.\n\n2. **Unavoidable quantile crossing**: Since the optimization of quantiles for different $\\tau$ values is performed independently, it is possible to observe quantile crossing in the final solution. This can introduce challenges in interpreting the results and may affect the overall performance of the proposed algorithm. Additionally, there seems to be a discrepancy between Equation (7) and the proposed algorithm. Equation (7) suggests obtaining all quantiles simultaneously, while the algorithm indicates that the quantiles are obtained separately. This discrepancy raises confusion and needs further clarification.\n\n3. **Extensive experiments need** :Another limitation of the paper is the lack of extensive experiments. In Section 4.3, the authors mention several methods mentioned in the references that are used to enhance the calibration of deep learning models. However, these methods are not included as baselines in the subsequent experiments. As a result, the proposed method is only compared with simple MSP, which may not provide sufficient evidence to support its effectiveness and superiority.\n\nQUESTIONS:\nAbout the toy example\n\nThe purpose of the toy example in the paper is to illustrate the construction of quantile representations, as depicted in Figure 1(b). However, the specific details regarding the generation of results in Figures 1(c) and 1(d) require further clarification. Since the classifier is only trained for class 1 and class 2, it is important to understand how the binary classifier is utilized to differentiate between inliers and outliers.\n\nTo gain a clearer understanding, more information is needed about the methodology employed in the paper. Specifically, it would be helpful to know the selected percentage $\\tau$ that is used in this context. This information will provide insights into how the classifier is used to distinguish between inliers and outliers based on the quantile representations.\n\nLIMITATIONS:\n.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem adapting quantile regression (QR) to  deep neural networks for classification tasks. QR takes into account the distribution of the dependent variable and fits a model that gives low error for all quantiles. Logistic regression is a special case of QR where only the tau=0.5 (median) quantile is considered. In previous work, application of QR is usually coupled to the loss function or the model used. In this paper, they decouple these things, so they can theoretically apply QR based learning to any loss function and any network. They establish a duality between quantiles and estimated probabilities in binary classification. Based on this duality, they train separate classifiers for different taus based on the predictions of the median (tau=0.5) classifier. Authors experimentally validate their method in two tasks: out of distribution detection and showing robustness to distortions.\n\nSTRENGTHS:\n+ Tackles an important problem: although quantile regression (QR) has a solid history in econometrics and statistics, it is relatively new in the machine learning community. Its application to modern deep neural networks is potentially useful and interesting.\n\nWEAKNESSES:\n- Limited evaluation. For out of distribution (OOD) detection, they compare their results only with OneClassSVM, a method from 2017. There are stronger and more recent methods in the literature. For example, in the ICLR2022 paper titled \"Open-Set Recognition: A Good Closed-Set Classifier is All You Need\", they compare many SOTA methods for this task and arrive to the conclusion that \"maximum logit score (MLS) thresholding\" is a strong method. The authors should have at least compared their results with MLS. Of course, there are many more relevant methods, which are all cited in that ICLR2022 paper. \n\n- Language and presentation issues prevent an easy reading of this paper. A non-exhaustive list follows. \n- L26: \"However, these techniques aren’t widely used in modern deep learning based systems since [5]\" -> what  does this mean? Is \"since\" the wrong word? \n- L28: \"might not compatible with\" \n- L55: \"taken to in-distribution\" \n- Fig 1 is not easily understandable. E.g., an intuitive explanation as to why we get those classifier in 1 (b) for different taus? which classifier correspond to which tau? Transition from 1(b) to 1(c) not clear. \n- L76: what is l in the subscript of f? \n- In the last step of Algorithm 1, you wrote \"train the classifier\" but I think you mean \"train a separate classifier\", right? \n- I found Fig2 to be completely not understandable. I cannot even point to any questions.\n\nQUESTIONS:\nL60: \"Figures 1c and 1d demonstrate the advantage of having several classifiers as opposed to one.\" -> Can't other models (e.g. deep ensembles and DUQ, to cite on top my of hat) achieve the same? \n\nRegarding Algorithm 1, what is the point of training a new classifier for a tau value? Can't you obtain it directly from $I[f_\\theta (x_i)>1-\\tau]$?\n\nLIMITATIONS:\nWhile listing the areas where QR is applied it would be useful to provide citations (L108).", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-26"}