Variational Imbalanced Regression
Authors: Ziyan Wang
Hao Wang
Venue: Submitted to ICLR 2023 (2023)
Keywords: probabilistic methods
variational inference
imbalanced regression
uncertainty estimation

Abstract: Existing regression models tend to fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. In this paper, we propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR), which not only performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct. Different from typical variational autoencoders assuming I.I.D. representation (a data point's representation is not directly affected by other data points), our VIR borrows data with similar regression labels to compute the latent representation's variational distribution; furthermore, different from deterministic regression models producing point estimates, VIR predicts the entire normal-inverse-gamma distributions and modulates the associated conjugate distributions to impose probabilistic reweighting on the imbalanced data, thereby providing better uncertainty estimation. Experiments in several real-world datasets show that our VIR can outperform state-of-the-art imbalanced regression models in terms of both accuracy and uncertainty estimation. 

Under review as a conference paper at ICLR 2023
VARIATIONAL IMBALANCED REGRESSION
Anonymous authors
Paper under double-blind review
ABSTRACT
Existing regression models tend fall short in both accuracy and uncertainty estimation when the label distribution is imbalanced. In this paper, we propose a probabilistic deep learning model, dubbed variational imbalanced regression (VIR), which not only performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct. Different from typical variational autoencoders assuming I.I.D. representations (a data point's s representation is not directly affected by other data points), our VIR borrows data with similar regression labels to compute the latent representation's 's variational distribution; furthermore, different from deterministic regression models producing point estimates, VIR pre- dicts the entire normal-inverse- inverse-gamma distributions and modulates the associated conjugate distributions to impose probabilistic reweighting on the imbalanced data, thereby providing better uncertainty estimation. Experiments in several real-world datasets show that our VIR can outperform state-of-the-ard imbalanced regression models in terms of both accuracy and uncertainty estimation.
INTRODUCTION
Deep regression models are currently the state of the art in making predictions in a continuous label space and have wide range of successful applications in computer vision (Yin et al., [021). natural language processing (Jiang et al. 2020), etc. However, these models fail however when the label distribution in training data is imbalanced. For example, in visual age estimation (Moschoglou et al.) 2017), where model infers the age of a person given her visual appearance, models are typically trained on imbalanced datasets with overwhelmingly more images of younger adults, leading to poor regression accuracy for images of children or elderly people (Yang et al. (02T). Such unreliability in imbalanced regression settings motivates the need for both improving performance for the minority in the presence of imbalanced data and, more importantly, providing reasonable uncertainty estimation to inform practitioners on how reliable the predictions are (especially for the minority where accuracy is lower).
Existing methods for deep imbalanced regression (DIR) only focus on improving the accuracy of deep regression models by smoothing the label distribution and reweighting data with different labels (Yang et al. 202I). On the other hand, methods that provide uncertainty estimation for deep regression models operates under the balance-data assumption and therefore do not work well in the imbalanced setting (Amini et al. 2020 Mi et al. 2022 Charpentier et al.) 2022].
To simultaneously cover these two desiderata, we propose a probabilistic deep imbalanced regres- sion model, dubbed variational imbalanced regression (VIR). Different from typical variational autoencoders assuming I.I.D. representations (a data point's representation is not directly affected by other data points), our VIR assumes Neighboring and Identically Distributed (N.I.D.) and borrows data with similar regression labels to compute the latent representation" 's variational distribution. Specifically, VIR first encodes a data point into a probabilistic representation and then mix it with neighboring representations (i.e., representations from data with similar regression labels) to produce its final probabilistic representation; VIR is therefore particularly useful for minority data as it can borrow probabilistic representations from data with similar labels (and naturally weigh them using our probabilistic model) to counteract data sparsity. Furthermore, different from deterministic regres- sion models producing point estimates, VIR predicts the entire normal-inverss -nverse-gamma distributions and modulates the associated conjugate distributions by the importance weight computed from the smoothed label distribution to impose probabilistic reweighting on the imbalanced data. This allows the negative log likelihood naturally put more focus on the minority data, thereby balancing the

---

Under review as a conference paper at ICLR 2023
accuracy for data with different regression labels. Our VIR framework is compatible with any deep regression models and can be trained end to end.
We summarize contributions below:
1. While previous work has studied imbalanced regression and uncertainty estimation sepa- rately, none of them has considered uncertainty estimation in the imbalanced setting. We identify the problem of probabilistic deep imbalanced regression as well as two desiderata, balanced accuracy and uncertainty estimation, for the problem.
2. We propose VIR to simultancously cover these two desiderata and achieve statc-of-the. performance compared to existing methods.
3. As a byproduct, we also provide strong baselines for benchmarking high-quality uncertainty estimation and promising prediction performance on imbalanced datasets.
2 RELATED WORK
Variational Autoencoder: Variational autoencoder
(VAE) (Kingma & Welling 2014) an unsupervised leamn ing model that aims to infer probabilistic representations from data. However, as shown Figure VAE typically assumes LLD. representations, where data point's s rep- resentation is not directly affected by other data points. In contrast, our VIR borrows data with similar regression labels to compute the latent representation' 's variational distribution.
Φ
z
z
Φ
x
x
Figure 1: Comparison on inference net- works between typical VAE (Kingma & Welling 2014) and our VIR. In VAE (left), a data point's latent representation (i.e. z) is affected only by itself, while in VIR (right), neighbors participate to modulate the final representation.
Imbalanced Regression. Imbalanced regression is under- explored in the machine learning community. Most exist- ing methods for imbalanced regression are direct exten- sions of the SMOTE algorithm (Chawla et al.) 2002). commonly used algorithm for imbalanced classification, where data from the minority classes is over-sampled. These algorithms usually synthesize augmented data for the minority regression labels by either interpolating both inputs and labels (Torgo et al.) 2013) or adding Gaussian noise (Branco al., 2017 2018).
Such algorithms fail to the distance in continuous label space and fall short in handling high- dimensional data (e.g., images and text). Recently, DIR (Yang et al. (02T) addresses these issues by applying kernel density estimation to smooth and reweight data on the continuous label distribution, achieving otate-of-thhearr performance. However, DIR only focuses on improving the accuracy, especially for the data with minority labels, and therefore does not provide uncertainty estimation, which is crucial to assess the predictions' reliability. Ren et al. (2022) focuses on re-balancing the mean squared error (MSE) loss for imbalanced regression, and Gong et al. (202)) introduces ranking similarity for improving deep imbalanced regression. In contrast, our VIR provides a principled probabilistic approach to simultaneously achieve these two desiderata, not only improving upon DIR in terms of performance but also producing reasonable uncertainty estimation as a much-needed byproduct assess model reliability. There also related work on imbalanced classification (Deng et al. 2021), which is related to our work but focusing on classification rather than regression.
Uncertainty Estimation in Regression. There has been renewed interest in uncertainty estimation in the context of deep regression models (Kendall & Gal (0177) Kuleshov et al., (2018) Song et al.
2019 Zelikman et al.] 2020 Amini et al.) 2020 Mi et al. 2002] van Amersfoort al. 202 I
Liu et al. [2200] Gal & Ghabraman] 2016 Stadler et al.] 2021] Snock et al.) 2019 Heiss et al.
2022). Most existing methods either directly predict the variance the output distribution as the
estimated uncertainty (Kendall & Gal 2017, /Zhng et al. 2019 Amini et al.) 2020) or rely on
post-hoc confidence interval calibration (Kuleshov et al. 2018 Song et al.) 2019 Zelikman et al.
2020). Meanwhile, Posterior Networks methods Charpentier et al. (2020) 2022 ; Stadler et al. (2021) consider conjugate distribution, pseudo-count interpretations, posterior updates, and variational losses for fast and high-quality uncertainty estimation. Closest to work is Deep Evidential Regression (DER) (Amini et al. 2020), which attempts to estimate both aleatoric and epistemic
uncertainty (Kendall & Gal 2017 Hüllermeier & Waegeman, 2019) regression tasks by training
2

---

Under review as a conference paper at ICLR 2023
the neural networks to directly infer the parameters of the evidential distribution, thereby producing uncertainty measures. While Posterior Networks Charpentier et al. (2020) 2022) are designed for general classification//egresstt consider imbalance in regression tasks, which is the focus this paper. DER (Amini et al. 2020) is designed for the data-rich regime and therefore fails to reasonably estimate the uncertainty if the data is imbalanced; for data with minority labels, DER (Amini et al. (020) tends produce unstable distribution parameters, leading poor uncertainty estimation (as shown in Sec. In contrast, our proposed VIR explicitly handles data imbalance in the continuous label space to avoid such instability; VIR does so by modulating both the representations and the output conjugate distribution parameters according to the imbalanced label distribution, allowing training/inference to proceed as the data balance and leading to better performance as well as uncertainty estimation (as shown in Sec. 4..
tasks and achieve promising performance, they do not explicitly
3 METHOD
In this section we introduce the problem setting, provide an overview of our VIR, and then describe details on each of VTR's key components.
3.1 PROBLEM SETTINGS
Assuming imbalanced dataset in continuous space {x, M1 where N is the total num- ber of data points, x; E Rd is the input, and yi E y R is the corresponding label from a continuous label space y. In practice, y is partitioned into equal-interval bins ,,,,)),))).. with slight notation overload. To directly compare with baselines, we use the same grouping index for target value b E B as in (Yang et al. 2021). We denote representations as 24, and use (z.", 2.)) (z)x;; () to denote the probabilistic representations for input x; generated by a probabilistic encoder parameter- ized by 6. Similarly we (Üi, 8.) to denote the mean and variance of the predictive distribution generated by a probabilistic predictor po (y: (z). Furthermore, we denote z as the mean of representation z; in each bins (i.e., letting No i==
[y(0), ( (y(+), y (B(B-1), y(B)),
Z
θ
2
Σ
in bin with Ng data points).
N
3.2 METHOD OVERVIEW
Figure 2: Overview of our VIR method.
Left: The inference model infers the la- tent representations given input x's in the neighborhood. Right: The gener- ative model reconstructs the input and predicts the label distribution (including the associated uncertainty) given the la- tent representation.
In order to achieve both desiderata in probabilistic deep imbalanced regression (i.e., performance improvement and uncertainty estimation), our proposed variational im- balanced regression (VIR) operates on both the encoder (z, (x; 1 ) and the predictor (()))lz)).
Typical VAE (Kingma & Welling 2014) dower-bounde input x,'s 's marginal likelihood; in contrast, VIR lower-bounds the marginal likelihood of input x; and labels y;:
log po (xi, yi) = Dxce (gs (z. (x: (po w:)) L(O, di Xi, v.).
Note that our variational distribution (z. [x,] 1 (1) does not conditions on labels ye, since the task to predict ys and (2) conditions on (neighboring) inputs (x,), rather than just x,, The second term C(O, oi xt, y:) is VIR's evidence lower bound (ELBO), which is defined as:
[(,, Φ: Xs, v.) B.g log pa px(((2))) E,, log pa (v.l2)) (xs(gg(2))))) (x. (l)))lpp(
(1)
where the Pe (z,) is the standard Gaussian prior OO I). following typical VAE (Kigmma & Welling 2014), and the expectation is taken over (z, ) which infers z, by borrowing data with sumilar regression labels to produce the balanced probabilistic representations, which is beneficial especially for the minority (see Sec. 3.3 for details).
3

---

Under review as a conference paper at ICLR 2023
Different from typical regression models which produce only point estimates for ys, our VIR's predictor, po z,), directly produces the parameters of the entire NIG distribution for ys and further imposes probabilistic reweighting the imbalanced data, thereby producing balanced predictive distributions (more details in Sec. 3.4).
3.3 CONSTRUCTING ((z)) (x.)
To cover both desiderata, one needs to (1) produce balanced representations to improve performance for the data with minority labels and (2) produce probabilistic representations to naturally obtain reasonable uncertainty estimation for each model prediction. To learn such balanced probabilistic representations, we construct the encoder of our VIR (i.e., Io (2,)(x;)) )) by (1) first encoding a data point into probabilistic representation, (2) computing probabilistic statisties from neighboring representations (i.e., representations from data with similar regression labels), and (3) producing the final representations via probabilistic whitening and recoloring using the obtained statistics.
Probabilistic Representations. We first encode each data point into probabilistic representation. Note that this is in contrast to existing work (Yang et al. (202]) that uses deterministic representations. We assume that each encoding z; Gaussian distribution with parameters {z z,),, which are generated from the last layer in the deep neural network.
From LLD. to Neighboring and Identically Distributed (N.LD.). Typical VAE (Kingma & Welling 2014) is an unsupervised learning model that aims to learn a variational representation from latent space reconstruct the original inputs under the I.I.D, assumption: that is. in VAE, the latent value (i.e., zı)) is generated from its own input x;. This I.I.D. assumption works well for data with majority labels, but significantly harms performance for data with minority labels. To address this problem. we replace the I.I.D. assumption with the N.LD, assumption; specifically. VIR's 's variational latent representations still follow Gaussian distributions (i.e., (2x"", 2²), but these distributions will be first calibrated using data with neighboring labels. For a data point (Xi, Mi) where yi is in the b'th bin, i.e.,
y: E (b-1), ()), we compute ((..) {x ) NN2;;; 2 xf as
Mean and Covariance of Initial 2, 2 2²² I(x;):
(2)
Statistics of Bin b's Statistics: wh: 1b: Σi, MN = A((x2, I)).
(3)
w.5. St E S(tuf mh Ex 10 Smoothed Statistics of Bin 6's Statistics: 2ig 140
(4)
E Mean and Covariance of Final z,: 7 z Fizt z5 1o pb E/ 24
M4 2b
where the details of functions I(.). S(.), and F()) are described below.
Function I(-): From Deterministic to Probabilistic Statistics. Different from deterministic statistics in Yang et al. 2021), our VIR's encoder uses probabilistic statistics (i.e., statistics of statistics). Specifically, VIR treats z; as distribution with the mean and covariance (z 2 = I(x,) rather than a deterministic vector. As result, all the deterministic statistics, Mbg Σp, Hb, and Σg are replaced by distributions with the means and covariances, (H4, H,),, (%%. D (7h H4, ), and (E"", "")) respectively (more details in the following three paragraphs A(-), S(-), and F(-)).
-E
Function A(-): Statistics of the current Bin b's Statistics. As part of our probabilistic overall statistics, the probabilistic overall mean becomes distribution with the mean (letting H4 Z) and covariance (assuming diagonal covariance):
E No Σ :=1
Σn z =
m =
NS
Similarly, our probabilistic overall covariance becomes matrix-variate distribution (Gupta & Nagar 2018) with the mean:
Σ N, (z. 2)
2 (2,)) - (uET) + ((.
since E[z] - μ and V = μ. Note that the covariance of Σg, i.e., Σg. involves computing the fourth-order moments, which computationally prohibitive. Therefore in practice, directly set D to zero for simplicity; empirically we observe that such simplified treatment already achieves promising performance improvement upon the state of the art.
4

---

Under review as a conference paper at ICLR 2023
Function S()): Neighboring Data and Smoothed Statistics. Next, can borrow data with neighboring labels (from neighboring label bins) to compute the smoothed statistics of the current bin b by applying symmetric kernel k(.,-) (e.g., Gaussian, Laplacian, and Triangular kernels). Specifically, the probabilistic smoothed mean and covariance are (assuming diagonal covariance):
F EvEB k(yb, yo pay- 1
Ex
b''Es k26,,, yo Jpy ===== E. kyb. you )Ey.
Function F()): Probabilistic Whitening and Recoloring. We develop a probabilistic version of the whitening and re-coloring procedure (Sun et al. 2016) used in (Yang et al. (02II). Specifically, produce the final probabilisticc representation {2] 2, for each data point as:
Σ E
z = (z (1))
= (z.
+
(5)
Inspired by Yang al. 2021 , keep updating the probabilistic overall statistics, {1 m.: Ed}, and the probabilistic smoothed statistics, {ar p cross different epochs. The probabilistic repre- sentation (z"" } are then re- pe-prammeerrzed (Kingma & Welling 2014) into the final representation 2, and passed into the final layer (discussed in Sec. (344) generate the prediction and uncertainty
estimation. Note that the computation of statistics from multiple x's 's is only needed during training. During testing, VIR directly uses these statisties and therefore does not need to re-compute them.
3.4 CONSTRUCTING p(u: (z;)
Our VIR's predictor p(y 2.)) A Wy 8:) predicts both the mean and variance for yi by first predicting the NIG distribution and then marginalizing out the latent variables. It is motivated by the following observations on label distribution smoothing (LDS) in (Yang et al. (021] and deep
evidental regression (DER) in (Amini et al. 2020), as well as intuitions on effective counts in conjugate distributions.
LDS's Limitations in Our Probabilistic Imbalanced Regression Setting. The motivation of LDS (Yang et al., (22T)) is that the empirical label distribution can not reflect the real label distribution in imbalanced dataset with continuous label space; consequently, reweighting methods for imbalanced regression fail due to these inaccurate label densities. By applying a smoothing kernel on the empirical label distribution, LDS tries to recover the effective label distribution, with which reweighting methods can obtain 'better' weights to improve imbalanced regression. However, in our probabilistic imbalanced regression, one needs consider both (1) the performance for the data with minority labels and (2) uncertainty estimation for each model. However, LDS only focuses on improving the accuracy, especially for the data with minority labels, and therefore does not provide uncertainty estimation, which is crucial to assess the predictions' reliability.
DER's limitations in Our Probabilistic Imbalanced Regression Setting. In DER (Amini et al. 2020), the predicted labels with their correspond uncertainties are produced by the representation of the posterior parameters in Normal Inverse Gamma (NIG) distribution N NIG(y, v, a, 3), while the model is trained via minimizing the negative log- likelihood (NLL) of a Student-t distribution:
C.E
log( }) + (a 2)) log((u: 7)2v + 52) a log(S2) + log( r(a)
[(a+
where Ω = 2,311 + v). It is therefore nontrivial to properly incorporate a reweighting mechanism into the NLL. One straightforward approach is to directly reweight CPER for different data points (Ii,, ):). However, this contradicts the formulation of NIG and often leads to poor performance, as we verify in Sec. 4
Intuition of Pseudo-Counts for VIR. To properly incorporate different reweighting methods, our VIR relies on the intuition of pseudo-counts (pseudo-observations conjugate distributions (Bishop) 2006). Assuming Gaussian likelihood, the conjugate distributions would be an NIG distribu- tion Bishop 2006), i.e., (H,, Σ) NIG(Y, a, 3), which means:
1 ~~ N(,,
r- (a, B),
where (a, 9) is an inverse gamma distribution. With a NIG prior distribution NIG Vo. Cn, Bo), the posterior distribution of the NIG after observing n real data points are: I'm
tn
10 n,, an = a0 + 7
Bn = Bo + (7080) + 4,
(7)
5

---

Under review as a conference paper at ICLR 2023
where V x and @ [ -(gum) Here Vy and Ogg be interpreted as virtual observations, Le., pseudo-counts or bseudo-observation that contribute to the posterior distribution. Overall, the mean of posterior distribution above can be interpreted an estimation from (20g + n) observations, with 200 virtual observations and ni real observations. Similarly, the variance can be interpreted an estimation from (v + n) observations. This intuition is crucial in developing the predictor of our VIR.
From Pseudo-Counts to Balanced Predictive Distributions. Based on the intuition above, we construct our predictor (i.e.. p(ys 2:)) by (1) generating the parameters in the posterior distribution of NIG, (2) computing re-weighted parameters by imposing the importance weights obtained from LDS, and (3) producing the final prediction with corresponding uncertainty estimation.
we feed the final representation {z, } generated from the Sec. 3.3 Eqn. into linear layer to output the intermediate parameters n,, V,, , 2, for data point (x,, y:
Based on Eqn. 7
n4 Vi, 4; ((z,),
(( W (Z;
We then apply the importance weights E8'e EB k(b,, yb calculated from the smoothed label distribution to the pseudo-count n; to produce the re-weighted parameters of posterior distribution of NIG. Along with the pre-defined prior parameters (70: MD, on, BB), we are able to compute the parameters of posterior distribution NIG()) 1/2 Ctsy 3,) for (x;: Mi):
7.- 7000+ ( k(yn (60)
2
Σ M(bbBw))
Vs = uo +
Y1::
EEEB
af = og
k(yb yb
3 3o + 1 (70m0) + D;-
6' E'EB
Based on the NIG posterior distribution, we can then compute final prediction and uncertainty estimation as
T
We use objective function similar to Eqn. 6 but with different definitions of (7, 3), to optimize our VIR model:
c. = E (x. (xx Fار ())()))+ (a + log((Un v/ + af log(w() log( [(a
)]+
(8)
where w - (1 + vT). Note that C? is part of the ELBO in Eqn. [ Similar to (Amini al. 2020), we use additional regularization term to achieve better accuracy
2 (v + 2a) 1y; 0
CP and L together constitute the objective function for learning the predictor pzy:
3.5 FINAL OBJECTIVE FUNCTION
Putting together Sec. 3.3 and Sec.
3.4 final objective function (to minimize) for VIR is:
LVRR_
C) C.TRR
C.TRR
AC L(0, x;, w:) AC CP
CKe.
where L(O, 6; X, 1/:) additional regularizer and the ELBO, and thus lead to better result both on accuracy and uncertainty estimation.
L.
CP
CKC is the ELBO in Eqn. À adjusts the importance of the
3.6 DISCUSSION ON L.D.D AND N.I.D. ASSUMPTIONS
Generalization Error, Bias, and Variance. We could analyze the generalization error of our VIR by bounding the generalization with the sum of three terms: (a) the bias of our estimator, (2) the variance of our estimator, (3) model complexity. Essentially VIR uses the N.L.D, assumption increases our estimator's bias, but significantly reduces its variance in the imbalanced setting. Since the model complexity is kept the same (using the same backbone neural network) as the baselines, N.I.D. will lead to lower generalization error (see more discussion in Sec. of the Appendix).
Note that in DER, the total evidence + has value 2v + a, but the best of our knowledge, it would be reasonable
20 the total evidence for an NIG distribution (Hishop) 2006).

---

Under review as a conference paper at ICLR 2023
4 RESULTS
Datasets. In this work, we evaluate our methods in terms of prediction accuracy and uncertainty
estimation two imbalanced datasets AgeDB (Moschoglou et al., 2017 MMDBWIKI (Rothe
et al. 2018). We follow the preprocessing procedures in DIR YYngg et al. 2021). I) Details for label
density distributions and levels of imbalance are discussed in DIR (Yang et al. 2021).
AgeDB-DIR: We use AgeDB-DIR constructed in DIR (Yang et al. 2021), ). which contains 12.2K images for training and 2. 2 images for validation and testing. The maximum age in this dataset is 101 and the minimum age is O, and the number of images per bin varies between and 353.
WODB-WWKI-DIR: We use IMDB-WIKS WWII-DIR constructed in DIR (Yang al. 2021 which contains 191.5K training images and 1 11.0K validation and testing images. The maximum age is 186 and minimum age is O; the maximum bin density is 7149, and minimum bin density is 1.
STS-B-DIE We use STS-B-DIL constructed in DIR Yang et al. (02I), which contains 5.2K pairs of training sentences and 1.0K pairs for validation and testing. This dataset collection of
sentence pairs generated from news headlines, video captions, etc. Each pair is annotated by multiple annotators with a similarity score between and 5.
Baselines. We use ResNet-50 (He et al. 2016 as our backbone network, and we describe the baselines below.
Vanilla: We use the term VANILLA to denote plain model without adding any approaches.
Synthetic-
Methods: Various existing imbalanced regression methods are also included as baselines; these include SMOTER (Torgo et al. (013) and SMOGN (Branco et al. (017). Further- more, following DIR (Yang et al. 2021), in BMDB-WIKI-DE we also include another two methods: MIXUP (Zhang et al. 2018) and M-MIXUP (Verma et al. 2019).
Cost-Sensitive Reweighting: As shown in DIR (Yang et al. 2021). the square-root weighting variant
(SQINV) baseline (i.e. ( (Ev k(yb, yb - always outperforms Vanilla. Therefore, for simplicity and fair comparison, all our experiments (for both baselines and VIR) use SQINV weighting. To use
SQINV in VIR, one simply needs to use the symmetric kernel k(-.-) described Sec. 3.3 To use
SQINV in DER, we replace the final layer in DIR (Yang et al. 2021 with the DER layer (Amini
al. 2020 to produce the predictive distributions.
Evaluation Metrics Accuracy. We follow the evaluation metrics in Yang et al. 2021) to evaluate the accuracy of our proposed methods: these include Mean Absolute Error (MAE), Mean Squared Error (MSE), and Geometric Mean (GM). The formulas for these metrics are as follows:
MAE- Σ
N VAl.
MSE
A/R Σ
(34 9.)
GM
D1 01
Evaluation Metrics Uncertainty Estimation. We use typical evaluation metrics for uncertainty estimation in regression problems to evaluate our produced uncertainty estimation; these include Negative Log Likelihood (NLL), Area Under Sparsification Error (AUSE). Eqn. 8 shows the formula for NLL., and more details regarding to AUSE can be found in (llg et al. 2018).
Evaluation Process. Following (Liu et al. 2019 Yang al. 2021 , for data sample x; with its label which falls into the target bins 6,, we divide the label space into three disjoint subsets: many-shot region (b, E B yı E b; & 1yal 1003. medium-shot region (b. B y/s b; & 20 S [ysl S 100). and few-shot region {b; E B y: E b, & 1y:| < 20), where |- denotes the cardinality of the set. We report results on the overall test set and these subsets with the accuracy metries discussed above.
Implementation Details. We use ResNer-50 He et al., 2016) for all experiments AgeDB-DIR and IMDB-WIKI-DE WIKI-DIR. We use the Adam optimizer [Kingma & Bal (013) to train all models for 100 epochs, with same learning rate and decay by 0.1 and the 60-th and 90-th epoch, respectively. In order to determine the optimal batch size for training, we try different batch sizes and achieve the same
Among the five datasets proposed in (Yang et al. 2021). ) only four of them are publicly available. In this paper the largest (MDB-WIKI) and the smallest (AgeDB) among the four evaluate our method.
7

---

Under review as a conference paper at ICLR 2023
Table I: Evaluation results of accuracy on AgeDB-DIR.
Metis
Med
Few
Med
DEEF
79.30
249.18
7.33
13.90
6.50
11.35
01:35
344.52
9.16
8.65
17:38
4.65
5.69
SMUGN
2017
137.29
101 30
133.00 56
9.26
7.64
0111
12.00
5.36
4.90
6.15
8.44
5QtN
1.92
8.90
11.45
4.91
5.73
DER Amnra 11 13100
106.1
91:32
122 85
209. 76
8.11
7.36
9183
12.09
5.31
4.65
0.88
10:53
FDS
Tats 01
100.78
$0.99
124.96
216.97
9.12
7.52
8.63
12.25
5.13
4.80
5.97
8.85
18B Varg Segre-il
10
100.22
K3.62
135.25 73
234.64
7.57
6.95
8.86
10.89
4.85
4.90
5.00
7.45
LDS T7DS Yarl Yalg a. 1001
102.16
56.99
138.04
199,
7.52
7.19
9108
11.34
5,01
4.50
6.10
7.00
FDS RANK31A
63.51
71.99
99.14
149.05
7.02
6.40
7.84
9.68
4.53
4,13
5.37
6.89
1 FDS H.AMETTTTT on 1133
64.96
74.27
91.44
163.92
7.0%
7.68
9.90
4.45
4.07
5.21
6.35
LDS
2020
VIR
OCRS VS. VANILLA
+14.39
+0.71
+34.2
+110.56
+9.65
+0.03
+1.72
+4.46
+9.60
40.26
+1.66
+5.23
OURS va. SQINV
+17.07
+14.98
+301.99
+50.00
+0.78
+0.75
+1.30
+1.90
+0.45
+0.54
+0.03
+1.92
OCRS vs, DER
139.93
133.63
125.90
164.00
+0.97
10.69
+1.33
13.37
10.73
+0.35
+1.39
14.21
OURS Vs. LDS ISOTA DIR)
+15.2
+9.30
+31.49
+53.42
+0.68
+0.52
+1.38
+1.72
+0.43
+0.29
+1.01
+6.71
conclusion as the DIR paper, i.e., the optimal batch size is 256 when other hyperparameters are fixed. Therefore, we stick to the batch size of 256 through out the experiments the paper. Meanwhile, we use the same hyperparameters as in DIR (Yang et al. 2021).
We use PyTorch to implement our method. For fair comparison, we implemented a PyTorch version for the official TensorFlow implementation of DER(Amini et al. (2200) To make sure we can obtain the reasonable uncertainty estimations, we restrict the range for a to [1.5, oo) instead of [1.0, oo) in DER. Besides, in the activation function SoftPlus, we set the hyperparameter beta to 0.1. As discussed in Sec. S.c.3.4 we implement a layer which produces the parameters n, V, St We assign as the minimum number for and use the same hyperparameter settings for activation function for DER layer.
To scarch for combination hyperparameters of prior distribution {20, vo. Co. 30 for NIG, combine grid search method and random search method (Bergstra & Bengio 2012) to select the best hyperparameters. We first intuitively assign value and a proper range with some step sizes which correspond to the hyperparameters. then, we apply grid search to search for the best combination for the hyperparameters on prior distributions. After locating a smaller range for each hyperparameters. we use random search to search for better combinations, if it exists. In the end, we find our best hyperparameter combinations for NIG prior distributions.
4.1 RESULTS FOR IMBALANCED REGRESSION ACCURACY
We report the accuracy of different methods in Table ! and Table AgeDB-DIR and IMDB-WIKI- DIR, respectively [ In both tables, we can conclude that our methods outperform the baselines in their categories. For ablation studies, see Table 5 and Table the Appendix. Note that to ensure fair and solid comparison, we re-run the DIR methods based on our machine and software settings" ht
Overall Performance, As shown in the last category (i.e., last four rows) of both tables, our proposed method's 's best variants compare favorably against the state of the art including DIR variants (Yang) et al., 202T) and DER (Amini et al., 2020), especially on the imbalanced data samples (i.e., in the few-shot columns). This verifies the effectiveness of our methods in terms of overall performance.
4.2 RESULTS FOR IMBALANCED REGRESSION UNCERTAINTY ESTIMATION
Different from DIR Yang et al. 2021) which only focuses accuracy, we create new benchmark for uncertainty estimation in imbalanced regression. Table and Table 4 show the results on uncertainty estimation for two datasets AgeDB-DIR and IMDB- WIKI--IR, respectively. Note that most baselines from Table and Table 2 are deterministic methods (as opposed to probabilistic
3 Results for STS-B-DIR
reported in Table Table 8 and Table 2 of the Appendix.
WWe find that due to differences in PyTorch, GPU, and CUDA versions, as well as numbers of GPUs used for parallel training, the results in DIR may vary. Furthermore, the randomness in multiple workers in the Dataloader also affect the performance.
8

---

Under review as a conference paper at ICLR 2023
Table 2: Evaluation results of accuracy IMDB-WIKI-NI
Matris
MSE
MAE
Shot
Med.
New
All
Many
Med.
All
Many
Med.
VANILLA
Yau et
2021
135.48
107.01
352.02
973.75
7.99
7.18
14.88
25.72
4.51
4.12
10.45
21.40
MIXUI
141.11
109.13
369.95
1037.95
8.22
7.29
15.23
25.13
4.68
4.22
12.25
21.55
MIXIP
2019
133.45
108.35
362.72
957.53
8.22
7.39
25.70
4.80
10.85
21,86
SMOON
136.00
199.00
8.8%
7.90
14.07
25.01
4.63
8.74
20.12
SQNNy
Yanr 21 41. X021
134.36
111:25
305.63
834
7.87
7.24
12:44
22.76
4.47
4.22
7.25
15.10
DER
Amne ci14
133.81
107.51 51
352.90
916.18
7.85
7.18
13.35
24.12
4.47
4.18
8.18
15.15
FDS
23.20
4.39
13.42
LDS
320.25
530.11
7.91
13,02
22.43
4.48
7.72
13.75
LDS FTDS orageraa
2021
136.72
112.76
322.50
811.35
8.08
7.47
13.21
22.54
4.65
4.39
8.01
14.33
1,DS FISS STRC Volgers
1021 Ammii at. 2000
120.86
90.75
297.64
875, 10
7:24
6.64
11.87
23.64
3.93
3.69
6.64
16:00
VIR (OURS)
119.60
99.35
298.85
809.34
7.23
5.65
11.90
21.78
3,90
3.68
6.51
13,34
OURS VANILLA
+15.88
47.76
+53.17
+164.39
+0.76
+0.52
+2.98
+4.94
+1.61
+0.44
+3.55
48.06
OURS vs. SQINY
+14.76
+11.98
+9.7%
+34.74
40.64
+0.58
+0.54
+0.98
+1.57
+0.54
+0.74
+1.76
OURS DER
+14.21
+5.36
+34.05
110684
+0.62
10.53
+1.45
+2.34
+0.57
10.50
+1.87
+1.84
LDS
DIR)
+2365
methods like ours) and cannot provide uncertainty extimation; therefore they are not applicable here. To show the superiority of our VIR model, we create a strongest baseline by concatenating the DIR variants (LDS + FDS) with the DER (Amini al. 2020).
Results show that VIR outperform the
Table 3: Uncertainty estimation results on AgeDB-DIR.
baselines in all few-shot metrics. In some categories. VIR may not per- form better in the overall, many-shot and median shot metrics, but the gap tends to be minimal. Note that our pro- posed methods mainly focus on the imbalanced setting, therefore we also focus on the few-shot metrics. Lastly, comparing our model variant with the best performance against the baseline (DER), we can conclude that our methods successfully improve uncertainty estimation in the probabilistic imbalanced regression setting.
Skat
A1
Mar
Mec.
Foy
AB
Staxy
Mad.
Pon
Deer Ensersntm [ahageennggaans
2012
5311
4.31
6.726
3.535
0.541
0.06
Du00s
D:885
DES Arnnn
5.49
51400
We also observe that the improve- ments of the uncertainty estimation on IMDB-WIKI are larger than those on Age-DB. We suspect that this be- cause IMDB-W WIII contains much more training, validating and testing data, therefore enjoying more stable uncertainty estimation improvements brought by VIR compared to those in Age-DB.
Table 4: Uncertainty estimation results
Mary
100
AB
Many
Med
DER Jenmic
0351
CONCLUSION
We identify the problem of probabilistic deep imbalanced regression, which aims to both improve accuracy and obtain reasonable uncertainty estimation in imbalanced regression. We propose VIR, which can use any deep regression models backbone networks. VIR borrows data with similar regression labels to produce the probabilistic representations and modulates the conjugate distributions to impose probabilistic reweighting on imbalanced data. Furthermore, we create new benchmarks for uncertainty estimation on imbalanced regression. Experiments show that our methods outperform Kate-of-theear imbalanced regression models terms of both accuracy and uncertainty estimation. Future work may include (1) improving VIR by better approximating variance of the variances in probability distributions, and (2) developing novel approaches that can achieve stable performance even on imbalanced data with limited sample size, and (3) exploring techniques such as mixture density networks (Bishop, 1994) to enable multi-modaa modality in the latent distribution, thereby further improving the performance.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus. Deep evidential regression. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florin Balcan, and Hsuan-Tire Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Mach.
Learn. Res., 13:581-305, 2012.
Christopher M. Bishop. Mixture density networks. Technical report, 1994.
Christopher M Bishop. Pattern recognition and machine learning. springer, 2006.
Paula Branco, Luís Torgo, and Rita P. Ribeiro. SMOGN: pre-processing approach for imbalanced regression. In First International Workshop on Learning with Imbalanced Domains: Theory and Applications, LIDIA @ PKDD/ECML 2017, 22 September 2017, Skopje, Macedonia, volume 74 of Proceedings of Machine Learning Research, pp. 36-50. PMLR, 2017.
Paula Branco, Luis Torgo, and Rita P. Ribeiro. REBAGG: resampled bagging for imbalanced regression. In Second International Workshop on Learning with Imbalanced Domains: Theory and Applications, LIDTA @ ECML/KKD 2018, Dublin, Ireland, September 10, 2018, volume 94 of Proceedings of Machine Learning Research, pp. 67 PMLR, 2018.
Bertrand Charpentier, Daniel Zügner, and Stephan Günnemann. Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Counts. In Advances in Neural Information Processing Systems 33; Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
Bertrand Charpentier, Oliver Borchert, Daniel Zügner. Simon Geisler, and Stephan Günnemann. Natural Posterior Network: Deep Bayesian Predictive Uncertainty for Exponential Family Distri- butions. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357. 2002.
Zongyong Deng, Hao Liu, Yaoxing Wang, Chenyang Wang, Zekuan Yu, and Xuehong Sun. PML: progressive margin loss for long tailed age classification. In IEEE Conference on Computer Vision and Puttern Recognition, CVPR 2021, virtual, June 19-25, 2021.
Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proceedings of the 33nd International Conference Machine Learning. ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pp. 1050-1059 JMLR.org. 2016
Yu Gong, Greg Mori, and Frederick Tung. RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression. In International Conference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 7634-7649. PMLR, 2022.
Arjun K Gupta and Daya K Nagar. Matrix variate distributions, volume 104, CRC Press, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 20/6, Las Vegas, NV, USA, June 27-30, 2016, pp. 770-778. IEEE Computer Society, 2016.
Jakob Heiss, Jakob Weissteiner, Hanna S. Wutte, Sven Seuken, and Josef Teichmann. NOMU: Neural Optimization based Model Uncertainty. In International Conference on Machine Learning. ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research, pp. 8708-8758. PMLR, 2022.
Eyke Hüllermeier and Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: A tutorial introduction. CoRR, abs/1910.09457,, 2019.
10

---

Under review as a conference paper at ICLR 2023
Eddy 1lg, Özgün cCiecek, Silvio Galesso, Aaron Klein, Osama Makansi, Frank Hutter, and Thomas Brox. Uncertainty estimates and multi-hypotheses networks for optical flow. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss (eds.), Computer Vision ECCV 2018 15th European Conference, Munich, Germany, September 8- 14, 2018, Proceedings, Part VII, volume 11211 of Lecture Notes in Computer Science, pp. 677-693. Springer, 2018.
Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. SMART: robust and efficient fine-tuning for pre-trained natural language models through principled regu- larized optimization. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pp. 1990. Association for Computational Linguistics, 2020.
Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? arXiv preprint arXiv: 0407779777 2017.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICL. 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In Yoshua Bengio and Yann LeCun (eds.), 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14- 16, 2014, Conference Track Proceedings, 2014.
Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. In International Conference on Machine Learning, pp. 2796-2804.
PMLR, 2018.
Balaji Lakshminarayanaan Alexander Pritzel, and Charles Blundell. Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pp. 6402-6413, 2017.
Jeremiah Liu, Zi Lin, Shreyas Padhy, Dustin Tran, Tania Bedrax Weiss, and Balaji Lakshminarayanan. Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness. In Advances in Neural Information Processing Systems, volume 33. Curran Associates, Inc., 2020.
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang. Boqing Gong, and Stella X. Yu. Large-scale long-tailed recognition in an open world. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pp. 2537-2546. Computer Vision Foundation IEEE, 2019.
Lu Mi, Hao Wang. Yonglong Tian, and Nir Shavit, Training-free uncertainty estimation for neural networks. In AAAI, 2022.
Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: The first manually collected, in-the-wild age database. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2017 Honolulu, HI, USA, July 21-26, 2017, pp. 1997-2005, 2017.
Jiawei Ren, Mingyuan Zhang, Cunjun Yu, and Ziwei Liu. Balanced MSE for Imbalanced Visual Regression. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-22 24, 2022, pp. 7916-7925. IEEE, 2022.
Rasmus Rothe, Radu Timofte, and Lue Van Gool. Deep expectation of real and apparent age from single image without facial landmarks. Int. J. Comput. Vis., 16(2-4))14441577 2018.
Jasper Snoek, Yaniv Ovadia, Emily Fertig, Balaji Lakshminarayanan. Sebastian Nowozin, D. Sculley, Joshua V. Dillon, Jie Ren, and Zachary Nado. Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 13969-1 11869-119800 13980, 2019.
11

---

Under review as a conference paper at ICLR 2023
Hao Song, Tom Diethe, Meelis Kull, and Peter Flach. Distribution calibration for regression. In International Conference on Machine Learning, pp. 5897-5906. PMLR, 2019.
Maximilian Stadler, Bertrand Charpentier, Simon Geisler, Daniel Zügner, and Stephan Günnemann. Graph Posterior Network: Bayesian Predictive Uncertainty for Node Classification. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14. 2021, virtual, pp. 18033-18048, 2021.
Baochen Sun, Jiashi Feng, and Kate Saenko. Return of frustratingly easy domain adaptation. In Dale Schuurmans and Michael P. Wellman (eds.), Proceedings of the Thirtieth AAAI Conference Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA, pp. 2058-2065. AAAI Press, 2016.
Luís Torgo, Rita P. Ribeiro, Bernhard Pfahringer, and Paula Branco. SMOTE for regression. In Luís Correia, Luís Paulo Reis, and José Cascalho (eds.), Progress in Artificial Intelligence 16th Portuguese Conference on Artificial Intelligence, EPIA 2013, Angra do Hervismo, Azores, Portugal, September 9-12. 2013. Proceedings, volume 8154 of Lecture Notes in Computer Science, pp. 378-389, Springer, 2013.
Joost van Amersfoort, Lewis Smith, Andrew Jesson, Oscar Key, and Yarin Gal. Improving De- terministic Uncertainty Estimation in Deep Learning for Classification and Regression. CoRR, abs/2102. 14409, 1409, 2021.
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, loannis Mitliagkas, David Lopez-Paz. and Yoshua Bengio. Manifold mixup: Better representations by interpolating hidden states. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019. Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pp. 6438 6447. PMLR, 2019.
Yuzhe Yang, Kaiwen Zha, Ying-Cong Chen, Hao Wang, and Dina Katabi. Delving into deep imbal- the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 11842 1851. PMLR, 2021.
anced regression. Marina Meila and Tong Zhang (eds.), Pruceedings the
Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, and Chunhua Shen. Learning to recover 3d scene shape from a single image. In IEEE Conference Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 213-213. Computer Vision Foundation IEEE, 2021.
Eric Zelikman, Christopher Healy, Sharon Zhou, and Anand Avati. Crude: calibrating regression uncertainty distributions empirically. arXiv preprint arXiv:2005. 12496, 2020.
Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018, Conference Track Proceedings. OpenReview.net, 2018.
Zizhao Zhang, Adriana Romero, Matthew Muckley, Pascal Vincent, Lin Yang, and Michal Drozdzal. Reducing uncertainty in undersampled mri reconstruction with active acquisition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2058, 2019.
12

---

Under review as a conference paper at ICLR 2023
A DISCUSSION ON I.I.D. AND N.I.D. ASSUMPTIONS
Generalization Error, Bias, and Variance. We could analyze the generalization error of our VIR by bounding the generalization with the sum of three terms: (a) the bias of our estimator, (2) the variance of our estimator, (3) model complexity. Essentially VIR uses the N.L.D, assumption increases our estimator's 's bias, but significantly reduces its variance in the imbalanced setting. Since the model complexity is kept the same (using the same backbone neural network) as the baselines, N.L.D. will lead to lower generalization error.
Variance of Estimators in Imbalanced Settings. In the imbalanced setting, one typically use inverse weighting to produced an unbiased estimator (i.e., making the first term of the aforementioned bound zero). However, for data with extremely low density, its inverse would he extremely large, therefore leading to a very large variance for the estimator. Our VIR replaces L.I.D with N.I.D, to "smooth out" such singularity, and therefore significantly lowers the variance of the estimator (i.e., making the second term of the aforementioned bound smaller), and ultimately lowers the generalization error.
B ADDITIONAL. EXPERIMENT RESULTS
B.I ABLATION STUDY ON VIR
In this section, we include ablation studies to verify that our VIR can outperform its counterparts in DIR (i.e., smoothing the latent space) and DER (i.e., NIG distribution layers).
Ablation Study on ((2ilxx J ) To verify the effectiveness of VIR's encoder q(2, [x,],² },=")) we replace VIR's predictor zz)(z.) with lin- car layer (as in DIR). Table 3 shows that compared to its counterpart, FDS (Yang et al. 2021), our encoder- only VIR still leads to considerable improvements even without generating the NIG distribution, therefore verifying the effectiveness of our VIR's q(z,) (x; W
Table 5: Ablation study AgeDB-DIR in terms of accuracy.
Metrics
MSE
MAE 1
Shot
AlI
Mary
Med
Fow
AB
Many
Med.
Few
FDS Yang 2021
109.78
93.09
124.66
7.52
(OURS)
DER Ammreaa pony
106.11
91.12
123.45
300 76
8.11
7.56
9:01
17.69
PREnECT (OURS)
88.96
74.79
95.85
263.6
7.28
6.68
7.76
11.3
Ablation Study on p(u 2.). To ver- ify the effectiveness of VIR's pre- dictor p(us v2). we replace VIR's en-
Table 6: Ablation study on AgeDB-DIR in terms of uncer- tainty estimation.
NJJ.
coder qz, x, )i=1) with simple de- terministic encoder as in DER (Amini al. 2020). Table 5land show that compared to DER, the counter- part of VIR's predictor, our VIR's s predictor still outperforms than DER, demonstrating its effective- ness; this verifies our claim (Sec. 3.4) that directly reweighting DER breaks NIG and leads to poor performance.
Shet
Med.
Mcl.
Fox
Armm Am
2020
393
3.700
3,80
4.431
0.900
0,500 IU387
0.649
(1MA8
PARDICTOROOOA TTR (outes)
3.887
3.558
3,854
4.994
8.448
0.990
6,417
B.2 RESULT ON STS-B-DIR DATASET
In this section, report the accuracy and uncertainty evaluation STS-B-DIR. (more details for the dataset is DIR (Yang et al. 202T). From Table 7
Table S and Table Ibelow, we can conclude
Table 7: Evaluation results of accuracy on STS-B-DIR.
Metrics
MSD 1
MAD
GM -
Shot
All
Many
Mod.
Fow
All
Many
Med.
Fow
All
Many
Mal.
Few
1.152
L.001
0.537
0.764
DIR (ANU ET AL. 50211
1.000
0.912
1.368
1.055
0,812
0.772
0.989
0.509
0.560
0.555
0.759
0,477
DIR EDER (YANOUT 2021 AMINI RT AL. 2020
1.007
0.580
1.535
1.085
0.813
0.757
1,046
0.542
0.558
0.518
0.765
0.574
0.960
0.669
that our model also outperforms all baselines in terms of both accuracy metrics and uncertainty
estimation metrics in this NLP dataset; this verifies the superiority of our model for NI.P datasets.
13

---

Under review as a conference paper at ICLR 2023
Table 8: Evaluation results of accuracy STS-B-DIR.
Metries
Pearson f
Spearman 1
Shot
All
Many
Med.
Few
All
Many
Med.
Few
0.718
0.612
0.723
0.530
DIR YYNG AL.. 2021
0.732
0.646
0.742
0.731
0.519
DIR +DER YYANG
2021
2020
0.729
0.714
0.635
0.730
0.526
VIR (OURS)
0.765
0.740
0.663
0.770
0.770
0.713
0.534
0.770
Table 9: Uncertainty estimation results STS-B-DIR.
Metries
NLL
AUSE 1
Shot
All
Many
Med.
Few
All
Many
Med.
Few
DER (YANG AL... 2021
AI.. 2020
2.561
2,880
0.672
0.609
VIR (OURS)
1.996
1.810
2.754
2.152
0.591
0.575
0.602
0.510
B.3 DIFFERENCE BETWEEN DIR'T AND OUR REPRODUCED RESULTS
To reproduce the results on AgeDB, we exactly the same settings as DIR's code (Yang et al. 2021> (i.e., by directly running their code on our machines without modifying hyperparameters). for each model DIR we report, we use five different random seeds to produce five results. We then report the performance by taking the average of them. Table T0 and Table 1T show the example for SQINV and LDS on AgeDB-DIR. From the table we can see that under our hardware and
Table 10: Results of running SQINV for 5 different random seeds on AgeDB.
Metrics
MSE
MAE 1
GM
Shot
Many
Many
Many
SQINV
SQINV
111.55
93:43
141.03
209.17
8.12
7.47
9.17
11.58
5.21
4.85
5.75
8.25
SQINY
114.33
7.59
SQINV
106.24
91.81
120.26
203.78
7.94
7.39
8.58
11.39
5.06
4,74
5.41
7.66
SQINV AVG
108.77
92.60
130.94
207 65
8.06
7.46
3.95
11.52
5.13
4.77
6.30
8.08
SQINV RESULTS FROM YANG ET AL. 2021
105.14
87.21
127.66
212.30
7.81
7.16
3.80
11.20
4.99
4.57
5.73
7.77
software environments, the SQINV model and LDS+FDS model (SOTA in DIR) could not perform
as well as is reported in DIR Yang et al. (202T), therefore for fair comparison, we use our replicated performance rather than theirs.
B.4 ABLATION STUDY ON À
In this section, we include ablation studies on the in our objective function. For À E (10.0, 1.0, 0.1, 0.01, 0.001),
our VIR model the AgeDB dataset. Table 12 shows the results. We can conclude that when X = 0.1, our model achieves the best performance.
14

---

Under review as a conference paper at ICLR 2023
Table 11: Results of running LDS+FDS for 5 different random seeds on AgeDB.
Metrics
MSB
MAE
GM
Many
Many
Many
LDS+
104.35
88.62
128.99
194.06
7.26
5,00
7.51
LDS+FDS
104.59
94.63
125.60
200.14
7.98
7.44
8.77
11.16
5.00
5.62
7.81
LDS+FDS
110.17
95.97
123.24
208.11
8.07
7.54
8.71
11.41
5.09
4.73
5.71
7.48
LDD
102.68
136.41
201.16
7.50
5.08
7,56
LDS
105.77
91.07
127.00
185.85
7.93
7.35
8.80
10.96
5.07
4.74
5.52
7.73
7.97
LDS XTD
6.41
11.70
3.52
55.
0.01
0.01
0.01
0.04
0.01
0.03
0.01
0.02
LDS RESUETS
2021
99.46
209.27
6.79
Table 12: Ablation study on À for VIR on AgeDB-DIR
Metries
MSE
MAE
NLL J
Shot
Many
Med.
Few
All
Many
Med.
Few
Many
Med.
Few
104.31
91.01
116.43
196.35
7.88
8.42
3.827
3.733
4.140
4.407
= 1.0
104.10
87.28
128.26
196.12
7.83
7.21
8.81
10.89
3.848
3.738
4.041
4.356
= 0.1
86,28
76.87
101.57
132.90
7.19
6.75
7.97
9.19
3.785
3.694
3.963
4.151
0.01
86.86
76.58
99.95
147.82
7.12
6.69
7.72
9.59
3.887
3.797
4.007
4,401
0.001
87.25
74.13
104.78
162.64
7.13
6.64
7.92
9.63
3.980
3.868
4.161
4.546
15


Reviews:
[{"review_id": "x8Sp4mV_NwU", "reviewer": "Reviewer_xTgV", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper looks at uncertainty estimation for regression on imbalanced datasets. It proposes a new method called VIR which combines variational inference, smoothed statistics, and conjugate distribution parametrization. In the experiments, VIR is evaluated on 2 imbalanced datasets on accuracy and calibration metrics.", "summary_of_the_review": "Overall, I vote for strong reject. The task is important and well motivated and the . My major concerns are about the related work, and the experiences (see cons beow). Hopefully the authors can address my concern in the rebuttal period.", "strength_and_weaknesses": "Pros:\n\n1. The task of uncertainty estimation for regression on imbalanced dataset is important.\n2. The idea of combining smoothed statistics to fix imbalanced dataset and produce uncertainty estimates via parametrizing conjugate distributions is interesting.\n3. The method achieves great improvement on 2 datasets.\n\n##########################################################################\n\nCons:\n\n- Related works: The paper misses many important related works for uncertainty estimation for regression like [1, 2, 3, 4, 5, 6, 7] but also many others. This includes very common baselines like Dropout [6] and Ensemble [1] which work for regression. The paper also does not discuss GP-based methods [2, 3] which can be used for regression. Further, the paper does not discuss Posterior Networks methods [4, 5, 7]. More specifically, [4, 5] shares many similarities with VIR. They use conjugate distributions, pseudo-count interpretations, posterior updates, and variational losses which are key parts of VIR. NatPN is also designed to work for regression. Action suggestion: discuss all of these methods in the related work section.\n- Desiderata: The two desiderata are sometimes not clear. It feels that the two desiderata are high quality *uncertainty estimation* and high performance on *imbalanced dataset*. However the paper mentions that the desiderata are “performance improvement and uncertainty estimation”. The paper also phrases the identification of the desiderata of imbalanced dataset and uncertainty estimation as a contribution.  However people already looked at imbalanced datasets and uncertainty estimation separately in previous works. Finally, the paper does not discuss existing desiderata for uncertainty estimation [8, 7, 9]. Action suggestion: I would recommend to change the phrasing of this contribution and have a discussion on existing desiderata in uncertainty estimation.\n- Loss: I felt that the description of the loss was sometimes confusing. I would be interested in the derivation fo the loss. This could also be provided in the appendix. In eq.(1), what is p_\\theta(z_i) ? Does this ELBO loss assume any prior ? What is the importance of the regularization loss in the total loss ? Action suggestion: Beyond the answer to these questions, it would be intresting to show the important of the regularization term in an experiment.\n- Clarity: The paper is sometimes hard to read. E.g. the paper mentions multiple times “see below” which leaves unclear where to exactly find the missing information. Action suggestion: make explicit reference when pointing to further details.\n- Bins: It is unclear what is the sensitivity of the method to the number of bins. It was also unclear to me when readin sec. 3.1. what data are used to build the mean representation from the bins. Action suggestion: I would recommend to clarify the bins construction since it is an essential part of the method. I would also be interested in an experiment on the number of bins.\n- Experiences: The experiments do not look very extensive to me. They consider only two datasets and a single backbone network. They do not compare to DropOut, Ensemble, GP, or NatPN which would be appropriate baselines. They do not look at common uncertainty estimation metrics like OOD detection scores. They also do not provide error bars which are key to assess the significance of the results. Action suggestion: I would recommend to add at least one dataset (e.g. depth estimation) with another backbone architecture. I would also recommend to add at least Ensemble which is a common and powerful baseline and NatPN which shares many similarities with VIR. I would also recommend to add error bars.\n\nI am happy to improve my score if a majority of the above points are addresses (e.g. with the action suggestions).\n\n[1] Simple and scalable predictive uncertainty estimation using deep ensembles, NeurIPS 2017\n\n[2] On feature collapse and deep kernel learning for single forward pass uncertainty.\n\n[3] Simple and principled uncertainty estimation with deterministic deep learning via distanceawareness, NeurIPS 2020.\n\n[4] Posterior Network: uncertainty estimation without ood samples via density-based pseudo-counts. NeurIPS 2020\n\n[5] Natural Posterior Network: deep bayesian uncertainty for exponential family distributions, ICLR 2022\n\n[6] Dropout as a bayesian approximation: representing model uncertainty in deep learning, ICML 2016\n\n[7] Graph Posterior Network: bayesian predictive uncertainty for node classification. NeurIPS 2021.\n\n[8] Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift, NeurIPS 2019.\n\n[9] NOMU: Neural Optimization-based Model Uncertainty. ICML 2022", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "R8lD11VAL0w", "reviewer": "Reviewer_EUDh", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a novel approach named variational imbalanced regression for the probabilistic deep imbalanced regression problem. It leverages data with similar regression labels to compute the latent representation’s variational distribution, and predicts the entire normal-inverse-gamma distributions and modulates the associated conjugate distributions to impose probabilistic reweighting on the imbalanced data.", "summary_of_the_review": "This paper provides some new perspectives for imbalanced regression. However, I have concerns about whether this work can be accepted in its current form. I will update my reviews based on the authors' responses.", "strength_and_weaknesses": "Strength\n\n- The problem studied in this paper is interesting and valuable. Both balanced accuracy and uncertainty estimation are important topics in regression.\n- The paper is well-organized and clearly written, which is easy to follow.\n- This paper provides some novel perspectives for probabilistic deep imbalanced regression. Leveraging Neighboring and Identically Distributed data is a valuable attempt for handling imbalance and uncertainty issues. \n\nWeaknesses\n\n- In Section 3, the authors declare some limitations of LDS and DER. However, the descriptions are somewhat confusing. In my opinion, the authors should provide theoretical perspectives or specific cases for the limitations, thus illustrating the advantage of VIR.\n- Since the I.I.D. assumption has been replaced with the N.I.D. assumption, some theoretical guarantees could be provided to further demonstrate the effectiveness of the algorithm.\n- The impact of $\\lambda$ is unclear. Further ablation studies should be conducted to illustrate the impact of $\\lambda$.\n- One can observe in Table 4 that the uncertainty estimation performance of the proposed VIR is worse than LDS+FDS+DER on Medium labels. The authors should further analyze the reasons for this phenomenon.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "MlyGZYHbIyb", "reviewer": "Reviewer_5JWG", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors introduce a probabilistic model called Variational Imbalanced Regression (VIR) to deal with the problem of imbalanced regression. VIR borrows data with similar regression labels to compute the latent representation’s variational distribution. VIR first encodes a data point into a probabilistic representation and then mixes it with neighboring representations. The proposed method not only performs well in imbalanced regression but naturally produces reasonable uncertainty estimation as a byproduct. ", "summary_of_the_review": "I like the idea of extending conventional deterministic encoding to be probabilistic for deep imbalanced regression based on the formulation of variational autoencoders. The presentation of the idea is clear and technically sound. However, the experimental evaluation needs to be improved to make the method more convincing. Therefore, I recommend rejection at this moment but encourage the authors to improve and submit to future venues. ", "strength_and_weaknesses": "Strength: \n\n(1) The authors introduce the setting of uncertainty estimation to the deep imbalanced regression. I appreciate this contribution since DIR was proposed very recently by Yang et al. (ICML 2021), and this can be a good contribution to the future investigation of DIR.\n\n(2) The method is interesting, especially for the reweighting scheme of the balanced predictive distribution. \n\nWeakness:\n\n(1) Considering the novelty, I think VIR is more like a combination of [1] and [2] from the current manuscript. Though VIR is empirically superior and introduces uncertainty estimation as a byproduct, it is still vague how VIR is motivated compared to the FDS in [1], since both methods have kernel estimation as the core contribution. And the idea of extending deterministic space to a probabilistic one is interesting but lacks more motivation. \n\n(2) Missing related work [3, 4]. Both were published before the submission deadline and the authors should discuss the difference and compare VIR with them.\n\n(3) The experimental setup can be improved. First, the authors only choose two age estimation datasets whose distributions are similar. Second, the reported numbers are not very convincing (please see the next section for quality and reproducibility). \n\n(4) I’m not fully convinced by the choice of Gaussian distribution as the prior distribution. For some specific datasets with regression labels like ages, this may possibly be a suitable assumption. But the authors claim the contributions to the task of general deep imbalanced regression, so I encourage the authors to conduct experiments on other types of regression datasets. It will strongly improve the contribution of this work if the authors can consider more datasets proposed in [1] or [3]. \n\n[1] Yang et al.,  Delving into Deep Imbalanced Regression. ICML 2021 \\\n[2] Amini et al., Deep Evidential Regression. NeurIPS 2020 \\\n[3] Ren et al., Balanced MSE for Imbalanced Visual Regression, CVPR 2022 \\\n[4] Gong et al., RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression, ICML 2022 \\\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "MiPDJP6z_-b", "reviewer": "Reviewer_ddev", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper simultaneously addresses the label imbalance problem and uncertainty qualification capability in regression.\nThe authors propose to enhance the reweighting technique dealing with the imbalance problem in (Yang et al., 2021) to be applicable to VAE and combine the output distribution and the corresponding loss in (Amini et al., 2020), which provides uncertainty qualification capability.\nExperimental results on several real-world datasets demonstrate that the proposed method performs better than state-of-the-art imbalanced regression models in terms of both accuracy and uncertainty estimation.", "summary_of_the_review": "Although the proposed method is a good combination of the two SOTA methods and performs well, the description of the proposed method does not seem self-complete.", "strength_and_weaknesses": "*Strength\n- The authors simultaneously address the label imbalance problem and uncertainty qualification capability in regression, which is a novel problem setting.\n- The non-trivial combination of two different approaches is proposed and works well.\n\n*Weaknesses\n- The intuition that each of the components of the proposed method works for their purpose is not described well in the main text. For example, there is no overall procedure for their reweighting technique, which is provided in the paper (Yang et al., 2021) though.\n- In Section 3.3, the authors proposed to use the covariance of z for probabilistic overall covariance. However, it is unclear why they do not use the covariance of z^{mu} and z^{sigma} instead.\n- There is no intuition why the proposed approach outperformed DIR, which could be more practical in terms of prediction performance in some cases, in my understanding.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "JWMRAL3dWq", "reviewer": "Reviewer_bYud", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a probabilistic deep learning model, namely variational imbalanced regression (VIR), when the distribution of outcome is imbalanced. This method uses data with similar outcomes to estimate the distribution of the latent representation, provides a point estimate of the outcome, and provides uncertainty quantification of the point estimate. The method can work well for the minority group in the data. ", "summary_of_the_review": "This paper provides a solution to address the problem of probabilistic deep imbalanced regression. The proposed method can potentially accurately estimate the outcome and quantify the uncertainty in the estimation. I think the paper can benefit from providing more practical guidance on using the proposed method, and providing some theoretical guarantee on the proposed method. ", "strength_and_weaknesses": "Strength: \n\nThis method uses a probabilistic representation approach that can quantify the uncertainty in the estimated prediction model. In addition, the method can potentially work well for the minority group in the data. \n\n\nWeakness:\n\nI think the paper can benefit from providing more practical guidance on implementing the proposed method. For example, how to select the number of bins B in Section 3.1? Why is it a good idea to partition the label space $\\mathcal{Y}$ into equal-interval bins, but not equal-size bins (with the same number of data points)? How to select the bandwidth in the kernel function $k(\\cdot)$ to compute the neighboring data and smoothed statistics in Section 3.3? How does the bandwidth vary with the sample size N?\n\nIn addition, can the authors provide some theoretical guarantee (e.g., unbiasedness, consistency) on the proposed method? ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject