Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint
Authors: Guangkun Nie
Gongzheng Tang
Shenda Hong
Venue: ICLR 2025 Poster (2025)
Keywords: Deep imbalanced regression
sparse data region optimization
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: Introducing Dist Loss to enhance deep learning models' performance in few-shot regions for imbalanced regression tasks

Abstract: Imbalanced data distributions are prevalent in real-world scenarios, presenting significant challenges in both classification and regression tasks. This imbalance often causes deep learning models to overfit in regions with abundant data (manyshot regions) while underperforming in regions with sparse data (few-shot regions). Such characteristics limit the applicability of deep learning models across various domains, notably in healthcare, where rare cases often carry greater clinical significance. While recent studies have highlighted the benefits of incorporating distributional information in imbalanced classification tasks, similar strategies have been largely unexplored in imbalanced regression. To address this gap, we propose Dist Loss, a novel loss function that integrates distributional information into model training by jointly optimizing the distribution distance between model predictions and target labels, alongside sample-wise prediction errors. This dual-objective approach encourages the model to balance its predictions across different label regions, leading to significant improvements in accuracy in fewshot regions. We conduct extensive experiments across three datasets spanning computer vision and healthcare: IMDB-WIKI-DIR, AgeDB-DIR, and ECG-KDIR. The results demonstrate that Dist Loss effectively mitigates the impact of imbalanced data distributions, achieving state-of-the-art performance in few-shot regions. Furthermore, Dist Loss is easy to integrate and complements existing methods. To facilitate further research, we provide our implementation at https://github.com/Ngk03/DIR-Dist-Loss.

Published as a conference paper at ICLR 2025
DIST Loss: ENHANCING REGRESSION IN FEW-SHOT REGION THROUGH DISTRIBUTION DISTANCE CON- STRAINT
Guangkun Niel2*, ,Gogghhngg Tang!= Shenda Hong
Institute of Medical Technology, Health Science Center of Peking University, Beijing, China
2 National Institute of Health Data Science, Peking University, Beijing, China
3 Institute for Artificial Intelligence, Peking University, Beijing, China
n Leuuagkkunsstu pku. edu. cn, gztang p.u. edu. en, howgshenda@pku.coo edu...c.ueedeuc......... cn
ABSTRACT
Imbalanced data distributions are prevalent in real-world scenarios, presenting sig- nificant challenges in both classification and regression tasks. This imbalance often causes deep learning models to overfit in regions with abundant data (many- shot regions) while underperforming in regions with sparse data (few-shot re- gions). Such characteristies limit the applicability of deep learning models across various domains, notably in healthcare, where rare cases often carry greater clini- cal significance. While recent studies have highlighted the benefits of incorporat- ing distributional information in imbalanced classification tasks, similar strategies have been largely unexplored imbalanced regression. To address this gap, we propose Dist Loss, a novel loss function that integrates distributional informa- tion into model training by jointly optimizing the distribution distance between model predictions and target labels, alongside sample-win wise prediction errors. This dual-objective approach encourages the model to balance predictions across different label regions, leading to significant improvements in accuracy in few- shot regions. We conduct extensive experiments across three datasets spanning computer vision and healtheare: IMDB SIBB-WIK-DDIR AgeDB-DIS -IR, and ECG-K- DIR. The results demonstrate that Dist Loss effectively mitigates the impact imbalanced data distributions, achieving state-of-the-arr performance in few-shot regions. Furthermore, Dist Loss is easy to integrate and complements exist- ing methods. To facilitate further research, provide our implementation https: /ttptttttttthto///////g/...h http:::://itttb...oo httph////cc//KKC/D3DCC -
INTRODUCTION
Imbalanced data distributions are prevalent in the real world, where certain target values sig- nificantly underrepresented [Buda et al. (0118): : Liu et al. (2019). In imbalanced regression tasks, deep learning models tend to bias their predictions toward regions with abundant data (many-shot regions) to minimize overall eтrоr, resulting in significantly higher errors in sparse-data regions (few-shot regions). This phenomenon severely limits the applicability of deep learning models in certain contexts, such as healthcare scenarios where rare cases often carry significant importance, and significant errors in these samples could lead to potential adverse events.
Taking the prediction of potassium concentration from electrocardiogram (ECG) signals as an ex- ample, the model takes ECG signals as input and predicts the corresponding potassium concentra- tion. Figure [ illustrates the distribution of potassium concentrations in real-world dataset, where the majority of samples fall within the normal range, while abnormal potassium concentrations (s 3.5 mmol/L 2 5.5 mmol/L) are and concentrated in the few-shot region. Due to this im- balance, deep learning models tend to bias their predictions towards the normal range in order to
Authors contributed equally to this research.
Corresponding author.

---

Published as a conference paper at ICLR 2025
Distribution of K" concentration
Vanilia model predictions & labels Noa aodel pootatkes
Dist Loss-hassed model prodictions & labols Dec lanse wneel prodictenn
Heportaledd
prequeency
Treequannc
L'requency
K- concentration (mmml/L)
K concenteation fammol/L)
concentration (mama//L)
15
(a)
(b)
(c)
Figure 1: A real-world healthcare task of potassium (K+) concentration regression from ECGs. (a) Both hyperkalemia (high K+) and hypokalemia (low k+) are predominantly found in the few-shot region, with normal K are located in the many-shot region. Hyperkalemia and hypokalemia are life-threatening conditions that can lead to cardiac arrest and ventrieular fibrillation, necessitating accurate and timely detection. Conversely, normal K+ concentrations (the many-shot region) are of little concern, as inaccurate and untimely detection of these samples has minimal impact. Here, we follow YYang et al.] (2021) to define the few-, median-, many-shot regions. (b) illustrates the significant distribution discrepancy between the vanilla model's predictions and the labels, stemming from the imbalanced data distribution. Here, the term "vanilla model" refers to a model that employs no specialized techniques to address imbalanced data. The orange histogram represents the label distribution, while the blue histogram depicts the prediction distribution from the vanilla model. It is evident that the model's predictions are heavily concentrated in the many-shot region and seldom fall into the few-shot region. (c) demonstrates the effectiveness of Dist Loss in reducing the distribution discrepancy. The orange histogram indicates the label distribution, and the blue histogram shows the prediction distribution from the model enhanced with Dist Loss. It is clear that the distribution discrepancy is significantly reduced.
However, abnormal potassium concentrations can severely af- feet metabolism and cardiac function, potentially leading life-threatening arrhythmias or sudden death [Ferreira et al. (2020); Crotti et al. (2020); Kim et al. (2023). Therefore, in clinical settings, accurately predicting abnormal potassium concentrations is far more critical than accurately identi- fying normal levels Galloway et al. 2019); Harmon et al. (2024), yet this remains a challenge for existing deep learning models. Enhancing model accuracy the few-shot region under imbalanced data distributions continues to be both significant challenge and an important goal Branco et al. 2017 Steininger et al. (202T).
minimize overall error (Figure Tb
In imbalanced regression tasks, a significant challenge lies in the substantial discrepancy between the distribution of the model's predictions and the true label distribution, as illustrated in Figure [b] The orange histogram represents the ground truth, while the blue histogram depicts the distribu- tion of predictions made by the vanilla model, which refers to a model trained without techniques specifically designed to address data imbalance (using LI loss here). It is evident that the model predominantly predicts values within the many-shot region, with very few predictions falling within the few-shot region. This discrepancy highlights a critical consequence of imbalanced data distribu- tions: the misalignment between the model's predictions and the target labels. While prior research has focused on addressing the adverse effects of class imbalance in classification tasks by incorporat- ing distributional information into the training process Feng et al. (2018); Zheng et al. (2020): TTin al. (020)), similar strategies have been largely unexplored in imbalanced regression. Therefore, key research direction is to investigate whether leveraging distributional information can effectively reduce prediction errors in the few-shot region by aligning the distribution of the model's predictions with the underlying label distribution.
Based on this concept, we propose Dist Loss, a novel loss function comprising two key compo- nents: distribution alignment optimization, which minimizes the discrepancy between the model's prediction distribution and the label distribution, and sample-wise prediction optimization, which ensures accurate predictions at the individual sample level. The distribution alignment optimization is achieved through three steps: (1) Generating pseudo-labels: we apply kernel density estima- tion (KDE) to the label set Parzen (1962) to model the label distribution and sample pseudo-labeke labels that reflect this distribution; (2) Constructing pseudo-predictionn predictions: we sort the model's predictions to construct pseudo-prediction that represent the prediction distribution; (3) Measuring distribu-
2

---

Published as a conference paper at ICLR 2025
tion discrepancy: approximate the discrepancy between the prediction and label distributions by computing the distance between the pseudo-labels and pseudo-predictiog distribution alignment and sample-wise prediction accuracy, Dist Loss mitigates errors in individual predictions while ensuring that the model's prediction distribution better conforms to the label distri- bution. As shown in Figure Tcl this approach effectively addresses the distribution mismatch caused by data imbalance and has been proven to improve prediction accuracy in the few-shot regions.
By jointly optimizing
To validate the effectiveness of Dist Loss, we conduct comprehensive experiments three datasets spanning computer vision and healthcare: curated ECG-K-DIR dataset. Our results demonstrate substantial improvement in accuracy for rare samples, leading to sfate-of-the-art (SOTA) performance. Furthermore, our experiments show that Dist Loss compatible with existing techniques, yielding further performance gains when integrated.
AgeDB-DIR, and our meticulously
In summary, the contributions of this paper are:
We analyze the impact of imbalanced data distributions in regression from a distributional perspective and introduce the concept of aligning the model's prediction distribution with the label distribution by leveraging distributional priors.
We propose a differentiable approach for measuring distribution distance in regression tasks, extending distribution alignment techniques from classification to regression.
Extensive experiments on diverse datasets demonstrate the effectiveness of Dist Loss in imbalanced regression, achieving SOTA performance in few-shot regions.
RELATED WORK
2.1 IMBALANCED CLASSIFICATION
Research on the problem of imbalanced classification mainly focuses on improving the loss function to enhance the model's ability identify the minority class. Weighted cross entropy King & Zeng (2001) gives higher weights to minority class samples, allowing the model to pay more attention to minority class samples when facing class imbalance. Focal loss Lin (2117) reduces the influence of the majority class by dynamically adjusting the weights in the loss function, further improving the performance of the minority class. Combining data augmentation and resampling techniques is also a common strategy. RUSBoost [Seiffert et al. (2009) combines random undersampling and boosting to reduce the majority class while maintaining the performance of the model. SMOTE Chawla et al. (2002) further improves the classification results by expanding the minority class data through synthetic samples. The combination of adversarial training and loss functions has also gradually attracted attention, and adversarial reweighting SSagaaa et al. (2019) improves the accuracy of minority classes.
2.2 IMBALANCED REGRESSION
Unlike classification tasks, where labels are discrete and bounded, regression tasks involve continu- and unbounded labels, and the distances between labels carry meaningful semantic information. These fundamental differences prevent methods designed for imbalanced classification from being directly applied to imbalanced regression. Existing methods for addressing imbalanced regression can be categorized into three levels: input, feature, and model output. At the input level, meth- ods primarily focus on resampling the training dataset. SMOTE Chawla et al. (2002): Torgo et al. (213)) and its variant SMOGN Branco et al. (2017) generate new samples by interpolating between minority samples and their nearest neighbors. BBancoo et al. (2018) further enhances this approach by integrating a bagging- based ensemble method with SMOTE to mitigate the impact of imbalanced data distributions. At the feature level. Yang et al. [202 introduces feature distribution smoothing (FDS), which transfers feature statistics between nearby target bins to smooth the feature space. VIR Wang & Wang (2024) borrows data with similar regression labels to compute the variational distribution of the latent representation. Ranksim Gong et al. (022) employs contrastive learning to bring the feature representations of samples with similar labels closer while pushing apart those with dissimilar labels. Similarly, ConR Keramati et al. (004) designs positive and negative sample pairs
3

---

Published as a conference paper at ICLR 2025
Empiriiaal distibution
Dorch sampling
KDE
Expecsed frequency
ditribation
Lentoro
4.3.3444444
Significant
2. hequenciess S. 11 3.
L()
discrepancy
Optimization
diserepancy
2...2.662..0)
ep- 2,,,1,,,,,,,,,,,,
Prodictions
Bunch sumplig
Sorted peodictions (Pesudo predictions)
Medel predictinns
predicions bacch
Training set
Batching
Dist loss calculation
Optimization objective
Figure 2: The presence of imbalanced data distributions introduces noticeable distribution dis- crepancy between the model's predictions and labels. Dist Loss mitigates this imbalance by simul- taneously minimizing this discrepancy and sample-wise prediction errors. Initially, KDE is applied to estimate the label distribution and compute the expected frequency of each label within a batch, thereby generating pscudo-labed labels that incorporate label distribution information. For example, given the labels [1. 3, 4, 6] and their computed expected frequencies [1, 2, 3, 1]. the resulting pseudo- labels would be [1, 3, 4, 4, 4, 6], where each label appears according its expected frequency. Subsequently, the model's predictions within batch are sorted to obtain an ordered sequence that captures the prediction distribution. For instance, if the model's initial predictions are [5, 2, 6, 3, 2, 7. IJ. sorting them yields [1, 2, 2, 3, 5, 6, 7], preserving the distributional characteristics of the pre- dictions. Measuring the distance between these pseudo-labels and iseudo-prediction encapsulate distribution information, provides an approximation of the distributional discrepancy. By optimizing both the distribution distance and sample-level prediction errors during training, the model effectively alleviates the adverse effects of imbalanced data, significantly enhancing accuracy, particularly in few-shot regions.
which both
based on label similarity, transferring label space relationships to the feature space in a contrastive manner. At the model output and label level, regressor retraining (RRT) Yang et al. (2021) decou- ples the training of the encoder and regressor, retraining the regressor with inverse reweighting after normal encoder training. DenseLoss Steininger et al. (2021) and label distribution smoothing (LDS) Yang et al. (2021) measure label rarity through KDE, assigning higher weights to rare samples to
enhance the model's focus on minority samples. Balanced MSE Ren et al. (022)) leverages the training label distribution prior to restore balanced predictions.
However, existing research on imbalanced regression often overlooks the significant distribution discrepancy between model predictions and labels, and distribution information, which has been proven effective in imbalanced classification, rarely utilized. In contrast, our approach intro- duces distribution distance optimization on top of the traditional focus sample-wise prediction error. By concurrently aligning the prediction distribution with the label distribution and minimiz- ing sample-level prediction errors, our method significantly improves accuracy in few-shot regions. This enhancement is achieved without incurring additional computational costs or requiring metic- ulous hyperparameter tuning. Extensive experiments demonstrate the superiority of our approach in handling critical and informative rare samples few-shot regions, achieving SOTA results.
METHOD
3.1
PROBLEM SETTING
Let D be training dataset consisting of samples, denoted as {(x))) Mo)) where X(s) nd represents the input and V(3) E RR denotes the corresponding label. To facilitate processing, the continuous label space y is discretized into B bins of equal width, such that y U [ys, Bb+1).
4

---

Published as a conference paper at ICLR 2025
where yb is the lower bound of bin and V1 1/ < for convenience, the lower bound yb of the bin [yo. Vb+11 will represent any label value y(i) that falls within that bin. The set of bins is denoted as the width of each bin, denoted Ay indicates the minimum resolution of interest when processing the label space. For example, in age estimation, one might set the bin width to I, resulting in Ay 96+1 yb 1, vb B. Additionally, Vi as Pi. and the probability distribution of labels can be estimated using KDE, which is a non- parametric statistical method used estimate the probability density function of a random variable without assuming a specific distribution form. Notably, the discretization of the label space serves to facilitate the of label distribution information while preserving the regression nature of the problem, rather than transforming it into classification task. Since continuous probability density functions are difficult to process directly, we discretize them into bins for practical computation and estimation. This discretization applies to the entire label space, which includes all possible values, not just those occurring in the training dataset.
yB. In subsequent discussions,
{1, 2, .... B}. In practical scenarios,
define the probability of observing label
3.2
DIST LOSS
One of the optimization objectives of Dist Loss is to minimize the distance between the predic- tion and label distributions in regression tasks. The core challenge lies in measuring the distance between these two distributions in a differentiable manner. Traditional metrics for measuring dis- tribution distance, such Kullback-Leibler divergence and Jensen-Shannon divergence, cannot be directly implemented in differentiable form for regression tasks. Therefore, we have devised an al- ternative approach in the implementation of Dist Loss to realize a differentiable distribution distance measurement in regression scenarios. Specifically, we approximate the distance between the label and prediction distributions by sampling from these distributions and quantifying the differences between the sampled values to estimate the distance.
3.2.1 CALCULATION OF DIST LOSS
As illustrated in Figure 2 we sample from the label and prediction distributions generate pscudo- labels and pseudo-pre peuudo-predictoos, which encapsulate the distribution information of the labels and pre- dictions. Taking the generation of pseudo-labels
example, we will now detail the process.
To generate pseudo-labels that contain label distribution information, we first randomly sample M points from the label distribution, where M corresponds the batch size during model train- ing. The expected frequencies of the label can be estimated by multiplying the number of sampling points M by the probability of that label ps. Based on this, we construct sequence NL (R1, n2.. , n.) to represent these expected frequencies, where ement in the obtained Ny, represents the expected frequencies of the corresponding label. Since these frequencies may be fractional, we need to convert them to integers while ensuring that the sum after conversion still equals M. Here, we denote the converted integer sequence by Nr. (n', n n'g). To acquire Nz., we first take the floor of each element in Nr. to obtain [mn]). Then we calculate the difference a, which represents the difference between the sum of the original expected frequencies (M) and the sum after applying the floor function, following M >' [n.]. Using the difference a, we construct an auxiliary sequence A, which determines how to distribute the difference to the elements of Ni, to ensure the sum is M:
= M Each el-
the sequence Nr,, = [[ni), [mz]....
1, if | or i B -
04
0, otherwise
Each n2 is determined by adding a; the corresponding element in Ni,, where ni [n.] + a, and i E B. Finally, we generate the corresponding pseudo-labels St based on the expected frequencies, here each element S2, represented as:
Sr., arg min.es (Em n42s): for j = I,..., M,
(2)
5

---

Published as a conference paper at ICLR 2025
Distribution
Secudo-predictions
Predictions
Flatten
Flatten
Labels
Pscudo-labee labels
Distribution
Per-sample loss
Figure 3:
To illustrate the core concept behind Dist Loss, the figure simplifies its computation by assuming that the batch size equals the total number of training samples.
where j l,..., M indexes the position in the sequence. To illustrate with concrete example, con- sider label sequence (V1, V2. Us) (4, 5. 6) and an obtained frequency sequence N₂. = (1, 21,2, 3). In this case, the generated pseudo-labels Sz would be (y1, yo y2, Va, Mss ys), corresponding to the sequence (4, 5, 6, 6).
Similarly, we can perform M-point sampling on the prediction distribution and apply the same procedure to obtain the pseudo-predictions Sp, which capture the characteristics of the prediction distribution. However, in practice, this process can be simplified by sorting the model predictions within batch. It is worth noting that although pseudo-labels can also be generated by sorting the labels within batch, we choose not to adopt this approach here. Instead, we aim to better leverage the full label distribution, the batch serves merely sample. To explain why sorted sequences capture distribution information, the pseudo-labels Sz and neeudo-predictions Sp can be viewed as one-dimensional representations of the label and prediction distributions, respectively (Figure 3)
By measuring the distance between the pseudo-predictions and pseudo-lahels, we can approximate the distance between their respective distributions. Let I.-) be function that measures the distance between two sequences; then, the distribution distance can be expressed as L(Sp,St, Furthermore, using the function L(-), can simultaneously evaluate the sample-wise prediction errors, which measure the discrepancy between individual predicted values and their corresponding ground truth labels. Specifically, let Ynurch and Yrutch denote the sets of ground truth labels and model predictions in batch, respectively. The sample-wise prediction error can then be formulated as L(Y Ybatch). Ultimately, by jointly optimizing both the distributional distance and the sample-level prediction emror during training, can mitigate the issue of distribution mismatch without compromising overall accuracy, thus addressing the challenges posed by imbalanced data distributions.
3.2.2 FAST DIFFERENTIABLE SORTING
As previously mentioned, the obtained pseudo-predictionn are in ascending order, whereas the order of the model's actual predictions is random in practical scenarios. Therefore, it is necessary to sort the model's predictions to obtain the pseudo-pred predddppeeiitions.. Since the sorting operation non- differentiable, we employ fast differentiable sorting algorithm Blondel et al. (2020) to ensure the differentiability of the entire computation process.
This method achieves the sorting operation by defining it as projections on permutation polytopes. Specifically, for any given vector w E R", we construct the permutation polytope P(w), which represents the convex hull of all possible permutations of w, i.c.
P(w) == conv((w, a E}).
where E denotes all permutations of [n]. The sorting operation s(0) is defined as the solution to the linear programming problem that maximizes the dot product with (a strictly decreasing vector) on P(0), i.e.,
( arg max (y- p)- yEr(
(4

---

Published as a conference paper at ICLR 2025
Sendbee regice
Den-diitrngiinn Neduaadfel regiee
repton Welknndll
Nunber somptos
Mokker del rogins
cxmppee
Marp-tive ingin
Age
Age
X* octcntration (mmott)
(a) IMDB-WIKI-DE WIKI-DIR
(b) AgeDB- DIR
(c) MCGG--DDIR
Figure 4: Overview of label distributions in the training sets for the IMDB-WIKI-II WIKI-DIR, AgeDB-DIR, and AgeDB-DIR
and ECG-K-DIR datasets. The classification of shot types for follows the definitions provided Yang et al. (202I).
To ensure the differentiability of the sorting operation, a regularization term is introduced, trans- forming the sorting operation into tractable projection problems:
Po (o w) arg min pEP(w)
=1 W(m)
(5)
where V is strongly convex function, ensuring the differentiability of the problem. This approach enables forward propagation with On log n) time complexity and backward propagation with O(n) time complexity.
4 EXPERIMENTS
4.1 BENCHMARKS AND BASELINES
We evaluated method three datasets, focusing tasks of age estimation and potassium concentration prediction. The IMDI-WIKI-DIR dataset Yang et al. (202I). derived from the IMDB- WIKI dataset Rothe al. (2018), consists of 213,553 facial image pairs annotated with age infor- mation. This dataset is partitioned into 191,509 samples for training, 11,022 for validation, and 11,022 for testing. The AgeDB-DID dataset Yang et al. (202T). derived from the AgeDB dataset Moschoglou et al. (2017), comprises 16,488 facial image pairs with age annotations. It is divided into 12,208 samples for training, 2,140 for validation, and 1.4 for testing. The ECG-K-DIR dataset, sourced from the MIMIC-IV dataset]Jhnson et al. (020)) includes 375,745 pairs of single- lead ECG signals paired with potassium concentration values. This dataset divided into 365,549 samples for training, 5,098 for validation, and 5,098 for testing. All these datasets character- ized by imbalanced training sets and balanced validation and test sets. The label distributions of these three datasets shown in Figure 4 Please refer to Appendix A.1 and A.2 for baseline and implementation details.
4.2 EVALUATION METRICS
Following the evaluation metries of Yang et al. (021). we report the results for four shots: all, many, median, and few, where all represents the entire dataset, and many/median/feww correspond to areas of high/medium//lw sample density within the dataset. For the MMDB-WIKI-IR and AgeDB- DIR datasets, we maintain consistency with previous studies, where few/median/many correspond areas with fewer than 20, between 20- 200, and more than 100 samples, respectively. For the ECG-K-DIR dataset, assuming that the maximum number of samples for single label is 1mmxx define areas with more than 0.5 Tmmax between 15-0.5 Tmmax, and fewer than 0.15 mmax samples many/median/feww shots areas, respectively. For each dataset, report the mean absolute error (MAE) and the geometric mean (GM).
4.3
MAIN RESULTS
Table I presents the results of baselines and our method in the few-shot region across three datasets, along with a comparison of these results. For detailed results on each dataset, please refer to Ap- pendix A.3] This table is divided into two sections. The first section displays the results of the

---

Published as a conference paper at ICLR 2025
Table I: Results are presented for the few-shot region on the IMDB-WIKI-DIR, AgeDB-DIR, and C-G-K-DIR datasets. The first section of the table reports the results of baselines and our method, with the best results highlighted bold and red. In the second section, improvements over corre- sponding baselines are reported in bold and green, while decreases in performance are reported in bold and blue.
MAE
GM
IMDB WIKI DIR
AgeDB DIR
BCG-K-DI
IMDB WIKI-DIR
AgeDB: DIR
ECG-K DIR
Vanilla
26,930
12.894
1.771
21,254
9.789
1.578
-DS
22.753
11.279
1.510
12.803
7.846
1.190
+ FDS
24.908
11.161
1.737
14.361
7,361
1,529
Ranksim
25.999
12.569
1.791
19.690
9.495
1.600
ConR
25,408
12.623
1.756
17.022
8.787
1.556
Balanced MSE
23,542
9.613
1.417
12.603
6,248
1.046
Dist Loss (Ours)
22.550
9.122
1.329
14.288
$,453
0.978
Ours vs. Vanilla
+4.380
+.772
0.442
6,966
+43336
+ 0,600
LDS
0.2003
2.157
0.181
1.485
+
+,221
Ours FDS
+23358
+ 2,039
0,408
0,073
+1.98
0.551
Ours vs. Ranksim
+.4449
+ 3.447
0.462
5,402
+40042
0.622
ConR
+22.588
+3.50
2.734
+3.334
Ours Balanced MSE
+0.992
+ 0.499
0,088
1.685
+0.795
0,068
Table 2: Results are presented for the few-shot region on the IMDB- WIKI-DIR, AgeDB-DIR, and ECG-K-DIR datasets. Each section of the table reports the results of baseline and the baseline incorporating our method, with the better results highlighted in bold.
MAE
GM
MMDB-WIKI-DIR
AgeDB-DIR
ECG-K-DIR
IMDB-WIKI-DIR
AgeDB-DIR
ECG-K-DIR
LLDS
22.753
11.279
12,803
1.DS Dist Loss
22.331
10.437
1.325
13.021
7,051
0.957
FDS
24.908
11.161
1.737
14.361
7.361
1.529
FDS Dist Loss
24.112
10.444
1.428
14,929
6.696
1.099
Ranksim
25.999
12.569
1.791
19.690
9.495
1.600
Ranksim Dist Loss
23.772
12.102
1.325
15.422
0.970
25.408
12.623
1.756
ConR Dist Loss
22.700
12.303
1,336
14.713
9.123
0.987
Balanced MSE
23.542
9.613
1.417
12.603
6.248
1.046
Balanced MSE Dist Loss
22.597
9.110
1.357
14.238
5.585
0.996
baselines and method, with the best results highlighted in bold and red. The second section shows the improvement our method over each bascline, with green bold indicating superior per- formance of our method and blue bold indicating otherwise. From the first section of the table, is evident that our method achieves the best results in five out of six metrics across the three datasets, with SOTA performances of 22.550, 9.122, and 1.329 on the ECG-K-DIR datasets, respectively. The second section reveals that our method outperforms in 28 of 30 metrics. Notably, compared Balanced MSE, which also involves fine-tuning the linear layers of pre-trained model and employs data distribution priors, our method demonstrates superior performance in the few-shot region, highlighting the effectiveness of our approach.
AgeDB-DIR, and
Table 2 further illustrates the complementary nature of our method with existing approaches. This table is divided into five sections, each showcasing the results of one baseline and the combined results with our method, with the best results within each section highlighted in bold and black. From this table, it shown that our method achieves better results in 26 out of metrics. Taking the MAE metric
example, incorporating few-shot region across all three datasets, achieving the best results of 22.331, 9.110, and 1.325 on the IMDB- WIKI-DIR, AgeDB-DIR, and ECG-K-DIR datasets, respectively. These experimental results demonstrate key advantage our method, namely its ability to effectively complement existing methods, thereby enhancing model performance in the few-shot region.
method leads to improved performance in the
8

---

Published as a conference paper at ICLR 2025
Table 3: Time consumption (in seconds) of one training epoch for the IMDB-WIKI-DIR, AgeDB-
DIR, and ECG-K-DIR datasets, with batch sizes of 64, 64, and 256, respectively.
IMDB-W WIDB-WIKI-DIR
AgeDB-DIR
ECG-K-DIR
Vanilla
399.8
31.8
94.6
LLDS
401.2
31.4
104.0
FDS
567.5
43.6
155.1
+ Ranksim
512.6
40.2
135.
ConR
1168.7
91.6
192.1
Balanced MSE
152.6
14.2
51.8
Dist Loss (Ours)
154.0
15.1
58.7
Table 4: Ablation study on loss functions measuring sequence difference. L1 represents MAE Loss, L2 represents MSE Loss, INV- denotes the the version of these loss functions that are inverse probability weighted. Results on the few-shot region are reported, with the best results in each section are in bold.
MAE
IMDE-WIKI-DIR
AgeDB-DIR
ECG-K-DIR
IMDB-WIKI-DIR
AgeDB-DIR
ECG-K-DIR
Vanilla
26.930
12.894
1.771
21.254
9.789
1.578
Dist Loss (LNV L.)
23.334
9,802
1,467
15.437
6,298
1.044
Dist Loss (FNV 122)
22.516
9.122
1.329
13.752
5,453
0.978
LDS
22.753
11.279
1,510
12.803
7,846
1.190
Dist Loss (INV
L.)
22.178
9.872
1.413
11.334
6.109
0.984
Dist Loss (INV -
22.331
10.437
1.325
13.021
7.051
0.957
FDS
24.908
11.161
1.737
14.361
7,361
1.529
Dist Loss (LNV
L.)
23.692
1,515
14.399
6.026
1.122
Dist LOss (/NV
24.112
10.444
1,428
14.929
6.696
1.099
Ranksim
25,999
12.569
1,791
19.690
9,495
1.600
Dist Loss (INV 122)
23.894
11.877
1.577
16.036
8.164
1,330
Dist Loss (INV
23.772
12.102
1.325
15.422
8.515
0.970
ConR
25,408
12.623
1.756
17.022
8.787
1.556
Dist Loss (LNV -
23.281
11.948
1,452
15.586
8,605
1. 1.044
Dist Loss (NNV 112))
22.700
12,303
1,336
14.713
9.123
0.987
Balanced MSE Dist Loss (NNV 12))
23,542
9.613
1.417
12.603
6.248
1.046
23.539
9.762
1.474
15.000
6.198
1.051
Dist Loss (NNV
22.597
9.110
1.357
14.238
5,585
0.996
4.4
TIME CONSUMPTION ANALYSIS
Table presents the time required to train each method for one epoch on the IMDB-WIKI-I WIKI-DIR, AgeDB-Die and ECG-K-DiR datasets, with all times reported in seconds. It can be observed that Balanced MSE and Dist Loss have the shortest training times, attributed to their approach of fine-tuning the model's linear layers. The time consumption of LDS and the vanilla model largely consistent, as these methods only weight the loss function without significantly increasing computational load. For methods operating the feature level, including FDS, Ranksim, and ConR, a notable increase in model training time is evident, due to the computational intensity associated with feature-level operations.
4.5
ABLATIONS AND ANALYSIS
4.5.1 DIFFERENT LOSS FUNCTIONS SEQUENCE DIFFERENCE MEASUREMENT
Dist Loss employs the loss function L(-) to measure the difference between two sequences. In this ablation study, we demonstrate the effects of using different functions, including the inverse probability weighted MAE and MSE losses. The experimental results shown in Table 4 with detailed results on each dataset provided Appendix [A.42 The table illustrates that Dist Loss reliably improves model accuracy in the few-shot region.

---

Published as a conference paper at ICLR 2025
Table 5: Ablation study on batch sizes for Dist Loss. Results on the few-shot region are reported.
MAE
GM
IMDB-W WIKI-DIR
AgeDB-DIR
256
22.323
9.013
13.787
5.632
512
22.516
9.122
13.752
5,453
768
22.550
9.148
14.288
5.223
4.5.2 DIFFERENT BATCH SIZES FOR DISTRIBUTION DISTANCE APPROXIMATION
Dist Loss estimates the overall distribution distance between predictions and labels by measuring batch-wise distances during training. This ablation study evaluates the sensitivity of model accuracy to batch size, as detailed in Table [ We examined batch sizes of 256, 512, and 768, adopting 256 as a standard based on prior research Yang et al. (20TT: Gong et al. (2022). The findings
show negligible variations in performance with different batch sizes. This could be attributed the fact that accurately reflecting the distribution information during sampling is more important than
requiring each value the generated pseudo-sequence to be perfectly accurate, which in turn allows for smaller batch size.
4.5.3 DIST LOss SURPASSES EXISTING METHODS IN THE MEDIAN-SHOT REGION
As depicted in the supplementary Tables 6 and 8 within Appendix A.3 Dist Loss delivers SOTA results, excelling not only in few-shot regions but also median-shot regions. In our compari- son with current methods, Dist Loss achieved the lowest MAE and the second-lowest GM on the IMDB-WIKI-DIR and AgeDB-DIE datasets, with scores tively. Similarly, on the ECG-K-DIE dataset, it secured the highest GM and the second- MAE, recording 0.445 and 0.674, respectively. Moreover, our experiments show that integrating Dist Loss with existing methods consistently improved performance in median-shot regions when measured by both MAE and GM,, surpassing the results of using those methods alone on IMDB-WIKI-DIR and AgeDB-DIR datasets. On the ECG-K-DIR dataset, this integration notably increased the GM. In conclusion, these findings validate Dist Loss's efficacy in enhancing model accuracy in both few- shot and median-shot regions.
2.611/7.686 and 7.315/4.563, respec-
CONCLUSION
In this study, we address the significant escalation of prediction errors in few-shot regions, a preva- lent challenge in imbalanced regression. By leveraging distribution priors, we introduce novel loss function, Dist Loss, designed to align the model's prediction distribution with the label dis- tribution throughout the training process. Our extensive experimental evaluation demonstrates that Dist Loss effectively enhances prediction accuracy few-shot regions, achieving state-of-the-ar performance. Furthermore, our results indicate that Dist Loss can be seamlessly integrated with existing methods to further augment their efficacy. We hope our work underscores the critical role of integrating distribution information in tackling deep imbalanced regression tasks.
ACKNOWLEDGEMENT
This work was supported by the Beijing Natural Science Foundations (QY23040); National Natural Science Foundation of China (62102008); Clinical Medicine Plus X Young Schol- Project of Peking University, the Fundamental Research Funds for the Central Universities (PKU2024LCXQQ030) PKU-OPPO Fund (B0202301);
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking. In International Conference on Machine Learning, pp. 950-959 PMLR, 2020.
Paula Branco, Luís Torgo, and Rita P Ribeiro. Smogn: pre-processing approach for imbalanced regression. In First international workshop on learning with imbalanced domains: Theory and applications, pp. 36-50. PMLR, 2017.
Paula Branco, Luis Torgo, and Rita P. Ribeiro. Rebagg: Resampled bagging for imbalanced gression. In Luís Torgo, Stan Matwin, Nathalie Japkowicz, Bartosz Krawezyk, Nuno Moniz, and Paula Branco (eds.), Proceedings of the Second International Workshop on Learning with Imbalanced Domains: Theory and Applications, volume 94 of Proceedings of Machine Learning Research, pp. 67-81. PMLR, 10 Sep 2018.
Mateusz Buda, Atsuto Maki, and Maciej A Mazurowski. A systematic study of the class imbalance problem in convolutional neural networks. Neural networks, 106:249-25 259, 2018.
Nitesh V Chawla, Kevin W Bowyer, Lawrence Hall, and W Philip Kegelmeyer, Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
Lia Crotti, Katja E Odening, and Michael C Sanguinetti. Heritable arrhythmias associated with abnormal function of cardiac potassium channels. Cardiovascular research, 116(9):1542-1556, 2020.
Lin Feng. Huibing Wang, Bo Jin, Haohao Li, Mingliang Xue, and Le Wang. Learning distance metric by balancing kl-divergence for imbalanced datasets. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 49(12):2384-2333 2018
João Pedro Ferreira, Javed Butler, Patrick Rossignol, Bertram Pitr, Stefan D Anker, Mikhail Kosi- borod, Lars H Lund, George Bakris, Matthew R Weir, and Faiez Zannad. Abnormalities of potassium in heart failure: Jacc state-of-the-art review. Journal of the American College of Car- diology, 75(22))2836-2888 2020.
Conner D Galloway, Alexander V Valys, Jacqueline B Shreibati, Daniel I. Treiman, Frank I. Pet- terson, Vivek P Gundotra, David E Albert, Zachi I Attia, Rickey E Carter, Samuel J Asirvatham, et al. Development and validation of a deep-lcarning model to screen for hyperkalemia from the electrocardiogram. JAMA cardiology, 4(5):428 436, 2019.
Yu Gong, Greg Mori, and Fred Tung. RankSim: Ranking similarity regularization for deep imbal- anced regression. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 7634-7649. PMLR, 17-23 Jul 2022.
David M Harmon, Chris K Heinrich, John J Dillon, Rickey E Carter, Kianoush B Kashani, Zachi I Attia, Paul A Friedman, and Jacob Jentzer. Mortality risk stratification utilizing artificial in- telligence clectrocardiogram for hyperkalemia in cardiac intensive care unit patients. JACC: Ad- vances, pp. 101 169, 2024.
Shenda Hong, Yanbo Xu, Alind Khare, Satria Priambada, Kevin O. Maher, Alaa Aljiffry, Jimeng Sun, and Alexey Tumanov. HOLMES: health online model ensemble serving for deep learning models in intensive care units. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (eds.), KDD '20: The 26th ACM SIGKDD Conference Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pp. 1614-1624. ACM, 2020. doi: 10.1 145/3394486. 3403212.
Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Steven Horng, Leo Anthony Celi, and Roger Mark. Mimic-iy. PhysioNet. Available online at: htts://physionet. orgfcontent/mmmicciv..c Of accessed August 23, 202/). I). pp. 49-55, 2020.
Mahsa Keramati, Lili Meng, and R. David Evans. Conr: Contrastive regularizer for deep imbalanced regression. In The Twelfth International Conference on Learning Representations. ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.

---

Published as a conference paper at ICLR 2025
Michael Kim, Christina Valerio, and Glynnis K Knobloch. Potassium disorders: hypokalemia and hyperkalemia. American Family Physician, 107(1):59-70A, 2023.
Gary King and Langche Zeng. Logistic regression in rare events data. Political analysis, 9(2): 137-163, 2001.
T Focal loss for dense object detection. arXiv preprint arXiv: 2017.
Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella Yu. Large- scale long-tailed recognition in open world. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 2537-2544 2019.
Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafciriou. Agedb: the first manually collected, in-the-wild age database. In Proceed- ings of the IEEE Conference Computer Vision and Pattern Recognition Workshop, volume 2, pp. 5, 2017.
Emanuel Parzen. On estimation of probability density function and mode. The annals of mathe- matical statistics, 165--166, 1962.
Jiawei Ren, Mingyuan Zhang, Cunjun Yu, and Ziwei Liu. Balanced mse for imbalanced visual regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recog- nition, pp. 7926-7935, 2022.
Rasmus Rothe, Radu Timofte, and Luc Van Gool. Deep expectation of real and apparent age from single image without facial landmarks. International Journal of Computer Vision, 126(2-4): 144-157, 2018.
Shiori Sagawa, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generaliza- tion. arXiv preprint arXiv:1911.0873/ 2019.
Chris Seiffert, Taghi M Khoshgoftaar, Jason Van Hulse, and Amri Napolitano. Rusboost: A hybrid approach alleviating class imbalance. IEEE transactions on systems, man, and cybernetic--par A: systems and humans, 40(1):185- 2009.
Michael Steininger, Konstantin Kobs, Padraig Davidson, Anna Krause, and Andreas Hotho. Density-based weighting for imbalanced regression. Machine Learning. 110:2187-2211, 2021.
Junjiao Tian, Yen-Cheng Liu, Nathaniel Glaser, Yen- Yen-Chang Hsu, and Zsolt Kira. Posterior re- calibration for imbalanced datasets. Advances in neural information processing systems, 33: 8101-811 13, 2020.
Luis Torgo, Rita P Ribeiro, Bernhard Pfahringer, and Paula Branco. Smote for regression. In Portuguese conference on artificial intelligence, pp. 378-389, Springer, 2013.
Ziyan Wang and Hao Wang. Variational imbalanced regression: Fair uncertainty quantification via probabilistic smoothing. Advances Neural Information Processing Systems, 36, 2024.
Yuzhe Yang, Kaiwen Zha, Yingcong Chen. Hao Wang, and Dina Katabi. Delving into deep imbal- anced regression. Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 11842-11851. PMLR, 18-24 Jul 2021.
Ming Zheng, Tong Li, Rui Zhu, Yahui Tang, Mingjing Tang, Leilei Lin, and Zifei Ma. Condi- tional wasserstein generative adversarial network- -rradiett penalty-based approach to alleviating imbalanced data classification. Information Sciences, 112:1009-10 1023, 2020.
12

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 BASELINES
To ensure fair comparison, we followed the experimental setup of Yang et al. (2021) on the IMDB- WIKI-DIR and AgeDB-DIE datasets, i.e. using ResNet-50 as the network architecture and train- ing for 90 epochs. For the ECG-K-DIR dataset,
employed the ResNet variant NetlD Hong ct al. (2020) as the network architecture. Given that previous work Yang et al. (202T); : Ren et al. (2022 : Gong et al. ( Keramati et al. (2024) has demonstrated superior performance over loss reweighting and RRT in deep imbalanced regression tasks, we do not include these methods as baselines this paper. Instead, we focused on widely recognized approaches in the field: LDS,
FDS Yang et al. 202T). Ranksim Gong et al. (2022). ConR Keramati et al. (2024), and Balanced MSE Ren et (2022). LDS and FDS encourage local similarities in label and feature space, while Ranksim and ConR leverage contrastive learning to translate label similarities into the feature space. Balanced MSE, based on label distribution priors, restores a balanced distribution from an imbal- anced dataset. Our experimental findings indicate that not only does our method achieve SOTA performance in few-shot regions, but it also enhances existing methods, offering a complementary strategy to boost their efficacy.
MSERen
A.2 IMPLEMENTATION DETAILS
We trained all models on the IMDB-WIKI-DIR and AgeDB-DIR datasets using a single NVIDIA GeForce RTX 3090 GPU and on the ECG-K-DIR dataset using single NVIDIA GeForce RTX 4090 GPU. To ensure a fair comparison. we followed the training, validation, and test set divisions from Yang al. (22T)) for the MIDB-WIKI-DIR and AgeDB datasets. During training with Dist Loss, we used the same strategy as Balanced MSF, fine-tuning the linear layer based on pre-trained model (vanilla model) parameters. This approach integrates our method with existing methods, using their model parameters the starting point for fine-tuning. Additionally, probability weighted MSE to measure sequence difference in Dist Loss for all datasets, setting the distribution loss component weight to 1.
used inverse
A.2.1 HIDB-WIKI-DIR
On the ing, the training epochs were set to 90, with an initial learning rate of 0.001, which was reduced to 1/10 of its value at the 60th and 80th epochs. We employed the Adam optimizer with a momentum of 0.9 and a weight decay of 0.0001. For our method and Balanced MSE, we used batch size followed the experimental setups from their original papers. It
dataset, we selected ResNet-50 as the network architecture. During train-
of 512. For the other baselines, should be noted that the original training epochs for ConR was 120, which we adjusted to 90 in our experiments ensure fair comparison.
A.2.2 AGEDB-DIR
On the AgeDB dataset, we employed ResNet-50 architecture for our model. The training consisted of 90 epochs with an initial learning rate of 0.001, which was reduced to 1/10 of its original value at the 60th and 80th epochs. We utilized the Adam optimizer with momentum of 0.9 and weight decay of 0.0001. For our method and Balanced MSE, we used a batch size of 512. For the other baselines, we followed the experimental configurations outlined in their respective original papers. To ensure fair comparison, we also set the training epochs for Ranksim and ConR to 90.
A.2.3 ECG-K-DIR
On the ECG-K-DIR dataset, we utilized the ResNet variant, NetlD Hong et al. 2020)) as our net- work architecture. The training was set for 10 epochs with initial learning rate of 0.001, which was reduced to 1/10 its initial value at the 5th and 8th epochs. We employed the Adam optimizer with momentum of 0.9 and weight decay of 0.00001. A batch size of 5 512 used for all meth- ods. Additionally, for ConR, we constructed positive and negative sample pairs by adding Gaussian noise.
13

---

Published as a conference paper at ICLR 2025
Table Comprehensive results on the WADB-WIKI-DIR dataset are presented. The table highlights the best results in each section using bold font. Additionally, the best result in each column is indicated in bold and red.
MAE
GM
Many
Median
Many
Median
Vanilla
8.143
7.260
15.758
26.930
4.642
4.211
21.254
Dist Loss
8,028
7.461
12,614
22.516
4.593
4.335
7.686
13.752
-LDS
8.036
7,445
12.869
22.753
4.570
4.322
7.528
12.803
Dist Lass
8.017
7.479
12.304
22.331
4.593
4.369
7.078
13.021
+FDS
7,954
7.272
13.523
24,908
4.499
4.192
8.633
14,361
Dist Loss
8.712
8.163
12.979
24.112
5.222
4.995
7.575
14,929
Ranksim
7.764
6.956
14,606
25.999
4.371
3.996
9.964
19,690
Dist Loss
7.721
7.129
12,401
23.772
4,422
4,183
7.091
15.422
ConR
7.842
7.033
14.772
25.408
4.329
3,951
10,250
17.022
Dist Loss
7.957
7.355
12.906
22.700
4.529
4.244
8.131
14.713
MSE
8.033
7.441
12.768
23.542
12.603
Dist Loss
8.075
7.511
12.625
22.597
4.616
4.354
7.754
14,238
Table 7: Comprehensive results on the AgeDB-DIR dataset are presented. The table highlights the best results in each section using bold font. Additionally, the best result in each column is indicated in bold and red.
MAE
GM
Many
Median
Few
All
Many
Median
Few
Vanilla
7.506
6,558
8.794
12.894
4.798
4.176
5.957
9.789
Loss
7.637
7.574
7.315
9.122
4.756
4.745
4.563
5.453
+LDS
7.783
7,070
8.957
11,279
5.088
4.599
6.142
7,846
Dist Loss
7,810
7.341
8,464
10.437
5,043
4.752
5,474
7.051
FDS
7,818
7.103
9.051
11.161
4.961
4.487
6.064
7.361
Dist Loss
7.799
7.351
8.374
10.444
4.863
4.615
5.181
6.696
Ranksim
7.272
6.363
8.458
12.569
4.617
3,939
6.120
9.495
Dist Loss
7.234
6.506
7.960
12.102
4.629
4.097
5.637
8.515
ConR
7.322
6.429
8.456
12.623
4.646
4,052
5.890
8.787
Dist Loss
7.383
6.572
8,373
12.303
4.657
4.112
5,591
9.123
Balanced MSE
7.663
7.540
7.353
9.613
4.658
4,558
4.511
6.248
Dist Loss
7.633
7.578
7.288
9.110
4.718
4.698
4.505
5,585
Table 8: Comprehensive results on the ECG-K-DIR dataset are presented. The table highlights the best results in each section using bold font. Additionally, the best result in each column is indicated in bold and red.
MAE
GM
Many
Median
Few
All
Many
Median
Few
Vanilla
1.235
0.274
0.685
1.771
0.835
0.193
0.622
1.578
Dist Loss
1.044
0.606
0.674
1.329
0.692
0.403
0.445
0.978
+LDDS
1.092
0.368
0.638
1.510
0.708
0.236
0.500
1.190
Dist Loss
1.031
0.557
0.671
1.325
0.671
0.363
0.455
0.957
+ +FDS
1.223
0.317
0.681
1.737
0.828
0.201
0.588
1.529
Dist Loss
1.055
0.557
0.688
1.428
0.744
0.375
0.490
1.099
Ranksim
1.249
0.275
0.696
1.791
0.841
0.190
0.629
1.600
Dist Loss
1.040
0.587
0,683
1.325
0.692
0.381
0.487
0.970
ConR
1.227
0.277
0.690
1.756
0.824
0.189
0.620
1.556
Dist Loss
1.045
0.581
0.684
11.336
0.696
0.376
0.480
0.987
Balanced MSE
1.106
0.606
0.727
1.417
0.722
0.383
0.475
1.046
Dist Loss
0.553
0.658
1.357
0.685
0.358
0.996
14

---

Published as a conference paper at ICLR 2025
Table Ablation study examining the impact of batch size on model performance across the IMDB- WIKI-DIR and AgeDB-DIR datasets.
MAE
GM
Dataset
Batch size
All
Many
Median
Few
All
Many
Median
Few
256
8.072
7.514
12.591
22.323
4.603
4.340
7.808
13.787
IMDE-WIKI-DIR
512
8.028
7.461
12.614
22.516
4.593
4.335
7.686
13.752
768
7.989
7.413
12.663
22.550
4.572
4.308
7.763
14.288
256
7.668
7.638
7.281
9.013
4.741
4.768
4.370
5.632
AgeDB-DIR
7.637
7.574
7.315
9.122
4.756
4.745
4.563
5.453
768
7.545
7.607
7.260
9.148
4.723
4.746
4,483
5.223
A.3 COMPREHIENSIEE EXPERIMENTAL RESULTS
Tables and present a comprehensive overview of our experimental results on the IMDB- WIKI- DIR, AgeDB -DRR, and ECG-K-DIR datasets. The results indicate that method achieves improvements in model performance on median-shot and few-shot regions without compromising overall error rates. This further demonstrates the effectiveness of method in sparse data regions.
A.4 ABLATIONS AND ANALYSIS
A.4.1 DIFFERENT BATCH SIZES FOR DISTRIBUTION DISTANCE APPROXIMATION
Table 9 illustraaee the impact of varying batch sizes on the final performance across IMDB-WIKE DIR and AgeDB-Di D-DDIIDRR datasets. The results indicate that there is no significant difference perfor- mance among different batch sizes. This observation suggests that the generation of pseudo-labels primarily requires an approximation of the distribution information, rather than the precise accuracy of every individual label value.
A.4.2 DIFFERENT LOSS FUNCTIONS FOR SEQUENCE DIFFERENCE MEASUREMENT.
Tables TO 11 and 12 present the comprehensive results using different loss functions on IMDB- WIKIDDRR AgeDB-DIR, and ECG-K-DIR, respectively. It is evident that existing methods, when augmented with Dist Loss, demonstrate superior performance on samples within few-shot regions.
A.4.3 PERFORMANCE OF DIST LOSS ACROSS DIFFERENT IMBALANCED RATIOS
We validated the effectiveness of Dist Loss by varying the imbalance ratios of the ECG-K-DIR dataset. The data distribution diagrams are shown in Figure 5] and the corresponding results in the few-shot regions are presented in Table Across eight datasets with different imbalance ratios, our method achieved the best performance in six cases and the second-best performance in the remaining two. These results collectively demonstrate the robustness of our approach across varying levels of data imbalance.
A.5 PERFORMANCE OF DIST LOSS ON THE GM METRIC
We observed that on the IMDB WIKI-DIR dataset, the performance of Dist Loss in the few-shot region, as measured by the GM metric, is inferior to that of Balanced MSE. To provide a more intuitive analysis of this phenomenon, we plotted the sorted error distribution curves for both Dist Loss and Balanced MSE in the few-shot region, shown in Figure 6 Specifically, for each method, the error values were first sorted in ascending order. The x-axis represents the rank of these sorted errors, while the y-axis denotes the corresponding error magnitudes. This visualization facilitates direct comparison of the error distributions between the two methods.
From the plot, it is evident that Dist Loss generally exhibits superior performance compared to Bal- anced MSE, as indicated by its curve lying below or aligning with the curve for Balanced MSE. However, localized discrepancy is observed around the x-axis values of approximately 5 and 30,
15

---

Published as a conference paper at ICLR 2025
Table 10: An ablation study loss functions on the IMDB-WIKI-DIR dataset. L1 represents MAE Loss, L2 represents MSE Loss, INV- denotes the probability-Men inversely weighted version of these loss functions. Results the few-shot region reported.
MAE
GM
All
Many
Median
Few
All
Many
Median
Few
Vanilla
8.143
7.260
15.758
26.930
4.642
4.211
11.522
21.254
+
Dist Loss (INV - L1)
7.807
7.210
12.608
23.334
4.458
4.189
7.717
15.437
+ Dist Loss (INV - L2)
8.028
7.461
12.614
22.516
4.593
4,335
7.686
13.752
+ LDS
8.036
7.445
12.869
22.753
4.570
4.322
7.528
12.803
+ Dist Loss (JNV I1)
8.054
7.545
12.030
22.178
4.678
4,486
6.717
11.334
+ Dist Loss (JNV - 12))
8.017
7.479
12.304
22.331
4.593
4,369
7.078
13.021
+ FDS
7.954
7.272
13.523
24,908
4,499
4.192
8.633
14,361
Dist Loss (LNV L1)
7.986
7.413
12.486
23.692
4.530
4.315
6.793
14.399
+ Dist (NNV L2)
8.712
8.163
12.979
24.112
5.222
4.995
7.575
14.929
+ Ranksim
7.764
6.956
14.606
25,999
4.371
3,996
9.964
19.690
+ Dist Loss (INV - -1)
7.501
6,888
12.372
23.894
4.150
3.910
7.035
16.036
+ Dist Loss (/NV 1.2)
7.721
7.129
12.401
23.772
4.422
4.183
7.091
15.422
+ ConR
7.842
7.033
14.772
25.408
4,329
3,951
10.25
17.022
+ Dist Loss (INV - L
7.538
6.924
12.499
23.281
4.169
3.893
7.643
15.586
+ Dist Loss (LNV - -2)
7.957
7.355
12.906
22.700
4.529
4.244
8.131
14.713
+ Balanced MSE
8.033
7.441
12.768
23.542
4.716
4.450
8.035
12.603
+ Dist (LNV L1)
7.788
7.175
12.732
23.539
4.460
4.182
7,900
15.000
+ Dist Loss (/NV L12)
8.075
7.511
12.625
22.597
4.616
4.354
7.754
14.238
Table ablation study on loss functions on the AgeDB-DIR dataset. L,1 represents MAE Loss, T2 represents MSE Loss, INV- denotes the probability-han -based inversely weighted version of these loss functions. Results on the few-shot region are reported.
MAE
GM
All
Many
Median
Few
All
Many
Median
Few
Vanilla
7.506
6.558
8.794
12.894
4.798
4.176
5.957
9.789
+
Dist Loss (INV L,)
7.552
7.282
7,660
9,802
4.700
4.528
4.800
6.298
+ Dist Loss (INV Lz))
7.637
7.574
7.315
9.122
4.756
4.745
4,563
5.453
+
LDS
7.783
7,070
8.957
11.279
5.088
4.599
6.142
7.846
+
Dist Loss (INV L,)
7.885
7.635
8.020
9.872
5.082
4.964
5.151
6.109
+ Dist Loss (INV Lz)
7.810
7.341
8.464
10.437
5.043
4.752
5.474
7.051
+ FDS
7.818
7.103
9,051
11.161
4.961
4.487
6.064
7.361
Dist Loss (INV L1)
7.911
7.665
8,010
9.969
5,010
4.933
4.941
6.026
Dist Loss (/NV [oz)
7.799
7.351
8.374
10.444
4.863
4.615
5.181
6.696
+ Ranksim
7.272
6.363
8.458
12.569
4.617
3.939
6.120
9.495
+ Dist Loss (INV - -L))
7.239
6.605
7.727
11.877
4.635
4.194
5.311
8.164
Dist Loss (JNV Lz)
7.234
6.506
7.960
12.102
4.629
4.097
5.637
8.515
+ ConR
7.322
6.429
8.456
12.623
4.646
4.052
5,890
8.787
+ Dist Loss (NNV L1)
7.398
6.683
8, 8,194
11.948
4.709
4.208
5,560
8.605
+ Dist Loss (/NV Ioz)
7.383
6.572
8.373
12.303
4.657
4.112
5.591
9.123
+ Balanced MSE
7.663
7.540
7.353
9.613
4.658
4.558
4,511
6,248
+ Dist Loss (/NV - L,)
7.537
7,300
7.540
9.762
4.751
4.623
4.737
6.198
+ Dist Loss (NNV [z))
7.633
7.578
7.288
9.110
4.718
4.698
4.505
5.585
16

---

Published as a conference paper at ICLR 2025
Table 12: ablation study on loss functions on the ECG-K-DIR dataset. 7, represents MAE Loss, T2 represents MSE Loss, INV- denotes the probability-basee inversely weighted version of these loss functions. Results on the few-shot region are reported.
MAE
GM
All
Many
Median
Few
All
Many
Median
Few
Vanilla
1.235
0.274
0.685
1.771
0.835
0.193
0.622
1.578
+
Dist Loss (LNV- L1)
1.088
0.458
0.648
1.467
0.680
0,300
0.463
1.044
+ Dist Loss (INV T22)
1.044
0.606
0.674
1.329
0.692
0.403
0.445
0.978
+ LDS
1.092
0.368
0.638
1.510
0.708
0.236
0.500
1.190
+
Dist Loss (/NV [1)
1.059
0.463
0.655
1.413
0.647
0.291
0.445
0.984
+
Dist Loss (TNV - T2)
1.031
0.557
0.671
1.325
0.671
0,363
0,455
0.957
+ FDS
1.223
0.317
0.681
1.737
0.828
0.201
0.588
1.529
+ Dist Loss (LNV Ln)
1.133
0.497
0.692
1,515
0.725
0.324
0.477
1.122
+ Dist Loss (NNV L2)
1.095
0.557
0.688
1.428
0.744
0.375
0,490
1.099
+
Ranksim
1.249
0.275
0.696
1.791
0.818
0.215
0.566
1.510
+ Dist Loss (UNV L1)
1.139
0.394
0.649
1.577
0.712
0.317
0.472
1.099
+
Dist Loss (NNV T22)
1.040
0.587
0.683
1.325
0.723
0.400
0.479
1.031
+ ConR
1.227
0.277
0.69
1.756
0.841
0.190
0.629
1.600
+ Dist Loss (LNV Ln)
1.085
0.484
0.651
1,452
0.780
0.272
0,503
1.330
+
Dist Loss (NNV L2)
1.045
0.581
0.684
1,336
0.692
0,381
0,486
0.970
+
Balanced MSE
1.106
0.606
0.727
1.417
0.824
0.189
0.620
1.556
+
Dist Loss (LNV L1)
1.092
0.457
0.65
1.474
0.678
0.307
0.443
1.044
+
Dist Loss (NNV - L2)
1.046
0.553
0.658
1.357
0.696
0.376
0.480
0.987
Table 13: Performance of Dist Loss the few-shot regions across eight datasets derived from the ECG-K-DIR dataset with varying imbalance ratios, with the best results highlighted bold.
MAE
Methods
Dataset ()
Dataset
Dataset
Dataset
Dataset 4
Dataset 5
Dataset 6
Dataset
Vanilla
2.701
2.676
2.658
1.979
2.679
2.647
2.624
1.888
LDS
2.684
2.703
2.642
1.962
2.672
2.507
2.644
1.901
FDS
1.865
2.368
2.191
1.790
2.223
2.625
1.908
1.665
Ranksim
2.470
2.327
2.273
1.831
2.314
2.192
2.258
1.725
ConR
2.461
2.343
2,308
1.828
2.193
2.274
2.255
1.742
Balanced MSH
1.997
1.984
1.981
1.831
1.906
1.863
1.815
1.708
Dist Loss
1.955
1.873
1.963
1.822
1.852
1.803
1.730
1.638
Derndbution cracstriike Catasi
Dasstaation cacestrviie DDtaai
Ratsbaatio cocentryian
Rutatbatte occurratio (Dutadd
Chintruios craeetrilke Chlest
Ubintedtive conettrite
Datast preestortan (l.
Rotattalae ovw.maation
www. (2204)1)
Figure 5: Data distribution diagrams for the eight datasets derived from the ECG-K-DIR dataset with varying imbalance ratios.
17

---

Published as a conference paper at ICLR 2025
Sorted Error Distribution Curves
Balanced MSE
70
Dist Loss
60
50
Error Value
40
30
20
10
0
0
20
40
60
80
100
Ranked Sample Index
Figure 6: Sorted error distribution curves for Dist Loss and Balanced MSE in the few-shot region on the IMDB-WIKI-DIR dataset.
where the errors of Dist Loss slightly exceed those of Balanced MSE. We hypothesize that this lo- calized discrepancy may contribute to the overall inferior performance of Dist Loss in terms of the GM metric, owing to the cumulative multiplicative effect intrinsic its caleulation. Unlike MAE, which averages еrrоr values, the GM metric calculates the geometric mean by multiplying error val- ues together. This process significantly amplifies the impact of small but frequent errors. For exam- plc, consider two eтrоr distributions: (40, 10.1, 10.1, 10.1, 1, 10.1, 10.1) and (42, 10, 10, 10, 10, 10). While the former achieves a lower MAE than the latter, its GM metric value is higher due to the cumulative effect, as 40 x 1.01 > 42 x 10a. This example underscores how the GM metric can magnify the influence of small deviations when they occur frequently.
In conclusion, the sorted error distribution curves demonstrate that Dist Loss consistently achieves better or comparable performance relative to Balanced MSE, except for minor localized discrepan- cies. These results suggest that the unique characteristies of the GM metric are the primary factors contributing to the observed differences in performance between the two methods.
A.5.1 PERFORMANCE VISUALIZATION OF DIST LOSS
Figure illustrates the performance comparison of three methods-vanille model, LDS, and Dist Loss on the HEDB-WIKK-DIR. AgeDB-DIR, and ECG-K-DIR datasets. As observed, Dist Loss achieves predictions that are systematically closer to the diagonal line (y = x, where the predicted values align with the ground truth values) across all datasets, indicating improved accuracy. The task on the ECG-K-DIR dataset, which involves estimating blood potassium concentration from single- lead ECG signals, is particularly challenging due to the inherently limited information provided by single-lead ECGs. This limitation exacerbates the regression dilution phenomenon, leading to larger deviations from the diagonal across all methods. Despite this difficulty, Dist Loss demonstrates superior performance, underscoring its robustness and effectiveness in addressing regression tasks with imbalanced and noisy data.
18

---

Published as a conference paper at ICLR 2025
Vanilla Model
LDS
Dist Loss
100
100
Predictent Values
80
Identical Line
Prodicted Vallues
Identical Line
80
Identical Line
60
60
40
40
Label Values
Label Values
Label Values
(a) Scatter plots on the IMDB-W WIKI-DIR dataset: vanilla (left), LDS (middle), Dist Loss (right).
Vanilla Model
LDS
Dist Loss
100
100
100
Dilettical Line
Edentical Line
Iilentical Line
Prodicted Valums
80
Prodicted Vallans
80
60
60
20
40
100
20
60
100
Label Values
Label Values
Label Values
(b) Scatter plots the AgeDB-DIR dataset: vanilla (left), LDS (middle), Dist Loss (right).
Vanilla Model
Dist
Identioal Line
Identical Line
Identical Line
Predicted Values
Prodicted Values
Prodicted Values
2
3
5
2
Label Values
Label Values
Label Values
(c) Scatter plots on the ECG-K- SCG-K-DIR dataset: vanilla (left), LDS (middle), Dist (right).
Figure 7: Performance visualization of vanilla, LDS, and Dist Loss
19


Reviews:
[{"review_id": "Fw94vc9tnr", "reviewer": "Reviewer_C9ay", "summary": "This paper tackles the issue of imbalanced regression, which often results in poor performance in regions with few samples, by proposing a new loss function termed Dist Loss. Dist Loss is designed to reduce the distribution gap between model predictions and actual labels, thereby promoting closer alignment with the target distribution, particularly in sparse data regions. The approach has shown enhanced performance in few-shot scenarios across several datasets, such as IMDB-WIKI-DIR, AgeDB-DIR, and ECG-Ka-DIR, and complements existing techniques well. Through extensive experimentation, Dist Loss demonstrates state-of-the-art accuracy for rare instances, highlighting its relevance for critical fields like healthcare.", "questions": "1. In Section 2.2, the authors claim that imbalanced image classification methods cannot be well adapted for imbalanced regression. However, with the chosen discretized distribution using fixed bin sizes, wouldn't classification-based methods be applicable? Could the authors elaborate on why classification-based methods might not work well in this setting and clarify the limitations or challenges?\n\n2. The proposed loss function does not show a strong performance improvement over Balanced MSE, as shown in Table I, and it even underperforms Balanced MSE on the IMDB/WIKI/DIR dataset using the GM metric. Can the authors explain why the proposed loss function struggles to outperform Balanced MSE in these cases and discuss any specific scenarios where it demonstrates advantages?\n\n3.Could the authors compare their method with other existing works, such as Wang & Wang (2024) and Keramati et al. (2023)? These works evaluate the task across different settings (all, many, medium, few), which seems like a more convincing framework to assess generalizability. Why did the authors choose not to include similar shot-based evaluations? Would this additional analysis provide deeper insights into the model’s robustness?\n\n\n4.The performance of ConR reported in the paper differs from what is reported in the original ConR paper for the AgeDB DIR dataset. Did the authors follow the same experimental settings as in these related works? If there were any differences in the setup, could they clarify these and explain how they might impact the results?\n\n5.Table 3 shows that Balanced MSE is more efficient in terms of training time compared to the proposed Dist Loss. Could the authors discuss the trade-offs between training time efficiency and performance gain? Given the additional time cost, what are the practical advantages of using Dist Loss?\nPerformance on AgeDB DIR Dataset in Few-Shot Settings:\n\n6. According to Keramati et al. (2023), ConR combined with LDS achieved better results on the AgeDB DIR dataset in the few-shot setting (MAE of 9.62 and 6.87). Why does the proposed method seem to underperform compared to this? Did the authors consider integrating ConR or LDS in their approach to enhance few-shot performance, and if not, what were the reasons?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces Dist Loss, a new loss function specifically designed to tackle the problem of imbalanced regression, improving model performance in low-sample regions.\n\n2. Dist Loss is easy to incorporate with current techniques and enhances their performance in few-shot areas, making it a practical addition to existing approaches.\n\n3. The proposed method is evaluated on different datasets.", "weaknesses": "1. According to the problem setting, the distribution has been discretized using fixed bin size, which can also be adapted into classification setting if one have a look at Figure 1. However the authors claimed in section 2.2 that the methods developed for imbalanced image classification can not be well adapted into imbalanced regression task. From my point of view, if bin is chosen then classification based method can also be used in regression task. More explanation should be provided by the authors.\n\n\n2. The performance gain of the proposed loss function is not promising compared with Balanced MSE in Table I. It even underperforms BalancedMSE on IMDB/WIKI/DIR dataset using GM metric.\n\n3. The authors are also suggested to compare with some other existing related works, e.g., a.\n\nIn a and b, the task is evaluated in four different settings, which are all, many, medium, few, which is also mentioned by the authors. However it seems in the provided Tables there are only results for few shot regions. I find the original different shot settings more convincing since it can showcase the generalizability of the proposed loss function among different shot settings.\n\nAnother question, why the reported performance of ConR is different compared with the performance reported by their paper for AgeDB DIR dataset? Did the authors follow the same experimental setting as introduced in ConR and other existing works?\n\na. Wang, Z., & Wang, H. (2024). Variational imbalanced regression: Fair uncertainty quantification via probabilistic smoothing. Advances in Neural Information Processing Systems, 36.\n\nb. Keramati, M., Meng, L., & Evans, R. D. (2023). Conr: Contrastive regularizer for deep imbalanced regression. ICLR 2024.\n\n4. In Table 3, it seems that balanced MSE is better than the proposed Dist Loss regarding the training time consumption. The benefits from the training efficiency pespective is now well highlighted.\n\n\n5. I have some concerns with Table 4. From the paper of ConR as mentioned in b, LDS with ConR can achieve better performance on AgeDB DIR dataset, with 9.62 for MAE and 6.87 for AgeDB DIR when we considering the few shot setting. The authors are suggested to compare with all other existing works in Table 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aD67vX66lW", "reviewer": "Reviewer_7CfX", "summary": "The paper addresses the challenge of imbalanced data distributions in regression tasks, especially the difficulty deep learning models face in accurately predicting few-shot settings. The authors introduce a new loss function, called Dist Loss, designed to align the distribution of model predictions with the label distribution using kernel density estimation and pseudo-predictions. The authors also contribute a differentiable relaxation for their loss function to make the model end-to-end training. The paper conducts experiments on the regression of three datasets, compares with five related works, and has demonstrated significant improvements.", "questions": "The reviewer has the following questions:\n\n\n(a) can you explain how the KDE is used in your formulation? What is input data? Do we need to use it in the testing part or only in the training steps?\n\n(b) can you explain why the pseudo-label is important to overcome imbalances in regression?\n\n(c) The ablation study in Table 4 is difficult to capture. In particular, the authors mentioned, *\"We demonstrate the effects of using different functions, considering the probability-based inversely weighed MAE and MSE losses.\"* There are two questions here. First, what is *probability-based inversely* (INV) ? and why $INV - L_1$ or $ INV - L_2$ shows benefit of your proposed one?\n\n(d) because dist_loss works with discrete values in sorted orders, how can you make the training process stable and avoid exploding gradients given large values from the loss function?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper has the following strengths:\n\n(i) The problem of imbalanced regression tasks is important to study but less well investigated in related works. The authors contributed a new loss function to train the model and showed good results compared to other baselines in two aspects: using the new loss function integrated into the baseline and combining the existing training loss with their proposed one. The running time is as fast as the mean squared error and significantly faster than other baselines. \n\n(ii) The idea of using a pseudo-label to define the distance between prediction and label distribution is interesting. Also, the authors applied different sorting techniques to make their Dist Loss ($L(S_P, S_L)$) trainable.", "weaknesses": "While the paper achieved good results, its presentation lacks details and information to help the reviewer understand core ideas and how the method efficiently addresses imbalanced problems in regression tasks.\n\n(i) In Figure 2, what is the role of **\"Labels\"** and **\"Predictions\"** terms (inside **Dist loss calculation**) while the Loss function already received **Pseudo-labels** and **Sorted Prediction**?\n\n(ii) In Section 3.2.1, authors jumped into detail about how they compute pseudo-labels and pseudo-predictions but they have not addressed two key questions:\n+ Why do we need to compute \"pseudo-here\"? How is it important to deal with the imbalanced dataset?\n+ Section 3.2.1 is based on the Empirical label distribution (KDE), but no details about KDE are discussed (either background and how it is used), which makes the readers unable to follow the details later. For e.g., is the KDE step applied at a batch size level or at the whole training dataset?\n\nAnother unclear problem is the standard regression formulation, which transforms continuous label space Y into discretized B bins (**Section 3.1**). How is the standard learning scheme trained with these discretized labels? \n\n(iii) The descriptions in Equation (1) and (2) and the definition of $N_{L'}$ are also hard to imagine without a figure to illustrate core ideas beyond.\n\nOverall, the paper's methodology section is not well presented, with information presented sporadically, and does not address core concerns about imbalanced settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EMxXlz0Orh", "reviewer": "Reviewer_ZuRS", "summary": "The paper proposes a new approach, called Dist Loss, for imbalanced deep regression. Specifically, the method generates pseudo-labels taking into consideration label distribution information, and at the same time generates pseudo-predictions. The paper is novel and well-written, but has a number of issues and can be strengthened.", "questions": "In Figure 2, what is the exact form of the loss function L(.)? This is not clear from both the figure and the main text. Specifically, in Figure 2, for the two sequences [1, 3, 3, 4, 4, 4, 6] and [1, 2, 2, 3, 5, 6, 7], how is the loss computed?\n\nThe first step of the algorithm is discretization which maps the continuous label space Y into B bins. In such a way, the regression problem is transformed into a classification problem. Could you also show classification metrics such as overall accuracy and few-shot accuracy, along with the regression metrics, so that we have a more thorough understanding of the result? Again, what are the correlation metrics (Pearson's or Spearman's) of the results? Could you please also present some actual-by-predicted correlation plots of the prediction?\n\nHow did you determine the number of bins B?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper proposes an interesting and concise idea by generating pseudo-labels and pseudo-predictions for imbalanced deep regression. The paper is clearly written with elucidatory illustrations for easier understanding.", "weaknesses": "All datasets are image datasets and all models are ResNet-like models. The method has not been evaluated on more datasets like text data or tabular data. \n\nThe experiment setting may not be representative enough. For all three real-world datasets, their imbalance rates are fixed. However, the input data could be manually tweaked to generate datasets covering a wider range of imbalance rates, and to investigate the algorithm's performance in scenarios from mildly to extremely heavy-tailed distributions. \n\nThere is no theoretical analysis of the algorithm.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)