Lifting Imbalanced Regression with Self-Supervised Learning
Authors: Weiguo Pian
Hanyu Peng
Mingming Sun
Ping Li
Venue: ICLR 2022 Submitted (2022)
Keywords: Imbalanced Regression
Self-Supervised Learning
Long-Tailed Learning

Abstract: A new influential task called imbalanced regression, most recently inspired by imbalanced classification, originating straightforwardly from both the imbalance and regression worlds, has received a great deal of attention. Yet we are still at a fairly preliminary stage in the exploration of this task, so more attempts are needed. In this paper, we work on a seamless marriage of imbalanced regression and self-supervised learning. But with this comes the first question of how to measure the similarity and dissimilarity under the regression sense, for which the definition is clear in the classification. To overcome the limitation, the formal definition of similarity in the regression task is given. On top of this, through experimenting on a simple neural network, we found that self-supervised learning could help alleviate the problem. However, the second problem is, it is not guaranteed that the noisy samples are similar to original samples when scaling to a deep network by adding random noise to the input, we specifically propose to limit the volume of noise on the output, and in doing so to find meaningful noise on the input by back propagation. Experimental results show that our approach achieves the state-of-the-art performance.

Under review as a conference paper at ICLR 2022
LIFTING IMBALANCED REGRESSION SUPERVISED LEARNING
WITH SELF-
Anonymous authors Paper under double-blind review
ABSTRACT
A new influential task called imbalanced regression has received great deal of attention. Different from previous works, this task particularly aims to solve the imbalanced problem in the regression world. The exploration of this task is still at a preliminary stage, so more attempts are needed. In this paper, we first discover that the benefit of Self-supervised Learning (SSL) can help reveal the bias caused by the severe skewness in the training data. Therefore, we work on a seamless marriage of imbalanced regression and SSL. But with this comes the first question of how to measure the similarity and dissimilarity between raw and augmented samples under the regression sense, for which the definition is clear the classification. To tackle this issue, we provide formal definition of similarity and dissimilarity in the regression task. In addition, a common practice in SSL produce augmented samples is to put noise onto the raw samples; the second problem is, it is not guaranteed that the augmented samples are similar to original samples when scaling to a deep network by directly adding random noise the input. To approach this problem, we specifically limit the volume of noise on the output to get a meaningful augmented sample by back propagation. Experimental results show that our approach achieves state-of-the performance.
INTRODUCTION
Regression is a fundamental task in machine learning and statistical analysis, which involves modeling the relationship, between one or more independent and dependent variables. As distinct from the discrete output values in classification, regression makes predictions for continuous values. The issue of imbalanced recognition, arising from the conjunction of imbalance and classification, has been drawing the attention of researchers recently (Shu et al., 2019; Cao et al., 2019; Ren et al., 2020; Jamal et al., 2020; Kang al., 2020; 2021). By analogy, is natural to question whether long- regression exists (or one can also call it imbalanced regression, the equivalence of these two names in the paper). Exactly what we expected, a fresh task, namely long-tailed regression, has recently been proposed by Yang et al. (2021). Regardless of the newness of the task, there is way to neglect its immense significance in terms of application and research. For example, age estimation from facial images; health indicators for various populations in healthcare (Soualhi et al., 2014), these quantities usually exhibit a long-tailed distribution. Going further, in terms of performance, progress in the long-tailed regression research is also far from satisfactory.
More recently, self-supervised learning (SSL) has emerged as new rage in the research community, owing its straightforward learning paradigm yet excellent performance (Noroozi & Favaro, 2016; Gidaris et al., 2018; Wu et al., 2018; He et al., 2020). The excellence of the approach has been admired not only on classification with rell-balanced datasets, but also surprisingly on long-tailed classification by Kang et al. (2021); Wang et al. (2021); Cui et al. (2021); Samuel & Chechik (2021) who found its benefits. It prompts some thoughts on the potential and practical applications, in particular, applications to SSL for long-tailed regression (Yang et al., 2021). Mindful that the majority of available techniques on SSL are in the context of classification (Oord et al., 2018; Hjelm et al., 2018; Chen et al., 2020a; Henaff, 2020). Due to the difference between classification and regression (no hard boundary existing between outputs in regression task as shown in Figure la) (Razi & Athappilly, 2005; Breiman et al., 2017; Liaw et al., 2002; Yang al., 2021), it is not guaranteed that SSL can help address the imbalanced regression problem well, and it may be difficult to make a straightforward migration from classification to regression task. Upon deeper pondering, there

---

Under review as a conference paper at ICLR 2022
randam avibe
runddven subic
C 222
Regression
Regresdion Medel
Medel
Male
Model
Modd
Model
Medel
3/2
0.800
0.15
[.00
0.71
[0:20
0199 bird
D Diu Mi
Di Di 12,
<
cat
dog
bird
cat
dog
I [1) 21)
I( x2)
(b)
Figure 1: (a) A more visual illustration of the difference between regression and classification with regard to their sensitivity to noise. When noise is imposed, the left image suggests a greater change with augmented data for regression model. The right image remains cat predicted under the same slight perturbation. It reveals the discrepancy between the sensitivity of regression and classification to noise (b) Concepts of similarity and dissimilarity in regression. The right images indicate that after adding noise to the input image, the difference between the age predicted by the augmented image and the original one less than the threshold 8, so they are considered as similar samples, on the contrary, the difference between the two images the left is greater than ô, they therefore recognized as dissimilar samples.
obstacles and problems that we face: Can SSL really relieve the long-tailed regression problem? If so, how might solve the imbalanced regression problem by utilizing SSL?
Figure la illustrates the motivation, by adding noise to the image of the cat, it is very unlikely to be misclassified into class that differs significantly from it, such as bird, since they discrete and the categories diverge greatly from each other. In the case of regression, however, is not at all feasible when similar migration was made. We imagine that if noise added to face image, then the network's ' estimation of the person's age may be significantly biased. In this regard, it becomes essential work out and comprehend the degree noise that induces "similarity and dissimilarity" amongst output values from regression standpoint. On the foundation of this definition, questions are raised about exploiting noise sensibly and effectively.
With this paper, try for the first time to propose answers these questions. And summarize our contributions as follows:
the best of our knowledge, we are the first to broaden the concept similarity and dissim- ilarity between augmented samples and original samples from classification to regression. And we are also the first to reveal imbalanced regression problem with SSL.
Having been convinced of the beneficial effect of applying SSL on a simple experiment, we extend the neural network-based long-tailed regression task. To ensure that the noise is manageable w.r.f. the similarity of the regression, we propose to generate noise on regression values within predefined threshold.
We obtain an optimization problem on noise with an efficient approximate solution algorithm based first-order Taylor expansion. The augmented input samples obtained by pack-propagation.
As practical trial for the proposed method, the results reveal that we attained not only the best results, but also the possibility to mix it freely with other training techniques.
2 PRELIMINARY
In this section, cover the description of imbalanced regression, and then, with reference to the concept of similarity and dissimilarity in classification, we broaden this concept to regression, and provide new definition of similarity and dissimilarity in regression.
2

---

Under review as a conference paper at ICLR 2022
Notations. In this paper, lowercase typeface letters (e.g., represent scalar, lowercase bold typeface letters (e.g., x) represent vectors, uppercase typeface letters (e.g.. X) stand for random variable, uppercase bold typeface letters (e.g., X) represent matrix. diag(-) means diagonalising vector into a matrix.
2.1 PROBLEM STATEMENT
validation samples {( v =e and testing samples {( H are presumed to be available, where of E R E {s, U, and yf are continuous variables. We suppose that yf is well-bounded and notate the maximum and mini- mum values of of as Vmaz and Vmmin accordingly. Further, we are able to partition yf into B bins [y vf), [ys 1/2 v/)..... ..... LV 1.1.-11/
collection of paired training data {(a), vf)
with equal interval Ay Mm -iisa--maa Vinia In doing we could put B
every yf in the dataset into the corresponding bin fulfilling the condition yi < y. S V where 1}. With three splits (training, validation, testing), if we count the number of data points that fall into each bin and form these into vector m (mf..... ..g)). The imbalanced character the normal regressions lies in the existence of m mi, E mn", allowing mš > mi, to hold. It is the opposite of this that the values of the elements of the two vectors m and m', are approximately equal. desire for imbalanced regression enables model f(x?: 0) produced on an imbalanced training set to be potentially generalized to balanced validation and testing set to the maximum extent possible. Typically, [ar;; 0) may he a neural network with parameter O because of its impressive fitting ability. In what follows, for the sake of convenience, we will drop the superscript
E {0, I,..,
In the introduction section, we elaborate on the predicament that the lack of clear definition of the concepts of similarity and dissimilarity in the regression, as regression involves continuous targets, where hard boundaries between classes do not exist. As a solution this problem, formally propose the following definition:
There exists some threshold 8, regarding two data samples (æ,, (t), (Z., Vi). 2s is obtained by adding some noise r to x,, and ÿ and Dt is inferved by the model f(: (;0) from x; and æ; respectively. We remark that (x), (æt, Di) are similar samples of the model (ax; f) (no need for well- model) if the distance between the two samples is less than 8 under some distance function D(: )), such as li or la distance. Mathematically, it can be phrased in this following:
Definition 1
[L. if {0. 0. if D(Er, U.) 2 8,
D(g U:) 8
I(x,, (()
where I(x;, ã..) indicates the trueness of the condition whether two samples æ; and ã; are similar samples. The parameter 8 here is manually specified and task-dependent. We display a virtual example this definition as illustrated in Figure 1b. select images from AgeDB-DIR and attach noises to them, afterward we estimate the ages of the original images and their corresponding noisy images with the network, Finally, we identify a pair original image and noisy image as not similar if their difference exceeds the threshold 6, and vice versa.
MOTIVATION
Herewith we illustrate, for simple shallow neural networks, that SSL facilitates long-tailed regression in positive way. Aside from this, we would characterize the difficulties involved extending our approach to deep neural networks, i.e. for controlling the level of noise to guarantee the creation of similar samples under the sense of regression. We first describe the data generation, together with a description of the specific experimental settings followed by an analysis. The network structure and implementation details are available in the appendix.
Data generation. Consider two components of data following two different Gaussian distribution, in particular, æ1 ~ ~ (m1 la, diag(7) 1 12)), 22 ~ N (f22 14, diag(a2 1.)). 1 denotes a vector with all I's. The values of (1((1)) and (2((2)) are quite different. With linear mapping æ and x2 with parameter w E IRd, it yields the output value = w x. As result, we get two N₁ collections of training data (wf W (2rā 1 Suppose Nf Nç. and measure
3

---

Under review as a conference paper at ICLR 2022
-Q- Cp-pariissd Leaming -C- Rupreseentation Leaniing
to
MBE Enjor
Regression Medel
Resressio Model
Regression
42.8
45,0
45.6
Imbalanced Coufficient
100
45.0
42.8)
2.2
2
[45.0
45.6)
0.5
dy
(a)
(b)
Figure 2: (a) Mean Square Error with different imbalanced coefficients. Very clearly it observed that under supervised learning, MSE grows as the imbalanced coefficient increases, contrary to the SSL case where MSE is considerably smaller with the same imbalanced coefficient. (b) A demonstration of the impact of noise generation on the regression values, when adding random noise to the original image, the difference between the output value of noisy image versus such component of the original image will most likely exceed the threshold.
imbalance here terms of R
N We pick up the value R in {1, 4, 10, 100). Despite
the training samples are imbalanced, the testing set {af.; (...)=1-11.,,, 1 we generate is balanced. Additionally, we produce similar samples by controlling the noise so that the condition min(y1,c, V2.,) > Vi E (1,..., NN), E (1...., N2) is satisfied.
Experimental settings. We compare two training methods here: (i) Representation Learning: It is trained the first Ti epochs using a self-supervised approach, and the resulting network treated as initialized network, and in the second T2 epochs it is trained in a standard supervised learning manner. (ii) Supervised Learning: it entirely trained using supervised learning in the T2 +72 epochs. As per the normal process, the model is trained the training set and tested on the testing set. To evaluate we use Mean Square Error (MSE) as a criterion. Here we use the vanilla mean squared loss function.
Experimental findings. We plot the variation curve of MSE with the imbalance coefficient on the testing for the training methods. As illustrated in Figure 2a, what has been clearly observed is that the MSE for both methods grows with an increase in the imbalance coefficient. More importantly, degree. The experimental findings, thus,
it is found that SSL could alleviate this problem lead us to mitigate the imbalance regression problem with SSL.
Remark It is true that the above experiments concede that SSL can be beneficial long-tailed regression, albeit with some restrictions that remain when we adapt it to neural networks. Well, yes, we have imposed settings for noise, namely, the noise added is controllable in terms of its impact on the regression values. Unfortunately, scaling the technique beyond shallow to deep networks, there is difficulty keeping the difference between the output values of similar samples under regression within a threshold, concern that arises in the case of sample augmented with random noise, where difference between the output values the noisy sample and the original sample is more than the threshold, shown in Figure 2b, hence the challenge stems from finding valid noise to ensure the generation of similar samples.
4 METHOD
Just as we showed before, is not feasible to merely port the above method to regression tasks. Instead, it will backfire if at all we do so. Hence, it is essential to control the noise r to limit the gap between y; and ÿ; in the length of bin Ay. We search for the most amisotropic noise causing the generated samples to be similar dissimilar to the original samples, which means that to impose constraints on the output values. Rather than adding noise to the input, we blaze new a trail, that is, apply noise the output value. Mathematically, it can be expressed by the following
have
4

---

Under review as a conference paper at ICLR 2022
formula:
arg min arg max D(g Msi 0),
(2)
0
|r <, where 91 = f 0) and y = + 2 and U = f(æ; + r;; 0)
(3)
(4)
where denotes the distance loss function, and y; the noisy output w.r.t noises 2j and rs.
Here 2 is randomly sampled from [-Ay, Ay). This allows us to assure that ÿr and ýi are similar, while simultaneously enhancing the local smoothness of the output. Normally is not possible to obtain to find closed-form solution. Since neural networks nonlinear, the analytical solutions can not be derived for r For the purpose of facilitating analysis and simultaneously providing efficient training of the network, approximate D(g. Vs: 0) by the first-order Taylor expansion:
highly dimensional and
Dio Ust O) = V. (ir 3et 0)" (a, x.)) + Va, D Di 0
(5)
s.t. || <E
(6)
Where (it, Mei mmm== is constant term around z;. Based on this, the close-form solution of Ty can be calculated as
T: arg max Va, D(u Dr: 0) t +
VR, D(ir, Dr; 0)
(8)
aV D(Ui, Us:
8.1. ||r Se
w., D(G. 0::0) V., P(
Therefore, is obviously that r; so can be further denoted as
Furthermore, Eq. presented, ÿ Ds 24.
Vz. D(ji, D: + 2;; 0)
T; v., 0. D(u 0: + 24: 0)
(10)
Algorithm 1: Self-Supervised Imbalanced Regression (SSIR)
Input: Mini-batch B with M samples, model f(: 0) with its parameters 0, bin length Ay, limitation coefficient of noise c, divergence distance function D, values range of input samples (Umin, Umax), function clip(,, Umazz Urinn) to clip values that out of range of (Uméns Umaz)
Result: Updated parameters of the model 0.
for æ; in B do
Initial random noise 2s, s.f. |za| < Ay;
0t f(æ;; 0), 01: 11, 2;i
Calculate T; by taking the gradient of D(g. T.: 0) w.t.. x;
9: V D (g:, y: 0) V,, ((f x 0), f(x; 0) + 2;)
r; +- 6 g:/ll/// l//119:l|2:
Update 0; taking a. + r the input of f: E; = clip(c (x, + Ti, Umazs Umin);
ÿi f(ã,; 0);
10
Update r; by by taking the gradient of D(ú. dc: 0 W...t. x;
11
9; Vz. D (i. D6:
+ e g./l1g.l12:
12
13
2; clip(æ;; + r:.Umans Uminn)
14
[ 0)
15
end
return 1 0 Vo(
D(ūr. I1, 0))
5

---

Under review as a conference paper at ICLR 2022
I ===+ obtained in this way, however, may exceed the boundary value of the feature (For example, the range of image pixels is [0. 255). To cope with such an accident, we constrain the feature values to this range by clipping their values. On the other hand, considering that single hack-propagation optimization may not be an accurate approximation, we therefore performed a two- step optimization of r. In our experiments, one-step optimisation approach does not adequately yield optimal performance, and more iterations will introduce more computational cost to training without significant performance gain. Up now, our proposed Seperruuee Imbalanced Regression (SSIR) algorithm is completely introduced. In order to show the SSIR more clearly. We give the algorithm workflow in Algorithm
Our training strategy includes normal regression training and self subersupereieed regression. For the regression training, we directly apply supervised training to minimize the loss function between the outputs of the model I and the corresponding ground truth y. Besides, an additional coefficient is introduced to the final loss. Our overall training function is shown below:
Ljoint LR + ALSSIR
(11)
D(ú yo; 0) XD(0r.
(12)
N
D(f(z;)), ( ((
(13)
EXPERIMENTS
In this section, we report the results three datasets demonstrate the superiority of our proposed methods, when compared with state-of-the-art baselines. Specifically, we compare our method with VANILLA. SMOTER (Torgo al., 2013), SMOGN (Branco al., 2017), RRT (Yang et al., 2021), INV, SQINV, FOCAL-R (Yang et al., 2021), LDS (Yang et al., 2021), FDS (Yang et al., 2021) and some of their combinations on three imbalanced regression datasets including IMDN-WIKI- DIR, AgeDB-DIR and NYUD2-DIR. Note that, in all the experiments, the LDS is combined with re-weighting methods i.e. SQINV or INV. In otherwise, our proposed SSIR can combined with various methods in training process that demonstrates the flexibility of SSIR. The evaluation metrics used on BMDB-WIKI-DE RMSE and 8 are also employed on NYUD2-DIR dataset. For the details of datasets, baselines, evaluation metrics and implementation details, we put them in the appendix.
and AgeDB-DIR datasets are MAE and GM (Yang et al., 2021), besides,
Table 1: Comparison on the MAE and GM with several competed baselines on IMDB- WIKI-DIR dataset. The underlined part denotes the second-best result, and the bold part denotes the best result.
MAE
GM
Method
All
Many
Med.
Few
All
Many
Med.
Few
VANILLA (Yang et al., 2021)
8.06
7.23
15.12
26.33
4.57
4.17
10.59
20.46
SMOTER (Torgo et al., 2013)
8.14
7.42
14.15
25.28
4.64
4.30
9.05
19.46
SMOGN (Branco et al., 2017)
8.03
7.30
14.02
25.93
4.63
4,30
8.74
20.12
SMOGN + LDS FDS (Yang et al., 2021)
7.97
7.38
13.22
22.95
4.59
4.39
7.84
14.94
RRT (Yang et al., 2021)
7.81
7.07
14.06
25.13
4.35
4.03
8.19
16.96
RRT LDS FDS (Yang al., 2021)
7.65
7.06
12.41
23.51
4.31
4.07
7.17
15.44
INV (Yang et al., 2021)
8.17
7.64
12.46
22.83
4.70
4.51
6.94
13.78
SQINV (Yang al., 2021)
7.87
7.24
12.44
22.76
4.47
4.22
7.25
15.10
SQINV + LDS + FDS (Yang al., 2021)
7.78
7.20
12.61
22.19
4.37
4.12
7.39
12.61
SQINY SSIR
7.63
7.06
12.17
23.10
4.26
4.02
6.82
14.93
FOCAL-R (Yang et al., 2021)
7.97
7.12
15.14
26.96
4.49
4.10
10.37
21.20
FOCAL-R + LDS FDS (Yang et al., 2021)
7.88
7.10
14.08
25.75
4.47
4.11
9.32
18.67
FOCAL-R + SSIR
7.88
7.08
14.75
25.16
4.39
4.01
10.20
18.04

---

Under review as a conference paper at ICLR 2022
5.1 MAIN RESULTS
In the experiments on IMDB-WIKI-DIE dataset, we combine our proposed SSIR with SQINV and FOCAL-R respectively. As shown in Table 1, our method achieves the best performance with 7.63 and 4.26 MAE and GM accordingly. Compared with the second-bes method RRT + LDS + FDS, there has improvements of 0.02 and 0.05 in MAE and GM with our approach. Besides, in comparison with other rOCAL-R-based methods, greater performance combination of FOCAL-R and SSIR with better GM score of 4.39 and the MAE score of 7.88, For re-weighting-bass methods, the combination of SQINV and SSIR outperforms the second-best re-weighting-bass method SQINV + LDS + FDS and improves 0.15 and 0.11 on MAE and GM respectively.
be achieved with the
Table 2: Comparison the MAE and GM with several competed baselines AgeDB-DIR dataset. The underlined part denotes the second- best result, and the bold part denotes the best result.
MAE
GM
Method
All
Many
Med.
Few
All
Many
Med.
Few
VANILLA (Yang et al., 2021)
7.77
6.62
9.55
13.67
5.05
4.23
7.01
10.75
SMOTER (Torgo et al., 2013)
8.16
7.39
8.65
12.28
5.21
4.65
5.69
8.49
SMOGN (Branco et al., 2017)
8.26
7.64
9.01
12.09
5.36
4,90
6.19
8.44
SMOGN LDS FDS (Yang et ul., 2021)
7.90
7.32
8.51
11.19
4.98
4.64
5.41
7.35
RRT (Yang al., 2021)
7.74
6.98
8.79
11.99
5.00
4.50
5.88
8.63
RRT I.DS FDS (Yang al., 2021)
7.66
6.99
8.60
11.32
4.80
4.42
5.53
6.99
INV (Yang et al., 2021)
7.97
7.31
8.81
11.62
5.05
4.64
5.75
8.20
SQINV (Yang et al., 2021)
7.81
7.16
8.80
11.20
4.99
4.57
5.73
7.77
SQINV + LDS + FDS (Yang et al., 2021)
7.55
7.01
8.24
10.79
4.72
4.36
5.45
6.79
SQINV SSIR
7.49
6.81
8.54
11.04
4.61
4.28
5.12
6.92
FOCAL-R (Yang et al., 2021)
7.64
6.68
9.22
13.00
4.90
4.26
6.39
9.52
FOCAL-R + LDS FDS (Yang et al., 202.)
7.47
6.69
8.30
12.55
4.71
4.25
5.36
8.59
FOCAL-R + SSIR
7.35
6.53
8.51
11.81
4.62
4.09
5.64
8.46
In the experiments on AgeDB-DIR, with the same experimental settings IMDB-WIKI-DIR, combine our proposed SSIR with SQINV and FOCAL-R respectively. Table 2 illustrates the main results of our method and baselines on AgeDB-DIR dataset. Clearly our method achieves the best performance on both MAE and GM with 7.35 and 4.61 respectively. As the combination of the re- weighting method and SSIR, SQINV SSIR performs better than the other four re- re-weighting-base based methods with 0.06 and 0.11 improvements on MAF and GM separately, compared to the second-best e--weighting-base based method SQINV LDSS FDS. For FOCAL-R--aase methods, the association with SSIR can further decrease the testing error than FOCAL-R and FOCAL-R + LDS + FDS.
For NYUD2-DIR dataset, combine SSIR with VANILLA and re-weighting method INV respec- tively. As presented in Table 3, our method INV + SSIR achieves the best performance among other methods. Specifically, INV + SSIR outperforms the second-best method INV + LDS + FDS on RMSE with 0.047, and our method achieves the best performance on ō1 with 0.075, Compared with VANILLA, the combination with SSIR can improve the performance significantly with 0.07 on RMSE and 0.007 on 51.
Table 3: Comparison the RMSE and & with several competed basclines NYUD2-DIR dataset. The underlined part denotes the second-best result, and the bold part denotes the best result.
RMSE
81
Method
All
Many
Med.
Few
All
Many
Med.
Few
VANILLA (Yang et al., 2021)
1.477
0,591
0,952
2.123
0.677
0.777
0.693
0.570
VANILLA + SSIR
1.407
0.580
0.883
2.022
0.684
0.772
0.704
0.590
INV LDS FDS (Yang al., 2021)
1.338
0.670
0,851
1.880
0.705
0.730
0.764
0.655
INV SSIR
1.291
0.766
0.894
1.756
0.705
0.669
0.763
0.718

---

Under review as a conference paper at ICLR 2022
9.00
-o-E-led -o-E=504 -o-E=1e-3 -C-E-Se-3
70
8.50
MAE
8.00
30
7.50
sample1
cample2
sample3
sampled
samples
samplet
(b)
Figure 3: (a) Parameter study of FOCAL-R + SSIR AgeDB-DIR dataset, for a fixed value of X, we vary the value of e. (b) Original outputs and noisy outputs of samples from AgeDB-DIR dataset.
5.2 PARAMETER AND CASE STUDY
Parameter Study. Figure 3a illustrates the results of parameter study of FOCAL-R SSIR AgeDB-DIR DIR dataset. As it shows, each line denotes the results of different À when fixing and we conduct such experiments with the of Se-3, le-3, 5e-4 and le-4. The overall results show that our method achieves the best performance with le 3 and 1.0.
Case Study. We also conduct case studies to visualize some results of selected examples. As Figure 3b shows, the gaps between the outputs and their associated noisy outputs of all the chosen examples do not exceed a bin length, which means our proposed method can successfully control the noise to limit the noisy outputs within reasonable range.
Visualization on Noisy Samples. Moreover, we also show three examples and their corresponding noisy examples with of 0.05, 0.1, 0.3, 0.5 and 1.0 respectively in Figure 4. Obviously, with the increasing, the noisy examples become unclearer that means the noise level is positively correlated with e.
6 = 0.05
= 0.1
= 0.3
= 0.5
= 1.0
Figure 4: Three groups of noisy samples generated from three samples chosen from AgeDB-DIR dataset with different e.
8

---

Under review as a conference paper at ICLR 2022
6 RELATED WORK
Presented here are the literature on imbalance problems and SSL. Concerning imbalance problems, the introduction focuses on classification and regression. For SSL, we review some of the advances and applications.
6.1 IMBALANCED PROBLEM
From the object of imbalanced problems, it can be divided into two categories: imbalanced classifica- tion and imbalanced regression.
Imbalanced classification. the imbalanced classification task, existing methods mainly adopt data-based model-basee solutions (Yang al., 2021). The core idea of data-based methods is reducing the gap of sampling numbers between many-shot classes and few-shot classes (Chawla al., 2002; He et al., 2008). For the model-based methods, several works have been proposed to tackle such challenges based on lots of aspects. Jamal et al. (2020) proposed to solve long-tailed classification through domain adaptation perspective. Kang et al. (2020) used two-stage learning strategy to learn representations for the classification layer using long-tailed training set. Kang al. (2021) explored the impact of balanced feature spaces when dealing with imbalanced data, and proposed SSL-based method to better learn the representation for imbalanced classification. Besides, SSL on unbalanced classification was similarly explored in Wang et al. (2021); Cui et al. (2021); Samuel & Chechik (2021). Furthermore, there also has some methods addressing the problem by well-designed loss functions (Zhang et al., 2017; Cao et al., 2019; Cui et al., 2019; Ren et al., 2020) and other learning paradigms, i.e. meta learning (Shu et al., 2019; Wang al., 2020).
Imbalanced regression. Imbalanced problem has also been explored in regression problems. Con- ventional works mainly applied data-based methods to solve the imbalanced problem in regression tasks. One of the first works to address this problem, is published by Torgo et al. (2013) using SMOTE (Chawla et al., 2002). Branco et al. (2017) proposed Gaussian noise-based synthetic case generation method for imbalanced regression. After that, they further proposed method aggregate various data pre-processing strategies (Branco al., 2018) for the same task. Recently, Yang et al. (2021) further delved into such problem by analyzing the error distribution and the statistic distribution of features, both w.r.t. labels, and proposed two distribution smoothing meth- ods for labels and features. And they also proposed the Focal-R, R, new loss function for imbalanced regression, motivated by Focal loss (Lin et al., 2017).
logging-based
6.2 SELF-SUPERVISED LEARNING
Recently, SSL. has received extensive attention in the field of deep learning. Different from con- ventional unsupervised learning and supervised learning, SSL can use the semantic or contexture supervision information to learn robust and well-generalization representations without manual annotations, include contrastive learning-based methods (Chen et al., 2020a;b;c; He et al., 2020) and some other pretext basks-based methods, such as image inpainting (Pathak et al., 2016; Jenni & Favaro, 2018), context prediction (Doersch et al., 2015), rotation prediction (Gidaris et al., 2018) and so on. Whereas the fact that we apply noise to the output directly rather than to the input, as well as the fact that the problem we analyze varies from semi-supervised classification problems to imbalanced regression problems.
tackle the optimization problem in a very similar way, the difference concerns
7 CONCLUSION
In this paper, we explore the Superiuper Learning (SSL) in imbalanced regression scenario. Specifically, we propose Self-Supervised Imbalanced Regression (SSIR) to better address the imbal- anced problem in regression tasks. In SSIR, instead of adding noise to the inputs directly to generate the noisy samples, we add noise, which is limited by the bin size, to the output of the model to avoid the excessive dissimilarity between samples and noisy samples, followed by back propagation steps to obtain the meaningful noisy samples. This work provides a paradigm tackle this challenge. Further, extensive experiments have shown the superiority of our proposed SSIR over existing state-of-the-an thr- baselines.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Paula Branco, Luís Torgo, and Rita P Ribeiro. Smogn: a pre-processing approach for imbalanced regression. First international workshop on learning with imbalanced domains: Theory and applications, pp. 36-50. PMLR, 2017.
Paula Branco, Luis Torgo, and Rita P Ribeiro. Rebagg: Resampled bagging for imbalanced regres- sion. In Second International Workshop on Learning with Imbalanced Domains: Theory and Applications, pp. 67-81, PMLR, 2018.
Leo Breiman, Jerome Friedman, Richard A Olshen, and Charles J Stone. Classification and regression trees. Routledge, 2017.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with label-distributionaaaar margin loss. Advances in Neural Information Processing Systems, 52:1567-1578, 2019.
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer, Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16:321-357, 2002.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020a.
Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in Neural Information Processing Systems, 33:22243-22255, 2020b.
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint (tl::2003.042997, 2020c.
Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 715-724, 2021.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge Belongie. Blass-balanced loss based on effective number of samples. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9268-9989999 9277, 2019.
Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision, pp. 1422-1430, 2015.
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv: 1803.07788, 2018.
Haibo He, Yang Bai, Edwardo A Garcia, and Shutao Li. Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE international joint conference on neural networks, pp. 1328-1328. IEEE, 2008
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020.
Olivier Henaff. Data-efficient image recognition with contrastive predictive coding. In International Conference on Machine Learning, 4 4182-4192. PMLR, 2020.
R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildeee Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv: 1808 06670, 2018.
10

---

Under review as a conference paper at ICLR 2022
Junjie Hu, Mete Ozay, Yan Zhang, and Takayuki Okatani. Revisiting single image depth estimation: Toward higher resolution maps with accurate object boundaries. In 2019 IEEE Winter Conference Applications of Computer Vision (WACV), pp. 1043- 1051. IEEE, 2019.
Muhammad Abdullah Jamal, Matthew Brown, Ming-Hsuan Hsuan Yang. Liqiang Wang, and Boqing Gong. Rethinking class-balanced methods for long-tailed visual recognition from domain adaptation perspective. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 7610-7619, 2020.
Simon Jenni and Paolo Favaro. Self-supervised feature learning by learning to spot artifacts. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pp. 2733-2742, 2018.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis. Decoupling representation and classifier for long-tailed recognition. In International Conference on Learning Representations, 2020.
Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, and Jiashi Feng. Exploring balanced feature spaces for representation learning. International Conference on Learning Representations, 2021.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1912.690, 2014.
Andy Liaw, Matthew Wiener, et al. Classification and regression by randomforest. R news, 2(3): 18-22, 2002.
Tsung-V Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pp. 2980-2988, 2017.
Christos Louizos and Max Welling. Structured and efficient variational deep learning with matrix gaussian posteriors. In International Conference on Machine Learning, pp. 7716-716.. PMLR, 2016.
Stylianos Moschoglou, Athanasios Papaioannou, Christos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: the first manually collected, in-the-wild age database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 51-59, 2017.
Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference computer vision, pp. 69-84. Springer, 2016.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 18Xiv:1807.03748, 2018.
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 2536-2544, 2016.
Muhammad A Razi and Kuriakose Athappilly. A comparative predictive analysis of neural networks (nns), nonlinear regression and classification and regression tree (cart) models. Expert systems with applications, 29(1):65-77) 2005.
Jiawei Ren, Cunjun Yu, Shunan Sheng, Xiao Ma, Haiyu Zhao, Shuai Yi, and Hongsheng Li. Balanced meta-softmax for longetailed visual recognition. In Proceedings of Neural Information Processing Systems 2020.
Rasmus Rothe, Radu Timofte, and Lue Van Gool. Deep expectation of real and apparent age from single image without facial landmarks. International Journal of Computer Vision 126(2):144-157 2018.
Dvir Samuel and Gal Chechik. Distributional robustness loss for toig-tail learning. arXiv preprint arXiv:2 (0::22/44.00 03066, 2021.
11

---

Under review as a conference paper at ICLR 2022
Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. Meta- weight-net: Learning an explicit mapping for sample weighting. Advances in Neural Information Processing Systems, 32:1919-1930, 2019.
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In European conference on computer vision, pp. 746-760. Springer, 2012.
Abdenour Soualhi, Kamal Medjaher, and Noureddine Zerhouni. Bearing health monitoring based on hilbert-huang transform, support vector machine, and regression. IEEE Transactions instrumentation and measurement, 64(1):52-62, 2014.
Xin Sun, Zhenning Yang. Chi Zhang, Keck- Voon Ling, and Guohao Peng. Conditional gaussian distribution learning for open set recognition. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13480-13489, 2020.
Luís Torgo, Rita P Ribeiro, Bernhard Pfahringer, and Paula Branco. Smote for regression. In Portuguese conference on artificial intelligence, pp. 378-389, Springer, 2013.
Peng Wang, Kai Han, Xiu-Shen Wei, Lei Zhang, and Lei Wang. Contrastive learning based hybrid networks for long-tailed image classification. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 943-952, 2021.
Qilong Wang, Peihua Li, and Lei Zhang. G2denet: Global gaussian distribution embedding network and its application visual recognition. In Proceedings of the IEFE conference computer vision and pattern recognition, pp. 2730-2739, 2017.
Renzhen Wang, Kaiqin Hu, Yanwen Zhu, Jun Shu, Qian Zhao, and Deyu Meng. Meta feature modulator for long-tailed recognition. arXiv preprint arXiv:2008.0344 2020.
Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non- parametric instance discrimination. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 3732-3742, 2018.
Yuzhe Yang, Kaiwen Zha, Ying-Cong Chen, Hao Wang, and Dina Katabi. Delving into deep imbalanced regression. In International Conference on Machine Learning (ICML), 2021.
Xiao Zhang, Zhiyuan Fang, Yandong Wen, Zhifeng Li, and Yu Qiao. Range loss for deep face recognition with long-tailed training data. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5409-5418, 2017.
12

---

Under review as a conference paper at ICLR 2022
8 APPENDIX
In the appendix, we first offer two comparative experiments. Then we give the implementation details in the motivation, as well the details of the datasets and baseline methods included in the experiments. Besides, we also present more detailed experimental results of parameter studies, and loss curves within different settings. Finally, We investigate the beneficial impact of our approach on feature imbalance.
8.1 COMPARISON WITH RANDOM NOISE METHOD
With the addition of random noise, the combined SSIR with FOCAL-R and SQINV approaches achieve the performance MAE with 7.57 and 7.71 respectively AgeDB-Dil -DR dataset. In contrast, the combined SSIR with FOCAL-R and SQINV approaches with noise generation proposed in our paper achieve MAE performance with 7.35 and 7.49 respectively. The experimental results demonstrate the superiority of our noise generation approach.
8.2 COMPARISON WITH ONE-STEP OPTIMIZATION METHOD
To quantitatively compare the benefits of two-step approximation against one-step approximation. we conducted a comparison experiment on the AgeDB-DIR dataset. The setup of the comparison experiment is following: we perform one-step optimization of and provide results combined with SQINV; except for this, we keep the other hyperparameters consistent with our experiments. When performing one-step optimization, the combined SSIR with SQINV approach can achieve the performance on MAE with 7.62. In contrast, the combined SSIR with SQINV approach with two- step approximation can achieve MAE performance with 7.49. The experimental results demonstrate the superiority of two-step approximation. However, we have found in our experiments that there is no significant increase in performance but an increase in computational overhead when the number of optimization steps is greater than or equal to 3. Therefore, in order to balance accuracy and efficiency, we have used two-step optimization in our paper.
8.3 EXPERIMENTS SETTINGS
Datasets We evaluate our proposed method on three large-scale deep imbalanced regression datasets, the tasks of which are age estimation and depth estimation. For all the datasets used in our experiments, we follow the settings in (Yang et al., 2021) and segment the target bins into three subsets: many-shot which contains more than 100 training images, medium-shut shot in which the number of training samples in each bin is in the range of 20 to 100, and few-shot that contains less than 20 training samples in following:
each bin. The details of the datasets
IMDB-W WIKI-DIR: IMDB- WIKI-DIR is an imbalanced age estimation dataset. This dataset is created by Yang et al. (2021) 1) based on the IMDB-WIKI dataset (Rothe et al., 2018). The original IMDB-WIK' contains 523.0K face images with associated ages. In IMDB-WIKI- DIR, the training set contains 191.5K images, and both validation and testing sets include 11.0K bin balanced images separately. The length of a bin is set to year with the range from 0 to 186, and the density of each bin is from to 7.149.
AgeDB-DIR: AgeDB-DIR is also a imbalanced age estimation dataset proposed by Yang et al. (2021) using AgeDB dataset (Moschoglou et al., 2017). In the dataset, the age range is from 0 to 101 and the bin length is I year. The maximal and minimal numbers of images in bin are 353 and I respectively. The training set contains 12,208 images with imbalanced densities among bins, while the validation and testing sets contain 2,140 bin-balanced images respectively.
The NYUD2-DIR is dataset constructed by Yang et al. (2021) for im- balanced indoor scenes depth estimation, which is an imbalanced version of NYU Depth Dataset V2 (Silberman et al., 2012). The dataset contains images of different indoor scenes and their corresponding depth maps. The bin length is set to 0. meters. The maximal and minimal depth is 10 meters and 0.7 meters separately in the dataset. Following Yang et al. (2021), there are 50K images in the training set and 654 images in the testing set. In each bin, the number of pixels is in the range from 1.13 10 to 1.46 10 Besides,
NYUD2-DIR:
13

---

Under review as a conference paper at ICLR 2022
the number of testing pixels selected for each bin from testing images is set to 9,357 (the minimal number of pixels in each bin in the testing set), which aims to make the testing set balanced. The total number of pixels in the training set and testing set are 3.51 x 10 and 8.70 x 10 respectively.
Baselines We conduct experiments to compare our proposed method with the following baselines:
Vanilla model: The pure backbone model that does not apply any techniques to improve the performance is presented as VANILLA Following (Yang et al., 2021), we use ResNet- 50 (He et al., 2016) as VANILLA for both IMDB-WIKI-DIR and AgeDB-DIR datasets, and ResNet-50-based encoder-decoder method (Hu et al., 2019) for NYUD2-DIR dataset.
Synthetic samples: We choose existing synthetic samples methods, which are used for imbalanced regression, as baselines, including SMOTER (Torgo et al., 2013) and SMOGN (Branco et al., 2017).
Focal-R: FOCAL-R (Yang et al., 2021) is the regression version of Focal loss (Lin et al., 2017), which uses a continuous function, egg. Sigmoid function, to map the absolute error into the range of 0 to instead of the scaling factor in Focal loss.
Regressor Re-Training (RRT): RRT (Yang al., 2021) is two-stage training strategy inspired by Kang et al. (2020), in which the feature and classifier are decoupled.
Re-weighting: We also select Re-weighting methods as baselines, including inverse- frequency weighting (INV) and its square-root weighting variant (SQINV) (Yang al., 2021).
Distribution Smoothing: Distribution Smoothing (Yang et al., 2021) includes Label Dis- tribution Smoothing (LDS) and Feature Distribution Smoothing (FDS). Such methods are designed to utilize the information from nearby bins by smoothing the distribution of labels or features. Note that, LDS should used with re-weighting methods, and we follow Yang et al. (2021) to combine LDS with INV and SQINV.
Evaluation metrics Here, we will introduce the details of all the evaluation metrics used in our experiments.
Mean Absolute Error (MAE): MAE common evaluation metric in regression prob- lems, which is used to measure the averaged absolute difference between ground truth and prediction results. MAE can be denoted as vV Σ li Wel. where N denotes the number of total samples, Dt and y/ are the prediction result and ground truth of the i-th sample respectively.
Root Mean Squared Error (RMSE): Similar to MAE, RMSE is also widely used evaluation metric in regression tasks. Different from MAE, RMSE is used to measure the rooted averaged squared difference between ground truth and prediction results, which can be presented as ΣN
N (0i )²) where N denotes the number of total samples, ÿr and are the prediction result and ground truth of the i-th sample respectively.
error Geometric Mean (GM): GM is a new evaluation metric proposed by Yang et al. (2021). Instead of applying arithmetic mean, GM uses geometric mean over the prediction errors, which is defined as (II ex)+,, i-th sample.
,, where e; - | - yil denotes the L₃ error of the
Threshold Accuracy (5.): For NYUD2-DIR dataset, follow Yang et al. (2021) and use standard depth estimation evaluation metric 6, which is defined max( d4
gi d/ 8, < 1.25', where 9i and d, denote the ground truth value and the the prediction value of pixel respectively. Here, in our experiments on NYUD2-DIR, we set to 1 as the settings in (Yang et al., 2021).
8.4 IMPLEMENTATION DETAILS IN MAIN EXPERIMENTS
IMDB-WWIL WIKI-DIR: For IMDB WIKI-DIR dataset, we use ResNet-50 (He et al., 2016) as backbone and train it with proposed SSIR method for 90 epochs using Adam optimizer (Kingma & Ba,
14

---

Under review as a conference paper at ICLR 2022
2014) with L1 loss. The initial learning rate is set to 1e-3 and decayed to le-4 and le-5 at the 60-th and 80-th epoch respectively. Besides, we set the batch size, and to 256, 0.1 and le-3 separately.
AgeDB-DIR: For AgeDB-DIR dataset, the settings of backbone. optimizer, learning rate, loss function and batch size are the same with that of R+SSIR, set the À and to 1.0 and le-3 respectively. Besides, they set to 0.1 and 1e-3 for SQINV+SSIR.
For the training of Focal-
For the experiments NYUD2-DIR, apply SisNee-55--basee encoder-decoder architecture (Hu et al., 2019). We train our method by Adam optimizer with 20 epochs. The initial learning rate is le-4 and decayed by 0.1 every 5 epochs. We set the batch size and weight decay to 32 and le-4 respectively. For À and we set them to 1.0 and 5e-4 for VANILLA+SSIR, and 2.5, 5e-4 for INVERSE+SSIR.
NYUD2-DIR:
8.5 IMPLEMENTATION DETAILS IN MOTIVATION EXPERIMENTS
For the settings in the data generation of our motivation, we set the mean and variance of the two Gaussian distribution μ1, 01 and 1/2, 02 to 10, 25 and -10, 25 respectively, and the dimension of the generated data d is set to 100. We apply a two-layer Multi-Layer Perception (MLP) with 128 hidden neurons as the prediction model the experiments. For the settings of the training process in the experiments, we set epochs Ti and T2 1000 and 500 respectively. The learning rates of the two training methods: Representation Learning and Supervised Learning, are set to 1e-3 and le-2 respectively. Besides, we the weight decay of the training process in the experiments to 5e-5.
8.6 PARAMETER STUDIES
Here, we study the different parameter settings of e and À on all three datasets and report all the results of the experiments that we conducted.
Table 4 illustrates the experiments results of different settings of e and X. The top part of the table is the results of different parameters settings of SQINV SSIR, and the bottom part shows the results of FOCAL-R SSIR on different settings of and X.
The results of the parameter study of our method on AgeDB-DIR dataset are presented in Table As the table shows, we present the results of different and À settings of our approach. The top part of the table is regarding with SQINV + SSIR, and the bottom part denotes the results of different parameter settings of FOCAL-R SSIR.
Table 4: Parameter study of our methods on
dataset.
MAE
GM
X
re-weighting
Loss
All
Many
Med.
Few
All
Many
Med.
Few
le-3
0.01
SQINV
L1
7.74
7.15
12.61
22.94
4.38
4.13
7.35
15.04
le-3
0.05
SQINV
L1
7.74
7.13
12.67
23.34
4.33
4.08
7.35
14.72
le-3
0.10
SQINV
L1
7.63
7.06
12.17
23.10
4.26
4.02
6.82
14.93
le-3
0.50
SQINV
L1
7.83
7.20
12.91
23.86
4.50
4.24
7.63
16.27
le-3
0.10
FOCAL-R
7.88
7.08
14.75
25.16
4.39
4.01
10.20
le-3 0.30
FOCAL-R
7.95
7.16
14.79
24.78
4.46
4.08
10.36
18.26
le-3 0.50
FOCAL-R
8.11
7.25
15.44
26.76
4.61
4.20
11.04
19.46
le-3
1.00
FOCAL-R
8.17
7.35
15.26
25.11
4.65
4.25
10.98
17.27
Table 6 is the parameter study regards and À on NYUD2-DIR dataset. The top part of the table illustrates the results of different parameters settings of VANILLA SSIR while the bottom part shows that of INVERSE SSIR.
8.7 TRAINING AND VALIDATION LOSSES
We present the training and validation loss of our proposed approach IMDB-WIKI-DI. and AgeDB-Dil D-DDB-DIR datasets. Figure 5a and 5b present the training and validation losses w.t.t training epochs of SQINV SSIR and FOCAL-R SSIR respectively HIDB-WIK-DDIR dataset. Figure 5c
15

---

Under review as a conference paper at ICLR 2022
Table 5: Parameter study of our method on AgeDB-DIR dataset.
MAE
GM
re-weighting
Loss
All
Many
Med.
Few
All
Many
Med.
Few
Se-4
0.1
SQINV
L1
7.06
8.32
10.26
4.90
4.52
5.67
7.00
5e-4
0.3
SQINV
T
7.67
6,98
8.86
10.95
4.88
4.44
7.27
5c-4
0.5
SQINV
L1
7.63
7.02
8.59
10.69
4.84
4.41
5.81
7.12
5e-4
1.0
SQINV
L1
7.65
7.01
8.47
11.49
4.96
4.58
5.67
7.22
le-3
0.1
SQINV
L1
7.49
6.81
8.54
11.04
4.61
4.28
5.12
6.92
le-3
0.3
SQINV
LA
7.73
7.05
8.74
11.34
4.86
4.38
5.97
7.33
le-3
0.5
SQINV
LA
7.61
6.94
8.74
10.80
4.93
4.53
5.81
7.00
le-3
1.0
SQINV
L1
7.70
7.02
8.69
11.37
4.83
4.37
5.90
7.34
Se-3
0.1
SQINV
LA
7.62
6.84
8.75
11.87
4.86
4.41
5.84
7.36
5e-3
0.3
SQINV
LA 7.65
6.86
8.83
11.80
4.88
4.34
6.05
8.12
Se-3
0.5
SQINV
L1
7.61
6.97
8.35
11.57
4.81
4.36
5.56
8.27
Se-3
SQINV
TA
8.04
7.15
9.56
12.20
5.13
4.50
8.37
le-4 0.1
FOCAL-R
7.62
6.75
8.85
12.43
4.94
4.41
5.97
8.67
le-4 0.3
FOCAL-R
6.59
8.26
12.39
4.79
4.32
5.46
8.90
lc-4 0.5
FOCAL-R
7.62
6.81
8.54
12.73
4.91
4.37
5.89
8.92
lc-4
1.0
FOCAL-R
7.53
6.66
8.59
12.73
4.76
4.14
5.98
9.46
le-4
1.5
FOCAL-R
7.60
6.71
8.79
12.62
4.92
4.28
6.25
9.33
Se-4 0.1
FOCAL-R
7.69
6.66
9.30
13.04
4.90
4.23
6.26
9.96
Se-4 0.3
FOCAL-R
7.51
6,49
9.05
12.95
4.65
3.97
9.74
5e-4 0.5
FOCAL-R
7.46
6.59
8.51
12.83
4.69
4.13
5.73
9.13
Se-4 1.0
FOCAL-R
7.61
6.64
9.14
12.64
4.91
4.24
6.36
9.53
5e-4 1.5
FOCAL-R
7.37
6.47
8.74
12.13
4.72
4.16
5.87
8.43
le-3 0.1
FOCAL-R
7.44
6.50
8.83
12.44
4.73
4.11
6.13
8.62
le-3 0.3
FOCAL-R
6.45
8.81
12.37
4.69
4.11
le-3
FOCAL-R
7.39
6.47
8.81
12.09
4.68
4.09
6.06
8.22
le-3 1.0
FOCAL-R
7.35
6.53
8.51
11.81
4.62
4.09
5.64
8.46
le-3 1.5
FOCAL-R
7.52
6.60
8.61
13.15
4.77
4.19
5.79
9,46
Se-3
0.1
FOCAL-R
7.48
6.54
8.99
12.10
4.81
4.21
5.98
9.06
Se-3
0.3
FOCAL-R
7.42
6.50
8.56
12.97
4.76
4.11
5.98
10.28
Se-3
FOCAL-R
6.53
8.98
13.84
4.89
4.14
6.40
11.28
Se-3
1.0
FOCAL-R
8.13
6.87
9.75
15.51
5.23
4.37
7.05
12.72
5e-3
1.5
FOCAL-R
9.00
7.33
11.54
17.79
5.85
4.68
8.66
16.16
Table Parameter study of our method on NYUD2-DIR dataset.
RMSE
81
re-weighting
Loss
All
Many
Med.
Few
All
Many
Med.
Few
Se-4
MSE
1.542
0.580
0.871
2.246
0.669
0.787
0.739
0.523
Se-4
0.3
MSE
1.531
0.536
0.878
2.238
0.663
0.796
0.690
0.519
5e-4
0.5
MSE
1.540
0.566
0.917
2.239
0.643
0.778
0.713
0.481
5e-4 1.0
MSE
1.407
0.580
0.883
2.022
0.684
0.772
0.704
0.590
5e-4 1.5
MSE
1.475
0.564
0.872
1.0
0.692
0.793
0.723
0.579
5c-4 2.0
MSE
1.402
0.601
0.893
2.005
0.692
0.755
0.704
0.625
5e-4 2.5
MSE
1.623
0.535
0.932
2.380
0.640
0.801
0.704
0.455
5c-4 3,0
MSE
1.476
0.560
0.899
2.137
0.686
0.789
0.723
0.570
le-3 0.1
MSE
1.476
0.564
0.919
2.134
0.693
0.791
0.723
0.585
le-3 0.3
MSE
1.540
0.560
0.932
2,238
0.680
0.792
0.724
0.550
le-3 0.5
MSE
1.520
0.568
0,876
2.212
0.667
0.787
0.733
0.521
le-3 1.0
MSE
1.537
0.557
0.898
2.239
0.658
0.787
0.716
0.507
5e-4
1.0
INVERSE
MSE
1.365
0.667
0.930
1.915
0.685
0.711
0.717
0.647
Se-4
1.5
INVERSE
MSE
1.391
0.679
0.849
1.968
0.670
0.713
0.734
0.602
5e-4
2.0
INVERSE
MSE
1.313
0.749
0.919
1.795
0.700
0.702
0.742
0.682
5e-4
2.5
INVERSE
MSE
1.291
0.766
0.894
1.756
0.705
0.669
0.763
0.718
16

---

Under review as a conference paper at ICLR 2022
Traihing Volidation
18.
Training Validation
loss
12:
1.-
20
40
60
20
60
Epochs
Epochs
(b)
Training Loss Validttio Loss
Training Loss
Val dation Loss
12
1059
10:
28-
(c)
(d)
Figure 5: (a) Training and validation loss win/ training epochs SQINV SSIR on IMDB-WIKI-DIR
dataset. (b) Training and validation loss w.r training epochs of FOCAL-R SSIR IMDB -III--
DIR dataset. (c) Training and validation loss w.r. training epochs SQINV + SSIR on AgeDB- DIR dataset. (d) Training and validation loss wer training epochs of FOCAL-R SSIR AgeDB-DIR dataset.
and 5d illustrate the training and validation losses of SQINV SSIR and FOCAL-R + SSIR on AgeDB-DIR dataset.
8.8 FEATURE BALANCE UNDER REGRESSION SENSE
To qualitatively and quantitatively evaluate the balance of features under the regression sense, we first attempt to offer a definition of feature balance, and then assess the impact of our approach and the baseline approaches experimentally.
8.8.1 DEFINITION OF FEATURE BALANCE
As mentioned in (Kang et al., 2021), the feature space can be more balanced if the features from the same class or bin are more similar, and the balanced feature space can handle the imbalanced problem well. In this subsection, we give the defimition of feature balance under regression sense. We assume that z with size c(z) is feature vector extracted from the neural network corresponding the input æ. It is often the case that z is semantically rich. Each element in signifies a different attribute. Without loss of generality, it is also reasonable to expect that the elements z are mutually independent. And then, we assume that for data within a bin, its features all conform to Gaussian distribution N N(P of). = 1,2,... c(z) for each dimension. The reasons for the Gaussian assumption are: Gaussian distribution is the most commonly used, and it can fit deep learning features relatively well (Wang et al., 2017; Sun et al., 2020; Louizos & Welling, 2016). Therefore, the variances of the distributions of each dimension can be used as the evaluation metric, evaluate the imbalanced ratio
17

---

Under review as a conference paper at ICLR 2022
VANILLA
SQINV
SQINV SSIR
40
30
Variance
20
10
0 All
Many
Med.
Few
Figure 6: Visualization of variance values of deep learning features on the IMDB-WIKI WIKI-DIR testing dataset for the VANILLA, SQINV and SQINV+SSIR methods. We compare the three methods on all bins, many-shot bins, middle-shot bins and few-shot bins respectively. SQINV+SSIR owns smaller variance a, which implies more balanced features are leaned.
of the feature space:
σ = Σo; s.t. i 1,2, c(2)
(14)
(15)
where smaller indicates balanced feature space that can handle the imbalanced problem well.
8.8.2 EVALUATION OF FEATURE BALANCE ON WIDB-WIKI-DIR DATASET
We conduct experiments on IMDB-WIKI-DIR testing dataset, recall that previously used f(x; 0) to denote a neural network. Here (ax; 0) is implemented via ResNet-50. z is the feature extracted from the penultimate layer in f(æ; 0), which has dimension of 2048. The methods evaluation are VANILLA. SQINV and our SQINV+SSIR. In Figure we show the variance values many-shot, middle-shot and few-shot classes, it is evident that
for
for the different methods method yields a much smaller variance a, which means the feature space generated by our method more balanced. Figure 7 shows more variance values of the three methods within different bins.
18

---

Under review as a conference paper at ICLR 2022
120
100
Varianca
60
20
60
Variance
10
20
40
60
80
100
60
50
40
Variance
20
40
60
80
100
Target value (Age)
Figure 7: Visualization of variance values of deep learning features the IMDB-WIKI- testing dataset for the VANILLA, SQINV and SQINV+SSIR methods on different bins. The three subfigures from the top to bottom correspond in turn to VANILLA. SQINV and SQINV+SSIR methods.
19


Reviews:
[{"review_id": "m9wIef1avZB", "reviewer": "Reviewer_QHt8", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths: \nThis paper seamlessly combines self-supervised learning and imbalanced regression by giving the formal definition of similarity in the regression task, and proves that self-supervised learning really relieve the long-tailed regression problem. The Experiments are substantial.\n\nWeaknesses: \n1)  In the penultimate paragraph of Section 4: “On the other hand, considering that a single back-propagation optimization may not be an accurate approximation, we therefore performed a two-step optimization of r.” Why a two-step optimization can be helpful for obtaining an accurate approximation? Please give an explanation.\n2)  In Section 4, why the authors apply noise to the output instead of the input? How to solve the imbalanced regression problem when applying noise to the input?\n3)  In Section 3, Data generation, why the test set generated by the authors is balanced? How would the proposed method perform if the test data is unbalanced?\n4)  In Section 3, the fourth line of second paragraph: y=w^Tx, the vector transpose symbol ‘T’ should not be bolded.\n5)  The sentence: “the left image suggest a greater change with augmented data.” “suggest” should be “suggests”.\n\n", "summary_of_the_paper": "This paper proposes a novel algorithm SSIR to address the imbalanced problem in regression tasks, which seamlessly combines self-supervised learning and imbalanced regression by giving the formal definition of similarity in the regression task. Besides, the authors specifically propose to limit the volume of noise on the output, and in doing so to find meaningful noise on the input by back propagation. Experimental results show that our approach achieves the state-of-the-art performance.", "summary_of_the_review": "I think this paper in its present form cannot be accepted for publication in ICLR.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "emdu3-F36Td", "reviewer": "Reviewer_cjZs", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n1. Although the self-supervision signal has been discussed under the context of classification, leveraging it in the imbalance regression task is novel.\n2.  The paper identifies and addresses an important problem: how to define positive pairs for the regression task and what is a valid augmentation to produce a positive pair. Solving the problem is a necessary step to introduce more self-supervised methods into the imbalanced regression domain.\n3. The solution proposed in the paper is novel. Augmenting on the label space and reversely finding the mapping in the image space is interesting. The method also shows promising empirical results in Tables 1,2, 3.\n\nWeakness:\n\n1. It is unclear why only noise-based augmentation is discussed in the paper. \n\n   The paper only discusses noise-based augmentation. However, random noise is only one of the many augmentation techniques. There are a number of other augmentations popular in self-supervised contrastive learning, for example, random cropping, random blur, and random color distortions. Can they transfer to the regression task? Can the proposed method apply to these augmentation techniques as well?\n2. The argument about \"infeasible migration\" in the Introduction is weakly supported and lacks empirical evidence.\n\n   The paper argues that \"In the case of regression, however, it's not at all feasible when a similar migration was made (from classification to regression) \". The authors support the argument using an illustration in Figure 1, where regression models make drastically varying predictions to different noises and hence prove that the random noises ambiguate regression targets. \n\n   However, the support is a little weak considering that a classifier will also make wrong predictions when noises are added [1]. For example, in Figure 1 (a), the cat could be classified as a dog/bird when strong enough noise is added. In another word, the illustration does not fully convince me why the use of noise should be different on classification and regression and why the \"migration\" is not feasible. Why would simple thresholding on the noise level like in classification [2] fail on the regression task?\n\n   Moreover, the empirical evidence for the argument is missing in the paper. There lacks a contrastive learning baseline that uses random noise to produce positive pairs like in the classification tasks. Compared with the sophisticated data generation procedure proposed in the paper, it would be interesting to know how a naive random noise augmentation performs. \n3. The settings in Motivation (Section 3) and Method (Section 4) are inconsistent, which undermines the paper's motivation.\n\n   In the motivation section, self-supervision is used as a pre-train and steady improvement is observed in Figure 2. However, in the method section, self-supervision is used as an additional training objective (Equation 11). Why not keep the pre-training setting in the method? \n\n4. There are some minor issues with the parameter study.\n\n   The paper concluded that the proposed method achieves the best performance with $\\epsilon = 1e-3, \\lambda=1.0$. However, no results are provided when $\\lambda>1$.\n\n\n\n[1]: Zheng et al., Improving the Robustness of Deep Neural Networks via Stability Training, CVPR 2016\n\n[2]: Chen et al., A Simple Framework for Contrastive Learning of Visual Representations, ICML 2020.\n\n\n", "summary_of_the_paper": "This paper proposes to leverage self-supervision to improve imbalanced regression. An augmentation method is designed to keep the regression target after augmentation close to the original target.", "summary_of_the_review": "I recommend a borderline acceptance. Although the motivation of the paper is not fully justified and some discussions are too limited to a specific domain (random noise augmentation), the paper explores a new topic and provides a technically sound solution. My overall attitude towards the paper is positive.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DQ74N9LJbY", "reviewer": "Reviewer_hPS1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positive points:\n1. This paper studies an important and practical task, namely long-tailed regression. The importance of this task can also be supported by a recent survey of deep long-tailed learning [1], which highlights long-tailed regression as an important future direction. \n\n\n2. This work explores contrastive self-supervised learning (CSL) for imbalanced regression, and finds the difference between CSL used for classification and CSL used for regression. That is, the regression is more sensitive to data augmentation (i.e., adding noise).  This may inspire future research when using CSL for imbalanced regression. \n\n\n3. This paper broadens the concept of similarity and dissimilarity between noise samples and original samples from classification to regression, and develops a new self-supervised learning based method for long-tailed regression. Extensive results verify the effectiveness of the proposed method.\n\n\nNegative points: \n1. In the definition of similarity and dissimilarity, it is unclear whether the model should be well-trained when evaluating the similarity or dissimilarity of two samples. Moreover, what is the distance function D? Do different distances influence a lot? The definition of the similarity and dissimilarity in the regression would be more solid if the authors can make these concepts more clear.\n\n\n2. It is unclear why the proposed self-supervised regression helps to handle imbalance. Based on the current method, it seems that self-supervised regression helps to train a more noise-invariant model. Although such an operation may empirically improve the performance of long-tailed regression (on some baselines in experiments), it is unclear how it helps to handle class imbalance. Does it learn a more balanced feature space? I am okay if the authors want to leave this as future work, but I still expect the authors to discuss it. \n\n\n3. Since this study explored a self-supervised learning based method for long-tailed regression and discussed contrastive self-supervised learning, I really suggest the authors to review more recent contrastive learning based long-tailed studies, in addition to the mentioned KCL (Kang 2021). According to the survey [1], contrastive learning based long-tailed methods also include Hybrid [2], PaCo[3] and DRO-LT [4].\n\n\n\nMinor suggestions:\n1. In the abstract, the noise is not defined and may confuse readers. The writing can be further improved.\n\n\n2. The caption of figure 1 is a little confused, and I suggest the authors further improve it.\n\n\n3. In the first paragraph of Section 5, it would be better to add citations for baselines.\n\n\nReferences:\n\n[1] Deep long-tailed learning: A survey. ArXiv, 2021.\n\n[2] Contrastive learning based hybrid networks for long-tailed image classification. In CVPR, 2021.\n\n[3] Parametric contrastive learning. In ICCV, 2021.\n\n[4] Distributional robustness loss for long-tail learning. In ICCV, 2021.\n\n", "summary_of_the_paper": "This paper studies long-tailed regression and explores self-supervised learning to alleviate class imbalance. More specifically, the work broadens the concept of similarity and dissimilarity between noise samples and original samples from classification to regression, and develops a new self-supervised learning based method for long-tailed regression.  ", "summary_of_the_review": "Overall, I like this paper because of its task importance and the new formulation of self-supervised learning for long-tailed regression. If the authors can fix the negative points mentioned above, this paper would be more solid and thorough. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "2VZXLD_M4jg", "reviewer": "Reviewer_TyLX", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\n[1] The imbalanced regression problem studied by this paper is of great significance not only to the academic research but also to the real-world applications. \n\n[2] Authors did comprehensive experiments on several popular benchmarks to show the effectiveness of the paper.\n\n[3] How to do noise generation, which ensures a high degree of similarity between the disturbed sample and the original sample, is very intuitive and easy to follow.\n\nCons:\n\n[1] The performance improvements to current state-of-the-art method is quite marginal. For NYUD2-DIR dataset, the improvements to RMSE and delta1 are 0.05 and 0.0, respectively. For AgeDB-DIR dataset, the improvements to MAE and EM are 0.12 and 0.16, respectively. For IMDB-WIKI-DIR dataset, the improvements to MAE and GM are 0.15 and 0.11, respectively. \n\n[2] The performance of the proposed method on few-shot classes is much worse than the previous SOTA on IMDB-WIKI-DIR dataset, as shown in Table 1. Is there any reason to explain it? \n\n[3] In Figure 1(a) and Figure 2(b), the author drew several conclusions about noise-related issues, including the sensitivity to noise and the impact of noise, based only on one selected sample. Such a special example is difficult to be convincing enough, especially when we are dealing with regression tasks that contain many outlier/noisy samples. My suggestion is that the author can try to provide statistical analysis on the overall datasets, and quantitatively compare the benefits of noise generation method against vanilla random noise method. \n\n[4] The idea of integrating self-supervised learning with imbalanced recognition has been investigated by previous works (1). Although this paper is working on a different task, i.e. regression, the technical contribution of how to integrate self-supervised learning with imbalanced learning is relatively limited.\n\n[5] There are several typos in the paper, for example, \"Update ri by by taking\" in the algorithm workflow \"Algorithm 1: Self-Supervised Imbalanced Regression (SSIR)\" should be \"Update ri by taking\". Authors may want to double check the draft.\n\n(1) Rethinking the Value of Labels for Improving Class-Imbalanced Learning. NuerIPs 2020.", "summary_of_the_paper": "This paper researched on the recently proposed long-tailed regression problem with self-supervised learning method. Two questions are investigated by this paper: 1) how to measure the similarity and dissimilarity under the regression sense; 2) it is not guaranteed that the sampled with perturbations are similar to the original samples without any noise. The former problem is addressed by providing a formal definition of similarity and the later question is addressed by limiting the volume of noise on the output. Authors report the results on three datasets to demonstrate the superiority of our proposed methods, including IMDN-WIKI-DIR, AgeDB-DIR and NYUD2-DIR.", "summary_of_the_review": "Although the imbalanced regression problem is very interesting and import, I have concerns on the technical contribution and effectiveness of the method. Therefore, my current rating is: \"5: marginally below the acceptance threshold\"", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to use self-supervised learning in the context of "imbalanced regression", where some values of the outcome variables are rare, such as in long-tailed regression. The author's proposal can be interpreted as a Monte Carlo approximation of a density smoothing technique, akin to Yang et al. 2021. They test their approach on three datasets. Overall, it provides marginal improvements, whose statistical significance are not assessed. All reviewers agreed that the paper has merits but that it should be further improved to demonstrate that the proposed method is indeed a step forward  in solving the problem of imbalanced regression. The authors should also provide stronger motivation for their pipeline details and experimental setup choices. I therefore recommend rejection, with encouragement for improvement in two directions: strengthening the experimental section, in particular by assessing statistical significance, and by improving the writing of the paper by developing a more rigorous exposition.
Decision: Reject