RankUp: Boosting Semi-Supervised Regression with an Auxiliary Ranking Classifier
Authors: Pin-Yen Huang
Szu-Wei Fu
Yu Tsao
Venue: NeurIPS 2024 poster (2024)
Keywords: Semi-Supervised Learning
Weakly Supervised Learning
Regression
Primary area: other
TL;DR: RankUp adapts semi-supervised classification techniques for regression by converting regression to a ranking problem.

Abstract: State-of-the-art (SOTA) semi-supervised learning techniques, such as FixMatch and it's variants, have demonstrated impressive performance in classification tasks. However, these methods are not directly applicable to regression tasks. In this paper, we present RankUp, a simple yet effective approach that adapts existing semi-supervised classification techniques to enhance the performance of regression tasks. RankUp achieves this by converting the original regression task into a ranking problem and training it concurrently with the original regression objective. This auxiliary ranking classifier outputs a classification result, thus enabling integration with existing semi-supervised classification methods.  Moreover, we introduce regression distribution alignment (RDA), a complementary technique that further enhances RankUp's performance by refining pseudo-labels through distribution alignment. Despite its simplicity, RankUp, with or without RDA, achieves SOTA results in across a range of regression benchmarks, including computer vision, audio, and natural language processing tasks. Our code and log data are open-sourced at [https://github.com/pm25/semi-supervised-regression](https://github.com/pm25/semi-supervised-regression).

RankUp: Boosting Semi-Supervised Regression with an Auxiliary Ranking Classifier
Pin-Yen Huang Academia Sinica
Szu-Wei Fu NVIDIA
Yu Tsao Academia Sinica
Taipei, Taiwan pyhuang37@gmail.com
Taipei, Taiwan szuweif@nvia wewiiiiiiiiii.......
Taipei, Taiwan
yu. ww.tsso@citii sini ca. edu.ua.eeeee.... tw
Abstract
State-of-the-art (SOTA) semi-supervised learning techniques, such as FixMatch and it's variants, have demonstrated impressive performance in classification tasks. However, these methods not directly applicable regression tasks. In this paper, we present RankUp, a simple yet effective approach that adapts existing semi- supervised classification techniques to enhance the performance of regression tasks. RankUp achieves this by converting the original regression task into a ranking problem and training it concurrently with the original regression objective. This auxiliary ranking classifier outputs a classification result, thus enabling integration with existing semi-supervised classification methods. Morcover, introduce regression distribution alignment (RDA), complementary technique that further enhances RankUp's performance by refining pseudo-labels through distribution alignment. Despite its simplicity, RankUp, with or without RDA, achieves SOTA results in across range of regression benchmarks, including computer vision, audio, and natural language processing tasks. Our code and log data are open- sourced at ht/https:/ / thub. com/pm25/sent http:/eeeisuureeeeeeeeeeeeeeeeee wup.eeiissddr-rereessssi.
Introduction
The effectiveness of deep learning models heavily depends on the availability of labeled data. However, obtaining labeled data can be challenging in various scenarios. For instance, tasks like quality assessment often require multiple human annotators label single data point [T5] [9] resulting in labor-intensive and time-consuming process. In domains where expert annotation frequently required, such as medical data, the cost of acquiring labeled data can be extremely expensive [14 21 36). To address these challenges, semi-supervised learning provides powerful
approach to reduce reliance on labeled data for training deep learning models [22 17 24 By
effectively leveraging the unlabeled data during model training, semi-supervised learning provides means to enhance model performance while minimizing the need for extensive labeled data.
Recent state-of-the-art (SOTA) semi-supervised learning methods, such as FixMatch and its variants, use confidence threshold technique to obtain high-quality pseudo-labels 22
This approach involves generating pseudo-labed from unlabeled data and then filtering out those with low confidence scores. The model is then trained produce predictions consistent with these high-quality pseudo-labels. Despite its success across various classification tasks, directly applying this technique to regression tasks encounters several challenges. First, unlike classification models, regression models typically lack confidence measures for their predictions, making the confidence threshold technique unfeasible. Additionally, one of the motivations behind using pseudo-labels to increase the model's confidence in its predictions for unlabeled data, based on the low-density assumption (6, (25)). However, as there are no confidence measures the predictions of regression models, relying on the low-density assumption and increasing the confidence of unlabeled data becomes unfeasible.
34 27
4
3T
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Prediction
Confidence Threshold
?
Weak Augmentation
Neural Network
?
Yes Prediction Pseudo-label
Unlabeled
?
Strong Augmentation
Neural
Yes No
Yes No
Network
Figure I: Illustration of using FixMatch on Auxiliary Ranking Classifier (ARC). This diagram uses the age estimation task example, where the goal is predict the age of person in an image. The auxiliary ranking classifier transforms this task into a ranking problem by comparing two images to determine which person is older. (Image sourced from the UTKFaae dataset (37)).
In this paper, introduce RankUp, simple yet effective semi-supervised regression framework that leverages existing superviee classification methods. RankUp achieves this by using an auxiliary ranking classifier, which concurrently solves a ranking task alongside the original regression task. The ranking task is derived from the original regression problem, where the objective is to compare the labels of pairs of samples determine their relative rank (i.e., which one is larger or smaller). Since ranking problem is type of classification problem, existing semi-supervised classification methods can be applied to assist in training the auxiliary ranking classifier (see Fig. [
Our empirical results demonstrate that enhancing the performance of the auxiliary ranking classifier also improves the performance of the original regression task, as measured by metrics, such as mean absolute еггог (MAE) [29]. coefficient of determination (R²) [23]. and Spearman rank correlation coefficient (SRCC) 33 Furthermore, we show that applying existing semi-supervised classification methods to the auxiliary ranking classifier can effectively utilize unlabeled data, leading to further improvements in the classifier's performance. This improvement, turn, translates enhanced performance in the original regression task, showcasing the potential of applying cuiissuperiieed classification techniques to enhance regression models.
One of the key advantages of using the auxiliary ranking classifier its ability to enhance the ranking relationship of pseudo-labels. Building upon this effect, we propose novel Regression Distribution Aligmment (RDA) method, designed to further improve RankUp's performance by refining the distribution of regression pscudo-labels. RDA adjusts the distribution these pseudo-labels better align with the true underlying distribution of the unlabeled data. This approach assumes that the distributions of labeled and unlabeled data are similar, allowing us estimate the distribution of the unlabeled data based on that of the labeled data distribution. This assumption holds true in many cases, especially when labeled data are randomly sampled from the same pool the unlabeled data. By aligning these distributions, RDA improves the quality of the pseudo-labels, ultimately leading to better model performance when training with these refined pseudo-labels.
Our experimental results demonstrate that RankUp, even without RDA, achieves state-of-the-art (SOTA) results across a variety of regression datasets, including tasks in computer vision, audio, and natural language processing. Moreover, integrating RDA with RankUp provides an additional performance boost, leading the highest performance observed our experiments. For example, Ra alone achieves at least a 13% improvement in MAE and a 28% improvement in R compared SOTA methods on the image age estimation dataset (UTKFace) with 50 labeled samples. The addition of RDA further boosts these results by additional 6% and 7% in MAE and R², respectively. The empirical results our experiments demonstrate that existing semi-supervised classification methods can be effectively leveraged to improve the performance of semi-supervised regression tasks. These findings bridge the gap between future research in semi-supervised regression and classification, paving the way for further advancements in the field.

---

2
Related Works
In this section, we review related research in semi-supervised learning. We categorize the literature into two groups: methods applicable to regression tasks, which will be discussed in Section 2.1 and methods applicable only to classification tasks, detailed in Section 2.2
2.1 Semi-Supprvised Regression
In semi-supervised regression, methods commonly rely on the smoothness assumption 6 25 which suggests that nearby data points in the feature space should share similar labels. Consistency regularization is a popular technique employed to achieve this assumption. It encourages models to generate consistent predictions for slightly perturbed data.
For example, the II-model [7 applies data augmentation to unlabeled data and minimizes the squared difference between the predictions of the augmented data and their original counterparts. Techniques like Mean Teacher (24) involve model-weight ensembling to align the predictions of the model with its ensemble counterpart. Similarly, UCVME [10] employs a bayesian neural network to ensure consistency in uncertainty predictions across co co-trained models. Additionally, CLSS utilizes contrastive learning to encourage features of similar labels to be closer together.
2.2 SuperSupee Classification
In semi-supervised classification, in addition to the smoothness assumption, another commonly relied- upon assumption is the low-density assumption 66 25). This assumption suggests that classifier's decision boundary should ideally pass through low-density regions the feature space. Pseudo- labeling (18) is common approach used to achieve this assumption, where the highest probability class predictions on unlabeled data are utilized as pseudo-labels for training. By incorporating pseudo-labels, the model's confidence in predicting unlabeled data is increased, effectively pushing the decision boundary away from high-density regions towards low-density regions.
Recent SOTA semi remi-supervised learning methods combine pseudo-labeling with consistency regular- ization to achieve both the lew-density and smoothness assumptions, leading to improved performance. For example, MixMatch 3 utilizes a mixup (35] technique and averages predictions from multiple augmented instances to ensure consistency, while also using a sharpening technique to boost predic- tion confidence on unlabeled data. Similarly, FixMatch [22) builds on this concept by generating high-quality pseudo-labels from weakly augmented unlabeled data using a confidence threshold and enforcing consistency between weakly and strongly augmented versions of the same input.
Despite the success of these methods on classification tasks. The low-density assumption doesn't directly translate to regression tasks, as regression models lack explicit confidence measures and decision boundaries like those in classification models. As a result, existing semi sunissupervieed learning methods based on the low-density assumption cannot be directly applied in regression settings.
3 Method
The proposed framework, RankUp, introduces two additional components: ARC and RDA. The design of ARC is inspired by RankNet (5). To provide clear understanding of the ARC's implemen- tation, we first present background information on RankNet in Section 3.1 Subsequently, we will detail the implementation of ARC in Section 3.2 and introduce RDA in Section 3.3 Furthermore, we propose a warm-up scheme and techniques for reducing the computational time of RDA in Sections 3.4 and 3.5 respectively. Lastly, outline the complete RankUp framework in Section 3.6
3.1 Background: RankNet
RankNet [5] is deep learning model designed to predict the relevance scores of documents. The core idea behind RankNet is the use of a pairwise ranking loss. It compares two samples and predicts their relative ranking (i.e., which document more relevant). This approach effectively transforms the relevance score prediction task into pairwise classification problem. In the following, we will provide a detailed explanation of how the pairwise ranking prediction performed and how the corresponding loss is calculated.
3

---

Pairwise Ranking Prediction. The output of RankNet is a single scalar value indicating the ranking score of the sample, where a higher score indicates greater relevance. To obtain the pairwise ranking prediction, two samples are fed separately into the model to get their respective ranking scores. The difference between these scores then passed through sigmoid function, which generates a prediction in the range [0, 1]. This prediction indicates the likelihood that the first sample is more relevant than the second. If the output is greater than 0.5, the model predicts that the first sample has higher relevance; if the output is less than 0.5, the second sample is considered more relevant. Mathematically, for two samples, I; and Tj, and the RankNet model g. the formula to obtain the pairwise ranking prediction Pij is follows:
1
Pij sigmoid(g(x.) g(2;)) 1 e (o(xx) (1(()=)-((())))))))
(I)
Here, g(z;) and g(ar) represent the ranking scores for samples 2; and zy. respectively. A higher value of Pij indicates higher likelihood that the ranking of x; will be higher than that of zz-
Pairwise Ranking Loss. The pairwise ranking loss calculated by comparing the model's predicted pairwise ranking Piş with the ground truth label Maj. The label Bij indicates the true relative ranking between samples H; and Zj. Specifically, Vyj 1 indicates that sample z; is ranked higher than sample zj. Mkj 0 indicates that sample æ; is ranked lower than 2z, and Ykj = 0.5 suggests the two samples are equally ranked. Since this is fundamentally a binary classification task, the pairwise ranking loss is calculated using the cross-entropy loss function. Mathematically, the pairwise ranking loss for batch of data is defined as follows:
N
N CE(y. Pij)
1
granknet
N2
;=1
Here, N denotes the batch size, CE is the cross-entropy loss function, Pty is the predicted pairwise ranking between samples zy and zj,, and yij is the corresponding ground truth label. The loss iterates through all possible pairs of samples in the hatch to calculate the average loss for the entire hatch.
3.2 Auxiliary Ranking Classifier (ARC)
The Auxiliary Ranking Classifter (ARC) is designed to solve ranking task alongside the primary regression task. It can be easily integrated into existing regression model architectures like ResNet [13, BERT [1) or Whisper [20). ARC is implemented an additional output layer that shares the same hidden layers with the original regression model. This transforms the model into a multi-task architecture with two output tasks: the original output header continues to provide the regression output, while ARC generates a ranking score for the sample.
The core idea behind ARC is to transform the original regression task into multi-class classification problem, allowing existing semi-supervised classification methods to assist in its training. To achieve this, ARC's design is inspired by RankNet, which can effectively convert the regression task into binary classification task. However, since multi-class classification output is required, we introduce two key modifications to RankNet adapt it for this purpose:
1. The scalar output value of RankNet is changed to two-class output, where each output class indicates which sample in a pair has a relatively greater ranking score.
2. The sigmoid function is replaced with softmax, which converts the model's output into a multi-class classification probability distribution.
Specifically, for the auxiliary ranking classifier r, which outputs two-class output, the formula to obtain the pairwise ranking prediction P.j of two data samples, xy and æy. is as follows:
Puj = softmax(r(a;) (a00)-(((r))) r(x;))
Here, P.g represents a two-class prediction that indicates which sample in the pair has relatively higher regression label. The loss calculation for remains the same described in Equation

---

This output format enables the integration of existing semi-superriied classification methods. We utilize FixMatch [22] as the semi-supervised classification technique for training ARC. An illustrative example of applying FixMatch to ARC can be found in Figure I with further details in Algorithm
Algorithm Auxiliary Ranking Classifier (with Fix.Match)
Input: Labeled batch X = (ai, yi) Jre, unlabeled batch U { us k=1" confidence threshold unlabeled loss weight with weak augmentation A strong augmentation A,
1: lu = (2 E Σソ= CE (softmax (r(mmaxx((Aw( - (2,)), 1ys > wit) Compute
cross-entropy labeled loss }
2: buth = 0 Initialize unlabeled loss
3: for 1 to Nus do
4: for j = 1 to Nutb do
5:
Рřj softmax A ( (u:)) (u Predict weak pairwise ranking
6:
PG softmax (T (A.A r (u;))) Predict strong pairwise ranking
7:
Lulb Lubs + 1 (max(p:)) (Ptr) > 7} CB(arg max(f Piy) { Accumulate unlabeled loss
8:
end for
9:
end for
10:
Latb (Nuuyy bulb { Average unlabeled loss
11: return fure = lu Walb luib
3.3 Regression Distribution Alignment (RDA)
Distribution alignment is a commonly used technique in semi-supervised classification 16 28 4 where pseudo-labels are refined by aligning their distribution with that of the labeled data. Training semi-supervised models with these refined pseudo-labels can lead to performance improvements. However, existing distribution alignment methods are designed for classification tasks involving discrete label distributions. making them unsuitable for regression settings. Moreover, applying distribution alignment to ARC impractical, as the two output classes always have equal proportions. To address these challenges, we propose Regression Distribution Alignment (RDA), enabling the direct application of distribution alignment to regression tasks.
The RDA process involves three key steps: (1) extracting the labeled data distribution, (2) generating the pseudo-label distribution, and (3) aligning the pseudo-label distribution with the labeled data distribution. These steps correspond to the orange, blue, and yellow parts of Figure 2 respectively.
Step 1: Extracting the Labeled Data Distribution. The labeled data is sorted according its label values. To ensure one-to-one correspondence with the pseudo-labed the labeled data distribution must contain the same number of data points the pseudo-labe label set. We use linear interpolation to extend the labeled distribution to match the size of the pseudo-labels.
Step 2: Generating the Eseudo-Label Distribution. The model generates pseudo-labels for all the unlabeled data. These pseudo-labels are then sorted by their values, either ascending or descending order, as long as the sorting direction is consistent with that of the labeled data distribution.
Step 3: Aligning the Distributions. Once both distributions have been sorted and resized the
same size, the alignment is performed by replacing each pseudo-label value with its corresponding value from the labeled data distribution.
In each training iteration, RDA is applied to refine the pseudo-labels. The loss is then computed between the model's predictions and the RDA-aligned pseudo-labels to minimize their difference. For an unlabeled data point u, with its corresponding regression prediction ÿ; and RDA-aligned pseudo-label Ür the RDA loss lda for batch of unlabeled data is defined as:
Nat Leeg (Dt, ū:)
Edda
(4)
Nuth
Here, Naa denotes the batch size of unlabeled data, Treg represents regression loss function (e.g.. MAE, MSE).
5

---

Sorting
Interpolation
Labeled Data
Sorted Labeled Data
Labeled Data Distribution
Sorting
Distribution Alignment
UD ui U2 u3 u4
04 UD uZ u3 UI
Pseudo-Labels
Bseudo-Labels Distribution
Aligned Pseudo-Labels
Figure 2: Illustration of RDA: This example includes three labeled data pairs {(r,, y4 yi) and five unlabeled data points with corresponding pseudo-labels ((us; út) Each data pair is represented by single bar in the graph. The x-axis indicates the sample indices, while the y-axis represents their corresponding regression label values. The orange bars demonstrate the process of obtaining the labeled data distribution, the blue bars illustrate how the pseudo-labe distribution is formed, and the yellow bars show the aligned pscudo-labela after applying RDA.
The design of RDA based on two key assumptions. First, assumes that the distributions of labeled and unlabeled data are similar, which is often true since labeled data is typically randomly sampled from the unlabeled pool. Second, assumes that the ranking of the pseudo-labels is reasonably accurate. Integrating RDA with ARC can reinforce this assumption, as ARC enhances the ranking relationships of pseudo-labels. Both assumptions are crucial for ensuring that RDA works properly.
3.4 Warm-Up Scheme for RDA
In the early stages of training, the pseudo-label rankings may be poorly predicted, which can degrade the quality of the pseudo-labele refined through RDA. To address this, we introduce linear warm-up scheme to stabilize the RDA process. The adjusted RDA loss, fa is defined as follows:
iitrr
l min
1.0) fni
OOwarm
Here, iter denotes the current training iteration, and Owarm is hyperparameter that controls the duration the warm-up phase. The min function ensures that the warm-up factor does not exceed 1.0, smoothly transitioning the model toward the full effect of L,aa-
3.5 Reducing Computational Time of RDA
Applying RDA can be computationally expensive, as it requires inference all unlabeled data and sorting all pseudo-labels every training iteration. This significantly increases the computational load compared to the original training process, especially when dealing with a large volume of unlabeled data, making the implementation of RDA impractical. To mitigate this challenge, propose several techniques aimed at reducing the computational burden of RDA.
Pseudo-label table. RDA creates table of the same size the unlabeled dataset. This table stores the model's predicted pscudo-labela for each instance of unlabeled data. For each training iteration, the model generates new pscudo-labels, which stored and updated within this table. This approach climinates the need to rerun inference on all unlabeled data when applying RDA, as it only requires a simple lookup from the pseudo-label table.
Applying RDA only every T steps. To further reduce computational costs, RDA applied only every T' steps, where T is hyperparameter. This is achieved by creating second table of the same size as unlabeled dataset, which stores the previously aligned results applying RDA. Between RDA updates, the model uses these stored aligned pseudo-labels, thereby avoiding the need to run RDA in every iteration. This strategy effectively reduces the computational cost associated with the RDA process to 1/T.
pseudo-labels generated by

---

3.6 Putting It All Together- RankUp
We introduce the term RankUp to describe our proposed framework, which integrates two key components: ARC and RDA. The use of RDA is optional, depending whether its underlying assumptions are satisfied. The final loss for Ra is a combination of the regression loss and the ARC loss. The regression loss consists of the original labeled regression loss plus the unlabeled RDA loss. Specifically, the RankUp loss frankup is defined as follows:
Taakup (Freg + Wedd (dda) + (Ware Parc)
(6)
In this equation, kreg represents the loss from the original labeled regression task, while eiin denotes the RDA loss, as defined in Equation 5 The hyperparameter widh controls the weight of the RDA loss. If RDA is not employed, the term Wida Li can be excluded from the equation. The term aaee corresponds to the loss from the ARC module, detailed in Algorithm T Additionally, the hyperparameter ware regulates the weight of the ARC loss.
4 Experiments
In this section, evaluate RankUp's performance across various tasks. The experimental settings are described in Section 4.1 The main results for RankUp under different label configurations are presented in Section 4.2 while Section 4.3 provides additional results audio and text datasets. 4ectionn4..eepllree the use of alternative semi-supervised classification methods in place of FixMMath Lastly, we discuss potential reasons why the smoothness and low-density assumptions are also effective for regression tasks in Section 4.5
4.1 Settings
Evaluation Metrics. We use three evaluation metrics: MAE, R², and SRCC, to assess the per- formance of semi-supervised regression methods. MAE measures the average absolute difference between the model's predictions and the actual values. The R² score indicates the proportion of variance in the data explained by the model. SRCC evaluates the correlation between the predicted rankings and the actual rankings.
Evaluation Robustness. To ensure the reliability of our evaluation results, each experiment is executed three times using fixed random seeds (0, 1, and 2). We report both the mean and standard deviation of each metric.
Fair Comparison. To ensure a fair comparison between our proposed methods and related works, we implement and evaluate all methods within the same codebase. Specifically, adapt the popular semi- supervised classification framework USB (26). modifying it for regression tasks to implement both our proposed methods and related works. Weak augmentation is applied consistently to the labeled data across all semi-supervised and supervised methods. For specific details on the modifications made to USB, please refer Appendix A.5| The code and full training logs of the experiments presented in this paper are open- sourced at
regressior
Hyperparameters. We use the hyperparameters of USB as the base for fine-tuning. We first fine-tuna the hyperparameters in the supervised baseline setting and find the hyperparameters that lead to lowest MAE score. These same hyperparameters to ensure a fair comparison. Only the additional hyperparameters specific to each semi-supervised method are further tuned. For more details on the hyperparameters, please refer to Appendix A.13
then applied all semi-supervised regression methods
Base Model. The base model used in our experiments varies depending on the data type. For image data, we use Wide ResNet-28-2 (32 which is not pre-trained. For audio data, we use the pre-trained Whisper-base 20). and for text data, we use the pre-trained Bert-Small [1].
Dataset. To simulate the semi-supervised setting, we randomly sample portion of the dataset as labeled data, treating the remainder as unlabeled. To evaluate performance, we use three diverse datasets: FTKeaae (77, an image age estimation dataset; BVCC [8) audio quality assessment dataset; and Yelp Review [ a text sentiment analysis (opinion mining) dataset. For more detailed information about these datasets, please refer to Appendix A.II
7

---

Table 1: Comparison of RankUp with and without RDA against other methods on the UTKFace dataset, evaluated under two settings: 50 and 250 labeled samples, with the remaining images treated as unlabeled. The original UTKFaaee dataset comprises 18,964 training images.
UTKFaace (Image Age Estimation)
Labels = 50
Labels = 250
MAET
R²1
SRCCT
MAEJ
R²T
SRCC1
Supervised
14. 2...6
0.090,0002
a.37
9.42a0.16
0,540z0.014
0.712a0010
II-Model
13.82
0.100
0.387a0092
9. 9.45
0,534
0.706za015
Mean Teacher
13.92+0.20
0.127-0037
0.423+0.02
8.85+0.25
0.586+0.020
0. 0.745+0013
CLSS
13.6120.92
0.138
0.44720.074
9.1 0.1020.15
0,58620.016
0.737
UCVME
13.49
0.157
8.6 8.63
0.6 0.626
0. 0.767
MixMatch
11. 11.44x0.45
0.401
0.674+0.035
7.95+0.15
0.692-0.013
0.832+0008
Ra (Ours)
9. 9,9622.22
0.514
0.703e0..09
7.06+0.11
0.751
0. 0.835+0008
RankUp + RDA (Ours)
9,33a054
0.552.0041
0.770z0.009
6.57
0.782:6.012
0.85620.005
Fully-Supervised
4.85.0m1
0.875-0.00
0.910+a.001
4.85
0.875+0.00
0.910+a.01
4.2
Main Results
To evaluate the performance of Ra under different labeled data settings, we conducted experi- ments using the UTKFace dataset with 50 and 250 labeled samples. We tested two configurations of RankUp: one incorporating the RDA (RankUp + RDA) and the other without it (RankUp). Their performance was compared against other semi-supervised regression methods, with MixMatch specif- ically representing the consistency regularization component of the approach. Additionally, we included supervised setting that used only the labeled data without incorporating any unlabeled data during training, as well as fully-supervised setting that used all available data (both labeled and unlabeled), assuming the unlabeled data had known true labels. We also conducted experiments with 2000 labeled samples setting; however, due to space limitations, the results for this configuration can be found in Appendix A..
The results are presented in Table T We observed that RankUp (without RDA) consistently outper- forms existing semi-supervised regression methods, especially when the amount of labeled data is scarce. Specifically, in the 50-label setting, RankUp achieves at least 12.9% improvement in MAE, a 28.2% improvement in R², and 4.3% improvement in SRCC compared to other semi-supervised regression methods. In the 250-label setting, Ra shows at least an 1 11.2% improvement in MAE, an 8.5% improvement in R², and 0.4% improvement SRCC.
Furthermore, the integration of RDA with RankUp further enhances the performance of RankUp. Specifically, in the 50-label setting, RankUp RDA achieves an additional 6.3% improvement in MAE, a 7.4% improvement in R2, and a 9.5% improvement in SRCC compared to RankUp alone. Similarly, in the 250-label setting, RankUp RDA achieves an additional 6.9% improvement in MAE, a 4. 1% improvement in R², and 2.5% improvement in SRCC relative to RankUp.
These empirical results demonstrate the effectiveness of Rank Up and RDA across different labeled settings. Another notable observation is that RankUp+ + RDA in the 50-label setting outperforms the supervised model that utilizes five times the labeled data (in the 250-label setting) across all three metrics. Specifically, Ra + RDA achieves a 1.0% improvement MAE, a 2.2% improvement in R², and 8.1 % improvement in SRCC while using only one-fifth of the labeled data, demonstrating its effectiveness in reducing labeling costs.
4.3 Additional Results on Audio and Text Data
To further assess the performance of RankUp across different data types and tasks, we evaluated it on the BVCC and Yelp Review datasets using 250 labeled samples. The results are presented in T The table demonstrates that RankUp also consistently outperforms existing semi-supervised regression methods on both audio and text datasets. Specifically, on the BVCC dataset, Rank (without RDA) achieves at least a 5.6% improvement in MAF, a 6.3% improvement in R², and a 0.3% improvement in SRCC compared to other semi-supervised regression methods. On the Yelp
8

---

Table 2: Comparison of RankUp with and without RDA against other methods on the BVCC and Yelp Review datasets, evaluated under the 250-labeled samples setting. The BVCC dataset consists of 4,975 training audio samples, while the Yelp Review dataset contains 250,000 training text comments.
BVCC (Audio Quality Assessment)
Yelp Review (NLP Opinion Mining)
Labels = 250
Labels = 250
MAEI
R-1
SRCCT
MAEL
R-1
SRCCT
Supervised
2.533
0.49020018
0.741
0.723
0.56620.019
0.76920.010
I1-Model
0.534
0.489
0. 0.740
0.730a0.24
0.565an.19
0.76920.009
Mean Teacher
0.532-0006
0.492-0018
0. +0.44
0.730+0.004
0.565+0.019
0.769+0.009
CLSS
0,49920010
0.534.0.027
0.74 0.74820.00s
0.721
0.543a0.011
0.74820.002
UCVME
0.553
0.7 0.774
0.775
0.540sna0s
0.763a0.00s
MixMatch
0.597-0017
0.353+0044
0.626+0.031
0.886+0004
0.38 +0.005
O. 0.660+6.004
RankUp (Ours)
0.470=0012
0.588=a
0.776H1.10
0.661
0.645.0.01
0.829+0.m0
RankUp + RDA (Ours)
0.463:0013
0.598
0.783x0.011
0.632200009
0.651 100000
0.81 00000000
Fully-Supervised
0.351-000s
0.764-0002
0.874+0.001
0.
0.799+0.002
0.896+0.011
Table 3: Comparison of using different semi- cemi-supervised classification methods for training UankUp's ARC component. Results are evaluated on the UTKFace dataset with a setting of 250 labeled samples.
MAEJ
R²T
SRCCT
None
9,42a0.16
0,540s0.014
0.712z0010
Supervised
9.03em00
0.588
0.746xaox8
I1-Model
8.8 8.81an.11
20.01
0.75 20002
Mean Teacher
8. 8.76+a.13
0.607-0.088
0.750=a00s
FixMatch
7.06.0.11
0.751
0.835
Review dataset, RankUp shows at least a 8.3% improvement in MAE, 14.2% improvement in R?, and 7.8% improvement in SRCC relative other semi-supervised regression methods.
Moreover, integrating RDA with RankUp further enhances Rank performance on both datasets. In the BVCC dataset, RankUp + RDA achieves an additional 1.5% improvement in MAE, 1.7% improvement in R², and a 0.9% improvement SRCC. the Yelp Review dataset, RankUp + RDA shows an additional 4.4% improvement in MAE and 0.9% improvement in R². However, in the Yelp Review dataset, Ra + RDA did not improve SRCC compared to RankUp without RDA, resulting in 2.3% decrease in SRCC. This decline is likely due to the limited distinct label values (only five distinct values) in the Yelp Review dataset; applying RDA may cause many pseudo-labels to align with the same value, thereby disrupting the ranking relationships and leading degradation in SRCC performance.
4.4 Analysis of Different Semi-Supervised Classification Methods ARC
To better understand the effect of RankUp's ARC component and its ability to utilize the unlabeled data, analyzed ARC by training it with various semi-supervised classification methods. In this evaluation, we did not apply RDA to isolate its effect on ARC. Additionally, we compared these methods against baseline with no ARC (denoted as "None") and a supervised setting where only labeled data was used to train ARC (denoted as "Supervised").
The results are presented in Table 3 The table indicates that using an ARC without any unlabeled data can already improve performance, with a 4.1 % improvement in MAE, an 8.9% improvement in R², and a 4.8% improvement in SRCC compared to the "None" setting. This suggests the beneficial impact of training ARC concurrently with the original regression head, even without leveraging unlabeled data.

---

Supervised
MixMatch
RankUp
60
100
40
Dimension
20
20
Dimerslon
20
Label (Age)
-20
-20
40
-40
-60
-40
-50
Dimension
Dimension
Figure 3: Comparison of t-SNE visualizations of feature representations for different semi-supervised regression methods on evaluation data. The supervised model is displayed on the left, MixMatch is in the center, and RankUp (without RDA) is shown on the right.
Furthermore, using different semi-supervised classification methods to utilize the unlabeled data further boost performance compared to using only labeled data. Specifically, tested the IT-Model, Mean Teacher, and FixMatch, all which demonstrated improvements over the "Supervised" setting across MAE, R², and SRCC metries. Among these methods, FixMatch achieved the best results, showing a 21.8% improvement in MAE, 27.7% improvement in R², and an 11.9% improvement in SRCC compared to the "'uupervied"" setting. This highlights the effectiveness of leveraging unlabeled data and semi-supervised classification techniques to improve ARC and RankUp 's overall performance.
4.5 Understanding Smoothness and Low-Density Assumptions in Regression
The low-density assumption is crucial for understanding the effectiveness of semi-supervised learning methods. However, it does not directly apply regression tasks due to the absence of decision boundaries and confidence measures. In this section, we explore why RankUp performs well in regression tasks by leveraging semi-supervised classification techniques that utilize the low-density assumption. This understanding can broaden our perspective on these assumption.
We adopt broader interpretation of the smoothness and low-density assumptions. Rather than viewing them solely in the context of classification, we interpret the smoothness assumption as an effort to group features with similar labels together, while the low-density assumption aims to separate features with dissimilar labels. These perspectives align with RankUp's 's approach, By training ARC with pseudo-labels, the model is encouraged to have greater confidence in the pairwise ranking predictions of the unlabeled data, thus pushing the features with dissimilar pseudo-labels further apart. Additionally, ensuring consistent predictions between weakly and strongly augmented data assists in grouping features with similar labels. The t-SNE visualization demonstrated in Figure 3 supports this claim, showing that within RankUp, similar labels closer together, while dissimilar labels are pushed further apart in the feature space.
5
Conclusion
Recent advancements in semi-supervised learning have achieved impressive results across various classification tasks; however, these methods are not directly applicable to regression tasks. In this work, we investigate the potential of leveraging existing semi-supervised classification techniques for regression tasks. We propose novel framework, RankUp, which introduces two key components: the Auxiliary Ranking Classifier (ARC) and Regression Distribution Alignment (RDA). The empirical results of our experiments demonstrate the effectiveness of our methods across various labeled data settings (50, 250, and 2000 labeled samples) and different types datasets (image, audio, and text). These findings show that semi-supervee supirvupe classification techniques can be effectively adapted to regression tasks, bridging the gap between research in semi-super regression and classification, and paving the way for more advanced research in this area.
10

---

Acknowledgments
I sincerely appreciate everyone who made this work possible. I am especially thankful to my coauthors, Szu-Wei Fu and Prof. Yu Tsao for their mentorship and our weekly discussions; their insights and comments have been invaluable shaping this research. I am also very grateful for my experience at CLLab and the guidance of Hsuan-Tien Lin, which sparked my interest in weakly- supervised learning and greatly inftuenced my research mindset and approach. My heartfelt thanks go to my family for their unwavering support throughout my academic journey. also thank my friend Chi-Chang, whose discussions helped me develop a deeper understanding of how to approach research. Lastly, want to thank Chia-Ling for her continuous support and encouragement during the development of this research. This work would not have been possible without all these individuals. Thank you!
References
[1] Yelp dataset: http://wwww.yll.commdaaaaae
chaallngge...
[2] D. Berthelot, N. Carlini, E. D. Cubuk, A. Kurakin, K. Sohn, H. Zhang, and C. Raffel. Remix- match: Semi-supervised learning with distribution matching and augmentation anchoring. In International Conference on Learning Representations, 2020.
[3] D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, Oliver, and C. A. Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural information processing systems, 32, 2019.
[4] D. Berthelot, R. Roelofs, K. Sohn, N. Carlini, and A. Kurakin. Adamatch: A unified approach semi-supervised learning and domain adaptation. In International Conference on Learning Representations, 2022.
[5] C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender. Learning to rank using gradient descent. In Proceedings of the 22nd international conference on Machine learning, pages 89-96, 2005.
[6] O. Chapelle, B. Scholkopf. and A. Zien. Semi-supervised learning (chapelle, o. et al., eds.; 2006)]bookk reviews]. IEEE Transactions on Neural Networks, 20()):542-522, 2009.
[7] H. Chen, R. Tao, Y. Fan, Y. Wang, J. Wang, B. Schiele, X. Xie, B. Raj. and M. Savvides. Softmatch: Addressing the quantity-quality tradeoff in semi-supervised learning. In The Eleventh International Conference on Learning Representations, 2023.
[8] E. Cooper and J. Yamagishi. How do voices from past speech synthesis challenges compare today? arXiv preprint 1rXiv:2105.023377 2021.
[9] E. D. Cubuk, B. Zoph, Shlens, and Q. Le. Randaugment: Practical automated data augmentation with reduced search space. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition workshops, pages 702-703, 2020.
[10] W. Dai, X. Li, and K.-T. Cheng. Semi-supervised deep regression with uncertainty consistency and variational model ensembling via bayesian neural networks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 7304-7313, 2023.
[11] W. Dai, D. Yao, Bai, K.-T. Cheng, and X. Li. Semi-supervised contrastive learning for deep regression with ordinal rankings from spectral seriation. In Thirty-sevnnth Conference on Neural Information Processing Systems, 2023.
[12] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.048005 2018
[13] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 778, 2016.
11

---

[14] S. C. Hoi, R. Jin, J. Zhu, and M. R. Lyu. Batch mode active learning and its application to medical image classification. In Proceedings of the 23rd international conference on Machine learning, pages 417-424, 2006.
[15] V. Hosu, H. Lin, T. Sziranyi, and D. Saupe. Koniq- 10k: An ecologically valid database for deep learning of blind image quality assessment. IEEE Transactions on Image Processing, 29:4041 -056, 2020.
[16] Kim, Y. Hur. S. Park, E. Yang. S. J. Hwang, and Shin. Distribution aligning refinery of pseudo-label for imbalanced semi-supervised learning. Advances in neural information processing systems, 33:14567-14579, 2020.
[17] S. Laine and T. Aila. Temporal ensembling for semi-supervised learning. In International Conference on Learning Representations, 2017.
[18] D.-H. Lee et al. Pseudo-label: The simple and efficient semi- superviper learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, volume 3, page 896. Atlanta, 2013.
[19] J. Lorenzo-Truehen J. Yamagishi, T. Toda, D. Saito, E. Villavicencio, T. Kinnunen, and Z. Ling. The voice conversion challenge 2018: Promoting development of parallel and nonparallel methods. In The Speaker and Language Recognition Workshop (Odyssey 2018), page 195. ISCA, 2018.
[20] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recognition via large-scale weak supervision. In International Conference Machine Learning, pages 28492-28518. PMLR, 2023.
[21] S. Rahimi, O. Oktay, J. Alvarez- and S. Bharadwaj. Addressing the exorbitant cost of labeling medical images with active learning. In International Conference on Machine Learning in Medical Imaging and Analysis, volume 1, 2021.
[22] K. Sohn, D. Berthelot. N. Carlini, Z. Zhang. H. Zhang, C. A. Raffel, E. D. Cubuk, A. Kurakin, and C.-L. Li. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems, 33:596 008660888 2020.
[23] R. G. D. Steel, J. H. Torrie, al. Principles and procedures of statistics. Principles and procedures of statistics., 1960.
[24] A. Tarvainen and H. Valpola. Mean teachers are better role models: Weight-averaged consis- tency targets improve semi-supervised deep learning results. Advances in neural information processing systems, 30, 2017.
[25] E. Van Engelen and H. H. Hoos. A survey on semi-supervised learning. Machine learning. 109(2):373-440, 2020.
[26] Y. Wang. H. Chen, Y. Fan, W. Sun, R. Tao, W. Hou, R. Wang. L. Yang, Z. Zhou, L.-Z. Guo. H. Qi, Z. Wu, Y.-F Li, S. Nakamura, W. Ye, M. Savvides, B. Raj. T. Shinozaki, B. Schiele, J. Wang, X. Xie, and Y. Zhang. Ush: A unified semi-supervised learning benchmark for classification. In Thirtt--sxthh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
[27] Y. Wang, H. Chen, Q. Heng, W. Hou, Y. Fan, 7. Wu, J. Wang, M. Savvides, Shinozaki, B. Raj. B. Schiele, and X. Xie. Freematch: Self-adaptive thresholding for semi-supervised learning. In International Conference on Learning Representations (ICLR), 2023.
(28) C. Wei, K. Sohn, C. Mellina, A. Yuille, and F. Yang. Crest: A class-rebalancingg self-training framework for imbalanced semi-supervised learning. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 10857-10866, 2021.
[29] C. Willmott and K. Matsuura. Advantages of the mean absolute error (mae) over the root mean square error (rmse) assessing average model performance. Climate research, 30(1):79-82, 2005.
12

---

[30] Q. Xie, Z. Dai, E. Hovy, T. Luong, and Q. Le. Unsupervised data augmentation for consisteney training. Advances in neural information processing systems, 33:6256- 336226-6268, 2020.
[31] Y. Xu, L. Shang, Ye, Q Qian, Y.-F Li, B. Sun, H. Li, and R. Jin. Dash: Semi-supervised learning with dynamic thresholding. In International Conference Machine Learning, pages 11525-1133 11536. PMLR, 2021.
[32] S. Zagoruyko and N. Komodakis. Wide residual networks. arXiv preprint arXiv: 146, 2016.
[33] H. Zar. Significance testing of the spearman rank correlation coefficient. Journal of the American Statistical Association, 67(339):578-580, 1972.
[34] B. Zhang, Y. Wang. W. Hou, H. Wu, J. Wang, M. Okumura, and T. Shinozaki. Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Advances in Neural Information Processing Systems, 34: 18:18-1888111 2021.
[35] H. Zhang, M. Cisse, N. Dauphin, and D. Lopez-Paz. mixup: Beyond empirical risk mini- mization. In International Conference on Learning Representations, 2018.
[36] W. Zhang, L. Zhu, J. Hallinan, S. Zhang, A. Makmur, Q. Cai, and B. C. Ooi. Boostmis: Boosting medical image semi-supervised learning with adaptive pseudo labeling and informative active annotation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 20666-20676, 2022.
[37] Z. Zhang, Y. Song, and H. Qi. Age progression/regrssion by conditional adversarial autoen- coder. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8810-5818, 2017.
13

---

A Appendix supplemental material
A.1 Limitations
One of the limitations of RDA is that it relies two key assumptions: the labeled and unlabeled data have similar label distributions, and the ranking of the pseudo-labels is accurate. If either of these assumptions is not met, the performance of RDA can be significantly impacted.
Another limitation of RDA is that may not perform well tasks where distinct label values are few. As shown in Table 2 in the Yelp Review dataset, SRCC decreases when RankUp is combined with RDA compared to using Ra alone. This is likely because the limited number distinct label values causes many pseudo-labels to align to the same value, disrupting the ranking relationships.
Despite these limitations, RDA still a powerful component that can work with RankUp to further boost its performance. However, users should carefully evaluate their dataset characteristics to determine whether applying RDA is an appropriate choice.
A.2 Analysis on UTKFace 2000 Labeled Setting
In T we present the results of RankUp against other methods using 50 and 250 labeled samples settings on the UTKFace dataset. Here, we further investigate the performance of Rank Up in a 2000- label setting, with the results shown in Table Notably, Rank Up continues to exhibit performance improvements over other semi-supprrvied regression methods in a 2000-label setting. Specifically, RankUp (without RDA) demonstrates a 7.0% improvement in MAE, a 1.7% improvement in R², and 0.5% improvement in SRCC. Meanwhile, RankUp with RDA demonstrates an additional 1.8% improvement in MAE, a 0.7% improvement R², and a 0.3% improvement in SRCC compared to RankUp alone. Comparing the results in [ and 250 labeled samples) with those in Table 4 (2000 labeled samples), we observe that RankUp with or without RDA continue to outperform other methods as the number of labeled samples increases. However, as expected, the advantages of semi-supervised learning diminish with the availability of more labeled training data.
Table 4: Comparison of RankUp with and without RDA against other methods on the UTKFace
dataset with 2000 labeled samples.
UTKFaae (Image Age Estimation)
Labels 2000
MAE,
Ra1
SRCC1
Supervised
6,28an.06
0.79420004
0.862s000
TT-Model
6.3 +0.10
0.790-o006
0.860+n.03
Mcan Teacher
6,29 10093
0.79320004
0.86220001
CLSS
6.29
0.79420003
0.862#n001
UCVME
5.90+n.77
0.821 -00007
0.877+0000
MixMatch
6.03
0.824=0004
0.883+0002
RankUp (Ours)
0.838
0.887
RankUp+ RDA (Ours)
5.5
0.84420004
0.89020008
Sully-Sup
4.85an.01
0.87520000
0.91020001
A.3 Analysis of Diffferntt Semi-Supervised Regression Methods on RankUp
We evaluated the impact of different cuni-super regression methods on training RankUp's regression output. The ARC was trained using FixMatch, while the regression output was trained with various cemi-supervised regression techniques. We compared these results with a supervised setting, where only labeled data was used for training the regression output (denoted as "Supervised"". The results, shown Table indicate that using different semi-supervised regression methods can further improve performance in MAE, R², and SRCC metrics compared to the supervised method alone. Demonstrating the effectiveness of leveraging unlabeled data to directly train the regression head.
14

---

Among all the semi- supervisee regression methods we tested, our proposed RDA achieves the best performance terms of MAE and R², with least 7.7% improvement MAE and a 1.7% improvement in R² compared to other methods. However, it shows decrease in SRCC compared to MixMatch, with a 1.1% % drop. In theory, RDA can also be used alongside the IT-Model, Mean Teacher, and MixMatch, as it focuses of these methods. However, due to increased computational expense and the goal of maintaining a simpler framework, our proposed RankUp only uses supervised method or RDA for training the regression output.
refining pseudo-labels, which is distinct from the approaches
Table 5: Comparison of using different semi-supervised regression methods for training RankUp's regression output. Results are evaluated on UTKe dataset with a setting of 250 labeled samples.
MAEL
R
SRCCT
Supervised
7.06+0.11
00.51
0.835-a008
II-Model
6.95
0.758s0.010
0.837
Mcan Teacher
a0.1
0.752:0.013
0.831 50004
MixMatch
7.12a0.09
0.76920:006
0.86620002
RDA (Ours)
6.57
0.782:0.012
0.85620005
A.4 Ablation Studies on RankUp Components: ARC and RDA
To further understand the effect of ARC and RDA in RankUp, we conducted ablation studies comparing the performance of supervised baseline, RDA only, ARC only, and ARC RDA on the UTKFace dataset, using settings of 50 and 250 labeled samples.
The results, shown in Table 6 demonstrate that RDA alone does not consistently outperform the supervised baseline, particularly the 50 labeled samples setting (as reflected in the MAE and R² values). This may be due to one of the assumptions of RDA not being met, where it requires the pseudo-labels to have reasonably accurate ranking (the supervised bascline with 50 labeled samples only has an SRCC score of 0.371). In contrast, ARC alone ("RankUp" in the table) significantly improves performance over both the supervised baseline and RDA alone. The combination of ARC and RDA yields the best performance, highlighting their synergistic relationship, where it is beneficial to use RDA with ARC.
Table 6: Ablation studies on the ARC and RDA components introduced in RankUp. In this table, "RankUp" refers to the use of the ARC component only, while "RDA" refers to the use of the RDA component only. "RankUp= + RDA" refers to the combined use of both ARC and RDA. Results are evaluated on the UTKKaaee dataset with 50 and 250 labeled samples.
UTKFace (Image Age Estimation)
Labels 50
Labels 250
MAE
R2 f
SRCC
MAE J
R2 f
SRCC
Supervised
14.13
0.090.0022
0.37
9.4220.16
0.540
0.71220010
RDA (Ours)
14.34
0.060so.12
0.442a0.104
8. 8.64
0.609
0.772
RankUp (Ours)
9,9620.22
0.5142001
0. 0.703
7.062a11
0.751
2.
RankUp I RDA (Ours)
9,33+0.54
0.552-0041
0.770+a.09
6.57+0.18
0.782+0.12
0.856=0005
A.5
Modifications for Adapting USB Codebase for Regression Tasks
The USB 26 codebase is originally designed for semi-supervised classification tasks and includes implementations of various existing semi-supervised classification methods. To adapt it for regression tasks, enabling the implementation of our proposed framework RankUp and other semi- cumi-supervieed regression methods, we made the following key modifications:
1. Replaced the cross entropy loss function with the MAE loss function.
15

---

2. Adjusted the output layer to produce single continuous output instead of multiple outputs used for multi-class classification.
3. Removed one-hot encoding from the codebase.
4. Normalized the regression labels the 0-1 range.
5. For methods like Mean Teacher [24] and I1-Model [77, no changes needed to the core algorithms, as they are inherently applicable to regression tasks.
6. For MixMatch [3]. we excluded components specifically designed for classification, such as sharpening and one-hot label encoding, while retaining the input mixing and consistency regularization aspects, which are valuable for regression tasks.
A.6 Influence of Data Augmentation on Labeled Data
To further analyze the effect of weak augmentation on labeled data, we conducted experiments the UTKFaae dataset with a setting of 250 labeled samples, comparing the results with and without weak augmentation applied to the labeled data.
Table 7 presents comparison of the results, illustrating the significance of weak augmentation. The performance metrics (MAF, R², SRCC) show a decline when weak augmentation is not applied (note that weak augmentation was applied to all the baselines in the paper). Despite the observed drop in performance when weak augmentation is not applied, the relative order of effectiveness among the methods remains consistent, regardless the application of weak augmentation. This consistency further underscores the robustness of our proposed method.
Table Analysis of the effect of weak augmentation on labeled data. Results are evaluated on the UTKFaae dataset with setting of 250 labeled samples. Best results for applying or not applying weak augmentation for each method are highlighted bold.
Weak Augmentation
MAE
R2 f
SRCC f
Yes
9.422016
0.540
0.712:0.010
Supervised
No
11.73a0.17
0.3
0.556a0.220
Yes
9.45+030
0.534+0.000
0.706+0.115
11-Model
No
11.97
0.3 0.31920.029
0.56710.026
Yes
7.95-a11
0.692-0.13
0.832+0:008
MixMatch
10. 10.80s0.99
0.446=0.01
0.7 6.0.00
Yes
6.57
0.78220.012
0.85620.005
RankUp + RDA (Ours)
No
7.16s025
0.742=0.02
0.843+0.00
Yes
4.85.00t
0.875.0.000
0.910.0.00
Supervsdde
No
5.58
0.83720.00
0.88810.000
A.7 Data Augmentation Operators
We followed the settings the USB 26 codebase for augmentation operators, with an adjustment to the strong augmentation for audio data. This adjustment was necessary because our task involves quality assessment, and the original strong augmentation method would have affected the quality of the data. Specifically, we used the following augmentation techniques:
Image
Weak augmentation: Random Crop, Random Horizontal Flip
Strong augmentation: RandAugment 9
Audio
Weak augmentation: Random Sub-sample
Strong augmentation: Random Sub-sample, Random Mask, Random Trim, Random Padding
16

---

Text
Weak augmentation: None
Strong augmentation:
Back-Translation
300
A.8
Hardware Specifications
All experiments reported this paper were conducted using an Nvidia Titan XP with 12 GB of VRAM and an Nvidia GefFrcee RTX 2080 Ti, also equipped with 12 GB of VRAM.
A.9 More Feature Visualization Results
Additional feature visualization results beyond those presented in this paper are available. This includes t-SNE and UMAP visualizations, in both 2D and 3D, for different semi-supervised regression methods, accessible at thttps://githur commpm25/ pmmppsemiimmmmmmmmmmmmmmmmmmm semi -
A.10 Dataset Processing
In our experiments, if the dataset provides a pre-defined train-eval-test split, we utilize the training split to train the model and evaluate its performance on the evaluation or test split. the dataset does not provide such split, we randomly sample 80% of the data as the training set and the remaining 20% the test set. We open-source the train-test splits used for conducting the experiments this paper at http:///w.....aaaooooo/////////a
com www.://mpoo/rrgesooooooooooooe datasets
A.11 Datasets
Three datasets are utilized in the experiments conducted in this paper: UTKFace [37, BVCC and the Yelp Review [1 dataset. Below, we provide a brief introduction to each dataset.
UTKFace. The UTKFace dataset is image age estimation dataset, where the goal is to predict the age of the person in an image. The labels range from I to 116 years old. The dataset consists of 23,705 face images, which split into 18,964 training samples and 4,741 test samples. The dataset available two versions: the original images and an aligned and cropped version. The experiments conducted in this paper the aligned and cropped version of the UTKFaaee dataset.
The VoiceMOS2022 (BVCC) dataset an audio quality assessment dataset, where the objective is to predict the quality of audio sample. The labels, ranging from 1 to 5, are obtained by averaging the scores provided by multiple listeners. The dataset is split into training (4,974 samples), evaluation (1,066 samples), and testing (1,066 samples) sets. In the experiments reported in this paper, we utilize only the training and evaluation splits to evaluate performance.
BVCC,
Yelp Review. The Yelp Review dataset is a text opinion mining task, where the goal is to predict the rating of customers based on the comments they leave on the Yelp website. There are only five distinct ratings (0 to 4). We use the processed Yelp Review data provided by the USB [26] codebase. The dataset comprises training (250,000 samples), evaluation (25,000 samples), and testing (10,000 samples) sets. We only utilize the training split for model training and the evaluation set for evaluation.
A.12 Hyperparameter Fine- Sine-Tuning
We began by fine-tuning the hyperparameters of the supervised baseline for each dataset, initially setting them based on the configurations provided the USB [26] codebase. We then fine-tuned the learning rate, weight decay, and layer decay hyperparameters for the supervised baseline model to identify the optimal set of hyperparameters that produced the lowest MAE score. The same hyperparameter settings were subsequently applied to all other methods evaluated in this study. Additionally, each method underwent further fine-tuning by adjusting its own specific additional peerpaaramesess, distinct from those used in the supervised setting.
17

---

A.13 Hyperparameters
In this section, we list the hyperparameters used in each experimental setting presented in the paper. Ta- provides the common hyperparameters for the base models: Wide ResNet-28-2 (for image data), Whisper- (for audio data), and Bert-Small (for text data). Specific hyperparameter configurations for each semi-supervised regression method are detailed in Table(9) The full code and hyperparameters are open-sourced ltttt::///// gi titt....... wommpmmm../mmmmmmmmmmmmmmmmm semi
Table 8: Common hyperparameters for the base models: Wide ResNet-28-2 (for image data), Whisper-Base (for audio data), and Bert-Small (for text data).
Wide ResNet-28-2
Whisper-Base
Bert-Small
Training Iterations
262,144
102,400
102,400
Evaluation Iterations
1,024
1,024
1,024
Training Batch Size
32
8
Optimizer
SGD
AdamW
Adam
Momentum
0.9
Criterion
MAE
MAE
MAE
Weight Decay
1c-03
2e-05
5c-04
Layer Decay
1.0
0.75
0.75
Learning Rate
1c-02
2c-06
1c-05
EMA Weight
0.999
Pretrained
False
Truc Random
True
Sampler
Random
Random
Image Resize
40x40
Max Length Seconds
6.0
Sample Rate
16,000
Max Length
512
Table 9: Specific hyperparameters for each semi-supervised regression methods.
II-Model, MeanTeacher
RankUp
MixMatch
UCVME
CLSS
RankUp
+RDA
Unlabeled Batch Ratio
1.0
1.0
0.25
7.0
7.0
Regression Unlabeled Loss Ratio
0.1
0.1
0.05
Regression Unlabeled Loss Warmup
0.4
0.4
Mixup Alpha
0.5
Dropout Rate
0.05
Ensemble Number
CLSS Lambda
2.0
Labeled Contrastive Loss
1.0
Unlabeled Contrastive Loss
0.05
Unlabeled Rank Loss Ratio
0.01
ARC Unlabeled Loss Ratio
1.0
1.0
ARC Loss Ratio
0.2
0.2
Confidence Threshold
0.95
0.95
Temperature
0.5
0.5
RDA Refinement Steps
1,024
18

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claim aligns with our contribution, which is that existing semi- supervised classification methods can be successfully adapted for use in semi-supervised regression tasks. Our empirical results demonstrate the effectiveness of our proposed framework in achieving this adaptation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations of our proposed method in Section A.1
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
19

---

Answer: [NA]
Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
We have provided a detailed implementation of our proposed framework in Section including the hyperparameters used our experiments, which can be found in This information can be used to reproduce the results presented in the
Justification:
Appendix A.13
paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
20

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: We have provided details about the datasets used in our experiments in Section AA.II] These datasets can be easily downloaded from the internet. If dataset did not come with predefined splits, we manually created the train-test splits. The splits used are open-sourced The code used in the experiments is also open-sourced at httpps: / ggthub.......... com/ com/pm25/ wwwuuperviiedrrree regreassio and can be used reproduce the results presented in the paper.
penttts://githee
com/ 1atommm25/reeeeee /
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
Saaiides/CodeSumiisstee
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurlPS code and data submission guidelines (https
//nips cc/ http://llcggg.dee/eeeeeeeeteeuee http:////ooooouusssssoooooooolae
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training and testing details of our experiments are outlined in Appendix
A.12
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All of our experimental results were obtained by running each experiment three times with fixed random seeds 0, 1, and 2. We have provided the mean and standard deviation for experimental results.
21

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: For more detailed information about the computer resources used for conduct- ing our experiments, please refer to Appendix A.8 A.8
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't 'tttt make it into the paper).
paper (e.g., preliminary or failed experiments that
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics httpge////aarrrrs..cc//////////p
Answer: [Yes]
Justification: We have thoroughly reviewed and ensured that we have met all the statement outlined in the ethical guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
22

---

Answer:
[NA]
Justification:
There is no societal impact of the work performed.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have properly cited all the code, data, and models used in this paper to ensure proper attribution and transparency.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
23

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The documentation, along with the code used in the paper, is open-sourced It provides detailed instructions on how to use the code. The code derived from the USB [ codebase, and both modifications and the original USB code licensed under the MIT license, which allows for modification and redistribution. More details about the license can be found at
at :tttps://gttee
com/ 2oom/pm25//smmii ppmp///semn-uuerrrreeeeeeeee sepperviseddreeresssee
https://githur wtw.tttu.cooooocec wwt.commppppoo wwwmpm//./eccccccmmccococmemme s0t55/eemi--e www.-miippeisseeeeeeeeeeeeeeeeee http:///ew..aaaeeeeeeeeeeeeeeeea Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

---

Answer:
[NA]
Justification:
This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
25


Reviews:
[{"review_id": "yu0PHsmWfs", "reviewer": "Reviewer_PMad", "summary": "The work introduces a novel SSL-regression method called RankUp. A a pairwise ranking loss enables the SSL-method FixMatch to utilize also the unlabled split of the data to learn a regression task. The addition of the Regression Distribution Alignment (RDA) loss enables the method to also take the overall distribution of the labeled samples into account. RankUp achieves SOTA on multiple SSL regression benchmarks.", "questions": "Regarding the baseline methods: \"Specifically, we adapt the popular semi-supervised learning codebase USB [20], modifying it for regression tasks.\" What modifications are necessary to turn e.g. the Mean Teacher approach into a regression method?", "limitations": "The authors have adequately addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written and easy to follow. Although simple, the proposed  combination of pseudo-label based SSL approach FixMatch with a pairwise ranking loss is novel. The effect of RDA is plausible and its limitations are discussed. Both variants report promising performance gains in small scale regression benchmarks, especially when the number of labeled samples is small. The qualitative analysis via t-SNE visualization underlines the effect of RankUp on the learned  representation.", "weaknesses": "The scale of the experiments. Although this might be an issue of the SSL-regression domain in general.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments and insightful questions. We're pleased you found our approach novel and well-supported. Regarding the weakness and questions:\n\n### **W1) Scale of the Experiments** \n\nWe acknowledge that the scale of our experiments is limited, which is indeed a common challenge in semi-supervised learning for regression tasks. Traditionally, research in this area has focused on small-scale image datasets \\[1\\]\\[2\\]. To address this limitation and enhance the robustness of our findings, we have evaluated our methods not only on the image datasets but also across diverse datasets including image, audio, and text modalities. These varied experiments consistently demonstrated promising results, reinforcing the efficacy of our approach.\n\n### **Q1) Modifications for Adapting USB Codebase for Regression Tasks**\n\nThe USB \\[3\\] codebase is designed for semi-supervised classification tasks. To adapt it for regression tasks, we made the following modifications:\n\n1. We replaced the softmax loss function with the mean absolute error (MAE) loss function.  \n2. We adjusted the output layer to produce a single continuous output instead of multiple outputs used for multi-class classification.  \n3. For methods like Mean Teacher and $\\pi$ Model, we didn't need to change the core algorithms as they are inherently applicable to regression tasks.  \n4. For MixMatch, we excluded components specifically designed for classification, such as sharpening and one-hot label encoding. We retained the input mixing and consistency regularization aspects, as these are valuable for regression tasks as well.\n\nWe appreciate you highlighting the need for clarity on these modifications. We will include these details in the final paper to provide a clearer understanding of our experiments.\n\n---\n\n\\[1\\] Dai, Weihang, Xiaomeng Li, and Kwang-Ting Cheng. \"Semi-supervised deep regression with uncertainty consistency and variational model ensembling via bayesian neural networks.\" *Proceedings of the AAAI Conference on Artificial Intelligence*. Vol. 37\\. No. 6\\. 2023\\.\n\n\\[2\\] Dai, Weihang, et al. \"Semi-supervised contrastive learning for deep regression with ordinal rankings from spectral seriation.\" *Advances in Neural Information Processing Systems* 36 (2023): 57087-57098.\n\n\\[3\\] Wang, Yidong, et al. \"Usb: A unified semi-supervised learning benchmark for classification.\" *Advances in Neural Information Processing Systems* 35 (2022): 3938-3961.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZiLZuN8NuF", "reviewer": "Reviewer_41bS", "summary": "The authors present two components to improve the problem of semi-supervised regression - 1) RankUp which considers the regression problem as a ranking problem and then adapts existing semi-supervised classification methods, and 2) Regression distribution alignment (RDA) which is to refine pseudo-labels. The experiments demonstrate the effectiveness on the problem.", "questions": "To better understand the effectiveness of each component on the semi-supervised regression, it can be better if the authors add more discussion and ablation studies on RankUp and RDA individually - e.g. only with RDA.", "limitations": "N/A", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The manuscript is clear and easy to follow;\n- The method is technically sound;\n- The experimental demonstration shows effectiveness over other baselines.", "weaknesses": "I'm concerned about the technical novelty. \n\n  The proposed RankUp and RDA are combination from existing methods [1][2][3], and the core idea of adding ranking auxiliary objective to the regression object is not novel [4][5]. The author should include more discussion on the related work of using ranking auxiliary objective to improve regression problems.  \n\n  The two proposed components seem to be orthogonal, especially for RankUp is directly designed to improving semi-supervised problems, but for more general regression tasks. \n\n[1] Burges et al., Learning to rank using gradient descent. ICML 2005\n\n[2] Sohn et al., Fixmatch: Simplifying semi-supervised learning with consistency and confidence, NeurIPS 2020\n\n[3] Kim et al., Distribution aligning refinery of pseudo-label for imbalanced semi-supervised learning, NeurIPS 2020 \n\n[4] Rank-N-Contrast: Learning Continuous Representations for Regression, NeurIPS 2023\n\n[5] Gong et al., RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression, ICML 2022", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and for bringing these important points to our attention. We appreciate your feedback on the clarity and technical soundness of our manuscript. Here are our responses to your concerns and questions:\n\n### **W1)  Regarding concerns about technical novelty**  \nWe acknowledge that RankUp and RDA are developed based on existing methods. However, our work uniquely integrates and adapts these methods for **semi-supervised regression tasks.**  Below, we highlight the key differences between our work and the relevant existing methods:\n\n**\\[1\\] Burges et al., Learning to rank using gradient descent. ICML 2005:** While RankNet uses pairwise ranking loss for ranking the data, the original study focuses on information retrieval tasks. We adapt the concept of RankNet for semi-supervised regression tasks, transforming the regression problem into a classification problem. This allows us to apply existing **semi-supervised** classification models to **regression problems**, bridging a gap in the field.\n\n**\\[2\\] Sohn et al., FixMatch: Simplifying semi-supervised learning with consistency and confidence, NeurIPS 2020:** FixMatch is a popular method for semi-supervised classification tasks. Our work first verified that FixMatch can also be used for semi-supervised **regression tasks.** Additionally, we would like to point out that we adopted FixMatch as a representative method to train RankUp's auxiliary ranking classifier. Our framework is flexible and can incorporate various semi-supervised classification methods, not just FixMatch.\n\n**\\[3\\] Kim et al., Distribution aligning refinery of pseudo-label for imbalanced semi-supervised learning, NeurIPS 2020:**   \n   1. Similarity: Align the pseudo-labels distribution with the labeled data distribution.  \n   2. Difference: DARP can’t be applied to regression tasks, and thus we designed a fundamentally different approach called RDA that can be used for **regression tasks**. Additionally, we introduced techniques to **overcome computational bottlenecks** for RDA, as detailed in Section 3.3 of our paper.  \n\n**\\[4\\] Rank-N-Contrast: Learning Continuous Representations for Regression, NeurIPS 2023:**   \n   1. Similarity: Uses ranking loss in regression tasks.  \n   2. Difference: Rank-N-Contrast is designed for supervised settings and can’t directly apply to semi-supervised settings. The focus of our study is **semi-supervised regression tasks**. Rank-N-Contrast requires the true labels of the data, whereas in a semi-supervised setting, the majority of the data is unlabeled. Moreover, one of the motivations of our proposed method is to leverage existing semi-supervised classification techniques for regression tasks, which Rank-N-Contrast can't achieve since it's a contrastive learning approach, not a classification approach.\n\n**\\[5\\] Gong et al., RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression, ICML 2022:**  \n   1. Similarity: Uses ranking loss in regression tasks.  \n   2. Difference: RankSim is similar to Rank-N-Contrast, which can't directly apply to semi-supervised settings. It requires the true labels of the data, whereas in a semi-supervised setting, the majority of the data is unlabeled. More detail can reference the above response to Rank-N-Contrast.\n\nWe will expand our related work section to provide a more comprehensive discussion of existing ranking-based regression methods.\n\n### **W2) Interconnection between RankUp and RDA**  \nAlthough RankUp and RDA may appear unrelated at first glance, they are suitably interconnected in our semi-supervised regression framework:\n\n1. One of the key assumptions of RDA is reliant on the ranking of the pseudo-labels. The better the quality of the ranking of pseudo-labels, the better RDA will perform.  \n2. RankUp enhances the ranking of pseudo-labels by incorporating ranking information.  \n3. The enhanced pseudo-label quality from RankUp directly benefits RDA's effectiveness, creating a synergistic relationship.\n\n### **Q1) Add more discussion and ablation studies on RankUp and RDA**\n\nWe appreciate your suggestion for additional ablation studies. Here are the results comparing supervised learning, RDA only, RankUp only, and RankUp \\+ RDA on the UTKFace dataset:\n\n|  |  | Labels \\= 50 |  |  |  | Labels \\= 250 |  |\n| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n|  | MAE &#8595; | $R^2$ &#8593; | SRCC &#8593; | | MAE &#8595; | $R^2$ &#8593; | SRCC &#8593; |\n| Supervised | 14.13±0.56 | 0.090±0.092 | 0.371±0.071 | | 9.42±0.16 | 0.540±0.014 | 0.712±0.010 |\n| **RDA (Ours)** | **14.34±1.27** | **0.060±0.125** | **0.442±0.104** | | **8.64±0.22** | **0.609±0.023** | **0.772±0.012** |\n| RankUp (Ours) | 9.96±0.62 | 0.514±0.043 | 0.703±0.019 | | 7.06±0.11 | 0.751±0.011 | 0.835±0.008 |\n| RankUp \\+ RDA (Ours) | 9.33±0.54 | 0.552±0.041 | 0.770±0.009 | | 6.57±0.18 | 0.782±0.012 | 0.856±0.005 |\n\nThese results demonstrate that:\n\n1. RDA alone may not improve upon the supervised baseline (MAE and $R^2$ of labeled = 50 setting).\n2. RankUp significantly outperforms the baseline and RDA alone.\n3. The combination of RankUp and RDA yields the best performance, showcasing their synergistic relationship.\n\nWe will include this ablation study and a detailed discussion of the results in our revised manuscript. This will provide a clearer understanding of each component's contribution and their combined effect on semi-supervised regression performance.\n\nThank you again for your insightful comments. We believe addressing these points will significantly strengthen our paper and provide a more comprehensive understanding of our method's contributions to semi-supervised regression.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lLmxci3ySU", "reviewer": "Reviewer_tpBB", "summary": "This paper presents a simple yet effective approach that adapts existing semi-supervised classification techniques to enhance the performance of regression tasks. The perspective is novel and can effectively use existing technologies to solve regression problems.", "questions": "1. The quality of the figures in the article needs to be improved.\n2. In the method, several variables are unclear. For example, ℓarc, ℓregression, etc.\n3. The description of some processes in Section 3.3 is not very clear and needs to be improved.", "limitations": "as above", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 4, "strengths": "The article has novel ideas and solid experimental results.", "weaknesses": "The method description needs to be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your positive comments on the novelty of our ideas and the strength of our experimental results. We appreciate your constructive feedback and will address your concerns as follows:\n\n### **W1) The method description needs to be improved**  \nWe will thoroughly revise Section 3 to enhance clarity by:\n\n1. A more detailed step-by-step explanation of our approach  \n2. Clearer definitions of all variables and components  \n3. Illustrative examples to aid understanding  \n4. Revise the unclear sentences.\n\n### **Q1) The quality of the figures in the article needs to be improved.**  \nWe have thoroughly improved the quality of figures by:\n\n1. Increasing the resolution of all figures  \n2. Enlarging the font size in all figures  \n3. Changing the font color to black for better readability  \n4. Adding more informative captions and labels  \n5. Unifying the size of subfigures in Figure 3\n\nThe revised figures are included in the uploaded PDF in the global response section for your review.\n\n### **Q2) Several variables are unclear. For example, ℓarc, ℓregression, etc.**  \nWe will provide clear definitions for all variables in the revised manuscript. To address the specific examples you mentioned:\n\n1. ℓarc represents the loss of the auxiliary ranking classifier (ARC).  \n2. ℓregression denotes the regression loss.\n\n### **Q3) The description of some processes in Section 3.3 is not very clear and needs to be improved**\n\nWe will thoroughly revise Section 3.3 to improve clarity by:\n\n1. Providing a more detailed explanation of each process  \n2. Using consistent terminology throughout the section  \n3. Adding a step-by-step algorithm or flowchart to illustrate the processes  \n4. Including examples to demonstrate how each process works in practice\n\nThank you again for your constructive feedback. We believe these revisions will significantly improve the clarity and readability of our paper, making our method and contributions more accessible to readers.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UnLNsG86bp", "reviewer": "Reviewer_Ngei", "summary": "The submission presents a new semi-supervised learning algorithm for deep regression models: in addition to the standard supervised regression head, an auxiliary classification head is trained using a semi-supervised learning algorithm for classification that learns to classify pairs of examples. A pair is classified correctly if the example with the larger target value is ranked above the example with the smaller target value. The loss for supervised regression is used in a weighted sum with the loss for semi-supervised classification. A further improvement to the algorithm is obtained by introducing an additional component in the loss that encourages the distribution of predicted target values to be aligned with the distribution of ground-truth target values. Experiments on three regression problems, one involving images, one involving audio, and one involving text, indicate that the proposed method outperforms semi-supervised regression algorithms from the literature.", "questions": "Was weak augmentation on the labeled data applied consistently across all the semi-supervised and supervised learning algorithm configurations evaluated in the comparison?\n\nWhich forms of augmentation were applied in the three different domains?", "limitations": "The submission discusses limitations of the proposed approach.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The submission presents a nifty idea for semi-supervised regression: formulate a label ranking problem using the regression task at hand to be able to apply a semi-supervised learning algorithm for classification. It additionally introduces a distribution alignment loss for regression that yields further improvements in performance.", "weaknesses": "The only potential problem I see is the influence of augmentation on the results. The FixMatch method used to train the auxiliary classifier, shown in Algorithm 1, applies weak augmentation for the supervised component of the loss. For a fair comparison, it seems crucial to apply this weak augmentation also on the labeled data for all the other semi-supervised and purely supervised configurations evaluated in the empirical comparison. The submission does not state whether this was done. \n\nAnother, smaller issue is that the submission does not state exactly which operators were used for augmentation in the three different domains.\n\nMy negative ratings for soundness, contribution, and the overall quality are based purely on this concern regarding the use of augmentation. *** I HAVE INCREASED BY RATINGS AFTER THE REBUTTAL ***", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and highlighting the strengths of our approach. We appreciate your careful consideration, particularly regarding augmentation. We'd like to address your concerns and questions:  \n\n### **W1) Influence of augmentation on results**  \nWe confirm that **weak augmentation was applied** consistently to the labeled data across **all** semi-supervised and supervised learning algorithm configurations in our evaluation. This ensures a fair comparison between our proposed method and the baselines, ensuring that performance improvements are not due to differences in augmentation strategies. We will include these details in the revised paper for better clarity. Our codes along with implementation details will be published if the paper is accepted. \n\nTo further analyze the effect of weak augmentation, we conducted a **new ablation study** on the UTKFace dataset with 250 labeled samples, **without** weak augmentation applied to the labeled data. Below is a comparison of the results:\n\n|  |  |  |  |  |\n|:----------------------|:------------------:|:--------------:|:-----------:|:----------:|\n|                 | Weak Augmentation |     MAE ↓      |   $R^2$ ↑   |   SRCC ↑   |\n| Supervised            | Yes                | 9.42±0.16      | 0.540±0.014 | 0.712±0.010|\n| **Supervised**        | **No**             | **11.73±0.17** | **0.315±0.012** | **0.556±0.020** |\n| $\\pi$ Model           | Yes                | 9.45±0.30      | 0.534±0.030 | 0.706±0.015|\n| **$\\pi$ Model**       | **No**             | **11.97±0.39**      | **0.319±0.029**        | **0.567±0.026**       |\n| MixMatch              | Yes                | 7.95±0.15      | 0.692±0.013 | 0.832±0.008|\n| **MixMatch**          | **No**             | **10.80±0.09**      | **0.446±0.021**        | **0.716±0.003**       |\n| RankUp + RDA (Ours)          | Yes                | 6.57±0.18      | 0.782±0.012 | 0.856±0.005|\n| **RankUp + RDA (Ours)**      | **No**             | **7.16±0.25**      | **0.742±0.022**        | **0.843±0.010**       |\n| Fully-Supervised      | Yes                | 4.85±0.01      | 0.875±0.000 | 0.910±0.001|\n| **Fully-Supervised**  | **No**             | **5.58±0.02**  | **0.837±0.002** | **0.888±0.000** |\n|  |  |  |  |  |\n\nThis table illustrates the importance of weak augmentation. The performance metrics (MAE, $R^2$, SRCC) decrease when weak augmentation is not applied **(please note that weak augmentation was applied on all the baselines in the paper)**. Despite this drop, the order of effectiveness among the methods remains consistent, whether weak augmentation is used or not. This consistency in ranking highlights the robustness of our proposed method.\n\n### **W2) Lack of specificity regarding augmentation operators**  \nWe followed the settings in the USB \\[1\\] codebase for augmentation operators, with an adjustment to the strong augmentation for audio data. This adjustment was necessary because our task involves quality assessment, and the original strong augmentation method would have affected the quality of the data. Specifically, we used the following augmentation techniques:\n\n1. Image:  \n   * Weak augmentation:  Random Crop, Random Horizontal Flip  \n   * Strong augmentation: RandAugment \\[2\\]  \n2. Audio:  \n   * Weak augmentation: Random Sub-sample  \n   * Strong augmentation: Random Sub-sample, Random Mask, Random Trim, Random Padding  \n3. Text:  \n   * Weak augmentation: None  \n   * Strong augmentation: Back-Translation \\[3\\]\n\n### **Q1) Consistent application of weak augmentation**  \nYes, weak augmentation on the labeled data **was applied consistently** across all semi-supervised and supervised learning algorithm configurations in our evaluation. More information can be found in the response to W1.\n\n### **Q2) Forms of augmentation in different domains**  \nAs detailed above (W2), we used domain-specific augmentation techniques based on the USB \\[1\\] codebase, with an adjustment for audio data. \n\nWe appreciate you bringing these important points to our attention. Addressing these issues will improve the clarity and reproducibility of our work. We will revise our manuscript to incorporate these details.\n\n---\n\n\\[1\\] Wang, Yidong, et al. \"Usb: A unified semi-supervised learning benchmark for classification.\" *Advances in Neural Information Processing Systems* 35 (2022): 3938-3961.\n\n\\[2\\] Cubuk, Ekin D., et al. \"Randaugment: Practical automated data augmentation with a reduced search space.\" *Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops*. 2020.\n\n\\[3\\] Xie, Qizhe, et al. \"Unsupervised data augmentation for consistency training.\" *Advances in neural information processing systems* 33 (2020): 6256-6268.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all the reviewers for taking their time to review our work. We greatly appreciate the thoughtful feedback and insightful comments, which have significantly contributed to improving the quality of our paper. We have attached the revised figures in the PDF for the reference.

Meta-review: This paper modifies existing semi-supervised classifications setups for regression. The reviews raised several issues, most significant among them was novelty. During rebuttal and discussions the authors satisfactorily addressed most of the concerns. This paper improves the performance of regression tasks by converting them into ranking problem.
Decision: Accept (poster)