RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Rank Correlation between Labels
Authors: Zhiqiang Kou
Yucheng Xie
Hailin Wang
Junyang Chen
Jing Wang
Ming-Kun Xie
Shuo Chen
Yuheng Jia
Tongliang Liu
Xin Geng
Venue: NeurIPS 2025 poster ()
Keywords: multi-label learning
label distribution learning
Primary area: general_machine_learning

Abstract: Pseudo label based semi-supervised learning (SSL) for single-label and multi-label classification tasks has been extensively studied; however, semi-supervised label distribution learning (SSLDL) remains a largely unexplored area.   Existing SSL methods fail in SSLDL because the pseudo-labels they generate only ensure overall similarity to the ground truth but do not preserve the ranking relationships between true labels, as they rely solely on KL divergence as the loss function during training.  These skewed pseudo-labels lead the model to learn incorrect semantic relationships, resulting in reduced performance accuracy.   To address these issues, we propose a novel SSLDL  method called \textit{RankMatch}.  \textit{RankMatch} fully considers the ranking relationships between different labels during the training phase with labeled data to generate higher-quality pseudo-labels. Furthermore, our key observation is that a flexible utilization of pseudo-labels can enhance SSLDL performance. Specifically, focusing solely on the ranking relationships between labels while disregarding their margins helps prevent model overfitting. Theoretically, we prove that incorporating ranking correlations enhances SSLDL performance and establish generalization error bounds for \textit{RankMatch}. Finally, extensive real-world experiments validate its effectiveness.

RankMatch: A Novel Approach to Semi-Supervised Label Distribution Learning Leveraging Rank Correlation between Labels
Zhigiang Kout.2, Yucheng Xie Hailin W Junyang Chen " Wang Ming-Kun Xie*, Shuo Chen Yuheng Jía 2 Liu?, Xin Gengl.2+
'School of Computer Science and Engineering, Southeast University, China
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan *Sydney AI Centre, The University of Sydney, Australia 5 School of Mathematics and Statistics, Xian Jiaotong University, China
fzhiqiang_ kou, xieyc, {ming-kur xxneei,nei,,, shuo. chen. yalerikkn..nnn.. jp, vanghailin9701663 com, tongliang liu@sydney, wwwpeyyddeyyeeuu...... edu.au
wangjing91, yhjia, xgengh@seu edu. en,
Abstract
Pseudo label based semi-supervised learning (SSL) for single-label and multi- label classification tasks has been extensively studied; however, semi-supervised label distribution learning (SSLDL)) remains a largely unexplored area. Existing SSL methods fail in SSLDL because the pseudo-labeit they generate only ensure overall similarity to the ground truth but do not preserve the ranking relationships between true labels, as they rely solely on KL divergence as the loss function during training. These skewed pseudo-labels lead the model to learn incorrect se- mantic relationships, resulting in reduced performance accuracy. To address these issues, we propose a novel SSLDL method called RankMatch. RankMatch fully considers the ranking relationships between different labels during the training phase with labeled data to generate higher-quality pseudo-labels. Furthermore, our key observation is that a flexible utilization of pseudo-labels can enhance SSLDI. performance. Specifically, focusing solely on the ranking relationships between labels while disregarding their margins helps prevent model overfitting- Theoretically, prove that incorporating ranking correlations enhances SSLDL performance and establish generalization error bounds for RankMatch. Finally, extensive real-world experiments validate its effectiveness.
Introduction
Label Distribution Learning (LDL) [TTO 26 is machine learning paradigm designed to address
label ambiguity [| 53 Unlike Multi-label Learning [633, B0J] which assigns fixed number of
labels to each instance [59], LDL extends this framework by quantifying the importance of each
label through description degrees [20]], thereby providing richer supervision information [09, 28
Corresponding authors.
2I.DI is similar to learning from soft labels, but the soft-label formulation focuses single-label problems probleems (i.c., there is only true label for each instance), while I.DI. considers multi-label prohlems (i.c., each instance can have multiple true labels).
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

0.5
Margh
KL KT, 0.018
Label Distnibation
RankError (CT,
RankError (GT.Da)
0.3
D,
@
D2
D, is better
0.50
02-
of
Luclraniigg By
Libel eakkng
5
beration
25
(b)
Figure 1: Illustration of the effectiveness of rank-aware pseudo-labelingg (a) Comparison of Pseudo- Labels by FixMatch (De) and RankMatch R) on a sample from the Twitter-LDL dataset [[]]]. DE fails to preserve the true label ranking (GT), despite a low KL divergence. In contrast, DR maintains ranking relationships with higher Kendall tau (7). (b) Performance on RAF-IDI. dataset [24]] Flexible pseudo-label utilization improves performance by focusing ranking relationships without strict margin alignment.
Deep learning has demonstrated remarkable success various domains, primarily due to its ability to leverage large-scale and accurately labeled datasets [59, 39], which are essential for train- ing deep neural networks (DNNs) with strong generalization. However, obtaining labeled data for LDL is particularly challenging and costly [G1L 29). For example, annotating the RAF-LDL dataset [B4] required 315 trained annotators, with each image annotated multiple times generate appro- priate label distributions [6_]] 27]. This highlights the significant burden of creating labeled datasets for LDL. Given these challenges, the importance of semi-supervised LDL becomes evident.
Semi-supervised learning (SSL) [II 9] has made significant progress, particularly in the areas of single-label [52] [33 and multi-label learning [5%, 3%] based on classical deep learning. However, Semi-Supervised Label Distribution Learning (SSLDL) remains relatively underexplored. One of the key techniques in SSL is leveraging trained models generate pseudo-labels (56), 633 for
labeled data, with the most well-known methods being FixMatch [52]] and MixMatch [2]. These methods [62,, 2 fail in SSLDL because, during training with labeled data, the model focuses solely on minimizing the overall similarity between predicted and ground truth label distributions (e.g.. us- ing KL divergence loss function for training) without learning label ranking correlations, leading to biased pseudo-labels. As shown in Fig. II Maaa,, for a sample from the Twitter-LDL dataset [6]]], FixMatch generates pseudo-labels with a low KL divergence (KL 0.018) but entirely incorrect label ranking correlations. This phenomenon is common across other datasets: ignoring the rank- ing relationships among labels during training produces distorted pscudo-label distributions (PLDs), and training with these PLDs causes the models performance to degrade. Moreover, we found that foreing the model to exactly match the pseudo-label distributions during training leads to overfitting. In contrast, using only the relative ranking among pseudo-labels preserves the underlying structure more robustly. Experiments on the RAF-LDL dataset (Fig. (bb)) demonstrate that imposing only ranking constraints on pseudo-labelsrather than enforcing strict numeric matchingsignificante im- proves model performance.
In this paper, we first propose pscudo-label-basel SSLDL method called RankMatch, incorpo- rates the ranking correlations between labels into the supervised training process. We introduce novel loss function called the Pairwise Ranking Relationship Loss (PRR Loss) to enhance the abil- ity of pseudo-labed to capture the ranking correlations between labels. Furthermore, we introduce a flexible pseudo-label training strategy that prioritizes ranking relationships between labels while disregarding margins, which prevents the model from overfitting to absolute label differences and enables a more robust utilization of pseudo-labels. In the theoretical aspect, we prove that incorpo- rating ranking correlations between labels can enhance the performance of SSLDL and provide gen- eralization error bounds for the RankMatch. Finally, extensive experiments on real-world datasets validate the effectiveness of our method. Our contributions can be summarized as follows:
To the best of our knowledge, this the first deep learning-based SSLDL algorithm uti- lizing pseudo-labels. Compared to existing SSL methods, our approach generates pseudo- labels that better align with the label distribution setting. Additionally, we propose flexible pscudo-label utilization strategy for SSLDL.

---

We theoretically demonstrate that incorporating label ranking correlations enhances model performance and provide a generalization bound for RankMatch.
Extensive experiments on multiple datasets validate the effectiveness of RankMatch, con- sistently outperforming existing SSL./I. methods.
2 RELATED WORK
Label Distribution Learning (LDL) (TO. b assigns a distribution over labels to each instance, es- tablishing direct mapping between instances and their label distributions. Originally proposed for facial age estimation [[2]] LDL generates distributions across all possible age categories, offering richer supervisory signals than traditional single-label approaches. This paradigm has also demon- strated strong performance in facial emotion recognition, where it effectively models ambiguous emotional states by capturing uncertainty within the label space [19, HX 38, 47.]
Beyond facial analysis, LDL has shown broad applicability across diverse domains. For instance, NASA employed LDL to infer the chemical compositions of Martian meteorites [[2]], refining the method to predict elemental abundances from crystallographic data. In mental health, LDL has been used for depression detection via the Deep Joint Label Distribution and Metric Learning framework, which identifies subtle variations in facial expressions associated with different depression levels [66]]- In crowd analysis, Ling 3SS applied LDL estimate indoor crowd densities by assigning label distributions that more accurately describe population levels in video frames.
Despite its success, LDL still faces challenges due to the scarcity of precisely annotated data [57, 60]. To mitigate this, several Semi-Supervised Label Distribution Learning (SSLDL) methods have been proposed. Hou [[X]] inferred the label distribution of unlabeled data by averaging the labels of its nearest neighbors, using both labeled and unlabeled samples for training. Jia [22] enhanced label distribution recovery by exploiting graph structrrd relationships among instances. Liu [39] further developed a co-regularization-bee based SSLDL framework that leverages dual model structures to improve robustness and consistency.
However, these SSLDL methods are often not end-to-end and rely heavily on manual feature en- gineering. limiting their scalability to high-dimensional or large-scale data. They also underuti- lize unlabeled information. In contrast, deep learning provides a natural mechanism for automatic representation learning and has demonstrated remarkable success in data-rich environments. Con- sequently, integrating deep learning into SSLDL offers a promising direction to address existing limitations and unlock the full potential of label distribution learning under limited supervision.
3 Problem Statement and Notation
In SSLDL,, the training data consists of a labeled dataset Dr, = [(x;, d;)li 1,2,..., n) and an unlabeled dataset Du (xylg 1,22...,, m). Here, and represent the number of labeled and unlabeled samples, respectively. In the labeled dataset Dz. x; is a labeled sample, and d; = (dR) duz ..., ddc } is the corresponding label distribution, where d%; represents the importance or relevance of label yj to sample x,. The label distribution satisfies the normalization constraint ES= d 1. c denotes the number of labels in the label space y = {y), 1/,,,,,, Me)
4
The Method
4.1
The Supervised Training Phase
In LDL, we transition from using the traditional binary cross-entropy loss, commonly employed in multi-label learning [TZ]], to adopting Ballback-Leibler (KL) divergence as the loss function. This transition is essential because LDL predicts continuous veal-valued label distributions instead of dis- crete binary outcomes. The KI, divergence [[Z]] well-suited for measuring the difference between ground-truth and predicted label distributions. The supervised loss formulated as:
C,
2% In
h(ys Aug. (x,):
(I)
n1
3

---

(I). Labeled Pairwise Relevance Ranking Loss (LR,
(I). Unlabeled Pairwise Relevance Ranking Loss (LpRRR)
Sapez
(4) Ground Truth
same
Unier) nieet
1000
61-
Supas
Toar
Supaie Jea
Fre
Tagne Your
Case.l
Case.2
(d) Case.3
10 pornts-hhhl distrbations
(b) Case.1
Case.2
Figure 2: An example to illustrate the LPRR loss.
where A Aug (x:) represents a weakly augmented version of the i-th labeled sample [52]. The term
hyy Auga (x:); ());;) () denotes the predicted importance degree of label My for the augmented instance Aug (X:), as determined by the model. This is computed as:
exp(fx(x;;0) 0))
h(y, x.; 0)
(2)
Egm (x,(f(xx;;0) 0))
where [x(;;:00 0) represents the raw output of the DNN for label with respect to instance x;. This formulation ensures that the predicted label distribution h(y; xx;0) satisfies the normalization
constraint ΣØ=1 h(yz x,; 1.
Existing SSL methods like FixMatch [52] and MixMatch [ fail in SSLDL because they focus only on minimizing the KL divergence between predicted and ground truth distributions, neglecting label ranking relationships. This oversight leads to pseudo-labels that may have low KL divergence but incorrect label rankings, as shown in the example where FixMatch produces reversed label impor- tance. In contrast, SSLDL requires preserving both the absolute importance and the relative ranking of labels ensure semantic consistency and accurate predictions.
To produce more reliable pseudo-labels, we propose the Pairwise Relevance Ranking (PRR) loss LPAR. which aligns predictions with the inherent semantic structure of label distributions. For labeled data, LYRR, strictly enforces alignment between the predicted and ground-truth label rank- ings while preserving meaningful margins. For example, when label description degrees d = 0.32 and d = 0.33, their negligible difference, likely caused by annotation noise, avoids unnecessary ranking adjustments. Let (xxx)) denote the predicted relevance for the j-th label after weak aug- mentation Aug The LPRR, loss defined as:
LPRRE
Σ (BLi.k) go(2, k) + sk,, 36(k:,)))
(3)
l<jckee
Here, 6 = d%i -d and ʃ(j, k) and go(i, k) are defined as follows:
11, if df; > dbc and dii d > l
s(i k) =
(4)
0, otherwise.
0,
if hx, - (x(x)) 2 6,
go( k))
(5)
66 (hy((x)) l.x
otherwise.
Fig. D(a) shows an example of the LIRR, loss, using sample from the RAF-LDL dataset. This loss penalizes two key scenarios (i) when the predicted ranking labels deviates from the ground truth (Case 1); (ii) when the ranking is correct but the difference between label scores does not match the ground-truth margin (Case 2). Only (iii), where both the ranking and the margin exactly match the ground truth (Case 3), does the loss drop to zero, indicating perfectly correct prediction.

---

4.2 Self-Training Phase by Pseudo-label distribution
Pseudo-label distribution generation: To improve prediction stability and effectively utilize unla- beled data, we adopt an ensemble cerning-basee approach [67]] This method generates pseudo- label distributions (PLDs) for unlabeled instances by averaging the model's outputs from multiple weak augmentations of the same image [52]].
The pseudo-label generation process is as follows: given an unlabeled image the model computes raw outputs (logits) for H weakly augmented versions Aug (x). The PLD for denoted as p is defined as:
exp
# Ex (Aug.(())x:
(i(w))
(6)
Eq= exp 1 E (Aug (x)):
where (A)g.. 0) is the models raw output (logit) for label yj on the k-th weak augmentation of x.
Then, we define the unsupervised consistency loss, Lucs which aligns the PLD with the predictions on strongly augmented versions of the same instances [52]] It is expressed as:
pxa (x.,(
Lue
Pia In
(3s
where (y/ Aug, (x,);; is the predicted importance degree for label yj after applying strong augmentation Xu- This loss encourages the model to exploit the underlying structure of the unlabeled data, improving learning from these instances.
In the unsupervised component, we adopt a more flexible strategy to utilize PLDs for training. Rec- ognizing the potential inaccuracies in pseudo-labels, we focus on aligning the ranking relationships among labels rather than enforcing strict adherence to absolute values. To achieve this, propose the unsupervised pairwise relevance ranking loss, LPRR.. which prioritizes capturing inter-label ranking while ignoring margins. Let h (x)) denote the predicted relevance of the j-th label after strong augmentation Aug,- The loss is defined as:
LPRR.
Σ
(s(),) ko(kk) + sk:, go(k:
(8)
l<jckee
where:
[1,
if p >x and pxi pxt >
s(j,k) = 10, otherwise.
(9)
0, [xx)x)) (xx)),
ix - he hx())) 2 O,
Go(j, k) =
(10)
otherwise.
As shown in Fig. [kb), we illustrate the unlabeled Pairwise Relevance Ranking loss CPRR.. In this examplea sample from the RAF-LDL datasetthe pseudolabel distribution is (0.6, 0. The loss only penalizes cases where the predicted ranking conflicts with the order suggested by the pseudolabel distribution, i.e., when a label with higher pseudolabel score is ranked lower(Case.1 1), and it does not impose any margin constraints. Our experiments demonstrate that this flexible use of pseudolabels significantly improves SSLDL performance, as evidenced by the faster convergence and lower KL divergence shown in the topright inset of Fig. [kb).
Finally Loss Function: Overall, the RankMatch algorithm utilizes dual-phase training strategy effectively differentiate between labeled and unlabeled data. The combined application of super- vised and unsupervised ranking losses under the PRR framework is modulated by hyperparameter A. The total loss is computed as follows:
Crotal = C. + Lue + MCPRRR + LPRR.),
(11)
Theoretical Analysis
In this section, we first investigate how the proposed PRR loss influences the generalization behavior of the SSLDI. framework. Intuitively, incorporating PRR encourages the model to capture inter- label correlations and refine label ranking consistency, which helps the network generalize beyond
5

---

the labeled set. To formalize this intuition, we derive the following theorem, which shows that adding the PRR term leads to a tighter generalization bound compared with the KL-only objective.
Theorem 5.1. Let F be hypothesis class of scoring functions, and define the empirical risks RKL. RRRR and the combined risk Reot as above. Denote the corresponding minimizers fKL and PRR+ Then, with probability at least 6,
In(1/5) 2(n + m)
Reo (fKL < (RoL(fKL) + + (l BV
This result theoretically confirms thatwhen the PRR term effectively reduces the empirical total risk the overall population risk under PRR regularization becomes strictly smaller than that of the KL-only formulation, up to the standard complexity and confidence terms. In other words, the PRR loss not only improves empirical optimization but also strengthens the generalization guarantee of the model. This provides theoretical foundation for the performance gains observed in our experiments.
Next establish theoretical foundation for our RankMatch by defining a generalization bound.
Theorem 5.2. Let I the true risk minimizer and f the empirical risk minimizer. Assume the loss
function E(-) is bounded by B and that the pseudo-labeling error satisfies ΣØ=1 T[J[ (xx))
I(d);) / S for all & E [q]. For given Lipschitz constant Lg, Rademacher complexity (x) of the function class F. and confidence parameter 5 >> 0, the generalization gap is bounded as:
log 2 2NN
R(F) (Lf)-R(f")) s =
4gL +
2qB
where N = m + the total number of labeled and unlabeled samples. Theorem 3.2 provides theoretical guarantee the performance of the proposed RankMatch algorithm. Furthermore, it highlights key factors influencing the generalization error SSLDL, including the pseudo-labeling error e, the complexity of the hypothesis space captured by the Rademacher complexity Rv (F), and the total number of training samples N. Moreover, increasing the training set size N further tightens the bound, reinforcing the benefits of leveraging large-scale unlabeled data in SSLDL. All the proof detail can be find in Appendix A and B.
Experiments
Experimental Datasets. We evaluate our approach four real-world datasets: Twitter-LLDL [6]]] (10,045 Twitter images labeled for eight emotions), Flickr-I.DI. [[6]] (10,700 Flickr images anno- tated for eight emotions by 11 annotators), Emotionó [3 (1,980 Flickr images labeled for six emo- tions), and RAF-LDL
(5,000 multi-label facial expression images).
Implementation Following [C. 58, 6SJ, we employ ResNet-50 [T6. 9EL 551 pre-trained ImageNet 32. 57 for training the classification model. For training images, we adopt standard flip-and-shift strategy [52) 51 b4] for weak data augmentation, and Rand-ugment [M Rn and Cutout [8. 14. 3] for strong data augmentation. We employ A [62. [ optimizer and one-cycle policy scheduler [15]]] to train the model with maximal learning rate of 0.0001. For all datasets, the number of epochs is set as 30 and the batch size is set as 32, Furthermore, we perform exponential moving average (EMA) [25, ![! for the model parameter 0 with decay of 0.98. We adjust the parameter À across range of values, specifically (0.005, 0.01, 0.05, 0.1 }. We perform all experiments on GeForce RTX 3090 GPUs. The random seed is set to for all experiments. The datasets detail and the other implementation detail can be find in Appendix C.
Comparing Methods. To evaluate the effectiveness of our proposed RankMatch method, we bench- mark it against four distinct groups of algorithms:
Semi-Superviser Multi-Label Learning (SSMLL) Algorithms: We introduce two advanced algorithms, SSMLLLCAPPCCAP [59] and PCLP [GG]], developed to address the challenges of semi-supervised multi-label learning by improving the reliability pseudo-labeling and leveraging label correlations within multi-label datasets.

---

Table 1: Comparison of testing results on the Emotion6, Flickr, RAF, and Twitter datasets using Canberra, Clark, Intersection, and Cosine metrics. The table reports performance under different labeled data proportions (10% 20%, and 40%) used for training. The best performance in each metric is highlighted in bold.
Emotico6
FlickrLLDL
Twitter- LDL
RAF-LDL
Method
10%
20%
40%
10%
20%
.10%
10%
20%
40%
10%
20%
40%
Rankmatch
3.3902
3.3176
3.2504
4.41660
3.9964
3.9013
3.7370
3.6962
3.2913
3,0178
2.9358
2,8341
SSMLL-CAP
3.7951
3,7613
3.7248
5.3827
5.3235
5.2676
5.8983
5.7639
5,6366
3,4385
3,2808
3.1966
PCLP
3,7011
3.6017
3.6030
5,2781
5,2292
5.1966
5,4909
5,3738
5.4133
3.3696
3.3383
3.3310
Fixmmathh-LDD
3,5080
3.5680
3,6050
5,5570
5,5310
5,4350
6.1750
6.0060
5,8340
3.1230
3.0920
3.0770
Can.
Mixmatct-LDD
3.6080
3,4860
3,4890
5.6450
5,5026
5,5750
6,3530
6,2489
6,2960
3.1580
3.1111
3.0630
GCT-LDI
3,5980
3,5490
3.6410
5,5860
5,5872
5,5260
6.3010
6.3078
6.2380
3,1920
3.1260
3.1470
SALDI
3,4836
3.3737
3.1931
5.4612
4.7789
4.8199
5,030
4.0808
4,0742
3.1947
3.1415
3,0527
SLDLF
4.4164
4.3398
4.1322
6,2280
6.1238
6,2589
5,3084
6.0008
6.1910
4.0586
4.1705
4.1189
DF-LDL
4.2427
4.0717
3.7221
5.5348
5.5549
5,5207
6,4184
6.3120
6.2588
3.328]
3,3865
3.3582
LDL-LRR
4,6528
4.0496
3,7719
5.6325
5,4988
5,4319
6,1215
6.3295
6.2905
3.8677
4,0116
4.1890
Adam- LCL
4.0815
4.1128
4.1204
6.1634
5,9889
5,6508
6.5220
6.408
6,3575
3.089]
3.0242
2,9912
Rankmatch
1,298
1,5050
1,4834
1.8189
1,7051
16737
L6480
16190
1,5188
1.4506
1,4190
1,3843
SSMLL-CAP
1,6205
1.6611
1. 1,6502
2.1222
2,0988
08000
2,2590
2.2155
2.1733
1.5918
1.5332
1.882
PCLP
1,6397
1.6059
1,6083
2.0601
2,0478
2.0328
2.1002
2.0623
2.0728
1.5689
1.5636
1 1,593
Fixmatch-LDL.
1,5950
1.6230
1,6390
2.2220
2.2110
2.1910
2.3830
2,3310
2.2820
1.5130
1.5060
1.5050
Cla. 4.
Mixmatch-I.DM
1.6240
1,5810
1,5840
2.2330
2.1996
2.2100
2,4280
2,4034
2,4150
1,5150
1.5020
1,770
GCT-LDL
1.6090
1.6050
1,6390
2.2200
2.2238
2,2080
2,4170
2.4216
2,4066
1,5350
1.5170
1.5290
SALDI.
1.6019
1,5100
2.1967
2,0346
2.0446
2.12888
1.8988
1.8964
1.5445
1.5288
1.5035
sLDLF
1,8922
1.8566
1,8049
2.3722
2,3436
2.3761
2.1480
2.3384
2,3746
1.9300
1.9645
1,9750
DF-LDL
18217
1.7746
1.678]
2.2255
2.2072
2.1992
2.4313
2.4108
2.4033
1.6071
1.6229
1.6138
LDD-LERR
1,9899
1.7745
1,6953
2.2385
2.2022
2.1919
2,4429
2.4223
2,4121
1.79077
1.8298
1.8919
Adum: Adum-LDL-Se LDL-SCI
1.7851
1.7976
1,8014
2.3534
2,3093
2.2312
2,4639
2,4324
2,4160
1.5134
1,4980
1,4905
Rankmatch
0.6735
0.6832
0.6940
0.6921
0.7073
0.7151
0.7036
0.7190
0.7316
0.6551
0.6813
0.7044
SSMLL-CAP
0,5479
0.5587
0.5666
0.5815
0.6125
0.6034
0.6324
0.6577
0.5264
0.5876
0.6092
PCLP
0.6059
0.6370
0.6363
0.6392
0.6469
0,6490
0,6707
0.6784
066880
0.5471
0.5588
0.5590
6xmatch-LDL
0.6638
0.6797
0.6916
0.6857
0.7042
0.7119
0.7009
0.7147
0.7283
0.6570
0.6760
0.6987
Mixmatch-I.DI.
0.6372
0.6418
0.6496
0.6639
0.6686
0.6831
0.6819
0.6806
0.6986
0,6133
0.6381
0.6534
GCT-LDL
0.6116
0.6602
0.6770
0.6639
0.6879
0.6863
0,6787
0.7018
0.7102
0.6321
0.5669
0.6910
SALDE.
0.6457
0,6612
0.6723
0,5559
0,5108
0.5091
0.6632
0.5724
015687
0.62298
0.6504
0.6208
sLDLF
0.5935
0,6162
0,4813
0,4750
0,4616
0.6487
0.5652
0.5336
0,2433
0.2199
DF-LDL
0.5057
0.5461
0.6353
0.4173
0,4176
0.4169
0.3541
0.3536
0.3505
0,7022
0.7083
0.7085
I.D
0.3721
0.6213
0.6626
0,5322
0,5811
0.5600
0.5746
0.5904
(15979
0.5649
0,5389
0.4411
Adum- LCL
0.3409
0.5627
0,6040
0,4724
0.3933
0,4628
0.5488
0.5828
0.5200
0.6177
0.5768
0,4843
Rankenatch
0.8121
0.8257
0.8331
0,8489
0,8614
0,8679
0,8544
0.8698
0,8790
0.7901
0,8140
0.8375
SSMLL-CAP
0.6850
0.6994
0,7185
0.7634
0.7885
0.8144
0.8109
0.8270
(18442
0.6456
0.7119
0.7329
PCLP
0,7421
0,7778
0,8057
0.8146
0.8151
0.8391
0.8448
0.6815
0.6962
0.6969
Fixmatch-LDD
0.8079
0.8200
0.8312
0.8487
0.8573
0.8673
0.8517
0.8647
0.8758
0.7881
0.8123
0.8311
Cos. 1
Mixmatch-I.IDI. .II.
0,7585
0.7863
0.7901
0.7888
0.8381
0.8468
0.8463
0,8552
(18602
0,7536
0.7680
0.7820
GCT-LDL
0,7530
0.8134
0.8313
0.8508
0.8499
0.8716
0,7660
0.8181
SALDI.
0.7784
0.7874
0.7981
0.7361
0.5643
0.6624
0.8479
0,7612
0.7615
0.7711
0.7938
0.8135
sLDLF
0,7037
0.6980
0,7350
0.6276
0,6066
0,5897
0,8002
0.7454
0.6988
0.3262
0.3506
0.3459
DF-LDL
0.6035
0.6470
0.7689
0.5436
0.5539
0.5569
0.5069
0.5209
0.8427
0.5470
LDL-LRR
0.4604
0.7362
0.7905
0.7020
0.7316
0.7399
0.7767
0.8027
0.8125
0.7253
0.6938
0.5757
Adam--DDL-SCL
0.4311
0.6670
0,7144
0.6104
0,4888
0.6166
0,7163
0.7661
0,7403
0,7717
0.7337
0.6191
Table 2: Evaluation of Label Distribution Ranking Relationships for Test Samples. The table com- pares Kendall tau (Tx) and Spearmans rank (ps) correlation coefficients across datasets and different methods. Bold values indicate the best-performing method for each dataset and metric.
Meaic
Dutuset
10% Labdlal Data
20's Labelol Data
RankMatch
GCT
CAP
PCLP
FisMadd
MisMathh
RaskMadd
OCT
CAP
PCL.P
FeMacc
MisMurch
RAF
0,5686
0.4258
0.2079
0,2750
0,4645
0.9066
0,5463
0.5811
0.3558
0.2949
0,6965
0.4524
Eautionó
0.5536
0.3718
0.1257
0.3325
0.4899
0.4562
0,5620
0.4594
0.1536
0.4394
0.4985
0.4528
Flickr
0,5618
0.5005
0.4030
0.6904
0,5215
0.5119
0.5627
0.5215
0.4475
0.5005
0.5415
0.5265
Twitter
0.4877
0.4744
0.5816
0,5452
0.808
0.5099
0.5177
P5
RAF
0.6726
0.3122
0.2474
0.3365
0.5564
0,0900
0,6649
0.5754
0.4277
0.3602
0.5917
0.5497
Eautison
0.6545
0-4495
0.1520
0.4087
0.5933
0.5528
0,6312
0.5634
0.1933
0.5332
0.5989
n.5550
Flickr
0,6537
0.5934
0.4831
0.5793
0.6097
0.6019
0.6551
0.6112
0.5335
0.5903
0.6304
0.6171
Twittr
0.5740
0.5637
0.5193
0.7755
0.5517
0.5864
0.6315
0.5966
0.5658
0.5898
0.5853
0A4044
Dual-Network SSLDI Algorithm: We present and evaluate our own GCT-LDL(GGCT), dual-network [+] SSLDL approach that we developed, which leverages mutual supervision of unlabeled data between two independent networks for enhanced learning.
* Deep Learning SSLDL Algorithms: We introduce two novel algorithms, FixMatch-LDL. [52]] and MixMatch-L.DI. [D],, designed to bridge the gap in open-source semi-supervised LDL (SSLDL) approaches within deep learning frameworks.
7

---

aepar
Aggen
Scar
Detural
serpette
far
digadt
hapgs
Gd
Ruscsact-IDL
DAC
004
600
QAB
0.58
285
825
0.51
8.52
832
DAB
8.21
Fouask E00
0M0
289
080
0.0
60
2:2
UCTLUL
020
010
agper
6gm
14
netural
drga
happ:
BAAT
Uneed fret
000
100
RA4
133
Unsed Irch
0:21
167
0.31
120
052
822
Pbraack-..i D1
040
012
606
020
289
8.45
Flowsad
666
6.19
900
922
031
106
0114
103
8.2
6:37
4112
8.34
Figure 3: Examples illustrating RunkMatch-L.DI. ability to generate superior pseudo-label distribu- tions compared to existing semi-supervised methods. The first two images are from the Emotion6 | dataset, and the last two from the RAF dataset [[44]] RankMatch- LDL more accurately aligns with the ground truth label ranking.
Traditional SSLDL Algorithm: The traditional SA-LDL [[8] algorithm, originally for tab- ular data, is adapted for image datasets through necessary feature engineering, detailed in Appendix D.
SOTA LDL Algorithms:: Comparisons are also made with state-of-the-art LDL algorithms including Adam-LDL-SCLL [20], sLDLF [SD]] DF-LDL [[3]], and LDL-LRR [2T]] high- lighting their potential limitations in SSLDL contexts.
Evaluation Metries. We evaluate LDL methods using eight metrics [[]]] Chebyshev, Clark, Can- berra distances, and Kullback-Leiblee divergence (lower hetter), along with Intersection and Co- sine similarities, Spearmans rank correlation (ps), and Kendall tau correlation (Tx) (higher is better).
6.1 Comparative Experiment Analysis
employed a range of labeled data proportions (10%, 20%, and 40%) to simulate varying levels of label availability, critical factor in semi-supervised learning scenarios. The experiments are presented in Table and Table from that we can draw the following conclusions
Mathh consistently achieves top performance across all datasets and metrics. Com- pared to SSMLL-CAP and PCLP, RankMatch shows stronger label relationship model- ing, leveraging PRR losses refine pscudo-label rankings and outperforming traditional SSLDL methods like FixMatch-LDL and MixMatch-LDD.
The results Table confirm the consistent superiority RankMatch in capturing label ranking relationships, achieving the highest Kendall tau (TK) and Spearmans rank (ps) correlation coefficients across almost all datasets and metries. Notably, RankMatch main- tains robust performance as the proportion of labeled data increases, demonstrating strong generalization and adaptability under varying supervision levels.
With increasing labeled data, RankMatch scales effectively, achieving significant perfor- mance improvements. On Twitter, the Canberra distance improves from 3.7370 (10% la- beled data) to 3.2913 (40% labeled data).
6.2 Analysis of Pseudo-Labed Performance
To evaluate the performance pseudo-labeling, we assess the pseudo-labeling quality of different algorithms and visualize two sample images from the Emotionó and RAF datasets examples. The experimental results are presented Table & and Fig. CL Based on these results, we derive the following conclusions:
Our method effectively captures the true label distribution, achieving the best performance in overall distance metrics, evidenced by the lowest KI. divergence across all datasets.
By incorporating the PRR loss, our approach generates pseudo-labels with ranking struc- tures that more closely align with the ground truth, leading higher-quality pseudo-labels and improved model performance.

---

Table 3: Comparison of pseudo-labeling performance with different methods trained on 10% and 20% labeled data. Kendalls Tau (Tk) and Spearmans rank correlation (ps) measure ranking quality (higher is better), while KL divergence quantifies distribution alignment (lower is better).
10% Labeled Data
20% Labeled Data
Dataset
RankMatch
FixMarch
MixMatch
GCT
CAP
PCLP
RankMatch
FixMatch
MixMatch
GCT
CAP
PCLP
Ps
Emotionó
0,5372
0.4763
0.6287
0.5685
0.4929
0,4449
0.3325
RAF
0.6617
0,5348
0,5024
0.4714
0.2593
0.3471
0,6730
0.5772
0.5480
0,5730
0,2354
0.4313
Flickr
0.6646
0.6262
0.6153
0.6024
0,4902
0.5933
0.6618
0.6490
0.6377
0.6328
0.5528
0.5988
Twitler
0.6446
0.5551
0.6006
0.5729
0.5197
0.5681
0.6440
0.5990
0.6147
0.6108
0.5602
0.5830
Emnotion6
2.5458
3,4885
3.9461
5.1156
5.6353
5,0965
2.4344
3.0090
4,1522
3.7812
3.5549
4,8452
RAF
2.0564
2,9951
3,9584
3,5445
4,9070
4.5420
1,3800
2.5717
3.1612
2.7052
5.4632
3,9629
Plickr
2.6495
3,7454
3.6983
4.2128
6.0978
3.7667
2,6802
2,9860
4,0924
3.7720
5,2461
3,7492
Twitter
3.1827
3,4360
6.5147
3.2599
2.5900
3.9771
3.2475
5.7258
3.0865
Table 4: Ablation Results on Flickr and RAF Datasets.
Che.
Cla.j
Can.j
KT.
Cos.t
Int.1
pretrain
0.2411
2,2594
0.5371
0,8427
0.6873
Flickr
peetrain + consistency
0.2262(6.2% f) (.113166.5%7) 5.1536(9.491) 0.5293(L5981)
0.8633(2.488 0.7188(4.6%.)
pretrain consistency+PRR loss 0.2184(3.4% 1) 2.0158(4.6) t) 4.9008(4.91) 0.5227(1.2 f) 0.8714(0.9% f) 0.7208(0.3%1)
Che.j
Cla.j
Can.
KT.
Cosf
Int.1
pretrain
0.2938
1,5422
3.206
0.5146
0.7687
0.6411
RAF
pretrain consistency
0.255(13.2%% 1.5021(2.5%11) 3.1345(2.2%6) 0.3699(8.131)) 0.889(28.1 198 0.7073(00.35/)
pretrain consistency loss 0.2341(8.2% f) 1.4914(0.7% 3.0459(.891) 0.3464(6.4%) 0.8476(3551) 0.7194(1.7%-1)
6.3 Further Analysis
Ablation Study Our ablation study analyzed the impact of PRR loss and unsupervised consistency loss on the performance of Mankhatch. Initially, the model was pre-trained with only 10% of labeled data establish a baseline. This phase highlighted the model's ability utilize minimal data effectively.
Next, unsupervised consistency loss was applied to enhance learning from unlabeled data. In the final phase, PRR loss was introduced, leveraging the same 10% labeled data to refine the model further with supervised ranking loss. Ablation experiment results are shown in Table BL From this, we can draw the following conclusions
The integration of unsupervised consistency loss markedly improves RankMatch's 's perfor- mance across datasets, as observed in the ablation results. This confirms the effectiveness of using unsupervised data enhance model accuracy.
The incorporation of pairwise relevance ranking (PRR) loss significantly boosts perfor- mance, particularly in scenarios where surpasses the baseline. This improvement demonstrates the PRR loss's critical role in refining label discrimination within the semi- supervised learning framework.
The Impact of LraR Loss on Bseudi-Labeling: We evaluated the effect of LPRR loss on pseudo- labeling by comparing models trained with and without it on the RAF and Emotion6 datasets. Pseudo-labels were generated for part of the validation set during training. and their ranking perfor- mance was analyzed. As shown in Fig. EL incorporating PRR loss consistently improved the ranking quality of pseudo-labels, aligning them more closely with the ground truth. This demonstrates the ability of CPAR loss to effectively capture inter-label ranking relationships, thereby enhancing both the training process and the overall model performance.
Parameter Sensitivity Analysis Fig. illustrates the impact of the parameter À on RankMatch's 's performance across the Emotion6, Flickr-LDI, RAF-LDI, and Twitter-I.DL datasets, focusing on KI. divergence and Cosine similarity metrics. Fig. 5 shows that RankMatch performs consistently well when À ranges from 0.01 to 0.05, with minimal variations in performance metrics such as KL

---

Raf
Fmotion6
WithitaniLess
NoRasklews
NoPRRLoss
0.30
0.20
20:33
9.55
0.25
045
0.00
Figure 4: The effect of incorporating PRR Loss on the ranking performance of pseudo-labels during training.
Figure 5: Parameter sensitivity analysis Emo- tion6, Flickr-LDL, RAF-LDL, and Twitter-DDL datasets.
divergence, Intersection, and Cosine similarity across all datasets. However, 0.005, notice- able performance drop, particularly in the Emotionó dataset, highlights the reduced effectiveness of regularization. Conversely, higher values, such À 0.1, slightly hinder performance, suggest- ing weerreegularization. This analysis indicates that À values between 0.01 and 0.05 strike the best balance for effective learning. Detailed results for additional metrics can be found in Appendix D.
7 Conclusion and Limitations
Conclusion. In this paper, we introduce RankMatch, the first deep-learningbased semi-supervised label distribution learning (SSLDL) method that explicitly models inter-label ranking relationships via pseudo-labels. By combining the standard KL-divergence loss with our novel Pairwise Ranking Relationship (PRR) loss within single training framework, RankMatch produces higher-quality pseudo-label distributions and flexibly leverages unlabeled data without overfitting to noisy abso- lute values. In our theoretical analysis, we prove that adding the PRR loss tightens the models generalization bound and provide explicit generalization error bound for RankMatch. Finally, we empirically validate on four weal-world LDL benchmarks (Twitter-LDL, Flickr-LDL, Emotion6, and RAF-LDL) that Mank consistently outperforms all baselines.
Limitations. While SSLDL significantly lowers the demand for fully annotated data, it still depends on a nontrivial amount of manual labeling. Going forward, we plan to investigate how to harness large language models generate cost-effective, high-quality pseudo-labels and to integrate LLMs directly into the SSLDL training pipeline.
Acknowledgments
This work supported in part by the Jiangsu Science Foundation (BG2024036, BK20243012), the National Natural Science Foundation China (62125602, U24A20324, 92464301, 62306073), the China Postdoctoral Science Foundation (2022M720028), the Xplorer Prize, and the National Natural Science Foundation of China under Grant U24A20322. Additional support was provided by the Big Data Computing Center of Southeast University.
References
[1] Hritam Basak and Zhaozheng Yin. Pseudo-label guided contrastive learning for semi- supervised medical image segmentation. IEFF/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 19786- 19797. IEEE, 2023.
[2] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi sumi-super learning. Advances in neural information processing systems, 32, 2019.
[3] Linbo Cao and Jinman Zhao. Pretraining the test set is no longer all you need: A debate- driven approach to QA benchmarks. Second Conference on Language Modeling, 2025.
[4] Xiaokang Chen, Yuhui Yuan, Gang Zeng, and Jingdong Wang. Semi-supervised semantic seg- mentation with cross pseudo supervision. In IEEE Conference on Computer Vision and Pattern
10

---

Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 2613-2622, Computer Vision Foun- dation / IEEE, 2021.
[5] Elijah Cole, Oisin Mac Aodha, Titouan Lorieul, Pietro Perona, Dan Morris, and Nebojsa Jojic. Multi-label learning from single positive labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 933-942, 2021.
[6] Richard Combes. An extension of modiarmid's inequality. arXiv preprint 2015.
[7] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical au- tomated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702-703, 2020.
[8] Terrance DeVries and Graham W Taylor. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04553. 04557000..455,, 2017.
[9] Enrico Fini, Pietro Astolfi, Karteek Alahari, Xavier Alameda Saaeea--inneda, Julien Mairal, Moin Nabi, and Elisa Ricci. Semi-supervised learning made simple with self-supervised cluster- ing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver. BC. Canada, June 17-24, 2023, pages 3187-3197. IEEE, 2023.
[10] Xin Geng. Label distribution learning. IEEE Transactions on Knowledge and Data Engineer- ing, 28(7):1334-1748, 2016.
[11] Xin Geng, Xin Qian, Zengwei Huo, and Yu Zhang. Head pose estimation based multivariate label distribution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[12] Xin Geng, Chao Yin, and Zhi-Hua Zhou. Facial age estimation by learning from label distri- butions. IEEE transactions pattern analysis and machine intelligence, 35(10):240122412 2013.
[13] Manuel González, Germán González-Almagro, Isaac Triguero, José-Ramón Cano, and Sal- vador Gareía. Decooppoittion-fusionn for label distribution learning. Information Fusion, 66:64- 75, 2021.
[14] Shumxin Guo, Hongsong Wang, Shuxia Lin, Zhiqiang Kou, and Xin Geng, Addressing skewed heterogeneity via federated prototype rectification with personalization. IEEE Transactions on Neural Networks and Learning Systems, 36(5):8442-844) 2025.
[15] Mahammad A Hannan, Dickson NT How, Muhamad Bin Mansor, Md S Hossain Lipu, Pin Jern Ker, and Kashem M Muttaqi. State-of-charge estimation of li-ion battery using gated recur- rent unit with one-cycle learning rate policy. IEEE Transactions on Industry Applications, 57(3):2964-299) 2021.
[16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattem recogni- tion, pages 770-778, 2016.
[17] John Hershey and Peder A Olsen. Approximating the kullback leibler divergence between gaussian mixture models. In 2007 IEEE International Conference Acoustics, Speech and Signal Processing-CCASSPOO volume 4, pages IV-317. IEEE, 2007.
[18] Peng Hou, Xin Geng, Zeng-Wer Wei Huo, and Jia-Qi Lv. Semi-supervised adaptive label distribu- tion learning for facial age estimation. In Thirty-First AAAI Conference on Artificial Intelli- gence, 2017.
[19] Xiuyi Jia, Weiwei Li. Junyu Liu, and Yu Zhang. Label distribution learning by exploiting label correlations. Proceedings of the Thirty-Thine International Joint Conference on Artificial Intelligence, volume 32, 2018.
[20] Xiuyi Jia, Zechao Li, Xiang Zheng, Weiwei Li, and Sheng-Jun Huang. Label distribution learning with label correlations on local samples. IEEE Transactions on Knowledge and Data Engineering, 33(4)11619-1611 2019.

---

[21] Xiuyi Jia, Xiaoxia Shen, Weiwei Li, Yunan Lu, and Jihua Zhu. Label distribution learning by maintaining label ranking relation. IEEE Transactions on Knowledge and Data Engineering. 2021.
[22] Xiuyi Jia, Tao Wen, Weiping Ding, Huaxiong Li, and Weiwei Li. Semi-supervised label distri- bution learning via projection graph embedding. Information Sciences, 581:840-8555 2021.
[23] Yangbangyan Jiang, Xiaodan Li, Yuefeng Chen, Yuan He, Qianqian Xu, Zhiyong Yang, Xi- aochun Cao, and Qingming Huang. Maxmatch: Semi-supervised learning with worst-case consistency. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5970- 5987, 2022.
[24] Jae Yong Kim, Eun-Young Lee, Jin Kyun Park, Yeong Wook Song, Jae-Ryong Kim, and Kyung-Hyun Cho. Patients with rheumatoid arthritis show altered lipoprotein profiles with dysfunctional high-density lipoproteins that can exacerbate inflammatory and atherogenic pro- cess. PLoS One, 11(10)ee014564, 2016.
[25] Frank Klinker. Exponential moving average versus moving exponential average. Mathematis- che Semesterberiihtee 58:97-107, 2011.
[26] Zhiqiang Kou, Si Qin. Hailin Wang, Jing Wang. Mingkun Xie, Shuo Chen, Yuheng Jia, Tongliang Liu, Masashi Sugiyama, and Xin Geng. Label distribution learning with biased annotations assisted by multi-label learning. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, 8 2025.
[27] Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Inaccurate label distribution learning. IEEE Transactions on Circuits and Systems for Video Technology, 34(10):10337-1024 2024.
[28] Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Progressive label enhancement. Pattern Recognition, 160:11 1172, 2025.
[29] Zhiqiang Kou, Jing Wang, Yuheng Jia, Biao Liu, and Xin Geng. label distribution learning. IEEE Transactions on Neural Networks and Learning Systems, 36(1):1425-143) 2025.
inaccurate
[30] Zhiqiang Kou, Jing Wang, Jiawei Tang, Yuheng Jía, Boyu Shi, and Xin Geng. Exploiting multi- label correlation in label distribution learning. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 4326-4334, 8 2024.
[31] Zhiqiang Kou, Haoyuan Xuan, Jingyu Zhu, Hailin Wang, Ming-kun Xie, Changwei Wang. Jing Wang, Yuheng Jia, and Xin Geng. Tail-aware reconstruction of incomplete label distributions with low-rank and sparse modeling. IEEE Transactions on Circuits and Systems for Video Technology, pages 1-1, 2025.
[32] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012.
[33] Hengduo Li, Zuxuan Wu, Abhinav Shrivastava, and Larry S Davis. Rethinking pseudo labels for semi-supervised object detection. In Proceedings of the AAAI conference artificial intelligence, volume 36, pages 1322, 2022.
[34] Shan Li and Weihong Deng, Blended emotion in-the-wild: Multi-label facial expression recog- nition using crowdsourced annotations and deep locality feature learning. International Jour- nal of Computer Vision, 1276677)884--966 2019.
[35] Miaogen Ling and Xin Geng. Indoor crowd counting by mixture of gaussians label distribution learning. IEEE Transactions on Image Processing, 28(11)5561-5570, 2019.
[36] Biao Liu, Ning Xu, Xiangyu Fang, and Xin Geng. Correlation-induced label prior for semi- supervised lulti-label learning. In Forty-first International Conference on Machine Learning.
[37] Biao Liu, Ning Xu, Xiangyu Fang, and Xin Geng. Correlationiinduced label prior for semi- supervised multi-label learning. In Proceedings of the 41st International Conference on Ma- chine Learning, volume 235, pages 32224-32238, 21-27 Jul 2024.
12

---

[38] Biao Liu, Ning Xu, Jiaqi Lv, and Xin Geng. Revisiting pseudo-label for single- positive multi- label learning. In Proceedings of the 40th International Conference on Machine Learning. volume 202, pages 22249-22265, 23-29 Jul 2023.
[39]
Xinyuan Liu, Jihua Zhu, Qinghai Zheng, Zhiqiang Tian, and Zhongyu Li. Semi-supervised label distribution learning with co-regularization. Neurocomputing. 491:353-364, 2022.
[40] Lei Meng. Zhuang Qi, Lei Wu, Xiaoyu Du, Zhaochuan Li, Lizhen Cui, and Xiangxu Meng. Improving global generalization and local personalization for federated learning. IEEE Trans- actions on Neural Neaworks and Learning Systems, 36, 2024.
[41] Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learn- ing. MIT press, 2018.
[42] Shaunna M Morrison, Feifei Pan, Olivier C Gagné, Anirudh Prabhu, Ahmed Eleish, Pe- ter Arthur Fox, Robert T Downs. Thomas Bristow, Elizabeth B Rampe, David Frederick Blake, al. Predicting multi-component mineral compositions in gale crater, mars with label distri- bution learning. In AGU Fall Meeting Abstracts, volume 2018, pages P211-3438, 2018.
[43]
Peng, Tsuhan Chen, Amir Sadovnik, and Andrew C Gallagher. A mixed bag of emotions: Model, predict, and transfer emotion distributions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 860-868, 2015.
[44] Zhuang Qi, Lei Meng, Zitan Chen, Han Hu, Hui Lin, and Xiangxu Meng. Cross-silo proto- typical calibration for federated learning with non-iid data. Proceedings of the 31st ACM International Conference Multimedia, pages 3099-3107, 2023.
[45] Zhuang Qi, Lei Meng, Zhaochuan Li, Han Hu, and Xiangxu Meng. Cross-silo feature space alignment for federated learning on clients with imbalanced data. In The 39th Annual AAAI Conference Artificial Intelligence (AAAI-25), pages 19986-19994 2025.
[46] Zhuang Qi, Sijin Zhou, Lei Meng, Han Hu, Han Yu, and Xiangxu Meng. Federated de- confounding and debiasing learning for out-o--disrrbbutio generalization. arXiv preprint arXiv:2505.04979, 2025.
[47] Congyu Qiao, Ning Xu, and Xin Geng. Decompositional generation process for instance- dependent partial label learning. In International Conference on Learning Representations, 2023.
[48] Congyu Qiao, Ning Xu, Yihao Hu, and Xin Geng. Ularef: unified label refinement frame- work for learning with inaccurate supervision. In International Conference on Machine Learn- ing, 2024.
[49] Congyu Qiao, Ning Xu, Jiaqi Lv, Yi Ren, and Xin Geng. Fredis: fusion framework of re- finement and disambiguation for unreliable partial label learning. In International Conference on Machine Learning, pages 28321-28336. PMLR, 2023.
[50] Wei Shen, Kai Zhao, Yilu Guo, and Alan L Yuille. Label distribution learning forests. Ad- vances in neural information processing systems, 30, 2017.
[51] Boyu Shi, Shiyu Xia, Xu Yang. Haokun Chen, Zhiqiang Kou, and Xin Geng. Building variable- sized models via learngene pool. Proceedings of the AAAI Conference on Artificial Intelligence, 38(13):14946-144
Mar. 2024.
[52] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raf- fel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. Fixmatch: Simplifying semi- supervised learning with consistency and confidence. Advances in neural information process- ing systems, 33:596-608. 2020.
[53] Jing Wang and Xin Geng. Label distribution learning machine. In Proc. Int. Conf. Mach. Learn., pages 10759. PMLR, 2021.
[54] Jing Wang, Zhiqiang Kou, Yuheng Jia, Jianhui Lv, and Xin Geng. Label enhancement by manifold fusion of feature and label spaces. Pattern Recognition, 168: 11185,, 2025.
13

---

[55] Yining Wang, Jinman Zhao, Chuangxin Zhao, Shuhao Guan, Gerald Penn, and Shinan Liu. X-grpo: Unifying the grpo frameworks with learnable token preferences, 2025.
[56] Yuchao Wang, Haochen Wang, Yujun Shen, Jingjing Fei, Wei Li, Guoqiang Jin, Liwei Wu, Rui Zhao, and Xinyi Le. Semi-supervised semantic segmentation using unreliable pseudo-labels. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pages 4248 4257, 2022.
[57] Chunlin Wen, Yu Zhang, Jie Fan, Hongyuan Zhu, Xiu-Shen Wei, Yijun Wang, Zhiqiang Kou, and Shuzhou Sun. Object-level correlation for few-shot segmentation, 2025.
[58] Ming-Kun Xie, Jia-Hao Xiao, Gang Niu, Lei Feng, Zhiqiang Kou, Min-Ling Zhang, and Masashi Sugiyama. What makes "good" distractors for object hallucination evaluation in large vision-language models?, 2025.
[59] Mung-Kun Xie, Jiahao Xiao, Hao-Zhe Liu, Gang Niu, Masashi Sugiyama, and Sheng-Jun Huang. dlass-distibbutionaaee aware pseudo-labeling for semi-supervised multi-label learning. Ad- vances in Neural Information Processing Systems, 36, 2024.
[60] Miao Xu and Zhi-Hus Zhou. Incomplete label distribution learning. In Proc. Int. Joint Conf. Artif. Intell, pages 3175-3181, 2017.
[61] Jufeng Yang, Ming Sun, and Xiaoxiao Sun. Learning visual sentiment distributions via aug- mented conditional probability neural network. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
[62] Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xi- aodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. Large batch optimization for deep learning: Training bert in 76 minutes. arXiv preprint arXiv: /900:0094900 2019.
[63] Min-Ling Zhang and Hui-Hua Zhou, A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8):1819-1888 2014.
[64] Xueyan Zhang, Jinman Zhao, Zhifei Yang. Yibo Zhong, Shuhao Guan, Linbo Cao, and Yining Wang. UORA: Uniform orthogonal reinitialization adaptation in parameter efficient fine-tuning of large models. Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting the Association for Computa- tional Linguistics (Volume I: Long Pupers), pages 11709-11728, Vienna, Austria, July 2025. Association for Computational Linguistics.
[65] Yibo Zhong, Jinman Zhao, and Yao Zhou. Low-rank interconnected adaptation across layers. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguisties: ACL 2025, pages 17005-17029, Vienna, Austria, July 2025. Association for Computational Linguistics.
[66] Xiuzhuang Zhou, Zeqiang Wei, Min Xu, Shan Qu, and Guodong Guo. Facial depression recognition by deep joint label distribution and metric learning. IEEE Transactions on Affective Computing, 1303):1605-1615 2020.
[67] Zhi-Hua Zhou and Zhi-Hua Zhou. Ensemble learning. Springer, 2021.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the papers contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state our three major contribu- tions(i) the RankMatch SSLDL framework, (ii) the novel PRR loss, and (iii) the the- oretical generalization boundand these are fully supported by the experiments (Sec. 1).
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the au-
thors?
Answer: [Yes]
Justification: We include a dedicated Limitations paragraph at the end Sec. not- ing that SSLDL still requires some manual labels and outlining future directions for LLMbased low-cost annotation and integration of LLMs into the SSLDL loop.
3. Theory assumptions and proofs
For each theoretical result, does the paper provide the full set of assump- tions and complete (and correct) proof?
Question:
Answer: [Yes]
Justification: All assumptions are stated with each theorem (Sec. 5), and full proofs appear in Appendix A and B.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results?
Question:
Answer: [Yes]
Justification: We provide dataset splits, pseudocode, hyperparameters, training details in Sec. 6 and Appendix C.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient
instructions to reproduce the main experimental results?
Answer: [Yes]
Justification: The benchmark datasets used in this work are publicly available, and the download links are provided in Appendix C. The implementation code and pretrained models will be released our GitHub repository upon publication to facilitate full reproducibility.
6. Experimental setting/details
Question: Does the paper specify all training and test details necessary to understand the results?
Answer: [Yes]
Justification: We describe data splits, augmentations, optimizers, and hyperparameter sweeps in Appendix C.
7. Experiment statistical significance
Does the paper report error bars or other appropriate information about statistical significance?
Question:
Answer: [No]
Justification: For all experiments, we used a random seed 1.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on com- pute resources needed to reproduce the experiments?
Question:
15

---

Answer: [Yes]
Justification: Appendix C reports GPU type (NVIDIA 3090), run time per epoch, and total training hours.
9. Code of ethics
Question:
Does the research conform the NeurIPS Code of Ethics?
Answer: [Yes]
Justification:
We have reviewed the NeurIPS Code of Ethics and found no ethical
issues.
10. Broader impacts
Question: Does the paper discuss both potential positive and negative societal im- pacts?
Answer: [NA]
Justification:
We found no socictal impact of this work.
11. Safeguards
Question: Does the paper describe safeguards for responsible release of data models with high misuse risk?
Answer: [NA]
Justification: Our contributions do not pose highrisk dualuse scenarios requiring spe- cial release controls.
12. Licenses for existing assets
Question: Are existing assets properly credited with license information?
Answer: [Yes]
Justification: We cite datasets (Twitter-DDL, RAF-LDI, etc.) and reference their original licenses (Sec. 6).
13. New assets
Question: Are new assets introduced in the paper well documented alongside the as- sets?
Answer: [NA]
Justification: his paper does not release new assets.
14. Crowdsoureing and human subjects
Question:
For crowdsourcing experiments, does the paper include full instructions and compensation details?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
15. IRB approvals
Question: Does the paper describe IRB approvals equivalent for research with hu- man subjects?
Answer: [NA]
Justification: We did not conduct new humansubjects research.
16. Declaration of LLM usage
Question: Does the paper describe the usage LLMS they are an important compo-
nent of the core methods?
Answer: [NA]
Justification: Our core SSLDL method does not rely on external LLMs; future work will explore LLM-based labeling but that is out of scope here.
16

---

A The Proof of Theorem 3.1
We now show that adding the pairwise relevance ranking (PRR) loss the usual KLdivergence objective strictly tightens the generalization bound in semisupervised label distribution learning.
Theorem A.I. Let F be hypothesis class of scoring functions f: X' - IRC Define the empirical risks on the labeled set {0x, d,) and unlabeled set {zz 1 by
())( = Σ 1=1
(d (((r))) RPRR (f)
LPRn (PRn(J(()).d))
where DKI,(-l) is the KullbackLeibler divergence and LeaR is the pairwise ranking loss computed against pseudo-labels dz. Let the combined empirical risk be
Rio(f) PKKIf) + APRR(()
Denote
fKL. arg min (f) fKL+PRR arg min Phot (f). JEF
FEF
Assume all losses bounded by B and Lipschitz continuous. Define the population risks
RKL (f) E (()) RPRR (f) E (PPRR((()))))))) (f() Reoi (f)
R
Then for any E (0, 1), with probability least 1 over the training draw,
In(1/6) 2(n m)
Riot (JKL.PRR)
< Resot (FKL) + 2m FcoF) + B
where R Rntm (lom(LoF) is the Rademacher complexity of the combined loss class ((x, y) = (uu(y) ALPRR(Z, y). In particular, if adding LPRR strictly lowers the empirical total risk, then the resulting population total risk is strictly smaller than that of the Kl.only solution.
Proof. By standard Rademachercomplexitt bounds (see [+]]), for every J E F, with probability at least 1 5,
Reot (J) S Rroo(S) + 2 (eo + BB
ln(1/8) 2(++m)
By definition of the minimizers.
RKL (fKL) s (FL(FKuFRRK) Riot (fKL+PAR) s Reol (fKL).
Subtracting these two inequalities shows that
Rhot (fKL+PRR) Recor (JKL.)
Reot (fKL+PRR) < Reot (JKI.).
Apply the uniformeonvergence bound from Step 1 to both fKL+PRR and fKI- The difference in their population risks is upperbounded by the difference in empirical risks (which is negative by Step 2) plus the same complexity term. Hence
Root (fKL+PRR) < Reot (fKL) + 22nmmooo + Bv
(n(1))
2(++m))
as claimed.
B The Proof of the Theorem 3.2
We study the generalization performance of Rankmatch. Before providing the main results, we first define the true risk with respect to the classification model of (x;
R() = B(z,,) [T(J(x),
17

---

Our goal is to learn a good classification model by minimizing the empirical risk R(f) - R2(f) (f),), where Rf) and Rf) are respectively the empirical risk of the labeled loss L1 (f((x), and unlabeled loss Lu (f(x),));
- n Σ aU), Ro(s) 1 m Σ Lu (f(x)),,)))
R2( =
Note that during the training. we cannot train model directly by optimizing Ru since the labels of unlabeled data are inaccessible. Instead, we train the model with (f) = 1 EE= Lo (f(xx), dz), where dj represents the pseudo-label vector of the instance Xj.
dht Let La (J(x)) d In h(ve (x)) be the loss for the label k, and LE be any (not necessarily the best) Lipschitz constant of L. Let Rx(FF) be the expected Rademacher complexity of FF with N m + training points, Let f be the empirical risk minimizer, where FF is function class, and f be the true minimizer. We derive the following theorem, which provides a generalization error bound for the proposed method.
Theorem B.1. Suppose that f(.) is bounded by B. For some >> 0, if ET I(h (x2)) - (x)
/m S for any l E [g). any 5 > O, with probability at least - 8, we have
log 2N
R(') ( S 2qBe + AgLpRN(()) + BqB
From Theorem 2, can be observed that the generalization performance of f mainly depends on and the number of training examples N. Apparently,
two factors, i.e., the pseudo-labeling a smaller pscudo-labeling error often leads to better generalization performance. Thanks to its robustness and the empirical evidence supporting the model, we anticipate strong performance in practical applications.
Theorem B.2. Suppose that E(-) is bounded by B. For some >> 0, if Ef I(A (xx)) - dgg
/m < for any ls E [4] for any 8 > 0, with probability at least 6, we have
log z
R(f) (Lf)-R(f"))) 2g 4gLg B2
2N
Proof. Before proving the theorem, we first provide two useful lemmas as follows. We primarily derive the uniform deviation bound between R(f) and R(f).
Lemma B.3, Suppose that the loss function is Lg-Lipschitz continuous with respect to f. For any
5 > O, with probability at least 1 5, have
log 3
IR()) R()) < 2qb Ra qB)
(12)
2(n + m)
Proof. In order to prove this lemma, we define the Rademacher complexity of T. and F with m training examples follows:
Ba (T (L F) = Ex.d.a sup
azl (f (xx), d:) + Σ oge (1 (x;), 1=1
fer
where o; and oy are Rademacher variables.
Considering that C(f(x), d) = Σm E(ʃk, dx), we have
Rn+m oL F) S gRn (lo F) s (Lg
18

---

where the second line is due to the Lipschitz continuity of the loss function E.
Then, we proceed the proof by showing that one direction sup R(f) H(f) is bounded with probability at least 1 - 6/2, and the other direction can be proved similarly. According to McDi- armid''s inequality [6]]. for any 6 > 0, with probability at least 8/2, have
log 2
sup R(î) - R()) & sup R(f) - R(J) + BB SEF
SEF
2(n + m))
According to the result in -1 (Theorem 3.3) that shows Esupjer R(f) - R(f) S 2R (F), by
further considering the other direction Jer R(J) - R()), with probability at least 6, we have
log
R(J) - 25 + qB
sup
SEF
2n + m
which completes the proof.
Then, can bound the difference between R(f) and R(f) follows:
Lemma B.4. Suppose that E(-) is bounded by B. For some O. if iz 1(f (x,))
/m S for any k E [g] for any 8 >> O, we have:
[R(l) Rv(s) Is qBe
Proof. Without loss of generality, assume that is the largest pseudo-labeling error among q classes, - I du
i... max -ド I[fx (x,))
//m for any [q]- Obviously, consists
below pscudo-labeling error:
E Σラ f& (xx)
dg
(13)
Then, we prove the following side, which provide the bounds for (f(f). Firstly, we prove its upper bound:
1
R2(()=
= m
I (fa (xx)) (f& (x,))
m
(
((fx((x)) + I(dd) fu (xx) e(fk (xx))
m
(14)
- Σ c (xz)) dith
+
efx (x,,)
< RR,()) + qBe
where the second line holds based on Eq Then, we prove its low bound:
RC()))
(fa (xx)) (fe (x²))
m ΣΣΣ
dd e(fr (xx)) - I(d,) fo (xx) e(fa (xx))
(15)
- Σ c (f (xx) dett)
-
m
+ Σ ((fx (x,))
1=1
> Rf) + gBe
19

---

By combining these two sides, we can obtain the following result:
[fc - Rf) S qlee
which concludes the proof.
For any 8 O, with probability at least - 6, we have:
log
R(F) S R(S + (4l(f)))+LLlBnnoooe + /B/
2N
log 2
R(J) Rr(() + gBe 2gL + /BB
2N
log 3
R(f) RJ( 2q 2qL Rm (F) + q
2NV
log 2
$ R(()) + Ru (f) + (ew (F)
Bq
2N
log
s R(f) + 2gBc + Fg + B
2N
where the first and fifth lines are based Eq. 6, and second and fourth lines due to Lemma The third line is by the definition of f. Putting all these together, the proof is then finished.
BK
C Others
Experimental Datasets :In this paper, validate approach using four distinct real-world datasets B, The details of these datasets are as follows:
Twitter-DDL: : A large-scale Visual Sentiment Distribution dataset was constructed from Twitter, en- compassing eight distinct emotions Amusement, Anger, Awe, Contentment, Disgust, Excitement, Fear, Sadness. Approximately 30,000 images were collected by searching various emotional key- words, such as "sadness," "heartbreak," and "grief." " Subsequently, eight annotators were hired to label this dataset. The resulting Twitter LDL dataset comprises 10,045 images.
Flickr-LDL subset of the Flickr dataset unlike other datasets that searched for images using emotional terms, the Flickr dataset collected 1,200 pairs of adjective-noun pairs, resulting in 500,000 images. We employed 11 annotators label this subset with tags for eight common emotions. In the end, the Flickr LDL was created, containing 10,700 images, with roughly equal quantities for each class.
Emotion6: Emotion6: We collected 1,980 images from Flickr using six category keywords and syn- onyms search terms for Emotion6. A total of 330 images were collected for each category, and each image was assigned to only one category (dominant emotion). Emotionó represents the emo- tions related each image in the form of a probability distribution, consisting bins, including Ekman's 's 6 basic emotions and neutral.
RAF-LDL: RAF-LDL is multi-label distribution facial expression dataset, comprising approxi- mately 5,000 diverse facial images downloaded from the internet. These images exhibit variations in emotion, subject identity, head pose, lighting conditions, and occlusions. During annotation, 315 well-trained annotators are employed to ensure each image can be annotated enough independent times. And images with multi-peat label distribution selected out constitute the RAF-IDDI.
Comparing methods In order to assess the effectiveness the proposed approach, we benchmark it against four sets of methods:
For the semi-supervised multi-label learning (SSMLL) algorithms, we follow the hyperparame- ter configurations provided in their original papers. Specifically, for SSMLL-CAP (CAP) [59] and
'The dataset's author has made the dataset wtt.://www.....c.ooooooeeeeeeea.
publicly available at the following
link:
20

---

PCLP [36], we adopt the same backbone networks, learning rates, batch sizes, and training schedules as specified in their respective works. Additionally, adapt these methods to our supe-vsee la- bel distribution learning (SSLDL.) setting, we modify their final activation layer: the original sigmoid function is replaced with a softmax activation. This transformation enables the models to generate pseudo-label distributions rather than independent multi-labe probabilities, ensuring better
2) The second group consists of two deep learning SSLDL algorithms that we introduced, named FixMatch-LDL and MixMatch-LDL. Since there are currently no open-source semi-supervised LDL works in deep learning, these two algorithms were developed by us, based on the current most effective two deep learning SSL algorithms.
(i) FixMatch- Fixmatch-LDL is an adaptation we made based the classic semi semi-supervised algorithm fixmatch [ Specifically, we pre-trained on images using ResNet50, then trained the model with labeled data. Subsequently, we assigned pseudo-label distributions to the unlabeled data, and finally, we aligned the model's strongly augmented output with the pseudo-label distribution. For all datasets, the number of epochs set 30 and the batch size is set as 32. We perform all experiments on GeForce RTX 3090 GPUs. The random seed is set to I for all experiments.
(ii) MixMatch-LDL: Mixmatch is semi-supervised LDL algorithm designed by us. Specifically, we first use linear interpolation to blend images, creating new samples. Similarly, we generate the label distributions for these new samples. Following this, train the data using the same training strategy as Mixmatch. It's worth mentioning that producing new samples enhances the model's s ability to prevent overfitting. For all datasets, the number of epochs is is set as 32. We perform all experiments on GeForce RTX 3090 GPUs. The random seed is set to 1 for all experiments.
30 and the batch size
3) The thrid group of algorithms is a deep learning SSLDL algorithm based on the dual-network concept, which we named GCT-LDL. The core idea involves mutual supervision of the outputs from two independent networks using unlabeled data. GCT-LDL: Two models utilized two differ- ent pretrained initializations of ResNet50 provided by PyTorch (ResNet50- Weights Ita.ttt...AEEEETIII K- V2).. During training, labeled and unlabeled data were
VI and ResNet50-Weightss. mixed. The loss used is the cross-entropy loss, divided into two parts: for labeled data, the loss is calculated directly between the prediction results and the ground truth. For unlabeled data, the loss is calculated between the prediction results of each model and the results of the other model. Hyperparameter settings are the same as those used in other methods.
4) The fourth group consists of traditional SSLDL algorithms, referred to as SA-LDL [0]] Since SA-LDL is an SSLDL algorithm designed for tabular data, we needed to perform feature engineer- ing on image data, first, we use ResNet-50 for feature extraction from all datasets, followed by dimensionality reduction to 128 dimensions using PCA. For the remaining settings, we adhere to the defaults specified in the paper.
5) The finally category consists of existing LDL algorithms. As there is currently only one open- source SSLDL algorithm, which is SA-LDI [18), we compared it with some state-of-the-art LDL algorithms. this regard, we selected four state-of-the-art LDL. algorithms: AdaS-LDL-SCLL [20] sLDLF [50], DF-LDL [[3]], and LDL-LRR [21]]. These algorithm settings are defaulted to be con- sistent with those specified in the paper. Additionally, for these algorithms, we directly use labeled data to train the classifier. Then, we use the trained model to assign pseudo-labels to the unlabeled samples. Finally, we use the pseudo-labels to update the model.
We evaluate LDL algorithms using six metrics: five distance-based (Cheby- shev, Clark, Kullback-Leibler, and Canberra) and two similarity- based (Cosine and Intersection). Lower values indicate better performance for distance- based metrics (L), while higher values indi- cate better performance for similarity- metrics (t).
Evaluation Metrics:
21

---

Table 5: The distribution distance/similarity measures and ranking correlation metrics
Measure / Metric
Formula
Chebyshev 4
Dis (d, d) = max, dg - d2
(d, dd
Clark |
Dis (d, d) = V 2=j=1
(d +dy )
|d d,1
Canberra +
Diss (isa(d, d) = Ej=1
dr tg
Kullback-Leibler)
Dis, (d, d) Ej=1 dg In di
Cosine f
Sim (d, d) =
Σj=1 djdj
v V d3
Σラ d3
Intersection f
Sim - 2j=1 min dee
201 de:
65
k(K D)
Spearman's 's rank ps
PS =
Kendall tau correlation TK t
TK ]R(k-1)
C.1 The Rest Experimental Results
Convergence Analysis Fig. f illlustrates the convergence curves of the RankMatch algorithm on the Flickr-LDL and Twitter-LDL datasets. The rapid decline in the initial loss for Flickr-LDL indicates quick adaptation and efficient optimization during early epochs, stabilizing as the model converges. On the other hand, Twitter-LDL demonstrates a more gradual decline. reflecting steadier learn- ing process. These results confirm the robust optimization capability of RankMatch across diverse datasets.
C.2 Parameter Sensitivity Analysis
To investigate the robustness of our method with respect to the trade-off parameter À in the PRR regularization term, we conduct sensitivity analysis on four datasets: Emotion6, Flickr-LDL, RAF- LDL, and Twi,ter-LDL. As shown in Fig. D, the performance remains stable across wide range of X values from 0.005 to 0.1, demonstrating that the proposed framework is not overly sensitive to this hyperparameter. A small X (e.g., 0.01) generally achieves the best or near-best results across most metrics, indicating that a moderate contribution from the PRR loss is sufficient to capture label- ranking consistency without dominating the primary KL-divergence objective. These results validate the robustness and general applicability of the proposed PRR- regglaegler SSLDI. framework across diverse datasets.
Ensdion8
TAF
Twile"
LooG
15 Epoch
10
Epoch
Epoch
Epach
(a) Emotion6
(b) Flickr-LDL
(c) RAF-LDL
(d) Twitter-LDL
Figure 6: The convergence curve on Emotion6, FLickk-LLDI, RAF-MI, and Twitter-LDI.
0.70
Ereatorn,
0:22
Emotenn
0.70
0.50
0.54
0.55
0.00
0.65
0.7
Me
Wilter
0.63
0.82
0.80
twiter
0.05
0.45
025
0.64
0.025
0.0:
0.05
0.085
0.03
0.25
65
0.055
0.01
0.05
0.503
0.61
0.05
(a)
(b)
(c)
(d)
Figure 7: Parameter Sensitivity Analysis on 4 datasets.
22


Reviews:
[{"review_id": "mAF3ZmlAmW", "reviewer": "Reviewer_KiaF", "summary": "This paper studies semi-supervised label distribution learning (SSLDL), a generalization of semi-supervised multi-label classification in which a small set of examples has dense label distributions and a larger set is unlabeled. The goal is to learn to predict full label distributions for new samples. The authors argue that pseudo-labeling, when combined with similarity-based distribution matching (e.g. CE, KL-div), while very effective for SSL, falls short for SSLDL because it fails to learn ranking correlations between labels. To address this issue, the authors introduce RankMatch, a deep-learning framework based on consistency regularization and pairwise label ranking. The main methodological contribution is the Pairwise Relevance Ranking (PRR) loss, which has two components: (1) The supervised PRR term aligns predicted and ground-truth label rankings, preserving relative class margins.  (2) The unsupervised PRR term penalizes incorrect ranking relationships on pseudo-labeled data, without enforcing margin constraints. Pseudo-labeled distributions are generated via an ensemble of different augmented “views” of each sample. In addition, the authors theoretically show that PRR reduces the generalization error and provide a generalization bound for the proposed method. Extensive empirical evaluation on multiple datasets and supervision levels demonstrates that RankMatch outperforms adapted baselines in both distribution alignment and ranking quality.", "questions": "## Questions for the Authors\n\n- **Q1: Relation to sharpening methods:** Is the absence of entropy minimization the key difference between the unsupervised consistency loss (Sec. 2.2.1) and methods like FixMatch or MixMatch (the latter also uses ensemble-based soft pseudo-labels)?\n\n- **Q2: Definition of δ (L102)**: The paper currently defines\n\n  ```math\n  δ = d_{x_i}^{y_k} − d_{x_k}^{y_j}.\n  ```\n\n  Would it be more consistent to set\n\n  ```math\n  δ = d_{x_i}^{y_j} − d_{x_i}^{y_k}\n  ```\n  to compare distances to two labels for the same input?\n\n* **Q3: Hyperparameter tuning.**\n  Can you provide practical guidelines or heuristics for selecting λ and t under very limited labeled data? What minimum validation-set size ensures stable tuning?\n\n* **Q4: Additional baselines.**\n  Could you evaluate against recent SSLDL-specific approaches [21, 31]? Including such comparisons could clarify any performance gap.\n\n* **Q5: Complexity analysis.**\n  Can you comment on the computational overhead of ensemble pseudo-label generation, especially for large-scale datasets?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\n- Addresses the relevant SSLDL task, reducing annotation costs to train with dense distributions while leveraging modern deep architectures.\n\n- Introduces PRR losses that effectively tackle ranking alignment under supervised and unsupervised regimes.\n\n- Provides theoretical generalization bounds that connect PRR minimization to error reduction.\n\n- Demonstrates consistent empirical gains in diverse datasets and supervision regimes, showing improvements in the quality of both predicted distributions and label rankings.\n\n**Weaknesses**:\n\n- W1 - Hyperparameters (λ and t) exhibit sensitivity; tuning under very limited labeled data may be unstable or unfeasible, with unclear requirements for validation data.\n\n- W2 - Most baselines are adapted from related but distinct tasks (e.g. supervised distribution learning, semi-supervised multi-label learning); comparison to the latest SSLDL-specific methods (e.g., [21, 31]) is missing.\n\n- W3 - The ablation study does not disentangle the effect of the supervised and unsupervised components of the PRR loss.", "was_revised": true, "final_justification": "I have carefully checked all rebuttals. \n\nThe authors have proactively investigated sensitivity issues, provided empirical guidelines for hyperparameter tuning for practitioners, implemented comparisons with more SSLDL baselines, added missing loss ablations, and elaborated on run-time. In addition, they have addressed other reviewers' concerns, particularly regarding class imbalance and applicability to large-scale datasets (e.g. Pascal VOC and COCO) and clarified notation issues and their method's connection to related methods.\n\nConsidering all this, I've decided to raise my score to accept the paper.", "rebuttal": "**W1.provide practical guidelines or heuristics for selecting λ and t under very limited labeled data**\n\nRW1: Thank you for your valuable comment. We agree that tuning hyperparameters such as λ and t may be particularly challenging when labeled data is extremely limited. To address this issue, we conducted additional experiments under the low-label setting (10% labeled data) with varying validation set sizes (5%, 10%, 15%, and 20%) to analyze the stability and robustness of hyperparameter selection. \n\nTable 1. Validation performance under different values of λ \n| che↓      | Val-Label % | λ = 0 | λ = 0.1 | λ = 0.2 | λ = 0.5 | λ = 1.0 |\n|--------------|-------------|-------|--------|--------|--------|--------|\n| Emotion6     | 5           | 0.725 | 0.695  | 0.691  | 0.706  | 0.715  |\n| Emotion6     | 10          | 0.664 | 0.635  | 0.631  | 0.648  | 0.658  |\n| Emotion6     | 15          | 0.608 | 0.582  | 0.579  | 0.593  | 0.603  |\n| Emotion6     | 20          | 0.549 | 0.527  | 0.522  | 0.538  | 0.547  |\n| Twitter-LDL  | 5           | 0.286 | 0.267  | 0.264  | 0.274  | 0.280  |\n| Twitter-LDL  | 10          | 0.250 | 0.234  | 0.232  | 0.241  | 0.247  |\n| Twitter-LDL  | 15          | 0.241 | 0.225  | 0.222  | 0.231  | 0.237  |\n| Twitter-LDL  | 20          | 0.237 | 0.222  | 0.219  | 0.228  | 0.233  |\n\n\nTable 2. Validation performance under different values of t\n| che↓ | Val-Label % | t = 0 | t = 0.1 | t = 0.2 | t = 0.5 | t = 1.0 |\n|-------------|-------------|-------|--------|--------|--------|--------|\n| Emotion6    | 5           | 0.740 | 0.719  | 0.715  | 0.706  | 0.731  |\n| Emotion6    | 10          | 0.664 | 0.640  | 0.636  | 0.648  | 0.653  |\n| Emotion6    | 15          | 0.608 | 0.584  | 0.579  | 0.593  | 0.597  |\n| Emotion6    | 20          | 0.549 | 0.527  | 0.521  | 0.538  | 0.538  |\n| RAF-LDL     | 5           | 0.295 | 0.277  | 0.272  | 0.274  | 0.287  |\n| RAF-LDL     | 10          | 0.281 | 0.263  | 0.256  | 0.241  | 0.278  |\n| RAF-LDL     | 15          | 0.268 | 0.251  | 0.244  | 0.231  | 0.265  |\n| RAF-LDL     | 20          | 0.252 | 0.234  | 0.228  | 0.228  | 0.249  |\n\nThe results reveal the following practical guidelines:\n\n- **For λ**: We observed that moderate values (e.g., λ = 0.1 ~ 0.2) generally lead to the most stable or near-optimal performance across different datasets and validation splits. This choice effectively balances regularization strength and the risk of overfitting. In contrast, λ = 0 (i.e., no regularization) usually performs poorly, especially when the validation set is small. Although λ = 1.0 can occasionally yield good results, its performance tends to be less stable and more sensitive to validation bias.\n\n- **For t**: Extreme values (t = 0 or t = 1.0) typically result in suboptimal performance. In contrast, intermediate values (t = 0.2 ~ 0.5) often produce better or second-best results, particularly on the Emotion6 and RAF-LDL datasets. As the validation size increases, the performance gap between different t values narrows, but intermediate values consistently offer stronger robustness under low-resource conditions.\n\nThese findings provide empirical guidance for practitioners to tune hyperparameters effectively in low-label scenarios, especially when validation data is limited.\n\n\n  \n**W2. Lack SSLDL baseline**\n\n RW2: We sincerely thank the reviewer for the insightful suggestion. To address the concern regarding the lack of comparison with recent SSLDL-specific baselines (e.g., IS [21] and Neu[31]), we have conducted additional experiments on these two methods.\nSince the official implementations of IS [21] and Neu [23] are based on MATLAB and do not support direct processing of image data, we first extracted visual features using a ResNet-50 backbone and reduced the dimensionality to 128 via PCA to fit their input format. All other experimental settings strictly follow the original paper defaults.\nThe following table summarizes the Canberra distance (↓ lower is better) of RankMatch, IS [21], and Neu [31]  across four benchmark datasets under different label ratios.\n\n \n| Dataset Can↓ | Method | 10%   | 20%| 40%   |\n|---------------|-------------|-------|-------|-------|\n| Emotion6      | RankMatch   | **3.3902** | **3.3176** | **3.2504** |\n|               | IS [21]     | 3.7951 | 3.7613 | 3.7248 |\n|               | Neu [31]    | 4.0815 | 4.1128 | 4.1204 |\n| Flickr-LDL    | RankMatch   | **4.4060** | **3.9964** | **3.9013** |\n|               | IS  [21]| 5.3827 | 5.3235 | 5.2676 |\n|               | Neu [31]    | 6.1634 | 5.9889 | 5.6508 |\n| Twitter-LDL   | RankMatch| **3.7370** | **3.6962** | **3.2913** |\n|               | IS  [21]| 5.8983 | 5.7659 | 5.6366 |\n|               | Neu  [31]| 6.5220 | 6.4081 | 6.3575 |\n| RAF-LDL       | RankMatch   | **3.0178** | **2.9358** | **2.8341** |\n|               | IS  [21]| 3.4385 | 3.2808 | 3.1966 |\n|               | Neu  [31]| 3.0891 | 3.0242 | 2.9912 |\n\n\nThese results clearly demonstrate that our proposed method, RankMatch, consistently achieves lower Canberra distances across all datasets and label ratios when compared to the recent SSLDL-specific baselines IS [21] and Neu [31]. The performance margins are particularly pronounced on Flickr-LDL and Twitter-LDL, where RankMatch outperforms the second-best method by a notable margin, highlighting its robustness and effectiveness under limited supervision.\n\nThese additional results and analysis will be incorporated into Section 4.4: Comparison with SSLDL Methods in the revised version of the paper, along with discussions on computational setups and practical applicability.\n\n[21] Semi-supervised label distribution learning via projection graph embedding. Information Sciences.\n \n\n[31] Semi-supervised label distribution learning with co-regularization. Neurocomputing.\n \n\n**W3.  Ablation study on PRR_U loss**\n\nRW3: Thank you for your suggestion. We have added an ablation study to disentangle the effect of the two components in the PRR loss.\nAs shown below, both the unsupervised part (PRR_U) and the supervised part (PRR_L) contribute positively. Adding PRR_U already improves the results, and further including PRR_L brings additional gains. This shows both components are useful and complementary.\n \n| Dataset | Method                                 | Che. ↓          | Cla. ↓           |\n|---------|----------------------------------------|------------------|------------------|\n| Flickr| pretrain + consistency| 0.2262 (6.2%↑)    | 2.1131 (6.5%↑)    |\n|| + PRR_U loss| 0.2215 (2.1%↑)    | 2.0783 (1.6%↑)    |\n|| + PRR_U + PRR_L loss (full PRR loss)| **0.2184 (1.4%↑)** | **2.0158 (3.0%↑)** |\n| RAF| pretrain + consistency| 0.2550 (13.2%↑)   | 1.5021 (2.5%↑)    |\n|| + PRR_U loss| 0.2450 (3.9%↑)    | 1.4950 (0.5%↑)    |\n|| + PRR_U + PRR_L loss (full PRR loss)| **0.2341 (4.5%↑)** | **1.4914 (0.2%↑)** |\n \n\nWe will include these results in the revised version of Table 4.\n \n \n \n\n**Q1. Relation to sharpening methods**\n\n**R1:** Thank you for your insightful question regarding the relation between our unsupervised consistency loss (Section 2.2.1) and sharpening-based methods such as FixMatch and MixMatch. While our approach shares the overarching principle of enforcing prediction consistency under perturbations, it fundamentally differs in both the loss formulation and suitability for label distribution learning (LDL).\n\n1. **No Entropy Minimization or Confidence Filtering**: Methods like FixMatch often apply entropy minimization or confidence thresholding to encourage low-entropy (i.e., nearly one-hot) pseudo-labels. However, our approach **does not include any entropy-based terms**. Instead, we employ KL divergence between the predicted label distributions of the original and perturbed inputs to ensure consistency. This design avoids overconfident pseudo-labels that may harm distribution modeling.\n\n2. **Incompatibility of Sharpening with Label Distribution**: Sharpening functions, by design, amplify dominant labels while suppressing others, which may be beneficial for single-label classification. However, in our setting—where the goal is to predict a **full label distribution** that reflects label ambiguity—sharpening artificially distorts the soft structure by **disproportionately down-weighting minor but meaningful label components**. This leads to biased distributions that deviate from the ground-truth, especially in cases where multiple labels have comparable semantic relevance. Therefore, sharpening is **not appropriate for LDL**, and our consistency loss is carefully designed to preserve distributional fidelity while enhancing robustness.\n\n\n\n**Q2. Definition of δ (L102)**\n\n \nR2: Thank you for pointing this out. We agree with the reviewer that the more consistent definition should be:  \n$\n\\delta = d_ {\\\\mathbf{x} _i}^{y _j} - d _{\\\\mathbf{x} _i}^{y _k}\n$  \nThis change better aligns with the intended meaning of comparing the distances to two labels for the same input.  We will revise the notation in the final version accordingly.\n \n\n **Q3. Hyperparameter tuning**  \n\n\nR3: Details can be find on  **RW1**.\n \n \n\n**Q4. Additional baselines**  \n\n\nR4. Details can be find on **RW2**.\n\n \n **Q5. Complexity analysis**  \n\n\nR5: Our ensemble pseudo-label generation relies solely on forward passes with different dropout masks (or perturbations) and does not involve backpropagation or model re-training. As a result, the computational overhead is relatively low and scales linearly with the number of ensemble passes.\n \nTo illustrate, we measured the pseudo-label generation time on three datasets of varying scales using 5 ensemble rounds on a single NVIDIA RTX 3090 GPU:\n \n- Emotion6 (small-scale): ~1.8 seconds\n- Twitter-LDL (medium-scale): ~2.5 seconds\n- Flickr-LDL (large-scale): ~3.0 seconds\n \nThese results suggest that our method incurs negligible overhead, even on large-scale datasets. Moreover, this step is only performed once per iteration during training, making it practically efficient."}, {"review_id": "DMGky8zJHZ", "reviewer": "Reviewer_paT1", "summary": "This paper proposes RankMatch, a semi-supervised learning framework for label distribution learning (LDL) problem using pseduo-labels. As existing pseudo-labels fails to consider the ranking relationships between different true labels, RankMatch addresses this issue by including the ranking correlations between labels into the learning process. Specifically, the paper proposes a Pairwise Ranking Relation (PRR) Loss, which improves the pseudo-labels to capture the ranking correlations between labels. In addition, a flexible pseudo-label training stratygy is proposed to prevent the model from overfitting to the absolute label differences. Experiments are conducted on several LDL benchmarks and showed competitive results on the ranking-based metrics.", "questions": "- Please see the weaknesses above", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThis paper first improves the pseudo-labeling strategy of existing semi-supervised learning algorithms in label-distribution learning scenario, by introducing the ranking correlations between labels. This approach seems to be original, and is effective to improve the pseudo-labeling mechanism from the empirical results. In addition, the ranking correlation is supported by the theoretical gaurantee, making the proposed method plausible. \n\n\nWeaknesses\n\n- All the considered benchmarks are small-scale and not standardized, and the proposed method needs to be evaluated on more large-scale and commonly used benchmarks in multi-label classification and semi-supervised learning literature, such as Pascal VOC, COCO, and OpenImages for multi-label classificaiton and CIFAR10/100 and STL10 for semi-supervised learning. \n\n- The considered evaluation metrics include only correlation-based ones, and accuracy related metrics such as top-1 accuracy, precision/recall, and mAP are not considered, which are commonly adopted in single/multi-label classification domains. \n\n- The proposed method which utilizes the relationships across different classes has underlying assumption that the class distribution of labeled and unlabeled data are identical. As it is more natural that unlabeled data often violate the condition, it is unclear whether the proposed method can be robust under such a condition.", "was_revised": true, "final_justification": "I appreciate the authors for making clarifications and additional experiments requested for the rebuttal. It addresses my concern and the work is now solid. Therefore, I would like to raise my rating.", "rebuttal": "**Q1: Lack of results on PASCAL, COCO...datasets**\n\n**R1**: We sincerely thank the reviewer for suggesting evaluation on more standardized and larger-scale benchmarks. Our work focuses on **semi-supervised label distribution learning (SSLDL)**, where supervision is provided in the form of **soft label distributions**, as opposed to the hard labels commonly used in traditional multi-label classification or semi-supervised learning.\n\nThe datasets used in our main experiments (e.g., Emotion6, Twitter-LDL, RAF-LDL, and Flickr-LDL) are widely adopted standard benchmarks in the LDL and SSLDL literature [1,2,3], as they offer soft label annotations required for this task. Unfortunately, popular datasets such as  CIFAR10/100 do not natively provide label distributions and are thus not directly applicable to our setting.\n\nTo address the reviewer’s concern, we further constructed two new **multi-label distribution learning (LDL)** datasets from the Pascal VOC and COCO datasets, referred to as **VOC-LDL** and **COCO-LDL**, to validate the effectiveness of our method under more realistic and large-scale settings.\n\nNotably, both Pascal VOC and COCO are originally **semantic segmentation datasets** with bounding box and mask annotations. We first convert them into **multi-label classification datasets** by treating each image as containing multiple object categories. Then, we transform these into LDL datasets by calculating the proportion of the segmentation mask area that each category occupies within an image. These proportions are then normalized to form valid label distributions.\n We evaluate performance using the same protocol and report results in the below.\n\n| Method|Ratio|COCO-Can.↓|COCO-Che.↓|VOC-Can.↓|VOC-Che.↓|\n|-------|-----|----------|----------|---------|---------|\n|RankMatch|10%|**4.210**|**0.251**|**3.894**|**0.235**|\n|SSMLL-CAP|10%|4.496|0.272|4.088|0.251|\n|PCLP|10%|4.537|0.281|4.193|0.259|\n|Fixmatch-LDL|10%|4.802|0.293|4.361|0.266|\n|Mixmatch-LDL|10%|4.925|0.298|4.493|0.273|\n|GCT-LDL|10%|4.612|0.278|4.167|0.254|\n|SALDL|10%|4.469|0.269|4.073|0.246|\n|sLDLF|10%|5.220|0.318|4.851|0.281|\n|DF-LDL|10%|5.086|0.312|4.699|0.272|\n|LDL-LRR|10%|5.301|0.326|5.002|0.289|\n|Adam-LDL-SCL|10%|5.087|0.313|4.578|0.271|\n|RankMatch|20%|**3.982**|**0.232**|**3.672**|**0.219**|\n|SSMLL-CAP|20%|4.365|0.263|3.889|0.241|\n|PCLP|20%|4.418|0.271|4.002|0.248|\n|Fixmatch-LDL|20%|4.672|0.284|4.218|0.256|\n|Mixmatch-LDL|20%|4.793|0.289|4.336|0.263|\n|GCT-LDL|20%|4.482|0.267|4.010|0.246|\n|SALDL|20%|4.351|0.260|3.928|0.239|\n|sLDLF|20%|5.105|0.312|4.787|0.276|\n|DF-LDL|20%|4.979|0.308|4.610|0.268|\n|LDL-LRR|20%|5.203|0.319|4.912|0.282|\n|Adam-LDL-SCL|20%|4.987|0.310|4.482|0.265|\n|RankMatch|40%|**3.761**|**0.218**|**3.483**|**0.205**|\n|SSMLL-CAP|40%|4.184|0.246|3.728|0.229|\n|PCLP|40%|4.240|0.255|3.845|0.235|\n|Fixmatch-LDL|40%|4.487|0.266|4.081|0.243|\n|Mixmatch-LDL|40%|4.602|0.272|4.194|0.250|\n|GCT-LDL|40%|4.319|0.253|3.894|0.239|\n|SALDL|40%|4.206|0.247|3.813|0.232|\n|sLDLF|40%|4.988|0.296|4.609|0.266|\n|DF-LDL|40%|4.881|0.290|4.450|0.258|\n|LDL-LRR|40%|5.089|0.303|4.743|0.273|\n|Adam-LDL-SCL|40%|4.879|0.292|4.327|0.255|\n-------\n\nThese results clearly demonstrate that RankMatch generalizes well even on large-scale and realistic multi-label LDL datasets, significantly outperforming baselines across both evaluation metrics. The strong performance on COCO-LDL and VOC-LDL suggests that our method is not only effective in conventional LDL settings but also robust when extended to more challenging, fine-grained, and high-dimensional multi-label distribution scenarios. This further confirms the scalability and practicality of our framework in real-world applications. We will include this discussion in a future version of the paper.\n\n\n\n\n**Q2 Lack  evaluation results such as top-1 accuracy or mAP**\n\n\n**R2**: We sincerely thank the reviewer for the insightful question regarding evaluation metrics. Our work focuses on Label Distribution Learning (LDL) and its semi-supervised variant SSLDL, where both supervision and predictions are soft label distributions rather than discrete labels. Thus, conventional classification metrics like Top-1 accuracy, precision, recall, and mAP are not directly applicable, as they assume categorical ground truths. Instead, LDL models aim to recover continuous-valued probability distributions.\n\nFollowing prior works in LDL and SSLDL [1,2,3,4], we adopt distribution-based distance metrics such as Canberra, Chebyshev, Clark, and KL divergence. These are specifically designed to measure differences between distributions and better reflect prediction quality in LDL settings.\n\nThat said, for applications where hard labels may be derived (e.g., via argmax over the predicted distribution), classification metrics like accuracy or mAP can be optionally computed. However, such a process discards the rich soft distribution information, which is the essence of the LDL paradigm.\nTo further address the reviewer’s concern regarding classification-oriented evaluation, we conducted single-label (Top-1 Accuracy) and multi-label (mAP) assessments by converting the predicted soft label distributions into discrete label sets. Specifically, we retain the original training data format using soft label distributions, while during evaluation, we transform the predicted distributions into\n\n\t•\tSingle-label outputs by selecting the top-1 category (argmax), and\n\n\t•\tMulti-label outputs by selecting the top-k categories with the highest predicted probabilities.\n\nThis conversion allows us to assess classification performance while preserving the LDL training paradigm. All baseline methods are evaluated under the same protocol for fairness. The experimental results are as follows\n\n|Method|Flickr Top-1 10%↑|Flickr Top-1 20%↑|Flickr mAP 10%↑|Flickr mAP 20%↑|Twitter Top-1 10%↑|Twitter Top-1 20%↑|Twitter mAP 10%↑|Twitter mAP 20%↑|\n|------|------------------|-------------------|----------------|----------------|--------------------|--------------------|-----------------|-----------------|\n|RankMatch|**0.708**|**0.721**|**0.652**|**0.666**|**0.692**|**0.705**|**0.630**|**0.645**|\n|SSMLL-CAP|0.672|0.689|0.612|0.628|0.658|0.674|0.595|0.610|\n|PCLP|0.684|0.697|0.624|0.638|0.671|0.684|0.610|0.624|\n|Fixmatch-LDL|0.669|0.683|0.609|0.624|0.660|0.675|0.600|0.615|\n|Mixmatch-LDL|0.658|0.672|0.597|0.612|0.649|0.661|0.589|0.603|\n|GCT-LDL|0.675|0.687|0.615|0.631|0.667|0.680|0.606|0.620|\n|SALDL|0.663|0.678|0.603|0.619|0.653|0.667|0.592|0.607|\n|sLDLF|0.619|0.635|0.557|0.572|0.610|0.625|0.546|0.563|\n|DF-LDL|0.648|0.662|0.584|0.598|0.636|0.651|0.575|0.590|\n|LDL-LRR|0.604|0.618|0.536|0.550|0.592|0.606|0.520|0.534|\n|Adam-LDL-SCL|0.638|0.653|0.570|0.585|0.628|0.642|0.557|0.572|\n\nAs shown on the Flickr-LDL and Twitter-LDL datasets, our method RankMatch consistently achieves superior performance under both Top-1 Accuracy and mAP, across all label ratios (10%, 20%). This demonstrates that RankMatch not only excels at recovering accurate soft distributions, but also yields strong discrete predictions under both single-label and multi-label interpretations, showcasing its versatility and practical effectiveness. We will include this discussion in a future version of the paper.\n\n\n**Q3 Robustness of imbalanced data**\n\n**R3**: We appreciate the reviewer’s insightful comment regarding the potential mismatch in class distributions between labeled and unlabeled data. We agree that this assumption may not always hold in real-world settings, where unlabeled data often exhibit label shift.\n\nTo address this, we conducted additional experiments following the protocol of [4], which introduces a parameter $\\gamma$ to control the imbalance level between labeled and unlabeled label distributions. Specifically, $\\gamma$ denotes the ratio of cumulative label descriptiveness between the most and least represented classes, thereby quantifying the severity of the label distribution shift.\n\n\n\nWe constructed unlabeled subsets with varying $\\gamma $ = 1, 10, 100 to simulate balanced, moderately imbalanced, and severely imbalanced conditions, respectively. The labeled data remained fixed during training. Experimental results on four benchmark LDL datasets under 10%, 20%, and 40% label ratios. To rigorously evaluate whether performance differences under different $\\gamma $ values are statistically significant, we performed one-way repeated-measures ANOVA across the three \\gamma settings per dataset and label ratio. This test is suitable here as all conditions (different $\\gamma $) are applied to the same model and dataset settings. The experimental results are as follows\n\n\n\n|Dataset|LabelRatio|$\\gamma $=1(Che.)|$\\gamma $=10(Che.)|$\\gamma $=100(Che.)|Significant|\n|--------|----------|---------------|----------------|------------------|------------|\n|Flickr-LDL|10%|0.2184|0.2190|0.2195|No|\n|Flickr-LDL|20%|0.2110|0.2117|0.2125|No|\n|Flickr-LDL|40%|0.2027|0.2032|0.2034|No|\n|Twitter-LDL|10%|0.1947|0.1956|0.1958|No|\n|Twitter-LDL|20%|0.1879|0.1882|0.1886|No|\n|Twitter-LDL|40%|0.1813|0.1819|0.1825|No|\n|RAF-LDL|10%|0.2341|0.2351|0.2355|No|\n|RAF-LDL|20%|0.2263|0.2270|0.2274|No|\n|RAF-LDL|40%|0.2175|0.2177|0.2181|No|\n|Emotion6|10%|0.2112|0.2119|0.2123|No|\n|Emotion6|20%|0.2036|0.2041|0.2047|No|\n|Emotion6|40%|0.1943|0.1948|0.1954|No|\n\n\n\n\nThe results indicate that performance differences across  $\\gamma $ values are consistently **not statistically significant** (all p-values > 0.05). This confirms that our method is **robust to label distribution shift**, maintaining stable performance even under severe imbalance conditions. We will include this discussion in a future version of the paper.\n\n[1] Predicting Label Distribution From Tie-Allowed Multi-Label Ranking. TPAMI 2023.\n\n[2] Label Distribution Learning by Maintaining Label Ranking Relation. TKDE 2023.\n\n[3] Fast Label Enhancement for Label Distribution Learning. TKDE 2023.\n\n[4] Zhao X, An Y, Xu N, et al. Imbalanced label distribution learning. AAAI 2023."}, {"review_id": "7Hrv7pB7c7", "reviewer": "Reviewer_fect", "summary": "This paper addresses a critical yet underexplored problem in label distribution learning (LDL)—namely, the challenge of ranking preservation under weak supervision. Although this issue inherently exists in LDL scenarios, it has been largely overlooked by prior work. The proposed method presents an elegant solution by introducing a novel ranking-based formulation, which is both theoretically grounded and empirically validated. The experimental results are promising, and the theoretical insights enhance the method’s credibility. Given the clarity of the motivation and the demonstrated effectiveness, this work is a meaningful step forward and has the potential to influence future research in the area.", "questions": "1. Where is the Related Work section located? It would be helpful to clarify whether related work is integrated into the introduction or presented as a standalone section.\n\n2. Some notations (e.g., in Eq. 3 and Eq. 5) could benefit from brief clarification for readers unfamiliar with the setup.\n\n3. While the paper motivates the use of label distribution learning (LDL), it would be helpful to elaborate on specific real-world tasks where the distributional supervision provides a clear benefit.\n\n4. How sensitive is the method to the choice of hyperparameters? While exact tuning is not necessary, a short discussion would be useful.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\n• The paper proposes a novel ranking-based framework, *RankMatch*, to address label distribution learning, which offers a fresh perspective compared to traditional regression-based objectives.\n\n• The use of pairwise label ranking allows the model to capture relative importance among labels, aligning well with the goal of label distribution modeling.\n\n• The method is simple yet effective, and achieves consistent improvements across multiple benchmark datasets.\n\n• The writing is clear and the motivation is well established.\n \n**Weaknesses**:\n\nThe method section is divided into too many small subsections, which slightly affects the overall readability. A more concise structure with fewer headings may improve clarity.", "was_revised": true, "final_justification": "Thanks to the authors for their responses. I have carefully read the rebuttal and all my concerns have been addressed. I also reviewed the comments from the other reviewers, and believe this is quite a solid work. I would like to raise my score to Accept.", "rebuttal": "**W1. Structure of the Method Section**\n\nRW1: We thank the reviewer for the helpful suggestion regarding the structure of the Method section. Our intention in using multiple smaller subsections was to improve **modularity** and facilitate **step-by-step understanding** of each component in the proposed framework. However, we understand that too many headings may fragment the narrative and affect overall readability. In the final version, we will revise the section to adopt a **more concise structure** by merging closely related components and **reducing the number of subsections**, while still retaining the clarity of technical contributions. We appreciate the reviewer’s feedback and will ensure the revised version reads more fluently.\n\n\n\n**Q1. Where is the Related Work section located?**\n\nR1. We sincerely thank the reviewer for raising this important question regarding the placement of the Related Work section. \n\nCurrently, due to the **strict page limit imposed by the NeurIPS submission format**, we have opted to place the detailed *Related Work* discussion in **Appendix A**. This is a **common practice in NeurIPS submissions**, especially when the primary focus must remain on presenting novel methodology, theoretical insights, and extensive experimental results within the main paper. We have made this tradeoff to ensure that our core contributions are communicated with sufficient depth and clarity.\n\nThat said, we fully agree that making the positioning of the related work more transparent would improve the overall readability of the paper. In the **camera-ready version**, we plan to:\n\n- Either **integrate the essential related work into the introduction** to improve contextual flow and grounding for the reader,  \n- Or, where formatting permits, **create a dedicated Related Work section** prior to the methodology section.\n\nAdditionally, we note that many of the relevant works are **already cited in context throughout the introduction, method, and experiment sections** to support discussion of design choices and comparisons with prior efforts. This was done intentionally to ensure readers are continuously aware of how our work relates to the broader literature.\n\nWe appreciate the reviewer’s suggestion and will revise accordingly to provide a clearer structure in the final version.\n\n\n\n\n\n\n**Q2.  Clarification of notations in Eqs. (3)–(5)**\n\n\nR2: We thank the reviewer for pointing out the potential ambiguity in our notation. To enhance clarity, we will revise the manuscript to provide **brief explanations near Eqs. (3)–(5)**. Specifically:\n\n- **Eq. (3)** defines the PRR loss, a **symmetric pairwise ranking regularization** term that encourages the predicted scores to respect the relative ground-truth importance between label pairs $(j, k) $. It symmetrically considers both $s(j,k) \\cdot g_ \\delta(j,k)$ and $ s(k,j) \\cdot g  _\\delta(k,j) $, promoting consistency.\n\n- **Eq. (4)** defines the binary indicator function $ s(j,k) $, which equals 1 **only if label $ j $** is significantly more important than label $k $ in the ground-truth soft label distribution — that is, $d^{y^j} _{x _i} - d^{y^k} _{x _i} > t $, where $t $ is a small threshold. This **filters out weak or noisy label preferences**, enforcing only meaningful pairwise relations.\n\n- **Eq. (5)** defines the penalty function $ g _\\delta(j,k) $, which penalizes violations between predicted scores and true label relations. Specifically, if the predicted score gap $h _j(x _i) - h _k(x _i) $ fails to meet the margin implied by the ground-truth gap $\\delta = d^{y^j} _{x _i} - d^{y^k} _{x _i} $,  the penalty is activated. This encourages the model to reflect **label preference orderings** encoded in the soft labels.\n\nWe will include a concise version of the above explanations directly in the main paper near the corresponding equations to improve readability and support understanding for readers unfamiliar with the LDL setup. We sincerely appreciate the reviewer’s suggestion.\n\n\n\n\n**Q3. Real-world applications of label distribution learning (LDL)**\n\nR3:  We thank the reviewer for the valuable question regarding practical applications of label distribution learning (LDL). LDL is a general framework that models **soft supervision**, where each instance is associated with a distribution over multiple labels instead of a single hard label. This enables it to effectively capture ambiguity, uncertainty, and nuanced patterns in real-world scenarios where hard labels are either infeasible or oversimplified.\n\nHere we briefly highlight several representative real-world tasks where LDL has demonstrated clear benefits:\n\n- **Facial age estimation**: In age estimation from facial images, individuals may appear to belong to a range of plausible ages rather than a single ground-truth number. LDL allows for learning from age distributions (e.g., Gaussian over possible ages) rather than a hard label, improving robustness and interpretability [1].\n\n- **Emotion recognition**: Emotional expressions often exhibit *overlapping cues* (e.g., “happy” with hints of “surprise”). LDL enables learning from soft emotion distributions (e.g., 30% happy, 50% surprise, 20% neutral), which reflect human perception more accurately than one-hot labels [2].\n\n\n- **Medical imaging**: In diagnosis tasks (e.g., X-ray or MRI interpretation), multiple plausible conditions may be present, or expert disagreement may arise. LDL provides a way to encode diagnostic uncertainty via probabilistic supervision [3].\n\n- **Multi-label learning with incomplete supervision**: LDL has been shown to be effective in weakly supervised multi-label tasks by recovering soft label distributions when only partial labels are observed, which is the focus of our work [4].\n\nThese examples illustrate that LDL is especially suitable for scenarios involving **label ambiguity**, **subjectivity**, or **incomplete labeling**, all of which are common in practical machine learning pipelines. Our method extends LDL to **semi-supervised settings**, further enhancing its applicability when full supervision is costly or unavailable.\n\n\n\n[1] Facial age estimation by learning from label distributions. IEEE TPAMI.  \n[2] Emotion distribution learning from facial expressions. ACM MM.  \n[3] Class-aware multi-level attention learning for semi-supervised breast cancer diagnosis under imbalanced label distribution. Medical & Biological Engineering & Computing.\n\n[4] Multi-label classification: An overview. Data Warehousing and Mining: Concepts, Methodologies, Tools, and Applications.\n\n **Q4.  Sensitivity to Hyperparameters**\n\nR4:  Thank you for the thoughtful question. We have reported detailed sensitivity analyses of key hyperparameters (λ and t) in **Figure 5** of the main paper and **Table 11** in **Appendix D**. These results demonstrate that our method maintains robust performance under a range of hyperparameter settings."}, {"review_id": "yhmIUsGRlk", "reviewer": "Reviewer_DrpT", "summary": "The paper explores an important yet insufficiently studied challenge in LDL—the preservation of ranking structures in the presence of biased or partial annotations. The authors propose a principled approach to tackle this issue by integrating ranking constraints into the learning process. The method is intuitive and supported by solid theoretical analysis. Empirical evaluations demonstrate its effectiveness across multiple datasets. While there is room for improving the clarity of the presentation and broader comparisons, the paper makes a meaningful contribution and opens up new directions for improving robustness in LDL frameworks.", "questions": "Q1. Why did the authors not include commonly used multi-label evaluation metrics such as mean Average Precision (mAP) or Hamming Loss in the performance evaluation? These metrics might better reflect the model’s behavior in ranking or partially correct predictions.\n\nQ2. In Equation (3), a ranking margin \\delta is introduced. How is this hyperparameter chosen? Is the same value of \\delta used across all datasets? Is the model performance sensitive to this margin?\n\nQ3. The paper mentions filtering pseudo-labels based on ranking consistency (Eq. 5). Could the authors clarify the specific criterion used for filtering? For example, is a threshold applied, and if so, has it been tuned or validated experimentally?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.The paper proposes a novel method for semi-supervised label distribution learning (SSLDL), named RankMatch, which innovatively incorporates inter-label ranking correlations. This is a significant contribution, as ranking-based information has been largely underexplored in previous SSLDL approaches. By leveraging label ranking structures, the method enhances the flexibility and robustness of pseudo-labeling in weakly supervised scenarios.\n2.The proposed method demonstrates broad applicability, particularly in domains where label ambiguity is inherent—such as facial emotion recognition or crowd counting. Its ability to effectively utilize both labeled and unlabeled data reduces reliance on fully labeled datasets, thus improving the practicality and scalability of SSLDL in real-world applications.\n3.The paper is well-organized and clearly written. The theoretical development is presented in a logical and gradual manner, and the experimental results are comprehensive, using well-established metrics to demonstrate the method’s superiority. The empirical design is thoughtful, and results are easy to interpret.\n\nWeaknesses:\nThe paper would benefit from providing the implementation code of RankMatch. Open-sourcing the code would promote reproducibility and enable other researchers to validate and build upon the proposed method, thereby increasing its impact within the community.", "was_revised": true, "final_justification": "Thank you to the authors for their detailed and thoughtful responses to the reviewers' comments. I have carefully reviewed the replies. The authors have addressed the concerns raised in the previous review. I would like to raise my score to Accept.", "rebuttal": "**W1. Code Availability and Reproducibility**\n\nRW1: We thank the reviewer for highlighting the importance of reproducibility and community impact.\n\nWe fully agree that open-sourcing the implementation is crucial for promoting transparency and encouraging further research. We are currently in the process of cleaning and organizing the codebase of **RankMatch**, and we will publicly release it upon the paper’s acceptance. We believe this will greatly facilitate future work on semi-supervised label distribution learning and ensure the reproducibility of our results.\n\n **Q1.  On the Use of mAP and Hamming Loss**\n\nR1: We thank the reviewer for raising this insightful point. Our work is based on **Label Distribution Learning (LDL)**, where both the supervision signal and the prediction target are **soft label distributions** over classes, rather than binary vectors or ranking lists as in classical multi-label learning.\n\nConventional multi-label evaluation metrics such as **mAP**, **F1-score**, and **Hamming Loss** rely on thresholding and assume **discrete binary labels**, making them incompatible with the goals and outputs of LDL models. In LDL, the prediction is a **probability distribution** over all labels (e.g., `[0.3, 0.1, 0.2, 0.4]`), and evaluation focuses on how close this predicted distribution is to the **ground-truth distribution** using distributional metrics such as **KL divergence**, **Chebyshev**, and **Canberra** distances.\n\nFollowing standard practice in the LDL and semi-supervised LDL (SSLDL) literature [1, 2, 3], we adopted distribution-based metrics such as:\n\n- **Chebyshev distance**\n- **Canberra distance**\n- **Clark distance**\n- **Kullback–Leibler (KL) divergence**\n\nThese metrics are specifically designed to measure discrepancies between two probability distributions, which aligns well with the learning objectives of LDL.\n\nThat said, to further address the reviewer’s concern, we have also reported **mAP** and **Top-1 accuracy** in additional experiments (see ** Reviewer paT1 R2**). These are computed by converting soft distributions to **rankings** or **top-K predictions**. Our method achieves **state-of-the-art performance** under these metrics as well, demonstrating its robustness across both **soft (distributional)** and **hard (classification-style)** evaluation criteria.\n\n[1] Facial age estimation by learning from label distributions. IEEE TPAMI\n\n[2]  Imbalanced label distribution learning. AAAI\n\n[3] Label distribution learning. IEEE TKDE\n\n\n\n\n**Q2. Clarification on the Margin $\\delta$  in Equation (3)**\n\nR2: We thank the reviewer for the question regarding the interpretation and sensitivity of the margin in our ranking loss formulation.\n\nIn Eq. (5), $\\delta$ is **not** defined as the difference between the ground-truth label distributions. Instead, it is a **fixed hyperparameter** that serves as a confidence margin in the pairwise ranking regularization.\n\n\n\nOtherwise, a penalty is applied to the loss. This mechanism ensures that the model not only respects the label preference implied by the distribution but does so with **sufficient confidence**.\n\n\n\nWe will revise the paper to clarify this definition and remove any ambiguity in notation.\n\n\n\n\n **Q3. Clarification on Ranking Consistency Filtering (Eq. 5)**\n\nR3: We thank the reviewer for the question. Our method filters pseudo-labels based on the consistency between predicted scores and the ground-truth soft label distribution.\n\nSpecifically, for each label pair (j, k), if the ground-truth distribution satisfies $ d _{x _i}^{y^j} > d _{x _i}^{y^k} $, we expect the model prediction to follow $ h _j(x _i) > h _k(x _i) $. A violation occurs when this preference is not respected.\n\nTo ensure that we only enforce meaningful and robust preference relations, we introduce a **margin threshold** $ t $ in Eq. (5). We ignore label pairs where the difference in ground-truth importance is too small (i.e., $ |d _{x _i}^{y^j} - d _{x _i}^{y^k}| \\leq t $). This is important because small differences can be noisy or ambiguous, and forcing the model to learn from them may reduce robustness.\n\n This margin helps the model focus on more confident and informative ranking signals."}]

Author rebuttal:

Meta-review: This paper introduces a novel contribution to label distribution learning. Initially, the reviewers raised concerns about the lack of experimental details. However, after the rebuttal, the authors successfully addressed most of these concerns: they investigated sensitivity issues, provided empirical guidelines for hyperparameter tuning for practitioners, implemented comparisons with additional SSLDL baselines, added missing loss ablations, elaborated on run-time costs, evaluated imbalance and applicability to large-scale datasets (e.g., Pascal VOC and COCO), and clarified notation issues as well as their method’s connection to related approaches. The AC has carefully reviewed the paper, the reviewer comments, and the rebuttal, and agrees that the paper is well-motivated, clearly written, and supported by thorough experiments. Therefore, the AC recommends acceptance. It is strongly encouraged that the authors incorporate all additional experiments and discussions from the rebuttal into the final version.
Decision: Accept (poster)