Towards a Pairwise Ranking Model with Orderliness and Monotonicity for Label Enhancement
Authors: Yunan Lu
Xixi Zhang
Yaojin Lin
Weiwei Li
Lei Yang
Xiuyi Jia
Venue: NeurIPS 2025 spotlight ()
Keywords: label polysemy
label distribution learning
label enhancement
Primary area: general_machine_learning

Abstract: Label distribution in recent years has been applied in a diverse array of complex decision-making tasks. To address the availability of label distributions, label enhancement has been established as an effective learning paradigm that aims to automatically infer label distributions from readily available multi-label data, e.g., logical labels. Recently, numerous works have demonstrated that the label ranking is significantly beneficial to label enhancement. However, these works still exhibit deficiencies in representing the probabilistic relationships between label distribution and label rankings, or fail to accommodate scenarios where multiple labels are equally important for a given instance. Therefore, we propose PROM, a pairwise ranking model with orderliness and monotonicity, to explain the probabilistic relationship between label distributions and label rankings. Specifically, we propose the monotonicity and orderliness assumptions for the probabilities of different ranking relationships and derive the mass functions for PROM, which are theoretically ensured to preserve the monotonicity and orderliness. Further, we propose a generative label enhancement algorithm based on PROM, which directly learns a label distribution predictor from the readily available multi-label data. Finally, extensive experiments demonstrate the efficacy of our proposed model.

Towards a Pairwise Ranking Model with Orderliness and Monotonicity for Label Enhancement
Yunan 22, Xixi Zhang', Yaojin Lin Weiwei Li*, Lei Yang?, Xiuyi Jia
Nanjing University of Science and Technology, Nanjing, China
The Hong Kong Polytechnic University, Hong Kong, China
3 Minnan Normal University, Fujian, China
Nanjing University of Aeronautics and Astronautics, Nanjing, China (luyn zhangxixi, jjaaxy (ajaax)e@jjut...... @n edu. en, yj yjlin@mnnu. edu. en, iiveiveinnua.. edu. en, ray. yang@polyu. edu. hk
Abstract
Label distribution in recent years has been applied in a diverse array of complex pocision-making tasks. To address the availability of label distributions, label enhancement has been established as an effective learning paradigm that aims automatically infer label distributions from readily available multi-labe data, e.g., logical labels. Recently, numerous works have demonstrated that the label ranking significantly beneficial to label enhancement. However, these works still exhibit deficiencies in representing the probabilistic relationships between label distribution and label rankings, or fail to accommodate scenarios where multiple labels are equally important for given instance. Therefore, we propose PROM, a pairwise ranking model with orderliness and monotonicity, to explain the prob- abilistic relationship between label distributions and label rankings. Specifically, we propose the monotonicity and orderliness assumptions for the probabilities different ranking relationships and derive the mass functions for PROM, which are theoretically ensured to preserve the monotonicity and orderliness. Further, we propose generative label enhancement algorithm based on PROM, which directly learns a label distribution predictor from the readily available multi-laber data. Finally, extensive experiments demonstrate the efficacy of our proposed model.
Introduction
Label polysemy, i.e., the cases where an instance is associated with multiple labels simultane- ously, a common phenomenon in real-world tasks. To preserve label polysemy, multi-label learning (Tsoumakas and Katakis, 2006) employs binary values to indicate the presence or absence of each label for a given instance. However, multi-label learning cannot directly handle a further question with more polysemy: How much is each label associated with instance? Hence, Geng (2016) introduced LDL (Label Distribution Learning). Unlike multi-label learning, LDL assigns a real-valued vector each instance, resembling probability distributions, where each element, called label description degree, represents the extent to which the label describes the instance. By providing richer label information, LDL. has been applied in various fields, including age estimation (Gao et al., 2018, Geng et al., 2013) and affective analysis (Jia et al., 2019, Machajdik and Hanbury, 2010).
A significant bottleneck hindering the broader application of LDL is the challenge of acquiring ground-truth label distributions, as accurately quantifying these distributions can be costly. To address
"Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

this issue, LE (Label Enhancement) has been proposed to automatically infer label distributions from more readily available multi-label data by mining the underlying label polysemy informa- tion (Xu et al., 2018, 2020). Initially, most LE methods mainly focused on mining the underlying label polysemy information from instance correlation and label correlation. Recently, some novel findings (Jia et al., 2023b, 2024, Wang and Geng, 2021) that the label ranking is beneficial for improving the generalization of LDL have inspired a portion of LE research to explore the label polysemy information from the perspective of label ranking. For example, Jia et al. (2023a), Lu et al. (2023a.b) proposed to incorporate the ranking relationship between positive and negative labels into the loss function or the generation model of label distributions. Although these works have made headway in designing methods for regularizing LE processes by label rankings, there remain the following two research gaps. 1) Current works ignore the ranking relation of tie, i.e. the cases where multiple labels simultaneously describe the instance to almost the same degree, which is ubiquitous in real-world tasks. 2) Current works neglect the qualitative and quantitative modeling of the probabilistic relationships between label distributions and label rankings, which is essential for the interpretable and generative modeling of label enhancement. Besides, it should be noted that while some classic ranking models, such as Bradley- (Hunter, 2004) and Plackett-Luce (Guiver and Snelson, 2009) models, can be employed as a quantitative model, they also cannot explicitly model the tie relation.
Therefore, we propose PROM (a Pairwise Ranking model with Orderliness and Monotonicity) to qualitatively and quantitatively explain the probabilistic relationship between label distributions and label rankings, which can serve as generative distribution of label rankings in generative label enhancement or as a loss function for measuring the inconsistency between label rankings and label distributions. Specifically, we first conduct qualitative analysis of the probabilistic principles governing the generation process of label rankings from label distributions, and formalize these principles as the assumptions on the monotonicity and orderliness of the probabilities of different ranking relationships. Second, we design parameterized probability mass functions for the ranking model, and derive the conditions under which the model adheres probabilistic monotonicity and orderliness. Third, we propose a generative label enhancement algorithm based on the PROM model, called LE-PROM, which integrates the LE process and the LDL process into a unified framework. and directly learns an LDL mapping on the training instances with multi-label data. Finally, we validate our proposal through extensive experiments on real-world datasets. The experimental results demonstrate the superiority of our proposed method.
2 Related Work
To tackle the challenge of acquiring accurate ground-truth label distributions, LE (Label Enhancement) was proposed to automatically infer label distributions from more readily available multi-label data, such as logical labels (Xu et al., 2018, Kou et al., 2025), ternary labels Lu and Jia (2024), multi-label rankings (Lu and Jia, 2022, Lu et al., 2023c), or inaccurate label distributions Kou et al. (2024, 2023), Lu et al. (2025). Most LE methods mine the label polysemy from instance and label correlations. For example, in order to capture the instance correlation, the prototype-based LE algorithms (El Gayar et al., 2006, Jiang et al., 2006, Wang et al., 2023, Fan et al., 2024) identify the representative points and subsequently estimate the label distribution based on the representative points of each label. The graph-based LE algorithms (Xu al., 2018, Zhang et al., 2021, Xu et al., 2019. Liu et al., 2021) directly calculate affinities between instances based on the features, which is further utilized to regularize the label distribution matrix. The manifold-based LE algorithms (Hou et al., 2016, Wen et al., 2021, Tang et al., 2020, Zhang et al., 2018) learn coefficients for each instance, by which the feature vector of each instance can be linearly reconstructed from its neighbors, and then maintain the reconstruction within label distributions. In order to capture the label correlation, Luo et al. (2021) utilized the confusion matrix to estimate global label correlation. Besides, several works address the LE task under special data distribution. For example, VIB-ILE Song et al. (2024) addresses imbalanced label information in LE by adopting a variational information bottleneck mechanism and introducing consistency regularization.
Recently, portion of LE research attempts to explore the label polysemy information from the perspective of label ranking. For example, Jia et al. (2023a) utilized margin-based loss function to penalize the cases where the description degree of negative labels exceeds that of positive labels; Lu et al. (2023a) designs a conditional distribution of logical labels given label distributions, whose
2

---

parameters are regularized by the ranking between negative and positive labels. Lu et al. (2023b) derives a variational approximation for label distribution generative label enhancement, which is theoretically assured to uphold the ranking relation between negative and positive labels. Besides, Lu and Jia (2022), Lu et (2023c) theoretically and methodologically investigated the task of predicting label distributions directly from multi-label rankings. Despite the success of these efforts, there remains a gap in the qualitative and quantitative modeling of the probabilistic relationships between label distributions and the tie-allowed label rankings.
3 Methodology
In this section, we first introduce the notations commonly used in this paper, secondly elaborate our proposed PROM model, and finally illustrate the generative LE framework.
3.1 Notations
We denote the D-dimensional feature space by XD TR denote the M-dimensional label dis- tribution space by M = {o E R. Σm mm = 1), denote the label space by y = {lm 1M We cope with the training datasets that appear as data pairs ((x,. )x)] where In, E x and yn denote the feature vector and the vector of the easily available label values of the n-th instance, respectively. The data of y, can directly yield pairwise ranking relation among labels. We denote the ranking relation between label é; and label Ej by Eij E {4;
L, > ly Cz}. where & < ej. & > Eg. and & = E denote that the label &, compared to the label lj, describes the instance to a higher, lower, and approximately the same degree, respectively. The goal of LE is to learn label distribution predictor f X² + M, i.e., a mapping from feature space to label distribution space, by the training dataset ( yn) n== n=1-
3.2 PROM
In this subsection, we first propose assumptions to formalize the monotonicity and orderliness of ranking probability. Then, we derive the parametric form of PROM.
3.2.1 Fundamental Assumptions of PROM
On the one hand, we study how the label description degree affects the probability of the label ranking, focusing on the monotonicity of the probability of the label ranking. Obviously, given any two labels C, éj of an instance, the label é, is more likely to rank above the label l (or the label Ej is more likely to rank below the label & alternatively) if the description degree of E exceeds that of E by larger margin. Besides, if the description degrees of two labels are close, then the strict ranking relation between the two labels will be difficult to distinguish, i.c., it is more likely that the two labels are tied. We formalize the above intuition as the probability monotonicity assumption.
Assumption 3.1 (Probability monotonicity). Given any two labels E, of an instance and their respective description degrees 2z, 2j. p(l, x <Egl2; ua 2j 1y)) < p(f x Eglz 0i 2j = 02) and p(l, > Łg12 ue 2j uy) p(e, E,(2; v., 2j = uz) holds for any u; -u < 2/2 Uj- p(e, = l
2j = Uz)) p(e, E5l 04 2j eÿ) holds for any [n: uyl < |1u Egl. where us, uj, Us Uy E [o, 1].
On the other hand, we explore how the label description degree affects the orderliness among the probabilities of different rankings. Obviously, given any two labels E,, Ey of an instance, these two labels are most likely to be tied if the description degrees of the two labels are sufficiently close; the label & is most likely to rank above the label É, (or the label E, is most likely to rank below the label l alternatively) the label description degree of l, exceeds that of & by a sufficiently large margin. We formalize the above intuition as the probability orderliness assumption.
Assumption 3.2 (Probability orderliness). There exists two real thresholds -1 < =_ <0 < 24 1,
p(l; Eg|zi,z)) 2j) > p(e, Eglz, 2,) > p(e, - E 2,) holds for any 2, - 2j < z=; p(l >
Lz 2j) p(l: Ezlzi, 25) < p(li x =1] 2z) holds for any zi 2j z+: p(e. ~ Ez(24, 2z)
max(p(e; > Eg 24, 2, p(), Eg1=x,=2) m,)] holds for any =_ < σi =j Z+
3

---

The intuition of probability monotonicity and orderliness is visualized in Figure 1. Actually, the above discussion also implicitly adheres to the following two assumptions. The one that the ranking relation between two labels depends solely on the difference in their description de- grees, and is independent of the description de- grees themselves. We refer to this assumption as the translational invariance assumption. The other is that the ranking relation between two labels is symmetric, i.c., & E is actually l > 4, We refer to this assumption as the rank- ing symmetry assumption. We formalize these two assumptions as follows.
M( 2x))
0.6
4
K6. -t)
0.4
Jouraann
0.2
the vodua
staallr
0.0
-0.5
0.0
0.5
1.0
Figure 1: Schematic of probability monotonicity and probability orderliness.
Assumption 3.3 (Translational invariance). Given any two labels L, E, of an instance and their
respective description degrees 2;, 2j. p(e; < fj|2; the 2j 1ty) p(f; < fglzz 0i 2j = 01))
and p(l; > r2]2 = U4 2j = Uz) p(e; > Eg|2, : = 01 2j = 0;) hold for any uy "y = 8/ 1/;
p(l, ~ 2j|z U4 2j uz) p(4 E 04 2j m2) holds for any [n unl = =lv, 0g4. where u, リj U/, UU E [0.
Assumption 3.4 (Ranking symmetry). Given any two labels 4,, lg of instance and their respective description degrees 2,, 2j. p(4; < Lglzi,z) ple; > Lí22, zi) and p(e; ~ Lg(z,, p(lj ~ (12),2)).
3.2.2 Probability Mass Functions of PROM
Since the ranking relation between two labels can be E, - Eyy, 4, > ly. or ls = Cj. we model the ranking relation by categorical distribution, which can be formalized
Esj 24, 2j Categorical(E55 [ø(z,, 2y), 0(z,, 21), 24, 27]]),
(1)
where o(2), 5(z,, 2,), and 0(2, 23) capture the rules of generating the label ranking relation 4 > lj. l < ly. and & = e,. respectively. Considering that the softmax function is most commonly used to model the probability of discrete events, we preliminarily assume the parametric form of ((2,,2) 27) as follows:
((,,,,) p(4 > G)2,, = 7 --l-zzlexppp eexp(aa(2 - 2;) + ba),
0(2,,2) p(l, Ly12, 2, 2j) = -- -xxopp(a; 2 +ba),
(2)
5(., 21) p(4: < k 21) = - ( 'exp((z(2z 24) + bz),
where Z is a normalization factor that ensures p(2,, 22) + ò( 2,, 24) + 5( 24) 1, a. < 0, < O.
ag > O, and bg. b3. and bg are arbitrary real numbers.
Figure visualizes PROM model at different values of parameters. It is evident that not any value of parameter can adhere to the probability monotonicity assumption and the probability orderliness assumption. For example, Figure 2(a) violates the probability monotonicity assumption, Figure 2(b) violates the probability orderliness assumption, and Figure 2(c) violates both. Figure 2(d) shows PROM model that we expect. Therefore, we next propose four theorems to ensure that the probability mass functions defined by Equation (2) adhere to the proposed assumptions.
Theorem 3.5. p(Eiş 2i, 22) defined by Equation (1) and Equation (2) adheres to the translation invariance assumption, i.e. Assumption 3.3.
Theorem 3.6. p(Csilzis =3) defined by Equation (1) and Equation (2) adheres to the ranking symmetry assumption, i.e. Assumption 3.4, iff az -as and by -bg-
Theorem 3.7. Given as > O, as < 0, as < < A = 2; 22.. 3 adheres to the mono-
tonicity assumption of p(l, ~ ()1,,2) 2j) if ag ag exp(bs by): adheres to the mono- tonicity assumption of p(l, x 2z) (az(20g)~1
-1) A -1 hz(-1) > 0) V (A' > -1 A 0): @ adheres the monotonicity assumption of ple, Eg(2)1zz)

---

0.8
Function
KG (1
08
0.8
0.8
0.6
Rh ((
Roa
Functon
all 2
0.6
Function
Functon
204
2 0.4
67
0.4
pl0 - (//4)))
0.2
02
0.2
-1.0
-05
0.0
0.5
10
0.0
1.0
-05
0.0
0.5
0.0
-1.0
-05
0.0
0.5
100
0.0
-10
-0.5
0.0
0.5
1.0
2-3
4-3
5-
(a) 0 = 1. = 3, = 1.
(b)
1.b 3,
3.
(c) a
1,b = 5, 8 = 3.
(d) a
16, b = 8,
3.
Figure 2: The shape of PROM with varying values of parameters.
1.0
1. R
1.0 Vifoa
Lengite of be imaenel
06
D6
+0 4/
62
02
02
80.5
0.0
(a) b = 10, = 10.
(b) a 10, = 10.
(c) a = 25, b 20.
(d) Length of tie interval.
Figure 3: Marginal effect of parameters on the shape of PROM.
if (ag(2ag) > 1) 2 1A (1)) > 0) V (A < AA (s(4)) > 0) where A" A",
(A). and (D) are defined by X == (az + as+ + V(03 - -aa)² 8az) A
a3 (0z + as - / (ag - -a))²= - 8az) (4ag) rz(4) == a azA² + bs by + log
2a م A-a
(A) == -a34 + ag A + by - by + log ag as ag
-2
Theorem 3.8. p(Eig |2t 2j) defined by Equation (1) and Equation (2) adheres to the probability orderliness assumption, i.e. Assumption 3.2, iff max az by + ag } < az + bz < min
4z. ba (g) and b3 > max(bs ba)-
The proof of the above theorems can be found in Appendix A. According to the above theorems, let a = 0g = -ag. -a -a. by = bbg = by by. we finally define the probability mass function of
PROM model as follows:
b(z, 2),,,,)) = Z le a(x 2,), 8(2, 23) = z-1e - -(z 2,)+ 2,) 7 a-1ca(t--2))
(3)
where the parameters are bounded by b,t) E F, where the feasible region F is defined by Equation (4):
2h Al < b+a log 26
F F= ((,,b,,) [ (a > 00-11)000120)
OAL> 0) ^ (102 2 4b²
2a (4)
V(a 2 26) V (t<
2ava2 a² +
+2 +a² a²
log
Va² + 2h-
a
+
^ a < 4bb - 22b) }.
4b
2a
3.2.3 Marginal Effect of Parameters PROM
1,0,,,1 > E,) and l, ey Intuitively, the parameters a and b control the uncertainty of l, < by (or l, respectively. As shown in Figure 3(a) and Figure 3(b), larger and b correspond to the steeper density curve of the probabilities of E, < Ey (or E > L1)) and E, ~ ey. respectively. More results are shown in Appendix B. We do not show p(e; > Ez|2,,2 22) for the sake of readability. The parameter is similar to the temperature coefficient the softmax function, whose effect on the PROM distribution is shown in Figure 3(c). Overall, these three hyper-parameters directly affect the length of tie interval of PROM distribution, i.e., the length of the interval of 2; - 2j with p(c. = L,) > max (p(s x ()),(( > 42), which is shown in Figure 3()).
5

---

y
a
E
PROM
duuly a;
softmax(w)
softmax(w)
u
z
q(u m))
(a) Generation Process
(b) Inference Process
Figure Schematic diagram of LE-PROM.
3.3 LE-PROM
LE-PROM consists of a probabilistic generation process for observed variables (features, available labels, pairwise rankings) and a posterior inference process for latent variables (label distribution), which will be illustrated in the following two subsections.
3.3.1 Generation Process
Here we illustrate the process of generating the observations, which is shown in Figure 4(a) and formalized as follows.
Generate a sample of label distribution:
(a) Generate real-valued vector from standard multivariate normal distribution, i.e.,
u ~ N(u (u 0M, Js) where OM is an M-dimensional all-zero vector, IM is an M x M identity matrix.
(b) Transform the real-valued vector into the label distribution z, i.e., softmax(1) (u).
2. Generate the available label from a task-specific distribution given y ply 2)2.
3. Generate a sample of feature variables x from a multivariate normal distribution conditioned on the label distribution Formally, z N( Aa (z), (iag(o((z))) where u (-) and az (.) are two neural networks, and (iag(o)) is a diagonal matrix whose (i. i) entry is the i-th element of a.
4. Generate sample of pairwise rankings & from our designed PROM distribution conditioned on the label distribution z. Formally, E z II< PROM(E Ri Zz).
According to the above generation process, the joint distribution of the complete-dat data can be factorized as uu, æ, y, <) p( (x)p://)ll since z can be deterministically obtained from u.
It should be noted that omit label distribution
3.3.2 Model Inference
Here derive the inference method of our proposed generative model. The schematic is shown in Figure 4(b). We employ variational inference to approximate the true posterior, since the exact inference of the posterior of is intractable due to the nonlinear and non-conjugate relationships among variables. First, we assume that the variational posterior of u can be decomposed as
q(u (z, y, E (u [2, y. E), diag(o..(;; E)))
(5)
Then, we find the variational posterior distribution of u that is closest the true posterior distribution. It has been proven that the Kullback- the true posterior can be minimized by maximizing the ELBO (evidence lower bound):
(KL) divergence between the variational posterior and
ELBO- Ey(uln. [log p(æ, y, Elu)] (g (u/ x, y, {))))(
In the right-hand side of Equation (6), the first term is also known the reconstruction term, and the second term also known as the prior regularization term. The reconstruction term serves to
The task-specific distribution can be conditioned on u or since z can be deterministically obtained by u. Analogous to p(y|z), 2)) the multivariate normal distribution for features also be conditioned on u.

---

ensure that the label distribution can accurately reconstruct the observed variables. It is evident that an analytic form of reconstruction term is intractable due the integral of complicated likelihood function. Therefore, we turn to SGVB estimator. Since g u (r. y. {) is multivariate normal, it can be reparameterized by q(ulaz a y: {) nou (r. y: E) + ou (x. y: 5) @ 6, where e standard normal noise. Besides, benefiting from (Lu et al., 2023a), we utilize hyperparameters and / to re- re-weight the reconstruction quality of æ, y, and u to re-balance the magnitudes of different likelihood functions. Finally, the reconstruction term can be estimated by:
B y. Elu)] (1+a B)L
Σ logp (llu y. (x. e
to logp (lluu( E) + o. y. o d + Blog (E)mul(,, y. an y. o c(t)
where I. is the number of Monte Carlo samples, e(1) a sample from the standard normal distribution. The prior regularization term encourages the posterior approach the prior. Since both the u-posterior and u-prior are multivariate normal, their KL divergence can be computed analytically. Besides, in order to train a feature-conditione during the model inference phase, we add a prediction loss term to ELBO. Finally, the optimization objective can be formalized as follows:
label distribution predictor p( u x) N(u p(x),
arg max E &)) [log p(z, Elu)] PKL y, {))))(u) - XD (f(ul y, E))
(8) where À is trade-off hyperparameter. As suggested by Daunizeau (2017), since is the softmax normalization of Gaussian random vector, the expected label distribution based on pl u (z) can be approximated follows:
P4
pm
E Ep(ula) [Zm] ~
Umi
Ei exp(#mm))
v1 + (o + o
where μ4 and o, denote the i-th elements of ((z)) and (2x), respectively.
Experiments
4.1 Datasets and Evaluation Measures
We conduct experiments on 16 real-world datasets. These datasets are collected from diverse real- world tasks, including emotion analysis, movie rating prediction, and bioinformatics. Specifically, "Painting" (Machajdik and Hanbury, 2010), "Emotion6" (Peng et al., 2015), "Music" (Lee et al., 2021), "BU-3DFE" (Yin et al., 2006), and "JAFFE" (Lyons et al., 1998) are collected from emotion analysis tasks. "Movie" (Geng, 2016) is dataset from movie rating prediction task. "Alpha", "Cde", "Cold", "Diau", "Dtt", "Elu", "Heat", "Spo", "Spo5", and "Spoem" (Geng, 2016) are collected from gene expression analysis task. To accelerate convergence, we min-max normalization to preprocess the feature data. We evaluate the model performance by two representative LDL measures: KL "Kullback-Leible divergence) and Cosine (cosine similarity). The KL (or Cosine) metric with smaller values represents the better (or worse) performance. More details of the datasets and the results on more metrics be found in Appendix C.
4.2
Experiments on Logical Label Enhancement
Here, we aim to experimentally answer the question of whether our proposed LE-PROM is superior the state-of-the-art logical label enhancement algorithms.
4.2.1 Experimental Configurations
Experimental Method.
Initially, we randomly divide the dataset, allocating 70% of the dataset to the training set and the remaining 30% to the testing set. Then we reduce the label distributions of the training instances to logical labels, and the detailed reduction algorithm can be found in Appendix Subsequently, we apply LE algorithms to transform the logical labels into the label distribution for each training instance, and utilize the recovered label distributions to train an LDL
7

---

Dataset
LE-PROM
CWLD
VIB-ILE
GLLE
KMLE
FCMLE
KL (4)
Painting
(1) 0.567
(5) 0.99 0.9911o.00e
(6) 27. 422.2.4000 490
(2) 0.570
(4) 0.978
(3) 0.589 200000
Emotion6
(1) 0.616+0.013
(5) 0.080=
(6) 0.737 +0.017
(2) O. 00.027
(3) 0.646xa..22
(4) 0.082=0.012
Movie
(1) 0.114
(5) 0.211
(4) 0.193
(2) 0. 00.105
(6) 0.216x6.016
(3) 0.178=0.003
Music
(1) 0.113+0.000
(4) =0.02
(6) 0.225=+0.012*
(2) 0..05
(5) 0.187 +118740.000 #0.04
(3) 0.130=00.009
BU-3DFE
(1) 0.072
(5) 0..00
(4) 0.096
(1) 0.072
(6) 00.11100.00
(3) 0.079=0.002
JAFFR
(1) 0.054
(5) 0. 00.00
0.084=20.000
(2) 00.064
(6) 20.10
(3) 0.071 =0.000
Alpha
(1) 0.006
(5) 0.012
(1) 0.006
(4) 0.010=o.001
(6) 0.024
(1) 0.006g00.00
Cde
(1) 0.007
(5) 00.0200.000
(3) 0.008=20.00
(4) 20.001
(6) 0.019
(1) 0.00720.000
Cold
(1) 0.012
(5) 0.01 20.000
(2) 0.013g0.001
(4) 0.016
(6) 0.027 .02
(2) 0.013g00000
Diau
(1) 0.014
go.00
0..02
(4) 0.022
(6) 0.075
(2) 0.015g0.000
Du
(2) 0.007
0.010
(1) 0. 0.006+0.000
(4) 0.009
(6) 0.01444..00
(2) 0.007
Elu
0.006
0.01220.000
0.007 #0.001
(4) 0.001
(6) 0.019 0011
(1) 0.006
Heal
(1) 0.013 o00
(5) 0.017 200.000
(1) 0.01310.000 o01
(4) 0.014go.001
(6) 0.018 our
(1) 0.013go.000
Spo
(1) 0.025 om
g.030
(3) 0.027 20077000011
(2) 0.026g00000
(6) 0.034 002
(3) 0.02720.001
Spo5
(1) 0.030 o01
0.054go.00z
(4) 0.034400000
(3) 0.033g0.001
(4) 20 oo1
(1) 0.030 g0.000
Spoem
(1) 0.026
0.057
(L) 0.026
(I) 0. 0.006
(5) 0.029 0os
0.027
Cosine (t)
Painting
(1) 0.718.888..0
(4) 0.648
(6) 0.28 20.076
g..1800
(5) 0.633
(3) 0. 0.705
Emotion6
2.707700000
(4) 0. 0.608 2007
0.668 o0.
(2) 20.00
(3) 0.693 013
(6) 0.665g0.004
Movie
(1) 0.926+0.001
(5) 0.802
(3) 0.902+n.000*
(2) 0.925=
(4) 0.901
(6) 0.871
Music
0.915
0.897 1..77000000
0.832
(2) 0.903
(6) 0.877 .411
(3) 0.902=00.00
BU-3DFE
(2) 0.929
(5) O. 20.00
(4) 0.910
(1) 0.900
(6) 0.902 .022
(3) 0. 0.923
JAFFR
0.948
0.9099 20.00
(4) 0.920
(2) 0..40=0.000
(6) 0.816 047
(3) 0.933320.00
Alpha
(1) 0.994 .000
0.990
(1) 0.994
(4) 0.99002..001
(6) 0.979
(1) 0.994g0.001
Cde
0.993
(5) 0.98820.000
0.993
(4) 0.990
(6) 0.981
(1) 0.993=0.000
Cold
(1) 0.988
(5) 0.984g0.000
(1) 0.988
(4) 0.985g0.001
(6) 2..7330.0022
(1) 0.988=0..011
Diau
(1) 0.987
0.902
(4) 0.980
(3) 0..09
(6) 0.939
(2) 0.986=00.000
(1) 0.994
00.00
(1) 00.000
(4) 0.992=0.001
(G) 0.987 20.002
(1) 0.994=0.000
Elu
(1) 0.994
0.988
(1) 0.994
(4) 0..c0
(6) 0.982
(1) 0.994420.000
Heal
(1) 0.988
0.0
(1) 0.988 200000
(4) 0.986go..0r
(6) 0.982 out
(1) 0.98820.000
Spo
(1) 0.977
0. 0.97220.001
(2) 0.975
(2) 0.975
(6) 0.96810
(2) 0.975g0.001
Spo5
(2) 0.973 o01
0.95620.00
(4) 0.970
(3) 20.00
(4) 0.970 o01
(1) 0.97420.0000
Spoem
(1) 0.979
0. 0.9532o.mes
(2) 0.978
(2) 0.978g9.001
(5) 0.976 o0
(4) 0.97720.001
Table Prediction performance measured by KL and Cosine on logical label enhancement.
model (SABFGS (Geng, 2016) is utilized in this paper). Then, evaluate the prediction performance of SABFGS on the testing set. Finally, we repeat the above process ten times and report the mean and standard deviation.
Comparison Algorithms. We choose two state-of offte-of-the-arr LE algorithms: CWLD (Fan et al., 2024) and VIB-ILE (Song et al., 2024), and three baseline LE algorithms: GLLE KMLE, and FCMLE (Xu et al., 2018). The hyperparameter configurations adhere to the respective pa- pers. For CWLDD a selected among 00.,, ..., 1.0). For VIB-ILE, / selected among {10-3, 100 ..... 10 }. 7 selected among {2, 20}. and À is set to 0.1. For GLLE, À is selected among {10-2 - 10-1
102}. For KMLE,, å is selected among {1, 2, 3, 4, 5). For FCMLE, is se- 1.1, 1.2, ...
lected among (1, 1.1,
1.2. 3). For method, À and B are selected among (10-1 10 10 ...., 10 J. is selected among {10-2, 10-1 10--,..., 10a), the probability distribution p(y|z) is modeled by Bernoulli distribution, the parameters of PROM are set 16, b 3.
4.2.2 Results and Discussions
The prediction performance is shown in Table 1. The rankings of each method are provided in the parentheses before the mean value of performance. Additionally, we use pairwise two-tailed 1-test to perform more thorough comparative analysis. We denote o/ o as the cases that our proposed
8

---

107
0.45
10
10
0.1
0.42
0.39
0.60
0.33
0.60
0.2
0.30
10-1-
10-1
0.24
10+4
10
10
0.10
10
10
10
10
10
10
10
10
100
10
10
10
10r
Figure 5: KL-based prediction performance different values of hyperparameters a, 3, and X.
PROM
GLERB
GLEMR
Bradley-Tery
1.00
0.25
Coenne
0.15
0.10
0.06
JAFFE
Movia
Music
BU-3DFE
Apha-Spoom
JAFFE
Movie
Music
BU-3DFF
Alpha-Spoen
Figure Prediction performance measured KL and Cosine on label ranking enhancement.
LE-PROM significantly inferior/superior to the corresponding comparison algorithms. If there is no nor it indicates that there is significant performance difference between LE-PROM and the corresponding algorithm. Overall, our algorithm achieves competitive prediction performance. Our algorithm achieves an average ranking of 1.09 and wins 138 times, ties 19 times, and loses 3 times, out of 160 statistical significance comparisons.
4.2.3 Hyperparameter Analysis
In this subsection, our objective is to demonstrate the effectiveness of each component by varying the values of hyperparameters, i.e. the weights of different components, in LE-PROM. In Figure 5, we show the prediction performance measured by KL on "Movie" dataset with varying values of hyperparameters a, 8, and A. The experimental results show that both (the weight of logical label information) and 33 (the weight of label ranking information) have significant impact on the performance of our model. It can be found that an increase in performance improvement. However, their values cannot be arbitrarily increased, and when these two hyperparameters reach extremely high values, the performance of the model also decreases. This observation implies that features, logical labels, and label rankings all have an impact on prediction performance, and a proper assignment of weights to these loss terms is critical model performance. The last two figures Figure show that À affects the performance to some extent. However, regardless of X, optimal performance can typically be achieved by adjusting the and 13
/ usually leads to
4.3 Experiments on Label Ranking Enhancement
In this subsection, our objective is to experimentally answer the question of whether our proposed PROM is superior current ranking models for the label ranking enhancement task. The exper- imental procedure is analogous to Section 4.2. with the sole distinction lying in the input label data: one the logical label, while the other is the label ranking. The training set with label distri- butions can be directly transformed into the one with label rankings. We choose three comparison algorithms, GLERB (Lu al., 2023b) and GLEMR (Lu al., 2023a) are the two tfate-of-the-ar label- enhenceenhnn oriented ranking models and Bradley-Terry is classic ranking model. Since comparison algorithms cannot be directly applied for tie-allowed label rankings, we preprocess label rankings accommodate the comparison algorithms. Due to the page limit, we the details of label preprocessing Appendix C. From the results in Figure 6, can seen that LE-PROM outperforms the comparison algorithms in most cases.

---

Conclusion and Limitation Discussion
Conclusion. In this paper, we qualitatively and quantitatively study the probabilistic relationship between label distributions and label rankings. We propose assumptions to characterize the proba- bilistic monotonicity and orderliness of label rankings. Subsequently, we derive a pairwise ranking model PROM that theoretically preserves the probabilistic monotonicity and orderliness. Besides, propose a generative label enhancement algorithm based on the PROM model to directly learn an LDL mapping on the training instances with multi-label data. The experimental results on extensive real-world datasets demonstrate the superiority of our proposed method.
Limitation. Despite the contributions of this paper, is imperative to acknowledge that the feasible region of PROM exhibits considerable complexity. This complexity may potentially impede the hyperparameter selection. Besides, if users attempt to adaptively learn the parameters a, b, 4, the feasible region may lead to difficult optimization problem. Hence, future research efforts will be directed towards simplifying the boundaries of the feasible region, with the aim enhancing its compatibility with more established optimization methods.
Acknowledgments
This work partially supported by the National Natural Science Foundation of China (62176123, 62476130, 62576166), the Natural Science Foundation Jiangsu Province (BK20242045), the Innovation and Technology Fund (GHP/079/222,, ITSO33333FF) and the UGC/GRF (No. 1521 15215421).
References
Jean Daunizeau. Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables. arXiv, 2017.
Neamat El Gayar, Friedhelm Schwenker, and Günther Palm. A study of the robustness of KNN classifiers trained using soft labels. In Artificial Neural Networks in Pattern Recognition, pages 67-80, 2006.
Jun Fan, Heng- Ru Zhang, and Fan Min. Learning cluster-wise label distribution for label enhancement. International Journal of Machine Learning and Cybernetics, 2024.
Binbin Gao, Hongyu Zhou, Jianxin Wu, and Xin Geng, Age estimation using expectation of label distribution learning. In International Joint Conference on Artificial Intelligence, pages 712-718, 2018.
Xin Geng. Label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 28 (7)11744-7748, 2016.
Xin Geng, Kate Smith-Miles and Zhi-Hua Zhou. Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25:2401-2412. 2013.
John Guiver and Edward Snelson. Bayesian inference for Plackett-Luce ranking models. In Interna- lional Conference on Machine Learning, page 377-384, 2009.
Peng Hou, Xin Geng, and Min-Ling Zhang. Multi-label manifold learning. In AAAI Conference on Artificial Intelligence, pages 1680-1686, 2016.
David R Hunter. MM algorithms for generalized Bradley-Tery -erry models. The Annals of Statistics, 32 (1):384 406, 2004.
Xiuyi Jia, Xiang Zheng, Weiwei Li, Changqing Zhang, and Zechao Li. Facial emotion distribution learning by exploiting low-rank label correlations locally. IEEE Conference on Computer Vision and Pattern Recognition, pages 9841-9850, 2019.
10

---

Xiuyi Jia, Yunan Lu, and Fangwen Zhang. Label enhancement by maintaining positive and negative label relation. IEEE Transactions on Knowledge and Data Engineering. 35(2):1708-174 2023a.
Xiuyi Jia, Xiaoxia Shen, Weiwei Li, Yunan Lu, and Jihua Zhu. Label distribution learning by maintaining label ranking relation. IEEE Transactions on Knowledge and Data Engineering. 35 (2):1695-1707, 2023b.
Xiuyi Jia, Tian Qin, Yunan Lu, and Weiwei Li, Adaptive weighted aaking--rrietted label distribution learning. IEEE Transactions on Neural Networks and Learning Systems, 35(8): 1308::11002-13166 2024.
Xiufeng Jiang. Zhang Yi, and Jian Cheng Lv. Fuzzy SVM with new fuzzy membership function. Neural Computing & Applications, 15(3):268-270) 2006.
Zhiqiang Kou, Jing Wang, Yuheng Jia, Biao Liu, and Xin Geng, Instance-dependent inaccurate label distribution learning. IEEE Trunsactions on Neural Networks and Learning Systems, 36: 1437, 2023.
Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Inaccurate label distribution learning. IEEE Transactions on Circuits and Systems for Video Technology. 10237-10237710249 2024.
Zhiqiang Kou, Jing Wang. Yuheng Jia, and Xin Geng. Progressive label enhancement. Pattern
Recognition, 160:111172,2001
Harin Lee, Frank Hoeger, Mare Schoenwiesner, Minsu Park, and Nori Jacoby. Cross-cultural mood perception in pop songs and its alignment with mood detection algorithms. In International Society for Music Information Retrieval Conference, pages 366-373, 2021.
Xinyuan Liu, Jihua Zhu, Qinghai Zheng, Zhongyu Li, Ruixin Liu, and Jun Wang. Bidirectional loss function for label enhancement and distribution learning. Knowledge-Baid Based System, 213:106690 2021.
Yunan Lu and Xiuyi Jia. Predicting label distribution from multi-label ranking. In Advances in Neural Information Processing Systems, pages 36931-36943, 2022.
Yunan Lu and Xiuyi Jia. Predicting label distribution from ternary labels. In Advances in Neural Information Processing Systems, pages 70431-70457 2024.
Yunan Lu, Liang He, Fan Min, Weiwei Li, and Xiuyi Jia. Generative label enhancement with Gaussian mixture and partial ranking. In AAAI Conference on Artificial Intelligence, pages 8975-8983, 2023a.
Yunan Lu, Weiwei Li, Huaxiong Li, and Xiuyi Jia. Ranking-preserved generative label enhancement. Machine Learning, 1 112:4693-4721, 2023b.
Yunan Lu, Weiwei Li, Huaxiong Li, and Xiuyi Jia. Predicting label distribution from tie-allowed multi-label ranking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(12):
15364-15379, 2023c.
Yunan Lu, Weiwei Li, Dun Liu, Huaxiong Li, and Xiuyi Jia. Adaptive-grained lahel distribution learning. In AAAI Conference on Artificial Intelligence, pages 19161-19169, 2025.
Jianqiao Luo, Yihan Wang, Yang Ou, Biao He, and Bailin Li. Neighbor-based label distribution learning to model label ambiguity for aerial scene classification. Remote Sensing, 13(4):755, 2021.
Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba. Coding facial expressions with Gabor wavelets. In IEEE International Conference on Automatic Face and Gesture Recognition, pages 200-205, 1998.
Jana Machajdik and Allan Hanbury. Affective image classification using features inspired by psychology and art theory. In ACM International Conference Multimedia, page 83-92, 2010.
Kuan Chuan Peng, Tsuhan Chen, Amir Sadovnik, and Andrew Gallagher. A mixed bag of emotions: Model, predict, and transfer emotion distributions. IEFE Conference on Computer Vision and Pattern Recognition, pages 860-868, 2015.

---

Anning Song, Chao Tan, Jiaxi Zhang, and Zilong Xu. Imbalanced label enhancement based on variational information bottleneck. In International Joint Conference on Neural Networks, pages 1-8, 2024.
Haoyu Tang, Jihua Zhu, Qinghai Zheng, Jun Wang, Shanmin Pang, and Zhongyu Li. Label en- hancement with sample correlations via low-rank representation. In AAAI Conference on Artificial Intelligence, pages 5932-5939, 2020.
Grigorios Tsoumakas and loannis Katakis. Multi-label classification: An overview. International Journal of Data Warehousing and Mining, 3(3):1-13, 2006.
Jing Wang and Xin Geng. Label distribution learning machine. In International Conference on Machine Learning, pages 10749-10759, 2021.
Ke Wang, Ning Xu, Miaogen Ling, and Xin Geng. Fast label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering, 35(2):1502-1514, 2023.
Tao Wen, Weiwei Li, Lei Chen, and Xiuyi Jia. Semi-supervised label enhancement via structured semantic extraction, International Journal of Machine Learning and Cybernetcss, 13:1131-1144, 2021.
Ning Xu, An Tao, and Xin Geng. Label enhancement for label distribution learning. In International Joint Conference on Artificial Intelligence, pages 1632-1643, 2018.
Ning Xu, Jiaqi Lv, and Xin Geng. Partial label learning via label enhancement, In AAAI Conference on Artificial Intelligence, pages 5557-5564, 2019.
Ning Xu, Jun Shu, Yun-Peng Liu, and Xin Geng. Variational label enhancement. In International Conference on Machine Learning, volume 119, pages 10597-10606, 2020.
Lijun Yin, Xiaozhou Wei, Yi Sun, Jun Wang. and M. J. Rosato. A 3D facial expression database for facial behavior research. In International Conference on Automatic Face and Gesture Recognition, pages 211-216, 2006.
Min-Ling Zhang, Qian-Wen Zhang, Jun-Peng Fang, Yukun Li, and Xin Geng. Leveraging implicit relative labeling-importanin information for effective multi-label learning. IEEE Transactions on Knowledge and Data Engineering, 33:2057-20700, 2021.
Qian- Wen Zhang, Yun Zhong, and Min-Ling Zhang. ceature-induced labeling information enrichment for multi-label learning. In AAA/ Conference on Artificial Intelligence, pages 2018.
Dian-Wen
12

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract accurately reflects the three contributions the paper, which are further explained in paragraph 3 of the introduction. That monotonicity and orderliness assumptions are proposed for the probabilities of different ranking relationships, the quality function of PROM is derived, and generative LE algorithm based PROM is proposed.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work discussed in Section 5.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
13

---

Answer: [Yes]
Justification: In Section 3.2.2, the assumptions underlying the theoretical results are ex- plained in Assumptions 3.2, 3.3, and 3.4. The details of the proofs of Theorems 3.5, 3.6, 3.7, and 3.8 are provided in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: In Sections 4.2.1 and 4.2.2, for the comparison algorithms, the hyperparameter configurations and dataset division methods of each comparison method are fully disclosed. For our method, the parameters of the PROM model are specifically set to = 16, b = 8, and 3.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the partieular way they provide for reproducibilitty In the case of closed-source models, it may be that access to the model is limited in
14

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: terms of data access, the datasets used this article strictly follow the data usage agreement of the original creator, and this article does authorize their distribution. In terms of code implementation, PROM is very easy to implement, and functions can be reproduced without relying additional code files. Nevertheless, the code will be accessible after publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ce
htaiic//uuidds//oeeeSbbbbs www.sssiiiciyyooooooooooolliiipp for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //nips. /t/pc//pp//.. hwc//ublic/ggiiiee /tceggguudde//odeuubiiiii
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: In Section 4.2.1, 1, the dataset partitioning, selection method, and label distribu- tion learning model are described detail. Section 4.2.2, the hyperparameter settings are described in detail. In Appendix C.3, the algorithm for reducing the label distribution of training instances to logical labels is described in detail.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
15

---

Justification: Table displays the statistical significance, mean, and standard deviation of the prediction performance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [No]
Justification: The difference in computing resource configuration has no significant effect on the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurip: http:://nnrrppssccpppiieeeeeeeee wc/publicc/thicsuuuee.e.
Answer: [Yes]
Justification: The research conducted in this article complies in all respects with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
16

---

Question:
Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is significant social impact from the work done for this paper.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymeett of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There is such risk associated with the work described in this paper. Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The original papers of five comparative algorithm models and sixteen datasets are fully cited in this article.
Guidelines:
17

---

The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: There are no new assets released this article.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
There is no research involving crowdsourcing or human subjects in this article.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The research of this paper did not involve human subjects.
18

---

Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: In this research methodology, LLM is not significant, unique, non-standard component.
Guidelines:
The answer NA means that the core method development in this research does involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy cectps:/neeurps. / for what should or should not be described.
19


Reviews:
[{"review_id": "QHtzgRDQEu", "reviewer": "Reviewer_M8yu", "summary": "In this paper, the authors propose PROM, a pairwise ranking model with orderliness and monotonicity, to capture the probabilistic relationship between label distributions and label rankings.  \nThe proposed method aims to address the following two limitations of the current label enhancement (LE) process:\n1. Ignorance of the ranking relation in ties.\n2. Ignorance of the probabilistic relationships between label distributions and label rankings.\nSince existing classic ranking models, such as the Bradley–Terry and Plackett–Luce models, cannot explicitly model tie relations, the authors conduct a qualitative analysis of the generation process of label rankings from label distributions and design a parameterized probability mass function for the ranking model.  \nBased on these, they propose a generative label enhancement algorithm called LE-PROM and validate their approach through extensive experiments on real-world datasets.", "questions": "1. In ELBO terms, only the approximation of  \\mathbb{E}_{q(u|x,y,\\xi)}[\\log p(x,y,\\xi|u)]  is given. What about the detail calculation of \\mathcal{D}_{KL}(q||p(u))  and  \\mathcal{D}_{KL}(\\hat{p}||q) ? They are not clearly described.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n1. (Idea and Method) The proposed method adequately explores the relationship between label ranking and label distribution/description, with particular attention to the analysis of tie rankings. Specifically, the discussion on the relationship between label ranking and label distribution is thorough.\n2. (Theory) The theoretical foundation of the proposed method is rigorous. The probabilistic modeling of label ranking is reasonable under the stated assumptions.\n3. (Experiments) The experimental design is sufficient and valid. The authors conduct experiments on 16 real-world datasets, comparing PROM with several state-of-the-art methods. The datasets are preprocessed using min-max normalization, and the experiments are repeated 10 times to ensure statistical reliability. KL divergence and cosine similarity are used as evaluation metrics, which are appropriate for assessing the quality of label distributions.\n\nWeakness:\n\n1. There may be a critical misdescription in the formulation of the proposed method. Specifically, the interpretation appears to be reversed. The key issue lies in why a larger value of $z_i-z_j$ implies a larger probability $p(\\ell_{i}\\prec \\ell_{j}|z_{i},z_{j})$ in Assumption 2.1 and 2.2. This seems counterintuitive. Should it instead be that a larger $z_{i}-z_{j}$ implies a larger probability  $p(\\ell_{i}\\succ \\ell_{j}|z_{i},z_{j})$?\n2. The assumption underlying the generative model may be questionable. In the generative process, the authors assume that $p(x,y,u)=p(x|u)\\cdot p(y|u)\\cdot p(u)$ (Here the variable $\\xi$ is ignored).  This implies that, given $u$ is known, the variable $x,y$ is independent, i.e., $p(x,y|u)=p(x|u)\\cdot p(y|u)$, meaning that the logical label $y$ is independent of $x$ given $u$. This is clearly unreasonable, as many multi-label learning methods have demonstrated that there is a strong dependency between between $x$ and $y$. The questionable nature of this assumption casts doubt on the reliability of the experimental results. Although simplifying assumptions are common, this particular assumption appears overly strong.", "was_revised": true, "final_justification": "All my concerns have been resolved, and I am now inclined to recommend acceptance of the paper.", "rebuttal": "Dear Reviewer M8yu,\n\nThank you for your positive assessment on our work. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Question 1: Only provides the approximation for $\\\\mathbb E _ \\{q(u | x,y,\\\\xi)\\}[\\\\log p(x,y,\\\\xi | u)]$, while the computations of $\\\\mathcal D _ \\{KL\\}(q | p(u))$ and $\\\\mathcal D _ \\{KL\\} (\\\\hat p | q)$ are not clearly explained.***\n\nIn terms of $\\\\mathbb E _ \\{q(u|x,y,\\\\xi)\\}[\\\\log p(x,y,\\\\xi|u)]$, since the joint distribution $p(x, y, \\\\xi \\\\mid u)$ involves the generation process of $x$, $y$, and $\\\\xi$, its form can be quite complex, making the expectation with respect to $u$ intractable in closed form. Therefore, we employ Monte Carlo approximation to estimate the expectation term $\\\\mathbb\\{E\\} _ \\{q(u|x, y,\\\\xi)\\}[\\\\log p(x, y, \\\\xi \\\\mid u)]$.\n\nIn terms of $\\\\mathcal D _ \\{KL\\}(q | p(u))$ and $\\\\mathcal D _ \\{KL\\}(\\\\hat p | q)$, the KL divergence terms involve Gaussian distributions, for which closed-form solutions can be obtained analytically. Therefore, we did not repeat the derivation in the paper for the sake of page limits. Nevertheless, we acknowledge your concern regarding the lack of explicit descriptions. We provide the detailed formulations of the two KL divergence terms as follows.\nFor the KL divergence between the variational posterior $q(u|x, y, \\\\xi) = \\\\mathcal N (\\\\mu _ q, \\\\Sigma _ q)$ and the prior $p(u) = \\\\mathcal N (\\\\mu _ p, \\\\Sigma _ p)$, the closed-form is:\n\n$$\n\\\\mathcal D _ \\{\\\\mathrm KL\\}(q(u|x, y, \\xi) | p(u)) = \\\\frac{1}{2} \\\\sum _ \\{i=1\\}^d \\\\left[ \\\\log \\\\frac\\{(\\\\Sigma _ p)_i\\}\\{(\\\\Sigma _ q)_i\\} + \\\\frac\\{(\\\\Sigma _ q)_i + (\\\\mu _ q)_i^2 - 2(\\\\mu _ q)_i(\\\\mu _ p)_i + (\\\\mu _ p)_i^2\\}\\{(\\\\Sigma _ p)_i\\} - 1 \\right]\n$$\n\nSimilarly, for the KL divergence between the enhanced prior $\\\\hat p (u) = \\\\mathcal N (\\\\mu _ \\{\\\\hat p\\}, \\\\Sigma _ \\{\\\\hat p\\})$ and the posterior $q(u|x, y, \\\\xi)$, the expression is:\n\n$$\n\\\\mathcal D _ \\{ \\\\mathrm KL \\} (\\\\hat p (u) | q(u)) = \\\\frac12 \\\\sum _ \\{i=1\\}^d \\\\left[ \\\\log \\\\frac\\{ (\\\\Sigma_q) _ i \\} \\{(\\\\Sigma_\\{ \\\\hat p \\}) _ i\\} + \\frac\\{ (\\\\Sigma_\\{ \\\\hat p \\}) _ i + (\\\\mu_\\{\\\\hat p \\}) _ i^2 - 2(\\\\mu_\\{\\\\hat p \\}) _ i (\\\\mu _ q)_i + (\\\\mu _ q)_i^2 \\}\\{ (\\\\Sigma _ q)_i \\} - 1 \n\\\\right]\n$$"}, {"review_id": "lvwMXrwxP1", "reviewer": "Reviewer_QeX9", "summary": "The paper introduces the PROM method to handle the ranking issue of the probability between label distribution and tie-allowed label rankings; a label enhance approach designed to capture ranking relation. Based on PROM, LE-PROM is designed for logical label enhancement. The authors validate the effectiveness of LE-PROM through extensive experiments on sixteen benchmark datasets, comparing it with five existing label enhance algorithms.", "questions": "1. Please describe the addressed issue of this paper in a more concise manner in the abstract.\n2. The motivation of this paper is not clear. The problem of label ranking has been solved by many researchers. Authors should indicate the limitations of the existing works and proposed appropriate solutions to overcome them.\n3. In the section of related work, the advantages and disadvantages of the existing works should be analyzed and summarized. In addition, the differences between the proposed method and the existing works should be clearly indicated.\n4. For the proposed method, each step is not well described.\n5. Most of the paper contains many shorthand notations and abstract level information. Each and every term of the equation should be clearly explained. \n6. If you want to show you work in more detail, please upload both code and data to github, and provide the link in the paper.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. PROM introduces an approach to consider label ranking correlation and prove the monotonicity and orderliness, making it a contribution to the field of LDL.\n2. The paper conducts experiment on sixteen benchmark datasets, employing four evaluation metrics to demonstrate the label enhance effectiveness of LE-PROM. Additionally, LE-PROM is compared to five state of-the-art label enhance algorithms, highlighting its competitive performance.\n\nWeaknesses\n1. The motivation of this paper is not clear. The issue of label ranking has been solved by many researchers. Authors should indicate the limitations of the existing works and proposed appropriate solutions to overcome them.\n2. The analysis and discuss of related work are little. In the section of related work, the advantages and disadvantages of the existing works should be analyzed and summarized. In addition, the differences between the proposed method and the existing works should be clearly indicated.\n3. The presentation is not well. The description of the proposed method and the meanings of notations are ambiguous.", "was_revised": true, "final_justification": "Rating: 4: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.\n\nThe authors have answered all my concerned questions. However, there are some minor issues that need to be addressed, such as the incomplete information of reference. The presentation can be further improved. I maintain a borderline accept.", "rebuttal": "Dear Reviewer QeX9,\n\nThank you for your constructive comments on our paper. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Question 1: Describe the addressed issue in a more concise manner in the abstract.***\n\nWe carefully revise the abstract to better highlight the addressed issue of this paper. Specifically, numerous works have demonstrated that the label ranking is significantly beneficial to label enhancement. However, existing works either exhibit research gaps in quantitatively and qualitatively characterizing the probabilistic relationship between the label distribution and their corresponding TALR (tie-allowed label rankings), or cannot account for the cases where labels are equally important to an instance. We will revised the abstract according to your comments in the final version.\n\n***Q2: Authors should indicate the limitations of the existing works and proposed appropriate solutions to overcome them.***\n\nOur work aims to address a fundamental yet understudied problem in label enhancement, i.e., establishing both qualitative and quantitative relationships between label distributions and TALR. Specifically, we investigate: (1) how changes in label distributions affect trends in TALR, and (2) how the magnitudes of label distributions influence probabilistic relationships in TALR. These investigations provide crucial guidance for designing probabilistic generative processes from label distributions to rankings and developing loss functions that measure inconsistencies between label distributions and TALR.\n\n**Current Limitations in Ranking Probability Models.** While existing ranking probability models (e.g., Bradley-Terry and Plackett-Luce) can quantitatively describe relationships between object logits and strict rankings, they suffer from a critical limitation that they only model strict ordering relations ($\\succ$ or $\\prec$), i.e., they cannot handle tie cases where labels describe instances equally. Although [1] extended Bradley-Terry model to TALR by assumes ties occur when $z_i = z_j$, i.e., $p(\\ell_i\\prec\\ell_j)=p(\\ell_i\\succ\\ell_j)=0.5$, it fails to account for real-world uncertainties in tie relation. However, as claimed by [2], in real-world situations the labels with description degrees smaller than a certain distance can be treated as tied labels.\n\n**Current Limitations in Existing Approaches for Tie Modeling.** Several works have attempted to address tie cases. For example, [2] introduces margin concepts for uncertain ties, [3] assigns equal weights to tied labels in loss functions. [4] utilizes virtual labels and bucket orders to model uncertainty. However, these methods share a common shortcoming: They merely assume all distributions satisfying ranking constraints are equally probable, without deeper investigation into the fundamental qualitative and quantitative relationships.\n\n**Our Novel Contributions**\nTherefore, we attempt to investigate the qualitative and quantitative relationships between TALR and label distribution within the task framework of label enhancement. The qualitative study focuses on how changes in label distribution may alter trends in TALR, as well as how the magnitude of label distribution affects probabilistic ordinal relationships in TALR. The quantitative study aims to formalize the findings from qualitative research from a probabilistic perspective.\n\n[1] X. Jia, X. Shen, W. Li, Y. Lu and J. Zhu, \"Label Distribution Learning by Maintaining Label Ranking Relation,\" IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 2, pp. 1695-1707.\n\n[2] Y. Lu, W. Li, H. Li and X. Jia, \"Predicting Label Distribution From Tie-Allowed Multi-Label Ranking,\" IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, pp. 15364-15379.\n\n[3] X. Jia, T. Qin, Y. Lu and W. Li, \"Adaptive Weighted Ranking-Oriented Label Distribution Learning,\" IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 8, pp. 11302-11316.\n\n[4] X. Geng, R. Zheng, J. Lv and Y. Zhang, \"Multilabel Ranking With Inconsistent Rankers,\" IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, pp. 5211-5224.\n\n***Question 3: The advantages and disadvantages of the existing works should be analyzed and summarized. The differences between the proposal and the existing works should be clarified.***\n\n**Advantages and disadvantages of existing works.** Due to the character limits, we briefly discuss the pros and cons of existing works. In terms of LE (label enhancement), existing works can be divided into ranking-based LE and ranking-free LE. Ranking serving as the key information underlying the label distribution is essential for LE. Hence, a portion of LE research recently explores LE from the perspective of label ranking, such as LEPNLR, GLEMR, GLERB, DRAM. The detailed description of these methods can be found in the main text. These methods focus on how to design a ranking-preserved function to regularize the space of label distribution. However, these methods exhibit deficiencies in qualitatively and quantitatively representing the probabilistic relationships between the label distribution and the TALR. Specifically, they either assume that all TALR-preserving label distributions are equiprobable (i.e., they ignore the probabilistic differences among distinct label distributions that satisfy the same TALR, for instance, the probability of $\\ell_i\\prec \\ell_j$ is certainly higher when $z_i−z_j=0.9$ compared to when $z_i−z_j=0.1$, or they fail to account for tie cases. In terms of ranking model, most existing approaches either fail to model scenarios where labels are equally important for instances (e.g., Bradley-Terry and Plackett-Luce), or cannot represent TALR as a probability distribution conditioned on the label distribution (e.g., [Modifying Bradley–Terry and other ranking models to allow ties]).\n\n**Differences between the proposed method and the existing works.** Our paper establishes four key properties between TALR and label distributions: *Probability Monotonicity*, *Probability Orderliness*, *Translational Invariance*, and *Ranking Symmetry*. These properties are rigorously integrated into a ranking probability model, thereby bridging the gap in existing literature regarding both qualitative and quantitative relationships between label distributions and TALR.\n\n***Question 4: For the proposed method, each step is not well described.***\n\nIn summary, we propose a probability distribution named PROM and a generative label enhancement framework called LE-PROM based on PROM. The probability mass function of PROM is given by Equation (3) in the paper, with the value range of each parameter constrained by Equation (4). LE-PROM consists of a probabilistic generative process for observed variables (features, readily available labels, TALR) and a posterior inference process for latent variables (label distribution). We assume that the label distribution, as latent variables, generates all observed variables. Specifically, the feature vector is sampled from a Gaussian distribution conditioned on a nonlinear mapping of the label distribution, the readily available labels are  generated from a task-specific distribution, and TALR is generated from the PROM distribution conditioned on the label distribution. For inferring the latent label distribution, we adopt Auto-Encoding Variational Bayes (AEVB), which approximates the true posterior using a parameterized variational distribution. AEVB transforms latent variable inference into an optimization problem, where the objective combines a reconstruction term for observed variables (which can be estimated via Monte Carlo sampling), and a KL divergence term between the variational posterior and prior distributions (which can be computed analytically). To further obtain a feature-conditioned label distribution predictor, we introduce an additional loss term that minimizes the KL divergence between the predictor’s output and the variational posterior of the label distribution. Finally, the feature-conditioned label distribution predictor can be learned by Equation (8) in the paper.\n\n***Question 5: Most of the paper contains many shorthand notations and abstract level information. Each term should be clearly explained.*** \n\nWe carefully checked all formulas and summarized the meanings of the commonly used shorthand terms and mathematical symbols as follows:\n|Symbol|Meaning|\n|-|-|\n|LDL|label distribution learning|\n|LE|label enhancement|\n|ELBO|evidence lower bound|\n|$\\mathcal{X}^D$|the $D$-dimensional feature space|\n|$\\Delta^M$|the $M$-dimensional label distribution space|\n|$\\ell_m$|the $m$-th label|\n|$v_m$| the description degree of label $\\ell_m$|\n|$\\mathcal{Y}$| the label space containing all possible labels $\\{\\ell_1, \\ell_2,...,\\ell_M\\}$|\n|$N$|the number of instances in the training dataset|\n|$\\boldsymbol x_n$| the feature vector of the $n$-th instance|\n|$\\boldsymbol y_n$  | the vector of the easily available label values of the $n$-th instance|\n|$\\ell_i \\prec \\ell_j$, $\\ell_i\\succ\\ell_j$, $\\ell_i\\simeq\\ell_j$ | label $i$ describes the instance to a higher, lower, and approximately the same degree compared to label $j$|\n|$\\xi_{ij}$  | the ranking relation between labels $\\ell_i$ and $\\ell_j$ |\n|$z_i$  | the description degrees of labels $\\ell_i$|\n|$\\underline{\\phi}(z_i, z_j)$ | probability that $\\ell_i \\succ \\ell_j$|\n|$\\tilde{\\phi}(z_i, z_j)$| probability that $\\ell_i \\simeq \\ell_j$|\n|$\\overline{\\phi}(z_i, z_j)$ | probability that $\\ell_i \\prec \\ell_j$|\n\n***Question 6: Upload both code to github, provide links in the paper.***\n\nWe have now uploaded the code and data to GitHub and will include the link in the revised paper. We appreciate your feedback and hope this improves the reproducibility of our work."}, {"review_id": "3xBmjXajzt", "reviewer": "Reviewer_jHPx", "summary": "The authors theoretically investigate a number of properties of the mapping from the latent label distribution to the label rankings: probabilistic orderliness and probabilistic monotonicity. The authors rigorously define these two properties and derive the conditions under which the proposed distribution-conditioned label ranking model satisfies both properties. In addition, the authors propose a generalized label enhancement model that can be directly utilized as a label distribution predictor based on label ranking. Finally, the authors validate their proposal on several real-world datasets.", "questions": "1. Equation (4) seems difficult to understand; could it be transformed into an easily understandable form?\n\n2. In Equation (2), why should the probability distribution of $\\ell_i=\\ell_j$ be modeled by the square of the distance between $z_i$ and $z_j$, while the other two ranking relationships are modeled by linear functions of the distance.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths：\n1. This paper delves into a meaningful problem of what properties the relationship between the label ranking and the underlying label distribution possesses, and how a label ranking model can be constructed to maintain these properties.\n\n2. This paper theoretically and methodologically studies these two problems, and the theoretical results and the proposed PROM model potentially contribute many new ideas to the field of weakly-supervised machine learning.\n\n3. The authors propose a label distribution generation model based on PROM, which achieves outstanding performance on several real-world datasets.\n\nWeaknesses:\n1. There are some undefined terms in this paper, such as description degree in line 80-84.\n\n2. The writing of this paper can be further improved, such as splitting some complex sentences and embellishing the expressions.", "was_revised": true, "final_justification": "The author addressed my concerns regarding Equations 2 and 4 in the paper. Additionally, considering the feedback from other reviewers, I have decided to keep my score unchanged.", "rebuttal": "Dear Reviewer jHPx,\n\nThank you for your positive assessment on our work. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Question 1: Equation (4) seems difficult to understand; could it be transformed into an easily understandable form?***\n\nThe function $\\\\mathcal F$ is introduced to ensure that the predicted probabilities satisfy the required ordering constraints. The Equation (4) can be decomposed as follows.\n\n**Basic Conditions**\n\n- $b > 0$\n- $t > 0$\n- $a > |b - t|$\n\n**Additional Conditions (Branching Based on the Relationship Between $a$ and $b$)**\n\n- **If $a \\geq 2b$**: \n  No additional conditions are required (automatically satisfied).\n\n- **If $a < 2b$**: \n  Further check the following subconditions:\n\n  - **If $a^2 \\geq 4b^2 - 2b$**: \n    An additional constraint is required:\n    $t< b + a - \\log \\frac{2b - a}{2a}$.\n\n  - **If $a^2 < 4b^2 - 2b$**:\n    Then the following tighter constraint is needed:\n    $t< \\frac{2a \\sqrt{a^2 + 2b} + a^2}{4b} - \\log \\frac{\\sqrt{a^2 + 2b} - a}{2a} + \\frac{1}{2}$.\n\n***Question 2: In Equation (2), why should the probability distribution of $\\\\ell _ i=\\\\ell _ j$ be modeled by the square of the distance between $\\\\boldsymbol z_i$ and $\\\\boldsymbol z_j$, while the other two ranking relationships are modeled by linear functions of the distance?***\n\nConsidering the symmetry of the probability distribution of $\\\\ell _ i=\\\\ell _ j$ with respect to $\\\\boldsymbol z _i - \\\\boldsymbol z _j$, we initially employed the absolute difference between $\\\\boldsymbol z_i$ and $\\\\boldsymbol z_j$ in our formulation, which maintains the same power-law relationship with respect to $\\\\boldsymbol z _i- \\\\boldsymbol z _j$ as observed in the probabilities of both $\\\\ell _ i\\\\prec \\\\ell _ j$ and $\\\\ell _ i\\\\succ\\\\ell _ j$. However, this method gives rise to a very complex analytical form of PROM, which may hinder the model training process. Therefore, we model the probability distribution of $\\\\ell _ i=\\\\ell _ j$ by the square of the distance between $\\\\boldsymbol z_i$ and $\\\\boldsymbol z_j$."}, {"review_id": "VxjXizcmF4", "reviewer": "Reviewer_78KF", "summary": "This paper presents PROM (Pairwise Ranking model with Orderliness and Monotonicity) to quantitatively inscribes the probabilistic relationship between the label distribution and the tie-allowed label rankings. The main contributions include formulating the monotonicity and orderliness assumptions of the ranking probabilities, deriving a theoretically-guaranteed ranking model that preserves the monotonicity and orderliness properties, and finally develop LE-PROM, a PROM-grounded generative label enhancement framework. Finally, extensive experiments on 16 datasets are conducted to show the effectiveness of the proposal.", "questions": "1. This paper sets $a= 16$, $b= 8$, and $t= 3$. Could authors elaborate more in depth on the potential effects that $a$, $b$ and $t$ would have on the model performance?\n2. The feasible domain $\\mathcal{F}$ seems restrictive. Could authors provide a preliminary discussion on how to adaptively learn $(a,b,t)$?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths：\nThis work has several significant strengths. First, the paper qualitatively portrays the probabilistic relationship between label distribution and label ranking by making explicit assumptions (Assumptions 2.1-2.4). These assumptions (i.e., monotonicity, orderliness, translational invariance, and ranking symmetry) are carefully worked out and are highly applicable in real-world scenarios, providing a principled basis for the proposed PROM model. The thorough theoretical analysis (Theorems 2.5-2.8) ensures that the derived PROM model complies with the stated assumptions. The visualization of the theoretical results presented in Figures 2 and 3 also effectively validates the model's adherence to the principles of monotonicity and orderliness. Another major strength of the paper lies in the innovation of the methodology. This paper designs a PROM-grounded generative label enhancement framework, which integrates label ranking, multi-label data, and label distribution learning into a unified probabilistic model. Finally, the effectiveness of the paper's proposal is verified by extensive experiments.\n\nWeaknesses：\nThis work also has two minor limitations. First, the paper empirically sets the parameters of the PROM (e.g., $a=16, b=8, t=3$) without explaining the motivation of this setting and the sensitivity of the model performance w.r.t. $a$, $b$, and $t$. A brief discussion of the performance sensitivity could be added. Besides, the paper omits some experimental details, such as the choice of optimizer, the learning rate, and so on.", "was_revised": true, "final_justification": "The authors have addressed my concerns and thus I maintain my previous rating.", "rebuttal": "Dear Reviewer 78KF,\n\nThank you for your positive assessment on our work. We sincerely appreciate the time and effort that you have dedicated to evaluating our work. We have carefully considered all the comments and have revised the paper accordingly. Below, we provide a point-by-point response to your suggestions.\n\n***Question 1: Elaborate more on an in-depth explanation of how the parameters $a$, $b$, and $t$ influence the model's performance.***\n\nThe parameters $a$, $b$, and $t$ jointly control the sensitivity of the ranking probabilities. Specifically, $a$ controls the slope of $p(\\\\ell _ i\\\\prec\\\\ell _j)$ and $p(\\ell_i\\succ\\ell_j)$ with respect to the difference of label description degree $z _ i - z _ j$. A larger value of $a$ makes the model more sensitive to the difference of label description degree. The parameter $b$ determines the width of the “approximately the same” ($\\\\ell _ i\\\\simeq\\\\ell _ j$) region. A larger value of $b$ encourages the model to assign equivalence relations more frequently. The parameter $t$ adjusts the baseline probability. A lower value of $t$ increases the model’s tendency to prefer \"$\\\\ell _ i \\\\prec \\\\ell _ j$\" or \"$\\\\ell _ i\\\\succ \\\\ell _ j$\" relations when the difference of label description degree is small.\n\n***Question 2: Provide a preliminary discussion on how to adaptively learn $(a, b, t)$.***\n\nAs shown in Equation (4), $\\\\mathcal F$ imposes structural constraints on the parameters $(a, b, t) $ to ensure that the model satisfies the probability orderliness assumption. While these constraints are theoretically necessary, they do not impose a fundamental limitation on the training process. During optimization, one can apply projected gradient descent to update $(a, b, t)$ while ensuring that the parameters remain within $\\\\mathcal F$. Alternatively, a reparameterization approach can be used to express $(a, b, t)$ in terms of unconstrained variables through differentiable transformations, allowing implicit enforcement of the constraints. We will further explore adaptive strategies to better reconcile theoretical feasibility with practical flexibility in training."}]

Author rebuttal:

Meta-review: This paper focuses on label enhancement, which infers label distributions from multi-label data, and highlights the limitations of existing methods in modeling probabilistic relationships between distributions and tie-allowed label rankings. To address this, it proposes PROM, a pairwise ranking model with orderliness and monotonicity assumptions, derives corresponding mass functions with theoretical guarantees, and develops a generative label enhancement algorithm based on PROM. The reviewers generally provided positive feedback, recognizing both the methodological contributions and the theoretical underpinnings. Overall, the paper is recommended for acceptance.
Decision: Accept (spotlight)