Title: Towards a Pairwise Ranking Model with Orderliness and Monotonicity for Label Enhancement
Venue: NeurIPS 2025
Accepted: yes
Keywords: label polysemy

label distribution learning

label enhancement
Abstract: Label distribution in recent years has been applied in a diverse array of complex decision-making tasks. To address the availability of label distributions, label enhancement has been established as an effective learning paradigm that aims to automatically infer label distributions from readily available multi-label data, e.g., logical labels. Recently, numerous works have demonstrated that the label ranking is significantly beneficial to label enhancement. However, these works still exhibit deficiencies in representing the probabilistic relationships between label distribution and label rankings, or fail to accommodate scenarios where multiple labels are equally important for a given instance. Therefore, we propose PROM, a pairwise ranking model with orderliness and monotonicity, to explain the probabilistic relationship between label distributions and label rankings. Specifically, we propose the monotonicity and orderliness assumptions for the probabilities of different ranking relationships and derive the mass functions for PROM, which are theoretically ensured to preserve the monotonicity and orderliness. Further, we propose a generative label enhancement algorithm based on PROM, which directly learns a label distribution predictor from the readily available multi-label data. Finally, extensive experiments demonstrate the efficacy of our proposed model.

Towards a Pairwise Ranking Model with Orderliness and Monotonicity for Label Enhancement

Abstract

Label distribution in recent years has been applied in a diverse array of complex decision-making tasks. To address the availability of label distributions, label enhancement has been established as an effective learning paradigm that aims to automatically infer label distributions from readily available multi-label data, e.g., logical labels. Recently, numerous works have demonstrated that the label ranking is significantly beneficial to label enhancement. However, these works still exhibit deficiencies in representing the probabilistic relationships between label distribution and label rankings, or fail to accommodate scenarios where multiple labels are equally important for a given instance. Therefore, we propose PROM, a pairwise ranking model with orderliness and monotonicity, to explain the probabilistic relationship between label distributions and label rankings. Specifically, we propose the monotonicity and orderliness assumptions for the probabilities of different ranking relationships and derive the mass functions for PROM, which are theoretically ensured to preserve the monotonicity and orderliness. Further, we propose a generative label enhancement algorithm based on PROM, which directly learns a label distribution predictor from the readily available multi-label data. Finally, extensive experiments demonstrate the efficacy of our proposed model.

1 Introduction

Label polysemy, i.e., the cases where an instance is associated with multiple labels simultaneously, is a common phenomenon in real-world tasks. To preserve label polysemy, multi-label learning (Tsoumakas and Katakis, 2006) employs binary values to indicate the presence or absence of each label for a given instance. However, multi-label learning cannot directly handle a further question with more polysemy: How much is each label associated with an instance? Hence, Geng (2016) introduced LDL (Label Distribution Learning). Unlike multi-label learning, LDL assigns a real-valued vector to each instance, resembling probability distributions, where each element, called label description degree, represents the extent to which the label describes the instance. By providing richer label information, LDL has been applied in various fields, including age estimation (Gao et al., 2018, Geng et al., 2013) and affective analysis (Jia et al., 2019, Machajdik and Hanbury, 2010).

A significant bottleneck hindering the broader application of LDL is the challenge of acquiring ground-truth label distributions, as accurately quantifying these distributions can be costly. To address

this issue, LE (Label Enhancement) has been proposed to automatically infer label distributions from more readily available multi-label data by mining the underlying label polysemy information (Xu et al., 2018, 2020). Initially, most LE methods mainly focused on mining the underlying label polysemy information from instance correlation and label correlation. Recently, some novel findings (Jia et al., 2023b, 2024, Wang and Geng, 2021) that the label ranking is beneficial for improving the generalization of LDL have inspired a portion of LE research to explore the label polysemy information from the perspective of label ranking. For example, Jia et al. (2023a), Lu et al. (2023a,b) proposed to incorporate the ranking relationship between positive and negative labels into the loss function or the generation model of label distributions. Although these works have made headway in designing methods for regularizing LE processes by label rankings, there remain the following two research gaps. 1) Current works ignore the ranking relation of tie, i.e. the cases where multiple labels simultaneously describe the instance to almost the same degree, which is ubiquitous in real-world tasks. 2) Current works neglect the qualitative and quantitative modeling of the probabilistic relationships between label distributions and label rankings, which is essential for the interpretable and generative modeling of label enhancement. Besides, it should be noted that while some classic ranking models, such as Bradley-Terry (Hunter, 2004) and Plackett-Luce (Guiver and Snelson, 2009) models, can be employed as a quantitative model, they also cannot explicitly model the tie relation.

Therefore, we propose PROM (a Pairwise Ranking model with Orderliness and Monotonicity) to qualitatively and quantitatively explain the probabilistic relationship between label distributions and label rankings, which can serve as a generative distribution of label rankings in generative label enhancement or as a loss function for measuring the inconsistency between label rankings and label distributions. Specifically, we first conduct a qualitative analysis of the probabilistic principles governing the generation process of label rankings from label distributions, and formalize these principles as the assumptions on the monotonicity and orderliness of the probabilities of different ranking relationships. Second, we design parameterized probability mass functions for the ranking model, and derive the conditions under which the model adheres to probabilistic monotonicity and orderliness. Third, we propose a generative label enhancement algorithm based on the PROM model, called LE-PROM, which integrates the LE process and the LDL process into a unified framework, and directly learns an LDL mapping on the training instances with multi-label data. Finally, we validate our proposal through extensive experiments on real-world datasets. The experimental results demonstrate the superiority of our proposed method.

2 Related Work

To tackle the challenge of acquiring accurate ground-truth label distributions, LE (Label Enhancement) was proposed to automatically infer label distributions from more readily available multi-label data, such as logical labels (Xu et al., 2018, Kou et al., 2025), ternary labels Lu and Jia (2024), multi-label rankings (Lu and Jia, 2022, Lu et al., 2023c), or inaccurate label distributions Kou et al. (2024, 2023), Lu et al. (2025). Most LE methods mine the label polysemy from instance and label correlations. For example, in order to capture the instance correlation, the prototype-based LE algorithms (El Gayar et al., 2006, Jiang et al., 2006, Wang et al., 2023, Fan et al., 2024) identify the representative points and subsequently estimate the label distribution based on the representative points of each label. The graph-based LE algorithms (Xu et al., 2018, Zhang et al., 2021, Xu et al., 2019, Liu et al., 2021) directly calculate affinities between instances based on the features, which is further utilized to regularize the label distribution matrix. The manifold-based LE algorithms (Hou et al., 2016, Wen et al., 2021, Tang et al., 2020, Zhang et al., 2018) learn coefficients for each instance, by which the feature vector of each instance can be linearly reconstructed from its neighbors, and then maintain the reconstruction within label distributions. In order to capture the label correlation, Luo et al. (2021) utilized the confusion matrix to estimate global label correlation. Besides, several works address the LE task under special data distribution. For example, VIB-ILE Song et al. (2024) addresses imbalanced label information in LE by adopting a variational information bottleneck mechanism and introducing consistency regularization.

Recently, a portion of LE research attempts to explore the label polysemy information from the perspective of label ranking. For example, Jia et al. (2023a) utilized a margin-based loss function to penalize the cases where the description degree of negative labels exceeds that of positive labels; Lu et al. (2023a) designs a conditional distribution of logical labels given label distributions, whose

parameters are regularized by the ranking between negative and positive labels. Lu et al. (2023b) derives a variational approximation for label distribution in generative label enhancement, which is theoretically assured to uphold the ranking relation between negative and positive labels. Besides, Lu and Jia (2022), Lu et al. (2023c) theoretically and methodologically investigated the task of predicting label distributions directly from multi-label rankings. Despite the success of these efforts, there remains a gap in the qualitative and quantitative modeling of the probabilistic relationships between label distributions and the tie-allowed label rankings.

3 Methodology

In this section, we first introduce the notations commonly used in this paper, secondly elaborate our proposed PROM model, and finally illustrate the generative LE framework.

3.1 Notations

We denote the D -dimensional feature space by X D = R D , denote the M -dimensional label distribution space by ∆ M = { v ∈ R M + : ∑ M m =1 v m = 1 } , denote the label space by Y = { ℓ m } M m =1 . We cope with the training datasets that appear as data pairs { ( x n , y n ) } N n =1 , where x n ∈ X D and y n denote the feature vector and the vector of the easily available label values of the n -th instance, respectively. The data of y n can directly yield pairwise ranking relation among labels. We denote the ranking relation between label ℓ i and label ℓ j by ξ ij ∈ { ℓ i ≺ ℓ j , ℓ i ≻ ℓ j , ℓ i ≃ ℓ j } , where ℓ i ≺ ℓ j , ℓ i ≻ ℓ j , and ℓ i ≃ ℓ j denote that the label ℓ i , compared to the label ℓ j , describes the instance to a higher, lower, and approximately the same degree, respectively. The goal of LE is to learn a label distribution predictor f : X D → ∆ M , i.e., a mapping from feature space to label distribution space, by the training dataset { ( x n , y n ) } N n =1 .

3.2 PROM

In this subsection, we first propose assumptions to formalize the monotonicity and orderliness of ranking probability. Then, we derive the parametric form of PROM.

3.2.1 Fundamental Assumptions of PROM

On the one hand, we study how the label description degree affects the probability of the label ranking, focusing on the monotonicity of the probability of the label ranking. Obviously, given any two labels ℓ i , ℓ j of an instance, the label ℓ i is more likely to rank above the label ℓ j (or the label ℓ j is more likely to rank below the label ℓ i alternatively) if the description degree of ℓ i exceeds that of ℓ j by a larger margin. Besides, if the description degrees of two labels are close, then the strict ranking relation between the two labels will be difficult to distinguish, i.e., it is more likely that the two labels are tied. We formalize the above intuition as the probability monotonicity assumption.

Assumption 3.1 (Probability monotonicity) . Given any two labels ℓ i , ℓ j of an instance and their respective description degrees z i , z j , p ( ℓ i ≺ ℓ j | z i = u i , z j = u j ) < p ( ℓ i ≺ ℓ j | z i = v i , z j = v j ) and p ( ℓ i ≻ ℓ j | z i = u i , z j = u j ) > p ( ℓ i ≻ ℓ j | z i = v i , z j = v j ) holds for any u i -u j < v i -v j . p ( ℓ i ≃ ℓ j | z i = u i , z j = u j ) > p ( ℓ i ≃ ℓ j | z i = v i , z j = v j ) holds for any | u i -u j | < | v i -v j | , where u i , u j , v i , v j ∈ [0 , 1] .

On the other hand, we explore how the label description degree affects the orderliness among the probabilities of different rankings. Obviously, given any two labels ℓ i , ℓ j of an instance, these two labels are most likely to be tied if the description degrees of the two labels are sufficiently close; the label ℓ i is most likely to rank above the label ℓ j (or the label ℓ j is most likely to rank below the label ℓ i alternatively) if the label description degree of ℓ i exceeds that of ℓ j by a sufficiently large margin. We formalize the above intuition as the probability orderliness assumption.

Assumption 3.2 (Probability orderliness) . There exists two real thresholds -1 < z -< 0 < z + < 1 , p ( ℓ i ≻ ℓ j | z i , z j ) > p ( ℓ i ≃ ℓ j | z i , z j ) > p ( ℓ i ≺ ℓ j | z i , z j ) holds for any z i -z j < z -; p ( ℓ i ≻ ℓ j | z i , z j ) < p ( ℓ i ≃ ℓ j | z i , z j ) < p ( ℓ i ≺ ℓ j | z i , z j ) holds for any z i -z j > z + ; p ( ℓ i ≃ ℓ j | z i , z j ) > max { p ( ℓ i ≻ ℓ j | z i , z j ) , p ( ℓ i ≺ ℓ j | z i , z j ) } holds for any z -< z i -z j < z + .

The intuition of probability monotonicity and orderliness is visualized in Figure 1. Actually, the above discussion also implicitly adheres to the following two assumptions. The one is that the ranking relation between two labels depends solely on the difference in their description degrees, and is independent of the description degrees themselves. We refer to this assumption as the translational invariance assumption. The other is that the ranking relation between two labels is symmetric, i.e., ℓ i ≺ ℓ j is actually ℓ j ≻ ℓ i . We refer to this assumption as the ranking symmetry assumption. We formalize these two assumptions as follows.

Assumption 3.3 (Translational invariance) . Given any two labels ℓ i , ℓ j of an instance and their respective description degrees z i , z j , p ( ℓ i ≺ ℓ j | z i = u i , z j = u j ) = p ( ℓ i ≺ ℓ j | z i = v i , z j = v j ) and p ( ℓ i ≻ r j | z i = u i , z j = u j ) = p ( ℓ i ≻ ℓ j | z i = v i , z j = v j ) hold for any u i -u j = v i -v j ; p ( ℓ i ≃ ℓ j | z i = u i , z j = u j ) = p ( ℓ i ≃ ℓ j | z i = v i , z j = v j ) holds for any | u i -u j | = | v i -v j | , where u i , u j , v i , v j ∈ [0 , 1] .

Assumption 3.4 (Ranking symmetry) . Given any two labels ℓ i , ℓ j of an instance and their respective description degrees z i , z j , p ( ℓ i ≺ ℓ j | z i , z j ) = p ( ℓ j ≻ ℓ i | z j , z i ) and p ( ℓ i ≃ ℓ j | z i , z j ) = p ( ℓ j ≃ ℓ i | z j , z i ) .

3.2.2 Probability Mass Functions of PROM

Since the ranking relation between two labels can be ℓ i ≺ ℓ j , ℓ i ≻ ℓ j , or ℓ i ≃ ℓ j , we model the ranking relation by a categorical distribution, which can be formalized as:

where ϕ ( z i , z j ) , ϕ ( z i , z j ) , and ˜ ϕ ( z i , z j ) capture the rules of generating the label ranking relation ℓ i ≻ ℓ j , ℓ i ≺ ℓ j , and ℓ i ≃ ℓ j , respectively. Considering that the softmax function is most commonly used to model the probability of discrete events, we preliminarily assume the parametric form of ϕ ( z i , z j ) as follows:

where Z is a normalization factor that ensures ϕ ( z i , z j ) + ˜ ϕ ( z i , z j ) + ϕ ( z i , z j ) = 1 , a ϕ < 0 , a ˜ ϕ < 0 , a ϕ > 0 , and b ϕ , b ˜ ϕ , and b ϕ are arbitrary real numbers.

Figure 2 visualizes PROM model at different values of parameters. It is evident that not any value of parameter can adhere to the probability monotonicity assumption and the probability orderliness assumption. For example, Figure 2(a) violates the probability monotonicity assumption, Figure 2(b) violates the probability orderliness assumption, and Figure 2(c) violates both. Figure 2(d) shows a PROM model that we expect. Therefore, we next propose four theorems to ensure that the probability mass functions defined by Equation (2) adhere to the proposed assumptions.

Theorem 3.5. p ( ξ ij | z i , z j ) defined by Equation (1) and Equation (2) adheres to the translation invariance assumption, i.e. Assumption 3.3.

Theorem 3.6. p ( ξ ij | z i , z j ) defined by Equation (1) and Equation (2) adheres to the ranking symmetry assumption, i.e. Assumption 3.4, iff a ϕ = -a ϕ and b ϕ = b ϕ .

Theorem 3.7. Given a ϕ > 0 , a ϕ < 0 , a ˜ ϕ < 0 , ∆ = z i -z j . ˜ ϕ adheres to the monotonicity assumption of p ( ℓ i ≃ ℓ j | z i , z j ) if a ϕ = -a ϕ exp( b ϕ -b ϕ ) ; ϕ adheres to the monotonicity assumption of p ( ℓ i ≺ ℓ j | z i , z j ) if ( a ϕ (2 a ˜ ϕ ) -1 < -1) ∨ (∆ ⋆ ≤ -1 ∧ h ϕ ( -1) > 0) ∨ (∆ ⋆ > -1 ∧ h ϕ (∆ ⋆ ) > 0) ; ϕ adheres to the monotonicity assumption of p ( ℓ i ≻ ℓ j | z i , z j )

(a)

a

= 1

, b

= 3

, t

= 1

.

(b)

a

= 1

, b

= 3

, t

= 3

.

(c)

a

= 1

, b

= 5

, t

= 3

.

(d)

a

= 16

, b

= 8

, t

= 3

.

1.0

0.8

Function

0.8

0.8

p(e;e;12i2)

0.8

0.6

p(ei<e|2i,2j)

p(e;>e;12i,2)

0.6

Function

p(e;e;丨2i,2j)

0.6

Function

p(e;e;丨2i,2j)

N

0.6

Function

p(e;e;丨zi,2j)

u

0.4

0.4

p(e;<e;|2i2)

0.4

p(e;<e;|2i2j)

0.4

p(e;<e;1zi,2j)

P

P

p(e;>e;12i，2j)

0.2

0.2

0.2

0.2

0.0

0.0

0.0

-1.0

0.5

0.0

0.5

1.0

-1.0

0.5

0.0

0.5

1.0

-1.0

0.5

0.0

0.5

1.0

-1.0

0.5

0.0

0.5

1.0

2i-j

2i-2j

2ij

-j

(a)

b

= 10

, t

= 10

.

(b)

a

= 10

, t

= 10

.

(c)

a

= 25

, b

= 20

.

(d) Length of tie interval.

1.0

a(e;~l;)

1.0

b(e;~e)

1.0

t(~e)

15

15

3

interval

1.5

a

0.8

30

0.8

30

0.8

N

0.6

0.4

90

45

60

75

a (e<e)

15

30

0.6

0.4

45

60

75

90

b (e;<e;)

30

15

r.

0.6

0.4

6

12

15

18

t (e;<e)

3

6

1.0

t

0.2

45

60

0.2

60

45

0.2

9

12

0.5

75

75

15

0.0

90

0.0

90

0.0

18

-1.0

-0.5

0.0

0.5

1.0

-1.0

0.5

0.0

0.5

1.0

-1.0

-0.5

0.0

0.5

1.0

25

50

75

100

i—j

i-j

i—j

α/b/t

Theorem 3.8. p ( ξ ij | z i , z j ) defined by Equation (1) and Equation (2) adheres to the probability orderliness assumption, i.e. Assumption 3.2, iff max { b ϕ -a ϕ , b ϕ + a ϕ } < a ˜ ϕ + b ˜ ϕ < min { b ϕ + a ϕ , b ϕ -a ϕ } and b ˜ ϕ > max { b ϕ , b ϕ } .

The proof of the above theorems can be found in Appendix A. According to the above theorems, let a = a ϕ = -a ϕ , b = -a ˜ ϕ , b ϕ = b ϕ , t = b ˜ ϕ -b ϕ , we finally define the probability mass function of PROM model as follows:

where the parameters are bounded by ( a, b, t ) ∈ F , where the feasible region F is defined by Equation (4):

3.2.3 Marginal Effect of Parameters on PROM

Intuitively, the parameters a and b control the uncertainty of ℓ i ≺ ℓ j (or ℓ i ≻ ℓ j ) and ℓ i ≃ ℓ j , respectively. As shown in Figure 3(a) and Figure 3(b), larger a and b correspond to the steeper density curve of the probabilities of ℓ i ≺ ℓ j (or ℓ i ≻ ℓ j ) and ℓ i ≃ ℓ j , respectively. More results are shown in Appendix B. We do not show p ( ℓ i ≻ ℓ j | z i , z j ) for the sake of readability. The parameter t is similar to the temperature coefficient in the softmax function, whose effect on the PROM distribution is shown in Figure 3(c). Overall, these three hyper-parameters directly affect the length of tie interval of PROM distribution, i.e., the length of the interval of z i -z j with p ( ℓ i ≃ ℓ j ) > max { p ( ℓ i ≺ ℓ j ) , p ( ℓ i ≻ ℓ j ) } , which is shown in Figure 3(d).

(a) Generation Process

(b) Inference Process

y

C

y

C

PROM

softmax(u)

softmax(u)

u

u

3.3 LE-PROM

LE-PROM consists of a probabilistic generation process for observed variables (features, available labels, pairwise rankings) and a posterior inference process for latent variables (label distribution), which will be illustrated in the following two subsections.

3.3.1 Generation Process

Here we illustrate the process of generating the observations, which is shown in Figure 4(a) and formalized as follows.

Generate a sample of label distribution:

(a) Generate a real-valued vector u from a standard multivariate normal distribution, i.e., u ∼ N ( u | 0 M , I M ) , where 0 M is an M -dimensional all-zero vector, I M is an M × M identity matrix.

(b) Transform the real-valued vector u into the label distribution z , i.e., z = softmax ( u ) .

Generate the available label y from a task-specific distribution given z : y | z ∼ ˜ p ( y | z ) 2 .

Generate a sample of feature variables x from a multivariate normal distribution conditioned on the label distribution 3 . Formally, x | z ∼ N ( x | µ x ( z ) , diag ( σ 2 x ( z ))) , where µ x ( · ) and σ x ( · ) are two neural networks, and diag ( σ ) is a diagonal matrix whose ( i, i ) entry is the i -th element of σ .

Generate a sample of pairwise rankings ξ from our designed PROM distribution conditioned on the label distribution z . Formally, ξ | z ∼ ∏ i<j PROM ( ξ ij | z i , z j ) .

According to the above generation process, the joint distribution of the complete-data can be factorized as p ( u , x , y , ξ ) = p ( u ) p ( x | u ) p ( y | u ) p ( ξ | u ) . It should be noted that we omit label distribution z since z can be deterministically obtained from u .

3.3.2 Model Inference

Here we derive the inference method of our proposed generative model. The schematic is shown in Figure 4(b). We employ variational inference to approximate the true posterior, since the exact inference of the posterior of z is intractable due to the nonlinear and non-conjugate relationships among variables. First, we assume that the variational posterior of u can be decomposed as

Then, we find the variational posterior distribution of u that is closest to the true posterior distribution. It has been proven that the Kullback-Leibler (KL) divergence between the variational posterior and the true posterior can be minimized by maximizing the ELBO (evidence lower bound):

In the right-hand side of Equation (6), the first term is also known as the reconstruction term, and the second term is also known as the prior regularization term. The reconstruction term serves to

ensure that the label distribution can accurately reconstruct the observed variables. It is evident that an analytic form of reconstruction term is intractable due to the integral of a complicated likelihood function. Therefore, we turn to SGVB estimator. Since q ( u | x , y , ξ ) is multivariate normal, it can be reparameterized by q ( u | x , y , ξ ) = µ u ( x , y , ξ ) + σ u ( x , y , ξ ) ⊙ ϵ , where ϵ is standard normal noise. Besides, benefiting from (Lu et al., 2023a), we utilize hyperparameters α and β to re-weight the reconstruction quality of x , y , and u to re-balance the magnitudes of different likelihood functions. Finally, the reconstruction term can be estimated by:

where L is the number of Monte Carlo samples, ϵ ( t ) is a sample from the standard normal distribution. The prior regularization term encourages the posterior to approach the prior. Since both the u -posterior and u -prior are multivariate normal, their KL divergence can be computed analytically. Besides, in order to train a feature-conditioned label distribution predictor ˆ p ( u | x ) = N ( u | µ ( x ) , diag ( σ ( x ))) during the model inference phase, we add a prediction loss term to ELBO. Finally, the optimization objective can be formalized as follows:

where λ is a trade-off hyperparameter. As suggested by Daunizeau (2017), since z is the softmax normalization of a Gaussian random vector, the expected label distribution based on ˆ p ( u | x ) can be approximated as follows:

where µ i and σ i denote the i -th elements of µ ( x ) and σ ( x ) , respectively.

4 Experiments

4.1 Datasets and Evaluation Measures

We conduct experiments on 16 real-world datasets. These datasets are collected from diverse realworld tasks, including emotion analysis, movie rating prediction, and bioinformatics. Specifically, 'Painting' (Machajdik and Hanbury, 2010), 'Emotion6' (Peng et al., 2015), 'Music' (Lee et al., 2021), 'BU-3DFE' (Yin et al., 2006), and 'JAFFE' (Lyons et al., 1998) are collected from emotion analysis tasks. 'Movie' (Geng, 2016) is a dataset from movie rating prediction task. 'Alpha', 'Cdc', 'Cold', 'Diau', 'Dtt', 'Elu', 'Heat', 'Spo', 'Spo5', and 'Spoem' (Geng, 2016) are collected from a gene expression analysis task. To accelerate convergence, we use min-max normalization to preprocess the feature data. We evaluate the model performance by two representative LDL measures: KL (Kullback-Leibler divergence) and Cosine (cosine similarity). The KL (or Cosine) metric with smaller values represents the better (or worse) performance. More details of the datasets and the results on more metrics can be found in Appendix C.

4.2 Experiments on Logical Label Enhancement

Here, we aim to experimentally answer the question of whether our proposed LE-PROM is superior to the state-of-the-art logical label enhancement algorithms .

4.2.1 Experimental Configurations

Experimental Method. Initially, we randomly divide the dataset, allocating 70% of the dataset to the training set and the remaining 30% to the testing set. Then we reduce the label distributions of the training instances to logical labels, and the detailed reduction algorithm can be found in Appendix C. Subsequently, we apply LE algorithms to transform the logical labels into the label distribution for each training instance, and utilize the recovered label distributions to train an LDL

model (SABFGS (Geng, 2016) is utilized in this paper). Then, we evaluate the prediction performance of SABFGS on the testing set. Finally, we repeat the above process ten times and report the mean and standard deviation.

Comparison Algorithms. We choose two state-of-the-art LE algorithms: CWLD (Fan et al., 2024) and VIB-ILE (Song et al., 2024), and three baseline LE algorithms: GLLE, KMLE, and FCMLE (Xu et al., 2018). The hyperparameter configurations adhere to the respective papers. For CWLD, α is selected among { 0 . 1 , 0 . 2 , . . . , 1 . 0 } . For VIB-ILE, β is selected among { 10 -3 , 10 -2 , . . . , 10 0 } , γ is selected among { 2 , 20 } , and λ is set to 0.1. For GLLE, λ is selected among { 10 -2 , 10 -1 , . . . , 10 2 } . For KMLE, δ is selected among { 1 , 2 , 3 , 4 , 5 } . For FCMLE, β is selected among { 1 , 1 . 1 , 1 . 2 , . . . , 3 } . For our method, λ and β are selected among { 10 -1 , 10 0 , . . . , 10 5 } , α is selected among { 10 -2 , 10 -1 , . . . , 10 4 } , the probability distribution ˜ p ( y | z ) is modeled by Bernoulli distribution, the parameters of PROM are set as a = 16 , b = 8 , t = 3 .

4.2.2 Results and Discussions

The prediction performance is shown in Table 1. The rankings of each method are provided in the parentheses before the mean value of performance. Additionally, we use a pairwise two-tailed t -test to perform a more thorough comparative analysis. We denote ◦ / · as the cases that our proposed

KL on Movie

KL on Movie

KL on Movie

105

0.45

105

105

0.61

0.42

103

0.39

103

103.

0.60

0.36

0.33

0.60

101.

101

101

0.27

0.30

β

0.59

10-1

0.21

10-1-

0.24

10-1

0.58

10-3

0.15

10-3

0.18

10-3

0.57

10-5.

0.09

10-5

0.12

10-5.

0.56

10-5

10-2

101

104

10-5

10-2

101

104

10-5

10-2

101

104

β

入

入

PROM

GLERB

GLEMR

Bradley-Terry

1.00

0.25

0.20

0.95

sine

0.15

S

0.10

0.90

0.05

0.85

0.00

JAFFE

Movie

Music

BU-3DFE

Alpha-Spoem

JAFFE

Movie

Music

BU-3DFE

Alpha-Spoem

LE-PROM is significantly inferior/superior to the corresponding comparison algorithms. If there is no ◦ nor · , it indicates that there is no significant performance difference between LE-PROM and the corresponding algorithm. Overall, our algorithm achieves competitive prediction performance. Our algorithm achieves an average ranking of 1.09 and wins 138 times, ties 19 times, and loses 3 times, out of 160 statistical significance comparisons.

4.2.3 Hyperparameter Analysis

In this subsection, our objective is to demonstrate the effectiveness of each component by varying the values of hyperparameters, i.e. the weights of different components, in LE-PROM. In Figure 5, we show the prediction performance measured by KL on 'Movie' dataset with varying values of hyperparameters α , β , and λ . The experimental results show that both α (the weight of logical label information) and β (the weight of label ranking information) have significant impact on the performance of our model. It can be found that an increase in α or β usually leads to a performance improvement. However, their values cannot be arbitrarily increased, and when these two hyperparameters reach extremely high values, the performance of the model also decreases. This observation implies that features, logical labels, and label rankings all have an impact on prediction performance, and a proper assignment of weights to these loss terms is critical to model performance. The last two figures in Figure 5 show that λ affects the performance to some extent. However, regardless of λ , optimal performance can typically be achieved by adjusting the α and β .

4.3 Experiments on Label Ranking Enhancement

In this subsection, our objective is to experimentally answer the question of whether our proposed PROM is superior to current ranking models for the label ranking enhancement task . The experimental procedure is analogous to Section 4.2.1, with the sole distinction lying in the input label data: one is the logical label, while the other is the label ranking. The training set with label distributions can be directly transformed into the one with label rankings. We choose three comparison algorithms, GLERB (Lu et al., 2023b) and GLEMR (Lu et al., 2023a) are the two state-of-the-art label-enhancement oriented ranking models and Bradley-Terry is a classic ranking model. Since comparison algorithms cannot be directly applied for tie-allowed label rankings, we preprocess the label rankings to accommodate the comparison algorithms. Due to the page limit, we put the details of label preprocessing in Appendix C. From the results in Figure 6, it can be seen that LE-PROM outperforms the comparison algorithms in most cases.

5 Conclusion and Limitation Discussion

Conclusion. In this paper, we qualitatively and quantitatively study the probabilistic relationship between label distributions and label rankings. We propose assumptions to characterize the probabilistic monotonicity and orderliness of label rankings. Subsequently, we derive a pairwise ranking model PROM that theoretically preserves the probabilistic monotonicity and orderliness. Besides, we propose a generative label enhancement algorithm based on the PROM model to directly learn an LDL mapping on the training instances with multi-label data. The experimental results on extensive real-world datasets demonstrate the superiority of our proposed method.

Limitation. Despite the contributions of this paper, it is imperative to acknowledge that the feasible region of PROM exhibits considerable complexity. This complexity may potentially impede the hyperparameter selection. Besides, if users attempt to adaptively learn the parameters a, b, t , the feasible region may lead to a difficult optimization problem. Hence, future research efforts will be directed towards simplifying the boundaries of the feasible region, with the aim of enhancing its compatibility with more established optimization methods.

6 Acknowledgments

This work was partially supported by the National Natural Science Foundation of China (62176123, 62476130, 62576166), the Natural Science Foundation of Jiangsu Province (BK20242045), the Innovation and Technology Fund (GHP/079/22SZ, ITS/034/23FP), and the UGC/GRF (No. 15211024, 15215421).

References

Jean Daunizeau. Semi-analytical approximations to statistical moments of sigmoid and softmax mappings of normal variables. arXiv , 2017.

Neamat El Gayar, Friedhelm Schwenker, and Günther Palm. A study of the robustness of KNN classifiers trained using soft labels. In Artificial Neural Networks in Pattern Recognition , pages 67-80, 2006.

Jun Fan, Heng-Ru Zhang, and Fan Min. Learning cluster-wise label distribution for label enhancement. International Journal of Machine Learning and Cybernetics , 2024.

Binbin Gao, Hongyu Zhou, Jianxin Wu, and Xin Geng. Age estimation using expectation of label distribution learning. In International Joint Conference on Artificial Intelligence , pages 712-718, 2018.

Xin Geng. Label distribution learning. IEEE Transactions on Knowledge and Data Engineering , 28 (7):1734-1748, 2016.

Xin Geng, Kate Smith-Miles, and Zhi-Hua Zhou. Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence , 35:2401-2412, 2013.

John Guiver and Edward Snelson. Bayesian inference for Plackett-Luce ranking models. In International Conference on Machine Learning , page 377-384, 2009.

Peng Hou, Xin Geng, and Min-Ling Zhang. Multi-label manifold learning. In AAAI Conference on Artificial Intelligence , pages 1680-1686, 2016.

David R Hunter. MM algorithms for generalized Bradley-Terry models. The Annals of Statistics , 32 (1):384-406, 2004.

Xiuyi Jia, Xiang Zheng, Weiwei Li, Changqing Zhang, and Zechao Li. Facial emotion distribution learning by exploiting low-rank label correlations locally. In IEEE Conference on Computer Vision and Pattern Recognition , pages 9841-9850, 2019.

Xiuyi Jia, Yunan Lu, and Fangwen Zhang. Label enhancement by maintaining positive and negative label relation. IEEE Transactions on Knowledge and Data Engineering , 35(2):1708-1720, 2023a.

Xiuyi Jia, Xiaoxia Shen, Weiwei Li, Yunan Lu, and Jihua Zhu. Label distribution learning by maintaining label ranking relation. IEEE Transactions on Knowledge and Data Engineering , 35 (2):1695-1707, 2023b.

Xiuyi Jia, Tian Qin, Yunan Lu, and Weiwei Li. Adaptive weighted ranking-oriented label distribution learning. IEEE Transactions on Neural Networks and Learning Systems , 35(8):11302-11316, 2024.

Xiufeng Jiang, Zhang Yi, and Jian Cheng Lv. Fuzzy SVM with a new fuzzy membership function. Neural Computing & Applications , 15(3):268-276, 2006.

Zhiqiang Kou, Jing Wang, Yuheng Jia, Biao Liu, and Xin Geng. Instance-dependent inaccurate label distribution learning. IEEE Transactions on Neural Networks and Learning Systems , 36: 1425-1437, 2023.

Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Inaccurate label distribution learning. IEEE Transactions on Circuits and Systems for Video Technology , 34(10):10237-10249, 2024.

Zhiqiang Kou, Jing Wang, Yuheng Jia, and Xin Geng. Progressive label enhancement. Pattern Recognition , 160:111172, 2025.

Harin Lee, Frank Hoeger, Marc Schoenwiesner, Minsu Park, and Nori Jacoby. Cross-cultural mood perception in pop songs and its alignment with mood detection algorithms. In International Society for Music Information Retrieval Conference , pages 366-373, 2021.

Xinyuan Liu, Jihua Zhu, Qinghai Zheng, Zhongyu Li, Ruixin Liu, and Jun Wang. Bidirectional loss function for label enhancement and distribution learning. Knowledge-Based System , 213:106690, 2021.

Yunan Lu and Xiuyi Jia. Predicting label distribution from multi-label ranking. In Advances in Neural Information Processing Systems , pages 36931-36943, 2022.

Yunan Lu and Xiuyi Jia. Predicting label distribution from ternary labels. In Advances in Neural Information Processing Systems , pages 70431-70452, 2024.

Yunan Lu, Liang He, Fan Min, Weiwei Li, and Xiuyi Jia. Generative label enhancement with Gaussian mixture and partial ranking. In AAAI Conference on Artificial Intelligence , pages 8975-8983, 2023a.

Yunan Lu, Weiwei Li, Huaxiong Li, and Xiuyi Jia. Ranking-preserved generative label enhancement. Machine Learning , 112:4693-4721, 2023b.

Yunan Lu, Weiwei Li, Huaxiong Li, and Xiuyi Jia. Predicting label distribution from tie-allowed multi-label ranking. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(12): 15364-15379, 2023c.

Yunan Lu, Weiwei Li, Dun Liu, Huaxiong Li, and Xiuyi Jia. Adaptive-grained label distribution learning. In AAAI Conference on Artificial Intelligence , pages 19161-19169, 2025.

Jianqiao Luo, Yihan Wang, Yang Ou, Biao He, and Bailin Li. Neighbor-based label distribution learning to model label ambiguity for aerial scene classification. Remote Sensing , 13(4):755, 2021.

Michael Lyons, Shigeru Akamatsu, Miyuki Kamachi, and Jiro Gyoba. Coding facial expressions with Gabor wavelets. In IEEE International Conference on Automatic Face and Gesture Recognition , pages 200-205, 1998.

Jana Machajdik and Allan Hanbury. Affective image classification using features inspired by psychology and art theory. In ACM International Conference on Multimedia , page 83-92, 2010.

Kuan Chuan Peng, Tsuhan Chen, Amir Sadovnik, and Andrew Gallagher. A mixed bag of emotions: Model, predict, and transfer emotion distributions. In IEEE Conference on Computer Vision and Pattern Recognition , pages 860-868, 2015.

Anning Song, Chao Tan, Jiaxi Zhang, and Zilong Xu. Imbalanced label enhancement based on variational information bottleneck. In International Joint Conference on Neural Networks , pages 1-8, 2024.

Haoyu Tang, Jihua Zhu, Qinghai Zheng, Jun Wang, Shanmin Pang, and Zhongyu Li. Label enhancement with sample correlations via low-rank representation. In AAAI Conference on Artificial Intelligence , pages 5932-5939, 2020.

Grigorios Tsoumakas and Ioannis Katakis. Multi-label classification: An overview. International Journal of Data Warehousing and Mining , 3(3):1-13, 2006.

Jing Wang and Xin Geng. Label distribution learning machine. In International Conference on Machine Learning , pages 10749-10759, 2021.

Ke Wang, Ning Xu, Miaogen Ling, and Xin Geng. Fast label enhancement for label distribution learning. IEEE Transactions on Knowledge and Data Engineering , 35(2):1502-1514, 2023.

Tao Wen, Weiwei Li, Lei Chen, and Xiuyi Jia. Semi-supervised label enhancement via structured semantic extraction. International Journal of Machine Learning and Cybernetics , 13:1131-1144, 2021.

Ning Xu, An Tao, and Xin Geng. Label enhancement for label distribution learning. In International Joint Conference on Artificial Intelligence , pages 1632-1643, 2018.

Ning Xu, Jiaqi Lv, and Xin Geng. Partial label learning via label enhancement. In AAAI Conference on Artificial Intelligence , pages 5557-5564, 2019.

Ning Xu, Jun Shu, Yun-Peng Liu, and Xin Geng. Variational label enhancement. In International Conference on Machine Learning , volume 119, pages 10597-10606, 2020.

Lijun Yin, Xiaozhou Wei, Yi Sun, Jun Wang, and M. J. Rosato. A 3D facial expression database for facial behavior research. In International Conference on Automatic Face and Gesture Recognition , pages 211-216, 2006.

Min-Ling Zhang, Qian-Wen Zhang, Jun-Peng Fang, Yukun Li, and Xin Geng. Leveraging implicit relative labeling-importance information for effective multi-label learning. IEEE Transactions on Knowledge and Data Engineering , 33:2057-2070, 2021.

Qian-Wen Zhang, Yun Zhong, and Min-Ling Zhang. Feature-induced labeling information enrichment for multi-label learning. In AAAI Conference on Artificial Intelligence , pages 4446-4453, 2018.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract accurately reflects the three contributions of the paper, which are further explained in paragraph 3 of the introduction. That is, monotonicity and orderliness assumptions are proposed for the probabilities of different ranking relationships, the quality function of PROM is derived, and a generative LE algorithm based on PROM is proposed.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations of the work are discussed in Section 5.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: In Section 3.2.2, the assumptions underlying the theoretical results are explained in Assumptions 3.1, 3.2, 3.3, and 3.4. The details of the proofs of Theorems 3.5, 3.6, 3.7, and 3.8 are provided in Appendix A.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: In Sections 4.2.1 and 4.2.2, for the comparison algorithms, the hyperparameter configurations and dataset division methods of each comparison method are fully disclosed. For our method, the parameters of the PROM model are specifically set to a = 16 , b = 8 , and t = 3 .

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in

some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [No]

Justification: In terms of data access, the datasets used in this article strictly follow the data usage agreement of the original creator, and this article does not authorize their distribution. In terms of code implementation, PROM is very easy to implement, and functions can be reproduced without relying on additional code files. Nevertheless, the code will be accessible after publication.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: In Section 4.2.1, the dataset partitioning, selection method, and label distribution learning model are described in detail. In Section 4.2.2, the hyperparameter settings are described in detail. In Appendix C.3, the algorithm for reducing the label distribution of training instances to logical labels is described in detail.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: Table 1 displays the statistical significance, mean, and standard deviation of the prediction performance.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [No]

Justification: The difference in computing resource configuration has no significant effect on the experimental results.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in this article complies in all respects with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: There is no significant social impact from the work done for this paper.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: There is no such risk associated with the work described in this paper.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The original papers of five comparative algorithm models and sixteen datasets are fully cited in this article.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: There are no new assets released in this article.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: There is no research involving crowdsourcing or human subjects in this article.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The research of this paper did not involve human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: In this research core methodology, LLM is not a significant, unique, or non-standard component.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents PROM (Pairwise Ranking model with Orderliness and Monotonicity) to quantitatively inscribes the probabilistic relationship between the label distribution and the tie-allowed label rankings. The main contributions include formulating the monotonicity and orderliness assumptions of the ranking probabilities, deriving a theoretically-guaranteed ranking model that preserves the monotonicity and orderliness properties, and finally develop LE-PROM, a PROM-grounded generative label enhancement framework. Finally, extensive experiments on 16 datasets are conducted to show the effectiveness of the proposal.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\nThis work has several significant strengths. First, the paper qualitatively portrays the probabilistic relationship between label distribution and label ranking by making explicit assumptions (Assumptions 2.1-2.4). These assumptions (i.e., monotonicity, orderliness, translational invariance, and ranking symmetry) are carefully worked out and are highly applicable in real-world scenarios, providing a principled basis for the proposed PROM model. The thorough theoretical analysis (Theorems 2.5-2.8) ensures that the derived PROM model complies with the stated assumptions. The visualization of the theoretical results presented in Figures 2 and 3 also effectively validates the model's adherence to the principles of monotonicity and orderliness. Another major strength of the paper lies in the innovation of the methodology. This paper designs a PROM-grounded generative label enhancement framework, which integrates label ranking, multi-label data, and label distribution learning into a unified probabilistic model. Finally, the effectiveness of the paper's proposal is verified by extensive experiments.\n\nWeaknesses：\nThis work also has two minor limitations. First, the paper empirically sets the parameters of the PROM (e.g., $a=16, b=8, t=3$) without explaining the motivation of this setting and the sensitivity of the model performance w.r.t. $a$, $b$, and $t$. A brief discussion of the performance sensitivity could be added. Besides, the paper omits some experimental details, such as the choice of optimizer, the learning rate, and so on.\n\nQUESTIONS:\n1. This paper sets $a= 16$, $b= 8$, and $t= 3$. Could authors elaborate more in depth on the potential effects that $a$, $b$ and $t$ would have on the model performance?\n2. The feasible domain $\\mathcal{F}$ seems restrictive. Could authors provide a preliminary discussion on how to adaptively learn $(a,b,t)$?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors theoretically investigate a number of properties of the mapping from the latent label distribution to the label rankings: probabilistic orderliness and probabilistic monotonicity. The authors rigorously define these two properties and derive the conditions under which the proposed distribution-conditioned label ranking model satisfies both properties. In addition, the authors propose a generalized label enhancement model that can be directly utilized as a label distribution predictor based on label ranking. Finally, the authors validate their proposal on several real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths：\n1. This paper delves into a meaningful problem of what properties the relationship between the label ranking and the underlying label distribution possesses, and how a label ranking model can be constructed to maintain these properties.\n\n2. This paper theoretically and methodologically studies these two problems, and the theoretical results and the proposed PROM model potentially contribute many new ideas to the field of weakly-supervised machine learning.\n\n3. The authors propose a label distribution generation model based on PROM, which achieves outstanding performance on several real-world datasets.\n\nWeaknesses:\n1. There are some undefined terms in this paper, such as description degree in line 80-84.\n\n2. The writing of this paper can be further improved, such as splitting some complex sentences and embellishing the expressions.\n\nQUESTIONS:\n1. Equation (4) seems difficult to understand; could it be transformed into an easily understandable form?\n\n2. In Equation (2), why should the probability distribution of $\\ell_i=\\ell_j$ be modeled by the square of the distance between $z_i$ and $z_j$, while the other two ranking relationships are modeled by linear functions of the distance.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces the PROM method to handle the ranking issue of the probability between label distribution and tie-allowed label rankings; a label enhance approach designed to capture ranking relation. Based on PROM, LE-PROM is designed for logical label enhancement. The authors validate the effectiveness of LE-PROM through extensive experiments on sixteen benchmark datasets, comparing it with five existing label enhance algorithms.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. PROM introduces an approach to consider label ranking correlation and prove the monotonicity and orderliness, making it a contribution to the field of LDL.\n2. The paper conducts experiment on sixteen benchmark datasets, employing four evaluation metrics to demonstrate the label enhance effectiveness of LE-PROM. Additionally, LE-PROM is compared to five state of-the-art label enhance algorithms, highlighting its competitive performance.\n\nWeaknesses\n1. The motivation of this paper is not clear. The issue of label ranking has been solved by many researchers. Authors should indicate the limitations of the existing works and proposed appropriate solutions to overcome them.\n2. The analysis and discuss of related work are little. In the section of related work, the advantages and disadvantages of the existing works should be analyzed and summarized. In addition, the differences between the proposed method and the existing works should be clearly indicated.\n3. The presentation is not well. The description of the proposed method and the meanings of notations are ambiguous.\n\nQUESTIONS:\n1. Please describe the addressed issue of this paper in a more concise manner in the abstract.\n2. The motivation of this paper is not clear. The problem of label ranking has been solved by many researchers. Authors should indicate the limitations of the existing works and proposed appropriate solutions to overcome them.\n3. In the section of related work, the advantages and disadvantages of the existing works should be analyzed and summarized. In addition, the differences between the proposed method and the existing works should be clearly indicated.\n4. For the proposed method, each step is not well described.\n5. Most of the paper contains many shorthand notations and abstract level information. Each and every term of the equation should be clearly explained. \n6. If you want to show you work in more detail, please upload both code and data to github, and provide the link in the paper.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors propose PROM, a pairwise ranking model with orderliness and monotonicity, to capture the probabilistic relationship between label distributions and label rankings.  \nThe proposed method aims to address the following two limitations of the current label enhancement (LE) process:\n1. Ignorance of the ranking relation in ties.\n2. Ignorance of the probabilistic relationships between label distributions and label rankings.\nSince existing classic ranking models, such as the Bradley–Terry and Plackett–Luce models, cannot explicitly model tie relations, the authors conduct a qualitative analysis of the generation process of label rankings from label distributions and design a parameterized probability mass function for the ranking model.  \nBased on these, they propose a generative label enhancement algorithm called LE-PROM and validate their approach through extensive experiments on real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n1. (Idea and Method) The proposed method adequately explores the relationship between label ranking and label distribution/description, with particular attention to the analysis of tie rankings. Specifically, the discussion on the relationship between label ranking and label distribution is thorough.\n2. (Theory) The theoretical foundation of the proposed method is rigorous. The probabilistic modeling of label ranking is reasonable under the stated assumptions.\n3. (Experiments) The experimental design is sufficient and valid. The authors conduct experiments on 16 real-world datasets, comparing PROM with several state-of-the-art methods. The datasets are preprocessed using min-max normalization, and the experiments are repeated 10 times to ensure statistical reliability. KL divergence and cosine similarity are used as evaluation metrics, which are appropriate for assessing the quality of label distributions.\n\nWeakness:\n\n1. There may be a critical misdescription in the formulation of the proposed method. Specifically, the interpretation appears to be reversed. The key issue lies in why a larger value of $z_i-z_j$ implies a larger probability $p(\\ell_{i}\\prec \\ell_{j}|z_{i},z_{j})$ in Assumption 2.1 and 2.2. This seems counterintuitive. Should it instead be that a larger $z_{i}-z_{j}$ implies a larger probability  $p(\\ell_{i}\\succ \\ell_{j}|z_{i},z_{j})$?\n2. The assumption underlying the generative model may be questionable. In the generative process, the authors assume that $p(x,y,u)=p(x|u)\\cdot p(y|u)\\cdot p(u)$ (Here the variable $\\xi$ is ignored).  This implies that, given $u$ is known, the variable $x,y$ is independent, i.e., $p(x,y|u)=p(x|u)\\cdot p(y|u)$, meaning that the logical label $y$ is independent of $x$ given $u$. This is clearly unreasonable, as many multi-label learning methods have demonstrated that there is a strong dependency between between $x$ and $y$. The questionable nature of this assumption casts doubt on the reliability of the experimental results. Although simplifying assumptions are common, this particular assumption appears overly strong.\n\nQUESTIONS:\n1. In ELBO terms, only the approximation of  \\mathbb{E}_{q(u|x,y,\\xi)}[\\log p(x,y,\\xi|u)]  is given. What about the detail calculation of \\mathcal{D}_{KL}(q||p(u))  and  \\mathcal{D}_{KL}(\\hat{p}||q) ? They are not clearly described.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}