Predicting Label Distribution from Multi-label Ranking
Abstract
Label distribution can provide richer information about label polysemy than logical labels in multi-label learning.
There are currently two strategies including LDL (label distribution learning) and LE (label enhancement) to predict label distributions.
LDL requires experts to annotate instances with label distributions and learn a predictive mapping on such a training set.
LE requires experts to annotate instances with logical labels and generates label distributions from them.
However,
LDL requires costly annotation, and the performance of the LE is unstable.
In this paper, we study the problem of predicting label distribution from multi-label ranking which is a compromise w.r.t.
annotation cost but has good guarantees for performance.
On the one hand, we theoretically investigate the relation between multi-label ranking and label distribution.
We deﬁne the notion of EAE (expected approximation error) to quantify the quality of an annotation, give the bounds of
EAE for multi-label ranking, and derive the optimal range of label distribution corresponding to a particular multi-label ranking.
On the other hand, we propose a framework of label distribution predicting from multi-label ranking via conditional
Dirichlet mixtures.
This framework integrates the processes of recovering and learning label distributions end-to-end and allows us to easily encode our knowledge about current tasks by a scoring function.
Finally, we implement extensive experiments to validate our proposal.
Introduction
The label polysemy problem has been a popular research topic in machine learning area, in which an instance is described by multiple labels simultaneously.
MLL (multi-label learning) [23] deals with label polysemy by assigning a vector with logical values to the instance, in which each logical value indicates whether the corresponding label is associated with the instance.
However, MLL only gives which labels can describe the instance, but cannot directly answer a question with more polysemy, i.e., how much does each label describe the instance.
Hence, label distribution [4], a real-valued vector that explicitly gives the description degrees of labels to an instance, is introduced to answer this question.
Obviously, label distribution provides richer information about label polysemy than logical labels, and it has been applied in many practical application scenarios, such as sentiment analysis [14, 35, 39], facial age estimation [3, 6, 30], and so on.
There are two main methods for obtaining label distribution to characterize label polysemy.
The ﬁrst type is LDL (label distribution learning) [4], that is, learning a predictive mapping from a feature vector to a label distribution.
LDL requires experts to directly annotate the instances with label distributions as a training set.
Such methods focus on how to design a well performed LDL algorithm.
Typical works include some algorithms [10, 20, 25, 37, 38] that improve LDL performance by mining label correlations, and some [21, 31] that improve the ability to ﬁt complex label distributions by introducing more ﬂexible models.
The second type is LE (label enhancement) [34], that is, generating label distributions from a vector of logical labels.
LE requires experts to annotate the instances with logical labels ﬁrst, and then recover the label distribution by analyzing the features and labels of the training instances, ﬁnally train a predictive model by these recovered label distributions.
In other words, LE can be regarded as the pre-processing of LDL to obtain the label distribution.
Such methods focus on how to recover the true label distribution accurately from the given logical labels.
For example, most LE algorithms [12, 22, 29, 34, 40] consider recovering more accurate label distributions by mining sample and label correlations.
Since LDL methods are trained directly on instances with true label distributions, they usually produce better performance.
However, annotating instances with label distributions is costly and even impractical in some cases [34].
In contrast, LE methods only require experts to annotate the instance with logical labels, thus reducing the annotation cost.
However, there is no reliable theory to guarantee that the label distribution recovered from logical labels converges to the true label distribution.
In terms of the annotation form, two labels taking the same logical value are indistinguishable in the
MLL case, while the label distribution usually characterizes them as labels with different description degrees.
In terms of solution, logical labels provide a large solution space for the label distribution (e.g., the label distribution component corresponding to a label with a logical value of 1 can take any real value from 0 to 1 ), making the solution unstable and inaccurate.
Therefore, we propose a hypothesis that the label distribution recovered from logical labels is not guaranteed to have the same label ranking as the true label distribution, while accurate label ranking is the key to recovering and predicting an accurate label distribution [13, 26, 27].
Fortunately, multi-label ranking [1, 2, 7] is a good annotation form to address the above problems.
Multi-label ranking 2 requires experts to give which labels are relevant to the instance and further to give the ranking (strict order) of these relevant labels.
Although the annotation cost of multi-label ranking is slightly higer than that of logical labels, it guarantees a ranking consistent with the true label distribution and constrains the approximated label distribution in a narrow solution space.
Hence, in this paper, we investigate the problem of predicting label distribution from multi-label ranking.
On the one hand, we theoretically investigate the relation between multi-label rankings and label description vectors 3 (unnormalized label distributions).
We deﬁne the notion of EAE (expected approximation error) to quantify the quality of annotation w.r.t.
recovering the true label description vector; we derive the EAE for multi-label ranking and logical labels to clarify the advantage of multi-label ranking; we give the bounds of EAE for multi-label ranking, and derive the optimal range of label description vector corresponding to a particular multi-label ranking.
On the other hand, we propose a generic framework named DRAM (label D istribution predicting from multi-label RA nking via conditional Dirichlet M ixtures).
This framework ﬁrst forms a semi-adaptive prior p (cid:63) ( d ) for the label distribution via a scoring function with a predeﬁned functional form and adaptive parameters, then models the predictive distribution p ( d | x ) by conditional Dirichlet mixtures, ﬁnally learns the model parameters by minimizing the cross-entropy of p ( d | x ) relative to p (cid:63) ( d ) .
This framework has two merits: 1) It allows us to ﬂexibly encode our prior knowledge about the tasks by a scoring function, and 2) it integrates the processes of recovering and learning label distributions end-to-end.
Besides, we design a comparison method whose main idea is to transform the dataset with multi-label rankings into the dataset with logical labels such that any existing LE method can be borrowed.
Finally, to validate our proposal, we conduct experiments on reduced LDL datasets and a new real-world dataset that we create directly according to the task.
The experimental results show that our method signiﬁcantly outperforms the comparison methods and also outperforms the LDL methods directly trained on the examples with true label distribution in most cases.
Theoretical analysis 2.1 Preliminary
Let x denote the feature vector of the instance and Y =
{ y i } Mi =1 denote the label set.
The label description vector z is the expert’s internal view of how much does each label describe the instance; z i ∈ [0 , 1] indicates the description degree of y i to x .
If the expert is asked to annotate the instance with logical labels, the internal label description vector will be degenerately expressed as a logical label vector l ∈ { 0 , 1 } M ; the element l i = 1 (or l i = 0 ) in l means that the label y i is the relevant (or irrelevant) to the instance x .
Let m denote the number of relevant labels.
If the expert is asked to annotate the instance with a multi-label ranking, the internal label description vector will be degenerately expressed as a permutation σ (which represents a total strict order) on the relevant labels; σ i indicates that the label y σ i is at the i -th position in ascending order of the description degree; for j ∈ [ M ] \ σ , the label y j is an irrelevant label for x , where [ M ] (cid:44) { 1 , 2 , · · · , M } .
Since both logical labels and multi-label ranking are reduced versions of the internal label description vector, there is consistency between them, which can be described in the following two assumptions:
Assumption 1 If the expert’s internal label description vector z is expressed as a logical label vector l , and δ > 0 is the minimum margin 4 of label description degrees, then we have z ∈ S δ l , where
S δ l = { z | ( ∀ l i = 0 , z i = 0) ∧ ( ∀
l j = 1 , δ ≤ z j ≤ 1) } .
Assumption 2 If the expert’s internal label description vector z is expressed as a multi-label ranking σ , and δ > 0 is the minimum margin of label description degrees, then we have z ∈ S δ σ , where S δ σ = (cid:8) z | ( ∀ i ∈ σ , δ ≤ z i ≤ 1) ∧ ( ∀ i ∈ [ m − [Equation] ] , z σ i ≤ z σ i +1 − δ
[Equation] ∧ [Equation] M ] \ σ
[Equation]
z i = 0) (cid:9) .
Note that the margin δ is an implicit variable in the annotation process, which is not explicitly indicated by the annotation results.
Therefore, the range of internal label description vector (e.g. S δ σ and S δ l ) is also implicit, so the range determined by the implicit interval δ is called the implicit range.
Although δ is implicit, we can predeﬁne an explicit margin ˆ δ ; the range determined by the explicit margin (e.g. S ˆ δ σ and S ˆ δ l ) is called the explicit range.
We are then able to generate a label description vector from the explicit range to approximate the internal label description vector.
2.2 Theoretical results
We ﬁrst deﬁne EAE to measure the quality of a certain annotation form w.r.t.
approximating the internal label description vector.
Deﬁnition 1 Suppose that an instance is annotated with an annotation r ; δ and ˆ δ are implicit and explicit margins, respectively; S δ r and S ˆ δ r are the implicit and explicit ranges, respectively.
Then the expected approximation error of r to the internal label description vector is
[Equation]
Eq.
(1) is essentially derived from the expectation of the squared Euclidean distance between z and ˆ z , i.e., E z , ˆ z (cid:2) (cid:107)
z − ˆ z (cid:107) 22 (cid:3) , which measures the average distance between the estimated label description vector and the internal one given the ranges S δ r and S ˆ δ r .
Since z and ˆ z are independent, p ( z , ˆ z )
= p ( z ) p (ˆ z ) .
Besides, we do not consider additional assumptions to reduce the uncertainty of z and ˆ z ; we assume that z and ˆ z follow the uniform distributions on S δ r and S ˆ δ r , i.e., p ( z ) p (ˆ z ) =
( V δ r V ˆ δ r ) − 1 , and derive the Eq.
Next, we give the EAE of multi-label ranking.
Theorem 1 If an instance is annotated by a multi-label ranking σ , m is the number of relevant labels, δ and ˆ δ are the implicit and explicit margins, respectively, then the EAE of σ is
[Equation]
[Figure]
Theorem 1 can be proved by mathematical induction, and the details can be found in the appendix.
Before giving more corollaries, we need to specify the range of the margins, i.e., δ and ˆ δ .
Lemma
If an instance is annotated by a multi-label ranking σ , then the margins δ and ˆ δ satisfy that 0 ≤ δ ≤ m − 1 and 0 ≤ ˆ δ ≤ m − 1 .
Next we give some interesting corollaries to understand Theorem 1.
Corollary
If an instance is annotated by a multi-label ranking σ , m is the number of relevant labels, the explicit margin ˆ δ (cid:63) minimizing the EAE of σ is ˆ δ (cid:63) =
((2 m + 1) δ + m )( m + 1) − 2 .
It is clear that the optimal explicit margin ˆ δ (cid:63) depends on the implicit margin δ ; hence we cannot obtain an exact optimum for ˆ δ (cid:63) .
Nevertheless, Corollary 1 helps us to narrow down the range of the optimal explicit margin considerably, i.e., m ( m + 1) − 2 ≤ ˆ δ (cid:63) ≤ m − 1 .
Corollary
If an instance is annotated by a multi-label ranking σ , m is the number of relevant labels, 0 ≤ δ ≤ m − 1 , m [Equation] m + 1) − 2 ≤ ˆ δ ≤ m − 1 , then the EAE of σ is bounded by:
[Equation]
Corollary 2 gives m -dependent bounds on the EAE of the multi-label ranking.
See the appendix for details of the proof.
Corollary
If an instance is annotated by a multi-label ranking σ , m is the number of relevant labels, δ and ˆ δ are uniform over (cid:2) 0 , m − 1 (cid:3) and (cid:2) m ( m + 1) − 2 , m − 1 (cid:3) , respectively, then we have:
[Equation]
Corollary 3 can be obtained by a simple integral calculation, as detailed in the appendix.
In Fig. 1(a), we visualize the expected value in Corollary 3 and the upper bound in Corollary 2.
It is obvious that we can obtain a signiﬁcant performance gain by ranking a small number of labels.
Theorem
If an instance is annotated by a logical label vector l , m is the number of relevant labels, δ and ˆ δ are the implicit and explicit margins, respectively, then the EAE of l is
[Equation]
Theorem 2 gives the EAE of the logical label vector, and the proof is detailed in the appendix.
Corollary 4 Suppose that (cid:15) δ, ˆ δ l l and (cid:15)
δ, ˆ δ σ σ are the EAE of the logical label vector l and the EAE of the multi-label ranking σ , respectively, we have the following inequality holds for m ≥ 3 :
[Equation]
Corollary 4 shows the advantage of the multi-label ranking over the logical labels w.r.t.
approximating the true label description vector.
It is obvious that as the number of relevant labels m increases, (cid:15)
δ, ˆ δ l l − (cid:15) δ, ˆ [Equation] σ σ increases at least at the rate of O ( m ) , which is visualized in Fig. 1(b).
Algorithms
In this section, we consider how to train a model on the dataset { ( x n , σ n ) } Nn =1 for predicting label distributions.
We propose a framework DRAM to deal with this problem.
Besides, we also design a comparison method.
3.1 DRAM framework
First, we describe how to enhance multi-label rankings into label distributions.
Then, we formally give the predictive model.
Finally, we derive a generic EM algorithm for our framework that works for any instantiation of the basic models and consider a concrete instantiation.
3.1.1 Recovering label distributions from multi-label rankings
In order to learn a mapping from instance features to label distributions, we consider enhancing multi-label rankings to label distributions.
The process of enhancing the simple label (e.g., multi-label ranking and logical label) into the label distribution can be viewed as selecting label distributions that satisfy a predeﬁned prior assumption from those consistent with simple labels.
For example, some algorithms [9, 34] select those label distributions that satisfy the smoothness assumption [24]; In fact, according to the no-free-lunch axiom, no prior assumption can work for all tasks; hence we do not consider a concrete assumption in our framework.
We provide a semi-adaptive scoring function φ ( d ; θ ) such that any assumption can be easily encoded.
The scoring function allows us to build a prior distribution of label distribution p (cid:63) [Equation] d ) :
[Equation]
where S ˆ δ σ is the explicit range of σ , Z p (cid:63) is a normalization constant, and I ( · ) denotes the indicator function.
The integral term in Eq.
(7) intuitively indicates how many label description vectors can be normalized to d .
According to Corollary 1, we set ˆ δ corresponding to the example ( x n , σ n ) as | σ n | ( | σ n | + 1) − 2 .
The functional form of φ ( d ; θ ) is predeﬁned and the parameters can be learned adaptively.
For example, we can predeﬁne φ ( d ; θ ) as a Gaussian likelihood function, and leave its mean and variance to be learned.
3.1.2 Predictive model: conditional Dirichlet mixtures
Here we need to determine the distribution form of d conditioned on x .
We use Dirichlet distribution to model d | x .
Since φ is any non-negative real-valued function, the prior distribution of d is usually multimodal.
Therefore, we model p ( d | x ) with the mixture of Dirichlet distributions:
[Equation]
where α , β 1 , · · · , β K are learnable parameters; f
( x ; α ) outputs a K -dimensional positive realvalued vector with a sum of 1 , and f k ( x ; α ) is its k -th value; f ( x ; β k ) outputs a M -dimensional positive real-valued vector; Dir ( · ) denotes Dirichlet distribution whose details are in the appendix.
Require: training set { ( x n , σ n ) } Nn =1 , testing instance x (cid:63) , score function φ , number of mixture components K , number of Monte Carlo samples L ; 1: α , θ , β 1 , · · · , β K ← Initialize model parameters; 2: while the likelihood is not converged do 3: for n = 1 , 2 , · · · , N ; i = 1 , 2 , · · · , L do 4: ˆ δ n ← | σ n | ( | σ n | + 1) − 2 ; 5: z ( i ) n ← Generate a sample uniformly from S ˆ δ n σ n ; 6: q ( c ( i ) n ) ← Infer the posterior of latent variable c ( i ) n as in Eq.
(9); 7: α , θ , β 1 , · · · , β K ← Update the model parameters as in Eq.
8: d (cid:63) ← Predict the label distribution for instance x
(cid:63) as in Eq.
(11); 9: return the label distribution d (cid:63) for instance x (cid:63) ;
3.1.3 Learning algorithm
We consider minimizing the cross-entropy of p ( d | x ) relative to p (cid:63) ( d ) , i.e., maximizing E p (cid:63) ( d ) [ln p ( d | x )] .
Since p (cid:63) ( d ) is usually a complex distribution and E p (cid:63) ( d ) [ln p ( d | x )] involves the integration of p (cid:63) ( d ) , it is often intractable.
Therefore, we approximate it by the importance sampling method (whose detailed derivation can be found in the appendix):
[Equation]
where z ( i ) ∼ Uni( z |S ˆ δ σ ) denotes that z ( i ) is sampled uniformly from S ˆ δ σ , and Z ( i ) equals to the sum of all elements in z ( i ) .
Since our model contains discrete latent variables, we use the EM algorithm [8] to train the model.
We introduce the variational distribution q ( c ( i ) n ) and obtain
[Equation]
We then alternate between M-step (maximizing ELBO) and E-step (minimizing the c -posterior error).
E-step : Infer the posterior of latent variables to minimize the c -posterior error:
[Equation]
M-step : Update the model parameters to maximize the ELBO:
[Equation]
Once the model parameters are learned, we can predict the label distribution d for the test instance x (cid:63) according to Eq.
To evaluate DRAM, we take the expectation of the label distribution based on p ( d | x (cid:63) ) as a deterministic output:
[Equation]
The overall learning process is shown in the Algorithm 1.
3.1.4 Instantiation: DRAM with linear learner and noninformative scoring function
Here we consider a concrete instantiation for our framework.
For simplicity, we use the noninformative scoring function, i.e., φ ( d ; θ ) = 1 .
f ( x ; α ) and f ( x ; β k ) are modelled as linear models with the softmax and softplus activation functions, respectively.
To avoid over-ﬁtting, we regularize the parameters { β k } Kk =1 by L 2 norm.
Then Eq.
(10) can be rewritten as:
[Equation]
where α k is the k -th parameter vector in α , β kj is the j -th parameter vector in β k , and λ is a trade-off parameter.
We use L-BFGS [17] to optimize Eq.
We denote this instantiation as DRAM-LN.
3.2 Comparison method: data transformation
Since there is no existing method that can directly predict label distribution from multi-label ranking, we propose a comparison method called DT (dataset transformation).
The main idea of DT is to transform multi-label rankings into logical labels.
Then, we can recover label distributions by any existing LE algorithm and learn a label distribution predictor by any existing LDL algorithm.
Speciﬁcally, the multi-label ranking dataset { ( x n , σ n ) } N n =1 will be transformed into (cid:83) Nn =1 (cid:8)(cid:0) x n , (cid:80) j ∈ σ n, : i v j , | σ n | − 1 (cid:1)(cid:9) | σ n | i =1 , where σ n, : i is the set of the 1 st, 2 nd, · · · , and i -th elements in σ n vector, v j is an | σ n | -dimensional one-hot vector with the j -th element being 1 , and the instance weight | σ n | − 1 is to avoid that the model learning favours instances with more relevant labels.
Related work
Our research is mainly related to LE [34] and LDL [4].
LE is a process of recovering label distributions from logical labels in the dataset.
Most of the existing LE algorithms follow some basic assumptions.
For example, some LE algorithms [9, 15, 22, 34] assume that instances with similar features have similar label distributions; some LE algorithms [12, 32] assume that semantically related labels also have close values in the label distribution; some LE algorithms [33, 16] assume that the label distribution is the low-dimensional representation of feature and logical label vectors.
LDL is the learning process on the instances annotated by label distributions.
The LDL problem can be addressed in two ways.
The ﬁrst is to directly design algorithms that match the prerequisites of the LDL problem.
Some prominent examples include the LDL algorithms [11, 19, 20, 25, 37] that mine label correlation and the LDL algorithms that maintain label ranking [13] or classiﬁcation accuracy [26, 27, 28].
Another way is to extend existing learning algorithms.
For example, LDSVR [5] ﬁts each component of the label distribution by a support vector machine; LDLLogitBoost [31] extends the boosting method by additive weighted regressors; LDLF [21] designs a normalization layer to model the multi-modal distribution by extending differentiable decision trees.
Experiments 5.1 Datasets and evaluation measures
We adopt several widely used label distribution datasets, including Movie [4], Emotion
[18], TwitterLDL, and Flickr-LDL [36].
5 We manually reduce the label distributions in these datasets to multi-label rankings, train the model on these multi-label rankings, and then evaluate the model using the original label distributions in these datasets.
In addition, we create a dataset called NSRD ( N atural S cene with multi-label R ankings and label D istributions).
The instances and label sets in NSRD are the same as in the Natural-Scene [7].
Three experts are requested to annotate the instances with multilabel rankings and label distributions.
Then we can directly train and evaluate the model using the multi-label rankings and label distributions, respectively.
Details of these datasets can be found in the [Table] appendix.
We used the six distance-based measures suggested in the paper [4] and a ranking-based measure suggested in the paper [13] to evaluate the performance of the model [Table] which are Cheb (Chebyshev distance), Clark (Clark distance), Canber (Canberra distance), KL (Kullback-Leibler divergence), Cosine (cosine coefﬁcient), Intersec (intersection similarity), and Rho (Spearman’s rho coefﬁcient).
Due to page limitations, we only show the results on Cheb, Canber, Cosine, and Rho.
Results on other measures are similar.
5.2 Comparison methods
On the one hand, we compare DRAM with the baseline method DT proposed in Section 3.2.
GL (Graph Laplacian LE) [34] and SA (specialized LDL algorithm with BFGS optimizer) [4] are the classical LE and LDL algorithms respectively.
VI (LE with variational inference) [33] and
DM (LDL with label distribution manifold) [25] are the state-of-the-art LE and LDL algorithms, respectively.
We combine them in pairs to construct four comparison methods, i.e., DT+GL+SA,
DT+GL+DM, DT+VI+SA, and DT+VI+DM.
The hyperparameter conﬁguration of GL, VI and DM [Table] Cheb ( ↓ )     7 .
Canber ( ↓ )    10 K (number of mixture components) 0 .0 .0 .
Cosine ( ↑ )    10 K (number of mixture components) 0 .0 .0 .
Rho ( ↑ ) follows their respective literature.
For our method, we set K = 3 and L = 20 , and λ is selected from { 10 − 5 , 5 × 10 − 5 , 10 − 4 , 5 × 10 − 4 , · · · , 10 1 , 5 × 10 1 } by ﬁve-fold cross-validation.
For the above comparison methods, since the label distributions are unavailable during training, the hyperparameter conﬁguration that gives the highest Rho on the validation set will be used.
On the other hand, we directly train DM and SA on the ground-truth label distributions for comparison.
We refer to these two as GT+DM and GT+SA for short, respectively.
For these two comparison methods, the hyperparameter conﬁguration that gives the best Cheb, Canber, Cosine, and Rho on the validation set will be used.
Each method is run for ten times on random dataset partitions (70% for training and 30% for test); the average values and standard derivations are recorded.
5.3 Results and discussions
Table 1 shows the four performance measures of each method on four reduced datasets and NSRD dataset.
Since the NSRD is annotated by three experts, we can obtain three corresponding datasets, denoted by the “-e1”, “-e2” and “-e3” sufﬁxes, respectively; these three datasets have the same feature vectors and different annotation values.
Each experimental result is formatted as “(rank) mean ± std t -test”; “(rank)” denotes the rank of each method among the seven comparison methods; • / ◦ indicates whether DRAM-LN is statistically superior/inferior to the corresponding methods (pairwise two-tailed t -test at 0 .
05 signiﬁcance level)
; if neither • nor ◦ is shown, it means that there is no signiﬁcant difference between the corresponding method and DRAM-LN; “ ↓ ” denotes “the lower the better”, and “ ↑ ” denotes “the higher the better”.
Table 2 shows the average rank of each comparison method on each measure.
Overall, our method achieves signiﬁcant advantages.
On the
Emotion6, Twitter-LDL, and Flickr-LDL datasets, our method signiﬁcantly outperforms almost any comparison methods.
On the NSRD and Movie datasets, our method is only inferior to the GT-based methods on Cheb and Cosine measures.
It is worth noting that DRAM-LN and DT-based methods outperform GT-based methods in many cases, such as the performance on Emotion6, Twitter-LDL, and Flickr-LDL datasets.
We believe this is because some datasets are difﬁcult to annotate; thus, the label distributions given by experts are noisy; then, ﬁtting such label distributions exactly may lead to overﬁtting.
This argument can be further supported by the fact that GT-based methods outperform our method on the NSRD dataset (where the label distributions are carefully annotated and less noisy).
Figure 2 shows how the number of mixture components K affects the performance of our method on NSRD dataset.
To save space, we show the average performance on NSRD-e1, NSRD-e2 and
NSRD-e3 rather than showing them separately.
It is obvious that the mixture model ( K > 1 ) always outperform the single model ( K = 1 ).
In addition, it can be seen that appropriately increasing the Dirichlet components in the mixture can improve the model capacity and thus improve the predictive performance, but too many Dirichlet components may lead to overﬁtting and thus degrade the predictive performance.
Limitations and conclusion
Limitations.
1) EAE is deﬁned for the label description vector and does not directly reﬂect the approximation error to the label distribution; this limitation arises because normalizing the label description vector to a label distribution will lead to an extremely complex closed form of EAE.
We do not believe that this limitation have a signiﬁcant impact on our main results since the approximation error to the label distribution does not exceed that to the label description vector.
For example, if the true and estimated label description vectors are z and t z (where t is a scaler), respectively, they will produce non-zero errors w.r.t.
the label description vector; but z and t z are the same after normalization, i.e., they do not produce errors w.r.t.
the label distribution.
If several labels actually describe the instance to the same degree, then requiring experts to give the strict order of these labels may lead to errors and invalidate Theorem 1; we plan to extend the theoretical results to this case in the future.
Fortunately, DRAM framework can suit this case by a minor modiﬁcation on the explicit range, i.e., allowing the description degree of these labels to be identical in line 5 of Algorithm 1.
Conclusion.
We derive some theorems and corollaries to reveal the relation between multi-label ranking and label distribution, and propose a generic framework, DRAM, for predicting label distribution from multi-label ranking.
DRAM is cost-effective: It is trained on the examples with multi-label rankings and achieves performance comparable to that of LDL methods which require expensive label distribution annotations; DRAM is ﬂexible: It allows users to easily encode their prior knowledge by a scoring function; DRAM is end-to-end: It integrates the processes of recovering and learning label distributions into one learning criterion, rather than performing them separately.
Experimental results show the superiority of our proposal.
Acknowledgments
This work was partially supported by the National Key Research and Development Program of
China under Grant 2019YFB1706900, the National Natural Science Foundation of China (62176123,
U20B2064), and the Fundamental Research Funds for the Central Universities (30920021131).
Checklist
For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[Yes] (c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] All proofs are detailed in the appendix.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] The supplemental material provides a detailed instruction for reproducing the main results.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[No] 4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] (b) Did you mention the license of the assets?
[No] (c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] We include the new dataset we created in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[No] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
[Yes] The data we are using doesn’t contain personally identiﬁable information or offensive content 5.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]