Understanding Partial Multi-label Learning via Mutual Information
Year: 2021

Abstract:
To deal with ambiguities in partial multi-label learning (PML), state-of-the-art methods perform disambiguation by identifying ground-truth labels directly. However, there is an essential question:“Can the ground-truth labels be identified precisely?". If yes, “How can the ground-truth labels be found?". This paper provides affirmative answers to these questions. Instead of adopting hand-made heuristic strategy, we propose a novel Mutual Information Label Identification for Partial Multi-Label Learning (MILI-PML), which is derived from a clear probabilistic formulation and could be easily interpreted theoretically from the mutual information perspective, as well as naturally incorporates the feature/label relevancy into consideration. Extensive experiments on synthetic and real-world datasets clearly demonstrate the superiorities of the proposed MILI-PML.

heading: 1 Introduction
text: Partial multi-label learning (PML) [1, 2] is a weakly supervised learning problem, where each instance is associated with a set of candidate labels, but only a part of them are the ground-truth labels while others are false positive labels.
In recent years, many real-world applications are arising due to the growing demand for identifying ground-truth labels from partially labeled data, which are easier and less costly to obtain. For example, in crowdsourcing image annotation, a web image might be annotated online by a potential unreliable annotator with many specific labels, but only some of them are accurate.
PML aims to train a classifier from partially labeled data so as to predict the ground-truth labels for an unseen instance automatically. The main challenge is how to deal with the ambiguities caused by false positive labels in candidate label set. One straightforward way is to simply treat all candidate labels equally as the ground-truth labels, and then solve the PML problem by standard multi-label classification methods [3, 4]. However, these methods can be easily misled by the noisy false-positive labels in the candidate set, and fail to generalize well in testing. To deal with the problem, the state-of-the-art PML research attempts to identify the ground-truth labels directly from the candidate label set, which becomes a popular and effective disambiguation strategy.
However, there are two fundamental issues, “Can the ground-truth labels be identified precisely?". If yes, “How can the ground-truth labels be found?". Moreover, the state-of-the-art PML methods are constructed by hand-designed heuristic modeling under considerations like feature/label correlations with no theoretical interpretation.
In this paper, instead of hand-made heuristic modeling, we propose a novel MILI-PML model, which is derived from a clear probabilistic formulation and can be easily interpreted theoretically by information theory. Besides, MILI-PML naturally incorporates the feature/label relevancy considerations rather than that of heuristic modeling.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
To answer the first question “CAN", we define an -identifiable score function to evaluate whether a PML method is workable for identifying the ground-truth labels precisely. Moreover, we demonstrate that the derived MILI-PML is -identifiable under certain assumptions and conditions theoretically.
To answer the second question “HOW", we design an efficient optimization algorithm called MILI Algorithm to optimize the objective function.
Extensive experiments are conducted on six synthetic datasets as well as four real-world datasets. The experimental results demonstrate that the proposed MILI-PML consistently outperforms the state-of-the-art methods in terms of five widely-used multi-label metrics including ranking loss, hamming loss, one error, coverage, and average precision.

heading: 2 Related Work
text: Partial multi-label learning (PML) [5, 6] is different from multi-label learning [7, 8, 9] or partial label learning [10, 11, 12, 13]. In PML, each instance is associated with a set of candidate labels, which contains multiple ground-truth labels and the others are false positive labels. The state-of-the-art PML studies attempt to identify the ground-truth labels directly from the candidate label set.
[14] propose PARTICLE to extract credible labels with high confidence via propagation matrix and use the identified labels to train multi-label classifiers. [15] propose DRAMA to get the reliable labels with high confidence by employing the feature manifold, and then use the identified labels to train the multi-label classifier.
Moreover, [1] propose PML-lc and PML-fp to optimize the label ranking confidence matrix in training classifiers which considers the label correlations and the feature prototype respectively. [16] develop fPML to optimize the label confidentce matrix by considering feature and label correlations. [17] propose PML-LRS to get label ranking which utilizes the low-rank and sparse decomposition to train classifiers while considering the feature and label interdependencies. [18] develop MUSER to train classifiers by decreasing the feature noise and label redundancy via mapping with orthogonality constraint and graph Laplacian regularization, which considers the feature correlation and label correlation simultaneously.
However, existing methods have two common issues. First, models are hand-made heuristic with no theoretical interpretation. Second, existing partial multi-label learning (PML) methods assume that the ground-truth labels exist in the candidate label set and identify them directly. We then have the question:“Can the ground-truth labels be identified precisely?". If yes, “How can the ground-truth labels be found?".

heading: 3 The Model Derivation
text: In this section, we derive our method, i.e., MILI-PML, from the following probabilistic formulation. Assume S = {(xi, yi) : i = 1, . . . , n} are drawn i.i.d. n times from a probability distribution P . Each observation is a pairwise example (x, y), consisting of a d-dimensional feature vector x = [x1, . . . , xd]T , and a q-dimensional candidate label vector y = [y1, . . . , yq]T , drawn from the underlying random variables X = [X1, . . . , Xd]T and Y = [Y1, . . . , Yq]T .
We adopt a q-dimensional binary vector v : a 1 indicating the label is selected, a 0 indicating it is discarded. Notation yv indicates the vector of selected labels, that is, the full vector y projected onto the dimensions specified by v. Notation yv̄ is the complement, i.e., the vector of remaining candidate labels. The full label vector y can therefore be expressed as y = {yv, yv̄}, which means y is the concatenation of yv and yv̄ . In PML, yv is selected from y.
Assume that Q is a hypothetical predictive model with two parameters: v representing which labels are selected and w representing parameter used for prediction. In order to identify the ground-truth labels from the candidate label set for PML, we can maximize the conditional likelihood of training dataset with respect to parameters {v, w}. The conditional likelihood given all training examples S = {(xi, yi) : i = 1, . . . , n} can be expressed as follows:
L(v, w|S) = n∏
i=1
Q(yiv|xi, w) (1)
For convenience, we use the conditional log-likelihood corresponding to Eq. (1) as follows:
`(v, w|S) = 1 n n∑ i=1 logQ(yiv|xi, w) (2)
By multiplying and dividing classifier Q by the distribution of identified ground-truth labels given features, i.e., P (yv|x), we can re-write the above as follows:
`(v, w|S) = 1 n n∑ i=1 log Q(yiv|xi, w) P (yiv|xi) + 1 n n∑ i=1 logP (yiv|xi) (3)
Similarly, we can expand the second term in Eq. (3) by multiplying and dividing the probability P (y|x), and get the following formulation:
`(v, w|S) = 1 n n∑ i=1 log Q(yiv|xi, w) P (yiv|xi) + 1 n n∑ i=1 log P (yiv|xi) P (yi|xi) + 1 n n∑ i=1 logP (yi|xi) (4)
Furthermore, we use E(X,Y) to denote the expectation of the random variables (X,Y). When n→ +∞, Eq. (4) can be reformated as follows:
`(v, w|S) =E(X,Y) {
log Q(yv|x, w) P (yv|x)
} +E(X,Y) { log P (yv|x) P (y|x) } + E(X,Y) { logP (y|x) } (5)
The first term of Eq. (5) equals to the following KL-divergence: E(X,Y) {
log Q(yv|x, w) P (yv|x)
} = −DKL(P (Yv|X)||Q(Yv|X, w)) (6)
As y = {yv, yv̄}, the second term of Eq. (5) can be derived into the following formulation: E(X,Y) {
log P (yv|x) P (y|x)
} = −E(X,Y) { log
P (y|x) P (yv|x)
} =− ∑ (x,y) P (x, y) log P (yv̄, x|yv) P (x|yv)
=− ∑ (x,y) P (x, y) log P (yv̄, x|yv) P (x|yv)P (yv̄|yv) − ∑
(yv̄,yv)
P (yv̄, yv) logP (yv̄|yv)
= −I(Yv̄,X|Yv) +H(Yv̄|Yv) = −I(Yv̄,X|Yv)
(7)
In Eq. (7), I(Yv̄,X|Yv) is the conditional mutual information and H(Yv̄|Yv) is the conditional entropy. The last equality holds as a result of the conditional entropy H(Yv̄|Yv) = 0. This is because in PML setting, Yv is selected from Y, once Yv is known, then Y is of course known in advance, meanwhile, Yv̄ is complementary to Yv , thus, the uncertainty remaining in Yv̄ is zero, i.e., H(Yv̄|Yv) = 0. The detailed derivation of Eq. (7) can be found in the supplementary materials. Moreover, the third term of Eq. (5) equals to the conditional entropy −H(Y|X), i.e.,
1
n n∑ i=1 logP (yi|xi) = −H(Y|X) (8)
Combining together, we get the derived objective function as follows:
`(v, w|S)=−DKL(P (Yv|X)||Q(Yv|X, w))− I(Yv̄,X|Yv)−H(Y|X) (9)
The first term is a KL-divergence between the true and predicted ground-truth label distributions given features. The value of this term depends on how well the model Q can approximate P . The second term is the conditional mutual information between the remaining candidate labels and the features, given the identified ground-truth labels. The value of this term depends solely on the choice of ground-truth labels, and will decrease as the selected ground-truth label set Yv explains more about features X, until eventually becoming zero when the remaining candidate labels Yv̄ are all false positive labels, which contain no additional information about X . The last term is the conditional entropy of all candidate labels given features. This term quantifies the uncertainty remaining in the candidate labels after we know all features of training data, which is independent of parameters v and w as a constant.

heading: 4 Theoretical Analysis
text: To answer the first question:“Can the ground-truth labels be identified precisely?", we define an -identifiable score function to decide whether a PML method can identify the ground-truth label precisely. Intuitively, the score of an identifiable score function increases when a ground-truth label is added to the identified label set compared to that of a false positive label. Thus, the -identifiable score function can be defined as follows, Definition 1 ( -identifiable Score Function). Let Ŝ be the identified label set; yk be a ground-truth label and yk̄ be a false positive label. Let be a non-negative constant. A score function g : 2
[q] → R is said to be -identifiable if the following property holds: for every yk /∈ Ŝ and yk̄ /∈ Ŝ, we have g(Ŝ ∪ {yk})− g(Ŝ ∪ {yk̄}) ≥ .
We will use this definition to derive the following theorem, which demonstrates that our objective function is an -identifiable score function and is capable of identifying the ground-truth label from the candidate label set. Theorem 1. Let yk and yk̄ denote any unidentified ground-truth label and any false positive label respectively. Let Yk and Yk̄ represent the random variables of yk and yk̄ respectively, and that yk, yk̄ ∈ v̄ and yk, yk̄ /∈ v. Then, for any identified ground-truth label set v, our objective function `(v, w|S) is an -identifiable score function on condition that,
I(Yk,X)− I(Yk,Yv) + I(Yk,Yv|X)−DKL1 ≥ I(Yk̄,X)− I(Yk̄,Yv) + I(Yk̄,Yv|X)−DKL2 where DKL1 = DKL ( P (Yv ∪ Yk|X)||Q(Yv ∪ Yk|X, w) ) and DKL2 = DKL ( P (Yv ∪ Yk̄|X)||Q(Yv ∪ Yk̄|X, w)
) (10)
Proof. g(Ŝ ∪ {yk})− g(Ŝ ∪ {yk̄}) = { − I ( Yv̄\Yk,X ∣∣Yv ∪ Yk)+ I(Yv̄\Yk̄,X∣∣Yv ∪ Yk̄)}+{
−DKL ( P (Yv ∪ Yk|X)||Q(Yv ∪ Yk|X, w) ) +DKL ( P (Yv ∪ Yk̄|X)||Q(Yv ∪ Yk̄|X, w) )} (11)
Firstly, we derive the first part of Eq. (11), i.e., −I ( Yv̄\Yk,X ∣∣Yv ∪Yk)+ I(Yv̄\Yk̄,X∣∣Yv ∪Yk̄).
Due to the chain rule that I(Y,X) = I(Yv,X) + I(Yv̄,X|Yv), we have −I ( Yv̄\Yk,X ∣∣Yv ∪ Yk) = −{I(Y,X)− I(Yv ∪ Yk,X)} = I(Yv ∪ Yk,X)− I(Y,X)
= I(Yv,X) + I(Yk,X|Yv)− I(Y,X) = I(Yv,X) + I(Yk,X)− I(Yk,Yv) + I(Yk,Yv|X)− I(Y,X)
(12)
Similarly, we can get, − I ( Yv̄\Yk̄,X ∣∣Yv ∪ Yk̄) = I(Yv,X) + I(Yk̄,X)− I(Yk̄,Yv) + I(Yk̄,Yv|X)− I(Y,X) (13)
Therefore, we have the following expression, The first part of Eq. (11) = −I ( Yv̄\Yk,X ∣∣Yv ∪ Yk)+ I(Yv̄\Yk̄,X∣∣Yv ∪ Yk̄)
= { I(Yk,X)− I(Yk,Yv) + I(Yk,Yv|X) } − { I(Yk̄,X)− I(Yk̄,Yv) + I(Yk̄,Yv|X) } (14) where I(Yk,X) is the mutual information between a ground-truth label yk and features, which means the correlation between the two items; I(Yk,Yv) is the mutual information between a ground-truth label yk and the existing identified ground-truth label set, which means the redundancy between the two items; I(Yk,Yv|X) is the feature-conditional redundancy, which means the good information still remaining between ground-truth label yk and the existing identified label set v after features are known.
Thus, the interpretation of the first part of Eq. (14), i.e., I(Yk,X)− I(Yk,Yv) + I(Yk,Yv|X), is the overall correlation of a ground-truth label yk and features x, which equals the correlation between a ground-truth label yk and features x minus the overall redundancy of label yk with existing identified label set, based on which we can evaluate the effectiveness to include the label yk into the ground-truth label set.
Similarly, the second part of Eq. (14) stands for the overall correlation between a false-positive label yk̄ and features x, which equals the correlation between a false-positive label yk̄ and features x minus the overall redundancy of label yk̄ with existing identified label set.
It is naturally hold that the first part should be larger than the sencond part as the overall correlation between a ground-truth label yk and features ought to be stronger than that of a false-positive label yk̄ and features.
After that, we illustrate the second part of Eq. (11), i.e.,−DKL ( P (Yv∪Yk|X)||Q(Yv∪Yk|X, w) ) +
DKL ( P (Yv ∪ Yk̄|X)||Q(Yv ∪ Yk̄|X, w) ) . The interpretaion of DKL ( P (Yv ∪ Yk|X)||Q(Yv ∪
Yk|X, w) )
is how well can the predictive model Q approximate the true probability distribution P when a ground-truth label yk is added to the existing identified label set v; that is, the information loss when using Q to approximate P . The more precise the approximation is, the smaller is the KL-divergence value.
Similarly,DKL ( P (Yv∪Yk̄|X)||Q(Yv∪Yk̄|X, w) ) can be interpreted as how well can the predictive
model Q approximate the true probability distribution P when a a false-positive label yk̄ is added to the existing identified label set v. The more imprecise the approximation is, the bigger is the KL-divergence value.
It is naturally hold that the KL-divergence value should be smaller when the ground-truth label yk is added to the existing identified label set v than that of the false-positive label yk̄, that is, DKL ( P (Yv ∪ Yk̄|X)||Q(Yv ∪ Yk̄|X, w) ) ≥ DKL ( P (Yv ∪ Yk|X)||Q(Yv ∪ Yk|X, w) ) .
Therefore, combined with the derivation of the first part of Eq. (11), we can get the condition for our objective function `(v, w|S) to be an -identifiable score function as follows, I(Yk,X)− I(Yk,Yv) + I(Yk,Yv|X)−DKL1 ≥ I(Yk̄,X)− I(Yk̄,Yv) + I(Yk̄,Yv|X)−DKL2
where DKL1 = DKL ( P (Yv ∪ Yk|X)||Q(Yv ∪ Yk|X, w) ) and DKL2 = DKL ( P (Yv ∪
Yk̄|X)||Q(Yv ∪ Yk̄|X, w) ) , which concludes the proof.
Remark. The condition of Eq. (10) requires that the overall correlation between a ground-truth label yk and features should be stronger than the overall correlation between a false-positive label yk̄ and features. From the above theoretical analysis, we can see that the ground-truth labels can be identified correctly by the proposed MILI-PML under given conditions and assumptions in Theorem 1.

heading: 5 Optimization
text: To answer the second question:“How can the ground-truth labels be found?", we design an efficient algorithm to optimize the proposed MILI-PML.
The goal of partial multi-label learning (PML) is to maximize `(v, w|S) of Eq. (9), which is equivalent to minimizing −`(v, w|S). Consequently, we can rewrite the objective function of MILI-PML as follows:
−`(v, w|S) =DKL(P (Yv|X)||Q(Yv|X, w)) + I(Yv̄,X|Yv) +H(Y|X) (15)
As the last term H(Y|X) is a constant, we only need to consider optimizing the first term and the second term together in Eq. (15). Thus, minimizing −`(v, w|S) is equivalent to minimizing DKL(P (Yv|X)||Q(Yv|X, w)) + I(Yv̄,X|Yv), that is,
arg min v,w −`(v, w|S) = arg min v,w DKL(P (Yv|X)||Q(Yv|X, w)) + I(Yv̄,X|Yv) (16)
Algorithm 1 Alternating Optimization (MILI Algorithm) Goal: Solve the Optimization Problem of Eq. (16) : identifying the ground-truth label set v as well as training a predictive model parameter w; Input: training data S = {(xi, yi) : i = 1, . . . , n}; Output: the predictive model parameter w.
1: Initialize the ground-truth label set vi for each instance by randomly choosing from the corresponding candidate label set v̄i; t = 0; F 0(v, w) = 0; 2: repeat 3: Fix v, update w in Eq. (17) according to Gradient Descent Algorithm. 4: Fix w, update v in Eq. (19) according to Algorithm 2; 5: t = t+1; 6: Calculate F t+1(v, w); 7: until |F
t+1(v,w)−F t(v,w)| |F t(v,w)| < δ;
8: Output the optimal solution vi for each instance xi, i ∈ {1, . . . , n} and the predictive model parameter w.
We design an alternating optimization algorithm called Mutual Information Label Identification Algorithm (i.e., MILI Algorithm) to optimize the two parameters v (the ground-truth label set parameter) and w (the predictive model parameter) in Eq. (16) by fixing one and updating the other. Complete procedures of MILI Algorithm are summarized in Algorithm 1.
For convenience, we denote the objective function in Eq. (16) as F (v, w) = DKL(P (Yv|X)||Q(Yv|X, w)) + I(Yv̄,X|Yv), which is used to set the stopping creteria in Algorithm 1.
Specifically, when v is fixed to update w, Eq. (16) is equivalent to optimize the following problem as the second term I(Yv̄,X|Yv) becomes a constant.
arg min w E(X,Y) logP (yv|x)− E(X,Y) logQ(yv|x, w) = arg minw −E(X,Y) logQ(yv|x, w) (17)
Moreover, we can use the sample estimation to approximate E(X,Y) logQ(yv|x, w) based on the Law of Large Numbers as follows:
E(X,Y) logQ(yv|x, w) ≈ 1
n n∑ i=1 logQ(yiv|xi, w) (18)
To this end, we can employ the gradient descent algorithm to optimize Eq. (17).
Next, we fix w to update v. The first term of Eq. (16) can be decomposed to be E(X,Y) logP (yv|x)− E(X,Y) logQ(yv|x). Meanwhile, due to the chain rule that I(Y,X) = I(Yv,X) + I(Yv̄,X|Yv), minimizing I(Yv̄,X|Yv) is equivalent to maximizing I(Yv,X). Therefore, Eq. (16) can be reformulated as follows:
arg max v −E(X,Y) logP (yv|x) + E(X,Y) logQ(yv|x) + I(Yv,X) (19)
Moreover, we can identify ground-truth labels yk from v̄ one by one and add to v. Thus, Eq. (19) is equivalent to the following formulation:
arg max yk∈v̄ −E(X,Y) logP (yk|x, yv) + E(X,Y) logQ(yk|x, yv) + I(Yk,X|Yv) (20)
In practice, we can use the empirical distribution and sample estimation to approximate the three terms in Eq. (20) based on the Law of Large Numbers. To this end, we can employ greedy strategy to optimize Eq. (19). Procedures of this implementation can be found in Algorithm 2.
Here, we explain some main steps of Algorithm 1. Step 2–7 summarizes the core iteration procedures, among which Step 3 and Step 4 are the alternating strategy to optimize the two parameters w and v, respectively. At Step 6, the objective function value F (v, w) is calculated. Finally, the stopping condition is set to δ-optimal and we choose δ to be 10−5 in practice (Step 7).
Algorithm 2 Fix w, update v. Goal: Solve the Optimization Problem of Eq. (19). Input: training data S = {(xi, yi) : i = 1, . . . , n}; Output: the ground-truth label set vi for each instance i ∈ {1, . . . , n}.
1: Initialize t = 0; vi0 = ∅ and let v̄i0 be the initial candidate label set for each instance xi, i ∈ {1, . . . , n}; 2: while I(Yk,X|Yvit) > ε do 3: Identifying the ground-truth label yik by Eq. (20); 4: Update the ground-truth label set by vit+1 = v i t ∪ yik; 5: Update the remaining candidate label set by v̄it+1 = v̄ i t\yik; 6: t = t+ 1; 7: end while 8: Output vi by vi = vi
t̂ .

heading: 6 Experiments
text: In this section, we conduct experiments to evaluate the classification performance of the proposed MILI-PML and compare it with six state-of-the-art PML methods.

heading: 6.1 Datasets
text: Experiments are conducted on six synthetic PML datasets1 and four real-world PML datasets (i.e. YeastBP [16], Music-emotion [19], Music-style [19], MIRFlickr [19, 14] of different scales, the characteristics of which are summarized in the supplementary materials. For real-world PML datasets, candidate labels are collected from web users which are further examined by human labelers to specify the ground-truth labels. For synthetic datasets, given the configuration strategy over multilabel datasets in [1, 14], we construct the candidate label set by randomly choosing irrelevant labels together with the ground-truth labels (GLs) for each multi-label instance. Specifically, different settings are considered by varying the average number of candidate labels (#CLs (avg.)) for each multi-label dataset. Accordingly, in this paper, we generate thirty synthetic PML datasets. For brevity, we report the detailed results of two configurations for each dataset, i.e. avg. #CLs (avg.) being 7 and 11 for Enron, Corel5k, Eurlex-sm; 9 and 13 for Eurlex-ed and Mediamill; 45 and 65 for CAL500.

heading: 6.2 Baselines
text: We compare the proposed MILI-PML method with the six state-of-the-art PML approaches. 1http://mulan.sourceforge.net/datasets-mlc.html
For all PML baselines, we set the trade-off parameters as suggested in the original papers. Details can be found in the supplementary materials.
For MILI-PML, we employ the simple Binary Relevance [20, 21] as the predictive classifier Q in the MILI-PML model. In addition, LIBLINEAR [22] with L2-regularized square hinge loss is also employed to train the binary classifiers in Binary Relevance (BR).
Evaluation metrics: We employ five widely-used multi-label metrics including ranking loss, hamming loss, one error, coverage, and average precision to evaluate the performance of all methods. More details about these evaluation metrics can be found in [23, 24, 25]. Besides, on each dataset, five-fold cross validation is performed where the mean metric value as well as standard deviation are recorded for each comparing method.

heading: 6.3 Experimental Results
text: Due to the page limit, we only report the performance comparisons of the proposed MILI-PML with six state-of-the-art PML methods on four real-world datasets and six synthetic datasets in terms of ranking loss, average precision and coverage in Tables 1, 2, 3 respectively. Besides, similar results can be observed in other metrics and we report the detailed results of hamming loss and one error metrics in the supplementary materials. From the overall results, we make the following observations:
• The proposed MILI-PML consistently outperforms all baselines on most real-world datasets, like Music-emotion, Music-style and MIRFlickr datasets, while is comparable to the best performance on YeastBP dataset. For example, MILI-PML is comparable to MUSER in terms of ranking loss and average precision, while comparable to DRAMA in terms of coverage.
• MILI-PML is superior to all baselines on most synthetic datasets, like Enron, Corel5k, Eurlex-sm, CAL500 and Mediamill while is comparable to the best performance on Eurlexed dataset. Specifically, MILI-PML is comparable to MUSER in terms of ranking loss and average precision, comparable to PML-LRS and MUSER in terms of coverage.
• It is obvious to observe that MILI-PML performs the best on most real-world as well as most synthetic datasets, except on YeastBP and Eurlex-ed. This is maybe because the number of feature and class combinations is relatively large compared with the number of instances in YeastBP and Eurlex-ed datasets, which increases the difficulty for identifying the ground-truth labels.
• The overall results demonstrate the superiorities of MILI-PML, which mainly owes to the natural consideration of the relevancy between features and labels as well as the overall redundancy of including an identified label into the ground-truth label set in the derived MILIPML. This aligns with our theoretical analysis that MILI-PML is -identifiable and capable of identifying the ground-truth labels correctly on the given conditions and assumptions of the data distribution.

heading: 7 Conclusion
text: This paper provides a new insight into partial multi-label learning problem from the perspective of mutual information. Instead of using hand-made heuristic modeling, we propose a novel method called MILI-PML, which is derived from a clear probabilistic formulation directly and can be easily interpreted theoretically by information theory as a result of naturally incorporating the relevancy considerations. Moreover, we present two fundamental issues about the assumption of existing PML research. To answer the first question “CAN", we define an -identifiable score function to evaluate whether a PML method is workable for identifying the ground-truth labels precisely. Furthermore, we demonstrate that the derived MILI-PML is -identifiable under certain assumptions and conditions theoretically. To answer the second question “HOW", we design an efficient optimization algorithm called MILI Algorithm to optimize the objective function. Extensive experiments are conducted on six synthetic PML datasets and four real-world PML datasets. The experimental results clearly demonstrate that the proposed MILI-PML consistently outperforms the state-of-the-art methods in terms of five widely-used multi-label metrics including ranking loss, hamming loss, one error, coverage, and average precision.

Review summaries:
The paper addresses the problem setting of partial multi-label learning (PML). The task is to select the ground-truth labels from a set of label candidates and disregard false-positive or noisy labels that may occur in the labelling process. The authors propose MILI-PML that exploits dependencies between labels and features for selection and relies on the assumption that ground-truth labels have a higher overall dependency between labels and features than false-positive ones. The paper provides theoretical analysis to show that the model can identify ground-truth labels from a candidate set correctly and propose a corresponding optimization procedure. Further, the authors conduct experiments to demonstrate the effectiveness of MILI-PML and claim that the method achieves superior performance compared to other SOTA methods.  ** Thanks for the response to this review. 

This paper presents two fundamental questions about the assumption of existing PML research. The authors derive a novel PML method from a probabilistic formulation which meets the $\epsilon$-identifiable definition. They provide theoretical analysis and algorithms to verify the proposed two questions. Thorough comparative experiments validate the superiorities of the proposed method.