Instance-Dependent Partial Label Learning
Year: 2022

Abstract:
Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels. However, this assumption is not realistic since the candidate labels are always instancedependent. In this paper, we consider instance-dependent PLL and assume that each example is associated with a latent label distribution constituted by the real number of each label, representing the degree to each label describing the feature. The incorrect label with a high degree is more likely to be annotated as the candidate label. Therefore, the latent label distribution is the essential labeling information in partially labeled examples and worth being leveraged for predictive model training. Motivated by this consideration, we propose a novel PLL method that recovers the label distribution as a label enhancement (LE) process and trains the predictive model iteratively in every epoch. Specifically, we assume the true posterior density of the latent label distribution takes on the variational approximate Dirichlet density parameterized by an inference model. Then the evidence lower bound is deduced for optimizing the inference model and the label distributions generated from the variational posterior are utilized for training the predictive model. Experiments on benchmark and real-world datasets validate the effectiveness of the proposed method. Source code is available at https://github.com/palm-ml/valen.

heading: 1 Introduction
text: Partial label learning (PLL) deals with the problem where each training example is associated with a set of candidate labels, among which only one label is valid [7, 5, 37]. Due to the difficulty in collecting exactly labeled data in many real-world scenarios, PLL leverages inexact supervision instead of exact labels. The need to learn from the inexact supervision leads to a wide range of applications for PLL techniques, such as web mining [25], multimedia content analysis [38, 4], ecoinformatics [24, 31], etc.
To accomplish the task of learning from partial label data, many approaches have been proposed. Identification-based PLL approaches [17, 28, 24, 5, 37] regard the ground-truth label as a latent variable and try to identify it. Average-based approaches [15, 7, 40] treat all the candidate labels equally and average the modeling outputs as the prediction. For confidence-based approaches [8, 34, 42], the confidence of each label is estimated instead of identifying the ground-truth label. These approaches always adopt the randomly picked candidate labels to corrupt benchmark data into partially labeled version despite having no explicit generation process of candidate label sets. To depict the instance-independent generation process of candidate label sets, Feng [9] proposes a statistical model and deduces a risk-consistent method and a classifier-consistent method. Under the
∗Corresponding author
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
same generation process, another classifier-consistent risk estimator is proposed for deep model and stochastic optimizers [26].
The previous methods assume that the candidate labels are randomly sampled with the uniform generating procedure [26, 9], which is commonly adopted to corrupt benchmark datasets into partially labeled versions in their experiments. However, the candidate labels are always instance-dependent (feature-dependent) in practice as the incorrect labels related to the feature are more likely to be picked as candidate label set for each instance. These methods usually do not perform as well as expected due to the unrealistic assumption on the generating procedure of candidate label sets.
In this paper, we consider instance-dependent PLL and assume that each instance in PLL is associated with a latent label distribution [33, 35, 11] constituted by the real number of each label, representing the degree to each label describing the feature. Then, the incorrect label with a high degree in the latent label distribution is more likely to be annotated as the candidate label. For example, the candidate label set of the handwritten digits in Figure 1(a) contains “1”, “3” and “5”, where “1” and “3” are not ground-truth but selected as candidate labels due to their high degrees in the latent label distribution of the instance. The object in Figure 1(b) is annotated with “bird” and “airplane” as the degrees of these two labels are much higher than others in the label distribution. The intrinsical ambiguity increases the difficulty of annotating, which leads to the result that annotators pick the candidate labels with high degrees in the latent label distribution of each instance instead of annotating the ground-truth label directly in PLL. Therefore, the latent label distribution is the essential labeling information in partially labeled examples and worth being leveraged for predictive model training.
Motivated by the above consideration, we deal with the PLL problem from two aspects. First, we enhance the labeling information by recovering the latent label distribution for each training example as a label enhancement process [33, 35]. Second, we run label enhancement and train the predictive model with recovered label distributions iteratively. The proposed method named VALEN, i.e., VAriational Label ENhancement for instance-dependent partial label learning, uses the candidate labels to initialize the predictive model in the warm-up training stage, then recovers the latent label distributions via inferring the variational posterior density parameterized by an inference model with the deduced evidence lower bound, and trains the predictive model with a risk estimator by leveraging the candidate labels as well as the label distributions. Our contributions can be summarized as follows:
• We for the first time consider the instance-dependent PLL and assume that each partially labeled example is associated with a latent label distribution, which is the essential labeling information and worth being recovered for predictive model training.
• We infer the posterior density of the latent label distribution via taking on the approximate Dirichlet density parameterized by an inference model and deduce the evidence lower bound for optimization, in which the topological information and the features extracted from the predictive model are leveraged.
• We train predictive model with a proposed empirical risk estimator by leveraging the candidate labels as well as the label distributions. We iteratively recover the latent label distributions and train the predictive model in every epoch. After the network has been fully trained, the predictive model can perform predictions for future test examples alone.
Experiments on the corrupted benchmark datasets and real-world PLL datasets validate the effectiveness of the proposed method.

heading: 2 Proposed Method
text: First of all, we briefly introduce some necessary notations. Let X = Rq be the q-dimensional instance space and Y = {y1, y2, ..., yc} be the label space with c class labels. Given the PLL training set D = {(xi, Si)|1 ≤ i ≤ n} where xi denotes the q-dimensional instance and Si ⊆ Y denotes the candidate label set associated with xi. Note that Si contains the correct label of xi and the task of PLL is to induce a multi-class classifier f : X 7→ Y from D. For each PLL training example (xi, Si), we use the logical label vector li = [l y1 i , l y2 i , . . . , l yc i ] > ∈ {0, 1}c to represent whether yj is the candidate label, i.e., lyji = 1 if yj ∈ Si, otherwise l yj i = 0. The label distribution of xi is denoted by di = [d y1 i , d y2 i , . . . , d yc i ] > ∈ [0, 1]c where ∑c j=1 d yj i = 1. Then L = [l1, l2, . . . , ln] and D = [d1,d2, . . . ,dn] represent the logical label matrix and label distribution matrix, respectively.

heading: 2.1 Overview
text: To deal with PLL problem, we iteratively recover the latent label distribution for each example x and train the predictive model by leveraging the recovered label distribution. We start with a warm-up period, in which we train the predictive model with the PLL minimal loss [26]. This allows us to attain a reasonable predictive model before it starts fitting incorrect labels. After the warm-up period, the features extracted from the predictive model can help for recovering the latent label distribution. Benefited from the essential labeling information in the recovered label distribution, the performance of the predictive model could be further improved.
VALEN implements label enhancement and classifier training iteratively in every epoch. In label enhancement, we assume the true posterior density of the latent label distribution takes on the variational approximate Dirichlet density parameterized by an inference model. Then the evidence lower bound is deduced for optimizing the inference model and the label distributions can be generated from the variational posterior. In classifier training, the predictive model is trained by leveraging the recovered label distributions and candidate labels with an empirical risk estimator. After the models has been fully trained, the predictive model can perform prediction for future test instances alone.

heading: 2.2 Warm-up Training
text: The predictive model θ is trained on partially labeled examples by minimizing the following PLL minimal loss function [26]:
Lmin = n∑ i=1 min yj∈Si `(f(xi), e yj ), (1)
where ` is cross-entropy loss and eY = {eyj : yj ∈ Y} denotes the standard canonical vector inRc, i.e., the j-element in eyj equals 1 and others equal 0. Similar to [26], the min operator in Eq. (1) is replaced by using the current predictions for slightly weighting on the possible labels in warm-up training. Then we could extract the feature φ of each x via using the predictive model.

heading: 2.3 Label Enhancement
text: We assume that the prior density p(d) is a Dirichlet with α̂, i.e., p (d) = Dir (d | α̂) where α̂ = [ε, ε, . . . , ε]> is a c-dimensional vector with a minor value ε. Then we let the prior density p(D) be the product of each Dirichlet
p(D) = n∏ i=1 Dir(di|α̂). (2)
We consider the topological information of the feature space, which is represented by the affinity graph G = (V,E,A). Here, the feature vector φi of each example could be extracted from the predictive model θ in current epoch, V = {φi | 1 ≤ i ≤ n} corresponds to the vertex set consisting of feature vectors, E = {(φi,φj) | 1 ≤ i 6= j ≤ n} corresponds to the edge set, and a sparse adjacency matrix A = [aij ]n×n can be obtained by
aij = { 1 if φi ∈ N (φj) 0 otherwise , (3)
where N (φj) is the set for k-nearest neighbors of φj and the diagonal elements of A are set to 1. Let features matrix Φ = [φ1,φ2, . . . ,φn], adjacency matrix A and logical labels L be observed matrix, VALEN aims to infer the posterior density p(D|L,Φ,A). As the computation of the exact posterior density p(D|L,Φ,A) is intractable, a fixed-form density q(D|L,Φ,A) is employed to approximate the true posterior. We let the approximate posterior be the product of each Dirichlet parameterized by a vector αi = [α1i , α 2 i , . . . , α c i ] >:
qw(D | L,Φ,A) = n∏ i=1 Dir (di|αi) . (4)
Here, the parameters ∆ = [α1,α2, . . . ,αn] are outputs of the inference model parameterized by w, which is defined as a two-layer GCN [20] by GCN(L,Φ,A) = Ã ReLU ( ÃZW0 ) W1, with
Z = [Φ; L] and weight W0, W1. Here Ã = Â− 1 2 AÂ− 1 2 is the symmetrically normalized weight
matrix where Â is the degree matrix of A.
By following the Variational Bayes techniques, a lower bound on the marginal likelihood of the model is derived which ensures that qw(D|L,Φ,A) is as close as possible to p(D|L,Φ,A). For logical label matrix L, feature matrix Φ, and the corresponding A, the log marginal probability can be decomposed as follows 1:
log p(L,Φ,A) = LELBO + KL[qw(D|L,Φ,A)||p(D|L,Φ,A)]. (5) where
LELBO = Eqw(D|L,Φ,A)[log p(L,Φ,A|D)]− KL[qw(D|L,Φ,A)||p(D)]. (6) Due to the non-negative property of KL divergence, the first term LELBO constitutes a lower bound of log p(L,Φ,A), which is often called as evidence lower bound (ELBO), i.e., log p(L,Φ,A) ≥ LELBO. According to Eq. (2) and Eq. (4) , the KL divergence in Eq. (6) can be analytically calculated as follows:
KL (qw(D|L,Φ,A)‖p(D)) = ∑n i=1 ( log Γ (∑c j=1 α j i ) − ∑c j=1 log Γ ( αji ) − log Γ (c · ε) + c log Γ (ε) + ∑c j=1 ( αji − ε )( ψ ( αji ) − ψ (∑c j=1 α j i ))) .
(7)
where Γ(·) and ψ(·) are Gamma function and Digamma function, respectively. As the first part of Eq. (6) is intractable, we employ the implicit reparameterization trick [10] to approximate it by Monte Carlo (MC) estimation. Inspired by [20], we simply drop the dependence on Φ:
p(L | A,D) = n∏ i=1 p (li | A,D) ,
p(A | D) = n∏ i=1 n∏ j=1 p (aij | di,dj) , with p (aij = 1 | di,dj) = s ( d>i dj ) .
(8)
Here, s(·) is the logistic sigmoid function. We further assume that p (li|A,D) is a multivariate Bernoulli with probabilities τi. In order to simplify the observation model, T(m) = [τ
(m) 1 , τ (m) 2 , . . . , τ (m) n ] is computed from m-th sampling D(m) with a three-layer MLP parame-
terized by η. Then the first part of Eq. (6) can be tractable:
Eqw(D|L,Φ,A)[log pη(L,Φ,A|D)] = 1
M M∑ m=1
( tr ( (I− L)> log ( I−T(m) )) + tr ( L> log T(m) ) − ‖A− S ( D(m)D(m)> ) ‖2F ) .
(9)
1More detailed calculations can be seen in Appendix A.1.
Algorithm 1 VALEN Algorithm Input: The PLL training set D = {(xi, Si)}ni=1, epoch T and iteration I;
1: Initialize the predictive model θ by warm-up training, the reference model w and observation model η; 2: Extract the features Φ from predictive model θ and calculate the adjacency matrix A; 3: for t = 1, . . . , T do 4: Shuffle training set D = {(xi, Si)}ni=1 into I mini-batches; 5: for k = 1, . . . , I do 6: Obtain label distribution di for each example xi by Eq. (4); 7: Update θ,w and η by forward computation and back-propagation by fusing Eq. (12) and
Eq. (13); 8: end for 9: end for
Output: The predictive model θ.
Note that we can use only one MC sample in Eq. (9) during the training process as suggested in [19, 35].
In addition, VALEN improves the label enhancement by employing the compatibility loss, which enforces that the recovered label distributions should not be completely different from the confidence ζ(xi) [9, 26] estimated by current prediction f(xi;θ):
Lo = − 1
n n∑ i=1 c∑ j=1 ζj(xi) log d yj i (10)
where
ζj(xi) =
{ fj (xi;θ) / ∑ yk∈Si fk (xi;θ) if yj ∈ Si 0 otherwise (11)
Now we can easily get the objective of label enhancement LLE as follows:
LLE = λLo − LELBO (12)
where λ is a hyper-parameter. The label distribution matrix D is sampled from q(D|L,Φ,A), i.e., di ∼ Dir(αi). Note that the implicit reparameterization gradient [10] is employed, which avoids the inversion of the standardization function, which makes the gradients can be computed analytically in backward pass.

heading: 2.4 Classifier Training
text: To train the predictive model, we minimize the following empirical risk estimator by levering the recovered label distributions:
R̂V (f) = 1
n n∑ i=1 ∑ yj∈Si d yj i∑ yj∈Si d yj i `(f(xi), e yj )  . (13) Here we adopt the average value of di sampled by di ∼ Dir(αi). We can use any deep neural network as the predictive model, and then equip it with the VALEN framework to deal with PLL. Note that we could train the predictive model and update the label distributions in a principled end-to-end manner by fusing the objective Eq. (12) and Eq. (13). The algorithmic description of the VALEN is shown in Algorithm 1.
Let f̂V = minf∈F R̂V (f) be the empirical risk minimizer and f? = minf∈F RV (f) be the optimal risk minimizer where RV (f) is the risk estimator. Besides, we define the function spaceHyj for the label yj ∈ Y as { h : x 7→ fyj (x) | f ∈ F } . Let Rn ( Hyj ) be the expected Rademacher complexity [2] ofHyj with sample size n, then we have the following theorem.
Theorem 1 Assume the loss function `(f(x), eyj ) is L-Lipschitz with respect to f(x)(0 < L <∞) for all yj ∈ Y and upper-bounded by M , i.e., M = supx∈X ,f∈F,yj∈Y `(f(x), e yj ). Then, for any
δ > 0, with probability at least 1− δ,
R ( f̂V ) −R (f?) ≤ 4 √ 2L c∑ j=1 Rn ( Hyj ) +M √ log 2δ 2n
The proof of Theorem 1 is provided in Appendix A.2. Theorem 1 shows that the empirical risk minimizer fV converges to the optimal risk minimizer f? as n → ∞ and Rn ( Hyj ) → 0 for all parametric models with a bounded norm.

heading: 3 Related Work
text: As shown in Section 1, supervision information conveyed by partially labeled training examples is implicit as the ground-truth label is hidden within the candidate label set. Therefore, partial label learning can be regarded as a weak supervision learning framework [18] with implicit labeling information. Intuitively, the basic strategy for handling partial label learning is disambiguation, i.e., trying to identify the ground-truth label from the candidate label set associated with each training example, where existing strategies include disambiguation by identification or disambiguation by averaging. For identification-based disambiguation, the ground-truth label is regarded as a latent variable and identified [17, 28, 24, 5, 37]. For averaging-based disambiguation, all the candidate labels are treated equally and the prediction is made by averaging their modeling outputs [15, 7, 40].
Most existing algorithms aim to fulfill the learning task by fitting widely-used learning techniques to partial label data. For maximum likelihood techniques, the likelihood of observing each partially labeled training example is defined over its candidate label set instead of the unknown ground-truth label [17, 24]. K-nearest neighbor techniques determine the class label of unseen instances via voting among the candidate labels of its neighboring examples [15, 40]. For maximum margin techniques, the classification margins over the partially labeled training examples are defined by discriminating modeling outputs from candidate labels and non-candidate labels [28, 37]. For boosting techniques, the weight over each partially labeled training example and the confidence over the candidate labels are updated in each boosting round [31]. For disambiguation-free strategies, the generalized description degree is estimated by using a graph Laplacian and induce a multi-output regression [34]. The confidence of each candidate label is estimated by using the manifold structure of feature space [42]. However, these methods just estimate the soft labeling information and train the predictive models in separate stages without considering the feedback of the predictive models.
The above-mentioned works were solved in specific low-efficiency manners and incompatible with high-efficient stochastic optimization. To handle large-scale datasets, the deep networks are employed with an entropy-based regularizer to maximize the margin between the potentially correct label and the unlikely ones [36]. [26] proposes a classifier-consistent risk estimator and a progressive identification, which is compatible with deep models and stochastic optimizers. [9] proposes a statistical model to depict the generation process of candidate label sets, which deduces a risk-consistent method and a classifier-consistent method.
The previous methods assume that the candidate labels are randomly sampled with the uniform generating procedure. However, the candidate labels are always instance-dependent (feature-dependent) in practice as the incorrect labels related to the feature are more likely to be picked as candidate label set for each instance. In this paper, we consider instance-dependent PLL and assume that each instance in PLL is associated with a latent label distribution [33, 35, 11] constituted by the real number of each label, representing the degree to each label describing the feature. Label enhancement (LE) [12, 33, 35] recovers the latent label distribution from the observed logical labels, in which the recovered label distribution is a kind of pseudo label [29, 23], actually. Note that the generation process of soft labels in label smoothing [30, 27] and distillation [14, 39] could also be regarded as a label enhancement process.

heading: 4 Experiments

heading: 4.1 Datasets
text: We adopt four widely used benchmark datasets including MNIST [22], Fashion-MNIST [32], Kuzushiji-MNIST [6], and CIFAR-10 [21], and five datasets from the UCI Machine Learning Repository [1], including Yeast, Texture, Dermatology, Synthetic Control, and 20Newgroups.
We manually corrupt these datasets into partially labeled versions 2 by using a flipping probability ξ yj i = P (l yj i = 1|l̂ yj i = 0,xi) , where l̂ yj i is the original clean label. To synthesize the instancedependent candidate labels, we set the flipping probability of each incorrect label corresponding to an example xi by using the confidence prediction of a clean neural network θ̂ (trained with the original clean labels) [43] with ξyji = fj(xi;θ̂)
maxyj∈Ȳi fj(xi;θ̂) , where Ȳi is the incorrect label set of xi. The
uniform corrupted version adopts the uniform generating procedure [26, 9] to flip the incorrect label into candidate label, where ξyji = 1 2 .
In addition, five real-world PLL datasets are adopted, which are collected from several application domains including Lost [7], Soccer Player [38] and Yahoo!News [13] for automatic face naming from images or videos, MSRCv2 [24] for object classification, and BirdSong [3] for bird song classification. The detailed descriptions of these datasets are provided in Appendix A.3.
We run 5 trials on the four benchmark datasets and perform five-fold cross-validation on UCI datasets and real-world PLL datasets. The mean accuracy as well as standard deviation are recorded for all comparing approaches.

heading: 4.2 Baselines
text: The performance of VALEN is compared against five DNN based approaches: 1) PRODEN [26]: A progressive identification partial label learning approach which approximately minimizes a risk estimator and identifies the true labels in a seamless manner; 2) RC [9]: A risk-consistent partial label learning approach which employs the importance reweighting strategy to converges the true risk
2The datasets corrupted by the instance-dependent generating procedure are available at https://drive. google.com/drive/folders/1J_68EqOrLN6tA56RcyTgcr1komJB31Y1?usp=sharing.
minimizer; 3) CC [9]: A classifier-consistent partial label learning approach which uses a transition matrix to form an empirical risk estimator; 4) D2CNN [36]: A deep partial label learning approach which design an entropy-based regularizer to maximize the margin between the potentially correct label and the unlikely ones; 5) GA [16]: An unbiased risk estimator approach which can be applied for partial label learning.
For all the DNN based approaches, we adopt the same predictive model for fair comparisons. Specifically, the 32-layer ResNet is trained on CIFAR-10 in which the learning rate, weight decay and mini-batch size are set to 0.05, 10−3 and 256, respectively. The three-layer MLP is trained on MNIST, Fashion-MNIST and Kuzushiji-MNIST where the learning rate, weight decay and mini-batch size are set to 10−2, 10−4 and 256, respectively. The linear model is trained on UCI and real-world PLL datasets where the learning rate, weight decay and mini-batch size are set to 10−2, 10−4 and 100, respectively. We implement the comparing methods with PyTorch. The number of epochs is set to 500, in which the first 10 epochs are warm-up training.
In addition, we also compare with five classical partial label learning approaches, each configured with parameters suggested in respective literatures: 1) CLPL [7]: A convex partial label learning approach which uses averaging-based disambiguation; 2) PL-KNN [15]: An instance-based partial label learning approach which works by k-nearest neighbor weighted voting; 3) PL-SVM [28]: A maximum margin partial label learning approach which works by identification-based disambiguation; 4) IPAL [41]: A non-parametric method that applies the label propagation strategy to iteratively update the confidence of each candidate label; 5) PLLE [34]: A two-stage partial label learning approach which estimates the generalized description degree of each class label values via graph Laplacian and induces a multi-label predictive model with the generalized description degree in separate stages.

heading: 4.3 Experimental Results
text: Table 1 reports the classification accuracy of each DNN-based method on benchmark datasets corrupted by the instance-dependent generating procedure. The best results are highlighted in bold. In addition, • / ◦ indicates whether VALEN is statistically superior/inferior to the comparing approach on each dataset (pairwise t-test at 0.05 significance level). From the table, we can observe that VALEN always achieves the best performance and significantly outperforms other compared methods in most cases. In addition, we also validate the effectiveness of our approach on uniform corrupted versions that is commonly adopted in previous works. From Table 2, we can observe that VALEN achieves superior or at least comparable performance to other approaches on uniform corrupted versions.
Table 3 and Table 4 report the classification accuracy of each method on UCI datasets corrupted by the instance-dependent generating procedure and the uniform generating procedure, respectively. VALEN always achieves the best performance and significantly outperforms other DNN-based methods in most cases on instance-dependent corrupted versions while achieves superior or at least comparable performance to other approaches on uniform corrupted versions. We further compare VALEN with five classical PLL methods that can hardly be implemented by DNNs on large-scale datasets. Despite the small scale of most UCI datasets, VALEN always achieve the best performance in most cases against the classical PLL methods as VALEN can deal with the high average number of candidate labels (can be seen in Appendix A.3) in the corrupted UCI datasets.
Table 5 reports the experimental results on real-world PLL datasets. We can find that VALEN achieves best performance against other DNN-based methods on the real-world PLL datasets. Note that VALEN achieves best performance against classical methods on all datasets except Lost and MSRCv2 as these datasets are small-scale and the average number of candidate labels in each dataset is low (can be seen in Appendix A.3), which leads to the result that DNN-based methods cannot take full advantage.
Figure 2(a) and Figure 2(b) illustrate the performance of VALEN on KMNIST corrupted by the instance-dependent generating procedure and the uniform generating procedure under different flipping probability, respectively. Besides, the performance of the ablation version that removes the label enhancement and trains the predictive model with PLL minimal loss (denoted by VALEN-NON) is recorded. These results clearly validate the usefulness of recovered label distributions for improving predictive performance. Figure 2(c) illustrates the recovered label distribution matrix over all training examples converges as the number of epoch (after warm-up training) on Kuzushiji-MNIST. We can see that the recovered label distributions converge fast with the increasing number of epoch.

heading: 5 Conclusion
text: In this paper, the problem of partial label learning is studied where a novel approach VALEN is proposed. We for the first time consider the instance-dependent PLL and assume that each partially labeled example is associated with a latent label distribution, which is the essential labeling information and worth being recovered for predictive model training. VALEN recovers the latent label distribution via inferring the true posterior density of the latent label distribution by Dirichlet density
parameterized with an inference model and deduce the evidence lower bound for optimization. In addition, VALEN iteratively recovers latent label distributions and trains the predictive model in every epoch. The effectiveness of the proposed approach is validated via comprehensive experiments on both synthesis datasets and real-world PLL datasets.

heading: 6 Acknowledgments
text: This research was supported by the National Key Research & Development Plan of China (No. 2018AAA0100104, No. 2018AAA0100100), the National Science Foundation of China (62125602, 62076063), China Postdoctoral Science Foundation (2021M700023), Jiangsu Province Science Foundation for Youths (BK20210220).

Review summaries:
This paper studies the problem of feature-dependent partial label learning, which is an interesting weakly supervised learning problem where the candidate labels of each instance are feature-dependent. Accordingly, the first attempt to feature-dependent partial label learning with latent label distribution is proposed.  The performance advantage of the proposed approach over state-of-the-art approaches is clearly validated via extensive experimental studies.

This paper proposes an algorithm for learning from feature-dependent partial labels. The paper is based on a latent distribution over the weak labels which is estimated during training. A variational approach is used to make approximate inference computationally tractable. The experiments show that the proposed algorithms outperform many algorithm in the state of the art. Experiments are done using real data with synthetic label corruptions and also with real weak labels.

This work approaches the feature-dependent partial-label learning problem where each training example has a set of candidate labels that are generated by feature-dependent procedure rather than random procedure. This paper assumes that each instance in PLL is associated with a latent label distribution which is the essential labeling information in partially labeled examples and worth being leveraged for predictive model training. Inspired by this, the authors propose a novel method to enhance the labeling information by recovering the latent label distribution and train the predictive model with recovered label distributions. In the experiment, the authors show the effectiveness of the method in the benchmark and real-world datasets.