Contrastive Label Disambiguation for Partial Label Learning

1 INTRODUCTION . The training of modern deep neural networks typically requires massive labeled data , which imposes formidable obstacles in data collection . Of a particular challenge , data annotation in the real-world can naturally be subject to inherent label ambiguity and noise . For example , as shown in Figure 1 , identifying an Alaskan Malamute from a Siberian Husky can be difficult for a human annotator . The issue of labeling ambiguity is prevalent yet often overlooked in many applications , such as web mining ( Luo & Orabona , 2010 ) and automatic image annotation ( Chen et al. , 2018 ) . This gives rise to the importance of partial label learning ( PLL ) ( Hüllermeier & Beringer , 2006 ; Cour et al. , 2011 ) , where each training example is equipped with a set of candidate labels instead of the exact groundtruth label . This stands in contrast to its supervised counterpart where one label must be chosen as the “ gold ” . Arguably , the PLL problem is deemed more common and practical in various situations due to its relatively lower cost to annotations . Despite the promise , a core challenge in PLL is label disambiguation , i.e. , identifying the groundtruth label from the candidate label set . Existing methods typically require a good feature representation ( Liu & Dietterich , 2012 ; Zhang et al. , 2016 ; Lyu et al. , 2021 ) , and operate under the assumption that data points closer in the feature space are more likely to share the same ground-truth label . However , the reliance on representations has led to a non-trivial dilemma—the inherent label uncertainty can undesirably manifest in the representation learning process—the quality of which may in turn prevent effective label disambiguation . To date , few efforts have been made to resolve this . This paper bridges the gap by reconciling the intrinsic tension between the two highly dependant problems—representation learning and label disambiguation—in one coherent and synergistic framework . Our framework , Partial label learning with COntrastive label disambiguation ( dubbed PiCO ) , produces closely aligned representations for examples from the same classes and facilitates label disambiguation . Specifically , PiCO encapsulates two key components . First , we leverage contrastive learning ( CL ) ( Khosla et al. , 2020 ) to partial label learning , which is unexplored in previous PLL literature . To mitigate the key challenge of constructing positive pairs , we employ the classifier ’ s output and generate pseudo positive pairs for contrastive comparison ( Section 3.1 ) . Second , based on the learned embeddings , we propose a novel prototype-based label disambiguation strategy ( Section 3.2 ) . Key to our method , we gradually update the pseudo target for classification , based on the closest class prototype . By alternating the two steps above , PiCO converges to a solution with a highly distinguishable representation for accurate classification . Empirically , PiCO establishes state-of-the-art performance on three benchmark datasets , outperforming the baselines by a significant margin ( Section 4 ) and obtains results that are competitive with fully supervised learning . Theoretically , we demonstrate that our contrastive representation learning and prototype-based label disambiguation are mutually beneficial , and can be rigorously interpreted from an ExpectationMaximization ( EM ) algorithm perspective ( Section 5 ) . First , the refined pseudo labeling improves contrastive learning by selecting pseudo positive examples accurately . This can be analogous to the E-step , where we utilize the classifier ’ s output to assign each data example to one label-specific cluster . Second , better contrastive performance in turn improves the quality of representations and thus the effectiveness of label disambiguation . This can be reasoned from an M-step perspective , where the contrastive loss partially maximizes the likelihood by clustering similar data examples . Finally , the training data will be mapped to a mixture of von Mises-Fisher distributions on the unit hypersphere , which facilitates label disambiguation by using the component-specific label . Our main contributions are summarized as follows : 1 ( Methodology ) . To the best of our knowledge , our paper pioneers the exploration of contrastive learning for partial label learning and proposes a novel framework termed PiCO . As an integral part of our algorithm , we also introduce a new prototype-based label disambiguation mechanism , leverages the contrastively learned embeddings . 2 ( Experiments ) . Empirically , our proposed PiCO framework establishes the state-of-the-art performance on three PLL tasks . Moreover , we make the first attempt to conduct experiments on fine-grained classification datasets , where we show classification performance improvement by up to 9.61 % compared with the best baseline on the CUB-200 dataset . 3 ( Theory ) . We theoretically interpret our framework from the expectation-maximization perspective . Our derivation is also generalizable to other CL methods and shows the alignment property in CL ( Wang & Isola , 2020 ) mathematically equals the M-step in center-based clustering algorithms . 2 BACKGROUND The problem of partial label learning ( PLL ) is defined using the following setup . Let X be the input space , and Y = { 1 , 2 , ... , C } be the output label space . We consider a training dataset D = { ( xi , Yi ) } ni=1 , where each tuple comprises of an image xi ∈ X and a candidate label set Yi ⊂ Y . Identical to the supervised learning setup , the goal of PLL is to obtain a functional mapping that predicts the one true label associated with the input . Yet differently , the PLL setup bears significantly more uncertainty in the label space . A basic assumption of PLL is that the groundtruth label yi is concealed in its candidate set , i.e. , yi ∈ Yi , and is invisible to the learner . For this reason , the learning process can suffer from inherent ambiguity , compared with the supervised learning task with explicit ground-truth . The key challenge of PLL is to identify the ground-truth label from the candidate label set . During training , we assign each image xi a normalized vector si ∈ [ 0 , 1 ] C as the pseudo target , whose entries denote the probability of labels being the ground-truth . The total probability mass of 1 is allocated among candidate labels in Yi . Note that si will be updated during the training procedure . Ideally , si should put more probability mass on the ( unknown ) ground-truth label yi over the course of training . We train a classifier f : X → [ 0 , 1 ] C using cross-entropy loss , with si being the target prediction . The per-sample loss is given by : Lcls ( f ; xi , Yi ) = ∑C j=1 −si , j log ( f j ( xi ) ) s.t . ∑ j∈Yi si , j = 1 and si , j = 0 , ∀j /∈ Yi , ( 1 ) where j denotes the indices of labels . si , j denotes the j-th pseudo target of xi . Here f is the softmax output of the networks and we denote f j as its j-th entry . In the remainder of this paper , we omit the sample index i when the context is clear . We proceed by describing our proposed framework . 3 METHOD . In this section , we describe our novel Partial label learning with COntrastive label disambiguation ( PiCO ) framework in detail . In a nutshell , PiCO comprises two key components tackling the representation quality ( Section 3.1 ) and label ambiguity respectively ( Section 3.2 ) . The two components systematically work as a whole and reciprocate each other . We further rigorously provide a theoretical interpretation of PiCO from an EM perspective in Section 5 . 3.1 CONTRASTIVE REPRESENTATION LEARNING FOR PLL . The uncertainty in the label space posits a unique obstacle for learning effective representations . In PiCO , we couple the classification loss in Eq . ( 1 ) with a contrastive term that facilitates a clustering effect in the embedding space . While contrastive learning has been extensively studied in recent literature , it remains untapped in the domain of PLL . The main challenge lies in the construction of positive sample set . In conventional supervised CL frameworks , the positive sample pairs can be easily drawn according to the ground-truth labels ( Khosla et al. , 2020 ) . However , this is not straightforward in the setting of PLL . Training Objective . To begin with , we describe the standard contrastive loss term . We adopt the most popular setups by closely following MoCo ( He et al. , 2020 ) and SupCon ( Khosla et al. , 2020 ) . Given each sample ( x , Y ) , we generate two views—a query view and a key view—by way of randomized data augmentation Aug ( x ) . The two images are then fed into a query network g ( · ) and a key network g′ ( · ) , yielding a pair of L2-normalized embeddings q = g ( Augq ( x ) ) and k = g′ ( Augk ( x ) ) . In implementations , the query network shares the same convolutional blocks as the classifier , followed by a prediction head ( see Figure 2 ) . Following MoCo , the key network uses a momentum update with the query network . We additionally maintain a queue storing the most current key embeddings k , and we update the queue chronologically . To this end , we have the following contrastive embedding pool : A = Bq ∪Bk ∪ queue , ( 2 ) where Bq and Bk are vectorial embeddings corresponding to the query and key views of the current mini-batch . Given an example x , the per-sample contrastive loss is defined by contrasting its query embedding with the remainder of the pool A , Lcont ( g ; x , τ , A ) = − 1 |P ( x ) | ∑ k+∈P ( x ) log exp ( q > k+/τ ) ∑ k′∈A ( x ) exp ( q > k′/τ ) , ( 3 ) where P ( x ) is the positive set and A ( x ) = { A\ { q } } . τ ≥ 0 is the temperature . Positive Set Selection . As mentioned earlier , the crucial challenge is how to construct the positive set P ( x ) . We propose utilizing the predicted label ỹ = arg maxj∈Y f j ( Augq ( x ) ) from the classifier . Note that we restrict the predicted label to be in the candidate set Y . The positive examples are then selected as follows , P ( x ) = { k′|k′ ∈ A ( x ) , ỹ′ = ỹ } . ( 4 ) where ỹ′ is the predicted label for the corresponding training example of k′ . For computational efficiency , we also maintain a label queue to store past predictions . In other words , we define the positive set of x to be those examples carrying the same approximated label prediction ỹ . Despite its simplicity , we show that our selection strategy can be theoretically justified ( Section 5 ) and also lead to superior empirical results ( Section 4 ) . Note that more sophisticated selection strategies can be explored , for which we discuss in Appendix B.4 . Putting it all together , we jointly train the classifier as well as the contrastive network . The overall loss function is : L = Lcls + λLcont . ( 5 ) Still , our goal of learning high-quality representation by CL relies on accurate classifier prediction for positive set selection , which remains unsolved in the presence of label ambiguity . To this end , we further propose a novel label disambiguation mechanism based on contrastive embeddings and show that these two components are mutually beneficial .

Summary: The paper proposes an innovative approach to partial label learning, where an instance is assigned with some false positive labels besides its true label (due to difficulty in labelling). The approach blends contrastive learning with prototype learning: (i) the former helps to form good clusters for the latter to learn prototype representations (ii) in return the latter helps to select positive samples for the former. Theoretically, the authors prove that the two of them work collaboratively in the EM fashion.   The paper demonstrates the effectiveness of the proposed approach using a typical setting for the task. Specifically, CIFAR 10 and CIFAR 100 were used and each instance was randomly assigned false labels with some probabilities. The proposed approach achieved impressive results, substantially outperforming five recent models in the literature. Moreover, its performance strongly approaches the one of supervised learning.   The paper also presents a much stricter setting when false positive labels are semantically correlated with true labels. The proposed approach also gained impressive results on CUB-200 and CIFAR-100-H datasets. Moreover, the paper shows several in-depth analyses.