Label Augmentation with Reinforced Labeling for Weak Supervision

1 INTRODUCTION . Supervised machine learning has proven to be very powerful and effective for solving various classification problems . However , training fully-supervised models can be costly since many applications require large amounts of labeled data . Manually annotating each data point of a large dataset may take up to weeks or even months . Furthermore , only domain experts can label the data in highly specialized scenarios such as healthcare and industrial production . Thus , the costs of data labeling might become very high . In the past few years , a new weak supervision ( WS ) approach , namely data programming [ Ratner et al . ( 2016 ; 2017 ) ] , has been proposed to significantly reduce the time for dataset preparation . In this approach , a domain expert writes heuristic functions named labeling functions ( LFs ) instead of labeling each data point . Each function annotates a subset of the dataset with an accuracy expected to be better than a random prediction . Data programming has been successfully applied to various classification tasks . However , writing LFs might not always be trivial , for instance , when data points are huge vectors of numbers or when they are not intuitively understandable . Developers can quickly code a few simple functions , but having heuristics to cover many corner cases is still a burden . Further , simple heuristics might cover only a tiny portion of the unlabeled dataset ( small coverage problem ) . The existing data programming framework Snorkel [ Ratner et al . ( 2016 ; 2017 ) ] implements a machine learning pipeline as follows . The LFs are applied to the unlabeled data points and the outcomes of LFs produce a labeling matrix where each data point might be annotated by multiple , even conflicting , labels . A generative model processes the labeling matrix to make single label predictions for a subset of data points , based on the agreements and disagreements on the LF outputs for a given data point x ( i ) using techniques such as majority voting ( MV ) or minimizing the marginalized log-likelihood estimates . Later , the label predictions are used to train a supervised model that serves as the end classifier ( discriminative model ) . This approach has two major limitations : 1 ) Coarse information about the dataset ( only LF outputs ) fed to the generative model , 2 ) Lack of generalization due to the sparsity of labeling matrix and relying only on end classifier to generalize . The current data programming does not take the data points ’ data features into account during the generative process , even though they are available throughout the pipeline . It utilizes the data features of only the data points with label predictions to train the end classifier . Various additional techniques are considered to complement the existing approach [ Varma & Ré ( 2018 ) ; Chatterjee et al . ( 2020 ) ; Varma et al . ( 2019 ) ; Nashaat et al . ( 2018 ) ; Varma et al . ( 2016 ) ] and improve the learning process . However , these approaches do not address this major problem in the design of existing data programming . This article proposes a label augmentation approach for weak supervision that takes the data features and the LF outputs into account earlier in the generative process . The proposed approach utilizes the features for augmenting labels . The augmentation algorithm , namely reinforcement labeling ( RL ) , checks similarities between data features to project existing LF outcomes to non-existing labels in the matrix ( i.e. , to unknown cases or abstains ) . Moreover , it uses a heuristic that considers unknown cases “ gravitate ” towards the known cases with LF output labels . In such a way , RL enables generalization early on and creates a “ reinforced ” label set to train an end classifier . Label augmentation extends the data programming to new scenarios , such as when LFs have low coverage , domain experts can implement only a limited number of LFs , or LFs outcome result in a sparse labeling matrix . Label augmentation can provide satisfactory performances in these cases , although data programming was previously non-applicable . The proposed approach can reduce the time spent by the domain experts to train a classifier as they need to implement fewer LFs . One advantage compared to the existing complementary approaches is that RL does not require any additional effort for labeling data , annotating data , or implementing additional LFs . In other words , the label augmentation enhances classification without any further development burden or assumption of available labeled datasets ( e.g. , the so-called “ gold data ” ) . Furthermore , one can easily combine this approach with the existing solutions . The RL method is implemented and tested compared to Snorkel ( Sn ) using different fully-supervised models as end classifiers . The experimental results span classification tasks from several domains ( YouTube comments , white/red wine datasets , weather prediction ) . The new approach outperforms the existing model in terms of accuracy and F1 scores , having closer outcomes to the fully-supervised learning , thanks to the improved coverage that enables end classifier convergence . 2 METHOD DESCRIPTION . 2.1 BACKGROUND ON DATA PROGRAMMING . In data programming [ Bach et al . ( 2019 ) ; Ratner et al . ( 2017 ) ] , a set of LFs annotate a portion ( subset ) of the original unlabeled dataset X = { x ( 1 ) , x ( 2 ) , . . . , x ( k ) } with a total labeling coverage of γ ∗ |X| , where γ ∈ [ 0 , 1 ] . Given a data point x ( i ) , an LFj takes x ( i ) as input and annotates the input with a label . LFs are considered weak supervisors implemented by application developers , and they can programmatically annotate many data points at once , as opposed to hand-labeling data points one by one . On the other hand , LFs may have lower accuracies than ground truth for the data points . For a binary classification task , an LF may return two classes or abstain from making a prediction . For simplicity , let us consider LFj returning 1 or 0 as the class labels and -1 ( abstain ) when it refrains from class prediction . LFs ’ outputs form a labeling matrix where rows represent the data point indices x ( 1 ) , x ( 2 ) , . . . , x ( k ) , and columns represent the LF indices LF1 , LF2 , . . . , LFm . A generative model takes the labeling matrix as input , filters out the data points with no label ( all LFs voted for abstains ) , and tries to predict a label for the remaining data points . An example of a generative model might be a majority voter ( MV ) based on LF outputs or by minimizing the negative log marginal likelihood [ Ratner et al . ( 2017 ) ] ( likelihood over latent variables , i.e. , LF outputs ) . Dependency structures between the LFs are learned as shown in Alg . 1 in [ Bach et al . ( 2017 ) ] . In both MV and marginal likelihood approaches , the generative model takes the labeling matrix as input and tries to make a label decision based on the agreements or disagreements of LFs . The generative model may fail to make a decision in certain cases , such as when equal numbers of LFs disagree on a data point . Lastly , the features of the weakly-labeled data points within X and the labels from the generative model are used to supervise an end classifier ( discriminative ) model . The design of the data programming model is agnostic to the end classifier ( discriminative ) model , so various supervised machine learning models can be candidates as the end model . 2.2 LABEL AUGMENTATION AND REINFORCED LABELS . In the described design of data programming , the two limitations mentioned above ( in Sec . 1 ) , namely the coarse information in the labeling matrix and sparsity , lead to failure for generalizing to new and unseen data points . Therefore , these limitations may lead to reduced performances . In such scenarios , the outputs of the existing generative model may not be satisfactory to train the end classifier . Therefore , the end classifier model may not converge or generalize well enough to cover different cases . Implementing many LFs that cover different cases is costly and not very straightforward in most scenarios . Although various additional techniques focus on the weak supervision problem [ Varma & Ré ( 2018 ) ; Chatterjee et al . ( 2020 ) ; Varma et al . ( 2019 ) ; Nashaat et al . ( 2018 ) ; Varma et al . ( 2016 ) ] , they rather extend the existing pipeline with additional features . On the other hand , label augmentation targets these major limitations by eliminating the sparsity using data features . Thus , it can lead to higher accuracy . Fig . 1 illustrates the new pipeline for label augmentation . The new generative process brings together the outputs of 〈LF1 , LF2 , . . . , LFm〉 , and the data features f1 , f2 . . . , fn of data points x ( i ) = { x ( i ) 1 , x ( i ) 2 , . . . , x ( i ) n } in the unlabeled dataset early on . Different methods can utilize data features in the generative process for augmenting the labels in the labeling matrix . This label augmentation approach differs from the existing “ data augmentation ” approaches [ Wang et al . ( 2019 ) ; Tran et al . ( 2017 ) ; Cubuk et al . ( 2019 ) ] that create new ( synthetic ) data points , as the new goal is to project LF outcomes to the existing data points as opposed to creating new data points . The outcome of the new generative process can be more representative of the data and weak supervisors than the outputs of the previously existing generative process due to additional coverage and accuracy gains without any additional LF implementation or data annotation . For instance , abstain values ( -1s ) from the LFs ’ outputs representing the unknown cases ( left side of the matrix in Fig . 1 ) can be predicted by the new generative process ( as outlined in Sec . 2.3 ) . The abstain values in the labeling matrix can be augmented with classification prediction values . Data programming applies statistics only to data points that are already covered by LFs , resulting in a single predicted label for a subset of the dataset . Similarly , by using likelihood estimation over the augmented matrix , the generative process predicts “ reinforced labels ” . The details of the generative model implementation can be seen in [ Bach et al . ( 2017 ) ] . After the generative process , the reinforced labels are used to train the end classifier . The end classifier can still be the same supervised machine learning model . As the augmented matrix has more density compared to the labeling matrix , it may lead to a larger training set . As a simple example , a disagreement between two LFs for a data point can be eliminated by a new prediction for the abstain case of a third LF . As a result , more data points can be used to train the end model . The benefits of using the end classifier include further generalization for the data points that are still not labeled . Furthermore , the reinforced labels can fine-tune pretrained machine learning models .

Summary: This paper proposes an extension to the data programming / weak supervision method of Ratner et. al. 2016, in which heuristic functions called labeling functions (LF) are used to label training data.  In the proposed extension, called "reinforced labeling", points that are close to those labeled by a labeling function in feature space (as determined by a heuristic "gravitation" algorithm) are also labeled, thereby augmenting the set of programmatically labeled examples.