Year: 2021

Abstract:
Weak supervision (WS) is an alternative to the traditional supervised learning to address the need for ground truth. Data programming is a practical WS approach that allows programmatic labeling data samples using labeling functions (LFs) instead of hand-labeling each data point. However, the existing approach fails to fully exploit the domain knowledge encoded into LFs, especially when the LFs’ coverage is low. This is due to the common data programming pipeline that neglects to utilize data features during the generative process. This paper proposes a new approach called reinforced labeling (RL). Given an unlabeled dataset and a set of LFs, RL augments the LFs’ outputs to cases not covered by LFs based on similarities among samples. Thus, RL can lead to higher labeling coverage for training an end classifier. The experiments on several domains (classification of YouTube comments, wine quality, and weather prediction) result in considerable gains. The new approach produces significant performance improvement, leading up to +21 points in accuracy and +61 points in F1 scores compared to the state-ofthe-art data programming approach.

heading: 1 INTRODUCTION
text: Supervised machine learning has proven to be very powerful and effective for solving various classification problems. However, training fully-supervised models can be costly since many applications require large amounts of labeled data. Manually annotating each data point of a large dataset may take up to weeks or even months. Furthermore, only domain experts can label the data in highly specialized scenarios such as healthcare and industrial production. Thus, the costs of data labeling might become very high.
In the past few years, a new weak supervision (WS) approach, namely data programming [Ratner et al. (2016; 2017)], has been proposed to significantly reduce the time for dataset preparation. In this approach, a domain expert writes heuristic functions named labeling functions (LFs) instead of labeling each data point. Each function annotates a subset of the dataset with an accuracy expected to be better than a random prediction. Data programming has been successfully applied to various classification tasks. However, writing LFs might not always be trivial, for instance, when data points are huge vectors of numbers or when they are not intuitively understandable. Developers can quickly code a few simple functions, but having heuristics to cover many corner cases is still a burden. Further, simple heuristics might cover only a tiny portion of the unlabeled dataset (small coverage problem).
The existing data programming framework Snorkel [Ratner et al. (2016; 2017)] implements a machine learning pipeline as follows. The LFs are applied to the unlabeled data points and the outcomes of LFs produce a labeling matrix where each data point might be annotated by multiple, even conflicting, labels. A generative model processes the labeling matrix to make single label predictions for a subset of data points, based on the agreements and disagreements on the LF outputs for a given data point x(i) using techniques such as majority voting (MV) or minimizing the marginalized log-likelihood estimates. Later, the label predictions are used to train a supervised model that serves as the end classifier (discriminative model). This approach has two major limitations: 1) Coarse information about the dataset (only LF outputs) fed to the generative model, 2) Lack of generalization due to the sparsity of labeling matrix and relying only on end classifier to generalize. The current data programming does not take the data points’ data features into account during the generative process, even though they are available throughout the pipeline. It utilizes the data features of only the data
points with label predictions to train the end classifier. Various additional techniques are considered to complement the existing approach [Varma & Ré (2018); Chatterjee et al. (2020); Varma et al. (2019); Nashaat et al. (2018); Varma et al. (2016)] and improve the learning process. However, these approaches do not address this major problem in the design of existing data programming.
This article proposes a label augmentation approach for weak supervision that takes the data features and the LF outputs into account earlier in the generative process. The proposed approach utilizes the features for augmenting labels. The augmentation algorithm, namely reinforcement labeling (RL), checks similarities between data features to project existing LF outcomes to non-existing labels in the matrix (i.e., to unknown cases or abstains). Moreover, it uses a heuristic that considers unknown cases “gravitate” towards the known cases with LF output labels. In such a way, RL enables generalization early on and creates a “reinforced” label set to train an end classifier.
Label augmentation extends the data programming to new scenarios, such as when LFs have low coverage, domain experts can implement only a limited number of LFs, or LFs outcome result in a sparse labeling matrix. Label augmentation can provide satisfactory performances in these cases, although data programming was previously non-applicable. The proposed approach can reduce the time spent by the domain experts to train a classifier as they need to implement fewer LFs. One advantage compared to the existing complementary approaches is that RL does not require any additional effort for labeling data, annotating data, or implementing additional LFs. In other words, the label augmentation enhances classification without any further development burden or assumption of available labeled datasets (e.g., the so-called “gold data”). Furthermore, one can easily combine this approach with the existing solutions.
The RL method is implemented and tested compared to Snorkel (Sn) using different fully-supervised models as end classifiers. The experimental results span classification tasks from several domains (YouTube comments, white/red wine datasets, weather prediction). The new approach outperforms the existing model in terms of accuracy and F1 scores, having closer outcomes to the fully-supervised learning, thanks to the improved coverage that enables end classifier convergence.

heading: 2 METHOD DESCRIPTION

heading: 2.1 BACKGROUND ON DATA PROGRAMMING
text: In data programming [Bach et al. (2019); Ratner et al. (2017)], a set of LFs annotate a portion (subset) of the original unlabeled dataset X = {x(1), x(2), . . . , x(k)} with a total labeling coverage of γ ∗ |X|, where γ ∈ [0, 1]. Given a data point x(i), an LFj takes x(i) as input and annotates the input with a label. LFs are considered weak supervisors implemented by application developers, and they can programmatically annotate many data points at once, as opposed to hand-labeling data points one by one. On the other hand, LFs may have lower accuracies than ground truth for the data points.
For a binary classification task, an LF may return two classes or abstain from making a prediction. For simplicity, let us consider LFj returning 1 or 0 as the class labels and -1 (abstain) when it refrains
from class prediction. LFs’ outputs form a labeling matrix where rows represent the data point indices x(1), x(2), . . . , x(k), and columns represent the LF indices LF1, LF2, . . . , LFm. A generative model takes the labeling matrix as input, filters out the data points with no label (all LFs voted for abstains), and tries to predict a label for the remaining data points. An example of a generative model might be a majority voter (MV) based on LF outputs or by minimizing the negative log marginal likelihood [Ratner et al. (2017)] (likelihood over latent variables, i.e., LF outputs). Dependency structures between the LFs are learned as shown in Alg. 1 in [Bach et al. (2017)]. In both MV and marginal likelihood approaches, the generative model takes the labeling matrix as input and tries to make a label decision based on the agreements or disagreements of LFs. The generative model may fail to make a decision in certain cases, such as when equal numbers of LFs disagree on a data point. Lastly, the features of the weakly-labeled data points within X and the labels from the generative model are used to supervise an end classifier (discriminative) model. The design of the data programming model is agnostic to the end classifier (discriminative) model, so various supervised machine learning models can be candidates as the end model.

heading: 2.2 LABEL AUGMENTATION AND REINFORCED LABELS
text: In the described design of data programming, the two limitations mentioned above (in Sec. 1), namely the coarse information in the labeling matrix and sparsity, lead to failure for generalizing to new and unseen data points. Therefore, these limitations may lead to reduced performances. In such scenarios, the outputs of the existing generative model may not be satisfactory to train the end classifier. Therefore, the end classifier model may not converge or generalize well enough to cover different cases. Implementing many LFs that cover different cases is costly and not very straightforward in most scenarios. Although various additional techniques focus on the weak supervision problem [Varma & Ré (2018); Chatterjee et al. (2020); Varma et al. (2019); Nashaat et al. (2018); Varma et al. (2016)], they rather extend the existing pipeline with additional features. On the other hand, label augmentation targets these major limitations by eliminating the sparsity using data features. Thus, it can lead to higher accuracy.
Fig. 1 illustrates the new pipeline for label augmentation. The new generative process brings together the outputs of 〈LF1, LF2, . . . , LFm〉, and the data features f1, f2 . . . , fn of data points x(i) = {x(i)1 , x (i) 2 , . . . , x (i) n } in the unlabeled dataset early on. Different methods can utilize data features in the generative process for augmenting the labels in the labeling matrix. This label augmentation approach differs from the existing “data augmentation” approaches [Wang et al. (2019); Tran et al. (2017); Cubuk et al. (2019)] that create new (synthetic) data points, as the new goal is to project LF outcomes to the existing data points as opposed to creating new data points.
The outcome of the new generative process can be more representative of the data and weak supervisors than the outputs of the previously existing generative process due to additional coverage and accuracy gains without any additional LF implementation or data annotation. For instance, abstain values (-1s) from the LFs’ outputs representing the unknown cases (left side of the matrix in Fig. 1) can be predicted by the new generative process (as outlined in Sec. 2.3). The abstain values in the labeling matrix can be augmented with classification prediction values.
Data programming applies statistics only to data points that are already covered by LFs, resulting in a single predicted label for a subset of the dataset. Similarly, by using likelihood estimation over the
augmented matrix, the generative process predicts “reinforced labels”. The details of the generative model implementation can be seen in [Bach et al. (2017)]. After the generative process, the reinforced labels are used to train the end classifier. The end classifier can still be the same supervised machine learning model. As the augmented matrix has more density compared to the labeling matrix, it may lead to a larger training set. As a simple example, a disagreement between two LFs for a data point can be eliminated by a new prediction for the abstain case of a third LF. As a result, more data points can be used to train the end model. The benefits of using the end classifier include further generalization for the data points that are still not labeled. Furthermore, the reinforced labels can fine-tune pretrained machine learning models.

heading: 2.3 IMPLEMENTING REINFORCED LABELING
text: Let us describe the RL method for label augmentation in the generative process. The intuition behind this method is that a data point x(j), that a LF does not label, might have very similar features compared to another data point, such as a data point x(i) labeled by the same LF. x(j) might not be labeled due to the conditions or boundaries in the LF or missing a subset of the data features in its heuristic implementation. Reinforcing the labels means predicting a label for the previously unlabeled data points by certain LFs and therefore creating a denser version of the labeling matrix during the generative process. Furthermore, RL could lead to a higher coverage γ
′ ∗ |X| ≥ γ ∗ |X| for training the end classifier.
Fig. 2 illustrates the RL method. The method iterates over 〈LF1, LF2, . . . , LFm〉 and for each LFl on the left part of the matrix, it finds the data points x(j)s that LFl abstains from (-1s on the matrix). The abstain points are compared with the points that are labeled by LFl for their distances based on data features. The similarity between a point x(i) labeled by LFl point and the unlabeled x(j) is represented by the “effect function” Eff(x(i), LFl(x(i)), x(j)). As inputs, the effect function has the data features of the labeled x(i) = {x(i)1 , x (i) 2 , . . . , x (i) n }, output label LFl(x(i)) and the features of the unlabeled x(j) = {x(j)1 , x (j) 2 , . . . , x (j) n }. Based on the label, the effect may take a positive value (if LFl(x(i)) = 1) or a negative value (if LFl(x(i)) = 0). For any other data point that the LFl also abstains, it takes the value zero. These values can be aggregated as ∑k i=1Eff(x (i), LFl(x (i)), x(j)) for any LFl and where k = |X|. We propose a heuristic algorithm to implement RL based on gravitation. Fig. 3 illustrates the heuristic method considering a simple case of having only two data features x1, x2. In the gravitation-based RL method, for the abstain point x(j) that is not labeled by LFl, each other point that LFl labels (shown as colored particles in Fig. 3) is considered as a particle that attracts x(j) towards positive (LFl(x(i)) = 1) or negative (LFl(x(i)) = 0) aggregated effect. Fig. 3 shows positive or negative attractions by different-colored lines between the data points. The attraction is inversely proportional to the pairwise distance of every labeled point x(i) to x(j). For optimizing the calculations, we
consider a maximum distance threshold εd above which attraction does not take place. Hence, the effect function is defined as follows:
Eff ( x(i), LFl(x (i)), x(j) ) =
{ β
Distance(x(i),x(j)) α if Distance(x(i), x(j)) < εd 0 otherwise, (1)
where α and β : are constants for optional adjustment of the effect and distance relationship. As a distance function Distance(x(i), x(j)) RL can use different metrics based on the data types (e.g., sensor readings, text, image) such as Euclidean, Levenshtein or Mahalanobis distances. The effects of all labeled points are aggregated based on their positive or negative attractions. The decision of updating an abstain value depends upon a parameter called the aggregated effect threshold ε. x(j) is labeled with that class for the given LFl if the following condition holds:∣∣∣∣∣∣ |X|∑ i=1 Eff ( x(i), LFl(x (i)), x(j) )∣∣∣∣∣∣ > ε. (2)
ε adjusts the degree of reinforcement and the density of the resulting augmented matrix. In the evaluation section, we consider various empirical parameters and a heuristic for automatic ε configuration.

heading: 3 EXPERIMENTAL EVALUATION

heading: 3.1 BENCHMARK FRAMEWORK
text: The experimental study includes four metrics: Classification accuracy (# correct/# tested), precision, recall, and F1 score (F1). Particularly, higher accuracy and F1 score are important to understand the performance for the given classification task for non-biased and biased tasks, respectively. Other than these performance metrics, the experimental evaluation includes insights on the labeling itself (labeling metrics) such as: Number of LFs: The number of LFs used as weak supervisors; Labeled samples: The number of samples labeled for training the end models; LF coverage: The ratio of the number of data points labeled by LFs to the number of samples; LF overlap: The ratio of LFs’ overlapping outputs with each other to the number of samples; LF conflicts: The ratio of LFs’ conflicting outputs with other each other to the number of samples. The last three metrics represent the mean values, averaged based on the number of LFs.
We evaluate the RL approach with four datasets from different domains: YouTube comments [Alberto et al. (2015)], red and white wine quality [Cortez et al. (2009)], and Australia Weather (rain) datasets1. YouTube comments dataset consists of texts that may be legit user comments or spam. This text-based dataset is used for benchmarking various data programming approaches [Chen et al. (2020); Evensen et al. (2020); Ren et al. (2020); Karamanolakis et al. (2021); Sedova et al. (2021); Awasthi et al. (2020)] and also as Snorkel’s tutorial for LFs. The YouTube dataset is largely unlabeled except for a small testing dataset. Only the text comment is used as a feature for the YouTube dataset, whereas we removed the others (such as user ID and timestamp). On the other hand, the end models get a sparse matrix of token counts computed with Scikit-learn CountVectorizer2. To classify the wine quality, there exist 12 real number features (e.g., acidity, residual sugar) for the two wine datasets. For training and testing, a wine is considered good (labeled with 1) when the wine quality feature is more than 5 out of 10; otherwise considered a bad wine (0). The Australia Weather dataset is widely used for the next-day rain predictions. It also consists of 62 features based on daily weather observations spanning ten years period. The wine and Australia Weather datasets are fully labeled with ground truth. For all datasets Euclidean distance metric computes distances between data point pairs. For the YouTube dataset, Euclidean distance is applied to the one-hot encoding of the tokenized texts3.
Snorkel (Sn) and fully-supervised learning (Sup) are the two main approaches for comparison. In addition to those, the majority voting (MV) approach is tested as the simple generative model. For
1https://www.kaggle.com/jsphyg/weather-dataset-rattle-package 2https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html 3https://www.nltk.org/api/nltk.tokenize.html
each data point, MV labels the point based on the majority of the LF outputs (i.e., 0 or 1) from the LFs that do not abstain. We also experiment with MV combined with RL (MVRL). For all results, the existing data programming approaches Sn and MV use the same set of LFs as RL. The LFs are listed in Supp. material A.5. We implement Sn and MV as well as the RL approach using the Snorkel library (version 0.9.5) with the existing features in the tutorial. The framework uses absolute latent labels for training end model (0,1), RL adapts the same scheme for the generative model and training end classifiers. Supervised learning leverages ground truth data (30% of the dataset) for the training. For the experimental scenario, supervised learning is considered the optimal result of machine learning, whereas the other two models are based on only programmatic labels. For supervised learning, the small testing dataset of YouTube comments is used for both training and testing due to lacking ground truth in the training set.
For the end classifier, different machine learning models are used for testing purposes. The models include two logistic regression models: the first one, namely “logit”, is the model that Snorkel uses by default (inverse of regularization strength C := 1000 and liblinear solver); the second one, “LogR”, uses lbfgs solver for optimization. In addition, we test the random forest (RF), naive Bayes (NaivB), decision tree (DT), k-nearest neighbor (knn), support vector machine (svm), and multi-layer perceptron (mlp) end models. In each experiment, all approaches use the same end classifier. Each experiment consists of 5 runs, and the results are averaged. We do not observe any notable difference between experiment runs. Our assumption is that the tested end models learning behavior is rather deterministic as they use the same the data points for their trainings. There are no additional hyperparameters used other than the stated in this section.

heading: 3.2 EXPERIMENTAL RESULTS
text: Benefits of RL The first set of results shows the advantage of using RL compared to the existing data programming approach Sn in terms of accuracy and F1 score gains. Table 1 includes results from the four datasets in terms of the four performance metrics. In all datasets, we observe substantial gains in accuracy and F1 scores compared to the benchmark Sn approach. Moreover, RL performance is closer to fully-supervised learning, although it does not use any ground truth labels, even when LFs are relatively few. For the YouTube dataset, we experience that although the Snorkel and RL approaches have the same set of LFs, RL provides up to 64% F1, whereas Sn provides less than 3% F1. In RL, the end model can converge thanks to the additional coverage by the reinforced labels. Table 2 reports the number of computed labels after the application of the generative model with reinforcement (RL) and without it (Snorkel). The gains for accuracy and F1 require no additional human effort or involvement. Only two parameters ε and εd configure the RL (see Table 2 for Table 1 experiments settings). Later in this section, we present how these parameters could be configured automatically for any given scenario based on labeling metrics (i.e., LF coverage, LF overlaps, and LF conflicts). Furthermore, our approach consistently outperforms Sn for training any of the tested end models. An extended table in Supp. material A.1 reports the experimental results varying end
models for each dataset. Lastly, we observe that εd affects the performance as it adjusts the trade-off between augmenting similar labels and the bias (noise) of the augmentation.
Auto-adjustment of the RL method We observe that ε is dependent on the dataset and the set of LFs. A simple heuristic method to choose ε is as follows: First calculate the distribution of aggregated effects of unlabeled data point for each LF (through a boxplot as in Fig. 4), then set ε to have boundaries bneg and bpos far enough from the quartiles Q1 and Q3. When using 2 such boundaries are symmetric to aggregate effect equal to 0 and they are chosen by the data scientist as we did for the results shown in Tables 1. Another way to calculate such boundaries is to have them symmetrical to the aggregated effect distribution. We can achieve this with the formula bneg = Q1− IQR ∗ hIQR (and similarly for bpos) where InterQuartileRange(IQR) = Q3−Q1 and hIQR is a parameter, namely the IQR factor. When hIQR = 1.5, the gravitation method labels only outliers of the aggregated effect distribution. At this point, there is no need to set the ε parameter and the boundaries automatically adapts to the aggregated effects that depends on the features range, distance metrics and the sparsity of the initial labeling matrix. However, varying hIQR (as tested for the range [0, 2]) affects the performances. Thus, the problem of finding an optimal hIQR still persists.
Fig. 5a-top row shows the comparison with Snorkel generative model, while Fig. 5a-middle row shows the comparison with MV. As in Fig. 5a-top, hIQR ← 0.5 cause a substantial F1 gain for fewer LFs (e.g., 5 to 8 LFs), whereas it may cause detrimental bias for the case of a higher number of LFs as the sum of LFs’ coverage increases. In this case, the reinforcement may not need to be applied as extensively as when the sum of LFs’ coverage is low. In the latter scenario, using hIQR ← 1.0 would be the most conservative approach (no reinforcement) that makes sure to add no additional noise. Similar behavior is observed with majority voter as label aggregation algorithm (Fig. 5a-middle row).
Fig. 5a-bottom row shows the effect of RL with different hIQR in terms of labeled samples, LFs’ coverage, mean overlaps, and mean conflicts. The smaller is IQR factor, the higher the values for those metrics are because the boundaries are closer to the IQR. One can infer that for a higher number of LFs, a smaller IQR factor results in excessive noise confusing the generative model (Sn or MV) and degrading the end model. Thus, we define a simple heuristic to automatically configure the gravitation method (shown in Alg. 2 in Supp. material A.3) through calculating hIQR by linking it to the LF statistics for the given dataset. The below formula uses these statistics and an empirical constant ξ.
hIQR = ξ ∗ ∑ LFl coverage ∗ ∑ LFl overlaps ∗ ∑ LFl conflicts (3)
We set ξ = 0.35 through empirical experiments. By this approach, the higher density the labeling matrix has (e.g., more LFs, higher LF coverage), the fewer the abstain labels updated by RL are.
We test the auto-adjustment approach and show the results in Fig. 5b for both Sn (top row) and MV (middle row). We have also set the second parameter εd virtually to infinite, relying on no hyper-parameter. When fewer LFs exist (e.g., 5, 6, or 7), RL provides significant performance gains, especially for F1 scores. On the other hand, when more LFs exist, RL adjusts itself by becoming more conservative on the reinforcement. Therefore, auto-adjustment preserves the good performances when having sufficient LFs, leading to a larger area under the curve for both Sn and MV. Our proposed RL approach, together with this heuristic, does not require any additional parameter setting or human effort by automatically adjusting its IQR factor. However, additional studies are due for more enhanced methods of auto-adjustment.
Fig. 5b shows the effect of the automatic IQR adaptation in terms of labeled samples and LF metrics. As expected, when fewer LFs exist, RL provides gains for the number of labeled data points and mean LF coverage, overlaps, and conflicts. Furthermore, it adapts itself and gradually provides lower numbers of additional labels for higher numbers of LFs, thus reducing the noise.
Varying distance metrics Our gravitation method highly rely on similarities between data points. In the experiments described above, we always use the Euclidean distance for the real number features. Therefore, we investigate how different distance metrics affect the performances. Table 3 shows the results on the white wine dataset with RF as the end classifier and the previous heuristics to calculate the aggregated effect boundaries with hIQR ← 1.4. RL consistently provides improved performance even when the distance metric changes.
Limitations As label augmentation relies on the data features, using a poor embedding for distance computation might lead to detrimental noise in the augmented labeling matrix. In addition, a more advanced augmentation adjustment would avoid performance degradation due to the tradeoff between the coverage and noise.

heading: 4 RELATED WORK
text: Data programming [Ratner et al. (2016)] enables programmatically labeling data and training using WS sources. In particular, the Snorkel framework [Ratner et al. (2017); Bach et al. (2019)] provides an interface for data programming where users can write LFs and apply them to generate a training dataset for their end models. The generation of the training dataset relies on the generative model, and several studies focus on this aspect [Ratner et al. (2017); Bach et al. (2017); Varma et al. (2017; 2019)].
Various recent works focus on extending the existing data programming approach. The extensions include multi-task classification [Ratner et al. (2019)], using small labeled gold data for augmenting WS [Varma & Ré (2018)], learning tasks (or sub-tasks) in slices of dataset [Chen et al. (2019)], user guidance for LF precision [Chatterjee et al. (2020)], making the training process faster [Fu et al. (2020)], generative adversarial data programming [Pal & Balasubramanian (2020; 2018)], user supervision for LF error estimates [Arachie & Huang (2019)], learning LF dependency structures [Varma et al. (2019)], user annotation of LFs [Boecking et al. (2021)], language description of LFs [Hancock et al. (2018)], active learning Nashaat et al. (2018), and so on. Varma et al. (2016) aims to learn common features by using a “difference model” and feeding these features back to generative model. Mallinar et al. (2019) takes advantage of the natural language processing query engine to expand gold labels and generate a label matrix as input for the generative model. Zhou et al. (2020) adopts a soft LFs matcher approach based on the distances between LFs’ conditions and data points.
Chen et al. (2020) uses pre-trained machine learning models to estimate distances for natural language processing. The last two studies focus on the semantic similarity of texts to improve the labeling.
Although the studies mentioned above may improve the existing generative model (e.g., through additional human supervision), they do not focus on the problem of LF abstraction with coarse information. Solving this problem would improve the validity of data programming in various scenarios especially since the human supervision is limited in its nature. In this paper, we identify this problem and propose the reinforced labeling that takes the data features into account early on in the generative process. Using this approach, one can leverage the data features and augment the matrix for further generalization and producing satisfactory performance without additional human supervision.
Weak supervision approaches outside the context of data programming consider learning from a set of rules, experts, or workers as in crowdsourcing. Platanios et al. (2017) infer accuracy from multiple classifier outputs based on the confidences. Safranchik et al. (2020) study the usage of Hidden Markov Models for tagging data sequences. Dehghani et al. (2018) train deep NNs using weakly-labeled data. Their approach is semi-supervised, where a teacher network based on rules adjusts the predictions of a student network trained iteratively by given data samples. In another study, Dehghani et al. (2017) propose WS to train neural ranking models in natural language processing and computer vision tasks.
Takeoka et al. (2020) consider leveraging unsure responses of human annotators as WS sources to improve the traditional supervised approach. Kulkarni et al. (2018) study labeling based on consensus and interactive learning based on active labeling for multi-label image classification tasks. Khetan et al. (2018) propose an expectation-maximization algorithm for learning workers’ quality, where each worker represents a WS source for image classification tasks. Qian et al. (2020) propose WS with active learning for learning structured representations of entity names. Guan et al. (2018) learn individual’s weights for predicting a weighted sum of noisy labelers (experts). Das et al. (2020) propose a domain-agnostic approach to replace the needs of LFs that apply affinity functions to relate samples with each other. This approach uses a small gold dataset with probabilistic distributions to infer probabilistic labels.
Similar approaches other than WS include domain-specific machine learning applications such as ontology matching. Doan et al. (2004) use a relaxation method to label a node into a graph dataset by analyzing features of the node’s neighborhood in the graph. The relaxation process is based on constraint and knowledge that leads to the final labeling. In a similar approach, Li & Srikumar (2019) describe a methodology framework to augment labels guided by external knowledge. In both approaches, label augmentation is in the final phase. These approaches do not involve any generative process. Lastly, literature related to semi-supervised learning (e.g., [Zhu et al. (2003)]) or other hybrid approaches (e.g., [Awasthi et al. (2020)]) consider using a mix of clean and noisy labels, whereas this paper focuses on using only the labels from LFs and improve the validity of the existing approach.

heading: 5 CONCLUSION
text: This paper proposes a novel method for label augmentation in weak supervision. In the new machine learning pipeline, the proposed RL method in the generative process leverages existing LF outputs and data features to augment the weakly-supervised labels. The experimental evaluation shows the benefits of RL for four classification tasks compared to the existing data programming approach in terms of substantial accuracy and F1 gains. Furthermore, the new method enables the convergence of the end classifier even when there exist few LFs. We consider applying RL for matching problems (e.g., entity matching) and active learning as future work. We consider RL as an initial approach for the identified limitation of the generative process, whereas the pipeline opens up the possibility for more advanced (e.g., machine learning) models to leverage data features during the generative process.

heading: 6 REPRODUCIBILITY STATEMENT
text: The results shown in the paper and the appendix can be reproduced as described below.
All of the four datasets are openly available and popularly used by many machine learning researchers. The open-source Snorkel framework tutorial [Snorkel (2021)] (version 0.9.5) can be run as it is, as the existing data programming approach. In the experiments the default features in these tutorial is used without any change, whereas we implemented new LFs for the new datasets and include all of these new LFs in Appendix A.5. The algorithm implemented for reinforcement of labels and IQR calculation are included in detail in Appendix 1 as Alg. 1 and Alg. 2. We plan to make the codes for these two algorithms open-source after our internal approval process.
A person skilled in programming (e.g., Python) can implement the two algorithms in a short time given the pseudocodes. The only two parameters (ε and εd in Table 1) used in the four experiments are listed in Table 2 and four distance metrics, instead of these two parameters, only one parameter value hIQR := 1.4 (for Alg. 2) is used as shown in Table 3.

heading: A APPENDIX
text: A.1 REINFORCED LABELING RESULTS ON DIFFERENT DATASETS VARYING END MODELS
Table 4 extends Table 1 by reporting the results of 8 end models and 4 datasets. The hyperparameter configurations of ε and εd are listed in Table 2. For all experiments and end models RL outperforms Snorkel and maintains closer performance to the fully-supervised machine learning.
A.2 SYMBOLS AND NOTATIONS IN THE PAPER
Table 5 lists and describes the frequently used symbols throughout the paper. Some listed parameters are not normalized (e.g., aggregated effects) or adjusted for simplicity, whereas they can be easily normalized to the range [0, 1], and so ε ∈ [0, 1] based on the observed values, without causing any change in the outcomes.
A.3 REINFORCED LABELING PSEUDOCODE FOR LABEL AUGMENTATION
Algorithm 1: REINFORCED LABELING ALGORITHM Input: LFs 〈LF1, LF2, . . . LFm〉 and unlabeled data points X = {x(1), x(2), . . . , x(k)}, where
x(i) has features 〈x(i)1 , x (i) 2 , . . . x (i) n 〉. Gravity parameters α, β. Distance threshold εd.
IQR adjusting parameter ξ Output: Label for a subset γ′ ∗ |X| of the data points (augmented labels)
Y = {y(1), y(i), . . . , y(k)}, where y(i) ∈ [0, 1] ∪ {−1} 1 Distances← ∅ 2 Effects← ∅ 3 Labels← ∅ 4 for i = 1, 2, . . . , |X| do 5 for l = 1, 2, . . . ,m do 6 Labels[i][l]← LFl(x(i)) 7 Effects[i][l]← 0 8 end 9 end
10 LFs_stats← CalculateLFsStats(X,Labels) 11 for l = 1, 2, . . . ,m do 12 for i = 1, 2, . . . , |X| do 13 if Labels[i][l] == −1 then 14 for j = 1, 2, . . . |X| do 15 if i 6= j & Labels[j][l] 6= −1 then 16 Distance←∞ 17 if Distances[i][j] is undefined then 18 Distance← FindDistanceBetweenDataPoints(x(i), x(j)) 19 Distances[i][j]← Distance 20 Distances[j][i]← Distance 21 if Distances[i][j] ≤ εd then 22 if Labels[j][l] = 1 then 23 Effects[i][l]← Effects[i][l] + β(Distances[i][j])α 24 else 25 Effects[i][l]← Effects[i][l]− β(Distances[i][j])α 26 end 27 end 28 end 29 end 30 boundary_min, boundary_max← CalculateIQRBoundaries(LF_stats, Effects, ξ) 31 for l = 1, 2, . . . ,m do 32 for i = 1, 2, . . . , |X| do 33 if Labels[i][l] == −1 then 34 if Effects[i][l] < boundary_min then 35 Labels[i][l]← 0 36 if Effects[i][l] > boundary_max then 37 Labels[i][l]← 1 38 end 39 end 40 return Labels
Alg. 1 shows the pseudocode of a similarity-based heuristic algorithm for reinforced labeling. Given m LFs and the unlabeled dataset X of size |X|, the algorithm outputs the augmented labeling matrix. The listed gravity parameters α := 1, β := 1 are constants used for all datasets in the experimental study. The distance threshold εd is an optional parameter to optimize the computation or to remove outliers, whereas it is not always used in the experimental study (εd is set to a high number). Labels, Effects are 2D arrays, where rows represent the index i of the data points (x(i))
Algorithm 2: CalculateIQRBoundaries(LF _stats, Effects, ξ) Input: LFs_stats = 〈LFs_coverage, LFs_overlaps, LFs_conflicts〉 where
LFs_coverage = 〈LF1_coverage, LF2_coverage, . . . , LFm_coverage〉 and similarly LFs_overlaps and LFs_overlaps; Effects: Array of aggregated effects, ξ ← 0.35
Output: boundary_min, boundary_max 1 sumcoverage ← 0 2 sumoverlaps ← 0 3 sumconflicts ← 0 4 for l = 1, 2, . . . ,m do 5 sumcoverage ← sumcoverage + LFl_coverage 6 sumoverlaps ← sumoverlaps + LFl_overlaps 7 sumconflicts ← sumconflicts + LFl_conflicts 8 end 9 hiqr ← ξ ∗ sumcoverage ∗ sumoverlaps ∗ sumconflicts
10 q1 ← calculatePercentile(Effects, 25) 11 q3 ← calculatePercentile(Effects, 75) 12 iqr ← q3 − q1 13 boundary_min← q1 − iqr ∗ hiqr 14 boundary_max← q3 + iqr ∗ hiqr 15 return boundary_min, boundary_max
and columns represent the index l of the LFs (LFl). Distances is a 2D array representing the distance between any two data points. For instance, Distances[i][j] represents the distance between x(i) and x(j).
The −1 values that represent abstains of LFs are updated based on their similarity given by the FindDistanceBetweeDataPoints function. This function can be implemented using various distance metrics such as Euclidean, Cosine, or Mahalanobis distances. Lastly, the updated Labels array represents the augmented labeling matrix that can be given to a generative process such as Snorkel’s generative model.
Alg. 2 shows the pseudocode of the CalculateIQRBoundaries used for the RL implementation. The heuristic in Alg. 2 leverages three LF statistics, that are LF coverage, overlaps, and conflicts, to calculate the boundaries that are effectively the aggregated effect threshold of RL.
A.4 ADDITIONAL EXPERIMENTAL INSIGHTS
In addition to the experimental study, we implement and test a hybrid approach of leveraging labels of LFs as well as strong supervision through a gold dataset. We call this approach the generative neural network for data programming (GNN). In GNN, the label outputs of LFs and data features are fed to a simple neural network (NN) along with the labels. The NN model contains two hidden layers (# nodes 12, 8) with ReLU activation function, and it uses Adam optimizer. The output layer has the sigmoid activation function. The NN model is used in different stages of the pipeline. First, GNN replaces the generative part using labels of Snorkel (Sn+GNN+<endmodel>) or RL (RL+GNN+<endmodel>). Then, the outputs of GNN are fed to an end classifier as usual. Second, the GNN itself serves as the end classifier as well as the generative model (Sn+GNN or RL+GNN). We applied the GNN model to both red wine and white wine datasets as these datasets have the available ground truth data to create gold datasets.
Fig. 6 shows results of the red and white wine datasets in more detail, including the hybrid GNN approach using RF and NaivB end models, respectively. The results of the following 7 approaches are listed in order: Sn+<endmodel>, RL+<endmodel>, Sup+<endmodel>, Sn+GNN+<endmodel>, RL+GNN+<endmodel, Sn+GNN, RL+GNN>. We observe that RL+RF outperforms the Sn+RF benchmark for the white wine dataset by +13 points accuracy and +34 points F1 and even provides better results than Sup (RF). For the red wine dataset, RL+NaivB outperforms Sn+NaivB by +10 points in accuracy and +7 points in F1 score. Moreover, although approaches such as Sup or GNN leverage ground truth labels in their training, outcomes of RL are competitive for the red wine dataset, whereas RL outperforms Sup (NaivB), Sn+GNN+NaivB, and RL+GNN+NaivB for the white wine dataset (see Fig. 6-right).
Fig. 7 shows the bar graph for the results of the YouTube comments and Australia Rain datasets (also listed in Table 1) in terms of the four metrics: Accuracy, precision, recall, and F1. The results of the following approaches are listed in order: Sn+<endmodel>, RL+<endmodel>, Sup+<endmodel>. We observe that RL+svm outperforms the Sn+svm benchmark for the YouTube dataset by +21 points
accuracy and +61 points F1. For the Australia Rain dataset, RL+RF outperforms Sn+RF by +5 points in accuracy and +34 points in F1 score.
A.5 LABELING FUNCTIONS
We use the Snorkel library to implement LFs and encode domain knowledge programmatically. The LFs are implemented using the interactive and user-friendly features of the Snorkel framework, such as providing LF statistics and allowing high-level definitions of LFs.
As in almost all the previous data programming studies, we do not follow a certain scheme or strict guidance on implementing LFs but rather rely on the best effort based on some understanding of the datasets by visualizing the features and interactively checking the LF coverages, overlaps, and conflicts. In general, the assumption is that the developers would make the best effort to write LFs.
The Australia Weather dataset is used to build a model that predicts for any given day if it will rain the day after4. LFs in Listing 1 use weather data features to label data points as GOOD (1) or BAD (0) based on the features such as Humidity, Rain Today, Temperature at 9am, Humidity at 9am, and pressure at 3pm.
1 from snorkel.labeling import labeling_function 2 ABSTAIN = -1 3 NO_RAIN = 0 4 RAIN = 1 5 6 @labeling_function() 7 def check_humidity3pm(x): 8 if x.Humidity3pm is None: 9 return ABSTAIN
10 elif x.Humidity3pm>0.75: 11 return RAIN 12 elif x.Humidity3pm<0.15: 13 return NO_RAIN 14 else: 15 return ABSTAIN 16 17 @labeling_function() 18 def check_rain_today(x): 19 if x.RainToday is None: 20 return ABSTAIN 21 elif x.RainToday==1: 22 return RAIN 23 else: 24 return ABSTAIN 25 26 @labeling_function() 27 def check_temp9am(x): 28 if x.Temp9am is None: 29 return ABSTAIN 30 elif x.Temp9am>0.60: 31 return RAIN 32 else: 33 return ABSTAIN 34 35 @labeling_function() 36 def check_rainfall(x): 37 if x.Rainfall is None: 38 return ABSTAIN 39 elif x.Rainfall>0.60: 40 return RAIN 41 else: 42 return ABSTAIN 43 44 @labeling_function() 45 def check_humidity9am(x): 46 if x.Humidity9am is None: 47 return ABSTAIN
4https://www.kaggle.com/jsphyg/weather-dataset-rattle-package
48 elif x.Humidity9am>0.90: 49 return NO_RAIN 50 elif x.Humidity9am<0.20: 51 return RAIN 52 else: 53 return ABSTAIN 54 55 @labeling_function() 56 def check_pressure3pm(x): 57 if x.Pressure3pm is None: 58 return ABSTAIN 59 elif x.Pressure3pm<0.05: 60 return RAIN 61 elif x.Pressure3pm>0.70: 62 return NO_RAIN 63 else: 64 return ABSTAIN
Listing 1: Australia rain labeling functions
The wine datasets have various data features such as alcohol, sulfates, citric acid levels, etc. Listing 2 and 3 include the LFs implemented for wine quality classification for the red wine and white wine datasets, respectively. LFs labels data points as GOOD (1) or BAD (0) quality wine.
1 from snorkel.labeling import labeling_function 2 ABSTAIN = -1 3 BAD = 0 4 GOOD = 1 5 6 @labeling_function() 7 def check_alcohol(x): 8 if x.alcohol is None: 9 return ABSTAIN
10 elif x.alcohol>0.75: 11 return GOOD 12 elif x.alcohol<0.15: 13 return BAD 14 else: 15 return ABSTAIN 16 17 @labeling_function() 18 def check_sulphate(x): 19 if x.sulphates is None: 20 return ABSTAIN 21 elif x.sulphates>0.3: 22 return GOOD 23 else: 24 return ABSTAIN 25 26 @labeling_function() 27 def check_citric(x): 28 if x.acidity_citric is None: 29 return ABSTAIN 30 elif x.acidity_citric>0.7: 31 return GOOD 32 else: 33 return ABSTAIN
Listing 2: Red wine labeling functions
1 from snorkel.labeling import labeling_function 2 ABSTAIN = -1 3 BAD = 0 4 GOOD = 1 5
6 @labeling_function() 7 def check_alcohol(x): 8 if x.alcohol is None: 9 return ABSTAIN
10 elif x.alcohol>0.75: 11 return GOOD 12 elif x.alcohol<0.15: 13 return BAD 14 else: 15 return ABSTAIN 16 17 @labeling_function() 18 def check_sulphate(x): 19 if x.sulphates is None: 20 return ABSTAIN 21 elif x.sulphates>0.3: 22 return GOOD 23 else: 24 return ABSTAIN 25 26 @labeling_function() 27 def check_citric(x): 28 if x.acidity_citric is None: 29 return ABSTAIN 30 elif x.acidity_citric>0.7: 31 return GOOD 32 else: 33 return ABSTAIN
Listing 3: White wine labeling functions
For the YouTube dataset, we implemented two LFs that search for the exact string "check out" or "check" in the text. Other than these two additional LFs, the LFs in the Snorkel tutorial [Snorkel (2021)] named “textblob_subjectivity”, “keyword_subscribe”, “has_person_nlp” are used in the experiments of Table 1. The rest of LFs available from the Snorkel tutorial are used for the experiments in Fig. 5.
1 from snorkel.labeling import labeling_function 2 ABSTAIN = -1 3 HAM = 0 4 SPAM = 1 5 6 @labeling_function() 7 def check(x): 8 return SPAM if "check" in x.text.lower() else ABSTAIN 9
10 @labeling_function() 11 def check_out(x): 12 return SPAM if "check out" in x.text.lower() else ABSTAIN
Listing 4: YouTube labeling functions

Review summaries:
This paper proposes an extension to the data programming / weak supervision method of Ratner et. al. 2016, in which heuristic functions called labeling functions (LF) are used to label training data.  In the proposed extension, called "reinforced labeling", points that are close to those labeled by a labeling function in feature space (as determined by a heuristic "gravitation" algorithm) are also labeled, thereby augmenting the set of programmatically labeled examples.

This paper describes a method to improve the coverage of labeling functions in weak supervision. Labeling functions in weak supervision can abstain if they do not vote on a particular point. The method replaces abstentions with a gravitation-based method based on the other points that the labeling function has voted on. The method uses similarities between features of points for the gravitation-based method. The authors claim performance benefits over Snorkel.

This paper tackles the problem of designing data programming, which is a practical approach in weak supervision. The authors state that prior effects neglect to utilize data features during the generative process, and therefore have suboptimal performance. Based on this, this paper proposes to exploit the sample similarities to augment the outputs of labeling functions. Empirical evaluations are provided to verify the effectiveness of the proposed method.