Original: ReGAL: Rule-Generative Active Learning for Model-in-the-Loop Data Programming
Revised: ReGAL: Rule-Generative Active Learning for Model-in-the-Loop Weak Supervision

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: One of the main bottlenecks to extending deep learning systems to new domains isthe prohibitive cost of acquiring sufﬁcient training labels.
Revised: One of the main bottlenecks to extending deep learning systems to new domains is the prohibitive cost of acquiring sufﬁcient training labels.

Original: While many previousworks have sought to alleviate this problem with weak supervision and data programming, rule and label noise prevent them from approaching fully-supervisedperformance.
Revised: While many previous works have sought to alleviate this problem with weak supervision and data programming, rule and label noise prevent them from approaching fully-supervised performance.

Original: This work-in-progress provides a principled, AI-guided approach toimprove rule-based and weakly supervised text classiﬁcation by performing activelearning not on individual data instances, but on entire labeling functions.
Revised: This work-in-progress provides a principled, AI-guided approach to improve rule-based and weakly supervised text classiﬁcation by performing active learning not on individual data instances, but on entire labeling functions.

Original: Weargue that such a framework can guide users and subject matter experts to selectlabeling rules that expand label function coverage without sacriﬁcing clarity.
Revised: We argue that such a framework can guide users and subject matter experts to select labeling rules that expand label function coverage without sacriﬁcing clarity.

Original: Ourexperiments show that our framework, ReGAL, is able to generate coherent labeling rules while simultaneously obtaining state-of-the-art performance in weaklysupervised text classiﬁcation.
Revised: Our experiments show that our framework, ReGAL, is able to generate coherent labeling rules while simultaneously obtaining state-of-the-art performance in weakly supervised text classiﬁcation.

Original:  Introduction
Revised:  Introduction

Original: One of the primary criticisms that has been leveled against deep learning is the high volume of dataneeded to produce models that generalize well and avoid overﬁtting.
Revised: One of the primary criticisms that has been leveled against deep learning is the high volume of data needed to produce models that generalize well and avoid overﬁtting.

Original: Numerous approaches have beenproposed to solve this problem, including transfer learning [1], active learning [2, 3], bootstrapping[4], distant supervision [5], rule-based supervision [6, 7], and crowd-sourced labeling [8].
Revised: Numerous approaches have been proposed to solve this problem, including transfer learning [1], active learning [2, 3], bootstrapping [4], distant supervision [5], rule-based supervision [6, 7] and crowd-sourced labeling [8].

Original: While this task is present across all domains of natural language processing (NLP), it is signiﬁcantlymore difﬁcult in the highly technical and low resource domains where labels must be curated bysubject matter experts (SMEs).
Revised: While this task is present across all domains of natural language processing (NLP), it is signiﬁcantly more difﬁcult in the highly technical and low resource domains where labels must be curated by subject matter experts (SMEs).

Original: Low-resource languages often have a very small pool of potentialannotators while technical domains such as scholarly articles contain such a wide breadth of information in literature means that a single SME cannot annotate a single corpus by himself or herself.
Revised: Low-resource languages often have a very small pool of potential annotators while technical domains such as scholarly articles contain such a wide breadth of information in literature means that a single SME cannot annotate a single corpus by himself or herself.

Original: These factors combined with the high cost of SME expertise motivates the development of systemsthat maximize both the speed and information content of annotation, thus allowing annotators toimpart more valuable data in shorter time.
Revised: These factors, combined with the high cost of SME expertise motivates the development of systems that maximize both the speed and information content of annotation, thus allowing annotators to impart more valuable data in shorter time.

Original: In this work, we develop a model called ReGAL for Rule-Generative Active Learning whichaccelerates data labeling by interactively soliciting human feedback on labeling functions insteadof individual data points.
Revised: In this work, we develop a model called ReGAL for Rule-Generative Active Learning which accelerates data labeling by interactively soliciting human feedback on labeling functions instead of individual data points.

Original: This framework, depicted in Figure 1 allows our active learner to seekfeedback on model weaknesses while simultaneously labeling large swaths of examples.
Revised: This framework, depicted in Figure 1 allows our active learner to seek feedback on model weaknesses while simultaneously labeling large swaths of examples.

Original:  Preliminaries 
Revised:  Preliminaries 2.1 Problem Formulation

Original: 2.1 Problem Formulation

Original: We assume that we are given a set of documents D
Revised: We assume that we are given a set of documents D

Original: = { d 1 , d 2 , . . .
Revised: = { d 1 , d 2 , . . .

Original: , d | D | } , each of whichis has a(possibly unknown) classiﬁcation label c i ∈ C .
Revised: , d | D | } , each of which has a (possibly unknown) classiﬁcation label c i ∈ C .

Original: Each document d i =
Revised: Each document d i =

Original: [ v i, 1 , v i, 2 , . .
Revised: [ v i, 1 , v i, 2 , . .

Original: . , v i,T ] represents a sequence of tokens from the vocabulary V , where tokens drawn from V could be words, subwords,characters, etc.
Revised: . , v i,T ] represents a sequence of tokens from the vocabulary V , where tokens drawn from V could be words, subwords, characters, etc.

Original: We assume that we do not have access to ground-truth labels for the documents in the trainingset but that we have a small number of heuristic labeling functions (LFs; i.e. labeling rules)
Revised: We assume that we do not have access to ground-truth labels for the documents in the training set but that we have a small number of heuristic labeling functions (LFs; i.e. labeling rules)

Original: R ={ r 1 , r 2 , . .
Revised: R = { r 1 , r 2 , . .

Original: , r l } , where each r j : D → C ∪ { c abstain } is a function that maps documents to a classlabel in C or abstains from labeling.
Revised: , r l } , where each r j : D → C ∪ { c abstain } is a function that maps documents to a class label in C or abstains from labeling.

Original: This set of LFs induces a vector of noisy labels for eachdocument, denoted (cid:96) i =
Revised: This set of LFs induces a vector of noisy labels for each document, denoted (cid:96) i =

Original: [ r 1 ( d i ) , r 2 ( d i ) , . . .
Revised: [ r 1 ( d i ) , r 2 ( d i ) , . . .

Original: , r l ( d i )] T .
Revised: , r l ( d i )] T .

Original: Because LFs act as rule-based labelers, wefreely interchange the terms “labeling function” and “rule” throughout the paper.
Revised: Because LFs act as rule-based labelers, we freely interchange the terms “labeling function” and “rule” throughout the paper.

Original: 2.2 Challenges
Revised: 2.2 Challenges

Original: Weakly supervised text classiﬁcation presents three main challenges: label noise and label incompleteness , and annotator effort .
Revised: Weakly supervised text classiﬁcation presents three main challenges: label noise and label incompleteness , and annotator effort .

Original: Label Noise Label noise is the problem of labeling functions generating incorrect labels forparticular data instances.
Revised: Label Noise Label noise is the problem of labeling functions generating incorrect labels for particular data instances.

Original: This problem generally occurs when a  labeling function is speciﬁed is toogeneral and thus is mislabeled into the wrong class.
Revised: This problem generally occurs when a speciﬁed labeling function is too general and thus mislabels into the wrong class.

Original: The diversity of language combined presentsan extremely large space of possible misapplications of a single labeling function, and enumeratingthem can be prohibitively expensive.
Revised: The diversity of language presents an extremely large space of possible misapplications for a single labeling function, and enumerating them can be prohibitively expensive.

Original: A number of recent works have sought to address this problem via numerous means, includinggenerative label denoising [6, 9], self-training on synthetic examples generated with latent variablemodels [10], using a neural network to identify improper applications of labeling functions usinglabeled rule exemplars [11], and active learning on instances with conﬂicting labels [12].
Revised: A number of recent works have sought to address this problem via numerous means, including generative label denoising [6, 9], self-training on synthetic examples generated with latent variable models [10], using a neural network to identify improper applications of labeling functions using labeled rule exemplars [11], and active learning on instances with conﬂicting labels [12].

Original: Whilesome of these methods have shown promising results, there is a distinct need for better methodsof automatically solving labeling functionconﬂicts conﬂicts.
Revised: While some of these methods have shown promising results, there is a distinct need for better methods of automatically solving labeling function conﬂicts.

Original: Our proposed methodology seeks toreduce label noise by automatically learning rules designed to differentiate between separate classes.
Revised: Our proposed methodology seeks to reduce label noise by automatically learning rules designed to differentiate between separate classes.

Original: Label Incompleteness A complementary problem to label noise is label incompleteness.
Revised: Label Incompleteness A complementary problem to label noise is label incompleteness.

Original: Labelincompleteness is the insufﬁciency of labeling functions to assign labels to particular slices of adataset.
Revised: Label incompleteness is the insufﬁciency of labeling functions to assign labels to particular slices of a dataset.

Original: This occurs when the syntactic and semantic patterns in a subset of examples do not liewithin the scope of the given labeling functions.
Revised: This occurs when the syntactic and semantic patterns in a subset of examples do not lie within the scope of the given labeling functions.

Original: Label incompleteness is particularly pervasive inthe long tails of a dataset, which may contain more informative data for better model performanceand generalization.
Revised: Label incompleteness is particularly pervasive in the long tails of a dataset, which may contain more informative data for better model performance and generalization.

Original: This can be especially manifest in low-resource or highly technical domains(e.g.biomedical research) where differences in nomemclature could lead to large labeling gaps.
Revised: This can be especially manifested in low-resource or highly technical domains where differences in nomenclature could lead to large labeling gaps.

Original: Approaches to tackle this problem include differentiable soft-matching of labeling rules to unlabeledinstances [13], automatic rule generation using pre-speciﬁed rule patterns [14, 15], cotraining a rule-based labeling module with a deep learning module capable of matching unlabeled instances[16, 10], and encouraging LF diversity by interactively soliciting LFs for unlabeled instances [12].
Revised: Approaches to tackle this problem include differentiable soft-matching of labeling rules to unlabeled instances [13], automatic rule generation using pre-speciﬁed rule patterns [14, 15], cotraining a rule-based labeling module with a deep learning module capable of matching unlabeled instances [16, 10], and encouraging LF diversity by interactively soliciting LFs for unlabeled instances [12].

Original: Annotator Effort Many domains such as biomedical or legal NLP require substantial subjectmatter expertise to annotate correctly.
Revised: Annotator Effort Many domains such as biomedical or legal NLP require substantial subject matter expertise to annotate correctly.

Original: However, subject matter experts (SMEs) are generally quitecostly and havevery limited time to devote to annotation.
Revised: However, SMEs are costly and have limited time to for annotation.

Original: This problem is strikingly evident whenone compares the size of labeled datasets in biomedicine [17, 18, 1] with those for general domaintasks (for example, the average training set in the general domain GLUE benchmark [19] is over 25xlarger than that of the average dataset in the contemporaneous BLUE benchmark [1] for biomedical NLP).
Revised: This problem is strikingly evident when one compares the size of labeled datasets in biomedicine [17, 18, 1] with those for general domain tasks. For example, the average training set in the general domain GLUE benchmark [19] is over 25x larger than that of the average dataset in the contemporaneous BLUE benchmark [1] for biomedical NLP.

Original: Accordingly, it is important to solicit the most useful possible SME feedback to maximizelabel signal while minimizing annotator time and effort.
Revised: Accordingly, it is important to solicit the most useful possible SME feedback to maximize label signal while minimizing annotator time and effort.

Original: By presenting annotators with candidatelabeling rules, ReGAL reduces the time necessary to specify rules by hand and thus offers largepotential gains to annotator efﬁciency.
Revised: By presenting annotators with candidate labeling rules, ReGAL reduces the time necessary to specify rules by hand and thus offers large potential gains to annotator efﬁciency.

Original: 2.3 Objectives
Revised: 2.3 Objectives

Original: This paper develops a model, ReGAL , that addresses these challenges by automatically proposinglabeling rules designed to (1) disambiguate instances with conﬂicting LF-induced labels and (2)extend coverage to unlabeled portions of the dataset.
Revised: This paper develops a model, ReGAL , that addresses these challenges by automatically proposing labeling rules designed to (1) disambiguate instances with conﬂicting LF-induced labels and (2) extend coverage to unlabeled portions of the dataset.

Original: This is done interactively as annotators generatelabels, allowing ReGAL to adapt to new labeling needs as the set of labels expands.
Revised: This is done interactively as annotators generate labels, allowing ReGAL to adapt to new labeling needs as the set of labels expands.

Original:  Related Work3.1 Data programming
Revised:  Related Work 3.1 Data programming

Original: Our work is closely related with recent works surrounding data programming and rule-based textclassiﬁcation.
Revised: Our work is closely related with recent works surrounding data programming and rule-based text classiﬁcation.

Original: One of the seminal works in this category proposed Snorkel [20, 6], which proposes arule-based labeling model where multiple, overlapping rules are proposed for each  and then denoisedwith a generative model to account for the inter-rule correlations.
Revised: One of the seminal works in this category proposed Snorkel [20, 6], which introduced a rule-based labeling model where multiple, overlapping rules are proposed for each class and then are denoised with a generative model to account for the inter-rule correlations.

Original: A number of alternative denoisingapproaches have since been proposed to improve the quality of aggregated labels.
Revised: A number of alternative denoising approaches have since been proposed to improve the quality of aggregated labels.

Original: Among these, Renet al [16] propose a self-training discriminative denoising module, while Awasthi et al.
Revised: Among these, Ren et al. [16] propose a self-training discriminative denoising module, while Awasthi et al.

Original: [11] proposelearning appropriate bounds on rule coverage from a small labeled set of data.
Revised: [11] propose learning appropriate bounds on rule coverage from a small labeled set of data.

Revised: ReGAL differs from this work by providing model-based assistance to deﬁne high-quality rules and augment human knowledge with global data patterns in addition to providing a module to denoise labeling rules.

Original: Follow-up studies [12] on Snorkel have also shown that Snorkel users tended to select rules bylooking at individual labeled instances and seeking to create rules reﬂecting the patterns they observein the data.
Revised: Follow-up studies [12] on Snorkel have also shown that Snorkel users tended to select rules by looking at individual labeled instances and seeking to create rules reﬂecting the patterns they observe in the data.

Original: This study further demonstrated that label efﬁciency can be improved by showing userscurrently unlabeled instances and instances with high label conﬂict, i.e. seeking labels in highlyvaluable areas of the training data.
Revised: This study further demonstrated that label efﬁciency can be improved by showing users currently unlabeled instances and instances with high label conﬂict, i.e. seeking labels in highly valuable areas of the training data.

Original: It also demonstrates that even subject matter experts tend to relyon the data to create their rules, rather than creating them from independent knowledge.
Revised: ReGAL provides the possibility to extend this by automatically proposing rules suited to conﬂicted or unlabeled data slices, thus reducing the manual effort involved.

Original: 3.2 Rule Learning
Revised: 3.2 Rule Learning

Original: Another key area of related work is rule learning, in which models seek to automatically label datasetsby generating their own label rules.
Revised: Another key area of related work is rule learning, in which models seek to automatically label datasets by generating their own label rules.

Original: MetaPad [21] mined and consolidated information extractionpatterns from large text corpora to improve the rule-based extraction of speciﬁc types of namedentities and their attributes from raw text.
Revised: MetaPad [21] mined and consolidated information extraction patterns from large text corpora to improve the rule-based extraction of speciﬁc types of named entities and their attributes from raw text.

Original: REPEL [15], on the other hand, focused speciﬁcally onrule-guided relation extraction.
Revised: REPEL [15], on the other hand, focused speciﬁcally on rule-guided relation extraction.

Original: The authors began with small sets of labeled seed instances anditeratively reﬁne a set of potential candidate rules basedbased on keywords in the shortest dependencypath between entities.
Revised: The authors began with small sets of labeled seed instances and iteratively reﬁned a set of potential candidate rules based on keywords in the shortest dependency path between entities.

Original: While their approach has many similarities to our own, it is limited in that itcan only produce a single type of rule and does not allow for iterative reﬁnement via human guidance.
Revised: While their approach has many similarities to our own, it is limited in that it can only produce a single type of rule and does not allow for iterative reﬁnement via human guidance.

Original: Snuba [14] generates candidate rules using a collection of weak learner primitives (e.g. decisionstumps, k-nearest neighbors) and then synthesizes and prunes this set of rules to generate ﬁnal rulesfor labeling.
Revised: Snuba [14] generates candidate rules using a collection of weak learner primitives (e.g. decision stumps, k-nearest neighbors) and then synthesizes and prunes this set of rules to generate ﬁnal rules for labeling.

Original: ReGAL combines elements  this framework to allow iterative communication betweendownstream classiﬁers and rule selectors to allow each to mutually enhance the other.This allows
Revised: ReGAL combines elements of this framework to allow iterative communication between downstream classiﬁers and rule selectors to allow each to mutually enhance the other.

Original: ReGAL to uniquely offer the ease of model-generated LFs while still gaining the nuanced insight ofhuman annotator input.
Revised: ReGAL differs from these by soliciting user feedback on rules, thus enabling it complement model-generated LFs with annotator guidance.

Original:  Methods
Revised:  Methods

Original: Here we describe ReGAL , a model that interactively generates labeling functions via repeated roundsof user feedback.
Revised: Here we describe ReGAL , a model that interactively generates labeling functions via repeated rounds of user feedback.

Original: This model is composed of three modules: (1) A contextualized word and documentencoder that produces global document embeddings and contextualized word embeddings; (2) a ruledenoising module that aggregates labeling functions to produce coherent labels for matched andunmatched documents; and (3) a rule proposal module that generates candidate labeling functions. 
Revised: This model is composed of three modules: (1) A contextualized word and document encoder that produces global document embeddings and contextualized word embeddings; (2) a rule denoising module that aggregates labeling functions to produce coherent labels for matched and unmatched documents; and (3) a rule proposal module that generates candidate labeling functions. A depiction of our model is shown in Figure

Original: 4.1 Contextualized Token Encoder
Revised: 4.1 Contextualized Token Encoder

Original: ReGAL begins with a token encoder.
Revised: ReGAL begins with a token encoder.

Original: The purpose of the encoder is to produce expressive, contextualized token embeddings containing the semantic content necessary  identifythey as potential keywordsin generated LFs.
Revised: The purpose of the encoder is to produce expressive, contextualized token embeddings containing the semantic content necessary to identify potential keywords in generated LFs.

Original: These embeddings will also be aggregated by the rule denoiser into a cohesivedocument representations.
Revised: These embeddings will also be aggregated by the rule denoiser into a cohesive document representation.

Original: We use transformers [22] to construct our encoder because they allow each token to draw contextualinformation from every other token in its same input document.
Revised: We use transformers [22] to construct our encoder because they allow each token to draw contextual information from every other token in its same input document.

Original: Speciﬁcally, we use a pretrained
Revised: Speciﬁcally, we use a pretrained

Original: BERT-base [23] model as our encoder.
Revised: BERT-base [23] model as our encoder.

Original: We use the outputs
Revised: We use the outputs

Original: h i,t of the last layer as token embeddings:(cid:104) h
Revised: h i,t of the last layer as token embeddings: (cid:104) h

Original: i, 1 , . . .
Revised: i, 1 , . . .

Original: , h i,T (cid:105) =
Revised: , h i,T (cid:105) =

Original: enc ([ v i, 1 , . . .
Revised: enc ([ v i, 1 , . . .

Original: , v i,T ])
Revised: , v i,T ])

Original: We will henceforth let H i =
Revised: We will henceforth let H i =

Original: (cid:104)
Revised: (cid:104)

Original: h
Revised: h

Original: i, 1 , . . .
Revised: i, 1 , . . .

Original: , h i,T (cid:105) represent the sequence token embeddings fromdocument d i 
Revised: , h i,T (cid:105) represent the sequence token embeddings from document d i .

Original: In addition to initializing this module with a BERT-base, we encourage the encoder to learn contextualinformation about rules.
Revised: In addition to initializing this module with a BERT-base, we encourage the encoder to learn contextual information about rules.

Original: We do so using masked language modeling (MLM), where our maskingbudget consists of all tokens used in keyword LFs as well as a random 10% of tokens from thesequence.
Revised: We do so using masked language modeling (MLM), where our masking budget consists of all tokens used in keyword LFs as well as a random 10% of tokens from the sequence.

Original: We either mask or noise each token according to the strategy in [23] and require ourmodel to predict the correct token in each case.
Revised: We either mask or noise each token according to the strategy in [23] and require our model to predict the correct token in each case.

Original: We optimize this using cross entropy loss over themasked/noised tokens, which we denote L MLM .
Revised: We optimize this using cross entropy loss over the masked/noised tokens, which we denote L MLM .

Original: 4.2 Rule Denoiser
Revised: 4.2 Rule Denoiser

Original: We develop a rule denoiser module similar to [16] that allows our model to learn document-levelapplicability of matched LFs.
Revised: We develop a rule denoiser module similar to [16] that allows our model to learn document-level applicability of matched LFs.

Original: This is co-trained with a neural prediction module that extends coverageto unmatched documents based on their learned semantic properties.
Revised: This is co-trained with a neural prediction module that extends coverage to unmatched documents based on their learned semantic properties.

Original: The rule denoiser also learnsclass-speciﬁc token importance scores that will be used to generate new LFs.
Revised: The rule denoiser also learns class-speciﬁc token importance scores that will be used to generate new LFs.

Original: The rule denoiser hastwo submodules which we discuss in detail: a token attention module and a rule attention module.
Revised: The rule denoiser has two submodules which we discuss in detail: a token attention module and a rule attention module.

Original: 4.2.1 Token attention
Revised: 4.2.1 Token attention

Original: The purpose of the TokenAttention submodule is to learn the token and document level informationnecessary to generate expressive, class-speciﬁc labeling functions.
Revised: The purpose of the TokenAttention submodule is to learn the token and document level information necessary to generate expressive, class-speciﬁc labeling functions.

Original: Speciﬁcally, TokenAttentioncomputes class-speciﬁc attention scores on each token, and the document’s probability of belonging toeach class.
Revised: Speciﬁcally, TokenAttention computes class-speciﬁc attention scores on each token, and the document’s probability of belonging to each class.

Original: These class probabilities also serve as suitability scores of how well-equipped a documentis to generate LF keywords for a given class.
Revised: These class probabilities also serve as suitability scores of how well-equipped a document is to generate LF keywords for a given class.

Original: TokenAttention takes as inputs the token embeddings from the document encoder and producesclass-speciﬁc token attention a ( c ) i,t , document embeddings e i , and document-level class probabilitiesp i
Revised: TokenAttention takes as inputs the token embeddings from the document encoder and produces class-speciﬁc token attention a ( c ) i,t , document embeddings e i , and document-level class probabilities p

Original: = (cid:104) p (1) i , . .
Revised: [Equation]

Original: , p ( C ) i (cid:105) .
Revised: [Equation] (cid:105) .

Original: We describe how each of these is computed as follows.
Revised: We describe how each of these is computed as follows.

Original: We begin by calculating class-speciﬁc attention scores for each token in our document.
Revised: We begin by calculating class-speciﬁc attention scores for each token in our document.

Original: These will beused by our rule proposal network to generate new labeling rules.
Revised: These will be used by our rule proposal network to generate new labeling rules.

Original: They are calculated as
Revised: They are calculated as

Original: [Equation]
Revised: [Equation]

Original: where W a 1 ∈ R m 2 × m 1 and W a 2 ∈ R c × m 2 .
Revised: where W a 1 ∈ R m 2 × m 1 and W a 2 ∈ R c × m 2 .

Original: These scores are then used to calculate a class-speciﬁcdocument representation˜ e ( c ) i = T (cid:88)
Revised: These scores are then used to calculate a class-speciﬁc document representation

Original: [Equation]
Revised: [Equation]

Original: These are in turn aggregated into an overall document representation 
Revised: These are in turn aggregated into an overall document representation with class weights

Revised: η c e i =

Revised: C (cid:88)

Original: [Equation]
Revised: [Equation]

Original: which will be used by the rule attention submodule to estimate conditional LF reliability.
Revised: This representation will be used by the rule attention submodule to estimate conditional LF reliability.

Original: The class-speciﬁc embeddings
Revised: The class-speciﬁc embeddings

Original: ˜ e
Revised: ˜ e

Original: ( c ) i are also used to compute the document’s class probabilities:
Revised: ( c ) i are also used to compute the document’s class probabilities:

Original: [Equation]
Revised: [Equation]

Original: where p ( c ) i =
Revised: where p ( c ) i =

Original: w ( c ) p T ˜
Revised: w ( c ) p T ˜

Original: e ( c ) i and w ( c ) p is a weight vector corresponding to each class.
Revised: e ( c ) i and w ( c ) p is a weight vector corresponding to each class.

Original: In addition toserving as this submodule’s prediction of the document’s label, these probabilities can additionallyserve as measures of the document’s suitability to generate keywords to be used in future LFs.
Revised: In addition to serving as this submodule’s prediction of the document’s label, these probabilities can additionally serve as measures of the document’s suitability to generate keywords to be used in future LFs.

Original: 4.2.2 Rule attention
Revised: 4.2.2 Rule attention

Original: The RuleAttention submodule learnsto learn measures of labeling function validity conditionedon the encoded document representation e i .
Revised: The RuleAttention submodule learns measures of labeling function validity conditioned on the encoded document representation e i .

Original: It outputs document-speciﬁc LF reliability scores as wellas soft labels to be used in co-training our neural classiﬁer.
Revised: It outputs document-speciﬁc LF reliability scores as well as soft labels to be used in co-training our neural classiﬁer.

Original: We ﬁrst learn conditional LF reliabilityscores
Revised: We ﬁrst learn conditional LF reliability scores

Original: z i for document d i as
Revised: z i for document d i as

Original: [Equation]
Revised: [Equation]

Original: We weight our masked LFs by these reliability scores using a masked elementwise product s i =
Revised: We weight our masked LFs by these reliability scores using a masked elementwise product s i =

Original: z i (cid:12) (cid:96) i .
Revised: z i (cid:12) (cid:96) i .

Original: This produces probabilistic class labels ˆ
Revised: This produces probabilistic class labels ˆ

Original: y i for each labeled document.
Revised: y i for each labeled document.

Original: We use these soft labels andthe class probabilities p i TokenAttention using probabilistic cross entropy loss:
Revised: We use these soft labels and the class probabilities p i TokenAttention using probabilistic cross entropy loss:

Original: [Equation]
Revised: [Equation]

Original: 4.3 Rule Proposal Network
Revised: 4.3 Rule Proposal Network

Original: ReGAL’s rule proposal network (RPN) enables ReGAL to generate new rules given a set of seed rules.
Revised: ReGAL’s rule proposal network (RPN) enables ReGAL to generate new rules given a set of seed rules.

Original: The RPN takes as inputs both the class-conditioned word level attention a ci,t and document-levelclass probabilities p i and outputs a scores τ ( c ) j for each v j ∈ V corresponding to how strongly v jrepresents class c .
Revised: The RPN takes as inputs both the class-conditioned word level attention a ci,t and document-level class probabilities p i and outputs a score τ ( c ) j for each v j ∈ V corresponding to how strongly v j represents class c .

Original: These scores are calculated as:
Revised: These scores are calculated as:

Original: [Equation]
Revised: [Equation]

Original: Here, γ ∈ [0 , 1] is a parameter that controls how much our ReGAL’s RPN balances between thecoverage of a token (i.e. how often it occurs) and its instance level importance.
Revised: Here, γ ∈ [0 , 1] is a parameter that controls how much our ReGAL’s RPN balances between the coverage of a token (i.e. how often it occurs) and its instance level importance.

Original: Low values of γ favortokens with high coverage while high values of γ favor rules with high impact tokens without regardfor coverage.
Revised: Low values of γ favor tokens with high coverage while high values of γ favor rules with high impact tokens without regard for coverage.

Revised: Since the types of rules needed may differ as training progresses, we allow users to choose γ for each round of proposed rules.

Original: We use γ = 0 . 6 for our experiments, which works well in practice.
Revised: In practice, we ﬁnd that γ ∈ [0 . 5 , 0 . 9] tend to produce good rules.

Original: After calculating token scores τ ( c ) j , the RPN proposes up to k new LFs for each class by choosing thetop k scoring tokens { v ( c ) 1 , . .
Revised: After calculating token scores τ ( c ) j , the RPN proposes up to k new LFs for each class by choosing the top k scoring tokens { v ( c ) 1 , . .

Original: . , v ( c ) k } that are not highly scoring for any other class c (cid:48) .
Revised: . , v ( c ) k } that are not highly scoring for any other class c (cid:48) .

Original: These tokenseach induce a labeling function of the form HAS(x, v ( c ) i )
Revised: These tokens each induce a labeling function of the form HAS(x, v ( c ) i )

Original: → c , where we assign the class label c toa text x  it contains the token v ( c ) i .
Revised: → c , where we assign the class label c to a text x if it contains the token v ( c ) i .

Original: These proposed LFs are then presented to a human annotator, whocan choose to accept or reject them.
Revised: These proposed LFs are then presented to a human annotator, who can choose to accept or reject them.

Original: 4.4 Model optimization
Revised: 4.4 Model optimization

Original: We optimize our entire model by minimizing the unweighted sum  the loss functions of its components: 
Revised: We optimize our entire model by minimizing the unweighted sum of the loss functions of its components: L

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original:  Experiments and Discussion
Revised:  Experiments and Discussion

Original: Weperform a variety of experiments to test the ability of ReGAL to generate meaningful, diverserules, as this is necessary to assist humans in efﬁcient data annotation.
Revised: We test the ability of ReGAL to generate meaningful, diverse rules to assist humans in efﬁcient data annotation.

Original: In doing so, we follow manyaspects of the experimental setup of [16] and speciﬁcally demonstrate how our model can be used toadd to their carefully curated rule-based labeling LFs.
Revised: In doing so, we follow many aspects of the experimental setup of [16] and speciﬁcally demonstrate how our model can be used to add to their carefully curated rule-based labeling LFs.

Original: We note that this a work  in progress and thus models have not undergone hyperparameter optimization.
Revised: We note that this a work is in progress and thus models have not undergone hyperparameter optimization.

Original: However, even without this optimization we demonstrate that ReGAL is both able to generate coherentrules and adeptly denoise rule-induced labels, making it a powerful toolfor data programming bothas a tool to assist annotators in generating LFs andas a model to disambiguate conﬂicted labels.
Revised: However, even without this optimization we demonstrate that ReGAL is both able to generate coherent rules and adeptly denoise rule-induced labels, making it a powerful tool to assist annotators in generating LFs and disambiguate conﬂicted labels.

Original: 5.1 Datasets and tasks
Revised: 5.1 Datasets and tasks

Original: We evaluate our model’s performance on two well-known sentiment classiﬁcation datasets.
Revised: We evaluate our model’s performance on two well-known sentiment classiﬁcation datasets.

Original: • Yelp is a collection of Yelp restaurant reviews classiﬁed according to their sentiment  • IMDB is a set of movie reviews classiﬁed according to sentiment 2 Baselines
Revised: • Yelp is a collection of Yelp restaurant reviews classiﬁed according to their sentiment [24] • IMDB is a set of movie reviews classiﬁed according to sentiment [25] 5.2 Baselines

Original: We compare the predictive performance of our model to three strong baselines for text classiﬁcationwith weak supervision: 
Revised: We compare the predictive performance of our model to three strong baselines for text classiﬁcation with weak supervision: [Table]

Original: • Majority voting of the labeling functions, where ties are broken by choosing the most common class as speciﬁed by the labling functions.
Revised: • Majority voting of the labeling functions, where ties are broken by choosing the most common class as speciﬁed by the labeling functions.

Original: • BERT [23] is a transformer-based language model that has shown exceptional performanceon a wide-range of NLP tasks.
Revised: • BERT [23] is a transformer-based language model that has shown exceptional performance on a wide-range of NLP tasks.

Original: We train BERT with the majority vote labels from our LFs.• MSWS [16] is a denoising method for multi-source weak supervision that co-trains cotrainsa rule denoiser with a neural classiﬁer to learn optimal weightings for rules and labelunmatched samples. 
Revised: We train BERT with the majority vote labels from our LFs. • Epoxy [26] is a recent weak supervision paradigm that uses combined pretrained embeddings with anchored weakly-labeled examples to enable interactive model training. • MSWS [16] is a denoising method for multi-source weak supervision that co-trains a rule denoiser with a neural classiﬁer to learn optimal weightings for rules and label unmatched samples. • Fully Supervised We train a BERT model with a fully connected layer to provide baseline performance for a model trained with ground-truth labels.

Original: 5.3 Rule Denoising
Revised: 5.3 Rule Denoising

Original: Parsing signal from noise is critical to learning from weak and rule-based supervision.
Revised: Parsing signal from noise is critical to learning from weak and rule-based supervision.

Original: Accordingly,we compare ReGAL’s ability to that of our baselines in accurately classifying instances based on aset of seed rules, which are shown in Table 1.
Revised: Accordingly, we compare ReGAL’s ability to that of our baselines in accurately classifying instances based on a set of seed rules, which are shown in Table 1.

Original: For each dataset, we provide exactly one seed LF foreach class. Each seed LF contains exactly six single-token keywords.
Revised: We provided exactly each class with exactly one labeling function consisting of six keywords or phrases adapted from [16].

Original: If any of these keywords isfound in document d
Revised: If any of these keywords is found in document d

Original: i , the LF assigns its label; otherwise, it abstains from labeling d i .
Revised: i , the LF assigns its label; otherwise, it abstains from labeling d i .

Revised: After providing seed rules

Original: One factor that majorly differentiates ReGAL from the above is its ability to interactively generatelabeling functions to expand label coverage.
Revised: [Table] we trained ReGAL with learning rate 0 . 001 and batch size 64, generating new rules every 100 batches. We continued iterating until we saw multiple consecutive rounds with for which at least one class proposed no high-quality rules.

Revised: In each case, proposed rules were adjudicated by university students to assess their utility and relevance.

Revised: One factor that majorly differentiates ReGAL from our baselines is its ability to interactively generate labeling functions to expand label coverage.

Original: This potentially gives ReGAL an advantage over ourbaselines which cannot generate new labeling rules.
Revised: This potentially gives ReGAL an advantage over our baselines which cannot generate new labeling rules.

Original: Accordingly, we show the performance of eachof these models using (1) only seed LFs and (2) seed + generated LFs from ReGAL.
Revised: Accordingly, we show the performance of each of these models using (1) only seed LFs and (2) seed + generated LFs from ReGAL.

Original: The results areshown in Table 2.
Revised: The results are shown in Table 2.

Original: Interestingly, although ReGAL generates coherent and intuitive rules (see section 5.4), these rulesactually degrade ReGAL’s performance.
Revised: Interestingly, although ReGAL generates coherent and intuitive rules (see section 5.4), these rules actually degrade ReGAL’s performance.

Original: This is particularly surprising considering that the added[Table] [Table] rules substantially boosts the performance of each of the baselines.
Revised: This is particularly surprising considering that the added rules substantially boost the performance of each of the baselines.

Original: We suspect that this may be dueto the masking of rule keywords in the encoder, which could inhibit the model from learning theirsignal.
Revised: We suspect that this may be due to the masking of rule keywords in the encoder, which could inhibit the model from learning their signal.

Original: We will investigate this anomaly further as we continue to develop the model
Revised: We will investigate this anomaly further as we continue to develop the model.

Original: [Table]

Original: 5.4 Evaluating Generated Rules
Revised: 5.4 Evaluating Generated Rules

Original: We evaluate the ability of our model to identify promising rules by providing it a small number ofseed rules and qualitatively evaluating its ability to discover additional high-quality rules.
Revised: We evaluate the ability of our model to identify promising rules by providing it a small number of seed rules and qualitatively evaluating its ability to discover additional high-quality rules.

Original: In tables3 and 4, we display the top 5 generated rules from the ﬁrst two iterations of ReGAL for Yelp and
Revised: In tables 3 and 4, we display the top 5 generated rules from the ﬁrst two iterations of ReGAL for Yelp and

Original: IMDB respectively.
Revised: IMDB respectively.

Original: From these results, one can see that on the ﬁrst iteration, ReGAL generates a very high proportion ofhighly informative rules.
Revised: From these results, one can see that on the ﬁrst iteration, ReGAL generates a very high proportion of highly informative rules.

Original: For example, ReGAL’s ﬁrst set of rules touches on multiple diverse aspectsof a restaurant’s quality, including food quality, speed, and friendliness of staff.
Revised: For example, ReGAL’s ﬁrst set of rules touches on multiple diverse aspects of a restaurant’s quality, including food quality, speed, and friendliness of staff.

Original: IMDB’s ﬁrst-iterationrules capture similarly strong signal, picking up that negative reviews call movies “boring” and “dull”while positive ones cite them as “touching” or “marvelous.”
Revised: IMDB’s ﬁrst-iteration [Table] [Table] rules capture similarly strong signal, picking up that negative reviews call movies “boring” and “dull” while positive ones cite them as “touching” or “marvelous.

Original:  Unfortunately, rule quality appears to degrade sharply after the ﬁrst iteration.
Revised: [Table] Unfortunately, rule quality appears to degrade sharply after the ﬁrst iteration.

Original: After a 90% acceptancerate in the ﬁrst round of generation on Yelp, ReGAL failed to generate a single accepted rule in round2.
Revised: After a 90% acceptance rate in the ﬁrst round of generation on Yelp, ReGAL failed to generate a single accepted rule in round 2.

Original: While the results on IMDB were somewhat more promising, ReGAL only generated meaningful 
Revised: While the results on IMDB were somewhat more promising, ReGAL only generated meaningful rules for a single class in the second round, which could potentially lead to label function imbalance and thus improper ﬁtting. Because ReGAL is an active learning framework

Original: [Table]
Revised: [Table]

Original: POS

Original: HAS(x, ‘beautifully’)

Original: Y HAS(x, ‘character’) N rules for a single class in the second round, which could potentially lead to label function imbalanceand thus improper ﬁtting. Because ReGAL is an active learning framework, users are able to peruse proposed rules and controlhow the model expands its labeling space.
Revised: users are able to evaluate proposed rules and control how the model expands its labeling space.

Original: This makes ReGAL resiliant  spurious generated rules andallows users to use it to quickly and meaningfully expand their rule space.
Revised: This makes ReGAL resiliant to spurious generated rules and allows users to use it to quickly and meaningfully expand their rule space [Table] [Table]

Original:  Future Work
Revised:  Future Work

Original: As we continue to develop ReGAL, we intend to make improvements to both our model and evaluation.
Revised: As we continue to develop ReGAL, we intend to make improvements to both our model and evaluation.

Original: First, we will add a more diverse set of baselines and evaluation datasets that allow us to compare
Revised: First, we will add a more diverse set of baselines and evaluation datasets that allow us to compare

Original: ReGAL to both traditional active learning frameworks as well as interactive data programmingparadigms [12].
Revised: ReGAL to both traditional active learning frameworks as well as interactive data programming paradigms [12].

Original: These, in turn, will be tested on a more diverse array of text classiﬁcation tasks,especially those requiring domain expertise (e.g. biomedical).
Revised: These, in turn, will be tested on a more diverse array of text classiﬁcation tasks, especially those requiring domain expertise (e.g. biomedical).

Original: Second, we will expand ReGAL’srule generating capacity to encompass multi-token sequences (e.g. for keywords that are split intomultiple tokens) or non-contiguous textual patterns.
Revised: Second, we will expand ReGAL’s rule generating capacity to encompass multi-token sequences (e.g. for keywords that are split into multiple tokens) or non-contiguous textual patterns.

Original: Finally, we will conduct user tests to determineand optimize the quality of human interactions to make ReGAL more user-friendly and accurate.
Revised: Finally, we will conduct user tests to determine and optimize the quality of human interactions to make ReGAL more user-friendly and accurate.

Revised: One promising area for future work is adapting ReGAL to work with crowdsourced labeling.

Revised: In such a framework, LFs could be adopted only after agreement from multiple annotators, which would lead to higher overall rule quality.

Revised: We forsee that this could natturally lead to a two-stage crowdsourcing framework in which earlier annotators would focus on broadening coverage with new LFs while subsequent annotators could with higher LF uncertainty (e.g. due to conﬂicting LF matches).

Revised: This would increase crowdsourcing efﬁciency by focusing annotators on providing instancelevel supervision on difﬁcult samples while still producing rule-induced labels on the remainder of the data.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we presented our work-in-progress ReGAL, a text classiﬁcation model capable ofboth denoising multiple sources of noisy supervision and generating high-quality labeling functionscapable of inductively labeling uncovered portions of dataset.
Revised: In this work, we presented our work-in-progress ReGAL, a text classiﬁcation model capable of both denoising multiple sources of noisy supervision and generating high-quality labeling functions capable of inductively labeling uncovered portions of dataset.

Original: In doing so, ReGAL offers the potentialto substantially reduces human labeling efforts by helping them identify quality labeling rules theywould otherwise miss.
Revised: In doing so, ReGAL offers the potential to substantially reduce human labeling efforts by helping them identify quality labeling rules they would otherwise miss.

Original: We demonstrate ReGAL’s utility by showing that it both generates coherent,reasonable rules and also that using the selected labeling rules improves the performance of each ofour baseline models.
Revised: We demonstrate ReGAL’s utility by showing that it both generates coherent, reasonable rules and also that using the selected labeling rules improves the performance of each of our baseline models.

Original: Broader Impact
Revised: Broader Impact

Original: This work has major potential to accelerate advancements in low-resource NLP domains by assistingsubject matter experts in discovering patterns capable of quickly labeling their data.
Revised: This work has major potential to accelerate advancements in low-resource NLP domains by assisting subject matter experts in discovering patterns capable of quickly labeling their data.

Original: This would allowthem to tackle more problems for which they previously lacked data while simultaneously reducingthe cost of doing so.
Revised: This would allow them to tackle more problems for which they previously lacked data while simultaneously reducing the cost of doing so.

Original: One could observe similar beneﬁts in NLP for scientiﬁc and biomedical texts.
Revised: One could observe similar beneﬁts in NLP for scientiﬁc and biomedical texts.

Original: The breadth of evena single scientiﬁc discipline is so broad that a single SME may miss key semantic indicators thatare not associated with his/her subﬁeld.
Revised: The breadth of even a single scientiﬁc discipline is so broad that a single SME may miss key semantic indicators that are not associated with his/her subﬁeld.

Original: Thus, automatic generation of labeling rules can potentiallyincrease the diversity of scientiﬁc and biomedical NLP tasks and thus accelerate the discovery of newinsights derived from text mining.
Revised: Thus, automatic generation of labeling rules can potentially increase the diversity of scientiﬁc and biomedical NLP tasks and thus accelerate the discovery of new insights derived from text mining.

Original: There are two potential negative impacts of this work, both of which hinge on the cognitive biases ofhumans involved in data annotation.
Revised: There are two potential negative impacts of this work, both of which hinge on the cognitive biases of humans involved in data annotation.

Original: First, our model exploits patterns in seed rules for generating newlabeling functions, which may lead it to miss important labeling signals that are independent fromthe given seed LFs.
Revised: First, our model exploits patterns in seed rules for generating new labeling functions, which may lead it to miss important labeling signals that are independent from the given seed LFs.

Original: Second, the reduction of human effort in creating LFs may lead to annotatorsaccepting proposed rule without sufﬁciently rigorous evaluation, thus degrading LF quality (and thuslabel quality).
Revised: Second, the reduction of human effort in creating LFs may lead to annotators accepting proposed rule without sufﬁciently rigorous evaluation, thus degrading LF quality. Acknowledgments and Disclosure of Funding

Revised: This work was supported by National Science Foundation CAREER grant 1944247, National Institute of Health grant R21-CA232249, Whitaker Biomedical Engineering Seed Grant for COVID-19,

Revised: Alzheimer’s Association Research Grant Award, Childrens Hospital of Atlanta Aﬂac Pilot Grant

Revised: Award, National Science Foundation award III-2008334, an Amazon Faculty Award, and a Google

Revised: Faculty Award.