DSReg: Using Distant Supervision as a Regularizer

1 INTRODUCTION . Consider the following sentences in a text-classification task , in which we want to identify text describing hotels with good service/staff ( as depicted as aspect-level sentiment classification in Tang et al . ( 2015 ) ; Li et al . ( 2016 ) ; Lei et al . ( 2016 ) ) : • S1 : the staff are great . ( positive ) • S2 : the location is great but the staff are surly and unhelpful .. ( hard-negative ) • S3 : the staff are surly and unhelpful . ( easy-negative ) S1 is a positive example since it describes a hotel with good staff . Both S2 and S3 are negative because staff are unhelpful . However , since S2 is lexically and semantically similar with S1 , standard models can be easily confused . As another example , in reading comprehension tasks like NarrativeQA Kočiskỳ et al . ( 2018 ) , truth answers are human-generated ones and might not have exact matches in the original passage . A commonly adopted strategy is to first locate similar sequences from the original passage using a pre-defined threshold ( using metrics like ROUGE-L ) and then treat them as positive training examples . Sequences that are semantically similar but right below this specific threshold will be treated as negative examples and will thus inevitably introduce massive labeling noise in training . This problem is ubiquitous in a wide range of NLP tasks , i.e. , some of the negative examples are highly similar to positive examples . We refer to these negative examples as hard-negative examples for the rest of this paper . Similarly , the negative examples that are not similar to the positive examples are refered to as easy-negative examples . Hard-negative examples can cause big trouble in model training , because the nuance between positive examples and hard-negative examples can cause confusion for a model trained from scratch . To make things worse , when there is a class-balance problem where the number of negative examples are overwhelmingly larger than that of positive examples ( which is true in many real-world use cases ) , the model will be at loose ends because positive features are buried in the sea of negative features . To tackle this issue , we propose using the idea of distant supervision ( e.g. , Mintz et al . ( 2009 ) ; Riedel et al . ( 2010 ) ) to regularize training . We first harvest hard-negative examples using distant supervision . This process can be done by a method as simple as using word overlapping metrics ( e.g. , ROUGE , BLEU or whether a sentence contains some certain keywords ) . With the harvested hard-negative examples , we transform the original binary classification setting to a multi-task learning setting , in which we jointly optimize the original target objective of distinguishing positive examples from negative examples along with an auxiliary objective of distinguishing soften positive examples ( comprised of positive examples and hard-negative examples ) from easy-negative examples . For a neural network model , this goal can be easily achieved by using different output layers to readout the final-layer representations . In this way , the features that are shared between positive examples and hard negative examples can be captured by the model . Models can easily tell which features that can distinguish positive examples the most . Using this unbelievably simple strategy , we improve the performance of in a range of different NLP tasks , including text classification , sequence labeling and reading comprehension . The key contributions of this work can be summarized as follows : • We study a general situation in NLP , where a subset of the negative examples are highly similar to the positive examples . We analyze why it is a problem and how to deal with it . • We propose a general strategy that utilize the idea of distant supervision to harvest hard- negative training examples , and transform the original task to a multi-task learning problem . The strategy is widely applicable for a variaty of tasks . • Using this unbelievably simple strategy , we can obtain significant improvement on the tasks of text-classification , sequence-labeling and reading comprehension . 2 RELATED WORK . Distant Supervision Mintz et al . ( 2009 ) ; Riedel et al . ( 2010 ) ; Hoffmann et al . ( 2011 ) ; Surdeanu et al . ( 2012 ) It is proposed to address the data sparsity issue in relation extraction . Suppose that we wish to extract sentences expressing the ISCAPITAL relation , distant supervision augments the positve training set by first aligning unlabeled text corpus with all entity pairs between which the ISCAPITAL relation holds and then treating all aligned texts as positive training examples . The idea has been extended to other domains such as sentiment analysis Go et al . ( 2009 ) , computer security event Ritter et al . ( 2015 ) , life event extraction Li et al . ( 2014 ) and image classification Chen and Gupta ( 2015 ) . Deep leaning techniques have significantly improved the results of distant supervision for relation extraction Zeng et al . ( 2017 ) ; Luo et al . ( 2017 ) ; Lin et al . ( 2017 ) ; Toutanova et al . ( 2015 ) . Multi-Task Learning ( MTL ) The idea of using data harvested via distant supervision as auxiliary supervision signals is inspired by recent progress on multi-task learning : models for auxiliary tasks share hidden states or parameters with models for the main task and act as regularizers . In addition , neural models often celebrate performance boost when jointly trained for multiple tasks Collobert et al . ( 2011 ) ; Chen et al . ( 2017 ) ; Hashimoto et al . ( 2017 ) ; FitzGerald et al . ( 2015 ) . For instance , Luong et al . ( 2015 ) use sequence-to-sequence model to jointly train machine translation , parsing and image caption generation models . Dong et al . ( 2015 ) adopt an alternating training approach for different language pairs , i.e. , they optimize each task objective for a fixed number of parameter updates ( or mini-batches ) before switching to a different language pair . Swayamdipta et al . ( 2018 ) propose using syntactic tasks to regularize semantic tasks like semantic role labeling . Hashimoto et al . ( 2017 ) improve universal syntactic dependency parsing using a multi-task learning approach . 3 MODELS . In this section , we discuss the details of the proposed model . We focus on two different types of NLP tasks , text classification and sequence labeling . 3.1 TEXT CLASSIFICATION . Suppose that we have text-label pairs D = { xi , yi } . xi consists of a sequence of tokens xi = { wi,1 , wi,2 , ... , wi , ni } , where ni denotes the number of tokens in xi . Each text xi is paired with a binary label yi ∈ { 0 , 1 } . The training set can be divided into a positive set D+ and a negative set D− . Let ŷi denote the model prediction . The standard training objective can be given as follows : L1 =− ∑ ( xi , yi ) ∈D logP ( ŷi = yi|xi ) =− ∑ ( xi , yi ) ∈D+ logP ( ŷi = 1|xi ) − ∑ ( xi , yi ) ∈D− logP ( ŷi = 0|xi ) ( 1 ) LetDhard-neg denote the hard-negative examples retrieved using distant supervision . Here we introduce a new label z : z = 1 for instances in Dhard-neg ∪D+ , and z = 0 for instances in D− −Dhard-neg . We regularize L1 using an additional objective L2 : L2 =− ∑ ( xi , zi ) ∈D+∪Dhard-neg logP ( ẑi = 1|xi ) − ∑ ( xi , zi ) ∈D−−Dhard-neg logP ( ẑi = 0|xi ) ( 2 ) L2 can be thought as an objective to capture the shared features in positive examples and hardnegative examples . Equ.2 can also be extended to another similar form , distinguishing between Dhard-pos∪D− ( i.e. , the union of positive examples that are similar to negative and negative examples ) and D+ −Dhard-pos . Empirically , we also find that adding one more three-class classification objective L3 , which separates positive vs hard-negative vs easy-negative introduces additional performance boost . We suggest that adding this three-class classification will additionally highlight the difference between hard negative examples and easy negative examples for the model . The label is denoted by l , where l = 0 for easy negative examples , l = 1 for positive examples and l = 2 for hard negative examples . This leads the final objective function at test time to be : L = λ1L1 + λ2L2 + λ3L3 ( 3 ) where λ1 + λ2 + λ3 = 1 are used to control the relative importance of each loss . For a neural classification model , p ( z|x ) , p ( y|x ) and p ( l|x ) share the same model structure . The input text x is first mapped to a d-dimensional vector representation hx using suitable contextualization strategy , such as LSTMs Hochreiter and Schmidhuber ( 1997 ) , CNNs Kim ( 2014 ) or transformers Vaswani et al . ( 2017 ) . Then hx is fed to three fully connected layers with softmax activation function to compute p ( y|x ) , p ( z|x ) and p ( l|x ) respectively : p ( y|x ) = softmax ( Wyhx ) p ( z|x ) = softmax ( Wzhx ) p ( l|x ) = softmax ( Wlhx ) ( 4 ) where Wy , Wz ∈ R2×d , Wl ∈ R3×d . 3.2 SEQUENCE LABELING . In sequence labeling tasks Lafferty et al . ( 2001 ) ; Ratinov and Roth ( 2009 ) ; Collobert et al . ( 2011 ) ; Huang et al . ( 2015 ) ; Ma and Hovy ( 2016 ) ; Chiu and Nichols ( 2016 ) , a model is trained to assign labels to each of the tokens in a text sequence . Suppose that we are to assign labels to all tokens in a chunk of text D = { x1 , x2 , ... , xnD } where nD denotes the number of tokens in D. Let us consider a simple case where we only have one type of tag and we will use the standard IOB ( short for inside , outside , beginning ) sequence labeling format . In this case , it is a three-class classification problem , assigning yi ∈ ( B , I , O ) to each token . We treat tokens with label B and I as D+ and tokens with label O as D− . The objective function for the vanilla sequence labeling task is given as follows : L1 =− logP ( y1 : nD |x1 : nD ) ( 5 ) P ( y1 : nD |x1 : nD ) can be computed using standard sequence tagging models such as CRF Lafferty et al . ( 2001 ) , hybrid CRF+neural models Huang et al . ( 2015 ) ; Ma and Hovy ( 2016 ) ; Chiu and Nichols ( 2016 ) ; Ye and Ling ( 2018 ) or purely neural models Collobert et al . ( 2011 ) ; Devlin et al . ( 2018 ) . To take into account negative examples that are highly similar to positive ones , we use the idea of distant supervision to first retrieve the set of hard-negative dataset Dhard-neg . Akin to the text classification task , we introduce a new label zi ∈ ( B , I , O ) , indicating whether the current token belongs to Dhard-neg . To incorporate the collected hard-negative examples into the model , again we introduce an auxiliary objective of assigning correct z labels to different tokens : L2 =− logP ( z1 : nD |x1 : nD ) ( 6 ) Similar to the text classification task , we also want to separate hard negative examples , easy negative examples and positive examples , so we associate each example with a label l. We thus have distinct “ outside ” and “ beginning ” labels for positive examples and hard-negative examples , i.e. , l ∈ ( Bpos , Ipos , Bhard-neg , Ihard-neg , O ) . Labels for different categories regarding y and z are shown in Table 2 . The final objective function is thus as follows : L = λ1L1 + λ2L2 + λ3L3 ( 7 ) Again λ1 + λ2 + λ3 = 1 . At the training time , the two functions are simultaneously trained . At the test time , we only use P ( yi|x1 : nD ) to predict yi as the final decision . For CRF-based models Huang et al . ( 2015 ) ; Ma and Hovy ( 2016 ) ; Chiu and Nichols ( 2016 ) , neural representations are fed to the CRF layer and used as features for decision making . As in Ma and Hovy ( 2016 ) , neural representation hxi is computed for each token/position using LSTMs and CNNs , and then forwarded to the CRF layer . The key issue with CRF-based models is that the CRF model is only able to output one single label . This rules out the possibility of directly feeding hx to three readout layers to simultaneously predict y , z and l. We propose the following solution : we use three separate CRFs to predict y ( for L1 ) , z ( for L2 ) and l ( for L3 ) , denoted by CRFy , CRFz and CRFl . The three CRFs use the same hidden representation hx obtained from the same neural model as inputs , but independently learn their own features weights . We iteratively perform gradient descent on the three CRF s , and the error from the three CRF s are back-propagated to the neural model iteratively .

Summary: This paper is aimed at tackling a general issue in NLP: Hard-negative training data (negative but very similar to positive) can easily confuse standard NLP model. To solve this problem, the authors first applied distant supervision technique to harvest hard-negative training examples and then transform the original task to a multi-task learning problem by splitting the original labels to positive, hard-negative, and easy-negative examples. The authors consider using 3 different objective functions: L1, the original cross entropy loss; L2, capturing the shared features in positive and hard-negative examples as regularizer of L1 by introducing a new label z; L3, a three-class classification objective using softmax.