DSReg: Using Distant Supervision as a Regularizer
Authors: Yuxian Meng
Muyu Li
Xiaoya Li
Wei Wu
Fei Wu
Jiwei Li
Venue:  (2020)

Abstract: In this paper, we aim at tackling a general issue in NLP tasks where some of the negative examples are highly similar to the positive examples, i.e., hard-negative examples). We propose the distant supervision as a regularizer (DSReg) approach to tackle this issue. We convert the original task to a multi-task learning problem, in which we first utilize the idea of distant supervision to retrieve hard-negative examples. The obtained hard-negative examples are then used as a  regularizer, and we jointly optimize the original target objective of distinguishing positive examples from negative examples along with the auxiliary task objective of distinguishing soften positive examples (comprised of positive examples and hard-negative examples) from easy-negative examples. In the neural context, this can be done by feeding the final token representations to different output layers. Using this unbelievably simple strategy, we improve the performance of a range of different NLP tasks, including text classification, sequence labeling and reading comprehension. 

Under review as a conference paper at ICLR 2020
DSREG: USING DISTANT SUPERVISION AS A REGU- LARIZER
Anonymous authors Paper under double-blind review
ABSTRACT
In this paper, we aim at tackling a general issue in NLP tasks where some of the negative examples are highly similar the positive examples, i.e., hand-negative examples). We propose the distant supervision as a regularizer (DSReg) approach to tackle this issue. We convert the original task to multi-task learning problem, in which we first utilize the idea of distant supervision to retrieve negatieeative examples. The obtained hard- -egaaive examples are then used as regularizer, and we jointly optimize the original target objective of distinguishing positive examples from negative examples along with the auxiliary task objective of distinguishing soften positive examples (comprised of positive examples and hard -ergtivetevve ex- amples) from easy-nege eesy-veee examples. In the neural context, this can be done by feeding the final token representations to different output layers. Using this unbe- lievably simple strategy, we improve the performance of range of different NLP tasks, including text classification, sequence labeling and reading comprehension.
INTRODUCTION
Consider the following sentences in text-classiffcation task, in which to identify text describing hotels with good service/staff (as depicted as aspect-level sentiment classification in Tang
al. (2015); Li al. (2016); Lei et al. (2016)):
$1: the staff are great. (positive)
$2: the location great but the staff are surly and unhelpful... (hard-negative)
S3: the staff are surly and unhelpful. (easy-negative)
S1 is positive example since it describes hotel with good staff. Both S2 and S3 negative because staff are unhelpful. However, since $2 is lexically and semantically similar with S1, standard models be easily confused. As another example, in reading comprehension tasks like Narrative Kočiský et al. (018). truth answers are human-generated ones and might not have exact matches in the original passage. A commonly adopted strategy is to first locate similar sequences from the original passage using a pre-defined threshold (using metrics like ROUGE-L) and then treat them positive training examples. Sequences that are semantically similar but right below this specific threshold will be treated as negative examples and will thus inevitably introduce massive labeling noise in training. This problem is ubiquitous in a wide range of NLP tasks, i.e., some of the negative examples are highly similar to positive examples. We refer to these negative examples hard-negative examples for the rest of this paper. Similarly, the negative examples that similar to the positive examples are refered to as easy-negative examples. Hard-negative examples can cause big trouble in model training, because the nuance between positive examples and hard-negative examples can cause confusion for a model trained from scratch. To make things worse, when there is blass-balance problem where the number of negative examples are overwhelmingly larger than that of positive examples (which is true in many real-world use cases), the model will be at loose ends because positive features are buried in the sea of negative features.
To tackle this issue, we propose using the idea of distant supervision (e.g., Mintz et al.; (2009); Riedel et al. (2010)) regularize training. We first harvest hard-negative examples using distant supervision. This process can be done by a method as simple as using word overlapping metrics (e.g., ROUGE, BLEU whether sentence contains some certain keywords). With the harvested hard-negative examples, we transform the original binary classification setting to a multi-task learning setting, in which we jointly optimize the original target objective of distinguishing positive examples

---

Under review as a conference paper at ICLR 2020
from negative examples along with auxiliary objective of distinguishing soften positive examples (comprised of positive examples and hard-negative examples) from negativee examples. For a neural network model, this goal can be easily achieved by using different output layers to readout the final-layer representations. In this way, the features that are shared between positive examples and hard negative examples can be captured by the model. Models can easily tell which features that can distinguish positive examples the most. Using this unbelievably simple strategy, we improve the performance of in range of different NLP tasks, including text classification, sequence labeling and reading comprehension.
The key contributions of this work can be summarized as follows:
We study a general situation NLP, where a subset of the negative examples are highly similar to the positive examples. We analyze why it is a problem and how to deal with it.
We propose a general strategy that utilize the idea of distant supervision to harvest hard- negative training examples, and transform the original task to multi-task learning problem.
The strategy is widely applicable for a variaty of tasks.
Using this unbelievably simple strategy, we can obtain significant improvement on the tasks of text-classification, sequence-labeling and reading comprehension.
RELATED WORK
Distant Supervision Mintz et al. (2009); Riedel et al. (2010); Hoffmann al. (2011); Surdcanu ct al. (2012) It is proposed to address the data sparsity issue in relation extraction. Suppose that wish to extract sentences expressing the ISCAPITAL relation, distant supervision augments the positve training set by first aligning unlabeled text corpus with all entity pairs between which the ISCAPITAL relation holds and then treating all aligned texts positive training examples. The idea has been extended to other domains such as sentiment analysis Go et al. (2009), computer security event Ritter et al. (2015), life event extraction Li et al. (2014) and image classification Chen and Gupta (2013). Deep leaning techniques have significantly improved the results of distant supervision
for relation extraction Zeng et al. (2017): )); Luo et al. (0017) Lin et al. (2017); Toutanova et al. (2015).
Multi- Learning (MTL) The idea of using data harvested via distant supervision as auxiliary supervision signals is inspired by recent progress on multi-task learning: models for auxiliary tasks share hidden states or parameters with models for the main task and act as regularizers. In addition. neural models often celebrate performance boost when jointly trained for multiple tasks Collobert itt al. [201T); Chen et al.] (2017); Hashimoto et al. (017); y; FitzGerald et al. (015)). For instance, Luong et al. 2015)) use sequence- teee-tto-s-uuuee sequence model jointly train machine translation, parsing and image caption generation models. Dong et al. (2015) adopt an alternating training approach for different language pairs, i.c., they optimize each task objective for fixed number of parameter updates (or mini-batches) before switching to a different language pair. (Swayamdipta et al. (2018) propose using syntactic tasks to regularize semantic tasks like semantic role labeling. |Hashimoto et al. (017) improve universal syntactic dependency parsing using a multi-task learning approach.
MODELS
In this section, we discuss the details of the proposed model. We focus on two different types of NLP tasks, text classification and sequence labeling.
3.1 TEXT CLASSIFICATION
Suppose that we have text-label pairs D ( vi). I; consists of sequence of tokens x;; (wi,1, W;.2, Wi,n }. where n; denotes the number of tokens in r;. Each text x; is paired with a binary label y; E 1). The training set can be divided into positive set D and negative set D= Let &J denote the model prediction. The standard training objective can be given as follows:
4 =
Σ (E.DJ)D
log P(e: 2x
Σ
log P(ùs = 1/x:))
Σ (x.D:)ED
log P(ù: = 0)
(
2

---

Under review as a conference paper at ICLR 2020
z1
y-
211
E:
y:0
z:0
60
softmuu (Wa
softmax Ks)
Softmax (w)
softmax (,
softmax Wihh)
softmax (W, Wy softmax Wih,)
softmax W A/
softmax (W))))
Sentence Representations
Sentence Representations
Sentence Representations
W
W4
W3
W4
Wg
Wg
Positive Examples
Hard- -aadt-ee Examples
Easy-Negative Examples
Figure illustration the classification model with supeevisu regularizier.
group positive examples hard-negative examples
notation
Z
D
1
1
Dhard neg
1
2
casy-negative examples
Dhard
0
Table 1: Labels for different instances in binary classification tasks.
Let Daand neg denote the hard-negative examples retrieved using distant supervision. Here we introduce new label z: z 1 for instances in Dhard-nog D+, and z 0 for instances in D Dhard-neg- We regularize Lı using an additional objective L2:
L2 (DD.S)DD
Σ
log P(2, 1[r;))
log P(2, = 0 0(2;))
(2)
(x..5.)D Drundrrr
L2 be thought objective capture the shared features in positive examples and hard- negative examples. Equ[ can also extended to another similar form, distinguishing between Dhart-pos UD- (i.e., the union of positive examples that similar to negative and negative examples) and D D pas.
Empirically, also find that adding one more three-class classification objective L3, which separates positive vs nard-negative easy-negative introduces additional performance boost. We suggest that adding this three-clas. classification will additionally highlight the difference between hard negative examples and easy negative examples for the model. The label is denoted by I, where | - 0 for easy negative examples, 1 for positive examples and for hard negative examples. This leads the final objective function at test time to be
(3) where A + A2 + A3 1 are used control the relative importance of each loss. For neural classification model, p(2[2), p(y|z) and p[[[z) share the same model structure. The input text is first mapped dd-dimensional vector representation h, using suitable contextualization strategy, such LSTMs Hochreiter and Schmidhuber (1997). CNNs Kim (2014) or transformers Vaswani] al. (2017). Then h fed three fully connected layers with softmax activation function to compute x()(=), p(z(z:) and p(1]z) respectively:
-= L1 + A2 + >aLas
p(y|=> softmax (W (Wyht.))
p(2) x = softmax(W> (Whh))
(4)
p(1]z:) softmax(Wih..)
where W,, W, R2xd, W. R3xd.
3.2
SEQUENCE LABELING
In sequence labeling tasks I Lafferty et al. (2001) Ratinov and Roth (2009); Collobert et al. (201 l); Huang al. 2015): Ma and Hovy (2016): Chiu and Nichols (201)), model is trained to assign

---

Under review as a conference paper at ICLR 2020
group
notation
y
B of positive examples positive examples
D
B
B
B1
D+
I
I
I
of nega-iegative examples hard negative examples
Dhurdaag
0
B2
Dhar-aeg
0
1
1g
Drand-ny
0
0
O
Table 2: Labels for different instances in sequence labeling tasks. short for beginning and short for inside.
labels to each of the tokens in a text sequence. Suppose that are to assign labels all tokens in chunk text = { I2,, Tnp) where n denotes the number tokens in D. Let us consider simple case where we only have one type of tag and we will use the standard IOB (short for inside, outside, beginning) sequence labeling format. In this case, it a three-class classification problem, assigning y/4 E (B, I, O) to each token. We treat tokens with label B and / as D+ and tokens with The objective function for the vamilla sequence labeling task given as follows:
label O as
L1 = log P( (1mmp)
P(yt:n Fi:mp be computed using standard sequence tagging models such CRF Lafferty ct al. ). hybrid CRF+neural models Huang et al. (2015); Ma and Hovy (2016); Chiu and Nichols (2016); Ye and Ling (2018) or purely neural models Collobert et al. (2011): Devlin et al. (2018).
B2
12
12
12
CRFY
CRF:
CRFI
W2
W3
W4
Hard HegdtNeettivee Examples
Figure 2: An illustration of the CRF tagging model with the distant-superrision regularizer.
To take into account negative examples that are highly similar to positive ones, we use the idea of distant supervision to first retrieve the set of hard-negative dataset Dhand-neg- Akin to the text classification task, we introduce new label 2; (B, O), indicating whether the current token belongs to Dhand-neg- To incorporate the collected hard-negative examples into the model, again we introduce an auxiliary objective assigning correct = labels to different tokens:
L = log P(zimp (1isp)
(6)
Similar to the text classification task, we also want separate hard negative examples, easy negative examples and positive examples, so we associate each example with a label We thus have distinct "outside" and "beginning" labels for positive examples and hard-negativ examples, i... I e (Bpos Apos- Bhard-ocg Ihard-nog O). Labels for different categories regarding y and z are shown in Table 2 The final objective function is thus follows:
L= L + A2L2 + Ag
Again \₁ + A2 + A3 = 1, At the training time, the two functions are simultancously trained. At the test time, we only use P(y:(=imp) predict y; as the final decision.
For CRF-based models Huang et al. (2015): Ma and Hovy 2016); Chiu and Nichols (006)). neural representations fed to the CRF layer and used as features for decision making- As Ma and Hovy (2016), neural representation /., is computed for each token/position using LSTMss and CNNs, and then forwarded the CRF layer. The key issue with CRF-based models is that the CRF model is only
4

---

Under review as a conference paper at ICLR 2020
able to output single label. This rules out the possibility of directly feeding h to three readout layers to simultaneously predict y, 2 and l. We propose the following solution: we use three separate CRFs to predict y (for L1). (for L2) and I (for L3). denoted by CRFy, CRF,, and CRFi. The three CRFs use the same hidden representation hg obtained from the same neural model as inputs, but independently learn their own features weights. We iteratively perform gradient descent on the three CRF's, and the error from the three CRF's s are back propaaatd to the neural model iteratively.
EXPERIMENTS
4.1 TEXT CLASSIHICATION
For the text classification task, we used three datasets, the Stanford Sentiment Treebank dataset (SST) Socher et al. (2013), the hotel review dataset for aspect-specific sentiment analysis in Lei et al. (2016)
and the financial statement dataset that we created.
The Stanford Sentiment Treebank (SST) Dataset associates each phrase within a sentence with sentiment label. The task is originally further divided into two-class paarsegraiined classification and five-class fine-grained classification. We only report binary classification results. For baseline, used BaRT-large Devlin al. (018)). The representation for CLS is output to the softmax layer for classification. We used distant supervision to retrieve hard-negative examples: similar to Mintz al. (2009), we employed a simple strategy, in which we treat negative reviews that contain positive sentiment lexicons as hard-negative examples. Positive sentiment words are retrieved using the MPQA corpus Wilson et al. (005)).
The Hotel Review Dataset contains roughly 50,000 reviews with an average length of 120 words. Each review contains ranking scores (integers from I to 5) for different aspects of the hotel, such as service, cleanliness, location, rooms, etc. The dataset constructed in a way that each review might contain diverse sentiments towards different aspects, and it is interesting to see how model manages or fails to identify these different aspects and their associated scores when entangled with other aspects. Following Li et al. (2016), the task is divided into four sub-tasks, each of which classifies the sentiment of one of the following four aspects: we focus on four aspects: value, rooms, service and location. We filtered neutral reviews (with score 3) and treat those with score 1 and 2 as negative ones, 4 and as positive one. The task is thus transformed to a binary classification task. The same review can thus carries positive sentiment regarding one aspect, but negative for the other. We report average accuracy for the four tasks. For fair comparison, we used the baseline in Li et al. (2016). The model combines Bi-LSTMs with neemory-netwood structure Sukhbaatar et al. (2015) at both the word level and the sentence level to obtain document-leve representations, which are then fed to a sigmoid function for binary classification. Hard- sard-negative examples retrieved is the same way as in the SST datasct.
The Financial Statement Dataset is a dataset created by us. It consists 97,736 sentences, each of which contains labels for the values of financial statement items (FSI) for individual businesses or services from annual reports of listed companies (Statistics shown in Table 6). The goal is to extract:
the [value] of [which financial statement item] of [which business or service] of listed company
The dataset can be used for both text classification and sequence labeling. For the task of text classification, the goal to identify whether a sentence contains useful FSI information, i.e., whether a sentence contains the FSI of interest and the corresponding value. Sentences with annotated financial statement items and the corresponding values as positive examples, and the rest are negative examples.
We used the following distant-supervision pattern to retrieve hard-negative examples: treat texts in negative examples that contain more than one mention of financial statement items as hard- -egatvee examples. For example, the sentence Benefiting from the decline in raw material costs and the further improvement of the company's 's management skills, the profit of the company increased during the reporting period is hard-negative example. Particularly, it is negative example since it does not specifically indicate the value of the profit and thus is not of interest. But this sentence is a hard-negative one because it contains the FSI keyword profit.
5

---

Under review as a conference paper at ICLR 2020
Model
SST
Hotel Review
Financial Statement
L1 Posu Hoo_HardNNg/FasyyNee - Pos|HardNCe
81.5
86.2
88.4
82.4
86.9
88.0
L3
79.2
85,8
81.3
Li L2
82.7
87.9
89.4
L1
Ls
82.4
87.1
88.5
L + L2 Ls
82.9
88.0
90.8
Table 3: Performances of different models the text classification task.
Bi-LSTM+ + CRF
Model
R
F
L1
82.06
83.26
82.65
La + L2
82.41
85.51
83.93
L1 + LB
82,22
85.14
83.65
L L2 L3
83.92
84.97
84.44
Table 4: Performances of different models on the sequence labeling task on the Financial Statement Dataset.
For the ease notations, we use Pos and Neg denote positive and negative examples, HardNeg to denote hard-negative examples and EasyNeg to denote easy-negative examples. We compare performances of the following models:
L1 the vanilla classification model to distinguish between positive examples (Pos) and negative examples.
HasddNee EasyNeg + Pos HardNeg: a hierarchical model that involves two states: 1) distinguishing between easy-negative (EasyNeg) and the union of positive and herd-negative (PosU/ardNNe 2) distinguishing positive (Pos) from hard-negative (HardNeg) examples.
L3: three-class classification model to distinguish Pos, HardNeg and EasyNeg.
L1 + Lg: the proposed multi-task learning model that jointly trains two objective functions: distinguishing Pos from Neg and positivetcasy negative HardNee) from hard negative (EasyNeg) examples.
L + Lz: combining the standard classification objective with the three class classification objective.
L1 + L2 + L3: combining the three.
Results Table 3 shows results for text classification tasks. As can be seen, three versions of the proposed DSReg models, i.e., T1 L2, Jn I-3 and I1 T.2 Lg. outperform the binary classification model and the pipelined model, which aligns with our expectation. For the pipelined model, since the eтror accumulates over stages, it underperforms not only the DSReg models, but also the binary classification model. By using both the three-class classification (Lg) and PosU Hard/a EasyNeg (L2) as regulations, the L1 La La setting leads the best performance.
4.2 SEQUENCE LABELING
For the sequence labeling task, again we used the Financial Statement Dataset. The task can be transformed to assigning 1OB (short for inside, outside, beginning) of each label category to each word. Labels extract include financial statement items (FSI), unit, value, the change of FSI (FSI- change), time, business & service (B&S), bases of comparison (BoC), as illustrated in the following example:
The (O) revenue (B-FSI) of education (B-B&S) sector (I-B&S) increased (B-FSI-change) by 25% (B-value) over (O) the (B-BoC) same (I-BoC) period (I-BoC-I) of (I-BoC) the (I-BoC) previous (I-BoC) year (I-BoC). (O)
We used the keyword matching strategy to retrieve hard-negative examples. We report statistics at the word-level Table/4 The three-class classification model (Lg) Pos| HardNeg EasyNeg and the pipelined model PosUHarddeegl /Fayyeee - Pos| HardNeg significantly underperform the others, and their results are omitted due to space limitations. From Table we can see that the proposed La + Ta2 La model significantly outperforms the Pos|Neg baseline by (+1.79).

---

Under review as a conference paper at ICLR 2020
4.3
READING COMPREHENSION
Model
Standard L1
DSReg (L1 + L2)
DSReg (L1 L2 Ls)
Human
summary: valid/test
BLEU-1
33,45/33.72
34,89/34,90
35.12/35.02
44.24/44.43
BLEU-4
15.69/15.53
16.89/16.89
17.17/17.14
18.17/19.65
Meteor
15.68/15.38
17.05/16.72
17.21/16.84
23.87/24.14
ROUGE-L
36.74/36.30
38.40/37.66
38.55/37.90
57.17/57.08
Model
Standard
DSReg (L1 + L2) DSReg (L1
+ L2 Ls)
Human
full version: valid/test
BLEU-1
5.82/5.68
7.60/7.55
7.81/7.77
44.24/44.43
BLEU-4
0.22/0.25
0.35/0.41
0.37/0.37
18.17/19.65
Meteor
3.84/3.72
5.17/5.02
5.22/5.05
23.87/24.14
ROUGE-L
6.33/6.22
7.40/7.17
7.66/7.22
57.17/57.02
Table 5: Results on the NarrativeQA datasct.
The narrativeQA dataset Kočiský et al. (2018) consists of 1,567 stories with 46,765 question-answe answer pairs. Following Kočisky et al. (2018), conduct experiments on both the summary setting and the full version setting. For the summary setting, answer spans need to be extracted from story summaries, and for the full version setting, they need be extracted from the entire books movie scripts. We followed the routines of the neural benchmarks in Kočiský et al. (2018), in which we first retrieve relevant chunks from the story using IR system. Then concatenate the selected chunks. Since the answer for each query is manually annotated. large proportion of answers do not have corresponding spans in the original passage that be exactly matched. In Kočiský et al. 2018). the span that achieves the highest ROUGE-L score with respect to the reference answer are used as gold spans. The start and end indices are predicted using BiDAF Sco et al. (0016). As in Kočiský ict al. (2018), spans with highest ROUGE-L scores are treated as positive examples. Suppose that for gold answer a, text span a' has the highest ROUGE-L score of ROUGE(a, a') and thus treated as the positive training example. Since documents and passages in this task are very similar, there are many text spans that highly similar the gold answer. These spans are treated examples. Specifically, we treat spans whose ROUGE-L scores are greater than a x ROUGE a') as hard- negativee examples, where (0, 1) is the parameter to tune on the development set.
hard-negative
We followed the criteria in Kočiský et al. (018 to train a vanilla BiDAF model. We used the splits of positive examples, hard-negative examples and negatiee examples to train DSReg models, using both the three-class classification and the (positivc+hard -neg) vs, casy-neg as regulations. We report scores for BLEU- 1, BLEU-4 Papineni al. (2002), Meteor Denkowski and Lavic (201T) and ROUGE-L scores Lin (004).. Results are shown in Table When combined with BiDAF, the proposed DSReg model outperforms the standard BiDAF model in both settings. As discussed in Kočisky et al. (2018), the span prediction model performs the best in the summary setting. This sets new sfate-of-the-as results for the summary setting the NarrativeQA dataset.
ABLATION STUDY AND VISUALIZATION
5.1 THE EFFECT OF L2: POSUHARDNEG EASYNEG
We examine the influence of the parameter À at the interval of 0.1 in the objective L - + on the hotel review dataset. It is worth noticing that À cannot take the value of here, since not able to make decisions purely based on Lg. Results are shown in Figure The performance first goes up when À is smaller than 0.4, and then declines dramatically. It accords with expectation that these losses are complementary to each other.
5.2 THE EFFECT OF Ly: Pos HARDNEG EASYNEG
We examine the influence of the parameter À the interval 0.1 in the objective -= (1- +AL3 on the hotel review dataset. Results are shown in Figure 3 Best performance is obtained when is set to 0.3.
7

---

Under review as a conference paper at ICLR 2020
90.50
Effect of L.2
87.75
Effect of L3
89.00
87.50
86
87
87.9
86.00
85 0
87.00
20
84.50
80.75
86
80 2
86.50
81.50
86.25
86
80.00
86.00
86
78.50
702
85.75
75.50
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
86.25
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Figure 3: Effect of L2 and La.
0.9
0.8
Vanille-11
S(neg, E)
0.7
0.6
DSREG-L1
S(neg, E)
0.5
SiposuhardNeggcc E)
0.4
DSREG-L3
S/hardNeg,
like
the
movie
though
plot
boring
0.1
0.9
0.8
Vanilla11
S(neg, E)
DSREGILI
Sineg. EI
0.6
DSREG-12
SiposthardNeg. E)
0.
SthardNeg.
very
surly
Figure Saliency maps from different models.
5.3 VISUALIZATION
For visualization, Saliency is a commonly adopted visualization technique to see how much each input unit contributes to the final decision using first-order derivatives Erhan et al. (2009); Li et al. (2015); (2015):(Smma [Smoonyan et al.) (2013). Given word embedding E = {e}. where denotes the value each embedding dimension, and class label the trained model associates the pair (e, c) with log likelihood S(c, e). For neural models, the class score S'c, e) a highly non-linear function. S(c, e) is approximated with linear function of by computing the first-order Taylor expansion
FIRST DERIVATIVE SALIENCY to unveil why DSReg helps. First Derivative
S(c, w(c, e
where w(c, e) is the derivative of S(c, e) with respect to the embedding dimension e.
w(c,e aS(c,e)
de
The magnitude (absolute value) of the derivative indicates the sensitiveness of the final decision to the change in particular dimension, telling how much specific dimension of the word
8

---

Under review as a conference paper at ICLR 2020
embedding contributes to making the decision c. The saliency score of specific word E is the average the absolute value of ec:
S(c, E)
= E
1
Σ [w(c,e)l RE
(10)
We use the example "I hate the movie though the plot is interesting" in SST and "Location is nice. Staff very surly and helpful" in aspect sentiment identification show the working mechanism of different constituents in DSREG to illustrate why works better. Both examples consist of two clauses with opposite sentiments, which inevitably makes model confused.
Figure 4 shows saliency maps regarding each word output from different models. Vanilla denotes the baseline classification model, the objective of which consists of only L1. As can be seen from S(pos, E) of Vanilla-L.1, though the model emphasizes more on hate in the first clause, but attaches significant amount of importance "interesting" in the second clause. This issue is largely alleviated in S(pos, E) of DSREG-LI, which offers a clearer focus on the first clause than the second one. The reason why this happens be explained by the saliency map from S(pos_harddNe,, E) of DSREG-LI: the example is hard-negative (it is negative but contains positive lexicon "interesting") and the sentiment from the distracting word "interesting" is captured in the JhadN,, E), making S(pos, E) more easily focus on the truly positive clause.
CONCLUSION
In this paper, we tackle general problem NI.,, i.e., the situation in which subset of the negative examples are highly similar to the positive ones. We propose the DSReg: model that utilizes distant supervision as a regularizer. We transform the original task to a multi-task learning problem, in which we first utilize distant supervision to retrieve hard-negative examples, which are then used as a regularizer. We show that the proposed strategy lead to significant performance boost text classification, sequence labeling and machine reading comprehension tasks.
REFERENCES
Xinchi Chen, Zhan Shi, Xipeng Qiu, and Xuanjing Huang. 2017. Adversarial multi-criteria learning for chinese word segmentation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguisties, ACL 2017, Vancouver, Canada, July 30 August 4, Volume 1: Long Pupers, pages 199--1203.
Xinlei Chen and Abhinav Gupta. 2015. Webly supervised learning of convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 1431-1439.
Jason P. C. Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional Istm-cnns. TACL, 4:357-370.
Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. 1. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12(Aug):2493-255) -5377.
Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the sixth workshop statistical machine translation, pages 85-91. Association for Computational Linguistics.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 11x.v:1810.04805..
Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-task learning for multiple language translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing ( (Volume I: Long Pupers), volume 1, pages 1723-1732.
Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of deep network. University of Montreal, 1341(3):1.

---

Under review as a conference paper at ICLR 2020
Nicholas FitzGerald, Oscar Täckström, Kuzman Ganchev, and Dipanjan Das. 2015. Semantic role labeling with neural network factors. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 960-970.
Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford, 1(12).
Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. 2017. A joint many-task model: Growing neural network for multiple NLP tasks. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen. Denmark, September 9-11, 2017.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
Raphael Hoffmann, Congle Zhang. Xiao Ling, Luke Zettlemoyer, and Daniel S Weld. 2011 1. Knowledge-baad based weak supervision for information extraction of overlapping relations. In Pro- ceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies- Volume I. pages 541-550. Association for Computational Linguistics.
Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional Istm-erf models for sequence tagging. arXiv preprint
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT. Special Interest Group of the ACL, pages 1746-1751.
Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gáabor Melis, and Edward Grefenstette. 2018. The narrativeqa reading comprehension challenge. Transactions of the Association of Compulational Linguistics, 6:317-328.
John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference Machine Learning (ICML 2001), Williams College, Williamstown, MA, USA, June 28 July 1, 2001.
Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. arXiv preprint arXiv:1606.04155
Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Jurafsky. 2015. Visualizing and understanding neural models in nlp. arXiv preprint 1506. 01066.
Jiwei Li, Will Monroe, and Dan Jurafsky. 2016. Understanding neural networks through representation erasure. arXiv preprint arXiv: 1612.08220.
Jiwei Li, Alan Ritter, Claire Cardie, and Eduard Hovy. 2014. Major life event extraction from twitter based on congratulation conoleessoolssseess speech acts. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1997-2007
Chin-Yer Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out.
Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2017. Neural relation extraction with multi-lingual attention. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 34 43.
Bingfeng Luo, Yansong Feng, Zheng Wang. Zhanxing Zhu, Songfang Huang, Rui Yan, and Dongyan Zhao. 2017. Learning with noise: Enhance distantly supervised relation extraction with dynamic transition matrix. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 430-439.
August 4, Volume 1: Long Papers, pages
Minh-Thang Luong, Quoe V Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2015. Multi-task sequence sequence learning. arXiv preprint arXiv: 1.06/ 14.
10

---

Under review as a conference paper at ICLR 2020
Xuezhe Ma and Eduard H. Hovy. 2016. End-to-end sequence labeling via bi-directional istm-enns-crf. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers.
Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Anmual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2- Volume 2, pages 1003-1011. Association for Computational Linguistics.
Kishore Papineni. Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting association for computational linguistics, pages 311-318. Association for Computational Linguistics.
Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning, pages 147-155. Association for Computational Linguistics.
Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Joint European Conference Machine Learning and Knowledge Discovery in Databases, pages 148-163. Springer.
Alan Ritter, Evan Wright, William Casey, and Tom Mitchell. 2015. Weakly supervised extraction of computer security events from twitter. In Proceedings of the 24th International Conference on World Wide Web, pages 896-905. International World Wide Web Conferences Steering Committee.
Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. arXiv preprint 1rXiv:161..0160.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint #rXiv:13/2.6034.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631-1642.
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 2015. End-to-end memory networks. In Advances in neural information processing systems, pages 2440-2448.
Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D Manning. 2012. Multi- instance multi-label learning for relation extraction. In Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, Association for Computational Linguistics.
pages 455-465.
Swabha Swayamdipta, Sam Thomson, Kenton Lee, Luke Zettlemoyer, Chris Dyer, and Noah A. Smith. 2018. Syntactic scaffolds for semantic structures. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, Brussels, Belgium, October November 4, 2018.
Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1432-1432.
Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015. Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 Conference Empirical Methods in Natural Language Processing, pages 1499-1509
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998-6008
Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing.
11

---

Under review as a conference paper at ICLR 2020
Zhi-Xin Ye and Zhen-Hual Ling. 2018. Hybrid semi-markov CRF for neural sequence labeling. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne. Australia, July 15-20, 2018, Volume 2: Short Papers, pages 235-240.
Wenyuan Zeng. Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2017. Incorporating relation paths in neural relation extraction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 1768-1777.
APPENDIX
Tag
examples income; revenue
of instances
financial statement items (FSI)
6543
unit
ton; Kilowatt hour
5622
value
5:100,000 25%
9073
bases of comparison (BoC) the change of FSI (FSI-change) business & service (B&C)
the same period of last year increase; decrease education sector, loan service
2585
2821
1617
Table 6: Details for labels of the financial statement item dataset.
12


Reviews:
[{"review_id": "rkeE7qr0YH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper is aimed at tackling a general issue in NLP: Hard-negative training data (negative but very similar to positive) can easily confuse standard NLP model. To solve this problem, the authors first applied distant supervision technique to harvest hard-negative training examples and then transform the original task to a multi-task learning problem by splitting the original labels to positive, hard-negative, and easy-negative examples. The authors consider using 3 different objective functions: L1, the original cross entropy loss; L2, capturing the shared features in positive and hard-negative examples as regularizer of L1 by introducing a new label z; L3, a three-class classification objective using softmax.\nThis authors evaluted their approach on two tasks: Text Classification and Sequence Labeling. This implementation showed improvement of performance on both tasks.\n\nStrenghts:\n+ the paper proposes a reasonable way to try to improve accuracy by identifying hard-negative examples\n+ the paper is well written, but it would benefit from another round of proofreading for grammar and clarity\n\nWeaknesses:\n- performance of the proposed method highly depends on labels of hard-negative examples. The paper lacks insight about a principled way to label such examples, the costs associated with such labeling, and impacts of the labeling quality on accuracy. The experiments are not making a convincing case that similar improvements could be obtained on a larger class of problems.\n- The objective function L3 is not well justified.\n- It would be important to see if the proposed method is also beneficial with the state of the art neural networks on the two applications. \n- Table 3 (text classification result) does not list baselines.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "r1x4lqk0Yr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a novel approach to leverage Distant Supervision for discriminating between positive examples and \"negative examples that share salient features with the positive ones.\" In spite of its simplicity, the method appears to be quite promising.\n\nThe main feedback for the authors is to describe \"early & in detail\" the distant supervision techniques used in the experiments. The paper would be greatly improved by adding:\n- an intuitive paragraph in the intro that explains a concrete example of DS high level, but with enough details for the reader to grasp the idea)\n- adding a new section right after related work (and before the current \"3. Models\") in which you present in great detail (and with concrete & complete examples) the two main DS techniques that are used in the experiments; with that solid understanding \n\n\nOther comments:\n- for sake of simplicity & understand-ability, you should avoid the use of the term \"soften positive examples\"  in the abstract\n- avoid using the term \"unbelievable\" (one in abstract & twice in intro\"\n- the last paragraph before \"Conclusion\" seems to refer to an earlier version of Figure 4, which, in its current form, does NOT \nhave S(pos,E) or the word \"interesting\"", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "r1lGMTihtr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to improve performance of NLP tasks by focusing on negative examples that are similar to positive examples (e.g. hard negatives). This is achieved by regularizing the model using extra output classifiers trained to classify examples into up to three classes: positive, negative-easy, and negative-hard. Since those labels are not provided in the original data, examples are classified using heuristics (e.g. negative examples that contain a lot of features predictive of a positive class will be considered as hard-negative examples), which are used to provide distant supervision. This general approach is evaluated on phrase classification tasks, one information extraction task, and one MRCQA task.\n\nAlthough the proposed approach is interesting, this paper has several weaknesses (i) the method is not sufficiently justified or analyzed; (ii) there are missing links with previous work (notably on domain adversarial training); (iii) experimental setting is rather weak.\n\n1) About the justification of the approach:\n1.1) I feel like the proposed are not intuitively justified enough. They point out that \"L_2 can be thought as an objective to capture the shared features in positive examples and hard-negative examples\". Why would that be good from an intuitive perspective ?\n1.2) L_3 is forcing the model to group all the hard-negative examples together. Do you have an intuition why would that be useful ?\n1.3) What happens if the model overfits the hard negative examples in the training set ? This would mean that it has captured some features that can distinguish positive / negative label. Why would L_3 help in that case ?\n\n2) About related approaches:\n2.1) How does this method relate to domain adversarial training applied to positive and hard-negatives and adversarial examples in general ?\n2.2) Would similar performance be obtained by virtual adversarial training for example ?\n\n3) About the experimental setting:\n3.1) The performance reported is well below the use of recent work on these datasets and recent models such as BERT. Would these improvements carry over to bigger architectures ?\n3.2) In SST, the paper say they use BERT large, but the baseline performance (81.5) is well below BERT large performance in the original paper (94.9, https://arxiv.org/pdf/1810.04805.pdf). Why the mismatch ?\n3.3) What's the proportion of hard-negative examples mined for training and test set ? While the heuristics used seem reasonable, without those numbers, it is impossible to know if the heuristics truly predict hard-negative examples.\n3.4) Does the performance gain comes from better predicting hard-negative examples in the test set ? One could analyze the performance per error type (i.e. true positive, false negative, false positive (easy), false positive (hard)) with the baseline model and of the various proposed regularizing tasks (e.g. L_2 and L_3, both in training and test).\n3.5) The heuristics are used to pick what should be adversarial examples, but there is no mentions of this concept in the text. Oversampling those adversarial examples could, potentially, improve the performance of the baseline model. It would be interesting to try this.\n3.6) If possible, it would be good to add standard deviation to the results obtained running multiple runs.\n3.7) The visualization sub-section is anecdotal and not especially illuminating, and its text seems to refer to a different example than the figures (\"interesting\" is not in the figures.)\n\nMinor points:\n\n- Section 3 (Models) may be made shorter, the models used are utterly simple. This could free up space for more experiments.\n- In the tables, simply adding the name of the used model to the \"L1\" rows would be clearer.\n- The description of the pipelined results in Section 4.1 does not match the results shown in the table.\n- The citations are not well integrated with the text (\\citep vs \\cite), and the formatting of CRF changes in the last paragraph of Section 3.2.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a way to handle the hard-negative examples (those very close to positive ones) in  NLP, using a distant supervision approach that serves as a regularization.   The paper addresses an important issue and is well written; however, reviewers pointed put several concerns, including testing the approach on the state-of-art neural nets, and making experiments more convincing by testing on larger problems.
 

Decision: Reject