Joint Training with Semantic Role Labeling for Better Generalization in Natural Language Inference (2020)

Abstract: End-to-end models trained on natural language inference (NLI) datasets show low generalization on out-of-distribution evaluation sets. The models tend to learn shallow heuristics due to dataset biases. The performance decreases dramatically on diagnostic sets measuring compositionality or robustness against simple heuristics. Existing solutions for this problem employ dataset augmentation which has the drawbacks of being applicable to only a limited set of adversaries and at worst hurting the model performance on other adversaries not included in the augmentation set. Instead, our proposed solution is to improve sentence understanding (hence out-of-distribution generalization) with joint learning of explicit semantics. We show that a BERT based model trained jointly on English semantic role labeling (SRL) and NLI achieves significantly higher performance on external evaluation sets measuring generalization performance.

## introduction nli is the task of determining the inference relationship between a premise and a hypothesis sentence which is usually formulated as a threeclass classification task with entailment, contradiction and neutral labels. it has been regarded as a central problem in natural language understanding and found its place in benchmarks such as glue @xcite . contemporary neural network based models achieve state-of-the-art (sota) results on these benchmarks. however, scoring high on test sets that have a similar distribution to the training sets does not guarantee wider generalization. models that top the leaderboards on standard test sets may perform poorly on specifically constructed evaluation sets targeting dataset biases. for instance, the hans challenge dataset @xcite showed that models trained on nli get fooled easily by heuristics when the input sentence pairs have high lexical similarity. the following example from hans can explain how this might happen. premise: the judge by the actor stopped the banker. hypothesis: the banker stopped the actor. a human reading this sentence pair carefully can conclude that the hypothesis can not be inferred from the premise. however, models relying on the lexical overlap heuristic will be fooled and predict the label as entailment since the premise contains all words of the hypothesis. existing approaches commonly tackle such adversaries by training the model with a dataset augmented with similar adversarial examples. as detailed in section 5, the problem with this approach is that it might lead to overfitting to the adversaries on the augmentation set. therefore, it can decrease the performance on other possible adversaries and hurt generalization @xcite . semantic role labeling (srl) asks the "who did what to whom, when and where etc." questions to find the semantic roles of words or phrases in a sentence @xcite . we hypothesize that using the srl task as a joint objective should improve the semantic knowledge of the models, thus making them less prone to dataset biases. consider the semantic roles in the previous example sentence pair, which are shown in hypoth. the banker stopped the actor. ## datasets in this section, we describe the datasets used in the experiments. we explain the shortcomings of the nli datasets and describe an srl dataset that can be used to alleviate these shortcomings. ## model description we propose a multi-task bert model to jointly predict semantic roles and perform natural language inference. bert is used as the shared encoder module and two separate decoder heads are appended on top of it to perform task-specific operations. the overall picture of the model can be seen from figure 1. following @xcite , the tasks share the encoder part of the model including the lexicon encoder and all bert layers. we follow the original sentence pair classification formulation for bert while training it for nli task. on the input side, we concatenate the premise and hypothesis tokens, add a [sep] token at the end of both sentences, and finally add a [cls] token at the beginning of the whole sequence. figure 2 shows the token embedding for an example sentence pair. while processing an nli input, we take the [cls] token embedding at the bert's output and treat it as the summary of the whole sequence. the dimension of this embedding is reduced to cls] judge by banker . [sep] the the [sep] . banker input e[cls] ethe e banker e[sep] e . e . e judge e by e[sep] e the e banker token embeddings segment embeddings position embeddings e a e a e a e b e a e b e a e a e a e b e b e 0 e 1 e 8 e 17 e 9 e 16 e 2 e 3 e 10 e 11 e 12 three after passing through a two-layer mlp since there are three labels in multinli and snli. finally, a softmax is applied to get the probability for each label class. following mccoy et al. (2019b) , when we evaluate the model on hans or comparisons, we collapse the predicted contradiction and neutral labels into a single negative label to output binary labels. for the srl training, we adapt an architecture similar to the one proposed by @xcite . in our implementation, we indicate the predicate using the segment embeddings by assigning 1 to the predicate word pieces and 0 to the other tokens. figure 3 denotes the embeddings used for the srl. we opt for a simple decoder and rely purely on the self attention to capture contextual information. the embedding of each token is directly passed through a two-layer mlp independently, and the srl tag is determined by a final softmax layer. we use the inside outside beginning (iob) foot_0 tagging for spans, and a viterbi decoder to ensure prediction of valid spans during testing. as figure 2 and 3 show, the segment embeddings represent different things for srl and nli inputs. in nli, segment embeddings separate the sentences whereas they indicate the target verb in srl. moreover, how the bert outputs are processed is also different between the tasks. in spite of these differences, the training is expected to optimize the bert weights so that it can generate embeddings suitable for both tasks. intuitively, this representation will be less prone to dataset biases in the nli thanks to explicitly forcing the model to pay attention to the semantic roles of the words. ## experiments and results in this section, we present the experiments we conducted by training the bert based model in singletask and multi-task learning setups. multinli and snli datasets are used to train the models whereas hans and comparisons are used for evaluation. for the multi-task learning experiments, we used the srl dataset for joint training with nli. in both of the hans and comparisons experiments, we tuned the hyperparameters using a validation set from the same distribution as the adversarial test set. nevertheless, we also tested our highest performing models on the original multinli and snli validation sets to make sure our multi-task bert model performs well on those as well. indeed, the multitask model performance on the original validation sets turned out to be similar (accuracy difference is @xmath0.5%, depending on the hyperparameters) to the single-task bert performance. ## related work in this section, we discuss various solution approaches proposed for the nli task. we start with a sentence embedding based approach and continue with a data augmentation method targeting the generalization problem of nli models. then, we discuss some models benefiting from syntax or semantic roles and touch on multi-task models. some previous studies used sentence embedding based approaches to solve nli. noticeably, in-fersent @xcite uses an lstm to encode the premise and hypothesis sentences independently. then, it concatenates the premise, hypothesis embeddings and two feature vectors obtained by their element-wise multiplication and absolute difference to get the overall sentence pair representation. finally, an mlp layer followed by a softmax is used to calculate the class scores. being trained on snli, this model suffers from the biases in its training data and performed close to the bow model on the comparisons evaluation set. there are a number of studies that use data augmentation to address the generalization problem revealed by nli challenge datasets like hans and comparisons. mccoy et al. (2019b); dasgupta et al. (2018); nie et al. (2018 ) created augmentation sets consisting of training instances with properties similar to the proposed adversarial evaluation set. the augmentation set is focused on the examined phenomena and considerably smaller than the original training set in general. nevertheless, the models are shown to achieve very strong results, even close to %100 accuracy on the evaluation sets after training with augmented datasets. however, there are some problems with an augmentation approach performed this way, i.e. using a new dataset targeting the inspected phenomena in the proposed evaluation set. first of all, it is not clear if they do result in improvement on the language understanding of the model in general. rather, the model at hand is patched so that it can excel on some specific cases that the new evaluation dataset examines. however, one can presumably find other adversarial example classes for a given training dataset, so creating an augmentation set for each possible adversarial class may not be feasible. moreover, @xcite showed that augmenting the training dataset by targeting some specific category of adversarial examples might be harmful to other types of adversarial examples. in other words, dataset augmentation with such limited focus might lead to overfitting to the targeted adversaries and hurt the overall robustness. therefore, in this work, we took a different approach and introduced an inductive bias on the model by explicitly enforcing it to produce representations suitable to extract semantic role information. some recent studies have investigated the benefits of semantic role labeling on the performance of natural language inference models. noticeably, @xcite used srl as a supplementary task for text comprehension tasks such as textual entailment and question answering. similar to our work, they used propbank @xcite style role annotations and treated srl as a sequence tagging problem. @xcite 's approach is different from ours in that they use a pre-trained, sota srl model to provide semantic embeddings to enrich the contextual embeddings from bert. they kept the srl model frozen and trained other parts of the model including the bert encoder. similarly, @xcite employed two different networks where one of them is an srl model responsible for generating the semantic embeddings to support the other network which is trained to solve the downstream task at hand. moreover, both networks in this model use pretrained word embeddings such as elmo @xcite or glove @xcite . the main difference of our approach from these is that we use a single network and train it in a multi-task fashion by sharing encoder representations among different tasks. moreover, unlike our work, they evaluated their models on the original datasets e.g. multinli, so their focus was not to improve the model performance on the adversarial evaluation sets. instead of semantic role information, some recent works investigated the benefits of syntax to support the natural language inference models. noticeably, @xcite used the hidden word representations of an externally trained, high performing dependency parser to enrich the bert based nli models. with this approach, the models achieved some modest increase on the overall hans results. multi-task models powered with pre-trained language model based encoders have achieved sota performance on natural language understanding benchmarks such as glue @xcite , and superglue @xcite . however, there is not much work focusing on simultaneously solving both a word-level semantics task such as srl and a sentence-level understanding such as nli which requires higher level reasoning. instead, the existing approaches such as @xcite ; @xcite combine multiple sentence-level understanding tasks to solve them jointly without any aid from lower level tasks focusing on syntax or word-level semantics. our approach differs from them in that we hypothesize using both word-level semantics and high level reasoning tasks might be a more suitable approach to learn deeper understanding of the sentences, thereby suffering less from the dataset biases in reasoning tasks such as nli. some previous work attempted to cast nli to a different natural language understanding task such as question answering. @xcite suggested a collection of various tasks including nli for a benchmark and proposed a novel approach to solve all those tasks using a single multi-task model. they casted each task to the question answering problem and trained a model to solve all of them jointly. ## conclusion and future work this work presents a multi-task learning approach using srl task to apply an inductive bias on a bert based nli model. our experiments show that joint training with srl makes the model more robust to the superficial patterns in the nli training data. as opposed to the augmentation based solutions focused on specific adversarial classes, this approach has the advantage of being applicable to a variety of adversaries without overfitting to some of them. having access to the semantic role information improves the sentence understanding of the model, hence making it generalize better to the unseen dataset distributions including the adversarial ones such as hans and comparisons. the srl task utilized in this work processes a single predicate per data instance. the future work might incorporate joint prediction of all predicates and corresponding roles to analyze its effect on adversarial nli evaluation performance.