Year: 2019

Abstract:
Even as pre-trained language encoders such as BERT are shared across many tasks, the output layers of question answering, text classification, and regression models are significantly different. Span decoders are frequently used for question answering, fixed-class, classification layers for text classification, and similarityscoring layers for regression tasks, We show that this distinction is not necessary and that all three can be unified as span extraction. A unified, span-extraction approach leads to superior or comparable performance in supplementary supervised pre-trained, low-data, and multi-task learning experiments on several question answering, text classification, and regression benchmarks.

heading: 1 INTRODUCTION
text: Pre-trained natural language processing (NLP) systems (Radford et al., 2019; Devlin et al., 2018; Radford et al., 2018; Howard & Ruder, 2018; Peters et al., 2018; McCann et al., 2017; Liu et al., 2019b) have been shown to transfer remarkably well on downstream tasks including text classification, question answering, machine translation, and summarization (Wang et al., 2018; Rajpurkar et al., 2016; Conneau et al., 2018). Such approaches involve a pre-training phase followed by the addition of task-specific layers and a subsequent re-training or fine-tuning of the conjoined model. Each task-specific layer relies on an inductive bias related to the kind of target task. For question answering, a task-specific span-decoder is often used to extract a span of text verbatim from a portion of the input text (Xiong et al., 2016). For text classification, a task-specific classification layer with fixed classes is typically used instead. For regression, similarity-measuring layers such as leastsquares and cosine similarity are employed. These task-specific inductive biases are unnecessary. On several tasks predominantly treated as text classification or regression, we find that reformulating them as span-extraction problems and relying on a span-decoder yields superior performance to using a task-specific layers.
For text classification and regression problems, pre-trained NLP systems can benefit from supplementary training on intermediate-labeled tasks (STILTs) (Phang et al., 2018), i.e. supplementary supervised training. We find this is similarly true for question answering, classification, and regression when reformulated as span-extraction. Because we rely only on the span-extractive inductive bias, we are able to further explore previously unconsidered combinations datasets. By doing this, we find that question answering tasks can benefit from text classification tasks and classification tasks can benefit from question answering ones.
The success of pre-training for natural language processing systems affords the opportunity to reexamine the benefits of our inductive biases. Our results on common question answering, text classification, and regression benchmark tasks suggest that it is advantageous to discard the inductive bias that motivates task-specific, fixed-class, classification and similarity-scoring layers in favor of the inductive bias that views all three as span-extraction problems.

heading: 1.1 CONTRIBUTIONS
text: Summarily, we demonstrate the following:
1. Span-extraction is an effective approach for unifying question answering, text classification, and regression.
2. Span-extraction benefits as much from intermediate-task training as more traditional text classification and regression methods.
3. Span-extraction allows for combinations of question answering and text classification datasets in intermediate-task training that outperform using only one or the other.
4. Span-extractive multi-task learning yield stronger multi-task models, but weaker singletask models compared to intermediate-task training.
5. Span-extraction with intermediate-task training proves more robust in the presence of limited training data than the corresponding task-specific versions.

heading: 2 RELATED WORK
text: Transfer Learning. The use of pre-trained encoders for transfer learning in NLP dates back to Collobert & Weston (2008); Collobert et al. (2011) but has had a resurgence in the recent past. BERT (Devlin et al., 2018) employs the recently proposed Transformer layers (Vaswani et al., 2017) in conjunction with a masked language modeling objective as a pre-trained sentence encoder. Prior to BERT, contextualized word vectors (McCann et al., 2017) were pre-trained using machine translation data and transferred to text classification and question answering tasks. ELMO (Peters et al., 2018) improved contextualized word vectors by using a language modeling objective instead of machine translation. ULMFit (Howard & Ruder, 2018) and GPT (Radford et al., 2018) showed how traditional, causal language models could be fine-tuned directly for a specific task, and GPT2 (Radford et al., 2019) showed that such language models can indirectly learn tasks like machine translation, question answering, and summarization.
Intermediate-task and Multi-task Learning. The goal of unifying NLP is not new (Collobert & Weston, 2008; Collobert et al., 2011). In Phang et al. (2018), the authors explore the efficacy of supplementary training on intermediate tasks, a framework that the authors abbreviate as STILTs. Given a target task T and a pre-trained sentence encoder, they first fine-tune the encoder on an intermediate (preferably related) task I and then finally fine-tune on the task T . The authors showed that such an approach has several benefits including improved performance and better robustness to hyperparameters. The authors primarily focus on the GLUE benchmark (Wang et al., 2018). Liu et al. (2019a) explore the same task and model class (viz., BERT) in the context of multi-tasking. Instead of using supplementary training, the authors choose to multi-task on the objectives and, similar to BERT on STILTs, fine-tune on the specific datasets in the second phase. Further improvements can be obtained through heuristics such as knowledge distillation as demonstrated in Clark et al. (2019). All of these approaches require a different classifier head for each task, e.g., a two-way classifier for SST and a three-way classifier for MNLI. Two recent approaches: decaNLP (McCann et al., 2018) and GPT-2 Radford et al. (2019) propose the unification of NLP as question answering and language modeling, respectively. As investigated in this work, the task description is provided in natural language instead of fixing the classifier a-priori.

heading: 3 METHODS
text: We propose treating question answering, text classification, and regression as span-extractive tasks. Each input is split into two segments: a source text which contains the span to be extracted and an auxiliary text that is used to guide extraction. Question answering often fits naturally into this framework by providing both a question and a context document that contains the answer to that question. When treated as span-extraction, the question is the auxiliary text and the context document is the source text from which the span is extracted. Text classification input text most often does not contain a natural language description of the correct class. When it is more natural to consider the input text as one whole, we treat it as the auxiliary text and use a list of natural language descriptions of all possible classification labels as source text. When the input text contains two clearly delimited segments, one is treated as auxiliary text and the other as source text with appended natural language descriptions of possible classification labels. For regression, we employ a process similar to classification; instead of predicting a floating-point number, we bucket the possible range and classify the text instead.
Our proposal is agnostic to the details of most common preprocessing and tokenization schemes for the tasks under consideration, so for ease of exposition we assume three phases: preprocessing, encoding, and decoding. Preprocessing includes any manipulation of raw input text; this includes tokenization. An encoder is used to extract features from the input text, and an output layer is used to decode the output from the extracted features. Encoders often include a conversion of tokens to distributed representation followed by application of several layers of LSTM, Transformer, convolutional neural network, attention, or pooling operations. In order to properly use these extracted features, the output layers often contain more inductive bias related to the specific task. For many question answering tasks, a span-decoder uses the extracted features to select a start and end token in the source document. For text classification, a linear layer and softmax allow for classification of the extracted features. Similarly, for regression, a linear layer and a similarity-scoring objective such as cosine distance or least-squares is employed. We propose to use span-decoders as the output layers for text classification and regression in place of the more standard combination of linear layer with task-specific objectives.

heading: 3.1 SPAN-EXTRACTIVE BERT (SPEX-BERT)
text: In our experiments, we start with a pre-trained BERT as the encoder with preprocessing as described in Devlin et al. (2018). This preprocessing takes in the source text and auxiliary text and outputs a sequence of p = m+n+2 tokens: a special CLS token, the m tokens of the source text, a separator
token SEP, and the n auxiliary tokens. The encoder begins by converting this sequence of tokens into a sequence of p vectors in Rd. Each of these vectors is the sum of a token embedding, a positional embedding that represents the position of the token in the sequence, and a segment embedding that represents whether the token is in the source text or the auxiliary text as described in Devlin et al. (2018). This sequence is stacked into a matrix X0 ∈ Rp×d so that it can be processed by several Transformer layers (Vaswani et al., 2017). The ith layer first computes αp(Xi) by applying selfattention with k heads over the previous layer’s outputs:
αk(Xi) = [h1; · · · ;hk]Wo (1) where hj = α(XiW 1j , XiW 2 j , XiW 3 j )
α(X,Y, Z) = softmax ( XY >√
d
) Z (2)
A residual connection (He et al., 2016) and layer normalization (Ba et al., 2016) merge information from the input and the multi-head attention:
Hi = LayerNorm(αp(Xi) +Xi) (3)
This is followed by a feedforward network with ReLU activation (Nair & Hinton, 2010; Vaswani et al., 2017), another residual connection, and a final layer normalization. With parameters U ∈ Rd×f and V ∈ Rf×d:
Xi+1 = LayerNorm(max(0, HiU)V +Hi)) (4)
LetXsf ∈ Rm×d represent the final output of these Transformer layers. At this point, a task-specific head usually uses some part of Xsf to classify, regress, or extract spans. Our proposal is to use a span-decoder limited to Xsf whenever a classification or similarity-scoring layer is typically used. In this case, we add only two trainable parameter vectors dstart and dend following Devlin et al. (2018), and we compute start and end distributions over possible spans by multiplying these vectors with Hf and applying a softmax function:
pstart = softmax(Xsfdstart) pend = softmax(Xsfdend) (5)
During training, we are given the ground truth answer span (a∗, b∗) as a pair of indices into the source text. The summation of cross-entropy losses over the start and end distributions then gives an overall loss for a training example:
Lstart = − ∑ i I{a∗ = i} log pstart(i) Lend = − ∑ i I{b∗ = i} log pend(i) (6)
With L = Lstart + Lend and at inference, we extract a span (a, b) as
a = argmax i pstart(i) b = argmax i pend(i) (7)

heading: 4 EXPERIMENTAL SETUP

heading: 4.1 TASKS, DATASETS AND METRICS
text: We divide our experiments into three categories: classification, regression, and question answering. For classification and regression, we evaluate on all the GLUE tasks (Wang et al., 2018). This includes the Stanford Sentiment Treebank (SST) (Socher et al., 2013), MSR Paraphrase Corpus (MRPC) (Dolan & Brockett, 2005), Quora Question Pairs (QQP), Multi-genre Natural Language Inference (MNLI) (Williams et al., 2017), Recognizing Textual Entailment (RTE) (Dagan et al., 2010; Bar-Haim et al., 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009), Question-answering as NLI (QNLI) (Rajpurkar et al., 2016), and Semantic Textual Similarity (STS-B) Cer et al. (2017). The Winograd schemas challenge as NLI (WNLI) Levesque et al. (2012) was excluded during training because of known issues with the dataset. As with most other models on the GLUE leaderboard, we report the majority class label for all instances. With the exception of STS-B, which is a regression dataset, all other datasets are classification datasets. For question answering, we employ 6 popular
datasets: the Stanford Question Answering Dataset (SQuAD) (Rajpurkar et al., 2016), QA Zero-shot Relationship Extraction (ZRE; we use the 0th split and append the token unanswerable to all examples so it can be extracted as a span) (Levy et al., 2017), QA Semantic Role Labeling (SRL) (He et al., 2015), Commonsense Question Answering (CQA; we use version 1.0) (Talmor et al., 2018) and the two versions (Web and Wiki) of TriviaQA (Joshi et al., 2017). Unless specified otherwise, all scores are on development sets. Concrete examples for several datasets can be found in Table 1.

heading: 4.2 TRAINING DETAILS
text: For training the models, we closely follow the original BERT setup Devlin et al. (2018) and Phang et al. (2018). We refer to the 12-layer model as BERTBASE and the 24-layer model as BERTLARGE. Unless otherwise specified, we train all models with a batch size of 20 for 5 epochs. For the SQuAD and QQP datasets, we train for 2 epochs. We coarsely tune the learning rate but beyond this, do not carry out any significant hyperparameter tuning. For STILTs experiments, we re-initialize the Adam optimizer with the introduction of each intermediate task. For smaller datasets, BERT (especially BERTLARGE) is known to exhibit high variance across random initializations. In these cases, we repeat the experiment 20 times and report the best score as is common in prior work (Phang et al., 2018; Devlin et al., 2018). The model architecture, including the final layers, stay the same across all tasks and datasets – no task-specific classifier heads or adaptations are necessary.

heading: 4.3 MODELS AND CODE
text: Pre-trained models and code can be found at MASKED. We rely on the BERT training library1 available in PyTorch Paszke et al. (2017).

heading: 5 RESULTS
text: Next, we present numerical experiments to buttress the claims presented in Section 1.1.
1https://github.com/huggingface/pytorch-pretrained-BERT/
Span-extraction is similar or superior to task-specific heads (classification or regression). Table 2 shows our results comparing BERT (with and without STILTs) with the corresponding variant of SpEx-BERT on the GLUE tasks Wang et al. (2018). For almost all datasets, the performance for SpEx-BERT is better than that of BERT, which is perhaps especially surprising for the regression task (STS-B). One can reasonably expect model performance to improve by converting such problems into a span-extraction problem over natural language class descriptions.
SpEx-BERT improves on STILTs. As in the case of Phang et al. (2018), we find that using supplementary tasks for pre-training improves the performance on the target tasks. We follow the setup of Phang et al. (2018) and carry out a two-stage training process. First, we fine-tune the BERT model with a span-extraction head on an intermediate task. Next, we fine-tune this model on the target task with a fresh instance of the optimizer. Note that Phang et al. (2018) require a new classifier head when switching between tasks that have different numbers of classes or task, but no such modifications are necessary when SpEx-BERT is applied. SpEx-BERT also allows for seamless switching between question answering, text classification, and regression tasks.
In Table 5, we present the results for SpEx-BERT on STILTs. In a majority of cases, the performance of SpEx-BERT matches or outperforms that of BERT. This is especially pronounced for datasets with limited training data, such as MRPC and RTE with SpEx-BERTLARGE and BERTLARGE: 85.2 vs 83.4 for RTE, and 90.4 vs 89.5 for MRPC). We hypothesize that this increase is due to the fact that the class choices are provided to the model in natural language, which better utilizes the pretrained representations of a large language model like BERT. Finally, we note, perhaps surprisingly, that question answering datasets (SQuAD and TriviaQA) improve performance of some of the classification tasks. Notable examples include SST (pre-trained from the Wiki version of TriviaQA) and RTE (pre-trained from any of the three datasets).
STILTs improves question answering as well. Table 3a shows similar experiments on popular question answering datasets. The transferability of question answering datasets is well-known. Datasets such as TriviaQA, SQuAD and ZRE have been known to improve each other’s scores and have improved robustness to certain kinds of queries (Devlin et al., 2018; McCann et al., 2018). We further discover that through the formulation of SpEx-BERT, classification datasets also help question answering datasets. In particular, MNLI improves the scores of almost all datasets over their baselines. For SQuAD, the benefit of STILTs with the classification dataset MNLI is almost as much as the question answering dataset TriviaQA.
STILTs can be chained. Pre-training models using intermediate tasks with labeled data has been shown to be useful in improving performance. Phang et al. (2018) explored the possibility of using one intermediate task to demonstrate this improvement. We explore the possibility of chaining multiple intermediate tasks in Table 3a. Conceptually, if improved performance on SQuAD during the first stage of fine-tuning leads to improved performance for the target task of CQA, improving performance of SQuAD through in turn pre-training it on MNLI would improve the eventual goal of CQA. Indeed, our experiments suggest the efficacy of chaining intermediate tasks in this way. CQA
obtains a score of 63.8 when fine-tuned from a SQuAD model (of score 84.0) and obtains a score of 65.7 when fine-tuned on a SQuAD model that was itself fine-tuned using MNLI (of score 84.5) as an intermediate task.
Multi-task STILTs yields stronger multi-task models, but weaker single-task models. We also experiment with multi-task learning during intermediate-task training. We present the results for such intermediate-multi-task training on RTE in Table 4a. In intermediate-multi-task training, we cycle through one batch for each of the tasks until the maximum number of iterations is reached. No special consideration is made for the optimizer or weighing of objectives. The results show that intermediate-multi-task training improves performance over the baseline for RTE, but this improvement is less than when only MNLI is used for intermediate-task training. Though not desirable if RTE is the only target task, such intermediate-multi-task training yields a better multi-task model that performs well on both datasets: the joint (single) model achieved 75.0 on RTE and 86.2 on MNLI, both of which are better than their single-task baselines. In some cases, the increased performance for one task (MNLI) might be preferable to that on another (RTE). We note that this observation is similar to the one of Phang et al. (2018).
SpEx-BERT on STILTs is more robust than BERT on STILTs when training data is limited. In Table 3b, we present results for the same models (BERT and SpEx-BERT) being fine-tuned with sub-sampled versions of the dataset. For this experiment, we follow Phang et al. (2018) and subsample 1000 data points at random without replacement and choose the best development set accuracy across several random restarts. The rest of the experimental setup remains unchanged. When used in conjunction with STILTs, the performance improves as expected and, in a majority of cases, significantly exceeds that of the corresponding baseline that does not use span-extraction.

heading: 6 DISCUSSION

heading: 6.1 PHRASING THE QUESTION
text: As described in Section 3, when converting any of the classification or regression problems into a span-extraction one, the possible classes or bucketed values need to be presented in natural language as part of the input text. This leaves room for experimentation. We found that separation of naturally delimited parts of the input text into source and auxiliary text was crucial for best performance. Recall that for question answering, the natural delimitation is to assign the given context document as the source text and the question as the auxiliary text. This allows the span-decoder to extract a span from the context document, as expected. For single-sentence problems, there is no need for delimitation and the correct span is typically not found in the given sentence, so it is treated as auxiliary text.
Natural language descriptions of the classes or allowable regression values are provided as source text for span extraction. For two-sentence problems, the natural delimitation suggests treating one sentence as source text and the other as auxiliary. The classification or regression choices must be in the source text, but it was also the case that one of the sentences must also be in the source text. Simply concatenating both sentences and assigning them as the source text was detrimental for tasks like MNLI.
For the case of classification, when experimenting with various levels of brevity, we found that simpler is better. Being terse eases training since the softmax operation over possible start and end locations is over a relatively smaller window. While more detailed explanations might elaborate on what the classes mean or otherwise provide additional context for the classes, these potential benefits were outstripped by increasing the length of the source text. We present these results on the development set of the MNLI dataset with BERTBASE in Table 4b. For regression, there exists a trade-off between brevity and granularity of the regression. We found that dividing the range into 10 – 20 buckets did not appreciably change the resulting correlation score for STS-B.

heading: 6.2 A FULLY JOINT MODEL WITHOUT TASK-SPECIFIC PARAMETERS
text: Unlike similar approaches using task-specific heads Liu et al. (2019a), SpEx-BERT allows for a single model across a broader set of tasks. This makes possible a single, joint model with all parameters shared. We present the results of this experiment in Table 5 in the Appendix; we multi-task over all datasets considered so far. Multi-task performance exceeds single-task performance for many of the question answering datasets (ZRE, SRL, CQA) as well as the classification dataset RTE. In some cases, these improvements are drastic (over 9% accuracy). Unfortunately, the opposite is true for the two tasks that are the greatest source of transfer, MNLI and SQuAD, and the remaining GLUE tasks. Understanding why such vampiric relationships amongst datasets manifest, why any particular dataset appears beneficial, neutral, or detrimental to the performance of others, and why question answering tasks appear more amenable to the fully-joint setting remain open questions. Nonetheless, a purely span-extractive approach has allowed us to observe such relationships more directly than in settings that use multiple task-specific heads or fine-tune separately on each task. Because some tasks benefit and others suffer, these results present a trade-off. Depending on which tasks and datasets are more pertinent, multi-task learning might be the right choice, especially given the ease of deploying a single architecture that does not require any task-specific modifications.
Joint models for NLP have already been studied Collobert et al. (2011); McCann et al. (2018); Radford et al. (2019) with a broad set of tasks that may require text generation and more general architectures. These approaches have yet to perform as well as task-specific models on common benchmarks, but they have demonstrated that large amounts of unsupervised data, curriculum learning, and task sampling strategies can help mitigate the negative influence multitasking tends to have on datasets that are especially good for transfer learning. This work represents a connection between those works and work that focuses on task-specific fine-tuning of pre-trained architectures.

heading: 7 CONCLUSION
text: With the successful training of supervised and unsupervised systems that rely on increasingly large amounts of data, more of the natural variation in language is captured during pre-training. This suggests that less inductive bias in the design of task-specific architectures might be required when approaching NLP tasks. We have proposed that the inductive bias that motivates the use task-specific layers is no longer necessary. Instead, a span-extractive approach, common to question answering, should be extended to text classification and regression problems as well. Experiments comparing the traditional approach with BERT to SpEx-BERT have shown that the span-extractive approach often yields stronger performance as measured by scores on the GLUE benchmark. This reduces the need for architectural modifications across datasets or tasks, and opens ways for applying methods like STILTs to question answering or a combination of text classification, regression, and question answering datasets to further improve performance. Experiments have further shown that spanextraction proves more robust in the presence of limited training data. We hope that these findings will promote further exploration into the design of unified architectures for a broader set of tasks.

heading: A MULTITASKING RESULTS
text: Below is the table that supports the commentary in Section 6.2.

Review summaries:
This paper asks whether it works to remove task-specific heads and treat classification and regression problems as span extraction, by formatting problems in such a way that a single span extraction model can be used.  This is a reasonable question to ask, and the authors performed a very large number of experiments attempting to answer this question.  The authors claim that using span extractive models instead of task-specific heads yields improved performance over separate heads.

This paper introduces a method for converting sentence pair classification tasks and sentence regression tasks into span into span extraction tasks, by listing all the possible classes (entailment, contradiction, neural) or the discretized scores (0.0, 0.25 ...) and concatenating them with the source text. With this formulation, one can train a BERT-based span-extraction model (SpEx-BERT) on classification, regression, and QA tasks without introducing any task-specific parameters. The purposed SpEx-BERT model achieves moderate improvement (0.3 points) over the BERT-large baseline on the GLUE test set when fine-tuned on intermediate STILTs tasks (Phang et al., 2018).