Paper:
Deep learning has proven useful on many NLP tasks including reading comprehension. However, it requires large amounts of training data which are not available in some domains of application. Hence we examine the possibility of using data-rich domains to pre-train models and then apply them in domains where training data are harder to get. Specifically, we train a neural-network-based model on two context-question-answer datasets, the BookTest and CNN/Daily Mail, and we monitor transfer to subsets of bAbI, a set of artificial tasks designed to test specific reasoning abilities, and of SQuAD, a question-answering dataset which is much closer to real-world applications. Our experiments show very limited transfer if the model is not shown any training examples from the target domain however the results are encouraging if the model is shown at least a few target-domain examples. Furthermore we show that the effect of pre-training is not limited to word embeddings.
1 INTRODUCTION
Machine intelligence has had some notable successes, however often in narrow domains which are sometimes of little practical use to humans – for instance games like chess (Campbell et al., 2002) or Go (Silver et al., 2016). If we aimed to build a general AI that would be able to efficiently assist humans in a wide range of settings, we would want it to have a much larger set of skills – among them would be an ability to understand human language, to perform common-sense reasoning and to be able to generalize its abilities to new situations like humans do.
If we want to achieve this goal through Machine Learning, we need data to learn from. A lot of data if the task at hand is complex – which is the case for many useful tasks. One way to achieve wide applicability would be to provide training data for each specific task we would like the machine to perform. However it is unrealistic to obtain a sufficient amount of training data for some domains – it may for instance require expensive human annotation or all domains of application may be difficult to predict in advance – while the amount of training data in other domains is practically unlimited, (e.g. in language modelling or Cloze-style question answering).
The way to bridge this gap – and to achieve the aforementioned adaptability – is transfer learning (Pan & Yang, 2010) and closely related semi-supervised learning (Zhu & Goldberg, 2009) which allow the system to acquire a set of skills on domains where data are abundant and then use these skills to succeed on previously unseen domains. Despite how important generalization is for general AI, a lot of research keeps focusing on solving narrow tasks.
In this paper we would like to examine transfer of learnt skills and knowledge within the domain of text comprehension, a field that has lately attracted a lot of attention within the NLP community (Hermann et al., 2015; Hill et al., 2015; Kobayashi et al., 2016; Kadlec et al., 2016b; Chen et al., 2016; Sordoni et al., 2016; Dhingra et al., 2016; Trischler et al., 2016; Weissenborn, 2016; Cui et al., 2016b;a;
∗These authors contributed equally to this work.
Li et al., 2016; Shen et al., 2016). Specifically, we would like to address the following research questions:
1. Whether we could train models on natural-language tasks where data are abundant and transfer the learnt skills to tasks where in-domain training data may be difficult to obtain. We will first look into what reasoning abilities a model learns from two large-scale readingcomprehension datasets using artificial tasks, and then check whether it can transfer its skills to real world tasks. Spoiler: both these transfers are very poor if we allow no training at all on the target task.
2. Whether pre-training on large-scale datasets does help if we allow the model to train on a small sample of examples from the target tasks. Here the results are much more positive.
3. Finally we examine whether the benefits of pre-training are concentrated in any particular part of the model - namely the word-embedding part or the context encoder (the reasoning part). It turns out that pre-training is useful for both components.
Although our results do not improve current state of the art in any of the studied tasks, they show a clear positive effect of large-dataset pre-training on the performance of our baseline machine-learning model. Previous studies of transfer learning and semi-supervised learning in NLP focused on text classification (Dai & Le, 2015; Mou et al., 2016) and various parsing tasks (Collobert et al., 2011; Hashimoto et al., 2016). To our knowledge this work is the first study of transfer learning in reading comprehension, and we hope it will stimulate further work in this important area.
We will first briefly introduce the datasets we will be using on the pre-training and target sides, then our baseline model and afterwards in turn describe the method and results of each of the three experiments.
2 DATASETS
2.1 PRE-TRAINING DATASETS
We have mentioned that for the model pre-training we would want to use a task where training data are abundant. An example of such task is context-dependent cloze-style-question answering since the training data for this task can be generated automatically from a suitable corpus. We will use two such pre-training datasets in our experiments: the BookTest (Bajgar et al., 2016) and the CNN/Daily Mail (CNN/DM) news dataset (Hermann et al., 2015).
The task associated with both datasets is to answer a cloze-style question (i.e. fill in a blank in a sentence) the answer to which needs to be inferred from a context document provided with the question.
2.1.1 BOOKTEST
In the BookTest dataset, the context document is formed from 20 consecutive sentences from a book. The question is then formed by omitting a common noun or a named entity from the subsequent 21st sentence. Among datasets of this kind, the BookTest is among the largest with more than 14 million training examples coming from 3555 copyright-free books avalable thanks to Project Gutenberg.
2.1.2 CNN/DAILY MAIL
In the CNN/DM dataset the context document is formed from a news article while the cloze-style question is formed by removing a named entity from one of the short summary sentences which often appear at the top of the article.
To stop the model from using world knowledge from outside the context article (and hence truly test the comprehension of the article), all named entities were replaced by anonymous tags, which are further shuffled for each example. This may make the comprehension more difficult; however, since the answer is always one of the anonymized entities, it also reduces the number of possible answers making guessing easier.
2.2 TARGET DATASETS
2.2.1 BABI
The first target dataset are the bAbI tasks (Weston et al., 2016) – a set of artificial tasks each of which is designed to test a specific kind of reasoning. This toy dataset will allow us to observe what particular skills the model may be learning from each of the three training datasets.
For our experiments we will be using an architecture designed to select one word from the context document as the answer. Hence we have selected Tasks 1,2,3,4,5,11,12,13,14 and 16 which fulfill this requirement and added task 15 which required a slight modification. Furthermore because both pre-training datasets are cloze-style we converted also the bAbI task questions into cloze style (e.g. ”Where is John?” to ”John is in the XXXXX.”).
For the models pre-trained on CNN/DM we also anonymized the tasks in a way similar to the pre-training dataset - i.e. we replaced all names of characters and also all words that can appear as answers for the given task by anonymous tags in the style of CNN/DM. This gives even models that have not seen any training examples from the target domain a chance to answer the questions.
Full details about these alterations can be found in Appendix A.
2.2.2 SQUAD
Secondly, we will look on transfer to the SQuAD dataset (Rajpurkar et al., 2016); here the associated task may be already useful in the real world. Although cloze-style questions have the huge advantage in the possibility of being automatically generated from a suitable corpus – the path taken by CNN/DM and the BookTest – in practice humans would use a proper question, not its cloze-style substitute. This brings us to the need of transfer from the data-rich cloze-style training to the domain of proper questions where data are much scarcer due to the necessary human annotation.
The SQuAD dataset is a great target dataset to use for this. As opposed to the bAbI tasks, the goal of this dataset is actually a problem whose solving would be useful to humans - answering natural questions based on an natural language encyclopedic knowledge base.
For our experiments we selected only a subset of the SQuAD training and development examples where the answer is only a single word, since this is an inherent assumption of our machine learning model. This way we extracted 28,346 training examples out of the original 100,000 examples and 3,233 development examples out of 10,570.
3 MACHINE LEARNING MODEL: AS READER
We perform our experiments using the Attention Sum Reader (AS Reader) (Kadlec et al., 2016b) model. The AS Reader is simple to implement while it achieves strong performance on several text comprehension tasks (Kadlec et al., 2016b; Bajgar et al., 2016; Chu et al., 2016). Since the AS Reader is a building block of many recent text-comprehension models (Trischler et al., 2016; Sordoni et al., 2016; Dhingra et al., 2016; Cui et al., 2016b;a; Shen et al., 2016; Munkhdalai & Yu, 2016) it is a good representative of current research in this field.
A high level structure of the AS Reader is shown in Figure 1. The words from the document and the question are first converted into vector embeddings using a look-up matrix. The document is then read by a bidirectional Gated Recurrent Unit (GRU) network (Cho et al., 2014). A concatenation of the hidden states of the forward and backward GRUs at each word is then used as a contextual embedding of this word, intuitively representing the context in which the word is appearing. We can also understand it as representing the set of questions to which this word may be an answer.
Similarly the question is read by a bidirectional GRU but in this case only the final hidden states are concatenated to form the question embedding.
The attention over each word in the context is then calculated as the dot product of its contextual embedding with the question embedding. This attention is then normalized by the softmax function and summed across all occurrences of each answer candidate. The candidate with most accumulated attention is selected as the final answer.
For a more detailed description of the model including equations check Kadlec et al. (2016b).
4 EXPERIMENTS: TRANSFER LEARNING IN TEXT COMPREHENSION
Now let us turn in more detail to the three kinds of experiments that we performed.
4.1 PRE-TRAINED WITHOUT TARGET ADJUSTMENT
In the first experiment we tested how a model trained on one of the large-scale pre-training datasets performs on the bAbI tasks without any opportunity to train on bAbI. Since the BookTest and CNN/DM tasks involve only cloze-style questions, we can’t expect a model trained on them to answer natural ?-style questions. Hence we did not study the transfer to SQuAD in this case, only the transfer to the (cloze-converted) bAbI tasks.
4.1.1 METHOD
First we tested how the AS Reader architecture (Kadlec et al., 2016b) can handle the tasks if trained directly on the bAbI training data for each task. Then we tested the degree of transfer from the BookTest and CNN/DM data to the 11 selected bAbI tasks.
In the first part of the experiment we trained a separate instance of the AS Reader on the 10,000- example version of the bAbI training data for each of the 11 tasks (for more details see Appendix B.1). On 8 of them the architecture was able to learn the task with accuracy at least 95% 1 (results for each task can be found in Table 4 in Appendix C). Hence if given appropriate training the AS Reader is capable of the reasoning needed to solve most of the selected bAbI tasks. Now when we know that the AS Reader is powerful enough to learn the target tasks we can turn to transfer from the two large-scale datasets.
The main part of this first experiment was then straightforward: we pre-trained multiple models on the BookTest and CNN/DM datasets and then simply evaluated them on the test datasets of the 11 selected bAbI tasks.
4.1.2 RESULTS
Table 1 summarizes the results of this experiment. Both the models trained on the BookTest and those trained on the CNN/DM dataset perform quite poorly on bAbI and achieve much lower accuracy than
1It should be noted that there are several machine learning models that perform better than the AS Reader in the 10k weakly supervised setting, e.g. (Sukhbaatar et al., 2015; Xiong et al., 2016; Graves et al., 2016), however they often need significant fine-tuning. On the other hand we trained plain AS Reader model without any modifications. Hyperparameter and feature fine-tuning could probably further increase its performance on individual tasks however it goes directly against the idea of generality that is at the heart of this work. For comparison with state of the art we include results of DMN+ (Xiong et al., 2016) in Table 1 which had the best average performance over the original 20 tasks.
the models trained directly on each individual bAbI task. However there is some transfer between the tasks since the AS Reader trained on either the BookTest or CNN/DM outperforms a random baseline2 and even an improved baseline which selects the most frequent word from the context that also appears as an answer in the training data for this task.
The results also show that the models trained on CNN/DM perform somewhat better on most tasks than the BookTest models. This may be due to the fact that bAbI tasks generally require the model to summarize information from the context document, which is also what the CNN/DM dataset is testing. On the other hand, the BookTest requires prediction of a possible continuation of a story, where the required kind of reasoning is much less clear but certainly different from pure summarization. Another explanation for better performance of CNN/DM models might be that they solve slightly simpler task since the candidate answers were already pre-selected in the entity anonymization step.
Readers interested in how the training-dataset size affects this kind of transfer can check (Kadlec et al., 2016a) where we show that the target-task performance is a bit better if we use the large BookTest as opposed to its smaller subset, the Children’s Book Test (CBT) (Hill et al., 2015).
Conclusions from this experiment are that the skills learned from two large-scale datasets generalize surprisingly poorly to even simple toy tasks. This may make us ask whether most teams’ focus on solving narrow tasks is truly beneficial if the skills learnt on these tasks are hard to apply elsewhere. However it also brings us to our next experiment, where we try to provide some help to the struggling pre-trained models.
4.2 PRE-TRAINED WITH TARGET ADJUSTMENT
After showing that the skills learnt from the BookTest and CNN/DM datasets are by themselves insufficient for solving the toy tasks, the next natural question is whether they are useful if helped by training on a small sample of examples from the target task. We call this additional phase of training target adjustment. For this experiment we again use the bAbI tasks, however we also test transfer to a subset of the SQuAD dataset, which is much closer to real-world natural-language question answering.
The results presented in this and the following section are based on training 3701 model instances.
4.2.1 METHOD
Common to bAbI and SQuAD datasets. In this experiment we started with a pre-trained model which we used in the previous experiment. However, after it finished training on one of the large pre-training datasets, we allowed it to train on a subset of training examples from the target dataset. We tried subsets of various sizes ranging from a single example to thousands. We tried training four different pre-trained models and also, for comparison, four randomly-initialized models with the same hyperparameters (see Appendix B.2 for details). The experiment with each task-model couple was run on 4 different data samples of each size which were randomly drawn from the training dataset
2The random baseline selects randomly uniformly between all unique words contained in the context document.
of the task to account for variations between these random samples – which may be substantial given the small sample size.3
bAbI. For each of these models we observed the test accuracy at the best-validation epoch and compared this number between the randomly initialized and pre-trained models. Validation was done using 100 examples which were set aside from the task’s original 10k training data.4 We perform the experiment with models pre-trained on the BookTest and also on CNN/DM.
SQuAD subset. In the SQuAD experiment, we trained the model on a subset of the original training dataset where answers were only single words and its sub-subsets. We report the best-validation accuracy on a development set filtered in the same way. This experiment was performed only with the models pre-trained on BookTest.
4.2.2 RESULTS
The results of these experiments are summarized in Figures 2 and 3.
3We are planning to release the split training datasets soon. 4The other models trained on the full 10k dataset usually use 1000 validation examples (Sukhbaatar et al., 2015; Xiong et al., 2016), however we wanted to focus on low data regime thus we used 10 times less examples.
bAbI. Sub-figure 2a shows mean test accuracy of the models that achieved the best validation result for each single task. The results for both BookTest and CNN/DM experiments confirm positive effect of pre-training compared to randomly initialized baseline. Figure 3 shows performance on selected bAbI tasks where pre-training has clearly positive effect, such plot for each of the target tasks is provided in Appendix C.2 (Figure 4).
Note that the CNN/DM models cannot be directly compared to BookTest results due to entity anonymization that seems to simplify the task when the model is trained on smaller datasets.
Since our evaluation methodology with different training set sizes is novel, we can compare our result only to MemN2N (Sukhbaatar et al., 2015) trained on a 1k dataset. MemN2N is the only weakly supervised model that reports accuracy when trained on less than 10k examples. MemN2N achieves average accuracy 93.2%5 on the eleven selected tasks. This is substantially better than both our random baseline (78.0%) and the BookTest-pre-trained model (79.5%), however our model is not tuned in any way towards this particular task. One important conceptual difference is that the AS Reader processes the whole context as one sequence of words, whereas MemN2N receives the context split into single sentences, which simplifies the task for the network.
SQuAD subset. The results of SQuAD experiment also confirm positive effect of pre-training, see Sub-figure 2b, for now compare just lines showing performance of the fully pre-trained model and the randomly initialized model – the meaning of the remaining two lines shall become clear in the next section.
More detailed statistics about the results of this experiment can be found in Appendix D.
We should note that performance of our model is not competitive with the state of the art models on this dataset. For instance the DCR model (Yu et al., 2016) trained on our SQuAD subset achieves validation accuracy 74.9% in this task which is better than our randomly initialized (35.4%) and pre-trained (51.6%) models6. However, the DCR model is designed specifically for the SQuAD task, for instance it utilizes features that are not used by our model.
4.3 PARTIALLY PRE-TRAINED MODEL
Since our previous experiment confirmed positive effect of pre-training if followed by target-domain adjustment, we wondered which part of the model contains the knowledge transferable to new domains. To examine this we performed the following experiment.
4.3.1 METHOD
Our machine learning model, the AS Reader, consists of two main parts: the word-embedding look-up and the bidirectional GRUs used to encode the document and question (see Figure 1). Therefore a natural question was what the contribution of each of these parts is.
To test this we created two models out of each pre-trained model used in the previous experiment. The first model variant uses the pre-trained word embeddings from the original model while the GRU encoders are randomly initialized. We say that this model has pre-trained embeddings. The second model variant uses the opposite setting where the word embeddings are randomly initialized while the encoders are taken form a pre-trained model. We call this pre-trained encoders.
bAbI. For this experiment we selected only a subset of tasks with training set of 100 examples where there was significant difference in accuracy between randomly-initialized and pre-trained models. For evaluation we use the same methodology as in the previous experiment, that is, we report accuracy of the best-validation model averaged over 4 training splits.
SQuAD subset. We evaluated both model variants on all training sets from the previous SQuAD experiment using the same methodology.
5MemN2N trained on each single task with PE LS RN features, see (Sukhbaatar et al., 2015) for details. 6We would like to thank Yu et al. (2016) for training their system on our dataset.
4.3.2 RESULTS
bAbI. Table 2 shows improvement of pre-trained models over a randomly initialized baseline. In most cases (all except Task 5) the fully pre-trained model achieved the best accuracy.
SQuAD subset. The accuracies of the four model variants are plotted in Figure 2b together with results of the previous SQuAD experiment. The graph shows that both pre-trained embeddings and pre-trained encoders alone improve performance over the randomly initialized baseline, however the fully pre-trained model is always the best.
The overall result of this experiment is that both pre-training of the word embeddings and pre-training of the encoder parameters are important since the fully pre-trained model outperforms both partially pre-trained variants.
5 CONCLUSION
Our experiments show that transfer from two large cloze-style question-answering datasets to our two target tasks is suprisingly poor, if the models aren’t provided with any examples from the target domain. However we show that models that pre-trained models perform significantly better than a randomly initialized model if they are shown at least a few training examples from the target domain. The usefulness of pre-trained word embeddings is well known in the NLP community however we show that the power of our pre-trained model does not lie just in the embeddings. This suggests that once the text-comprehension community agrees on sufficiently versatile model, much larger parts of the model could start being reused than just the word-embeddings.
The generalization of skills from a training domain to new tasks is an important ingredient of any system we would want to call intelligent. This work is an early step to explore this direction.
A CLOZE STYLE BABI DATASET
Since our AS Reader architecture is designed to select a single word from the context document as an answer (the task of CBT and BookTest), we selected 10 bAbI tasks that fulfill this requirement out of the original 20. These tasks are: 1. single supporting fact, 2. two supporting facts, 3. three supporting facts, 4. two argument relations, 5. three argument relations, 11. basic coreference, 12. conjunction, 13. compound coreference, 14. time reasoning and 16. basic induction.
Task 15 needed a slight modification to satisfy this requirement: we converted the answers into plural (e.g. ”Q: What is Gertrude afraid of? A: wolf.” was converted into ”A: wolves” which also seems to be the more natural way to formulate the answer to such a question.).
Also since CBT and BookTest train the model for Cloze-style question answering, we modify the original bAbI dataset by reformulating the questions into Cloze-style. For example we translate a question ”Where is John ?” to ”John is in the XXXXX .”
For the models pre-trained on CNN/DM we also replace two kinds of words by anonymized tags (e.g. ”@entity56”) in a style similar to the pre-training dataset. Specifically we replace two (largely overlapping) categories of words:
1. Proper names of story characters (e.g. John, Sandra) 2. Any word that can appear as an answer for the particular task (e.g. kitchen, garden if the
task is asking about locations).
B METHOD DETAILS
B.1 DIRECT TRAINING ON BABI – METHOD
Here we give a more detailed description of the method we used to arrive to our results. We highlight only facts particular to this experiment. A more detailed general description of training the AS Reader is given in (Kadlec et al., 2016b).
The results given for AS Reader trained on bAbI are each for a single model with 64 hidden units in each direction of the GRU context encoder and embedding dimension 32 trained on the 10k training data provided with that particular task.
The results for AS Reader trained on the BookTest and the CNN/DM are for a greedy ensemble consisting of 4 models whose predictions were simply averaged. The models and ensemble were all validated on the validation set corresponding to the training dataset. The performance on the bAbI tasks oscillated notably during training however the ensemble averaging does somewhat mitigate this to get more representative numbers.
B.2 HYPERPARAMETERS FOR THE TARGET-ADJUSTMENT EXPERIMENTS
Table 3 lists hyperparameters of the pre-trained AS Reader instances used in our experiments with target adjustment.
C DETAILED RESULTS
C.1 EXPERIMENTS WITHOUT TARGET ADJUSTMENT
Table 4 shows detailed results for the experiments on models which were just pre-trained on one of the pre-training datasets without any target-adjustment. It also shows several baselines and results of a state-of-the-art model.
C.2 TARGET-ADJUSTMENT EXPERIMENTS
C.2.1 RESULTS FOR ALL BABI TASKS
Figure 4 shows the test accuracies of all models that we trained in the target-adjustment experiments as well as lines joining the accuracies of the best-validation models.
Ta bl
e 4:
Pe rf
or m
an ce
of th
e A
S R
ea de
r w
he n
tr ai
ne d
on th
e bA
bI 10
k, B
oo kT
es ta
nd C
N N
/D M
da ta
se ts
an d
th en
ev al
ua te
d on
bA bI
te st
da ta
. T
he D
yn am ic M em or y N et w or k (D M N +) is th e st at eof -t he -a rt m od el in a w ea kl y su pe rv is ed se tti ng on th e bA bI 10 k da ta se t. It s re su lts ar e ta ke n fr om (X io ng et al ., 20 16 ). M em N 2N (S uk hb aa ta re ta l., 20 15 )i s th e st at eof -t he -a rt m od el on th e 1k tr ai ni ng da ta se t; fo rc om pl et en es s w e al so in cl ud e its re su lts w ith th e 10 k tr ai ni ng .
M od
el :
R an
do m
R nd
ca nd
. M
em N 2N (s in gl e) (P E L S R
N )
M em
N 2N
(s in
gl e)
(P E
L S
LW R
N )
D M
N +
(s in
gl e)
A SR
ea de
r
a a
a a
a a a
a a a
Te st
da ta
se t Tr ai
n da
ta se
t no
t tr ai ne
d bA
bI 10
k bA
bI 1k
bA bI 10 k
bA bI 10 k
bA bI 10 k
B oo
kT es t 14
M D
M +C
N N
1. 2M
1 Si
ng le
su pp
or tin
g fa
ct 7.
80 31
.2 0
10 0.
00 10
0. 00
10 0.
00 10
0. 00
37 .3
0 51
.5 0
2 Tw
o su
pp or
tin g
fa ct
s 4.
40 26
.9 6
91 .7
0 99
.7 0
99 .7
0 91
.9 0
25 .8
0 28
.9 0
3 T
hr ee
su pp
or tin
g fa
ct s
3. 40
19 .1
4 59
.7 0
97 .9
0 98
.9 0
86 .0
0 22
.2 0
27 .4 0 4 Tw oar gu m en tr el at io ns 10 .5 0 33 .5 8 97 .2 0 10 0. 00 10 0. 00 10 0. 00 50 .3 0 54 .9 0 5 T hr ee -a rg um en tr el at io ns 4. 40 21 .4 2 86 .9 0 99 .2 0 99 .5 0 99 .8 0 67 .6 0 68 .1 0 11 B as ic co re fe re nc e 6. 20 30 .4 2 99 .1 0 99 .9 0 10 0. 00 10 0. 00 33 .0 0 20 .8 0 12 C on ju nc tio n 6. 70 27 .2 5 99 .8 0 10 0. 00 10 0. 00 10 0. 00 30 .4 0 37 .7 0 13 C om po un d co re fe re nc e 5. 60 27 .7 3 99 .6 0 10 0. 00 10 0. 00 10 0. 00 33 .8 0 14 .0 0 14 Ti m e re as on in g 5. 00 27 .8 2 98 .3 0 99 .9 0 99 .8 0 95 .0 0 27 .6 0 50 .5 0 15 B as ic de du ct io n 5. 20 37 .2 0 10 0. 00 10 0. 00 10 0. 00 96 .7 0 39 .9 0 17 .6 0 16 B as ic in du ct io n 7. 50 45 .6 5 98 .7 0 48 .2 0 54 .7 0 50 .3 0 15 .1 0 48 .0 0 bA bI m ea n (1 1 ta sk s) 6. 06 29 .8 5 93 .7 3 94 .9 8 95 .6 9 92 .7 0 34 .8 2 38 .1 3
C.2.2 AVERAGE OVER ALL MODELS TRAINED ON BABI TASKS
Figure 5 plots mean accuracy of all models trained in our experiments. This suggests that pre-training helped all models, not only the top performing ones selected by validation as already shown in Figure 2a.
D MEANS, STANDARD DEVIATIONS AND P-VALUES BY EXPERIMENT
Table 5 shows the mean accuracy across all models trained for each combination of task, pre-training dataset and target-adjustment dataset size. Table 6 shows the corresponding standard deviations.
Table 7 then shows the p-value that whether the expected accuracy of pre-trained models is greater than the expected accuracy of randomly initialized models. This shows that the pre-trained models are statistically significantly better for all target-adjustment set sizes on the SQuAD dataset. On bAbI the BookTest pre-trained models perform convincingly better especially for target-adjustment dataset sizes 100, 500 and 1000, with Task 16 being the main exception to this because the AS Reader struggles to learn it in any setting. For the CNN+DM pre-training the results are not conclusive.
Ta sk
Pr et
ra in
in g
Ta rg
et -a
dj us
tm en
ts et
si ze
0 1
10 10
0 50
0 10
00 50
00 10
00 0
28 17 4 SQ uA D B oo kT es t 1. 01 e45 4. 07 e05 7. 40 e05 7. 82 e08 N A 5. 17 e08 N A 3. 93 e08 8.
52 e03 Ta sk 1 B oo kT es t 3. 34 e83 1. 81 e03 1. 33 e01 2. 35 e19 9. 41 e04 1. 67 e02 1. 32 e01 N A N A Ta sk 2 B oo kT es t 1. 24 e34 3. 86 e07 7. 29 e03 2. 59 e01 1. 39 e08 2. 63 e06 7. 54 e09 2. 04 e01 N A Ta sk 3 B oo kT es t 9. 84 e55 1. 27 e05 7. 66 e03 1. 48 e03 3. 18 e04 2. 18 e03 2. 16 e04 1. 03 e01 N A Ta sk 4 B oo kT es t 7. 25 e78 9. 50 e01 9. 71 e01 1. 04 e05 6. 38 e03 1. 70 e02 1. 81 e02 N A N A Ta sk 5 B oo kT es t 6. 55 e11 5 9. 88 e22 8. 87 e19 5. 25 e05 3. 66 e03 8. 61 e02 5. 65 e03 N A N A Ta sk 11 B oo kT es t 6. 78 e15 2 1. 00 e+ 00 9. 94 e01 4. 07 e09 2. 50 e04 2. 28 e02 6. 37 e02 N A N A Ta sk 12 B oo kT es t 2. 27 e90 9. 10 e01 6. 46 e01 1. 89 e05 2. 78 e04 1. 43 e02 2. 36 e02 N A N A Ta sk 13 B oo kT es t 5. 30 e91 9. 75 e01 9. 99 e01 2. 88 e02 2. 74 e02 1. 03 e01 7. 06 e02 N A N A Ta sk 14 B oo kT es t 1. 97 e20 0 1. 01 e03 6. 79 e01 2. 22 e14 3. 40 e05 2. 93 e03 3. 66 e06 3. 97 e01 N A Ta sk 15 B oo kT es t 3. 64 e09 4. 75 e01 4. 12 e01 6. 70 e01 1. 68 e03 3. 70 e03 1. 03 e05 4. 54 e01 N A Ta sk 16 B oo kT es t 1. 81 e05 8. 28 e04 4. 38 e01 2. 72 e01 4. 89 e01 5. 71 e01 7. 40 e03 N A N A Ta sk 1 C N N +D M 9. 43 e09 2. 99 e01 1. 11 e01 1. 05 e01 9. 54 e02 1. 45 e01 3. 97 e03 N A N A Ta sk 2 C N N +D M 9. 38 e17 6. 93 e01 9. 02 e01 9. 15 e01 1. 05 e03 4. 20 e01 2. 64 e03 8. 49 e02 N A Ta sk 3 C N N +D M 2. 42 e16 4. 95 e02 6. 30 e01 1. 75 e01 2. 13 e03 6. 59 e04 4. 68 e02 1. 24 e01 N A Ta sk 4 C N N +D M 5. 84 e03 9. 70 e01 1. 37 e01 4. 83 e03 3. 33 e01 8. 84 e01 1. 08 e01 N A N A Ta sk 5 C N N +D M 1. 17 e10 7. 00 e03 7. 93 e04 5. 20 e01 9. 70 e01 5. 66 e01 1. 83 e01 N A N A Ta sk 11 C N N +D M 1. 00 e+ 00 9. 84 e01 9. 73 e01 2. 58 e01 7. 17 e01 1. 45 e01 6. 95 e01 N A N A Ta sk 12 C N N +D M 1. 93 e14 9. 32 e01 9. 92 e01 2. 57 e02 4. 06 e01 6. 65 e02 2. 09 e01 N A N A Ta sk 13 C N N +D M 8. 69 e02 9. 61 e01 9. 72 e01 9. 89 e01 6. 22 e01 9. 44 e01 2. 83 e01 N A N A Ta sk 14 C N N +D M 2. 17 e12 6. 64 e02 1. 11 e01 2. 05 e02 3. 66 e02 4. 52 e01 9. 10 e01 8. 24 e01 N A Ta sk 15 C N N +D M 1. 36 e52 5. 30 e03 3. 48 e02 7. 21 e02 8. 36 e01 3. 09 e01 8. 47 e01 9. 84 e01 N A Ta sk 16 C N N +D M 6. 39 e35 4. 56 e02 9. 66 e01 5. 95 e01 7. 19 e01 4. 09 e02 2. 51 e02 2. 22 e03 N A
Ta bl
e 7:
O ne
-s id
ed p-
va lu
e w
he th
er th
e m
ea n
ac cu
ra cy
of pr
etr
ai ne
d m
od el
s is
gr ea
te rt
ha n
th e
ac cu
ra cy
of th
e ra
nd om
ly in
iti al
iz ed
on es
fo re
ac h
co m
bi na
tio n of ta sk pr etr ai ni ng da ta se t. pva lu es be lo w 0. 05 ar e m ar ke d in gr ee n.

Review:
First I would like to apologize for the delay in reviewing.
summary : This work explores several experiments to transfer training a specific model of reading comprehension ( AS Reader), in an artificial and well populated dataset in order to perform in another target dataset. 
Here is what I understand are their several experiments to transfer learning, but I am not 100% sure.
1. The model is trained on the big artificial dataset and tested on the small target datasets (section 4.1)
2. The model is pre-trained on the big artificial dataset like before, then fine-tuned on a few examples from the target dataset and tested on the remaining target examples. Several such models are trained using different sub-sets of fine-tuning examples. The results are tested against the performance of randomly intialized then fine-tuned models (section 4.2).
3. The model is pre-trained on the big artificial dataset like before. The model is made of an embedding component and an encoder  component. Alternatively, each component is reset to a random initialization, to test the importance of the pre-training in each component. Then the model is fine-tuned on a few examples from the target dataset and tested on the remaining target examples. (section 4.3)
I think what makes things difficult to follow is the fact that the test set is composed by several sub tasks, and sometimes what is reported is the mean performance across the tasks, sometimes the performance on a few tasks. Sometimes what we see is the mean performance of several models? You should report standard deviations also. Could you better explain what you mean by best validation ?
Interesting and unpretentious work. The clarity of the presentation could be improved maybe by simplifying the experimental setup? The interesting conclusion I think is reported at the end of the section 4.1, when the nuanced difference between the datasets are exposed.
Minor: unexplained acronyms: GRU, BT, CBT.
benfits p. 2
subsubset p. 6