A DVERSARIAL R EADING N ETWORKS FOR M ACHINE C OMPREHENSION
Anonymous authors
Paper under double-blind review
A BSTRACT
Machine reading has recently shown remarkable progress thanks to differentiable reasoning models.
In this context, End-to-End trainable Memory Networks ( MemN2N ) have demonstrated promising performance on simple natural language based reasoning tasks such as factual reasoning and basic deduction.
However, the task of machine comprehension is currently bounded to a supervised setting and available question answering dataset.
In this paper we explore the paradigm of adversarial learning and self-play for the task of machine reading comprehension.
Inspired by the successful propositions in the domain of game learning, we present a novel approach of training for this task that is based on the deﬁnition of a coupled attention-based memory model.
On one hand, a reader network is in charge of ﬁnding answers regarding a passage of text and a question.
On the other hand, a narrator network is in charge of obfuscating spans of text in order to minimize the probability of success of the reader.
We experimented the model on several question-answering corpora.
The proposed learning paradigm and associated models present encouraging results.
I NTRODUCTION
Automatic comprehension of text is one of the main goals of natural language processing.
While the ability of a machine to understand text can be assessed in many different ways, several benchmark datasets have recently been created to focus on answering questions as a way to evaluate machine comprehension (Richardson et al., 2013); (Hermann et al., 2015); (Hill et al., 2015a); (Weston et al., 2015); (Rajpurkar et al., 2016); (Nguyen et al., 2016).
In this setup, the machine is presented with a piece of text such as a news article or a story.
Then, the machine is expected to answer one or multiple questions related to the text.
The task is linked to several important incomes.
First, it provides tools that will shortly help users with efﬁcient access to large amounts of information.
Also, it acts as an important proxy task to assess model of natural language understanding and reasoning.
In this context, numerous large-scale machine comprehension/QA datasets (Hermann et al., 2015); (Rajpurkar et al., 2016); (Trischler et al., 2016a); (Nguyen et al., 2016) have been recently released and have contributed to signiﬁcant advancement.
From a model perspective, neural models are now approaching human parity on some of these benchmarks and a large corpus of novel and promissing research has been produced in the domain of attention, memory and parametric model with socalled reasoning capabilities.
However, the ﬁeld is currently bounded to the paradigm of supervised learning and strictly linked to the current annotated dataset.
As a counterpart, an increasing research activity has been dedicated since the 90’s to self-play and adversariality to overcome this boundary and allow a model to exploit his own decision to improve itself.
Two famous examples are related to policy learning in games.
Indeed, TD-Gammon (Tesauro, 1995) was a neural network controller for backgammon which achieved near top player performance using self-play as learning paradigm.
More recently, DeepMind AlphaGo uses the same paradigm to win against the current world best human go player.
The major advantage of such setting is to partially release the learning procedure to the limit of an available dataset.
The dual models learn and improve their performance by acting one against the other as so-called sparing patterns.
In this paper, we adapt this paradigm to the domain of machine reading.
One the ﬁrst hand, a reader network is trained to learn to answer question regarding a passage of text.
On the other hand, a narrator network learns to obfuscate words of a given passage in order to minimize to probability of successfull answering of the reader model.
We developed a sequential learning protocol in order to gradually improved the quality of the models.
This paradigm separates itself from the current research direction of joint question and answer learning from text as proposed on Wang et al.
Indeed, in comparison to question generation as regularizer of a reader model that sounds promising, we believe adversarial training unleashs from the constraint of strict and bounded supervision and brings robustness to the answering model.
Our contributions can be summarized as follows: (1) We propose a new learning paradigm for machine comprehension based on adversarial training.
(2) We show this methodology allows to overcome the boundaries of strict supervision and provides robustness to noise in question-answering settings through a set of experiments in several machine reading corpora and (3) visualizations of the models reveals some useful insights of the attention mechanism for reasoning the questions and extracting meaning passage of a text given a question.
Roadmap:
In Section 2, we formalize our adversarial learning protocol.
Also, the reader and narrator networks are presented.
In Section 3 the corpora used for evaluation are detailed.
Section 4 presents our current experimental results.
Section 5 details several vizualizations of the decisions and attention values computed by the coupled models.
Finally, Section 6 reviews the state-of-the-art of machine reading comprehension, Memory Network models, the paradigm of self-play and its links to adversarial learning.
A DVERSARIAL R EADING N ETWORKS
Several studies have recently challenged deep machine reading models with adversarial examples as
Miyato et al.
and Jia & Liang (2017).
This kind of approach is well known in computer vision (Goodfellow et al., 2014) but seems to also affects natural language processing.
More precisely, Jia & Liang (2017) demonstrates that a large majority of the recent state of the art deep machine reading models suffers from a lack of robustness regarding adversarial examples because of their so-called oversensibility.
Indeed average accuracies were decreased by half when these models were tested on corrupted data, i.e a document with an additional sentence at the end which normally does not affect the answer.
The model we propose is built to use this adversariality as an adaptive dropout by challenging the reader with more and more difﬁcult tasks during the learning.
Indeed, we extend the concept of asymmetric self-play to train a model that we called the narrator during an adversarial game with a reader .
The narrator is acquiring knowledge about the reader behaviour during the training and it generates harder adversarial examples.
Beyond increasing artiﬁcially the size of the available dataset, this adaptive behaviour of the narrator prevents catastrophic forgetting phenomena from the reader.
In this section, we explain the protocol of adversarial training we developed for robust machine comprehension.
Then, we describe the reader and narrator models used.
2. M AIN LEARNING PROTOCOL
The overall framework is a turn-based question answering game described in Figure 1.
At the beginning of each round, the narrator obfuscates one word for each document sampled from the training corpus.
We ﬁx the ratio of corrupted data / clear data to a ratio λ ∈ R [0 , 1] of the dataset.
Indeed, a too low percentage of corrupted data might have any effect on the training and a too high one will prevent the reader of learning well.
Then, the reader is trained on a subset of this obfuscated corpus and tested on the remaining subset.
Note that both train and test sets contain corrupted data.
Finally the narrator gets back a set of rewards regarding the reader performances on the obfuscated stories.
Given a tuple ( d, d obf , q ) where d is the original document, d obf the document with an obfuscated word proposed by the narrator and q the associated question, the reward r given to the narrator is deﬁned as follow:
[Equation]
The reward given to the narrator is a direct measurement of the impact of the obfuscation on the reader performance.
All the previously collected rewards are stored and used for experience replay throughout the turns.
After each learning turn, all the parameters of the narrator are reinitialized and retrained on all the recorded rewards.
Throughout the turns, the narrator accumulates information about the reader behaviour and proposes more challenging tasks as the game is playing.
Each narrator’s dataset is choosen to maximizes its expected rewards for 80% of the stories and randomly obfuscates a word in the remaining 20% in order to ensure exploration.
Finally, the reader keeps improving through the turn and any catastrophic forgetting is compensate at the next turn of the narrator by especially focusing on these ﬂaws.
Finally, categorical cross entropy L Reader =
− (cid:80) Ni =
y i log ( (cid:98) a i )
+ (1 − y i ) log (1 − (cid:98) a i ) , is the loss function for the reader network as the model decision is a distribution over a vocabulary.
Then, binary cross entropy L Narrator = − (cid:80)
Ni =1 (cid:80)
vj
=1 y ij log ( (cid:98) a ij ) is used as loss function for the narrator network.
B ASELINE PROTOCOL
As a reference protocol, one word is obfuscated in several stories of the dataset using a uniform sampling strategy.
This is a naive variation of the ﬁrst protocol where the narrator doesn’t learn from the reader feedbacks.
In fact, this protocol is similar to a dropout regularization that allows to avoid overﬁtting the training set.
However without the narrator learning of the ﬁrst protocol, we lost the adaptive dropout and all the curriculum learning notions of easier and harder inputs.
In practice this simple adversarial protocol improves the robustness of the results compared to a standard learning protocol.
This learning protocol have strong similarities with the one proposed by
Maaten et al.
2. R EADER N ETWORK
We use a Gated End-to-End Memory Network, GMemN2N , as reader which was ﬁrst introduced by Perez & Liu (2016).
This architecture is based on two different memory cells and an output prediction.
An input memory representation { m i } and an output representation { c i } are used to store embedding representations of inputs.
Suppose that an input of the model is a tuple ( d, q ) where d is a document, i.e. a set of sentences { s i } and q a query about d , the entire set of sentences is converted into input memory vectors m i =
A Φ( s i ) and output memory vectors c i =
C Φ( s i ) by using two embedding matrix A and C .
The question q is also embedded using a third matrix B , u = B Ψ( q ) of the same dimension as A and C .
where Φ and Ψ are respectively the sentence embedding function and the question embedding function described in the next paragraph.
The input memory is used to compute the relevance of each sentence in its context regarding the question, by computing the inner product of the input memory sentence representation with the query.
Then a softmaxis used to compute the probability distribution.
The response o = (cid:80) i p i c
i from the output memory is the sum of the output memory vectors { c i } weighted with the sentence relevances calculated before p i =
softmax ( u T m i ) , where softmax ( a i ) =
e a i / (cid:80) j ∈ [1 ,n ] e a j .
A gated mechanism is used when we updated the value of the controller u :
[Equation]
Finally, assuming we use a model with K hops of memory, the ﬁnal prediction is (cid:98) a = softmax ( W ( o K + u K )) where W is a matrix of size d × v and v is the number of candidate answers.
In this model, we do not use the adjacent or layer-wise weight tying scheme and all the matrix A k and B k of the multiple hops are different.
Text and question representations: To build the sentence representations, we use a 1-dimensional
Convolutional Neural Network (CNN) with a list of ﬁlter sizes over all the sentences as proposed in
Kim (2014).
Let [ s 1 , . . .
, s N ] be the vectorial representation of a document with N sentences where s i =
[ w i, 1 , w i, 2 , . . .
, w i,n ] is the
i − th sentence which contains n words.
Given a convolutional ﬁlter F ∈ R h ∗ d where h is the width of the convolutional window, i.e the number words it overlaps, the convolutional layer produces:
[Equation]
where (cid:12) is the elementwise multiplication, f a rectiﬁed linear unit (ReLU), b a bias term and E the embedding matrix of size d ∗ V where V is the vocabulary size and d the word embedding size.
Then, a max pooling operator is applied to this vector to extract features.
Given a ﬁlter F , after a convolutional operation and a max pooling operation, we obtain a feature ˆ c i = max j ( c i,j ) from the i − th sentence of the text.
Multiple ﬁlters with varying sizes are used.
Assume that our model uses N s different ﬁlter sizes and N f for each size, we are able to extract N s × N f features for one sentences.
The ﬁnal representation of the sentence Φ( s i ) =
[ˆ
[Equation] [Equation]
Nf ] is the concatenation of the extracted features from all the ﬁlters.
2. N ARRATOR N ETWORK
The objective of this model is to predict the probability of the reader to successfully respond to a question given a document with an obfuscated word.
This information will be use by the narrator to determine the position of the obfuscated word in the document which maximizes the probability of the reader to fail its task.
We use a GMemN2N similar to the reader.
However, on the last layer a sigmoid function is used to predict the probability of the reader to fail on this input: (cid:98) a = σ ( W ( o K + u K ))
where σ = 1 1+ e − x and (cid:98) a ∈ [0 , 1] is the predicted probability of failure of the reader and W a matrix of size
d × 1 .
An input of the reader is a tuple ( d obf , q ) where d obf is a document with an obfuscated word.
To obfuscate a word, we replace it by the word unk for unknow .
The output of the narrator is a real number r ∈ R [0 , 1] which is the expected probability of the reader to fail on the question.
The objective of the narrator is to select the stories which maximize this reward.
Finally, we use the same text passage and query representation than for the reader, based on a CNN with different ﬁlter sizes for the document and the two last hidden states of a bidirectional Gated Rectiﬁed Unit (GRU) recurrent network for the question encoding.
Both models are fully-differentiable.
D ATASETS AND D ATA P REPROCESSING
Cambridge Dialogs : We use the transactional dialog corpus proposed by Wen et al.
(2016) which was generated with a crowdsourcing version of the Wizard-of-Oz.
It was originally designed for a task-oriented dialog system but Perez (2016)) have shown that this task can also be considered as a reading task.
In such setting, the informable slots provided as metadata to each dialog were used to produce questions about the dialog comprehension.
The dataset deals with an agent assisting a user to ﬁnd a restaurant in Cambridge, UK.
To propose the best matching restaurant the system needs to extract three user constraints which correspond to the informable slots in the dialog state tracking task: Food, Pricerange, Area .
Given a dialog between an agent and a user, this three informable slots become questions for the model we propose.
The dataset contains 680 different dialogs aboutdifferent restaurants.
We preprocess the dataset to transform it into a question answering dataset by using the three informable slot types as questions about a given dialog ?? .
After this preprocessing operation, we end up with our question answering formatted dataset which contains 1352 inputs.
TripAdvisor aspect-based sentiment analysis : the dataset contains hotel reviews from the TripAdvisor website Wang et al.
This dataset contains a total of 235 K detailed reviews abouthotels.
Each review is associated to an overall rating, between 0 and 5 stars.
Furthermore, 7 aspects: value, room, location, cleanliness, checkin/front desk, service, and business service are available.
We transform the dataset into a question answering task over a given review.
Concretly, for each review a question is the aspect and we use the number of stars as answer.
This kind of machine reading approach to sentiment analysis was previously proposed in Tang et al.
Children’s Book Test (CBT) : the dataset is built from freely available books (Hill et al., 2015b)
thanks to Project Gutenberg 1 .
The training data consists of tuples ( S, q, C, a ) where S is the context composed by 20 consecutive sentences from the book, q is the query , C a set of 10 candidate answers and a the answer .
The query q is the 21 st sentence, i.e the sentence that directly follows the 20 sentences of the context and where one word is removed and replaced with a missing word symbol.
Questions are grouped into 4 distinct categories depending of the type of the removed word:
Named Entities (NE), (Common) Nouns (CN), Verbs (V) and Prepositions (P).
The training contains 669 , 343 inputs (context+query) and we evaluated our models on the provided test set which contains 10 , 000 inputs, 2 , 500 per category.
E XPERIMENTS 4. T RAINING D ETAILS
10% of the dataset was randomly held-out to create a test set.
We split the dataset before all the training operations and each of the protocol we propose was tested on the same test dataset.
For the training sessions, we split the training dataset to extract a validation set to perform early stopping.
We use Adam optimizer (Kingma & Ba, 2014) with a starting learning rate at 0 . 0005 .
We set the dropout to 0 .
9 which means that during the training session, 10% , randomly selected, of the parameters are not used during the forward pass and not updated during the backward propagation of error.
And we also add the gated memory mecanism of Perez & Liu (2016) that dynamically regulates the access to the memory blocks.
This mechanism had a very positive effect on the overall performances of our models.
All weights are initialized randomly from a Gaussian distribution with zero mean and σ = 0 .
Moreover, we penalize the loss with the sum of the L 2 of the parameters of the models.
Because some of the stories are quit long, we set a small batch size to 16 inputs and we use embedding word of size 300.
We initialize all the embedding matrix with pre-trained GloVe word vectors (Pennington et al.
and we randomly initialize the words of our document that are not in the
GloVe model.
It seems that for our experiments CNN encoding doesn’t improve only the overall accuracy of the model compared to LSTM but also the stability by decreasing the variance of the results.
So in practice we use 128 ﬁlters of size 2, 3, 5 and 8 so a total of 512 ﬁlters for the one dimensional convolutional layer.
We repeat each training 10 times for the two ﬁrst datasets and report maximum and average accuracy on the test set.
The maximum is the score on the test set of the best of the 10 trained models based on the validation set.
During the adversarial learning, the dataset contains 70% of clear dialogs and 30% of corrupted dialogs, λ = 0 .
Inside these corrupted data, 20% are randomly obfuscated by the narrator in order to make it learn from exploration and the narrator maximize his reward for the remaining 80% .
Eventually to ﬁt with the format of the dataset, we slightly modify the output layer of our reader for the CBT task.
Instead of projecting on a set of candidate answers the last layer of the reader make a projection on the entire vocabulary (cid:98) a = σ ( M (cid:12) W ( o K + u K )) where W is a matrix of size V ∗ d with V the vocabulary size, (cid:12) the elementwise product and M the mask vector of size V containing 1 if the corresponding word is proposed in the candidate answers 0 otherwise.
4. R ESULTS
[Table] [Table]
Performance results on the Cambridge dataset and TripAdvisor are displayed in table 1.
We present the results of our implementation of a standard GMemN2N , a uniform GMemN2N which is the reader trained with the baseline protocol 2.2 and the GMemN2N trained in the adversarial protocol 2.
against the narrator.
Each of the experiment was run 10 times and we report in this table the maximum score on the test (based on validation set) and the average score.
The precise number of hops needed to achieve the best performance with such models is not obvious so we are presenting all the results for reader and narrator between 4 and 6 hops.
Adaptive adversarial GMemN2N improves the accuracy of the model on the Cambridge task by 2.3 points for a model with with 6 hops.
The best performance on the TripAdvisor dataset was achieves by the adversarial GMemN2N with 4 hops.
It improves the accuracy by 1.5 points.
The uniform protocol improves the stability of the performances compare to a standard reader
but we went further with the adversarial protocol which improve both the overall accuracy and the stability of the performances.
It is not clear for this task that the number of hops, between 4 and 6, has an inﬂuence on the general behaviour but we achieve the best performance with our adversarial protocol and a reader with 6 hops.
All the average values of the models trained with the adversarial protocol are higher than the others , even for the 5 hops model which doesn’t achieve a very good max performance during the 10 replications we have run.
Performances on the CBT dataset are displayed in table 2.
Due to the large number of data in this dataset, we didn’t have time to repeat the training 10 times but only once.
Results of the uniform training seem similar to the performances of the standard reader in this case but the accuracy of the models trained with our adversarial protocol remain stronger than others.
[Figure]
V ISUALIZATIONS AND A NALYSIS 5. N ARRATOR PREDICTIONS
In order to better understand what the narrator learn from the reader behaviour during the adversarial protocol, Figure 2 depicts the rewards that the narrator expects for each word of a document after several rounds of the game.
Given a tuple ( d, q ) where d is a clear document and q a query and assuming the document contains k words, we generate k corrupted documents where one word is obfuscated in each of them.
We then feed the narrator with these corrupted data and report the results.
y-axis represents the document and x-axis the expected reward from the reader if the narrator decides to generate a corrupted document by obfuscated this word.
In red, we highlight the words of the documents that correspond to the answer of the question.
We can see that the narrator tends to obfuscate some important keywords of the dialogs.
We also observed that the narrator is not pointing on a single word but it points on a word and on its neighborhood.
We think that this is the result of the encoding which is not only a representation of a word but a representation of a word in its context.
5. S TANDARD VS A DVERSARIAL R EADER ATTENTION
Figure 3 depicts the attention values, presented by hops, over a document from the Cambridge dataset.
The document was choosen as only the adversarial protocol answers correctly to the question.
It displays attention distributions for a reader trained with the three different protocols: [top] standard, [middle] uniform, [bottom] adversarial.
The overall aspect seems similar for the two ﬁrst readers.
The readers quickly focus on what we assume to be an important span of text.
After two hops the readers start looking at the same position in the document.
On the contrary, the reader trained in the adversarial protocol seems to have a very different behavior regarding the attention mechanism.
It captures the important part of the sentence directly with the ﬁrst hop and uses theremaining hops to focus more largely on the end of the document.
We might interpret this as a consequence of the obfuscation protocol that forces the reader to look on different parts of the sentence instead of focusing on one precise point during the learning process.
R ELATED W ORK 6. E ND - TO -E ND M ACHINE R EADING
The task of end-to-end machine reading consist in learning to select an answer to question given a passage of text in supervised manner.
One of the popular formal setting of the problem, the clozestyle QA task, involves tuples of the form ( d, q, a, C ) , where d is a document (context), q is a query over the contents of d , in which a phrase is replaced with a placeholder, and a is the answer to q , which comes from a set of candidates C .
In this work we consider datasets where each candidate c ∈ C has at least one token which also appears in the document.
The task can then be described as: given a document-query pair ( d, q ) , ﬁnd a ∈ C which answers q .
Below we provide an overview of representative neural network architectures which have been applied to this problem.
LSTMs with Attention: Several architectures introduced in Hermann et al.
(2015) employ LSTM units to compute a combined document-query representation g ( d, q ) , which is used to rank the candidate answers.
These include the DeepLSTM Reader which performs a single forward pass through the concatenated (document, query) pair to obtain g ( d, q ) ; the Attentive Reader which ﬁrst computes a document vector d ( q ) by a weighted aggregation of words according to attentions based on q , and then combines d ( q ) and q to obtain their joint representation g ( d ( q ) , q ) ; and the Impatient Reader where the document representation is built incrementally.
The architecture of the Attentive Reader has been simpliﬁed recently in Stanford Attentive Reader, where shallower recurrent units were used with a bilinear form for the query-document attention Chen et al.
Attention Sum: The Attention-Sum (AS) Reader Kadlec et al.
(2016) uses two bidirectional GRU networks to encode both d and q into vectors.
A probability distribution over the entities in d is obtained by computing dot products between q and the entity embeddings and taking a softmax.
Then, an aggregation scheme named pointer-sum attention is further applied to sum the probabilities of the same entity, so that frequent entities the document will be favored compared to rare ones.
Building on the AS Reader, the Attention-over-Attention (AoA) Reader Cui et al.
(2016) introduces a two-way attention mechanism where the query and the document are mutually attentive to each other.
Multi-hop Architectures: Memory Networks (MemNets) were proposed in Weston et al.
(2014), where each sentence in the document is encoded to a memory by aggregating nearby words.
Attention over the memory slots given the query is used to compute an overall memory and to renew the query representation over multiple iterations, allowing certain types of reasoning over the salient facts in the memory and the query.
Neural Semantic Encoders (NSE) Munkhdalai & Yu (2016) extended MemNets by introducing a write operation which can evolve the memory over time during the course of reading.
Iterative reasoning has been found effective in several more recent models, including the Iterative Attentive Reader ?
and ReasoNet Shen et al.
The latter allows dynamic reasoning steps and is trained with reinforcement learning.
Other related works include EpiReader Trischler et al.
(2016b) consists of two networks, where one proposes a small set of candidate answers, and the other reranks the proposed candidates conditioned on the query and the context; Bi-Directional Attention Flow network (BiDAF) Seo et al.
(2016) adopts a multi-stage hierarchical architecture along with a ﬂow-based attention mechanism.
Large corpora of question-answering has been produced to permit the development of such paradigm.
A popular example of such corpus is SQuAD which deﬁnes an extractive QA task wherein answers consist of word spans in the corresponding document.
Wang & Jiang (2016) proposed one of the ﬁrst neural models for this corpus.
In their work, the authors demonstrated that learning to point to answer boundaries is more effective than learning to point sequentially to the tokens making up an answer span.
Many later studies adopted this boundary model and achieved near human performance on the task (Wang et al., 2017; Shen et al., 2016; Seo et al., 2016).
Beyond the boundary-pointing mechanism abstractive QA (Nguyen et al., 2016) has been adopted to more open-ended tasks.
6. A DVERSARIAL L EARNING AND S ELF -P LAY
The main principle of self-play consist in deﬁning a learning task where two, possible antagonist behaviours, will be learnt jointly by competing from one against the another.
In the context of twoplayer zero-sum games, such setting falls quite naturally.
Two models of the same nature compete regarding the rules of the considered game and learn from their sucessive performances.
A majority of prior work has focused on learning from self-play data using temporal-difference learning in backgammon Tesauro (1995), chess Mannen (2003), or using linear regression in othello van der Ree & Wiering (2013) and more recently Go Silver et al.
In the general context of board games, the main advantage of self-play as a method of training neural network controllers lies in the fact that every position will be the result of a game position from an actual board, rather than being contrived positions that may fail to teach the network about probabilities or prevent the network from properly generalizing from the results.
In other word, self-play contributes to exhibit challenging conﬁgurations to overcome as a controller.
In such setting, the network has the advantage of having seen over several million different board positions, which would have been hardly feasible in a network trained through a crafted set of training data.
In the domain of reading, it has been recently observed that the tasks of answering to a question given a passage of text and predicting the question regarding a text passage are interesting tasks to model jointly.
So, several papers have recently proposed to use the question generation as a regularization task to improve the passage encoding model of a neural reader.
In this paper, we claim these two tasks are indeed complementary but we think adversarial training of the nature used in two player games will lead to the same advantages than those observed previously.
As generating the question given a passage and a question is hard we inspired ourself from the recent work proposed by Tesauro and deﬁne a narrator network as complementary task to the reader learning one.
Such narrator have the task of ﬁnding the most meaningfull spans of text to obfuscate in a give passage and given a question in order to minimize the probability of successfull answering of the reader.
In the next section, we detailled the two antogonist models and the adopted learning protocol.
6. A DAPTIVE DROPOUT
Recent deep neural networks are composed of a lot of parameters and tend to easily overﬁt the training set.
One of the main idea which has been developped to prevent this overﬁtting is to randomly drop units from the network during the training session Srivastava et al.
Such approach results to combine many different neural networks to make a prediction.
In the same idea of avoiding to overﬁt the training data, training a model on a dataset which contains corrupted data is something usefull which has been studying in Maaten et al.
They have developed different ways to corrupt a document, for example by adding noise into the input features and our work refers to what they call the blankout corruption which consist of randomly delete features into the input documents (texts or images in this case) with probability q .
We think that this random corruption is not the most efﬁcient way to corrupt the data and can be dynamically adapted to the performance of the reader.
Indeed the obfuscation of one of a key word can be too hard for the reader at the beginning of the training and on the contrary, obfuscate a meaningless word regarding to a question won’t have any effect on the reader if it is good enough.
The learning protocol we propose aims to handle this by training jointly the narrator and the reader in order to adapt the corruption difﬁculty to the reader performances.
C ONCLUSION AND F UTURE W ORK
In this paper, we propose an adversarial protocol to train coupled deep memory networks for the task of machine comprehension.
This adversarial protocol where a narrator explicitely corrupts documents is a way to integrate adaptative dropout into the training process.
On all reported experiments, the models trained with this novel protocol outperform the equivalent models trained in a standard way.
Moreover it also improves the quality of the training by reducing the variance of the performances of trained models.
Moreover, we observed, in accord to recent papers of Dauphin et al.
that the CNN encoding instead of traditional RNN improves the training speed and the performances of the models on all the tasks we test them.
Then, we provide a visualization of the difference of the attention mechanism between a reader trained in a standard way and one trained with the adversarial protocol and a visualization of the knowledge extracted by the narrator.
In future work, we plan to investigate more precisely this difference of attention between the models trained with this novel protocol using an active question answering task.
Moreover, we currently investigate an adaptation of such protocol to Visual Question Answering.