Abstract: Several large cloze-style context-question-answer datasets have been introduced recently: the CNN and Daily Mail news data and the Children's Book Test. Thanks to the size of these datasets, the associated text comprehension task is well suited for deep-learning techniques that currently seem to outperform all alternative approaches. We present a new, simple model that uses attention to directly pick the answer from the context as opposed to computing the answer using a blended representation of words in the document as is usual in similar models. This makes the model particularly suitable for question-answering problems where the answer is a single word from the document. Ensemble of our models sets new state of the art on all evaluated datasets.

Introduction: Most of the information humanity has gathered up to this point is stored in the form of plain text. Hence the task of teaching machines how to understand this data is of utmost importance in the field of Artificial Intelligence. One way of testing the level of text understanding is simply to ask the system questions for which the answer can be inferred from the text. A well-known example of a system that could make use of a huge collection of unstructured documents to answer questions is for instance IBM's Watson system used for the Jeopardy challenge BIBREF0 . Cloze-style questions BIBREF2 , i.e. questions formed by removing a phrase from a sentence, are an appealing form of such questions (for example see Figure FIGREF1 ). While the task is easy to evaluate, one can vary the context, the question sentence or the specific phrase missing in the question to dramatically change the task structure and difficulty. One way of altering the task difficulty is to vary the word type being replaced, as in BIBREF3 . The complexity of such variation comes from the fact that the level of context understanding needed in order to correctly predict different types of words varies greatly. While predicting prepositions can easily be done using relatively simple models with very little context knowledge, predicting named entities requires a deeper understanding of the context. Also, as opposed to selecting a random sentence from a text as in BIBREF3 ), the question can be formed from a specific part of the document, such as a short summary or a list of tags. Since such sentences often paraphrase in a condensed form what was said in the text, they are particularly suitable for testing text comprehension BIBREF1 . An important property of cloze-style questions is that a large amount of such questions can be automatically generated from real world documents. This opens the task to data-hungry techniques such as deep learning. This is an advantage compared to smaller machine understanding datasets like MCTest BIBREF4 that have only hundreds of training examples and therefore the best performing systems usually rely on hand-crafted features BIBREF5 , BIBREF6 . In the first part of this article we introduce the task at hand and the main aspects of the relevant datasets. Then we present our own model to tackle the problem. Subsequently we compare the model to previously proposed architectures and finally describe the experimental results on the performance of our model.

Question: What is the performance of their model?
Answer: The different AS Reader models had average test accuracy of 71,35% and AS Reader (avg ensemble) had the highest test accuracy between all tested models with 75.4%

In case of Daily Mail average was  75.55% and greedy assemble had the highest value with 77.7%
CBT NE average was 69.65% and greedy ensemble had the highest value of 71% 

CBT CN had average of 65.5% and avg assemble had the highest value of 68.9%