Cloze-style queries are representative problems in reading comprehension. Over the past few months, we have seen much progress that utilizing neural network approach to solve Cloze-style questions. In this paper, we present a novel model called attention-over-attention reader for the Cloze-style reading comprehension task. Our model aims to place another attention mechanism over the document-level attention, and induces "attended attention" for final predictions. Unlike the previous works, our neural network model requires less pre-defined hyper-parameters and uses an elegant architecture for modeling. Experimental results show that the proposed attentionover-attention model significantly outperforms various state-of-the-art systems by a large margin in public datasets, such as CNN and Children's Book Test datasets. This work was done by the Joint Laboratory of HIT and iFLYTEK (HFL).
AoA Reader proposed by Yiming(2016) REF used another new attention mechanism on the basis of AS Reader for the same cloze-style task and outperforms the state-of-the-art models.
Attention-over-Attention Neural Networks for Reading Comprehension
ACL 2017 Vol.1 Long Papers 593-602
Computer Science