We introduce a neural network with a recurrent attention model over a possibly large external memory. The architecture is a form of Memory Network [22] but unlike the model in that work, it is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings. It can also be seen as an extension of RNNsearch [2] to the case where multiple computational steps (hops) are performed per output symbol. The flexibility of the model allows us to apply it to tasks as diverse as (synthetic) question answering [21] and to language modeling. For the former our approach is competitive with Memory Networks, but with less supervision. For the latter, on the Penn TreeBank and Text8 datasets our approach demonstrates slightly better performance than RNNs and LSTMs. In both cases we show that the key concept of multiple computational hops yields improved results. Our model takes a discrete set of inputs x 1 , ..., x n that are to be stored in the memory, a query q, and outputs an answer a. Each of the x i , q, and a contains symbols coming from a dictionary with V 1 arXiv:1503.08895v4 [cs.NE]
REF proposes a competitive memory network model that uses less supervision, called end-to-end memory network, which has a recurrent attention model over a large external memory.
End-To-End Memory Networks
NIPS
Computer Science