Title: Tracking the World State with Recurrent Entity Networks
Authors: Mikael Henaff
Jason Weston
Arthur Szlam
Antoine Bordes
Yann LeCun
Venue: ICLR 2017
Keywords: Natural language processing
Deep learning

Abstract: We introduce a new model, the Recurrent Entity Network (EntNet). It is equipped
with a dynamic long-term memory which allows it to maintain and update a rep-
resentation of the state of the world as it receives new data. For language under-
standing tasks, it can reason on-the-fly as it reads text, not just when it is required
to answer a question or respond as is the case for a Memory Network (Sukhbaatar
et al., 2015). Like a Neural Turing Machine or Differentiable Neural Computer
(Graves et al., 2014; 2016) it maintains a fixed size memory and can learn to
perform location and content-based read and write operations. However, unlike
those models it has a simple parallel architecture in which several memory loca-
tions can be updated simultaneously. The EntNet sets a new state-of-the-art on
the bAbI tasks, and is the first method to solve all the tasks in the 10k training
examples setting. We also demonstrate that it can solve a reasoning task which
requires a large number of supporting facts, which other methods are not able to
solve, and can generalize past its training horizon. It can also be practically used
on large scale datasets such as Children’s Book Test, where it obtains competitive
performance, reading the story in a single pass.

TL;DR: A new memory-augmented model which learns to track the world state, obtaining SOTA on the bAbI tasks amongst other results.

Review discussion:
I really enjoyed the visualization in Table 4. Have you checked key vectors in the case where you do not explicitly tie them to see if there is anything interpretable still, e.g. by looking at which s_t activates which w_j? Or is it mostly distributed and does not focus to a single concept/topic/entity?
Good question! We checked in the case using BoW encoding and untied keys, and found that the embedding for a given entity was often close to more than one key vector (in terms of cosine distance). We also plotted gate activations and found that more than 2 or 3 gates can open at a time (for the examples we looked at, up to 7). This suggests that the model is storing information distributed across several memory slots.
1. Error values in tables 1 and 2 range from 0 to 1 or 0 to 100? I can guess that it is from 0 to 1, but good to make it explicit in the caption.
2. You say that you report NTM results in Table 3, but it is missing.
3. Can you please modify table 4 and make the story part and key/NN part two separate figures or tables? My initial impression from the table was that for each line in the story you have a key/NN. It is quite confusing.
4. For CBT, authors say that setting U=V=0 and W=I helps. What is the performance of the general EntNet without these changes? Is it too worse? I think it is good to report those numbers as well.
5. I think EntNet+BoW results should go to Table 3 (and not to appendix). In table 3, all the models are not exactly comparable since different models use different encoder representations.
So authors should report the performance of EntNet with proposed encoder representation and EntNet with BoW encoder representation in the main table itself.
6. How much time does it take to train EntNet on babi tasks? 10 runs with different initialization for all tasks seems to be too much for me. It is worth investigating and solving the variance problem than simply increasing the computation.
Thank you for your helpful feedback. To answer your questions:
1. Yes, in Tables 1 and 2 the error ranges from 0 to 1. We will make this clear.
2. Thank you for pointing this out, we will add the NTM results to the table.
3. We will make the change to the figure.
4. We will run this experiment and add the results to the paper soon.
5. We included the results comparing EntNet to MemN2N using the exact same BoW encoding to help understand the effects of the model architecture while keeping the encoder fixed. Since none of the other published models shown in Table 3 use simple BoW (which is generally less effective than other encoding schemes), we felt it would be confusing to include the results with BoW encoding in Table 3. We view Table 3 as the “best results” for each model architecture, and Table 6 as an ablation study to understand the effects of architecture and weight tying. We will clarify this distinction in the paper.
6. With our current implementation, the EntNet takes about 3x longer to train than a MemN2N. We very much agree that trying different initializations is a suboptimal approach, and we are currently investigating the variance problem.
However, this is a common issue with models evaluated on the bAbI tasks and not specific to our model (all models reported in Table 3 except D-NTM select the best out of 10 or more runs).
1. The paper mentioned it could use POS tagging to define entities for finding entities. Have you tried it for some experiments/cases?
2. If the entities are extracted from data, would there be different number of entities for a testing data?
Then the length of your hidden state h_j would be varied based on the data.
1. We did not experiment with using a POS tagger ourselves.
However, in the CBT experiments we tied the key vectors to the embeddings of the words occurring in the candidate set, which were originally chosen using a POS tagger when the dataset was constructed.
2. Yes, it is possible that there could be a different number of entities in the training and testing sets. Entities not occurring in the testing data could have their memory slots removed, which might help performance. One strategy for dealing with entities occurring in the test set but not the training set would be to add memory slots for these entities and initialize the word embeddings/key vectors with embeddings trained on a larger corpus.
Dear all,
We have updated the paper with the following changes:
- Specified error ranges for tables in Section 5.1.
- Added NTM results to main table in Section 5.2.
- Separated world model visualization table in Section 5.2 into two separate tables.
- Added CBT results for general EntNet to table in Section 5.3
- Added note in Appendix C specifying the experiments are designed to better understand effect of architecture and weight tying.
- Added note to Section 3 about how weight tying can be used to handle unseen entities.
Please let us know if you have any other questions or comments.
Thanks!
The work proposes a variant of a recurrent neural network that can selectively update a fixed number of multiple memory slots to update entity states.
The architecture is well motivated, especially with the motivating example, and the operation is shown to validate the intuition as shown in visualizations.
Experimental results, datasets and the baselines used are sufficient to quantitatively show the strength of the proposed architecture.
A limitation is failing to (explicitly) generalize to unseen entities, however this is not a trivial problem on its own and the authors have addressed to this issue and proposed several ideas as workarounds.
I consider the work as a good conference contribution.
This paper proposes a new memory augmented neural network (MANN) model called recurrent entity network (EntNet). EntNet can be considered as a bank of RNNs with gating mechanism to update the hidden states of the RNNs and the hidden states act like the memory slots. The model is very much relevant to NTM style architectures. It is known that training the controller in NTM to read/write from memory slots is challenging. EntNet cleverly pushes the complexity of the controller to individual memory slots. It is as if each slot has a controller and they all act in a distributed manner.
Authors report strong results in bAbI tasks where the model achieves state of the art performance. Synthetic world experiments justify that the model learns to capture the world dynamics.
However it is not clear if this will be scalable to complex real tasks. EntNet also achieves reasonable performance with one-shot reading of the passage in CBT task.
I see EntNet as a generalization of RNNs and has some advantage over NTMs when it comes to training complexity. This is definitely a good contribution to the conference. I see that EntNet can have several other applications in future.
Authors have provided convincing answers to my pre-review questions.
Few more questions:
1. Do you fix the size of the f vector set in equation (1)? If so, to what size in all the experiments?
2. There are so many training details in the paper which makes it difficult to reproduce the results. Can the authors release the source code to reproduce all the results in the paper? I am willing to increase my rating if authors can release the code.
Thank you for the review! To answer your questions:
1. The f_i vectors are fixed to be the same size as the embedding dimension, which is d=20 for the world model experiments and d=100 for the others. The number of f_i vectors is equal to the maximum sentence/window length. For the bAbI tasks this was typically between 5 and 10 (depending on the task), for the CBT experiments it was 5 and for the world model experiments it was 4.
So, the words in a sentence or window are embedded, each word embedding is then multiplied pointwise with the corresponding f_i vector, and the resulting vectors are summed to produce a single vector representing the entire sentence or window.
2. Yes, we will release the source code in the near future, so that others can reproduce the results.
The paper proposed a multi-memory mechanism that memorizes different information into different components/entities. It could be considered as a mixture model in RNN. This is a very interesting model and result is convincing.
A limitation is that we do not know how to generalize to some unseen entities and how to visualize what entities the model learned.

Meta-review:
role: AC
data: comment: Hi authors, 

Have you had a chance to read over the reviews and the anonymous comments posted? If you have any rebuttals this would be a great chance to add them to the discussion.

Thanks!
role: PC
data: comment: Reviewers found this work to be a "well-motivated", "good contribution", and "clever". The idea was clearly conveyed and reviewers were convinced that the approach was simpler than others like NTMs. Experiments are sufficient, and the work will likely be used in the future. 
 
 Pros:
 - Well-explained and expected to be widely implemented
 - Experimental results convincing on the tasks.
 
 Cons:
 - Several questions about "generalization to some unseen entities". 
 - Reliance on synthetic tasks unclear if "scalable to complex real tasks"
 - Training process seems quite complicated (although again simpler that NTMs)

Decision: Accept (Poster)