Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: Summary:
This paper addresses abstract rule learning in high-dimensional data through constructing a recurrent neural network that exhibits a variable-binding ability. The proposed method, ESBN, is a RNN augmented with two memories, one for keys and one for values. The key memory captures the relations between items, which are important to produce symbolic answer (e.g.,  item/group index). The value memory stores the item embedding. Tested on several abstract rule learning tasks on visual inputs, ESBN exhibits excellent generalization with limited training data.

Pros:
- The problem of learning abstract rules that are decoupled from actual content (visual symbols in this case) is important, and is currently under-studied in machine learning.
- The solution, ESBN, is simple and effective. Empirical evaluations clearly show that ESBN woks well on tested tasks, while several well-studied neural  networks (LSTM, NTM, Relational network, Transformers) fail.

Cons:
- While the results are very encouraging on the simplified tasks, we still do not really know WHY the model works the way it does. After all, the relation between keys are transferred from the similarity between corresponding values. The tasks tested seem to emphasize on the object matching, and thus may not work on other types of rules, or with more variations of inputs.
- The paper is motivated well and the high-level idea is described, but certain important algorithmic/design details are missing in the main text (e.g., the role of confidence, gate). Algorithm 1 is at core of ESBN, but is left to Appendix, making it hard to follow the unrolling of the controller in step t. The concatenation of the key memory with the confidence score seems to make the dimension of k_r grows with t. Is this intended? If not, then how a fixed size input to the RNN controller is implemented?

Others:
- The paper conjectures that that the poor result of the Relation Network (RN) is due to its focus on pair-wise relations. More recent follow-ups have introduced higher-order relations (e.g., Temporal Relation Network [1] and Conditional Relation Network [2]) may be able to mitigate the issue. It would be nice to have these tested, or discussed in the context of abstract rules learning.
-  It would be important to explain/discuss the effect of temporal context normalization on ESBN.


Reference:
[1] Bolei Zhou, Alex Andonian, Aude Oliva, and Antonio Torralba. "Temporal relational reasoning in videos". In Proceedings ofthe European Conference on Computer Vision (ECCV), pages 803–818, 2018.
[2] Le, Thao Minh, et al. "Hierarchical Conditional Relation Networks for Video Question Answering." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

review_title: Decoupling symbol-like mechanism from concrete entities

final_score: rating: 7: Good paper, accept
review_content: This work introduces a set of tasks that require a simple symbolic reasoning and a new model - Emergent Symbol Binding Network (ESBN) to solve them. The tasks are designed such that it tests the model’s extrapolation ability. Specifically, they introduce novel input symbols that are unseen during training to evaluate the model. They show that previous memory-augmented neural networks fail on the tasks whereas their approach with an external memory that supports a specific variable-binding achieves an excellent performance. 

The main components of ESBN are an input encoder, a key-value memory and a LSTM controller.  The encoder is a CNN that takes an input symbol ((i.e. grayscale image) and outputs a value vector for the memory. The LSTM controller then produces a key vector by taking in a memory entry retrieved by using an attention mechanism. This retrieved memory entry consists of two parts: a key vector corresponding to a value that is selected by the attention and a confidence score. They calculate the attention and confidence scores based on the similarity between the current value and the other value vectors in the memory. The confidence score seems to be an important feature because it tells the LSTM that whether the new input symbol already existed in the memory or not.  Therefore, by design, the LSTM controller never sees the actual input symbols and it only sees their corresponding key vectors, which are in turn generated by the LSTM itself. This allows the LSMT to operate on a representation that is abstracted away from the (sensory) input and solve the reasoning tasks that require extrapolation over input symbols.

Pros:
- This is interesting work that addresses an important question in neuroscience and deep learning. 
- The tasks proposed here can be used to evaluate the other models

Cons:
- They test the model only on synthetic tasks
- The model feels ad-hoc for the task. It is unclear if the model can be useful for other more complex problems.
- I believe the confidence score is crucial as it tells LSTM whether the same object was already in the memory. Some ablation on this is useful. Specially, the confidence score seems to provide a quite reliable high-level feature for the LSTM controller, which is not necessarily a weakness but providing ablation could be nice.

Additional comments:
- Curious to see how more recent associative MANNs perform on these tasks, such as TPR-RNN (https://github.com/ischlag/TPR-RNN) and MNM (https://bitbucket.org/tsendeemts/mnm/src/master/). Since the code is already out there, it shouldn’t be too difficult to include them as baseline.
- Smolensky’s TPR, which is one of the earliest works on the topic is currently missing from the discussion

review_title: An interesting work on improving out-of-distribution generalization

final_score: rating: 7: Good paper, accept
review_content: This work proposes a neural network architecture comprising a two-stream memory structure: one block is populated with visual representations, and the other is populated by hidden state vectors from a controller. The interaction between these two blocks is suggested to be akin to a symbolic-like mechanism for indirection. Reasoning that indirection is a central component for solving problems in the abstract, the authors go on to show that, indeed, such a network can solve abstract reasoning problems out-of-distribution.

A crucial aspect of the proposed network is that the memory keys are not at all conditioned on the current visual input. This means that the model needs to learn how to produce representations that will allow it to solve tasks in an abstract sense. This is a clever maneuver, because it solves the "out of distribution" problem by not allowing the information used to solve a task to ever become out of distribution: the keys, which condition the model's output, are always produced by the LSTM, and the LSTM is never conditioned on anything that is different at test time from what it sees during training time. The difficult task of handling out of distribution data, then, reduces to the vision encoder being able to simply acknowledge that one image is different from another, and it must do so using images it has never seen before. Such a mechanism is in contrast to fully connected networks that need to produce representations from out-of-distribution images *and* reason with those representations. 

That being said, the out-of-distribution demands imposed on the visual encoder aren't as extreme as it would seem. This is because, for the current tasks, what is out-of-distribution is an object's *shape*. Convolutional encoders are predisposed through their inductive biases to detect shape differences. I would be quite interested to see how different encoders fair in this task, in this exact same two-memory network. For example, a simple random projection of the image, or an MLP would be interesting. I would hypothesize that the random projection method would work if images of the same type are identical, but would fail if images of the same type have subtle variation, as do, for example, a dataset of hand-written characters. Thinking about this further, it might be critical to also test the convolutional encoder on a dataset wherein shape types have variation between instances, because even the convolutional encoder might break down when "matching up" various instances of a type that are out-of-distribrution (whereas, in the work here, each representation it produces for a character type will necessarily be identical, even if the characters are out of distribution because each image instance is exactly the same). 

The reason I emphasize these latter points about the encoder is because the work is pitched as providing a mechanism for allowing a certain type of symbolic-like reasoning, which in theory, should be insensitive to the types of encoders. If indirection is truly the causal mechanism at play, then the representations that do the "referring" should be completely insensitive to the representations that are "referred". But, we need evidence to demonstrate that this is the case.  Otherwise, we cannot claim that abstract variable binding occurs. 

Altogether, the work is wonderfully put together. The experiments are simple, yet crisp, and they do the job. The work is well motivated, and well written, and there is ample background details to understand the models and experiments. Background work is adequately represented, and results are explained. My only reservations are in regards to the sensitivity of the encoder, and the sensitivity to the task design (particularly, the choice of using identical images per character type) to the overall performance. 
review_title: Nicely put together piece of work, but a couple more experiments might be needed

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary:

The authors present an architecture capable learning symbolic representations and rules over those symbols.  Also presented is a set of  tasks involving manipulation of symbolic rules over a set of symbols where held symbol sets may be used to measure generalization performance on the task set.  The goal of this approach is learn from high dimensional inputs useful symbolic representations which are difficult to generalize across from pixels alone.

The  architecture described in this paper is named the emergent symbol binding network (ESBN) which is composed of a two column external memory for indirection and two information processing streams, one for representing embeddings of concrete variables and another that is recurrent and trained to operate over task relevant variables.  The architecture uses Temporal Context Normalization across sequences to strengthen the models sensitivity to out-of-distribution samples.  This architecture is then trained and evaluated across a set of tasks involving symbolic entities and rules over which the ESBN must learn to generalize and is compared to other memory based architectures with a capacity for relational reasoning.  

 
Strength & Weaknesses:

The architecture divides the  memory into symbolic keys and pixel encoded values and allows gradients to flow between them (symbols -> encodings) via a similarity weighting function.  As keys are generated from an LSTM they should also encode the sequence history and learn to store the correct information over the sequence to perform the symbolic manipulation required to determine the correct answers to the tasks set out. 

The architecture to me seems not entirely novel to me.  There have been other approaches using key-value based episodic memories, for example the architecture in Fortunato et al. (2019) "Generalization of reinforcement learners with working and episodic memory".  The particular way that symbols are learned is interesting but I don't think necessarily exclusive to this model. I also have some concerns the model overfit the task.  It may have been useful to consider other the more complex tasks involving symbolic reasoning such as CLEVR or bAbI.  The tasks presented seem mainly to be toy tasks.  

The addition of the Temporal Context Normalization for improving generalization over out-of-distribution samples is a nice.  It'd be interesting to see an ablation to quantify the effect of this, for instance, how does it compare with Batch normal or when simply removed for example.

I'm surprised that the Relation Net failed at this task - were positional encodings included in the representation?  same with the transformer - this is crucial when order matters in the input sequence.

The RN performing poorly on the distribution-of-three and identity-rules tasks is surprising and might be due to 1) the relationships are limited to two entities or 2) the exclusion of sequence order info.  As such, I believe the results may not fully be reflecting the Relation Nets capacity on such a problem as I'd expect them to be able to solve these problems  e.g. The RN could easily be extended to consider all ternary relations and is mainly limited by the task definition.

More equations would have been helpful.  Possibly consider including Algorithm 1 in the main paper as it is the only description of how the model works.


Recommendation:

While I believe that this is a very interesting problem domain I think that the solution provided would have been more compelling if the scope of the problem was more ambitious and also I believe that the baselines could have been stronger.
review_title: Review of "Emergent Symbols through Binding in External Memory"

Meta-review: The paper proposes a recurrent neural network architecture for abstract rule learning. An LSTM is augmented with a two-stream memory structure: one block is populated with visual representations, and the other is populated by hidden state vectors from the RNN controller.

The authors also introduce a set of tasks that require a simple symbolic reasoning on visual inputs and strong extrapolation ability. They show that previous memory-augmented neural networks fail on these tasks, whereas their model exhibits excellent generalization with limited training data.

Pro: The work addresses an important and open question in neuroscience and deep learning. The proposed solution is simple and effective. The manuscript is well-written. It was also improved in a revised version after the first review round.

Con: The main criticism raised by the reviewers was that the considered tasks may be a too simplified synthetic task.  It would have been good to consider other the more complex tasks involving symbolic reasoning such as CLEVR or bAbI.

While this is a valid criticism, all reviewers agreed that this is an interesting and important work worth publishing. In particular, the considered question is of pivotal importance for the community and the work presents a significant progress.
Decision: Accept (Spotlight)