Venue: ICLR 2022 Track Blog

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This is a fantastic blog post describing the paper "Emergent Symbols through Binding in External Memory" presented at ICLR 2021. The major strengths are:

- The intro very nicely lays out the relevant background in an intuitive and accessible way. 
- The description of the proposed method and accompanying baselines is very thorough.
- The learned representations are explored in an extensive analysis featuring compelling visualizations. I was particularly impressed by the animations depicting the evolution of the learned representations over the course of training.
- The post concludes with an interesting set of questions for future research.

One major missing component is that the primary results (performance across different generalization regimes) are not really discussed. I appreciate the decision to focus on visualizing the learned representations, that seems like a good way to generate intuitions about what the model is doing, but probably it would also be good to at least briefly summarize the key results, especially as they pertain to the issue of out-of-distribution generalization.

Other than that, I have only a few minor corrections to the description of the methods, and some considerations about the visualizations:
- In the 'Symbolic tasks' section, in addition to mentioning the relatively small size of the training set, it would be good to also emphasize that in some cases $n$ is quite small, as this is a key part of the generalization regimes studied. Also the table shown in the section on temporal context normalization uses the variable $m$, which is what's used in the paper, rather than $n$. This should be consistent.
- The key written to memory at each time step $k_{w_{t}}$, is generated by the LSTM controller after it receives the read key from the *previous* time step $k_{r_{t-1}}$, rather than being generated from the read key at the current time step as the post states. These are not actually equivalent. Because the LSTM generates the write key for the current time step based on the read key from the previous time step, the write key at the current time step is not influenced by the image embedding at the current time step $z_{t}$.
- As reviewer FXWo points out, the confidence values are not generated using a linear layer. There is only a gain and bias term applied elementwise, rather than a full matrix of weights as in a linear layer.
- The LSTM output layers are not linear. The output layer for write keys uses ReLUs. The output layer for the read gate $g_{t}$ uses a sigmoid. The output layer for $y_{t}$ uses either sigmoid or softmax, depending on whether the task is 2-choice or N-choice.
- It's good to mention that the NTM has more sophisticated memory addressing mechanisms, but it would probably help the reader to also mention that a key difference between the NTM and ESBN is the ESBN's factorization between perception and control.
- The description of the standard transformer architecture is missing the shared MLP applied to each time step, before the time steps are averaged and passed to the final MLP.
- I don't think I agree that the permutation invariance in relation networks is what makes them perform poorly on the distribution-of-three or identity rules tasks. First, a temporal tag was concatenated to the embeddings before passing them to the RN, so the RN does actually have information about the temporal order. Second, as is mentioned in the post, a more significant issue that these tasks involve ternary relations whereas the RN is structured around binary relations. In the appendix of the original paper, there are some results showing that a version of the RN which considers ternary interactions does much better, which suggests that this is the key limitation on these tasks.
- It is difficult to visually assess the overlap of train vs. test representations when these are not shown on the same plot. I'm not sure what the best way to do this is, since combining the train and test plots will make for a lot of clutter. But it would be good to somehow facilitate a more direct visual comparison of the train vs test representations.
- I don't think I agree that the visualizations of the train vs. test representations for the distribution-of-three task demonstrate that the test representations 'do not align well with the train distribution'. It is true that there are some outliers, and these outliers visually dominate these plots, but an important question is how many data points are being visualized? I suspect (from my own version of this same analysis) that there are some very high density clusters that are not visually very salient since they are so highly overlapping. Ideally there would be some way of intuitively visualizing this density in these plots.

Overall I think this is really great work!

-Taylor Webb

review_title: Great blog post, very compelling visualizations and clear, comprehensive explanation of methods and results

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The submission presents the ICLR 2021 spotlight paper: “Emergent Symbols through Binding in External Memory”. It motivates very well the goal of the paper, which is to improve zero-shot generalization of logical tasks with visual cues. It describes clearly the tasks on which methods are going to compare against and the methodology of evaluation, and in addition it provides a largely self-contained explanation of the baseline architectures, which are qualitatively compared to the paper’s proposed architecture. Finally, some results are presented and the authors attempt to further solidify the findings of the paper by contributing extra empirical evidence via visualization of learned representations and noticing various significant invariances.

The blog post would improve if:  
* At the last paragraph of the introduction section, the authors say that “the paper … does not propose an alternative method for neuro-symbolic learning”. As opposed to what?
* At the ‘Models Evaluated’ section, when describing the read key generation mechanism, the authors state that the confidence values are computed as `c = \sigma(Linear(M_v \dot z_t))` whereas the original paper states `c = \sigma(\gamma (M_v \dot z_t) + \beta)`. The difference in the operations are not negligible. \gamma and \beta are learned gain and bias parameters that act elementwise on the vector representation `M_v \dot z_t`. Please rephrase this part.
* At the results table provided, under the ‘Temporal Context Normalization’ section, there is no mention of what variable `m` in the columns stands for. Making the blog post more self-contained improves its readability, even if it is meant as a companion to the original paper.
* In the interactive visualizations provided, it is not always apparent to what the color scheme refers to.

Strengths and weaknesses  
————————————  
\+ Nice introduction and motivation  
\+ Great report of baseline methods and literature review  
\+ Method and task description are clear  
\+ Contributes further empirical evidence  
\+ I find that the open questions raised are great research directions in the topic  
\- Not all of the visualizations provided contribute equally to solidifying the arguments of the original paper,
for example in the ‘Distribution of Three’ task, the reviewer is not sure what do we get from the key representations clustering well for each time step.

Typos  
———  
* While describing the Temporal Context Normalization technique, I find that `{z}` is not very readable as it is supposed to be.

Decision recommendation  
————————————  
I believe this will make a nice contribution to the workshop. I recommend acceptance.
review_title: Great baseline description and visualization of representations

Decision: Accept