Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper introduces a modification to Differentiable Neural Computer called Distributed Associative Memory (DAM) that comprises of 1) multiple independent memory blocks and 2) association reinforcement loss (ARS). Experimentally DAM improves upon DNC on multiple tasks and is showing comparable performance to some relation-aware architectures. 

**Paper strengths**
* It is interesting to see that a relatively simple modification can bring a prominent performance boost. While it probably still requires further investigation, the fact that an architecture with factorized memory blocks can match performance of an explicitly relational architecture suggests that  this is indeed a step in the right direction and/or the relational benchmarks currently used in the community are too simple.

**Paper weaknesses**
* Clarity. ARL which is one of the two novel components of DAM is not described clearly. $l_{ar}$ is not formally defined anywhere and its textual description is rather vague. What exactly does "sampled input sequence" mean? Should it be called "subsampled" instead? Is it always valid to simply subsample individual input tokens?
* Since the improved performance presumably comes from the multiple individual memory blocks, it is important to understand how exactly information is factorized across them and how each of the blocks is used. One can argue that a wide enough representation can potentially learn the factorization scheme and mimic it using multiple reading/writing heads. To me the basic intuition behind that is provided in the Introduction is not enough.
* I appreciate that authors do compare memory capacity across different DNC variants, but then it is important to do so for all the baselines and ideally evaluate all the baselines with the same number of floating point numbers reserved for memory. Otherwise, the exact source of the improvements is not clear.
* Authors may want discuss the following paper [1] which describes a highly relevant model.

I am happy to revise my score if the points above are addressed by authors.

** References **
[1] Marblestone, Adam, Yan Wu, and Greg Wayne. "Product Kanerva Machines: Factorized Bayesian Memory." arXiv preprint arXiv:2002.02385 (2020).
review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: The paper proposes two extensions for existing memory networks (e.g., DNC) to improve associative reasoning: (a) multiple memory blocks instead of just one; and (b) self-supervised training as auxiliary tasks. Experiments conducted on toy datasets show that these extensions lead to improved performance over DNC and the method is somewhat competitive against newer variants on several synthetic tasks.

Pros:
- Understanding why MANN works and doesn't, and how to improve it are still open problems. And thus this work is welcomed.
- The empirical results are positive, suggesting the introduced ideas have merits.

Cons:
-  In terms of novelty, this work is quite incremental. The first extension is simply about dividing the external memory in DNC into multiple blocks that connect via an attentive gate. The argument of having multiple memory blocks is to diversify the representation of the same input. A similar idea has been introduced in [3], although the motivation was different. The second extension is a reconstruction loss on the input signals. To what degree it works as  “association reinforcing” as claimed is unclear, even though the loss would work as a regulariser as in standard hybrid loss in existing neural networks.
-  In terms of experiments, the tasks are too simple and particularly favor the model design in the paper. For example, in the copy task, we need faithful information from the input signals for correctly decoding (copying), which can be enhanced via the reconstruction loss during the encoding phase. The same thing applies to the associative recall task.
- Note that for both copy and associative recall, the length of the input sequence ([8, 32]) is much smaller than the number of memory slots (64) (see Appdx A.2.1). With such redundancy in storage, it is not very surprising to me that dividing the memory into smaller blocks can improve convergence.
- The main baseline used for comparison in this paper is DNC, which, in my opinion, is out-of-date as it can be outperformed easily (e.g., existing works on memory networks [1, 2] show significant improvements over DNC on these toy tasks). 


Some minor comments:
- L_ar(i_t, y_t) is not defined in the paper. Do the authors use L1, L2 or binary cross-entropy loss?

[1] Relational recurrent neural networks, Santoro et. al., NIPS-2018
[2] Improving Differentiable Neural Computers through Memory Masking, De-allocation, and Link Distribution Sharpness Control, Csordas et. al., ICLR-2019
[3] Pham, Trang, Truyen Tran, and Svetha Venkatesh. "Relational dynamic memory networks." arXiv preprint arXiv:1808.04247 (2018).

review_title: Multiple memories and reconstruction loss for MANNs

final_score: rating: 6: Marginally above acceptance threshold
review_content: The authors propose a distributed memory architecture which shares some interface with the Differentiable Neural Computer however crucially segments memory into a collection of K units. The authors show that by increasing K the model learns to use its memory for algorithmic tasks such as copying and associative recall and learn faster. The authors also propose an auxiliary loss to improve memory representations, which involves reconstructing inputs from the representations in memory. 

I think the scientific statement is quite clear here and the paper is worth accepting; the only shame is that the authors did not apply this approach to a richer task than bAbI. 

Also it would have been nice to compare the approach to a multi-head attention transformer since these also use distributed representations (across heads). 

The authors may be interested in the following architecture MERLIN which also uses a reconstruction loss to improve memory representations: https://arxiv.org/abs/1803.10760
review_title: Convincing demonstration that distributing memory improves learning

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This paper proposes an extension of the Differenciable Neural Computer networks (DNC)
In these DNCs, the reading operation on the external memory are done by accessing a single memory block, which represents a single piece of information or knowledge. The architecture proposed in this paper aims instead to give the possibility to access multiple memory blocks at the same time. In this way, the approach of reading memory is more holistic (Chalmers, 1992). This is a desirable feature of distributed representations and, then, distributed memories. Otherwise, these memories are just similar to the classical approach of representing symbols (Fodor and Pylyshyn ,1988). This debate of what is the main charateristic of distributed representations is revitalized in Ferrone and Zanzotto (2020). It is then a needed extension of DNC.
The paper is well written and results are convincing.
However, there is a minor problem. There is not a direct link among equations in Section 2 and equations in Section 3. Clearly, DNC equations are extended by equations in Section 3. Are these equations linked only with the M, that is the Memory? 

References

Fodor, J. A., and Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71.
Chalmers, D. J. (1992). Syntactic Transformations on Distributed Representations. Dordrecht: Springer.
Ferrone, Zanzotto (2020), Symbolic, Distributed, and Distributional Representations for Natural Language Processing in the Era of Deep Learning: A Survey
review_title: Official Blind Review

final_score: rating: 4: Ok but not good enough - rejection
review_content: In this work the authors propose a novel memory architecture wherein memories are stored in multiple ways across a series of memory blocks. By "distributing" the memories in such a manner, the model can flexibly retrieve one version of a memory or another, which enables more flexible computations when conditioning on that memory. The authors demonstrate that such a memory network does well in tasks involving relational reasoning. 

One advantage of memory-based models is that they amortize compute across time: they pay an upfront cost to shape and store a memory *online*, but gain an advantages at read time, where they just need to do a basic lookup among the stored memories. A consequence of these  memory-based approaches is that the models need to anticipate how they should store a memory given what might come in the future. In this work, the authors propose the strategy of storing a memory in multiple different ways, which mitigates the risk that a single stored memory will be insufficient for what might come. 

In contrast, something like a Transformer pays a heavy cost at read time, as it needs to perform a full self-attention operation (rather than simple lookup) across all stored memories. Transformer-based approaches, however, profit immensely from tasks where memories need to be shaped differently at read time, as they can use the full power of self-attention to morph and condition memories given all the information they've accumulate to that point in time. Given the recent surge of evidence of the usefulness of self-attention based models, and the fact that they can easily be interpreted and/or used as memory models, the authors would be remiss to not include a self-attention based baseline to which they can compare their model. This is especially important given that the complex nature of memory models (i.e., the complexity associated with learning how to read, write, etc.) has recently given way to a more simple approach using memory buffers and self-attention.

This is not to say that there is no value in developing memory-based approaches, as there surely is. However, the memory-based approaches should demonstrate their value in domains that play to their strengths. As stated above, these models amortize the cost of shaping memories over time, and in addition, they can keep a constant size memory indefinitely in time. In contrast, a Transformer-based memory model would grow in memory cost as time increases, and the compute at read time would similarly grow quadratically with time. Thus, to demonstrate the value of a memory-based approach over self-attention, it is wise to pit the two against one another in a regime where self-attention simply becomes too costly; in other words, in a regime where a great number of time steps (and hence memories) need to be considered. Otherwise, the reader is left wondering how well this model compares to the simpler Transformer-based approach.

The authors also propose a new loss that forces the memory contents to be able to predict a sample sequence of previously observed inputs. Unfortunately, I believe the inclusion of this loss makes the absense of a Transformer baseline even more troublesome. This is because, for this loss to be implemented, we need to keep around a buffer of previous inputs, which is precisely the memory cost associated with using a Transformer! So, given the previous discussion on how memory models can in theory maintain a constant sized memory, in the DAM this is no longer the case. Memory costs grow linearly with time because of the need to preserve inputs for use in the ARL loss. 

Altogether, the paper is well put together and written well enough to understand the ideas and experiments. The authors did well to choose experiments that would demonstrate the strengths of their approach. Unfortunately, the empirical and rational comparison to Transformer-based approaches prevents me from recommending its publication. 

There are a few minor points scattered throughout, but I'll just call attention to the following:

"insufficient associating performance"
--> It is unclear what this means.

"lossy representation"
--> This term is used throughout, but I'm not sure it's warranted. How do we know that the distributed vector representation is truly lossy with respect to the information it must encode? In princple nothing prevents it from being lossless. Given a complicated enough decoder, one wouldn just need a handful of bits to encode very complicated things losslessly.  

However, even with its promising performance on a wide range of tasks, MANN still has difficulties in solving complex relational reasoning problems (Weston et al., 2015).
--> There has been much work since 2015 that has improved MANN performance on these tasks. For example, the Sparse DNC, as eventually shown in the results section. 

Through this attention-based reading process, DAM retrieves the most suitable information for the current task from distributed representations existing in the multiple memory blocks
--> As shown in this text, and as used throughout, the term "distributed representation" is overloaded in this work. Traditionally the term "distributed representation" is used to denote a vector with real-valued elements, whereas here it is used to denote a set of such vectors, "distributed" across multiple memories. 
review_title: New memory architecture, but unfortunately misses an obvious comparison to self-attention models

Meta-review: After carefully reading the reviews and the rebuttal, and after going over the paper itself, I'm not sure the paper it ready for ICLR. I do believe there is a lot of useful content in the current manuscript, and I urge the authors to keep working on the manuscript and resubmit it in due time. 

My concerns are as follows:
 (a) there is a lot of discussion about *relational information retrieval* -- however there is lack of any formalization of what this term means. I don't mind relational reasoning to be used as motivation, but when it is used to consider what are valid baselines and what are not, I feel compelled to understand what exactly it means. Why is *self-attention* retrieval not *relational*? Beside the task being seemingly relational in spirit, how do we test whether the retrieved mechanism carries any relational information whatsoever? I think the community had a learning lesson here in CLEVER dataset, which arguably does not require as much relational reasoning as it seemed. So I agree with Rev5, that there is a decent probability that the task we are using do not require relational information retrieval. While I understand that some of these systems are Transformer inspired, I feel transformer should be a baseline. 
 (b) I also feel the paper should take one of two paths. 
       - Either embrace larger scale tasks and baseline outside of the relational reasoning literature (like transformer) and particularly settings where potentially self attention will struggle due to the quadratic term or where they tend to be hard to train due to the difficulty of doing credit assignment through the attention mechanism 
       - Provide more careful ablation studies and formalize the claims a bit more. Regarding e.g. the discussion of a single larger memory vs multiple memory blocks. One of the main difference comes from the attention over which memory block to use in the proposed approach, which due to softmax has a unimodal behavior. So is the reason why it works better this potential hiding of part of the memory representation (so a better way of reading a subset of the memory entry). This could potentially be done differently (e.g. multiplicative interaction in the same style, for e.g. that they were used in WaveNet). This is just a random thought on this particular aspect. I have similar questions about the self-supervised loss. 

 I find the paper focusing on improving performance (unfortunately on toy domains) rather than ablation studies and an understanding and careful understanding of how things works. I realize there is some such analysis in the appendix. But I feel more of it should be in the main text. The paper is either proposing something that scales and works well at scale (and then understanding why is less important as it has direct application) or explores a very specific phenomena and then is fine to stay on toy tasks but there should be a bit of clarity in the claims, and an investigation whether the hypothesis (or intuition) put forward initially is the reason why the model works. 

Decision: Reject