Distributed Associative Memory Network with Association Reinforcing Loss
Authors: Taewon Park
Inchul Choi
Minho Lee
Venue:  (2021)
Keywords: memory augmented neural network
distributed memory
memorization
relational reasoning

Abstract: Despite recent progress in memory augmented neural network research, associative memory networks with a single external memory still show limited performance on complex relational reasoning tasks. The main reason for this problem comes from the lossy representation of a content-based addressing memory and its insufficient associating performance for long temporal sequence data. To address these problems, here we introduce a novel Distributed Associative Memory architecture (DAM) with Association Reinforcing Loss (ARL) function which enhances the relation reasoning performance of memory augmented neural network. In this framework, instead of relying on a single large external memory, we form a set of multiple smaller associative memory blocks and update these sub-memory blocks simultaneously and independently with the content-based addressing mechanism. Based on DAM architecture, we can effectively retrieve complex relational information by integrating diverse representations distributed across multiple sub-memory blocks with an attention mechanism. Moreover, to further enhance the relation modeling performance of memory network, we propose ARL which assists a task's target objective while learning relational information exist in data. ARL enables the memory augmented neural network to reinforce an association between input data and task objective by reproducing stochastically sampled input data from stored memory contents. With this content reproducing task, it enriches the representations with relational information. In experiments, we apply our two main approaches to Differential Neural Computer (DNC), which is one of the representative content-based addressing memory model and achieves state-of-the-art performance on both memorization and relational reasoning tasks.

Under review as a conference paper at ICLR 2021
DISTRIBUTED ASSOCIATIVE MEMORY NETWORK WITH ASSOCIATION REINFORCING LOss
Anonymous authors Paper under double-blind review
ABSTRACT
Despite recent progress in memory augmented neural network research, associa- tive memory networks with a single external memory still show limited perfor- mance on complex relational reasoning tasks. The main reason for this problem comes from the lossy representation of relational information stored in content- based addressing memory and its insufficient associating performance for long temporal sequence data. To address these problems, here we introduce a novel Distributed Associative Memory architecture (DAM) with Association Reinfore- ing Loss (ARL) function which enhances the relation reasoning performance of memory augmented neural network. In this framework, instead of relying on single large external memory, form set of multiple smaller associative memory blocks and update these sub-memory blocks simultaneously and inde- pendently with the content-based addressing mechanism. Based on DAM archi- tecture, we can effectively retrieve complex relational information by integrating diverse representations distributed across multiple sub-memory blocks with an attention mechanism. Moreover, to further enhance the relation modeling perfor- mance of memory network, we propose ARI. which assists a task's target objec- tive while learning relational information exist in data. ARL enables the memory augmented neural network to reinforce an association between input data and task objective by reproducing stochastically sampled input data from stored memory contents. With this content reproducing task, it enriches the representations with relational information. In experiments, Differential Neural Computer (DNC), which is one of the representative content- based addressing memory model and achieves state-of-the-art performance both memorization and relational reasoning tasks.
apply two main approaches
INTRODUCTION
The essential part of human intelligence for understanding the story and predicting unobserved facts largely depends the ability of memorizing the past and reasoning for relational information based on the pieces of memory. In this context, research on artificial intelligence has focused on designing human like associative memory network which can easily store and recall both events and relational information from a part of information.
In neural network research, many approaches generally model sequential data with memory sys- tems, such as Long Short Term Memory (LSTM) (Hochreiter & Schmidhuberi 1997) and memory- augmented neural networks (MANN). Especially, recent approach in MANN constructs associa- tive memory with a content-based addressing mechanism and stores both input data and its relational information to single external memory. MANN has already proven to be an essential component on many tasks which need long term context understanding (Weston et al,, 2014 Sukhbaatar et al.) 2015| Graves et al. 2014: 2016, Gulcehre et al. 2018). Also, compared to recurrent neural networks, it can store more information from sequential input data and correctly recall desired information from memory with a given cue. However, even with its promising performance on a wide range of tasks, MANN still has difficulties in solving complex relational reasoning problems (Weston) et al. 2015). Since content- based addressing model implicitly encodes data item and its relational information into one vector representation, they often result in a lossy representation of relational information which is not rich enough for solving relational reasoning tasks. To address such weak- ness, some researches find relational information by leveraging interaction between memory entities

---

Under review as a conference paper at ICLR 2021
with attention (Palm et al. 2018 Santoro et al. 2018 Others focus long sequence memoriza- tion performance of memory (Trinh et al. 2018; Le et al. 2019 Munkhdalai et al. 2019). Another attempts apply self-attention to memory contents and explicitly encode relational information to a separate external memory (Le et al. (020b). However, all those models need to explicitly find relational information among memory entities with highly computational attention mechanism and have to repeatedly recompute it on every memory update.
In this research, we approach the same problem in a much simpler and efficient way which do not require any explicit relational computation, such as self-attention, among memory entities. We hy- pothesize that lossy representation of relational information (Le al., 2020b) in MANN caused by both a single memory based representation and though MANN learns to correlate sequential events across time, its representation not rich enough to reflect complex relational information existing in input data. Therefore, for the enhanced relation learning, we focus on the richness of representation which implicitly embeds associations existing in input data. For this purpose, we introduce novel Distributed Associative Memory (DAM) ar-
data association performance. Al-
chitecture which is inspired by how the biological brain works (Lashley) T950 Bruce, 200T). In DAM, we replace the single external memory with multiple smaller sub- memory blocks and update those memory blocks simultaneously and independently. The basic operations for each associa- tive memory block are based on the content-based addressing mechanism of MANN, but its parallel memory architecture allows each sub-memory system to evolve over time independently. Therefore, similar the underlying insight of multi-head attention (Vaswani al., 2017). our memory model can jointly attend to information from different representation subspaces at different sub-memory blocks and is able provide a more rich representation of the same common input data. To retrieve rich information for relational reasoning, we apply a soft-attention based interpolation to the diverse representations distributed across multiple memories.
Moreover, to enrich long-term relational information in the memory, we introduce novel associa- tion reinforeing loss (ARL) which fortifies data associations of the memory and generally enhances the memorization capacity of MANN. The ARI. forces the memory network to learn to reproduce the number of stochastically sampled input data only based on the stored memory contents. As if, other associated pieces of memory are reminded together whenever person recalls certain event in his memory, the data reproducing task enables MANN to have better association and memorization ability for input data. It is designed to reproduce a predefined percentage of input representations in the memory matrix on average and, while optimizing two different tasks at the same time, keep the balance between ARL and target objective loss by dynamically re-weighting each task (Liu & Zhou 2006 Cui et al. 2019).
By combining the above two approaches, DAM, and ARL., our architecture provides rich representa- tion which can be successfully used for tasks requiring both memorization and relational reasoning. We apply our architecture to Differential Neural CoMp (Graves al. 2016), which one of the representative content- based addressing memory, to construct novel distributed associa- tive memory architecture with ARL. DNC has promising performance on diverse tasks but also known to be poor at complex relational reasoning tasks. In experiments, we show that our archi- tecture greatly enhances both memorization and relation reasoning performance of DNC, and even achieves the state-of-the-arr records.
DIFFERENTIABLE NEURAL COMPUTER
We first briefly summarize DNC architecture which is a baseline model for our approaches. DNC (Graves al. (016)) is a memory augmented neural network inspired by conventional com- puter architecture and mainly consists of two parts, controller and an external memory. When input data are provided to the controller, usually LSTM, it generates a collection of memory op- erators called as interface vector for accessing an external memory. It consists of several keys and values for read/write operations and constructed with the controller internal state he as &. Weh, at each time step t. Based on these memory operators, every read/write operation on DNC is performed.
During writing process, DNC finds writing address, w [0, 1]a, where A is memory address size, along with write memory operators, e.g. write-in key, and built- functions. Then it write-in values, en R2, in the external memory, M AxL, along with erasing value, e,
updates
2

---

Under review as a conference paper at ICLR 2021
Association Reinf orcing Loss
iy-2
1
Parget Task Loss
M [-1
M2
Attentive gate
h4
Tt
Controller
K Memory Blocks
Sampled input sequence
(a)
(b)
Figure 1: (a) The DAM with K sub-memory blocks and attentive interpolation, g.t (b) Association Reinforcing Loss.
[0, 1]4, where I. is a memory length size as follows:
M, M o (E w," e, + w. Ui T
where denotes element- multiplication and E is 1 AxL
In the reading process, DNC searches a reading address, w. E [0, 1]4, for R read heads, along with read memory operators, e.g. read-out key. Then, reads out information from the external memory:
r M.W,
Finally, the output is computed y: Wy r.] E Rd,, where fr E 1 sis R). Through these operations, DNC can learn how to store input data and utilize stored information to solve given task. These whole mechanisms make DNC suitable for general purposed memory augmented neural network.
3 PROPOSED METHOD
In this section, we introduce two methods that improve both memorization and relational reasoning ability of conventional DNC, a distributed associative memory architecture, and an ARL function. For a clear explanation, we illustrate DAM mechanism with single read head case. For R read head cases of DAM, the details are in the Appendix.
3.1 DISTRIBUTED ASSOCIATIVE MEMORY ARCHITECTURE
The distributed associative memory architecture consists of a controller network and K associative memory blocks where each memory block is content addressable memory similar the original DNC (Graves et al., 2016). Figure a () shows the overall read/write process of the proposed DAM. For the writing operation, the controller of DAM produces multiple writing operator vectors for mul- tiple memory blocks. Each writing operator vector is used for the content-based addressing of one of the multiple memory blocks, and is independent of other memory blocks. Since produced based on the current input and previous hidden states of the controller, it can independently store its own representation of the same input contents. This writing process enables DAM to store the diverse representations of the same input data to multiple memory blocks with much flexibility. Fur- thermore, for the reading process, all memory blocks are read at the same time, and read values are interpolated with soft attention to produce single read-out information. Through this attention-based reading process, DAM retrieves the most suitable information for the current task from representa- tions distributed in the multiple memory blocks. Based these read/write operations, DAM learss how to store and retrieve the diverse representations of input data for different purposed tasks. The following sections detail the main operations.
3

---

Under review as a conference paper at ICLR 2021
3.1.1 CONTROLLER FOR MULTIPLE ASSOCIATIVE MEMORY BLOCKS
At each time step the controller receives an external input, &, read-out of the previous time step, 7:-1, and previous hidden state of controller, h to update its current hidden state, h4 After layer normalization, it produces an interface vector, & E [Ka[<=n+3L+33 which includes read and write parameters for multiple memory access.
3.1.2 WRITE INTO MULTIPLE
BLOCKS
The multiple memory writing processes in our architecture based on the content- based memory accessing mechanism of DNC. A single memory block is addressed and updated with the same procedure DNC and such single memory block updating applied to blocks independently at the same time. As shown in Eq. 33. each memory block has its own interface vector relevant weight Wg...... Wg,,, where & E {1.... K}. Theses weights are multiplied with a controller hidden state vector, ht, and used for memory operations of each independent memory block following.
f: = 1 Et, 0ed1 = [W...... W.,K. Wa,
where Er, is interface vector for each memory block and 9 is an attentive gate at time
Based on a writing operator obtained from Ér DAM updates input information into each memory block, M4-1,k, independently and simultaneously, following Eq. (T). That independent and simul- taneous writing procedures of sub-mes mobomey blocks allow that our DAM learns to construct diverse representations for the same common input data.
The following attention-basee reading process is designed to integrate representations distributed across sub-memory blocks, and it contributes enrich representation for relational reasoning tasks.
3.1.3 READ FROM MULTIPLE SUB-MEMORY BLOCKS
As in the writing process, DAM obtains reading operator from &. and computes reading address, "h, E [0, 1] for each memory block. Based on those addresses, DAM reads values from each memory block and derives read-out value, E RR,, from them, using processed attentive gate, gf [0, 1] as follows:
K
Ti Σ gik M M. where g.t Softmaz(0(k)) for k 1,... ......., K.
(4)
Compared to Eq. of DNC, this reading process integrates representations stored in multiple memory blocks with attention gate and enables DAM to learn to provides the most appropriate distributed representation for target task.
3.2 ASSOCIATION REINFORCING LOSS
To enhance the association performance of memory network, we design novel auxiliary task, As- sociation Reinforcing Loss (ARL), which can further improve the memorization performance of any given MANN. The main role of ARI task is forcing a memory network to reproduce sampled input data based on its memory content. However, when ARL task trained with the main target task of the model in multi-task learning setting, main task related representation and its encoded asso- ciation can be further emphasized while training (Caruana & De Sa 1997 Bavid & Schuller,
2003 Alonso & Plank 2016 Rei 2017).
we define a task-specific target objective function of conventional sequential MANN as follows:
Coakk
Σ A(t) yt)
(5)
where T is whole sequence size, A(t) is function at time t, which indicates whether current phase in answer or not, its value is 1, then is in answer phases (otherwise 0). o, is a target answer and (east(;,))) .) is task target dependent loss function.
4

---

Under review as a conference paper at ICLR 2021
Our ARL function is defined to use sampled input sequence its target data as shown in Eq. (6) and this procedure leads the model to rememorize given input information while it is learning the given task.
4 (ar yt)
(6)
where -:)(: is input sequence dependent loss function., and é, is an input, y: is output time step 1, respectively. The error measure for ARL is adopted based on the input item type or main task characteristic. In this research, we use cross-entropy loss L2 loss depending on the given task.
As shown in Fig. [b). ARL forces a model to learn to reproduce sampled input sequences from stored representations in memory. When sampling input data, each item of input sequence is sam- pled with Bernoulli trial with probability, p, which we call reproducing probability and it is defined as follows:
P(a(1) =) P(a(1) 0) =
=
where (() is an indicator function that represents sampling status at time t.
For an input sequence length n, the series Bernoulli trial based samplings is same as a Binomial sampling of the input sequence. Therefore, for any input sequence, average, up samples are reconstructed by ARL because an expected value of Binomial sampling is a product between trial probability, p. and the number of trials, This random sampling policy prevents the model from learning to simply redirect given input to the output of the model at every time step.
When adding ARL the task sask-specific target objective for multi-tas learning, we also need new strategy that can control the balance between ARL and original target task loss. Since, as the number of the story input increases, the ARL overwhelm the total loss of the model. To prevent this loss imbalance problem, we apply a re-weighting method (Cui et al.) 2019 Liu & Zhou (006), which dynamically keeps the balance between the target task objective Crask and ARL L Moreover, we also introduce a scaling factor, to ensure the main portion of training loss can be the original target objective funetion.
7 {
if 1 otherwise.
where f ΣT S(t)a(t)
(8)
E
where S(t) is an indicator function which represents whether current time step t is in the story phase or not. Finally, the total loss for the training of proposed model follows:
L = 7 ctas
c(t)
From above two memory related tasks, Ltask and L"r, while a model learns to reproduce input representations, target task related representations and its association is further emphasized the same time. As result, ARL works as an auxiliary task which reinforces data association for target objective.
EXPERIMENTS
We evaluate each of our main contributions, Distributed Associative Memory architecture (DAM) and ARL, separately for ablation study, and show the performance of DAM-AR for complex rela- tional reasoning tasks, such as bAbI, 4 farthest task. In all experiments, we adopt well-known neural network generalization techniques that are used in Franke et al. (2018) for our baseline DNC model. The detailed parameter settings and adopted generalization techniques are shown in the Appendix.
4.1 DISTRIBUTED ASSOCIATIVE MEMORY ARCHITECTURE EVALUATION
The distributed memory architecture is evaluated in two aspects. First, for model efficiency in data association performance, DAM is configured to have similar total memory size as single memory model and evaluated with Representation Recall task. Second, scalability experiments of DAM show the effect of the number of sub-memory blocks the relation reasoning performance.
5

---

Under review as a conference paper at ICLR 2021
Segments
Sagments
Copy
Associative Recall
3.0
1.0
Q.B
mo:0
DNC
modo
DNC
06
0.6
DNC
Nozil
06N
D0P-2
D000-
1300-3
0.4
0.4
3003
22000
15000
30000
2000
30200
25002
2000C
2003
4000
2003
Iteration
4000
(a) Representation Recall task
(b) Al Algorithmic tasks
DNC-AR
DAMB-AR
DNC
DAMB-AR
3.0
3.0
Accursed
Acturacy
0.8
Prote
0.5
0.6
0.5
D-0:1
p-C-3 p-t.3
p=0.3
p=0.3 3
5-0.5
p-5.5
pw0 5
(-05
0.6
fteration
Beewtton
2010
ltatation
2010
Beration
(c) Copy task for ARL
(d) Associative Recall task for ARL
Figure 2: Mean training curves on (a) the Representation Recall task and (b) the algorithmic tasks. Mean training curves for different reproducing probability values on (c) the copy task and (d) the associative recall task. The shadowed area shows a standard deviation of 10 trials.
Representation Recall Task We design a new algorithmic task, called a Representation Recall (RR) task, which evaluates how much representation details a memory model can remember and recall from memory. This task uses randomly generated binary vectors as input sequences. From the sequence, binary vector is randomly selected and divided into 2N sub-parts. Among them, N sub-parts are provided as a cue for a model to predict the rest of sub-parts. In order to solve this task, the model is required to remember N1(2N-N combinations of relations existing in each input,
2N!
and therefore task complexity increases as N increases. To show the efficiency of model with a fair comparison, we configure DAM by dividing the original single external memory to 1/2 or 1/4 sized sub-memory blocks. The mean training curves of DAM-2 and are compared with original DNC while increasing the task complexity N as in shown Fig. 2 a). The result demonstrates that our proposed architecture learns the task much faster than other DNC based models, and also shows better accuracy and learning stability (smaller standard deviation in learning curve).
Algorithmic Tasks Graves et al] (2014). The copy task is designed to show whether a model can store and recall arbitrary long sequential data correctly, and the associative recall task is intended to show whether a model can recall the information associated with a given cue by remembering temporal relation between input data. As shown in Fig. (b), simply adopting DAM architecture enhances relation recall performance of the memory model. We can obtain more benefits by adding additional sub- memory blocks (DAM-2 and 3), however, for the copy task, as shown in Fig. Cc), the effect of the number of memory blocks is small because it is not task designed for relation reasoning.
We show the effect of DAM the copy and the associative recall tasks from
Scalability of DAM For the evaluation of the scalability of dis- tributed associative memory architecture without the effect of infor- mation loss at sub-memory block, we adopt a fixed size sub-memory block that has a larger length than a half of the input size and then in- crease the number of sub-mes sub-memory blocks to produce several models, DAM-2, 3, and 4. We evaluate all model's performance with complex reasoning tasks, bAbI task, to show the effect of K (representation di- versity) on relational reasoning performance. The bAbl task (Weston et al. 2015) is a set of 20 different tasks for evaluating text under- standing and reasoning, such basic induction and deduction. In Fig. 3, the DAM-1 represents a baseline model that has a single ex- ternal memory and includes modifications from FFanke et al.) 2018)
DAM-K
Mean Error rate
DAM-1
K
Figure 3: Mean error rate of DAM models

---

Under review as a conference paper at ICLR 2021
Table 1: Test accuracy [%] on /oth farthest task. The results of other models are reported from
Santoro al. (2018); Le et al. (020b))
Model
DNC
RMC
TPR
STM
DAM6-AR (0.1)
Best
25
91
13
98
97.8
Table 2: (Top) The mean word error rate [%] for 10 runs of different DNC based models trained jointly on all 20 bAbl task.(oottom) The mean word error rate [%] for best run of MAMN models trained jointly on all 20 bAbl tasks. The results of other models are reported from Graves et al.
(2016); Rae et al. (2016); Franke et al. (2018); Csordás Schmidhuber (2019) Le et al. (2020a)
Dehghani et al. (0018) 8); Munkhdalai et al. (2019); Banino et al. (220)) Le et al. 2020b).
Model
DNC
SDNC
ISDNC
DNC-MD
NUTM
DAM2-AR (0.1)
DAMZ-AR (0.3)
Mean
16.7
6.4 +
6.3 +
9 n/a
5.6 1.9
1,5 +.3
2.5
Best
3.8
2,9
3.6
3.3
0.16
0.14
Model
Transformer
UT
MNM-p
MEMO
STM
DAM2-AR (0.1) DAM2-AR (0.3)
Best
22.1
0.29
0.175
0.21
0.15
0.16
0.14
for the generalization performance enhancement. For the comparison with DAM-K (K=2, 3, and 4), we linearly increase its single external memory size. The overall graph shows that as the degree of distribution increases, performance on bAbl tasks is also enhanced accordingly for both mean error rate and standard deviation results. If we use more sub-memory blocks to further increase we can obtain gradual performance enhancement, which clearly shows the benefits of distributed associative memory architecture.
4.2 ASSOCIATION REINFORCING LOSS EVALUATION
To show the effect of ARL on MANN, we apply it to Algorithmic Tasks (Copy and Associative Recall task). In Figs. ((c and (d), we show the mean training curves according to the reproducing probability, p. on the copy task, and the associative recall task, respectively. For DAM-AR, although we show only DAM3-AR, other configurations(AMM22. shown in the figures, the ARL function expedites the learning speed of models in most cases. For the original DNC, makes model training speed much faster and is further increased by the high reproducing probability models but DAM is not sensitive to the change of reproducing probability. From those results, see that the effect of ARL is related to the property of given task. For the evaluation of generality of ARL approach, we apply ARL to the other MANN model (RMC) and show the experimental results in Appendix.
DAM4-AR) have similar results. As
both tasks. For DAM-AR, the ARL enhances the training speed of the
4.3 DAM-AR EVALUATION ON RELATIONAL REASONING TASKS
As shown in the ablation study, each component of the proposed architecture has a significant impact on the original DNC performance. To show the performance of the whole combined model, DAM- AR, we compare our architecture other DNC's variations and attention based MANN on following relational reasoning tasks. To support our argument relational information retrieving approach, we adopts recent memory network models which are applying extensive self-attention (Le et al.) 2020b) or multi-head attention for encoding relational information (Santoro et al. 2018) our counter parts. Specifically. Self-attentive Associative Memory(STM) (Le et al. 2020b) Relational Memory Core (RMC) (Santoro et al., 2018) and Universal Transformer (UT) (Dehghani al. 2018) use self-attention and multi-head attention in their memory architecture.
th Farthest This task evaluates a capacity for relational reasoning across time. It asks a model find the M farthest vector from given query vector and this requires the memorization of

---

Under review as a conference paper at ICLR 2021
Table 3: Test accuracy [%] on Convex hull task. The results of other models are reported from Le
et al. (2020b).
Model
LSTM
ALSTM
DNC
RMC
STM
DAM6-AR (0.3)
DAM8-AR (0.3)
N-
89.15
89.92
89.42
93.72
96.85
95.6
95.4
N 10
82.24
85.22
79.47
81.23
91.88
89.8
90.5
relational information between vectors, such as distance, and sorting mechanism. With this task, long temporal relation modeling performance of a model can be demonstrated.
Table shows a comparison the N Farthest task results between our model and other MANN models which designed for relational reasoning tasks. In the results, even though, the original DNC can not solve the task at all, our DAM-AR shows surprisingly high performance on the task. Even compared to Relational Memory Core (RMC) (Santoro et al.; 2018)), which explicitly finds relational information from memory based on multi-head attention mechanism, DAM6-AR shows superior performance. For STM (Le al., 2020b), our model shows slightly lower accuracy. However, if we consider STM's self-attention computations for finding every possible relations with outer products, our DAM-AR is a quite simple and efficient architecture that does not introduce any explicit relation seeking operations or high-order storage. Therefore, in the aspect of model efficiency, DAM-AR shows comparable performance to STM.
bAbl QA task The bAbl task (Weston al. 2015) a of 20 different tasks for evaluating text understanding and reasoning, such as basic induction and deduction. Fach task consists of stories for questions and correct answers for the questions, e.g. Daniel travelled the bathroom. Mary moved to the office. Where is Daniel? bethroom. In evaluation, model is supposed to remember the story and recall related information to provide correct answer for the given questions.
Table 2 shows experimental results on the bAbl task. In this experimental result, our proposed model, DAM2-AR with p = 0.1, shows the best mean performance on the bAbl task, among all other DNC based approaches. These results demonstrate that proposed architecture efficiently learns the bAbl task by using distributed associative memory architecture and association reinforcing loss. Particularly, in Table [ the best result of DAM2-AR records the tffteoff-hhe-ar performance on the bAbl task, even compared to other types of recent MANN models.
Convex hull task The convex hull task /Vinyals et al. 2015) is predicting a list of points that forms convex hull sorted by coordinates. The input list consist of N points with 2D coordinates. In this experiment, we trained the model with N E [5, 20] and test with N = 5, 10 cases. The output sequence of 20-dimensional one-hot vectors representing the features of the solution points in the convex-hull. As shown in Table 3] DAM-AR show better accuracy than RMC (Santoro et al. 2018) and similar performance with STM (e al. 2020b).
5 RELATED WORKS
Multiple Memory based MANN In memory slot based MANNs, the content- based addressing is implemented with a dynamic long-term memory which is composed of multiple memory slots (Dani- helka et al.) 2016)] Henaff et al.] 2016 Santoro et al., 2018 Goyal et al.) 2019). For multiple memory matrix- based models, researchers improve single memory architecture by adding task-relevant formation, asynchronous data input, and relational information to an additional memory matrix (e.g. dual memory) (Munkhdalai & Yu 2017 Le et al.) 2018, 2020b). Our DAM adopts a multiple of the same type of memory matrix for distributed representation. Compared to other approaches, dis- tributed memory architecture is much simpler and shows better performance on the same problems.
Memory Networks for Relational Reasoning For relational reasoning, some MANN models explicitly find relational information by comparing their memory entities. RMC (Santoro et al. 2018) leverages interaction mechanisms among memory entities to update memory with relational information. STM (Le et al. 2020b) adopts self-attention for memory contents and store relational
8

---

Under review as a conference paper at ICLR 2021
information to separate relation memory. Compared to those methods, DAM provides relational information through diverse representations of input data and long-term association performance of memory.
Losses for Long-term Dependency For long-term memorization of the input pattern, Munkhdalai et al. (2019) used a meta objective loss which forces a model to memorize input patterns the meta- learning framework. Also, for longer sequence modeling, Trinh et al. (2018) adopted unsupervised auxiliary loss which reconstructs or predicts a sub-sequence past input data. Compared to Trinh random anchor point and the sub-sequence reconstruction
et al.] (2018), ARL does not rely rather enforces memorization of every past input data that are associated with the target task. ARL focuses enhancing data association while reproducing input representations, but also considering a balance with target objective loss by applying dynamic weighting method for dual task optimiza- tion.
CONCLUSION
In this paper, we present a novel DAM architecture and an ARL function to enhance data association performance of memory augmented neural networks. The proposed distributed associative memory architecture stores input contents to the multiple sub-memory blocks with diverse representations and retrieves required information with soft-attention based interpolation over multiple distributed memories. We introduce a novel association reinforcing loss to explicitly improve the long-term data association performance of MANN. Our ARL is designed to reproduce the contents of associative memory with sampled input data and also provides a dynamic task balancing with respect to the target objective loss. We implement with challenging relational reasoning tasks. The evaluation results demonstrate that our DAM-AR correctly stores input information and robustly recall the stored information based the purpose of the given tasks. Also, it shows that our model not only improves the learning speed of DNC, but reinforce the relation reasoning performance of the model. Eventually, our DAM-AR significantly outperforms all other variations of DNC and shows the statc-of-tthe-ar performance on complex relation reasoning tasks, bAbl, even compared to other types of memory augmented network models. As future works, we are going to optimize task balancing strategy between ARL and target loss, and perform further research on how ARL affects multiple sub-memory blocks while optimizing memory model.
novel architecture with DNC and test its performance
REFERENCES
Héctor Martínez Alonso and Barbara Plank. When is multitask learning effective? semantic quence prediction under varying data conditions. arXiv preprint 1/X00:16/ 2016.
Andrea Banino, Adrià Puigdomènech Badia, Raphael Köster, Martin J. Chadwick, Vinicius Zam- baldi, Demis Hassabis, Caswell Barry, Matthew Botvinick, Dharshan Kumaran, and Charles Blundell. Memo: A deep network for flexible combination of episodic memories. In Interna- tional Conference on Learning Representations, 2020. URL
ew. net./
Shai Ben-David and Reba Schuller. Exploiting task relatedness for multiple task learning. Learn- ing Theory and Kernel Machines, pp. 567-580. Springer, 2003.
Darryl Bruce Fifty years since lashley's in search of the engram: Refutations and conjectures. Journal of the History of the Neurosciences, 10(3):308-335 318, 2001. doi: 10. 10.1076/jhin.100 10.3. 308. 9086. URL https:: / /wwww//www.//..////////////ww tand ine com/ doi abs/ 1.000.10700 1076/ thi
3. 3.308. 9086 PMID: 11770197.
Rich Caruana and Virginia R De Sa. Promoting poor features to supervisors: Some inputs work better as outputs. In Advances in Neural Information Processing Systems, pp. 389-395, 1997.
Róbert Csordás and Jürgen Schmidhuber. Improving differentiable neural computers through memory masking, de-allocation, and link distribution sharpness control. arXiv preprint arXiv: /904. 10278, 2019.

---

Under review as a conference paper at ICLR 2021
Yin Cui, Menglin Jia, Tsung-Y- Lin, Yang Song, and Serge Belongie. Class- balsned loss based on effective number of samples. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 9268-99899 9277, 2019.
Ivo Danihelka, Greg Wayne, Benigno Uria, Nal Kalchbrenner, and Alex Graves. Associative long short-term memory. arXiv preprini arXiv: 1rXivv1602.03022 2016.
Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. Universal transformers. arXiv preprint arXiv:1807.038// 2018.
Jörg Franke, Jan Niehues, and Alex Waibel. Robust and scalable differentiable neural computer for question answering, arXiv preprini arXiv:/807.0200 2018.
Anirudh Goyal, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Bengio, and Bernhard Schölkopf. Recurrent independent mechanisms. arXiv preprint arXiv: 10893, 2019.
Alex Graves, Greg Wayne, and Ivo Danihelka. arXiv:1400.54p 2014.
Neural turing machines. arXiv preprint
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwinska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538 (7626):471, 2016.
Caglar Gulcehre, Sarath Chandar, Kyunghyun Cho, and Yoshua Bengio. Dynamic neural turing machine with continuous and discrete addressing schemes. Neural computation, 30(4))857-88. 2018.
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. Tracking the world state with recurrent entity networks. arXiv preprint 16Xiv: 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Karl S Lashley, In search of the engram. 1950.
Hung Le, Truyen Tran, and Svetha Venkatesh. Dual memory neural computer for asynchronous two- view sequential learning. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1637-1645, 2018.
Hung Le, Truyen Tran, and Svetha Venkatesh. Learning remember more with less memorization.
arXiv preprint arXiv:1901.01344 2019.
Hung Le, Truyen Tran, and Svetha Venkatesh. Neural stored-program memory. In Interna- tional Conference on Learning Representations, 2020a. URL
net. Hung Le, Truyen Tran, and Svetha Venkatesh. Self-attentive associative memory. arXiv preprint arXiv:2002.035/9, 2020b.
Forum?id-rkxx. FFD
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey Hinton. Layer normalization. arXiv preprint 160.0:6607.064500. 2016.
Xu- Ying Liu and Zhi-Hua Zhou. The influence of class imbalance on cost-sensitive learning: An empirical study. In Sixth International Conference on Data Mining (ICDM pp. 970-974. IEEE, 2006.
Tsendsuren Munkhdalai and Hong Yu. Neural semantic encoders. In Proceedings of the conference. Association for Computational Linguistics. Meeting, volume pp. 397. NIH Public Access, 2017.
Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, and Adam Trischler. Metalearned neural memory. In Advances in Neural Information Processing Systems, pp. 13310-13321, 2019.
10

---

Under review as a conference paper at ICLR 2021
Rasmus Palm, UIrich Paquet, and Ole Winther. Recurrent relational networks. In Advances in Neural Information Processing Systems, pp. 3368-3366390 -378, 2018.
Jack Rae, Jonathan Hunt, Ivo Danihelka, Timothy Harley, Andrew W Senior, Gregory Wayne, Alex Graves, and Timothy Lillicrap. Scaling memory-augmented neural networks with sparse reads and writes. In Advances in Neural Information Processing Systems, pp. 3621-3629, 2016.
Marek Rei. Sepi-supervised multitask learning for sequence labeling. 1704.01700.01500 2017.
arXiv preprint
Adam Santoro, Ryan Faulkner, David Raposo, Jack Rae, Mike Chrzanowski, Theophane Weber, Daan Wierstra, Oriol Vinyals, Razvan Pascanu, and Timothy Lillierap. Relational recurrent neural networks. In Advances in neural information processing systems, pp. 7299-7310, 2018.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929-195) 2014.
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End-to-end memory networks. In Advances in neural information processing systems, pp. 2440-2448, 2015.
Trieu H Trinh, Andrew M Dai, Minh- Minh-Thang Luong, and Quoc V Le. Learning longer-term depen- 2018.
dencies in mns with auxiliary losses. arXiv preprint
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008 2017.
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly, Pointer networks. In Advances neural information processing systems, pp. 2692-2700, 2015.
Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks.
arXiv preprint
arXiv: 110033910 2014.
Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.0566 2015.

---

Under review as a conference paper at ICLR 2021
A EXPERIMENT DETAILS
In experiments, for model optimization, we adopt an RMSprop optimizer with a momentum value 0.9 and epsilon 10-10
A.1 MODEL CONFIGURATION DETAILS
We configure hyper-parameters of our model with two types of settings, one for the comparison with other memory network models and the other for the scalability experiment of distributed memory (DM), For the comparison with other models, we keep the almost same total memory size of DNC and divide it into smaller sub-memory blocks construct distributed memory system for fair comparison. Also, we adjust the memory length size of DAM so that it has a similar amount of for
trainable parameters with other DNC variants. Tables SobllesJande [nd show our model each task, including controller's internal state, dh. the number of read heads, R, the number of memory blocks, K, memory address size, and a memory length size, L.
For the scalability evaluation of DAM, we set the memory length of single sub-memory block, I, same as the memory block size of DAM-2. The size smaller than this causes the information loss because of too small memory matrix size compared to the input length. In this configuration, increase the degree of distribution by increasing the number sub-memory blocks. Since we fix the single memory block size, the total memory size increases linearly with K. Table the details of configuration.
A.2 EXPERIMENTAL TASK EXPLANATION
In this subsection, introduce our experimental setup and how to conduct experiments on each task.
A.2.1 ALGORITHMIC TASK DESCRIPTION
The experiments on algorithmic tasks are repeated 10 times with batch size of and a learning rate of 10-1, and training iterations of 20K on Representation Recall task and training iterations of 10K on Copy and Associative Recall tasks. We evaluate the performance on the algorithmic tasks based the accuracy which is defined as L1 norm between model outputs and targets. In each task, an input flag is provided with story (input) data which are required to produce an answer. After story input, query data along with the output flag are provided to the model. Based on the input and stored information, the model is required to predict the correct answer. For training, we randomly construct each task's data each iteration the following configurations.
Representation Recall Task. In this task, I binary vectors, each has W length, are randomly generated and are provided the model along with an input flag. Here, we divide each input binary vector into 2.V segments and use the half of them, N, as a cue vector. For an answer phase, Le cue vectors are constructed by random sampling from L, binary vectors with replacement and N segments without replacement. The network has to predict the remaining N segments when each cue vector provided. We use L; = 8, W = 64, E {2, 4, 8), and L. E [8, 16). Therefore, d, and d are 64 and 32, respectively.
Memory Length Size
Memory Address Size
Monsrry Btock
Figure 4: Composition of a memory block.
12

---

Under review as a conference paper at ICLR 2021
Table 4: Model paper-parameters for the copy task, associative recall task, and bAbl task.
Copy
Associative Recall
bAbi task
Hyper parameter
DNC
DAM-2
DAM-3
DNC
DAM-2
DAM-3
DAM-2
DAM-3
dp
128
128
128
128
128
128
256
256
R
1
K
1
2
3
2
3
A
64
64
64
32
32
32
128
128
L
36
36
36
36
36
36
48
40
Memory Capacity
2.3K
4.6K
6.9K
1.1K
2.3K
3.4K
12.3K
15.4K
Total Parameters
0.11M
0.13M
0.15M
0.11M
0.13M
0.15M
0.79M
0.80M
Table Model paramerrrrraaeeesss for the representation recall task, N farthest task, and the convex
hull task.
Representation Recall task
Noh Farthest task
Convex Hull
Hyper parameter
DNC
DAM-2
DAM-4
DAM-6
DAM-6
DAM-8
da
128
128
128
1024
256
256
1
2
4
32
32
32
16
20
20
120
60
30
128
64
64
Memory Capacity
3.8K
3.8K
3.8K
12.3K
7.7K
10.2K
Total Parameters
0.23M
0.20M
0.18M
12.6M
1.5M
1.7M
the copy task (Graves et al. 2014). L, binary vectors, each has W length, are randomly generated and are provided to the model along with an input flag. After receiving an output flag, the model is required vectors as an output. We use W = 8 and the number of binary vectors. L; E [8, 32], is randomly associative task (Graves al. 2014). we define an item as a sequence
Copy Task and Associative Recall Task.
sequentially produce the same L, binary
chosen at each iteration. of binary vectors with width W, and during the input phase, L, input items, each consists of N; binary vectors, are provided
model along with an input flag. Subsequently, query item, which is randomly chosen from input item sequence. is provided to the model along with output flag. In this phase, the model is required to predict a subsequent item that has placed right after the query item in the input item sequence. We W 8, N, 3, and the number of items, I, E [2,8], 8), is randomly chosen at each iteration. Therefore, d, and d, are 10 on both tasks.
A.2.2 BABI TASK
Our experiments on bAbl task are repeated 10 times with a batch size of 32 and learning rate of [1. 10] 10 and fine-tuned model with the learning rate 10 and training iterations of 10K. We compose experimental set on the bAbl task with 62,493 training samples and 6,267 testing samples and adopt fixed training iterations of about O. We use word embedding, which is general method in natural language processing domain, with embedding size of 64. Thus, d; is 64 and do is 160. We evaluate the performance on the bAbl task based on the word error rate for answer words in the test samples.
Pre-processing on bAbI Task Dataset The bAbl task dataset (Weston et al. 2015) consists of many different types sets that share similar properties. Among them, we only use en- en-10K set to fairly compare the performance with other researches. In data pre-processing, any input sequences which have more than 800 words are all excluded from the training dataset as in Franke et al. (2018) for computational efficiency. After the exclusion, samples and 6,267 testing samples. We remove all numbers from input data and split every sentence into words, and convert every word its corresponding lower case. After these processes, the
construct our dataset with 62,493 training
13

---

Under review as a conference paper at ICLR 2021
Table 6: Model hyper-parameters for scalability evaluation on the bAbl task.
Hyper parameter
DAM-1
DAM-2
DAM-3
DAM-4
dp
256
256
256
256
R
4
4
K
2
3
4
192
128
128
128
48
48
Memory Capacity
12.3K
12.3K
18.4K
24.6K
Total Parameters
0.80M
0.79M
0.88M
0.97M
whole vocabulary composed 156 unique words and four symbols, which are '[PADJ', ", " and '-', The ''' symbol represents that the model has to predict answer word at the current time step. Therefore, answer words are replaced to symbol from input data. The bAbl task dataset is available http:// www. www.pttteseeeeeeeeee httpa//a......aaeeeet//a/aaaaa/a v1 21-2.tar.gz
1-20
A.2.3 Nth FARTHEST TASK
We use the same task configuration used in Santoro et al. (018)) which uses Adam Optimizer with a batch size 1,600 and an initial learning rate of le consists eight 16-dimensional input vectors, and add 4-layers of MLP with ReLU activation function to the output layer. We set DAM configurations as shown Table 5
A.2.4 CONVEX HULL TASK
We use the same task configuration used in Le al. (2020b), which adopts RMSProp Optimizer with a batch size of 128 and an initial learning rate of le- We add 2-layers of MIP with ReLU activation function (each layer has 256 units) to the output layer and set DAM configurations as shown in Table
B ADDITIONAL ANALYSIS OF ASSOCIATION REINFORCING LOSS
Proposed Association Reinforcing Loss (ARL) chooses a subset of input sequence based on stochastic sampling with trial probability, p. which is called reproducing probability. It adap- tively decides the number of sampled input story, n', according to the story sequence length, n, by adjusting this reproducing probability. Since is binomial sampling with independent trials, the expected sampled story length is as shown in Eq. (1Op. Therefore, it can consistently enhance the memorization performance of memory with the reproducing probability of
E Eln' np
(10)
We used an input sequence dependent loss function, L of each task as shown in Table 7 We adopt CrossEntropy loss function for the Algorithmic, Representation Recall, and bAbl tasks since their input sequences are the binary vectors or one-hot encoding vectors, whereas L2 loss function is adopted Nar farthest task and the convex hull task because their input data are continuous values.
Table 7: The input data dependent loss function, Lar, for each task.
Task
Algorithmic
Representation Recall
bAbl
th Farthest
Couvex hull
Loss
CrossEntropy
L2
14

---

Under review as a conference paper at ICLR 2021
C MODEL IMPLEMENTATION DETAILS
In this section, we provide more detail of the DAM architecture including well-known neural net- work generalization techniques that are used in Franke al. (2018).
At each time step f. the controller, LSTM (Hochreiter & Schmidhuber 1997), receives a input, zy E R": previous hidden state, - R and previous memory read-out values, Pt-1 {r/-1 1
R < RJ. where L is memory length size and R is the number of read heads. Based on these values, the controller updates its internal state, h; Controller((xx:TT- h2-1)). Then, layer normalization (Lei Ba al. 2016) is applied to the updated internal state. From the normalized internal state I.N, the controller generates a memory operators, &, IR [R is called interface vector, as follows:
which
E1 [Er,1,.... 1LK [W....... We,K, Wg,ae/Re,LN
(11)
where É., E BB- is a memory operator for each memory block, K is the number of memory blocks, & E f1.... K}, and 8i R R.R is an attentive gate.
The interface vector, Er, is split into sub-components, and each sub-component and Jfth are used for each memory's 's read/write operations as follows:
fr, = [ki Pi Êe Vi,ki fl ". SBi 02 Di, ki.k." K BL. kk
.B,, ,
(12)
the write-in key ki E R";
the write strength S1.,& = ((3r,) E [1,
the erase values et,k = o (1,,)) E [0,1]',
the write-in values 12 E R.::; R free gates (fi, a (fi,.) [0, 1]: < sis R); the allocation gate Jh, 0 (.txk) E 0011];
the write gate g.*h = (Ht,k) E [0, 1];
the read-out keys { E ; S R3; the read strengths (8 < (( E oo) sis R); and
the attentive gate at.i = Softmar ( for k = 1, K R).
where <()) denotes a Oneplus function, (.)) denotes a Sigmoid function and Softmaz(.) denotes a Softmax function.
Based on those memory operators, the model performs a writing process for each memory block simultaneously. The controller finds writing address, unt [0, 1j4, where is memory address size, in two ways: (i) content-basse based addressing and (ii) memory usage statistic.
The content-base based addressing computes data address as following:
exp(D(k, M 1))33
C(M, k. B}[
(13)
E, Cxp((D(k: M j, 1)333
where D(.,.) is the cosine similarity, M E AAX is the memory matrix, and the 3 controls strength of the address's 's sharpness.
Therefore, It finds write-content addresses, c²,b E 10, 1]4, based on the cosine similarity between write-in keys and memory values, as follows:
c² = C(Mi, ki (
(14)
Next, it finds an allocation address, a,,k E [0. 1 considering current memory usage, such as unused already read memory space. It determines the retention of the most recently read address's
15

---

Under review as a conference paper at ICLR 2021
values for each memory block through memory retention vector, 2/ [0, 1| and it calculates current memory usage vector, ut E [0, 1 follows:
i II w
(15)
Ut,k = (Uc-1. + wi-1 1 w/-1,)) 11
(16)
where denotes a element-wise multiplication.
Afterwards, ar is determined based on current memory usage information as follows:
i-1
a,,, (1 ut (ill
II
ur un [o,,
(17)
f=1
where &e,& is free list which informs indices sorted with respect memory usage values, i.e. [1] is the index of the least used address.
The writing address for each memory block, wi,, determined by interpolation between ci and ac and each memory block is updated, as follows:
10 Si, De,k + (1-1111 g2 cr, w j.k 9i,k
(18)
M M1-1 o (E ---we w(ike(,k) + w
(19)
where L. controls a proportion between at,k and Gi Pi.k controls an intensity of writing, denotes element-wise multiplication and E is AAI,
After all memory blocks are updated, the model executes reading process for each memory block. finds out a read address for each memory block, wit E [o, 1]4, as follows:
f C(M kid ( 35d)
(20)
where i E (1,..., R}.
Then, it reads preliminary read-out value, ri, E R.,, from each memory block and interpolates the preliminary read-out values by of to produce read-out value, ri E RR,, as follows:
rikk
M. w
(21)
K
It.k
rtk
(22)
A=1
where Di controls which memory block should be used for final read-out value.
at,
The read-out values, r' [rl: < R), are provided to the controller. As in Franke et al. 2018
the model applies drop-out (Srivastava et al. 2014) to h2 with a drop-out probability, Pdp. which computed as htsdpp = DropOut (h, Pdp)). Finally, the controller produces an output, y: Rdo, along with fe and r. as follows:
y: Wy Wylh r']
(23)
16

---

Under review as a conference paper at ICLR 2021
D ADDITIONAL RESULTS ON BABI TASK
Reproducing Probabi ity
no
p=U.J
p-0.5
0.3
0.0
10000
20000
30000
40000
50000
Iteration
Figure 5: Mean convergence curves of DAM2-AR for different reproducing probability on the bAbl task. The shadowed area shows standard deviation of 10 trials.
E VISUALIZATION OF DISTRIBUTED MEMORY OPERATION (ATTENTIVE GATE)
Segments
Attentive
Time
Segments
Attontive Gete
05
Tima
Figure 6: The activated attentive gate of DAM-2 according to time step on Representation Recall task. (Top) sub-parts. (Bottom) 16 sub-parts.
8 Segments
Attentive
50
Time
16 Segments
23
Attentive Gate
06
Figure 7: The activated attentive gate of DAM-4 according to time step on Representation Recall task. (Top) 8 sub-parts. (Bottom) 16 sub-parts.
17

---

Under review as a conference paper at ICLR 2021
Fig. 6 and Fig. [ show relative weights of multiple memory blocks (DAM-2 and 4) obtained from attentive gate while performing Representation Recall (RR) task. During RR task, the model has to predict missing sub-pun parts when given other parts clue. The figures show how much each sub memory block is referenced when the model is predicting sub-parts. As shown in the above figures, for given tasks, the model effectively integrates well-distributed information from multiple memory blocks and utilizing all memory blocks for information retrieval.
F EVALUATION RESULT ON GENERAL APPLICABILITY OF DAM-AR APPROACH
Table 8: Test accuracy [%] on N farthest task.
Model
Accuracy
DNC
25
RMC
91
STM
98
RMC-AR (0.3)
94
DARMC3-AR (0.3)
96
DAM6-AR (0.3)
97.8
We applied DAM and ARL to RMC model to verify the general effect other MANN models. As shown in Table 8 RMC-AR which adopted ARL shows superior relational reasoning performance than original RMC. Furthermore, DARMC3-AR which applied both DAM and ARL RMC model shows even better performance than RMC-AR. Although RMC itself includes multi-head attention for relation finding, which is redundancy the DAM, we still can obtain performance enhancement. This result shows the generality of proposed approaches for improving the relational reasoning per- formance of MANN.
G ADDITIONAL EXPERIMENTAL ANALYSIS ON DAM-AR
In the experimental section of our paper, we set up each block size as {A = 64, L, 36) for the copy task and {4 = 32, I, = 36} for the associative recall task. To show the model performance with smaller number of memory slots and memory block length, show the additional experimental analysis.
G.1 SCALABILITY EXPERIMENT ON ASSOCIATIVE RECALL TASK
To compare with the experimental setup used in the paper, A 32, perform scalability periment on the associative recall task with = 16 and each of L values 108, 54, and 36 for each block, and show the performance of DAM according
the number of memory blocks, K in Figure g The result shown in Figure g (a) corresponds to the case when a single memory size of D is linearly increased and the same memory size is divided obtain several numbers of smaller memory blocks of DAM-2 and 3. As shown in the result, the smaller blocks provide more accuracy. The 8 c) shows the case when we adopt a single fixed total memory size for DAM- 2, and 3, and each of DAM-2 and 3 represents the number of sub memory blocks under the same condition. Similar to (a), as we divide the memory size to obtain more sub memory blocks, better accuracy is obtained if there is no information loss at single sub memory block. The Figure B)) shows the training curves for the case of (c). It shows DAM architecture can expedite the training speed of the model even with the smaller number of memory slots.
G.2 EXPERIMENTS WITH SMALLER MEMORY BLOCK ADDRESS SIZE
In our previous Algorithmic experiments, for the copy task, the number input binary vectors are randomly chosen from L4 E [8, 32), and the memory address size is set up A 64. For the
18

---

Under review as a conference paper at ICLR 2021
1.00
1.0
1.00
DAM-K
Accuracy
Accuracy
0.8
Accuracy
0.95
0.96
model
((L-1)
(4M-2(L-54)
DAM-3(L-36)
0.92
0
1503
3000
Iteration
K
(a)
(b)
(c)
Figure 8: Scalability Experiments with Associative Recall task. (a) Scalability test while increasing single memory size. (b) Training curve while iteratively dividing a fixed memory size. (c) Scalability test while iteratively dividing fixed memory size.
associative recall task, the number items is chosen from L, E [2, 8] and each input item has N; 3 length, and A is set to 32. In this experiment, we decreased A from 64 to 16 for the copy task, and from 32 to 16 for the association recall task, so that each memory block has less number of memory slots than the length of the input sequence. Figure T shows DAM performance copy and the association recall task according to the number of sub memory blocks. In the results, similar the previous experiments, the memory network performance both tasks is enhanced as we increase the number of sub memory blocks.
the
Copy
Associative Recall
1.0
1.0
Accuracy
0.8
0.8
0.6
model
0.6
model
DNC
DNC
DAM-3
DAM-3
2000 Iteration
4000
2000
4000
Iteration
Figure 9:
Mean convergence curves the algorithmic tasks with A 16. The shadowed area shows a standard deviation of 5 trials.
Figure 10 shows the experimental result for the effect of ARL on both copy task and associative recall task. In Figure a)) even with the decreased number of memory slots, A = 16, ARL still enhances the DNC and DAM performance on the copy task with increasing reproducing probability. For the associative recall task, the result shows similar pattern the copy task.
2200
Accurasy
Accureyy
groo
pno0
n0
0-0.1
D-0.1
0-0
p-0.3
p-0 3
p-03
p-0 5
2000
Ilration
feeratio
attation
(a) Copy task for ARL
(b) Associative Recall task for ARI
Figure 10: Mean training curves for different reproducing probability values (a) the copy task with A = 16 and (b) the associative recall task with A = 16. The shadowed area shows a standard deviation of 10 trials.
19

---

Under review as a conference paper at ICLR 2021
H DAM MODEL EQUATIONS DIRECTLY RELATED TO DNC
(DNC) & - Weh, = [Wa.i E ERL
(DAM) E. = Weeh, = -E,,1... _ [W....-. Wg.K. Wa 36
DNC generates the memory operators, &., called as interface vector, for its single memory operation, and DAM extends this vector for multiple independent memory blocks. DAM generates K number of DNC like memory operators, É,,,, (except for temporal linkage operator) and newly introduce attentive gate, of to read from those multiple memory blocks.
( w. e, + w."" (DAM) Mt. M1-1, o (E w. Wi.
(DNC) M, M,-10
T
The writing process of DAM is the same as DNC as shown in the above equations, except the same write operation is executed in multiple memory blocks independently the same time.
(DNC) r; M T
K
(DAM) r. Σ
Peh M, w
where Fi,k Softmar(((t)) for 1s = 1,... K.
1=1
In the reading process of DAM, the basic reading procedure for each memory block is the same as DNC, but, DAM integrates every read-out value from K memory blocks into single distributed representation with an attentive gate, The attentive gate, 9 is newly introduced part of DAM for the attentive interpolation.
20

---

Under review as a conference paper at ICLR 2021
run- 10
0.0
0.5
1.2
0.0
0.5
0.0
1.0
0.3
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.4
0.7
0.1
0.0
0.0
0.24
Table 9; Results from all runs of our proposed model with DAM2-AR (P = 0.1].
run-9
0.0
0.2
1.1
0.0
0.2
0.0
0.3
0.4
0.0
0.0
0.0
0.0
0.0
0.0
0.0
51.4
0.1
0.1
1.6
0.0
2.77
run-8
0.0
1.1
5.6
0.1
0.7
0.0
1.1
0.3
0.1
0.0
0.0
0.0
0.2
0.1
0.0
42.6
3.6
0.6
1.5
0.0
2.88
run-7
0.0
0.4
1.5
0.0
0.6
0.1
1.5
0.2
0.0
0.1
0.0
0.0
0.0
0.0
0.0
45.4
0.4
0.3
0.6
0.0
2.56
run-6
0.0
0.5
1.2
0.0
0.5
0.0
0.8
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.3
0.0
0.18
run-5
0.0
0.3
0.8
0.0
0.4
0.0
0.8
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.1
0.2
0.3
0.0
0.16
run-4
0.0
0.0
1.2
0.0
0.8
0.1
1.9
0.2
0.1
0.2
0.0
0.0
0.0
0.2
0.0
49.9
0.5
0.2
2.6
0.0
2.90
run-3
0.0
1.6
5.1
0.0
0.4
0.0
2.0
0.4
0.0
0.0
0.0
0.0
0.0
0.2
0.0
51.7
0.7
0.0
0.7
0.0
3.14
run-2
0.0
0.6
1.1
0.0
0.5
0.1
1.0
0.2
0.0
0.0
0.0
0.0
0.1
0.0
0.0
0.1
0.4
0.2
0.4
0.0
0.24
run-1
0.0
0.1
1.1
0.0
0.5
0.0
1.4
0.1
0.0
0.0
0.0
0.0
0.0
0.1
0.0
0.2
0.0
0.0
0.2
0.0
0.19
3: three supporting facts
I: one supporting fact
2: two supporting facts
4: two arg regumenttnn rels.
5: three argument rels.
6: yes/no questions
9: simple negation
10: indefinite knowl.
11: basic coreference
13: compound coref.
14: time reasoning
15: basic deduction
16: basic induction
17: positional reas.
18: size reasoning
12: conjunction
19: path linding
20: agent motiv.
Mean Er.. (%)
7: counting
8: lists/sets
Task
21

---

Under review as a conference paper at ICLR 2021
run- 10
0.0
0.5
0.8
0.0
0.6
0.0
1.4
0.2
0.0
0.1
0.0
0.0
0.0
0.0
0.0
44.6
0.1
0.3
0.5
0.0
2.46
Table 10: Results from all rums of our proposed model with DAM2-AR (p = 0.3].
run-9
0.0
0.2
0.6
0.0
0.5
0.2
1.6
0.0
0.0
0.0
0.0
0.0
0.0
0.2
0.0
49.3
0.4
0.4
0.6
0.0
2.70
run-8
0.0
0.2
0.5
0.0
0.5
0.0
0.4
0.2
0.0
0.1
0.0
0.0
0.0
0.3
0.0
0.1
0.0
0.0
0.5
0.0
0.14
run-7
0.0
0.6
3.8
0.0
0.3
0.0
0.8
0.8
0.0
0.3
0.0
0.0
0.0
0.0
0.0
47.6
18.8
0.2
1.8
0.0
3.75
run-6
0.0
0.8
0.0
0.5
0.0
0.3
0.9
0.0
0.0
0.0
0.0
0.0
0.1
0.0
46.0
0.1
0.1
1.3
0.0
2.62
run-5
0.0
0.5
2.8
0.0
0.3
0.0
1.2
1.0
0.0
0.5
0.0
0.0
0.0
0.0
0.0
47.2
0.4
1.6
0.0
2.79
run-4
0.0
0.3
1.5
0.0
0.9
0.0
1.7
0.3
0.0
0.0
0.0
0.0
0.0
0.2
0.0
47.2
1.2
0.0
0.3
0.0
2.68
run-3
0.0
0.9
2.7
0.0
0.4
0.0
1.3
0.1
0.0
0.1
0.0
0.0
0.0
0.1
0.0
46.5
1.0
0.1
0.5
0.0
2.69
run-2
0.0
0.4
4.7
0.0
0.2
0.2
0.7
0.2
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.4
0.2
0.0
0.4
0.0
0.37
run-1
0.0
1.1
1.9
0.0
0.8
0.0
1.5
0.7
0.0
0.2
0.0
0.0
0.1
0.0
0.0
48.7
0.1
0.2
0.7
0.0
2,80
3: three supporting facts
I: one supporting fact
2: two supporting facts
4: two arg rngueetttnn rels.
5: three argument rels.
6: yes/no questions
9: simple negation
10: indefinite knowl.
11: basic coreference
13: compound coref.
14: time reasoning
15: basic deduction
16: basic induction
17: positional reas.
18: size reasoning
12: conjunction
19: path linding
20: agent motiv.
Mean Er.. (%)
7: counting
8: lists/sets
Task
22


Reviews:
[{"review_id": "Y6ui8syiwWE", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes two extensions for existing memory networks (e.g., DNC) to improve associative reasoning: (a) multiple memory blocks instead of just one; and (b) self-supervised training as auxiliary tasks. Experiments conducted on toy datasets show that these extensions lead to improved performance over DNC and the method is somewhat competitive against newer variants on several synthetic tasks.\n\nPros:\n- Understanding why MANN works and doesn't, and how to improve it are still open problems. And thus this work is welcomed.\n- The empirical results are positive, suggesting the introduced ideas have merits.\n\nCons:\n-  In terms of novelty, this work is quite incremental. The first extension is simply about dividing the external memory in DNC into multiple blocks that connect via an attentive gate. The argument of having multiple memory blocks is to diversify the representation of the same input. A similar idea has been introduced in [3], although the motivation was different. The second extension is a reconstruction loss on the input signals. To what degree it works as  “association reinforcing” as claimed is unclear, even though the loss would work as a regulariser as in standard hybrid loss in existing neural networks.\n-  In terms of experiments, the tasks are too simple and particularly favor the model design in the paper. For example, in the copy task, we need faithful information from the input signals for correctly decoding (copying), which can be enhanced via the reconstruction loss during the encoding phase. The same thing applies to the associative recall task.\n- Note that for both copy and associative recall, the length of the input sequence ([8, 32]) is much smaller than the number of memory slots (64) (see Appdx A.2.1). With such redundancy in storage, it is not very surprising to me that dividing the memory into smaller blocks can improve convergence.\n- The main baseline used for comparison in this paper is DNC, which, in my opinion, is out-of-date as it can be outperformed easily (e.g., existing works on memory networks [1, 2] show significant improvements over DNC on these toy tasks). \n\n\nSome minor comments:\n- L_ar(i_t, y_t) is not defined in the paper. Do the authors use L1, L2 or binary cross-entropy loss?\n\n[1] Relational recurrent neural networks, Santoro et. al., NIPS-2018\n[2] Improving Differentiable Neural Computers through Memory Masking, De-allocation, and Link Distribution Sharpness Control, Csordas et. al., ICLR-2019\n[3] Pham, Trang, Truyen Tran, and Svetha Venkatesh. \"Relational dynamic memory networks.\" arXiv preprint arXiv:1808.04247 (2018).\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "RWrfnJfg9eM", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work the authors propose a novel memory architecture wherein memories are stored in multiple ways across a series of memory blocks. By \"distributing\" the memories in such a manner, the model can flexibly retrieve one version of a memory or another, which enables more flexible computations when conditioning on that memory. The authors demonstrate that such a memory network does well in tasks involving relational reasoning. \n\nOne advantage of memory-based models is that they amortize compute across time: they pay an upfront cost to shape and store a memory *online*, but gain an advantages at read time, where they just need to do a basic lookup among the stored memories. A consequence of these  memory-based approaches is that the models need to anticipate how they should store a memory given what might come in the future. In this work, the authors propose the strategy of storing a memory in multiple different ways, which mitigates the risk that a single stored memory will be insufficient for what might come. \n\nIn contrast, something like a Transformer pays a heavy cost at read time, as it needs to perform a full self-attention operation (rather than simple lookup) across all stored memories. Transformer-based approaches, however, profit immensely from tasks where memories need to be shaped differently at read time, as they can use the full power of self-attention to morph and condition memories given all the information they've accumulate to that point in time. Given the recent surge of evidence of the usefulness of self-attention based models, and the fact that they can easily be interpreted and/or used as memory models, the authors would be remiss to not include a self-attention based baseline to which they can compare their model. This is especially important given that the complex nature of memory models (i.e., the complexity associated with learning how to read, write, etc.) has recently given way to a more simple approach using memory buffers and self-attention.\n\nThis is not to say that there is no value in developing memory-based approaches, as there surely is. However, the memory-based approaches should demonstrate their value in domains that play to their strengths. As stated above, these models amortize the cost of shaping memories over time, and in addition, they can keep a constant size memory indefinitely in time. In contrast, a Transformer-based memory model would grow in memory cost as time increases, and the compute at read time would similarly grow quadratically with time. Thus, to demonstrate the value of a memory-based approach over self-attention, it is wise to pit the two against one another in a regime where self-attention simply becomes too costly; in other words, in a regime where a great number of time steps (and hence memories) need to be considered. Otherwise, the reader is left wondering how well this model compares to the simpler Transformer-based approach.\n\nThe authors also propose a new loss that forces the memory contents to be able to predict a sample sequence of previously observed inputs. Unfortunately, I believe the inclusion of this loss makes the absense of a Transformer baseline even more troublesome. This is because, for this loss to be implemented, we need to keep around a buffer of previous inputs, which is precisely the memory cost associated with using a Transformer! So, given the previous discussion on how memory models can in theory maintain a constant sized memory, in the DAM this is no longer the case. Memory costs grow linearly with time because of the need to preserve inputs for use in the ARL loss. \n\nAltogether, the paper is well put together and written well enough to understand the ideas and experiments. The authors did well to choose experiments that would demonstrate the strengths of their approach. Unfortunately, the empirical and rational comparison to Transformer-based approaches prevents me from recommending its publication. \n\nThere are a few minor points scattered throughout, but I'll just call attention to the following:\n\n\"insufficient associating performance\"\n--> It is unclear what this means.\n\n\"lossy representation\"\n--> This term is used throughout, but I'm not sure it's warranted. How do we know that the distributed vector representation is truly lossy with respect to the information it must encode? In princple nothing prevents it from being lossless. Given a complicated enough decoder, one wouldn just need a handful of bits to encode very complicated things losslessly.  \n\nHowever, even with its promising performance on a wide range of tasks, MANN still has difficulties in solving complex relational reasoning problems (Weston et al., 2015).\n--> There has been much work since 2015 that has improved MANN performance on these tasks. For example, the Sparse DNC, as eventually shown in the results section. \n\nThrough this attention-based reading process, DAM retrieves the most suitable information for the current task from distributed representations existing in the multiple memory blocks\n--> As shown in this text, and as used throughout, the term \"distributed representation\" is overloaded in this work. Traditionally the term \"distributed representation\" is used to denote a vector with real-valued elements, whereas here it is used to denote a set of such vectors, \"distributed\" across multiple memories. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "OX9N9PHnEb1", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an extension of the Differenciable Neural Computer networks (DNC)\nIn these DNCs, the reading operation on the external memory are done by accessing a single memory block, which represents a single piece of information or knowledge. The architecture proposed in this paper aims instead to give the possibility to access multiple memory blocks at the same time. In this way, the approach of reading memory is more holistic (Chalmers, 1992). This is a desirable feature of distributed representations and, then, distributed memories. Otherwise, these memories are just similar to the classical approach of representing symbols (Fodor and Pylyshyn ,1988). This debate of what is the main charateristic of distributed representations is revitalized in Ferrone and Zanzotto (2020). It is then a needed extension of DNC.\nThe paper is well written and results are convincing.\nHowever, there is a minor problem. There is not a direct link among equations in Section 2 and equations in Section 3. Clearly, DNC equations are extended by equations in Section 3. Are these equations linked only with the M, that is the Memory? \n\nReferences\n\nFodor, J. A., and Pylyshyn, Z. W. (1988). Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71.\nChalmers, D. J. (1992). Syntactic Transformations on Distributed Representations. Dordrecht: Springer.\nFerrone, Zanzotto (2020), Symbolic, Distributed, and Distributional Representations for Natural Language Processing in the Era of Deep Learning: A Survey", "extra_scores": {}, "extra_text": {}}, {"review_id": "GLesDNQuH_k", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper introduces a modification to Differentiable Neural Computer called Distributed Associative Memory (DAM) that comprises of 1) multiple independent memory blocks and 2) association reinforcement loss (ARS). Experimentally DAM improves upon DNC on multiple tasks and is showing comparable performance to some relation-aware architectures. \n\n**Paper strengths**\n* It is interesting to see that a relatively simple modification can bring a prominent performance boost. While it probably still requires further investigation, the fact that an architecture with factorized memory blocks can match performance of an explicitly relational architecture suggests that  this is indeed a step in the right direction and/or the relational benchmarks currently used in the community are too simple.\n\n**Paper weaknesses**\n* Clarity. ARL which is one of the two novel components of DAM is not described clearly. $l_{ar}$ is not formally defined anywhere and its textual description is rather vague. What exactly does \"sampled input sequence\" mean? Should it be called \"subsampled\" instead? Is it always valid to simply subsample individual input tokens?\n* Since the improved performance presumably comes from the multiple individual memory blocks, it is important to understand how exactly information is factorized across them and how each of the blocks is used. One can argue that a wide enough representation can potentially learn the factorization scheme and mimic it using multiple reading/writing heads. To me the basic intuition behind that is provided in the Introduction is not enough.\n* I appreciate that authors do compare memory capacity across different DNC variants, but then it is important to do so for all the baselines and ideally evaluate all the baselines with the same number of floating point numbers reserved for memory. Otherwise, the exact source of the improvements is not clear.\n* Authors may want discuss the following paper [1] which describes a highly relevant model.\n\nI am happy to revise my score if the points above are addressed by authors.\n\n** References **\n[1] Marblestone, Adam, Yan Wu, and Greg Wayne. \"Product Kanerva Machines: Factorized Bayesian Memory.\" arXiv preprint arXiv:2002.02385 (2020).", "extra_scores": {}, "extra_text": {}}, {"review_id": "19xJPGRdq47", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a distributed memory architecture which shares some interface with the Differentiable Neural Computer however crucially segments memory into a collection of K units. The authors show that by increasing K the model learns to use its memory for algorithmic tasks such as copying and associative recall and learn faster. The authors also propose an auxiliary loss to improve memory representations, which involves reconstructing inputs from the representations in memory. \n\nI think the scientific statement is quite clear here and the paper is worth accepting; the only shame is that the authors did not apply this approach to a richer task than bAbI. \n\nAlso it would have been nice to compare the approach to a multi-head attention transformer since these also use distributed representations (across heads). \n\nThe authors may be interested in the following architecture MERLIN which also uses a reconstruction loss to improve memory representations: https://arxiv.org/abs/1803.10760", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: After carefully reading the reviews and the rebuttal, and after going over the paper itself, I'm not sure the paper it ready for ICLR. I do believe there is a lot of useful content in the current manuscript, and I urge the authors to keep working on the manuscript and resubmit it in due time. 

My concerns are as follows:
 (a) there is a lot of discussion about *relational information retrieval* -- however there is lack of any formalization of what this term means. I don't mind relational reasoning to be used as motivation, but when it is used to consider what are valid baselines and what are not, I feel compelled to understand what exactly it means. Why is *self-attention* retrieval not *relational*? Beside the task being seemingly relational in spirit, how do we test whether the retrieved mechanism carries any relational information whatsoever? I think the community had a learning lesson here in CLEVER dataset, which arguably does not require as much relational reasoning as it seemed. So I agree with Rev5, that there is a decent probability that the task we are using do not require relational information retrieval. While I understand that some of these systems are Transformer inspired, I feel transformer should be a baseline. 
 (b) I also feel the paper should take one of two paths. 
       - Either embrace larger scale tasks and baseline outside of the relational reasoning literature (like transformer) and particularly settings where potentially self attention will struggle due to the quadratic term or where they tend to be hard to train due to the difficulty of doing credit assignment through the attention mechanism 
       - Provide more careful ablation studies and formalize the claims a bit more. Regarding e.g. the discussion of a single larger memory vs multiple memory blocks. One of the main difference comes from the attention over which memory block to use in the proposed approach, which due to softmax has a unimodal behavior. So is the reason why it works better this potential hiding of part of the memory representation (so a better way of reading a subset of the memory entry). This could potentially be done differently (e.g. multiplicative interaction in the same style, for e.g. that they were used in WaveNet). This is just a random thought on this particular aspect. I have similar questions about the self-supervised loss. 

 I find the paper focusing on improving performance (unfortunately on toy domains) rather than ablation studies and an understanding and careful understanding of how things works. I realize there is some such analysis in the appendix. But I feel more of it should be in the main text. The paper is either proposing something that scales and works well at scale (and then understanding why is less important as it has direct application) or explores a very specific phenomena and then is fine to stay on toy tasks but there should be a bit of clarity in the claims, and an investigation whether the hypothesis (or intuition) put forward initially is the reason why the model works. 

Decision: Reject