Plug, Play, and Generalize: Length Extrapolation with Pointer-Augmented Neural Memory
Authors: Hung Le
Dung Nguyen
Kien Do
Svetha Venkatesh
Truyen Tran
Venue: Accepted by TMLR ()

Abstract: We introduce Pointer-Augmented Neural Memory (PANM), a versatile module designed to enhance neural networks' ability to process symbols and extend their capabilities to longer data sequences. PANM integrates an external neural memory utilizing novel physical addresses and pointer manipulation techniques, emulating human and computer-like symbol processing abilities. PANM facilitates operations like pointer assignment, dereferencing, and arithmetic by explicitly employing physical pointers for memory access. This module can be trained end-to-end on sequence data, empowering various sequential models, from simple recurrent networks to large language models (LLMs). Our experiments showcase PANM's exceptional length extrapolation capabilities and its enhancement of recurrent neural networks in symbol processing tasks, including algorithmic reasoning and Dyck language recognition. PANM enables Transformers to achieve up to 100% generalization accuracy in compositional learning tasks and significantly improves performance in mathematical reasoning, question answering, and machine translation. Notably, the generalization effectiveness scales with stronger backbone models, as evidenced by substantial performance gains when we test LLMs finetuned with PANM for tasks up to 10-100 times longer than the training data.

Published in Transactions on Machine Learning Research (10/2024)
Plug, Play, and Generalize: Length Extrapolation with Pointer-Augmenter Neural Memory
Hung Le
that. le @daakin edu.on
Applied AT Institute
Deakin University, Austrolia
Dung Nguyen
Applied Al Institute
Deakin University, Austrolia
Kien Do
k. &dekkin edu.
Applied Al Institute Deakin University, Austrolia
Svetha Venkatesh
svetha. wwwkatesh Edeakin eda..ox
Applied AI Institute
Deakin Universily, Austrulia
Truyen Tran
truyen. tran Edeakin edu.
Applied Institate
Deakin University, Austrolia
Reviewed OpenReview: https: optps:/e/penrevieeu. net/ forum? id= dyQ9uFbbFD
Abstract
We introduce Pointer-Angmented Neural Memory (PANM), versatile module designed to enhance neural networks' ability to process symbols and extend their capabilities to longer data sequences. PANM integrates an external noural memory utilizing novel physical ad- dresses and pointer manipulation lechniques, emulating human and computer-like symbol processing abilities. PANM facilitates operations like pointer assignment, dereferencing, and arithmetic by explicitly employing physical pointers for memory access. This module be trained end-to-end sequence data, empowering various sequential models, from simple current networks to large language models (L..ss). Our experiments showcase PANM's ex- ceptional length extrapolation capabilities and its enhancement of recurrent neural networks in symbol procossing tasks, including algorithmic reasoning and Dyck language recognition. PANM enables Transformers lo achieve up lo 100% generalization accuracy in compositional learning tasks and significantly improves performance mathematical reasoning, question answering, and machine translation. Notably, the generalization effectiveness scales with stronger backbone models, evidenced by substantial performance gains when we test LLLSs finetuned with PANM for tasks up to 10-100 times longer than the training data.
1 Introduction
Systematic generalization underpins intelligence, and it relics on the ability to recognize abstract rules, extrapolating them to novel contexis that are distinct yet semantically similar to the seen data. Current neural networks or statistical machine learning fall short of handling novel data generated by symbolic rules even though they have achieved stato-of-tho-art results various domains. Some approaches can show decent generalization for single or set input data (Bahdanau 2020). Yet, neural networks in general still fail in sequential symbol processing tasks, even with slight
2018 Gao et al. [020)] Webb et al.,

---

Published in Transactions on Machine Learning Research (10/2024)
(c) Pointer-based Memory Access
(d) Decoding
Mode-1 Apoess
(b) Address Bank
080
0010
0011
0100
Pointer Unit
Controller
Mode-2
PPi
Access
(a) Data Memory
Figure 1: PANM architecture. (a) The data memory contains the encoded input sequence (b) The address bank contains physical addresses associated with data memory slots. The base and end addresses (aB, ag) define the address range of the imput sequence. (c) The Pointer Unit takes aB, dg, recurrently generates the current pointer 3f and gets its value "pf via Mode-1 (red)/2 (green) Access. (d) The Controller takes pointer information, decoding input (zt = yt-), and produce the 1-th output token de- We note that for illustration purposes, the figure depicts special case where the pointer perfectly matches address (c.g., Pf 0010),
in practice, the pointer may not point exactly to a single address.
novelty during inference [ake & Baroni 2018 Delétang et al. 2022). For instance, these models can easily learn duplicate sequences of 10 items, but they will fail to copy sequences of 20 items if they were not part of the training data. These models overfit the training data and perform poorly on aut-of-distribution samples such sequences of greater length or sequences with novel compositions. The issue also affects big models like Large Language Models, making them struggle with symbolic manipulation tasks (Qian et al. 2023 This indicates that currett methods lack principled mechanism for systematic generalization.
From a neuroscience perspective, it has been suggested that the brain can execute symbol processing through variable binding and neural pointers, wherein the sensory data are conceptualized into symbols that can be assigned arbitrary values (Kricte et al.] 2013). Like the brain, computer programs excel at symbolic computations. Programmers use address pointers dynamically access data or programs, and have flexible control over the variable. Their programs can work appropriately with unseen inputs.
Building on these insights, we propose pointer-based mechanism to enhance generalization to unseen length in sequence prediction, which is a crucial problem that unifies all computable problems (Solomonoff 2010).
Our mechanism based on two principles: (I) explicitly modeling pointers as physical addresses, and (II) strictly isolating pointer manipulation from input data. As such, we need to design a memory that supports physical pointers, and create a model that manipulates the pointers to perform abstract rules and access to the memory. Our memory, dubbed Pointer-Augmente (Von Neumann 1993 where each memory slot consists of two components: data and address. Unlike initial endeavors that implicitly model pointers as altention softmax (Vinyals el. al. 2015 Kurach et al. 2015 our addresses are generated to explicitly simulate physical memory addresses, i.e., incremental binary numbers, which is critical for generalization to longer sequences.
Neural Memory (PANM), is slot-based RAA
To manipulate a pointer, we create an address bank that contains physical addresses corresponding to the input sequence, and use a neural network called Pointer Unit that responsible for transforming pointers from an initial address in the address bank. Through attention to the address bank, a new pointer is generated as mixture of the physical addresses, which can point to different memory slots to follow the logic of the task. We aim to let the Pointer Unit learn the symbolic rules of the task in an end-to-end manner. Finally, given a (manipulated) pointer, the model can access the data through 2 modes of pointer- based access: pointer dereference (Mode-1) and relational access (Mode-2). Our memory can be plugged into common encoder-decoder backbones such as LSTM or Transformer.
Our contribution is novel memory architecture, PANM, which incorporates explicit pointer and symbol processing, seamlessly enhancing sequential models for better generalization. In our experiments, inte-
2

---

We are interested in the ability to handle inputs of arbitrary length, so we focus on settings in which the length of testing input sequences is larger than that of training ones: max ((X;) < min ((X,) with X; E Ptrain and Xj E Drest- In the following sections, when there is no confusion, we will drop the sample index or j for Published in Transactions on Machine Learning Research (10/2024)
grate PANM into range of deep learning backbone models of varying scales to assess the improvement in generalization:
First, we add PANM to recurrent neural networks (LSTM) and memory-angmennted neural net- works (SRNN), demonstrating excellent generalization in symbol-processing tasks like algorithms and context-free grammar.
We also apply PANM to Transformer models, improving their performance on compositional learning with SCAN and mathematics datasets, Additionally, PANM significantly enhances Transformer and BERT generalization in question answcring and machine translation tasks.
Finally, we demonstrate PANM's scalability by integrating with LLMs (Llama2-7B), showing significant improvement in length extrapolation for NLP tasks requiring symbolie reasoning.
Our focus is not on striving for state-of-the-arl results requiring specialized designs tailored to specific tasks. Instead, we aim to highlight that universal generalization improvement con be achieved by integrating our memory module into various sequential models, with minimal architectural changes, and showcase the importance of using fundamental generalizing principles to address limitations of current deep learning.
2 Methods
2.1 Problem Formulation
In sequence-to-sequeeee (s2s) problems, each data point is an input sequence X; = {=() -=1 associated ((Y:)
(X.)
with a target sequence Y, {vi Jr=1 where I is a function returning the length of the sequence. A model $
((%) ( where the predicted sequence terminates as the model outputs token (()) EOS where EOS is special end-of-sequence token. Each predicted token sampled from a categorical distribution, parameterized by t and conditioned on the input sequence and optionally with previous output tokens: o ~ pa (o, Xi, vi-) where vi_ can be { t-1 (true outputs) (-1 (predicted outputs) or even zero, depending on the setting (training or inference). We train $ or {0k J==1 by minimizing the cross-entropy loss:
takes the input X, and generates an output sequence Ý, {
- -
Σ Loogoo(si X,, vi-)
C= E,
case of reading. We note that autoregression is special case of the s2s formulation where the input and output are from the same domain, and the target sequence is one step ahead of the input sequence.
2.2 Pointer Modeling
Computers are powerful thanks to their ability manipulate pointers. These pointers store the address of data in memory. Following C programming language notation, let p denote a pointer associated with a data d in memory M, then p is also the address of the memory slot containing d, i.., p &d where & operator returns the pointer associated with value d. We can access the data pointed by as "p d where operator returns the memory value associated with pointer p, which is also known as pointer dereference.
We can manipulate the pointer to execute various tasks. For example, given a list X storing elements in consocutive memory slots, &X denotes the pointer of the first element of the list. If the task is to copy the list a new list Y, using pointers, this task can be executed by iterating over the elements of the list. and copying each element from X to Y regardless of the list length and the values in X. The copying
3

---

Published in Transactions on Machine Learning Research (10/2024)
process can be described as follows: (1, assignment) Initialize 2 pointers pointing at the first element of X and Y, respectively: px = PX: = &Y; (2, dereference) Access the value pointed to by px (i.e., the current element of X) and copy it to the location pointed to by pY (i.e., the corresponding position in Y): "py = -xx; (3, arithmetic) Increment both pointers to move to the next elements in their respective lists, px - px + P;pY = py + 1. Repeat this procedure until all elements in X have been copied to Y.
In this paper, we propose a way to model pointers by constructing a bank of addresses, analogous to the addresses in computer architecture. The address bank starts with base address an and increases to form an arithmetic sequence with the common difference of 1. For example, if the bank has 3 addresses and aB 3, the addressos are A {3, 53. We represent the address as b-bit binary vectors, so we have A {0010, 0011, 0100} when b = 4. The address space is 2 b-bit. binary vectors. Given a memory M containing ((M) slots, we bind each slot to an address in the address bank A such that Aft] = &M[[f] and AA(t] = Mid] (1 s t & (()).. We use ag as the end address to refer to the final address corresponding to the last item M.
The memory M stores the input sequence, and its size depends the imput length: ((M) ((X). To enable generalization, the address space should cover wide range of addresses that is greater than the sequence length range (i.e., 2 > max (()).. More importantly, during training, all possible addresses should be exposed to the model. Otherwise, any unexposed address will confuse the model when it appears during inference. As such, during training, we uniformly sample the base address ag from the address space to construct the address bank for each sequence memory M. This ensures any address in the address space will eventually appear between the base and end addresses. See Appendix for an illustrative example of the base address sampling mechanism and its complexity.
Given the address bank, can perform pointer-based procedures to achieve generalization. To do so, we need pointer variables Pt denoting pointers used by the model at timestep t. As for the copy task, the model outputs correctly by accessing and manipulating the pointer variables through 3 important pointer operations: ps = A[t] (assignment); z = pp (dereference); Pt Pt + (arithmetic), which will be described in the next section.
2.3 Pointer- Bantter-Augmented Neural Memory (PANM)
PANM acts as an external memory module for any neural network to support it handling sequence data. In such a memory-augmentee neural network, a neural Controller (Ctr1) interacts with the memory (M) to read/write data and make predictions on the output target. Unlike traditional neural memory, PANM is equipped with an address bank (A) and a Pointer Unit (PU) to support pointer operations. To simplify memory writing operations, PANM transforms the whole input sequence X to the memory M in the encoding phase such that L = ((M) ((X) using M Encoder, (X) where X E Rd. M E Rd xL and the Encoder, parameterized by 0, can be any neural encoder such LSTM or Transformer. The address bank A e {0, 13 is then created and bound to M as mentioned in the previous section. During decoding, the encoded information in M is not changed and the controller focuses only on reading from M to produce the right output sequence. An overview of PANM decoding process given in Fig. I
2.3.1 Pointer Unit
At each timestep t of the decoding process, PANM makes use of pointer variables pf, which are initialized as valid address in the address space and then updated by the Pointer Unit PU. In tasks like summarizing a lengthy article by extracting key points from specific paragraphs, the Pointer Unit functions like sophis- ticated "bookmark' system, starting at the targeted paragraph and sequentially progressing through each key point, effectively managing documents of any length by using relative positions. In particular, the PU, implemented as an GRU (Chung et al., (2014). takes an address from as its initial inputs, e.g., Pd = ag. and recurrently produces key hy that performs address atlention to create succeeding pointer variables pf:
4

---

Published in Transactions on Machine Learning Research (10/2024)
M GRU,p (pG-1:1811) 1121)
(1)
hig (A/zz) || 1193 (A(m))]
ii [n] softmax
(2)
pt Awu
(3)
where hō is initialized as 0' in Eq. (1). In Eq. (2), 1 n (X), y denotes the parameters of the PU and oet (-) is a leed-forward neural network to transform the address to the same space as hg. According to 5 [ principle I, pf is "softly" assigned a physical address value in the address bank. Our pointer, pf, offers several advantages over "implicit pointers" made up of the attention weights (wf),, which are commonly utilized in previous works (Vinyals et al.] 2015 Luong al. 2015). First, P is a combination of physical addresses represented by binary numbers, and therefore its dimension is generally independent. of the sequence length. In contrast, the dimension of wf varies with the input length. Therefore, arithmetic transformations on pi are easier than on wy. Second, longer testing length poses challenges for traditional attentions to accurately produce we pointing to unseen location. Using "physical key" A to compute we mitigates this issue by employing random physical address ranges (see $ 2.2
Following 9 s principle II, the PU recurrently transforms the original p to a scries of pointers {pf (rg] (P) suitable for the current task without using inpul dala. This prevents unseen testing inputs disturb PU's transformations. In the copy example, an ideal arithmetic transformation ensure P ag and pf+1 pf +1, which performs perfectly for any sequence whose length < 2 We aim to learn PU to automatically discover pointer manipulation rules from the task. As the rules are learned, generalization is achieved even when the testing sequence is longer or contains novel items.
2.3.2 Pointer- Addressing Modes
Mode 1 In this mode, PANM's pointer functions like an index, much like accessing a particular row in a data table or a sentence in a passage, so the PU points to a sentence, it directly retrieves that sentence, regardless of the passage content. Particularly, the content from memory M is retrieved directly by dereferencing pointers. To dereference pointer pt, we utilize the
binding and the address attention weight wy, retrieving the pointer value associated with pt as "rf = Mwf. Through this dereference operator, we can access to arbitrary data in the memory M without relying the content of the memory. This property enables robustness in handling new sequence when the memory content is novel and the process stays the same (e.g., copy a never-scen-before sequence). Accessing M indirectly via A allows more memory slots to be added to M during inference without alfecting the processing rule as long as the PU can transform the pointer variable correctly. During training, PU experiences pointer variables covering the whole address space because the base address sampled randomly. Hence, it is possible for to correctly transform pointers that point to extended addressos of a growing memory as long as the pointer manipulation rule does not change. The address attention can be used for multiple pointer variables. In this case, there would be H²
and { (r h=1 multiple pointer units [ responsible for several { Pf, f h=1
H"
where Ho is the number of attention heads. These pointer values will be used by the Controller for other memory reading.
Mode 2 This mode uses more complicated memory access mechanism to capture relations between pointers in complicated reasoning tasks. The accessed content is not the one associated with the current pointer, but those whose contents are related to the current pointer's value. As an example, selection sort algorithm may require comparing items in a list with the Mode-1 pointer's item to select the greater one. Another example is to summarize a document by identifying the most. important sentences based on their relevance to a query. In this mode, PANM's pointer goes beyond direct retrieval by using the content of a sentence to generate a query that ranks sentences based on relevance, much like running a scarch query to identify and select the most important sentences in a document based on their contextual significance. We simulate that using attention with the query as the current. pointer value:
5

---

Published in Transactions on Machine Learning Research (10/2024)
(c) Copy
LSTX
Eybrid Attention
DNC
Location Attertion
Transfocmer
Neural Stack
Contezt Attention
NTM
PirN ot
NEAN
ESON
PANN (Ours]
(a) Copy
(b) ID Sort
2
100
(d) ID Sort
Decoding Steps
Accuracy (%)
Input Maps To Output
M1 Mode-1 Access
M2 Mode-2 Access
Token (with
40
4. L+1 20L+1] 4(L- B(L-1) Length Test
L+1 201-1 1) 4(L-1) 8(L-1)
Figure 2: Exemplar results on 2 algorithms. (a, b) Test accuracy (mean + sid) over 5 runs on Copy and ID Sort on each length test, respectively. Random predictor would reach around 10% accuracy. (c,d) Visualization of data and pointer's slots for Copy and ID Sort, respectively.
q: g6 ((+ ((pin))mi)
)
(4)
w [n] softmnx
(5)
|| [M[x]]
Pp. Mun
(6)
as input, transforms them to query qi using ferd-forward neural network of (-), and returns the related pointer value "pf through attention mechanisms on M. Intuitively, the Pointer Unit manipulates the Mode-1 pointer p such that it retrieves the desired content pointed by the Mode-2 pointer pf. We can also have multi-head attention, which results in Hº
Here, the pointer attention takes the concatenated values ('EE. A-1
{ ('Pf.) 1=1
where He the nunber of attention heads.
2.3.3 The Controller
The Controller Ctrl is responsible for decoding the memory to produce outputs. Unlike other methods, we have pointer-based memory access to provide the controller with symbol-processing information. particular, at the f-th step of the decoding, Ctrl takes the pointer values (mode and 2) as input together with an optional decoding imput (2t =)-), and uses a GRU to recurrently produce the hidden state hf:
b = GRUA ([ Ha {" Jm 21) Mi-1))
where the hidden state hy is initialized Σ, M[i] and is the parameters of Ctrl. The GRU handles content-based input, empowered with pointer information to construct rich hidden state representations. Furthermore, the pointer-based data gives the GRU access to correct items in the soquence even when the memory content becomes different from the training due to encountering novel sequence length.
The Controller Ctrl uses the pointer values (mode 1), the related pointer values (mode 2) and the hidden state if to make the final prediction. simply concatenates them and forward to the go (-)-a MLP, to generale the output token
~ Po X,, 21) = ( ("
H mg
The pointer values allow 9e to fully utilize pointer information in producing the final output. & consists of the parameters of the Encodero, Pointer Unit PU, and Controller Ctrl,- Here, Ctrl can be put top of

---

Published in Transactions on Machine Learning Research (10/2024)
Section
Task
Backbone
3.1
Algorithmic Reasoning
RNN (LSTM)
3.2
Dyck Language Recognition
MANN (SRNN)
3.3
Compositional Learning
Transformer (6 layers, RPE)
3.4
Other NLP Tasks (QA and Machine Translation) Algorithmic Reasoning and BIG-bench Tasks
Transformer (6 layers, BERT)
3.5
LLM (Llama2-7B)
Table 1: Task and backbone summary.
Task
Key Metric
Best Baseline
PANM (Ours)
Algorithmic Reasoning
Token Match (%)
56.3
(Other Max)
68.2
Dyck Language Recognition
Sequence Match (%)
28.2
(SRNN)
70.0
Compositional Learning
Sequence Match (%)
82.6
(U. TRM+RPE)
88.6
bAbi QA
Accuracy (%)
77.5
(TRM)
83.0
SQUAD QA Machine Translation
F1 (%)
75.0
(BERT)
77.0
PPL
42.7
(TRM)
32.7
LLM Algorithmic Tasks
BLEU (%)
14.7
(Llama)
62.7
BIG-bench Tasks
Sequence Match (%)
11.5
(Llama)
22.5
Table 2: Average performance summary with key metrics.
another decoder to process the decoding input 2; = Decodero (v--). For example, we can use Transformer as the Decoder (see Appendix and D.3). The Encoder, and/or the Decodery will be commonly referred to as the backbone model in which PANM is built upon. summary of PANM's operation given in Algo. and Fig. in the Appendix.
3 Experimental Results
In our experiments, use two pointer variables in Mode-1 access and one for Mode-2 balance between performance and computing cost. (Ha 2, H 1, see more in Appendix C The two Mode-1 pointer variables are initialized as the base and end addresscs. All MLPs in PANM have 1 hidden layer of 128 dimensions. We use 256-dinensional GRUs for PU and Ctrl. The memory's 's address space has b = 10 bits, corresponding to maximum of 1024 unique addresses, which is greater than any sequence length in the experiments.
In 33.1 3.3, our chosen lasks are representative of various lypes symbolic reasoning and well-known bench- marks measure the symbol-processing capability of machine learning models. To showcase that these tasks report how further validate this point, we finetune Llama2-7B {Touvron et al. 2023) on similar tasks and show that the LLM also fails to generalize even with finetuning on task data (see $3.5). In addition, we validate the contribution of PANM in other practical tasks in $3.4 and (3.5) We summarize the tasks and backbones used in our experiment in Table I We also summarize the final average performance PNAM and the best baselines Table
non-trivial,
[Achiam et al.) 2023) failed our tasks in Appendir D.7 To
Baseline Choice Despite some being simple, our baselines are still very strong methods in our studied tasks. For example, in our algorithmic reasoning, LSTM with attention or Pointer Networks are still dom- inant baselines, outperforming the more recent Transformers. There are also other sophisticated methods focusing generalization such as ESBN (Webb et al. (020). In Dyck recognition, stack-based models are still SOTA because their inductive bias is suitable for the task. Experiments in $3.3) adopt (Universal) Positional Encoding (RPE), which a very strong Transformer variant focusing on gencralization. For experiments with LLMs, Llama2-7B stands out as the most capable open-source LLM that is suitable for our hardware. In our experiments, PANM-angmented models are ensured to have similar model size as the baselines and always share similar backbones for fair comparison.

---

Published in Transactions on Machine Learning Research (10/2024)
Task
Copy
Reverse
Mix
D. Recall
P. Sort
ID Sort
Other Max
60.2
63.6
64.0
47.6
60.6
42.1
PANM (Ours)
74.8
73.6
81.2
52.8
67.8
59.2
Table 3: Algorithmic reasoning: mcan soquence-level accuracy (%) over testing lengths Other Max is selected as the best numbers at each length mode from other baselines.
SCAN (L cul-oll)
Math
Task
22
24
25
26
27
28
30
32
33
36
40
a.s
p.V
C. TRM+RPE
20
12
71
100
100
100
100
100
100
100
100
97
75
TRM+RPE
20
12
31
61
100
100
100
94
100
100
100
91
0
U. TRM
5
14
21
26
0
35
0
94
20
TRM
19
29
30
8
24
36
0
89
12
PANM (Ours)
22
47
100
100
100
100
100
100
100
100
100
97
86
Table 4: SCAN (Left): Exact match accuracy (%, median of runs) on splits of various lengths. Mathematics (Right): mean accuracy over 5 runs. The basclines' numbers are from Csordás al. (2021) and we run PANM using the authors' codebase.
3.1 Algorithmic Reasoning
In our first experiment, we study the class of symbol processing problems where an output sequence is generated by a predefined algorithm applied to any input sequence (e.g., copy and sort). The tokens in the sequences are symbols from 0 to 9. The input tokens can be coupled with meta information related to the task such as the priority score in Priority Sort task. During training, the input sequences have length up to L tokens and can grow to I. 1, 2 1), 1( or 8(L+1) during testing. Our setting is more challenging than previous generalization tests on algorithmic reasoning because of four reasons: (1) the task is 10-class classification, harder than binary prediction in Graves et al. (2014). (2) the testing data can be eight time longer than the training and the training length is limited to L R 10, which is harder than Grelenstette et al. (2015), (3) there is no curriculum learning as in Kurach et al. (015)), and (4) the training label is the one-hot hot value of the token, which can be confusing in case one token appears multiple times in the input sequence and tougher than using label as the index/location of the token as in Vinyals et al. (2015).
Here, we design several tasks. Content-free tasks involve permuting tokens in input sequence using certain position-based rules: First-In-First-Out (Copy), rules demand linear pointer manipulations (traverse the sequence from the head or tail, to output the target token), the last one uses a non-linear, length-dependent manipulation rule: if is odd, y: = =1
-in-ii--t-uutt (Reverse) and Mtr. While the first two
if is even,
V/ = -1- Content-based tasks need the input's token value together with symbol processing to arrange the output sequence. We introduce 3 tasks: Dynamic Recall, Priority Sort and ID Sort. Readers can find the details of these tasks in Appendix D.1
Baselines are categorized into 4 groups: (1) Traditional RNNs such as LSTM Hochreiter & Schmidhuber
1997p, (2) Sequential attention models: Content Attention [Bahdanau et al. 2014). Location Attention
[uong et al.] 2015). Hybrid Attention (our bascline concatenates the attention vectors from content and
location attention). (3) MANNs such as NTM (Graves et al.] 2014). DNC (Graves et al. 2016 Neural Stack
Grefenstette et al. 2015) and Transformer (Vaswani et al. 2017), and (4) pointer-aware models: NRAM
(Kurach et al. 20155) PtrNet (Vinyals al.) 2015), ESBN (Webb et al., 2020) and our method PANM. In
this synthetic experiment, we adopt LSTM as the encoder for PANM. All baselines are trained with fixed number of steps (100K for ID Sort and 50K for the rest), which is enough for the training loss to converge. For each task, each baseline is trained 5 times with different random seeds and we use the best checkpoint on L 1 mode validation to evaluate the baselines.
Results We report the average accuracy across different testing length for each task in Table 3 Over- all, PANM significantly outperforms the best competitors ranging from 10-20% per task. Compared with individual baselines, the improvement is much higher (Appendix D.I.. We illustrate how the pointer ma-
8

---

Published in Transactions on Machine Learning Research (10/2024)
(a)
(b)
MENX
PANM
Transhenear
PANM
TRANSFORMER
PAWM
Perple xixty
1000
300
90
SANN
D2
PANM
100
50
30
0
20
40
60
0611) 08:23 09.3
05.01
35.22
0.5
Epochs
Layers)
Figure 3: (a) Dyck: mean std. accuracy over 5 runs with different testing lengths. (b) Machine translation task: Perplexity Multi30K dataset (the lower the better). We sort. the sequences in the data by length and create 2 settings using train/test split of 0.8 and 0.5, respectively. The baselines are Transformer and PANM. Left: The best test perplexity over 2 settings for different number of Transformer's layers (1 to 3 layers). Right: an example of testing perplexity curves over training epochs for the case of 0.5 train/test split (2 layers) where we run 3 times and report the msantstd. The y-axis is visualized using log scale.
nipulation works for Copy and ID Sort in Fig. 2 (c) and (d). In Copy, only Mode-1 access is needed. As decoding step t increases, Pointer Unit generates ps following the increment of the addresses as expected. In ID Sort, both Mode-1 and 2 are needed. The Pointer Unit generates pt incrementally tokens from left to right (Mode 1). Then, the Mode-2 pointer p is computed via attention to discover token with the same id, which will be the output at step Without Mode-2 access, PANM certainly fails this task. Experiments with varied number of heads are in Appendix D.6.
trace the input
3.2 Dyck Language Recognition
Truly understanding the hidden law of context-free grammars such as Dyck (Dn) is challenging for neural networks, even those with memory and attention YYu et al. 2019). The language consists of strings with balanced pairs of brackets of different types (11, (l..)z)...)n), generated by the following rules: S + |:SS, with probability p/n or 5S with probability or with probability - pp- Here, P,q are parameter of the language and 6 is equivalent to EOS token. We follow the sequence prediction task and datasets in Suzgun et al. (2019) where the input is an unfinished Dyck string, and the output is the set of possible brackets for the next token, e.g., for D2, (I) + or or [. We follow the authors to enable set prediction by representing output Y/ a multi-hot vector.
We adapt PANM to this autoregression task by masking M to ensure the decoder only see tokens up to the current decoding step. Since the token set is simple, we do not need to use any encoder, i.e., raw input tokens are stored in M. The SOTA baseline in this task is SRNN (Suzgun et al., 2019), an autoregressive model using stack as memory. We use this model as the decoder to make the setup of PANM close to SRNN. The only difference is that PANM has Pointer-Based Memory Access (Fig. (b)). To make the task more challenging, we limit the maximum training length L 10 (D2) and 20 (Dx) tokens, and the testing lengths are ++2 2, 2L, 4L, 8L. We choose L minimum numbers such that the model can perform decently training data. The standard training and lesting sizes are 5000. We train the models for 5 epochs and evaluate them on the training data at the end of each epoch to save model checkpoints. We use the best checkpoints for generalization test. Fig. 3 (a) reports the models' accuracy for D2 and D3. Under our extreme setting, SRNN generalization fades out quickly as test lengths increase, especially for Dy whereas PANM performance degradation happens at a much slower rate, outperforming SRNN by around 20% on average in both tasks at any test lengths.
3.3 Compositional Learning
SCAN In this task one needs to map an input sentence into an output sequence of commands (Lake & Baroni 2018). The sequences are compositional, consisting of reusable parts. For example, in one case, "jump twice" should be mapped to *JUMP JUMP" and in another, "walk twice" becomes "WALK WALK". We
9

---

Published in Transactions on Machine Learning Research (10/2024)
Split
Model
0.8-0.2
0.5-0.5
Transformer
0.79 + 0.01
0.76 = 0.01
U. TRM+ RPE
0.80 +: 0.02
0.75 + 0.01
PANM (Ours)
0.85 0.02
0.81 0.08
Table 5: bAbl QA: Testing accuracy (mean + std.) over 5 runs.
focus on the "length split" datasets where the training sequences are shorter than the test ones with 11 length modes = 22, 24,,,,40 40 (Newman et al. 2020). We adopt the benchmark, training procedure and baselines prepared by Csordás et a.. (2021), which achieves strong results under standard s2s learning.
Here, our aim is not to break SOTA, which can be achieve by hybrid-symbolic architectures (Chen et al, 2020: Shaw et al., )021). Instead, we focus on improving Transformer generalization in this task, hence the basclines are chosen as several variants of Transformers (TRM) targeted to sequence extrapolation, including those using Relative Positional Encoding (RPE (Dai et al. 2019)) and Universal Transformer (U. TRM (Dehghani et al.) 2018)), which an advanced Transformer variant that recurrently processes each token, and can dynamically adjust the number of processing steps. Following Csordás ct al. (2021), each baseline is trained 5 times for 50K steps and the resulting model after training is used for evaluation (no validation). Here, we Transformer as the Encoder, which is the same the TRM, and stack the Controller to another Transform Decoder (see details in Appendix D.3. Hence, the only difference is the decoding where PANM leverages pointer manipulation.
Table 4 shows that PANM outperforms other baselines in the hardest settings when the training length is up- to 22, 24, and 25. For 22 and 24 cases, general models like PANM cannot show perfect generalization because some testing compositions is entirely discarded from the train set. In casier settings, PANM shares the perfect median accuracy with the sophisticated U. TRM + RPE although it does not use RPE. Remarkably, despite sharing the same encoder, TRM performs much worse than PANM and even fails to learn in casy modes (33, 36, 40), indicating the importance of pointer handling in this testbed. One problem for other baselines is the EOS decision (when to generate ending token), which requires length tracking (Newman et al.) 2020). As they do not have content-free sequence iteration mechanisms, it is extremely hard to trace the length without overfitting to the training data. On the other hand, PANM can hypothetically generate pointer incrementally and capture the difference between the last and the first. pointers, i.e. the input length, and infer the output sequence length based on that information.
Mathematical Problems We test our model on mathematics (Saxton et al. 2018 where the in- put/output are questions and answers about math and each token is a character. For exam- ple, What is - 5 110911? + 110996 (add or_oo___o _ub) and What is the hundreds digit of 31253? + 2 (place_valuee The task requires not only math reasoning, but also natural language understanding. We follow the training from Csordás et al. (2021) to conduct experiments on 2 subsets: add_ ____or__ubb_____________________ (a.s) and place_ value (p.v), and compare our method with Transformer-basedd baselines. Here, we focus on the extrapolating test set involving larger nunbers, more numbers, more compositions, and thus longer input sequences than the training. We use TRM + RPE as the Encoder and the Controller is added to normal TRM decoder. As shown in Table 4 on place value, PANM does not suffer from performance crash as TRM - RPE (0% test accuracy, as admitted in the paper (Csordás et al.) 2021) even though it uses the same encoder). PANM achieves similar results as U. TRM+ RPE on add or sub while outperforming it by 11% on stace_value, We also report PANM +Transformer results in Appendix D.3
3.4 Other NLP Tasks
Question Answering Our objective is to explore the PANM's generalization beyond obviously composi- tional data by applying it in a more practical setting of question answering. For this purpose, we utilize two datasots, namely bAbl (Weston et al., 2015)) and SQUAD 1.1 (Rajpurkar et al.) 2016) where the input sequence is a context paragraph and a question, and the output the answer. To add complexity to the task, we ensure the length of test sequence is greater than that of the training by sorting the context paragraph by
10

---

Published in Transactions on Machine Learning Research (10/2024)
Split
Model
0.8-0.2
0.5-0.5
F1
FM
F1
FM
BERT
0.77
0.64
0.73
0.59
PANM (Ours)
0.78
0.65
0.76
0.61
Table 6: SQUAD 1.1: Testing accuracy after epoch fine-tuning. F1 score and exact match (EM) follows
the standard evaluation Kenton & Toutanova (2019).
Testing Length (# letters)
Task
Model
10
20
40
100
200
500
1000
Llama2-7B
0.1
010
0.10
010
010
010
010
Copy
Llama2-7B FT
98+2
16+4
7+5
0+0
0+0
0=e
0=0
PANM (Ours)
99
99:1
97:1
8642
8512
87:22
92::
Llama2-7B
0.0
0100
0.10
010
01.0
01:0
01.0
Recall
Llama2-7B FT
86#4
040
0+0
0+0
040
0+0
0=0
PANM (Ours)
95#2
80:5
50+8
9+1
0+o
0+o
0=0
Table 7: Syuthetic Algorithmic Tasks: (zero-shot) LLM's mean std. Training length 10 letters. Bold denotes best.
BLEU accuracy (%) over 5 runs.
length and splitting the sorted data into 0.8/0.2 and 0.5/0.5 ratio. Details of the data/task are in Appendix D.4 In bAbl, we configure the PANM similarly to the one described in & 3.3 using Transformer backbone, and test the models after 100-epoch training. The models predict the answer tokens given the context and question tokens. As shown in Table and Appendix Fig. (right), PANM helps Transformer generalize better, consistently improving around 6% and 5% using 0.8 and 0.5/0.5 splits, respectively. Notably, PANM's testing loss is not diverged quickly Transformer's ss indicating PANM's capability of reducing overfitting. In SQUAD, use BERT as the backbone to predict the start and the end of the answer as in Kenton & Toutanova (2019) PANM-assisted model outperforms the baselines by 1% and 2% exact match accuracy, respectively (Table 66. The improvement is significant as BERT is a big foundation model already pretrained with big data and robust against novel test data.
Machine Translation Here, we want lo verify the PANM in machine translation and show that PANM can work with different number layers of Transformer. The results are presented in Fig. 3 (b) where we report the model perplexity on Multi30K (en-de) dataset. The 30K-sample dataset is sorted by input length and split into training and testing s.t. testing sequences are longer, similar to QA task. The results demonstrate PANM can consistently improve the generalization performance of Transformer across different split ratios and the number of
layers.
Task
Model
Zero-shot
3-shot
Llama2-7B
0+0
32+2
Arithmetic
Llama2-7B FT
1744
1943
PANM (Ours)
3342
3842
Llama2-78
0+0
0+0
Abstract
Llama2-7B FT
010
94:
PANM (Ours)
3+1
1644
Table 8: BIG-bench Tasks: LLM's mean std. Exact Match (%) over 5 runs. The finetuned models are trained and tested with shortest and longest sequences, respectively. Bold denotes best.
11

---

Published in Transactions on Machine Learning Research (10/2024)
3.5 Scaled Generalization with LLMs
In this section, we explore the compatibility of PANM with LLMs. Specifically, we use Llama-2 (Touvron ct al. 2023) as the backbone model and integrate the PANM layer on top of the final attention layer. There are 3 baselines: the pretrained LLM (Llama2-7B), the finetuned LLM (Llama2-7B FT) and the finel
LLM 1.M-+PA PANM (PANM). The models are finetuned with the same training configuration such LoRA (Hu
ct al.
2021) and AdamW optimizer [Loshchllov & Hutter 2018). The evaluation is executed using Language Model Evaluation Harness library (Gao et al.] 2023).
Synthetic Algorithmic Tasks In the first test, we use Copy and Dynamic Recall (D. Recall) similar to those described in $3.11 We finetune both the LLM and LLM+PANM on a training datasct and then evaluate them on separate testing dataset. In this test, each token a letter sampled from the English alphabet. (case sensitive), and the training data consists of 100,000 sequences, each of maximum 10 letters long, combined with an instruction introducing the task (see Appendix D.5). After fine-tuning, we evaluate the models on multiple testing sets, each of 1000 testing sequences with sequence lengths ranged from 10 to 1000 letters.
The results in Table 7 show that normal LLM finetuning generally outperforms the pretrained LLM in downstream tasks, but performance degrades rapidly as the testing length increases. Specifically, without PANM, performance drops to 0% at test lengths of 100 and 20 for Copy and Dynamic Recall, respectively, In contrast, PANM achieves 86% and 80% at these testing lengths. Notably, PANM maintains rcasonable performance in the Copy task up to a lest. length of 1000. We also examine the evaluation setting when 5 few-shot examples are added to the prompt to enable LLM in-context learning. The results in Appendix Table [1 confirm that PANM significantly outperforms other baselines in this setting. Compared to the results of the same tasks in 43.1 we observe that PANM reaps more benefits in maintaining generalization with a stronger backbone like Llama2-7B, highlighting the versatility and potential of PANM.
BIG-bench Tasks Here, we foeus on challenging benchmarks designed for LLMs. We pick the first two tasks from the BIG-bench benchmark (Srivastava et al. 2023), one is symbolic (Arithmetic) and one is not (Abstract). To ensure the experiment is both practical and challenging, we finetune the model using only a small subset of the training set containing the shortest input sequences. We then evaluate the model on a lesting set with the longest input sequences to assess its generalization to longer sequences. Details of the chosen tasks are given in Appendix D.5
The results in Table demonstrate that PANM significantly improves the generalization accuracy of finctuned LLMs, purperforinng the original finetuned LLM by a large margin of 16% and 3% in Arithmetic and Abstract, respectively. Even when augmented with 5 in-context examples, PANM maintains substantial generalization gain (19% and 7%), confirming its effectiveness as plug-and-play memory module for LLMs.
4 Related works
There are many attempts to augment neural networks with external memory (MANN) to improve their symbol-processing ability. Pioneers such as NTM (Graves et al. 2014) and DNC (Graves et al. 2016) propose computer-lik, memory read/wiite operations with content- based attention mechanisms, and thus in principle, can execute any symbolic rules. However, learning the hidden law end-to-end from sequence data is extremely difficult. Therefore, MANNs including Transformers (Vaswani et al. in out-of-distribution testbeds, espccially length extrapolation (Delétang et al. good at reasoning and generalization, but bad at symbolic processing (Qian et al., 2023 Tang al., 2023). We use LLMs only to show our task difficulty (Appendix D.7., not baseline, because they not the same scale as our method.
2017). may fail miserably
2022 Recent LLMs are
Many recent works advocale the use of specialized memory architectures such as stacks (Grefenstette el al. 2015 Hao et al., 2018 Suzgun et al., 2019), key-value memory (Webb et al. 2020; Le et al. 2020a) and improved attentions (Kurach et al.J 22015] Russin et al., 2019 Dubois et al.] 2020 Le ct al. 2019). These methods employ different inductive biases in designing the memory and attention, yet not following the two principles advocated by our paper. Although they may work remarkably certain synthetic tasks, they are
12

---

Published in Transactions on Machine Learning Research (10/2024)
not examined on various benchmarks or compatible with different sequential backbones. Other orthogonal
approaches focus on model initialization (Zhang et. al. 2019), data augmentation (Andreas 2020 training details (Csordás et al., 2021). Besides differentiable models, there are major progress in compositional rule learning that leverage ncuro-symbolic architectures (Nye et al.) 2020; Shaw et al. 2021 Chen et al. 2020 or
reinforcement learning (Liu et al., 2020). We have not compared our model with these task-specific methods, as our focus is on improving the systematic generalization of fundamental differentiable models.
Our approach is mainly related to key-value memory because the address bank can be viewed the key and the data memory the value. However, the key in other works is either learned through backpropagation [Le ct al.] 2020a Le & Venkatesh] 2022) or computed based on the input data (Webb et al., 2020). In contrast, our "keys" are generated as fixed numbers (physical memory addresses [ principle 1), which is totally separated from the data and extendable to longer sequences. We argue that using addresses as keys is critical to symbol processing because it explicitly allows pointer assignment, dereference and arithmetic. A related generalizationcnnal
scheme is design positional encoding of tokens in sequence (Vaswani et al., 2017 Dai et. al., 2019 Li & McClelland] 2022). Recent. advancements in this area have shown competitive results for length extrapolation up to 4 and 8 times the training length (Chen et al.) 2024).. Unlike these approaches, our method isolates the physical addresses (i.c., "positional encoding") from the data, allowing pointer transformation through time steps and absolutely detaching pointer manipulation from the input. Consequently, our method allows the freedom to point to any timestep, making it adaptable to a variety of backbones, not limited to Transformers.
5 Conclusion
We introduce a neural memory model called PANM, designed to manipulate pointers and learn symbol processing rules for improved length extrapolation. PANM separates symbols from data and utilizes address bank to enable data-isolated pointer manipulation through address attention. PANM consistently outperforms strong basclines in tasks such as algorithm mining, compositional lcarning, mathematical rea- soning, context-free grammar recognition, and practical NLP lasks, even when test sequences are significantly longer than training sequences. Remarkably, PANM achieves these results across various backbone models, including LSTMs, Transformers, and LLMs.
Limitations While PANM provides significant benefits in enhancing sequence processing, it does come with certain limitations: (1) Computational Complexity: PANM introduces additional computational over- head, particularly when used with smaller models like LSTMs, where complexity increase by 20-30%. For larger models, such LLMs, this overhead is minimal, making PANM's operations relatively inexpensive. (2) Memory Requirements: The additional memory needed for the Address Bank and Pointer Unit is small relative to large backbone models. However, these components do contribute to overall memory usage, which may be a consideration in resource-constrained environments. (3) Implementation Challenges: Integrating PANM into existing architectures might require engineering effort. Ensuring seamless interaction wetn the backbone and PNAM and tuning PANM hyperparameters for optimal performance on specific tasks may require additional experimentation.
References
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
Jacob Andreas. Good-enough compositional data augmentation. In Proceedings of the 58th Annual Meeting of the Association for Compulational Linguistics, pp. 7556 7566, 2020.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning align and translate. XiXiv preprint ar 1rxi::1400.00. 2014.
13

---

Published in Transactions on Machine Learning Research (10/2024)
Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. Systematic generalization: What is required and can it be learned? In International Conference on Learning Representations, 2018.
Guanzheng Chen, Xin Li, Zaiqiao Meng, Shangsong Liang, and Lidong Bing. CLEX:
Continuous length extrapolation for large language models. In The Twelfth International Conference on Learning Represen- tations, 2024. URL
Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, and Denny Zhou. Compositional generalization via neural-symbolic stack machines. Adnances Neural Information Processing Systems, 33:1690-113 18:1690--771,, 1701, 2020.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv preprint arXiv:1412.3555, 2014.
Róbert Csordás, Kazuki Irie, and Jürgen Schmidhuber. The devil is in the detail: Simple tricks improve systematic generalization of transformers. arXiv preprint arXiv:2108. 12284, 2021.
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime G Carbonell, Quoc Le, and Ruslan Salakhutdinov. Transformer-el Attentive language models beyond a fixed-length context. In Proceedings of the 57th Annual Meeling of the Association for Compulational Linguislies, pp. 2978 2988, 2019.
Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. Universal trans- formers. In International Conference on Learning Representations, 2018.
Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, Li Kevin Wenliang, Flliot Catt, Marcus Hutter, Shane Legg, and Pedro A Ortega. Neural networks and the chomsky hierarchy. arXie preprint arXiv:2207.000 000009 2022.
Yann Dubois, Gautier Dagan, Dicuwke Hupkes, and Elia Bruni. Location attention for extrapolation to longer sequences. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 403-413, 2020.
Aaron Fisermann, Jac Hee Lee, Cornelius Weber, and Stefan Wermter. Generalization in multimodal lan- guage learning from simulation. In 2021 International Joint Conference on Newral Networks (IJCNN), pp. 1-8. IEEE, 2021.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Lau- rence Golding, Jeffrey Hsu, Alain Le Noachh, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot language model evaluation, 12 2023. URL httpges//eenoooooog///eeooooodd
Tong Gao, Qi Huang, and Raymond Mooney. Systematic generalization on gscan with language conditioned embedding. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Com- putational Linguistics and the 10th International Joint Conference on Natural Language Processing, pp. 491- 491-503, 2020.
Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. arXiv preprint dr1iv:1410.5401, 2014.
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Tvo Danihelka, Agnieszka Grabska-Barwifskk, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, (38(666)) 2016.
Edward Grefenstette, Karl Moritz Hermann, Mustafa Sulcyman, and Phil Blunsom. Learning to transduce with unbounded memory. Advonces in neural information processing systems, 28, 2015.
Yiding Hao, William Merrill, Dana Angluin, Robert Frank, Noah Amsel, Andrew Benz, and Simon Mendel- sohn. Context-free transductions with neural stacks. FMNLP 2018, pp- 306, 2018
14

---

Published in Transactions on Machine Learning Research (10/2024)
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 988):1735-178) 1997.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. orXiv preprint arXim:2106.0000 09685, 2021.
Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger P Levy. A systematic assessment of syntactic generalization neural language models. arXiv preprint arXiv:2005.0800 03692, 2020.
Jacob Devlin Ming- Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pp- 4171-4188 2019.
Trenton E. Kriete, David C. Noclle, Jonathan D. Cohen, and Randall C. OåReilly. Indirection and symbol- like processing in the prefrontal cortex and basal ganglia. Proceedings of the National Academy of Sciences. 110:16390 - 16395, 2013.
Karol Kurach, Marcin Andrychowiez, and llya Sutskever. Neural random-access machines. orXiv preprint arXiv:511100332, 2015.
Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sequence-to-ser losseueece recurrent networks. In International conference on machine learning, pp. 2873 2882. PMLR, 2018.
Brenden M Lake, Compositional generalization through meta sequence-to-sequene learning. Advances in neural information processing systems, 32, 2019.
Hung Le and Svetha Venkatesh. Neurocoder: General-purpose computation using stored neural programs. In International Conference on Machine Learning, pp. 12204-12221. PMLR, 2022.
Hung Le, Truyen Tran, and Svetha Venkatesh. Learning remember more with less memorization. arXiv
preprint XrXi::1901.0133 2019.
Hung Le, Truyen Tran, and Svetha Venkatesh. Neural storeed-rraann memory. In International Conference on Learning Representations, 2020a. URL http:e//pppparrooeee.eeeerttrrtr
Hung Le, Truyen Tran, and Svetha Venkatesh. Self-attentive associative memory. In Hal DaumAO IIII and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1892-5691, Virtual, 13-18 Jul 2020b. PMLR.
Yuxuan Li and James McClelland. Systematic generalization and emergent structures in transformers trained structured tasks. In NewrlPS '22 Workshop All Things Attention: Bridging Different Perspectives on Attention, 2022. URL https:///penrevvee.. httpeeewwwwwweeeeeeeeeeeeeeeeeee forum www.://ooormddddBdaaaaa.a 2id=BNNaKKYaduu..
Qian Liu, Shengnan An, Jian-Guang Lou, Bei Chen, Zeqi Lin, Yan Gao, Bin Zhou, Nanning Zheng, and Dongmei Zhang. Compositional generalization by learning analytical expressions. Advances in Newral Information Processing Systems, 33:11416-1144 2020.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2018.
Minh- Minh-Thang Luong, Hieu Pham, and Christopher D Manning, Effective approaches to attention attention-based neural machine translation. arXiv preprint ww0.::::00000000 04025, 2015.
Benjamin Newman, John Hewitt, Percy Liang, and Christopher D. Manning. The eos decision and length extrapolation. In BlackBoNNLPEMNNLP 2020. URL https://nlp. cttaaford.edu/uuuui
nemaan002eextrpplaaai...
Maxwell Nye, Armando Solar-Lezama, Josh Tenenbaum, and Brenden M Lake. Learning compositional rules via neural program synthesis. Advances in Neural Information Processing Systems,
2020.
15

---

Published in Transactions on Machine Learning Research (10/2024)
Jing Qian, Hong Wang, Zekun Li, Shiyang Li, and Xifeng Yan. Limitations of language models in arithmetic and symbolic induction. In Anna Rogers, Jordan L.. Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2028, pp. 1285-92298 Association for Computational Linguistios, 2023. doi: 10.18653/v v1
URL https: :/tp::://doii org/ 10. 1tt////..1163//1///2222 attp:///////....llooooo.g...
acl-long. .166
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-222 2292, 2016.
Jake Russin, Jason Jo, Randall C O'Reilly and Yoshua Bengio. Compositional generalization in deep seq2seq model by separating syntax and semantics. arXiv preprint arXic:1904.099008, 2019.
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. Analysing mathematical reasoning abilities of neural models. In International Conference on Learning Representations, 2018.
Peter Shaw, Ming-Was Wei Chang, Panupong Pasupat, and Kristina Toutanova. Compositional generalization and natural language variation: Can a semantic parsing approach handle both? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Conference Natural Language Processing (Volume 1: Long Papers), pp. 922 938, 2021.
Ray Solomonoff. Algorithmic probability, heuristic programming and agi. In 3d Conference on Artificial General Intelligence (AGI-2010), pp. 57 63. Atlantis Press, 2010.
Aarohi Srivastava et al. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https:
//openreview wtt.:/oorooomiiiddyl/...
Mirac Suzgun, Sebastian Gehrmann, Yonatan Belinkov, and Stuart M Shieber. Memory-augmented recurrent neural networks can learn generalized dyck languages. orXiv preprint arXi::1911.0333 2019.
Xiaojuan Tang, Zilong Zheng, Jiaqi Li, Fanxu Meng, Song-Chun Zhu, Yitao Liang, and Muhan Zhang, Large language models are in-context semantic reasoners rather than symbolic reasoners. arXiv preprint arXiv:2305.1400 14825, 2023.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arxiv preprint orXiv:2307.09280 2023.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks Advances in neural information processing systems, 28, 2015.
John Von Neumann. First draft of a report on the edvac. IEEF Annals of the History of Computing, 15(4): 27 75, 1993.
Taylor Whittington Webb, Ishan Sinha, and Jonathan Cohen. Emergent symbols through binding in external memory. In International Conference on Learning Represendations, 2020.
Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart Van Merriënboer, Armand Joulin, and Tomas Mikolov. Towards ai-complete question answering: A set of prerequisile toy tasks. arXiv preprint arXiv:1502.00 05598 2015.
Bo Wu, Haoyu Qin, Alireza Zareian, Carl Vondrick, and Shih-Fu Chang. Analogical reasoning for visually grounded language acquisition. arXiv preprint arXiv:2007.11668, 2020.
Greg Yang. Lie access neural turing machine. orXiv preprint ar
2016.
16

---

Published in Transactions on Machine Learning Research (10/2024)
Xiang Yu, Ngoc Thang Vu, and Jonas Kuhn. Learning the dyck language with attention-based seq2seq models. In Proceedings of the 2019 ACL Workshop BlackborNL.P: Analyzing and Interpreting Neural Networks for NLP, pp. 138 146, 2019.
Biao Zhang, Ivan Titov, and Rico Sennrich. Improving deep transformer with depth-scaled initialization and merged attention. In Proceedings of the 2019 Conference Empirical Methods in Natural Longuage Pro- cessing and the 9th International Joinl Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 898-988 909, 2019.
17

---

Published in Transactions on Machine Learning Research (10/2024)
Appendix
A More Discussion on Related Works
The proposed address attention in our paper is comparable to two known mechanisms: (1) location-basee
attention [Luong et al.) 2013 Dubois et al. 2020) and (2) memory shifting (Graves ct al. 2014 Yang 2016).
The former uses neural networks to produce atfention weights to the memory memory/sequence, /squeence which cannot help when the memory grows during inference since the networks never learn to generate weights for the additional slots. Inspired by Turing Machine, the latter aims to shift the current attention weight associated with a memory slot to the next or previous slot. Shifting- like operations can handle any sequence length. However, it cannot simulate complicated manipulation rules. Unlike our PU design which obeys $ [s principle II, the attention weight and the network trained to shift it depend on the memory content M. That is detrimental to generalization since new content can disturb both the attention weight and the shifting network as the memory grows.
Another line of works tackles systematic genoralization through meta-lcarning training (Lake. 2019), while our method employs standard supervised training. These approaches are complementary, with our method concentrating on enhancing model architecture rather than training procedures, making it applicable in diverse settings beyond SCAN tasks. Additionally, the study by Hu et al. (2020) addresses syntactic gener- alization (Hu et al. 2020), a different problem compared to our paper, which emphasizes length extrapolation across various benchmarks. Notably, our paper considers similar baselines, such as LSTM and Transformer, as those examined in the referenced papers. There are other lines of research targeting reasoning and gen- cralization using image input Wu et al., 2020 Eisermann et al., 2021). They are outside the scope of our paper, which specifically addresses genera generalization lization for longer sequences of text or discrete inputs
Our address bank and physical pointers can be viewed some form of positional encoding. However, we do not use simple projections or embeddings to force the attention to be position-only. Instead, we aim to learn a series of transformations that simulate the position- based symbolic rules. At each time step, new pointer ("position') is dynamically generated that reflects the manipulation rule required by the task (e.g. move to the next location), which is unlike the positional encoding approaches such as RPE (Dai et al., 2019) which aims to provide the model with information on the relative position or distance of the timesteps. We summarise the difference between our method and Transformer in Table g
B More Discussion on Base Address Sampling Mechanism
We provide a simple example to illustrate how base address sampling help in generalization. Assume the training sequence length is 10, and the desired manipulation is p' = p (copy task). Assume the possible address range is 0, 1, ..., 19, which is bigger than any sequence length. If an = 0 ; the training address bank contains addresses: 0, 1, ...8, 9. Without base address sampling, the model always sees the training address bank of 0, 1, 8,,8 9 and thus can only learn manipulating function for S p S 9, thereby failing when testing address bank includes addresses larger than 9.
Thanks to base address sampling, at some point of training, as = 10 the training address bank is 10, 19. The manipulating function sees p > 9 and can learn to transform p' = p + 1 for > 9, e.g., transform p = 10 + p' = 11. The learning happens because the pointer's value (*p) is used to predict the output sequence. The task loss will reward "p that follows the rule, and update the Pointer Unit such that it transforms the p following the rule to minimize the loss. During testing, the input length can be 12, we set an- and the address bank is 0, 1,..., 10, 11. The learned manipulation can still be applied to new locations 10th, and 11th.
We can prove that the complexity of exposing all addresses to the model is practically small compared to the normal training. Assume the training input sequence length is L, and the number of possible addresses is Lmaz. Here, Lmax indicates the possible testing length that the model can handle. When Lmaz + 0o the expected number of samples required for exposing all addresses is (n log n) where n = Lmaz/ (we can formulate this problem as Coupon collector's problem). For example, in even an extreme address range of Temar 106 (in practice we rarely need that big range) to train input sequences of length 10, we only need
18

---

Published in Transactions on Machine Learning Research (10/2024)
Difference
Transformer
PANM (Our)
Key
Keys are computed based on input data. Hence, when meeting novel data during testing, Transformer will observe novel keys, and cannot work properly.
The keys in our approach are generated as fixed numbers,
Generation
specifically physical memory addresses. These keys are entirely separate from the data. The fixed nature of our physical addresses allows pointers to be easily manipulated and extendable to longer sequences.
Extendable to Longer
The dimension of attention weights varies with input length, making arithmetic transformations on these attention weights infeasible as the sequence length increases.
Sequences
Symbol
The use of attention weights as implicit pointers may lack the explicitness needed for effective symbol processing.
Using physical addresses as keys in our approach is crucial for symbol processing as explicitly allows pointer assignment, dereference, and arithmetic operations.
Processing
Advantages
Physical Address Positional Encoding
Positional encoding be generated independently from data. However, they are not separated from the input data as our physical addresses. There is no explicit mechanism Transformer to attend only to these positional encodings to transform pointers to point to these positional encodings from one step to another.
Our physical addresses detached from the data, supporting the transformation of pointers through timesteps and isolating pointer manipulation from the input.
Table 9: PANM vs Transformer
to sample 10s log 10a sequences, which is oflen smaller than the size of the training datasets. Empirically, in our experiments, we always train our method with the same number of batch size and training steps other basclines to ensure a fair comparison, and we rcalize that it is always possible to expose all the addresses to our model during training.
C More Discussion on Model and Architecture
) H" ) to achieve generalization using pointer arithmetic. In other words, if aB, & (8 h-m we can always find task make PANM fail
We can see that it is critical to have He 2 2 and an ar E { {
I =1
{ (o I H²
to generalize. For example, ag $ Poos f-1 PANM cannot generalize in Reverse task. To see that, without loss of generality, we assume PANM only learn to produce the last token at address ag using
h=1 such that p' + ag. During training, the learned pointer arithmetic to perform Reverse at the first step of the decoding to produce value y1 can only be a function of p': yл *az= =' f(y',, that is, ar fz'). During testing, ar can receive arbitrary value, for whatever learned f, can always find test sequence such that ag # f(p)) Vf because ag # p'. A similar argument can be used for ag and Copy task.
information from some initial addresses p' E
{ (6
""
In the main manuscripl, we only experiment with 1 Mode-2 pointer (H = 1). If H = 0, obviously PANM will fail in tasks such as ID Sort. Using more H and Ha can still be beneficial in exchange for slower computation (sce Appendix D.6p. In all experiments, we use 250-dimensional GRUs for the PU and Ctrl. The encoder and decoder (to stack the Controller on) can vary across tasks. The general plug-and-play framework is illustrated in Fig. 4. We also summarize operations of our model in Algo. 1
19

---

Published in Transactions on Machine Learning Research (10/2024)
OUTPUT SEQUENCE
PANM MEMORY
PANM Module
PANM CONTROLLER
Backbone Model
ENCODER
DECODER
INPUT SEQUENCE
Figure 4: PANM plug-and-play architecture. The encoder and decoder can be any model (LSTM, Transformer or BERT). PANM Controller can be used as the last layer of the Decoder to access the memory during decoding. To reduce the number of parameters of the augmented architecture, the decoder's number of layers can be decreased.
D Experimental Details
All datasets and public codebases used licensed under Apache MIT License. We trained all the models single Tesla V100-SXMM GPU, The rumning time of PANM depends the Encoder and the specific tasks. Overall, with 2 Mode-1 pointers and Mode-2 pointer, PANM operates at 70-80% of the speed of the backbone model. For instance, in the Copy lask, PANM achieves 15 ilerations per second compared to LSTM's 20 iterations per second. When using a Transformer Encoder, PANM runs at 77 iterations per second, whereas the Transformer runs at 90 iterations per second. Notably, the larger the backbone, the smaller the difference in rumning time. In LLM experiments, there no significant difference in training/ inference speed between settings with PANM and without PANM, with speeds of 15.2 iterations per second and 15.6 iterations per second, respectively.
D.1 Algorithmic Reasoning
We first give the details of the content-based tasks below.
In Dynamic Recall, an arbitrary input token is chosen as the query and is added to the end of the input sequence. Depending on the length of the input, a.k.a, odd or even, the first target token will be on the left or right of the query, following its succeeding tokens in the input. This task requires both content matching (find query token the input sequence) and position-based access (shift. left or right).
In Priority Sort, each input token is associated with priority score sampled from the standard normal distribution. The target output will be tokens from the input sequence sorted ascending by their the score. This task can be solved in many ways and likely needs complicated symbol processing such as looping through items in the sequence and comparing the score of tokens.
Finally, in ID Sort, each input token is augmented with an id feature vector sampled from standard multivariate normal distribution such that every tokens share one id. For example, with input æ 22, #3 24. 21 and 24 may share one id while 22 and v3 shares another id. The pairing is chosen randomly. The output token at position i-th will be the input token that share id with the i-th input token. The correct output
20

---

Published in Transactions on Machine Learning Research (10/2024)
for the earlier example is 43, T2, 21. This task is specifically designed to test the ability to learn Mode 2 pointer-based memory access.
In this task, we implement the baselines such as LSTM, attention models and Transformer using Pytorch library. The hidden state dimension for these models are set to 512, which results in around 1-3 millions parameters. We tuned the number of layers of the encoder encoder/decoder /decoder for these baselines Copy task, and realized that 1-layer gave the best performance. For NTM and DNC, we use public repositoried with defanlt controller's hidden state of 256 dimensions and 128-slot external memory, which results in around 1.2 millions parameters. We use the ESBN's author codebase D with default parameter setting, resulting in 11.2 million parameters. For PtrNet, since we do not use token index as the training label, we produce the predicted token by performing weighted sum the input tokens using the PirNet's 's attention weights. PirNet's hyperparameters are the same as attention models. We could not find the authors' code for Neural Stack and NRAM so we implemented them and tuned hyperparamcters for the Copy task at length such that the model sizes are about 1.1 million paramcters. In this task PANM uses LSTM with hidden state of 256 the Encoder and does not stack the Controller any decoder models, resulting in 11.1 million parameters.
In this experiment, all the models are trained without teacher forcing as in Graves et al. (0144), i.e, the input to the decoder is zero (24 0). The detailed average accuracy (mean + std.) of each method together with the actual length of each testing mode are reported in Tables 13 18
Overall, PANM observes significant improvement ranging from 10-20% on each task. We note that when compared with individual baselines, the improvement is much higher. Consider Copy as an example (Fig. 2), PANM outperforms the worst baseline Transformer by around 60% at 2(T. 1) and 30% at 4(T. + 1); respectively. As stated carlier that our tasks challenging, thus, originally strong basclines such as NTM, DNC, and Neural Stack do not generalize well al extreme lengths, especially in ID Sort. ID Sort is trickier than content-fie free tasks, making some baselines fail at length L even though it is in the training data. The best other model in this case is Content Attention, which clearly underperforms our PANM from few % to 50% (Fig. pp). Without curriculum learning and under the 10-class prediction setting, methods that use implicit pointers, such as PirNet, NRAM, and ESBN, demonstrate mediocre performance on average when compared to PANM. Furthermore, PANM also outperforms in length-dependent tasks (Mix, D. Recall), indicating that it can track the sequence longth in extrapolation. We hypothesize that PANM's content-free pointer generation mechanism to simulate list iteration makes it possible.
In Copy, only Mode-1 access is needed. As decoding step t increases, Pointer Unit generates pt following the increment of the addresses as expected. That said, for several steps, the address attention is not sharp, showing other addresses pointed by the pointer, which is not surprising since we use soft. attention and it This problem gets worse as test length
is hard for neural network to learn the exact rule: pf+1 y increases as the error accumulates, especially when the same token can appear many times, which confuses the model. This explains why PANM's performance drops clearly in the hardest case 8(L + 1). Yet, it is still significantly better than others whose results are near random prediction.
D.2 Dyck Language Recognition
In this task, we adopt the SRNN code from Suzgun el al. (2019 using the default parameters. As explained in the main text, this task is auto-regression, hence, 2, Út-1. PANM adopts SRNN (an rugo-regressive model) as the encoder and does not stack the Controller on any decoder models. The result is visualized in Fig. (left).
D.3 Conpositional Learning
In this task, we adopt the code from Csordás et al. (2021) using the default parameters. When using Transformer Encoder, we need to have Transformer-like decoder to align the token representation of the encoding and decoding phases. As such, in SCAN, we utilize the 3-aayer Transformer decoder, replace
htttp::/ggitbb httpas///gghubb..mm/mbahuunngee www.//taaiiinggel////1
htttp://github..com http:/hhhb....////ooo/////////o
21

---

Published in Transactions on Machine Learning Research (10/2024)
100
90
Accuracy
22
60
PANM (8 5-4. 5)
40
TRANSFO 5-3.--55 TRANSPORMER (0.9-8.2)
20
100
Accurac
SRNN
D3
PANM
1.25
60
1.00
40
26
0.50
100
Figure 5: Dyck (Left): mean + std. accuracy over 5 runs with different testing lengths. bAbl QA (Right):
mean + std. testing accuracy and cross-entropy loss across 100 training epochs over 5 runs.
40
60
Mean
30
80
80
Median
Accuracy
40
60
60
run-l
20
run-2
20
run-3
10
20
20
run-4
L=22
L=24
L= 25
L=26
run-5
0
20K
4DK
20K
40K
20K
4DK
20K
40K
Stop
Step
Sleg
Figure 6: SCAN: PANM's exemplar learning curves.
its last layer by the Controller. Formally, 2, in Eq. 7 becomes Decoder(ıt-) where the Decoder is a 2- layer Transformer. In Mathematics reasoning task, we use similar integration except that the Encoder is Transformer with relative positional encoding (TRM + RPE). By reducing the number of decoding layer, we ensure PANM's hyperparameter number equivalent to that of the Transformer baseline (12M). All models are trained with teacher forcing as in Csordás et al. (2021).
SCAN The training size is 16990 and the test size is 3920. SCAN is well-known and standard benchmark for testing compositional learning and generalization in sequential models. One property of this dataset is that a new length often contains new rulcs that must be captured by the model to ensure generalization, and thus, if the model fails to learn hidden rule, its performance may drop significantly from one length split to another. Fig. G illustrates PANM's testing accuracy curves when 22, 24, 25, 26. Other learning 26 where PANM easily solves the task perfectly.
curves for L 26 looks similar
and standard deviation. Here, we augment TRM and TRM+RPE with PANM. Both shows improvement, especially for TRM+RPE, indicating that PANM is compatible with other methods designed to improve generalization in Transformer.
Mathematical Problems Table 19 reports the accuracy with
D.4 Other NLP Tasks
The bAbl dataset consists of 20 synthetic tasks that evaluate various reasoning skills, To prepare the data for each task, we combine hatp/vali//tesss into single set and sort it by length and split it into training and testing sets, as described in the main text. We train the models jointly all 20 tasks and measure the accuracy of thoir answers, which are considered correct only if they match the ground truth answer perfectly. The Transformer used here has 8 heads, 3 layers of encoding, 3 layers of decoding, and hidden dimensions of 512. PANM uses the same Transformer backbone except that the decoder has layers make the model size equivalent. We run each model times report the mean and standard deviation as in Fig. 5 (right). Table reports the detailed numbers.
follows exactly the standard protocol presented in (Le et al. (020b)). The
22

---

Published in Transactions on Machine Learning Research (10/2024)
The SQUAD dataset contains more than 100K realistic context/questioon-aswerr pairs. Again, combine train /estt into single set and sort it by length and split into new train/test sets. Following Kenton & Toutanova 2019), we use BERT model (tttps://huggingffel) http:e///uugiggggaacccooooetttte httpbbert-bas--uccaaeeeee uncased) to predict the start and end location of the answer, and finctune the model with the same sctting (c.g., epochs with a learning rale of 5e-5) except that our batch size 16 with our GPU. PANM appends the Controller BERT to predict the start and end. Both BERT and PANM have around 114 million parameters. Table reports the detailed numbers.
D.5 LLM Tasks
The training follows torchtune library: https:// http.//////ooooc/ooo/tcuuu.u using the default training hyperparameters. For experiments using LoRA finetuning, we use LoRA with the following configuration:
Rank: 8
Target layers: q proj.v proj
The optimizer configuration is:
Optimizer:
AdamW with weight decay of 0.01
Learning rate: 3c-4
Learning rate scheduler:
Cosine scheduling with 100 warm up steps
Training configuration:
The batch size is 8 with 4 gradient accumulation steps
Number of epochs: 3
Below the example of the prompt given to the LLM in the used tasks.
Copy
Prompt: "This is copy task. Given this: Fbp FbPiPBmsC An Your answer:" "
Target:
"F "FPPPBBs
Dynamic Recall
Prompt: "This is dynamic recall task. You are given a sequence of characters. The last character is the query and you need to retrieve the first character after or before the query the sequence the number of the characters the sequence odd even, respectively. Given this: E Vf V An Your answer:" Target: "g"
Bigbench Ari thmetic
Prompt: "Q: What is 63539 times 88329?\n choice: 1572879084 choice: 2152788438
choice: banana choice: 34615225\n choice: 5612336331 \n choice: 5564513399911 \n
choice: house "
Target: "5612336331"
23

---

Published in Transactions on Machine Learning Research (10/2024)
Bigbench Abstract
Prompt: "Given the choice: People who live in glass houses shouldn't throw stones stones\nGood nGood things come in small packages inDon' 't put new wine into old pottles\nBeal swords into aloughshares
what you preach And man who is his own lawyer has fool for his client\nA barking dog never bites WHe who pays the piper calls the tune\mApril showers bring forth May flowers\nPis always stink from the head down AnAbsolute power corrupts absolutely\nJack of all trades, master of none\nThe wages of sin is death\nLove of money is the root of all evil\nChristmas comes but once a year AnWhat's sauce for the goose is sauce for the gander\nIt's the squeaky wheel that gets the grease'nPractien army marches on its stomach\nIf the mountain won't come to Mohammed, then Mohammed must go to the mountain'nk soft answer turneth away wrathluHoney catches more flies than vinegar AnDed beat path to your doorLink to (uRveege is dish best served cold\nAll publicity is good publicity\nDon's meet troubles half-waylnFirst impressions are the most lasting'nIt takes two to tango nango\nAn Englishman's 's home is his castle\nlittle things please little minds'nCut your coat to suit your Gloth\nLook before you leap LeaplnCheaters never win and winners never cheat golden key can open any door prophet is not recognized in his own land\nSilence is volden'nSuccens has many fathers, while the best
makes perfect (nThe age of miracles is past
tell tales'nBuild better mousetrap and the world will
failure is an orphan AnGod helps those who help themselves' innauhhhe medicine AnTheee's no accounting for Pastes\nDo unto others you would have them do to you'nd stitch in time saves
judge book by its cover \nHard cases make bad law\nA house divided against itself cannot stand\nTime is moneylnThat which does not kill us makes us stronger LnSeek \nSeek and you shall find'nFailing to plan is plamning to fail AnThe cobbler always wears the worst shoes\nYou are never too old to learn Infe who laughs last laughs Longest\nnon'' shoot the messenger\nGood things come to those that wait\nSeeing is believinghullindsiihhh is always twenty-twentyylnnnly fools and horses work\nFlattery will get you nowhere \nNothing cured must be endured\'d cat may look at king\nño rest for the wicked\nVirton is its own reward\nOnce bitten, twice shy\nHaste makes iaste\nn nod's as good as a wink to a blind horse\nThe best things in life are freeknStrike while the iron is hot AnFom the sublime to the ridiculous is only one steplnDistane lends enchantment to the viewlnTo err is human; to forgive divine nnHe who hesitates is lost can't hold with the hare and run with the hounds poor workman always blames his tools (need cold and starve fever\nFinders keepers, losers weepers \Maae haste slowly AGrret oaks from little acorns grow\nLive for today for tomorrow never comes AnI takes a thief to catch a thief better to light candle than to curse the darkness oThhre''s honour among thieves chieves\uMoney doesn't grow on trees (neewenn two stools one falls to the ground\nGivg credit where credit is due\nThe apple never falls far from the tree\nNothing is certain but death and taxes taxessnnCleanliess what you make it\nThe customer is always right\n0ms hand washes the other\nDon't let the grass grow under your feet\nThe end justifies the means)nFor want of mail the shoe was lost; for want of shoe the horse lost; and for want of horse the man was lost (eestssssioon is nine points of the lawAnAn apple a day keeps the doctor aMarranasss are made in heaven'n.
under the sun\nWhat can't be
weake
next to godliness\nBel news travels fast\nLife
In what follows, we provide short narratives, each of which illustrates a common proverb. AnMarrative: Today was Kim's birthday, and she wanted to celebrate with all her friends. Cindy told she couldn't join them, because she wanted to get some work done. Kim asked if she could please take some fun and celebrate, but Cindy wouldn't. She just talked about all of her future plans that she had to prepare for. Later that night, Cindy saw all of the pictures from Kim's birthday celebration, and she felt sad. None of her future plans were happening any time soon, and in the meantime, she missed the party. AnThis narrative is a good illustration of the following proverb:"
away from work to have
Target: "Live for today for tomorrow never comes"
24

---

Published in Transactions on Machine Learning Research (10/2024)
Task
Train/ Size
Train Max Longth
Test Max Longth
bigbench_arithmenne arithmetic
1000/1000
126 characters
178 characters
generate until
understanding bigbench
biggene abstract
700/200
828 characters
1507 characters
narrative Sarralive_generaleeeeee until
Table 10: BIG-bench tasks.
Testing Length (# letters)
Task
Model
10
20
40
100
200
500
1000(
Llama2-7B 7B
100
99+1
98=1
98+1
95:12
8+1
0=0
Copy
Llama2-7B FT
100
9911
99
9811
9712
31:3
010
PANM (Ours)
100
100=0
100+0
100=0
99=1
39#3
3=0
Llama2-7B
10+1
6
1+1
3+1
5+1
3+1
1=1
D. Recall
Llama2-7B FT
1212
411
010
311
512
212
111
PANM (Ours)
60+7
38#5
18+3
9+1
11=1
5+1
3#1
Table 11: Synthetic Algorithmic Tasks: (5-shot) LLM's mean + std. BLEU accuracy(%) over 5 runs. Training length is 10. (+) In this case, input overflow happens because maximum input length for Llama2 is 4096 tokens.
D.6 Additional Experiments
Generalization to Shorter Sequence Here, explore whether PANM can maintain its performance in long-to-short generalization scenario. We selected a challenging synthetic task, Mix, and trained both PANM and Location Attention (the best baseline for this task) on sequences of length 20-30, testing them on sequences of length 10. The training and testing curves in Fig. 7 (left) reveal that while both models achieve near-perfect training results, only PANM shows smooth learning and signs generalization to the test length of 10, with an improvement margin of nearly 14% compared Location Attention.
Furthermore, in more practical Machine Translation task, we trained both PANM and Transformer (the best baseline for this task) on 50% longer sequences and tested on the remaining shorter sequences using the same 2-layer Transformer backbone. As shown in Fig 7 (right), the Transformer model overfits and fails to generalize to shorter sequences. In contrast, PANM achieves reasonable perplexity on the test data, outperforming the best Transformer result by approximately 25 points.
Pointer Hyperparamoters In this section, we confirm the logic presented in Appendix C by performing experiments that involve varying the number and type of pointers.
Mode-1 Pointers We test the PANM version in 3.1 with H² 0, 1, 3 on Copy, Reverse. We do not use Mode-2 pointer here to avoid confusion (H 0). Fig. g plots the testing accuracy over training time. As Ha 0, there is pointer information for the Controller, PANM should be equivalent to GRU and fail to generalize. As Ha = 1, the only pointer is initialized either with the base end address. As shown in Fig. PANM cannot generalize in both Copy and Reverse tasks with single Mode-1 pointer, which is proved in Appendix C In the caseH = 3, we initialize them with the base, end and middle addresses. We observe that increasing He to slightly reduces the performance these tasks. We speculate that too many Mode-1 pointers make the lcarning harder; in particular, learning to manipulate the third pointer may interfere with that of the first or second pointer, which are more important in these tasks. Generally, most tasks only require list iterations from the head to the tail vice versa. Hence, keep H 2 in all experiments to save the computation cost.
Mode-2 Pointers We fix Ha = 2, and vary Hº = 0, 0, 2 on Copy, Priority Sort, ID Sort. As shown in Fig. without Mode-2 pointers (H = 0), generalization in Priority Sort and ID Sort reduced significantly by 50% and 30%, respectively bocause these tasks focus more on the content of the input sequence and often demand comparing the content of different tokens. Interestingly, content-free task like Copy also suffers from performance drop if there is Mode-2 pointer. Specifically, find out that for 2/5 runs,
25

---

Published in Transactions on Machine Learning Research (10/2024)
Location Attention
PANM
PANM
TRANSFORMER
Train
Train
100
1000
300
100
60
Perplexity
30
Accuracy
10
Test
Test
60
300
40
100
50
20
28
0
20K
40K
0
20
40
60
Steps
Epochs
Figure Long-to-short generalization: Mix (left) and Machine Translation (right) learning curves. The curves are mean + std. over runs.
PANM (1 base Ha) PANM (1 end Ha)
PANM H PANM H*)
PANM (no pointer)
Copy
Reverse
90
80
80
70
Accuracy (%)
60
50
40
40
30
20
10
10K
20K
30K
40K
50K
10K
20K
30K
40K
50K
Step
Step
Figure 8: Testing accuracy (mean + std.) at 2(I+1) length over training steps. Different configurations of Mode-1 pointers are trained and evaluated 5 timess.
26

---

Published in Transactions on Machine Learning Research (10/2024)
PANM (HS=0)
PANM (HS=1)
PANM (HS=2)
Copy
Priorty Sort
Sort
100
80
50
80
60
Accuracy (%)
40
60
50
30
40
30
20
20
10
0
20K
40K
20K
40K
50K
100K
Figure 9: Testing accuracy (mean + std.) at 2(L++1) length training steps. Different configurations of Mode-2 pointers are trained and evaluated 5 times.
the model converges to a suboptimal solution, leading to high variance and slightly lower mean accuracy. Perhaps, Mode-2 pointer allows the docoder to access the input instantly (like content-based attention), avoid forgetting, and thus, improve the prediction as the sequence longer. Having more Mode-2 pointers generally improves the generalization in Copy and Priority Sort, yet the gain is small for H 2, or even negative in ID Sort. Therefore, we trade-off performance with computing efficiency by sctting H 1 in experiments.
D.7 Failures of Chat-GPT in Our Tasks
Large Language Models (LLMs), especially Chat-GPT, have shown remarkable results reasoning and gen- cralization. Directly comparing Chat-GPT with other models used in our experiments would be unfair be- cause Chat-GPT not directly trained with our datasets and it has much more parameters than model. Therefore, in this section, we merely use Chat-GPT simple, are non-trivial. The evaluated tasks are algorithmic reasoning and SCAN. We do not examine Dyck recognition because the output encoding is complicated to represent in text. Other datasets are more common and likely to be used for training Chat-GPT, thus, are not suitable for generalization test. For example, Mathematics task, we ask Chat-GPT the question from the data What the hundreds digit 31253?, it provide the correct answer (2). However, slightly modifying the question to ensure it does not appear in the training and testing set will successfully fool Chat-GPP
a tool to verify that our chosen tasks, despite being
Example 1:
Prompt: What is the hundreds digit 312537?
Chat-GPT answer: The hundreds digit of the mumber 312537 is 2.
Example 2:
-rromp:: What is the hundreds digit 319253?
Chat-GPT answer: The hundreds digit of the number 319253 is 9.
We use Open AT's Chat-GPT 3.5 version September 2023 and evaluate the model on our data using few-shot example prompts, following the format:
Examples:
input at, .... output yt v....
27

---

Published in Transactions on Machine Learning Research (10/2024)
Failure example Chat-GP GPat-GPT Output
Task
Chal-GPT
PANM
Input
True Output
Copy 100%
N/A % % &&8%S%%&
84%
Reverse 69%
8%&&8% a%#
#%o %
81%
Mix
42%
8%&&%% # S(&&S# 09 %
$7
98%
Dynamic Recall 14%
45%
Table 12: Failure of Chat-GPT algorithmic reasoning test cases of length 2L. Token-level accuracy is reported. We do not test Chat-GPT on Priority and ID sort because they have complicated token representations. PANM results cannot be directly compared, and shown for reference only.
input =2, output v2.
Question:
input 21,42 $2:-. output
Algorithmic Reasoning To ensure that Chat-GPT does not memorize the output answer from its vast training data, we use non-digit symbols: 45% & as 10 tokens of the datasets. For each task, we sample training examples of length
to build the in-context examples, and test on longer sequence of length 2L 10. We conduct 20 trials and report the average test accuracy. Table 12 summaries the evaluation results. Overall, except for Copy task where Chat-GPT shows excellent generalization, other tasks are very hard for Chat-GPT, indicating that the length extrapolation problem still poses big challenge to today AI techniques.
In this task, we sample 20 examples in the T-cutoff=40 split set (casiest) as in-context learning examples and evaluate on 10 unseen sequences. Chat-GPT totally failed in this task. When testing on the similar length or longer length as the examples, Chat-GPT cannot produce any exact match results (exact match accuracy=0). Below are some failure examples:
SCAN
IN: walk and turn opposite right OUT:
L Chat-GPT output: I_ TURN RIGHT TURN RIGHT I_ WALK
Truc output: __ WALK I_ TURN RIGHT I TURN RIGHT
IN: run around left twice and run around right OUT:
Chat-GPT output: RUN T LEFT RUN L LEFT R TURN RIGHT I RUN
True output: TURN LEFT I RUN TURN LEFT I RUN TURN LEFT I RUN TURN LEFTI RUNI TURN LEFTI RUN TURN LEFT RUN TURN LEFT
RUN TURN LEFT 1 RUN 1 TURN RIGHT RUN TURN RIGHT I RUN
I TURN RIGHT I RUN I TURN RIGHT I RUN
28

---

Published in Transactions on Machine Learning Research (10/2024)
Copy Task
9(L)
10(L+1)
20((L+1)
40((L+1)*)
80((L+))88)
LSTM
100++0
47+0
11+00
10+0
10+0
Location Attention
100+0
9312
51:5
28:4
20:1
Content Attention
100+0
92#1
5340
33
22:10
Hybrid Attention
100
9111
5011
2343
1320
Transformer
100+0
20+1
16+0
13+0
11+0
NTM
100+0
7414
1312
11:0
11:0
DNC
100+0
5412
11:1
11:0
11:0
Neural Stack
100+0
9044
4712
2910
1720
PtrNet
100+0
90#2
52+1
32#1
20
NRAM
100+00
8143
1542
11:0
11:1
ESBN
100+0
92:00
3440
11:0
11:0
PANM
10040
100:0
84=1
5211
3611
Table 13: Copy: accuracy (mean + std. over 5 runs)
Reverse Task
9(L)
10(L+1)
20((L+1)*2)
40((L+1)*4)
80((L+1)*8)
LSTM
100+0
96:0
53:10
33
22:10
Location Attention
100+0
26113
18:1
14:0
1240
Content Attention
100+0
81=25
38111
23
1642
Hybrid Attention
100+00
98-+1
50+7
24+2
15+1
Transformer
100+0
18:1
1513
1341
11:0
NTM
100+0
95:17
65:27
26=13
1311
DNC
100+0
9313
60118
2316
1211
Neural Stack
100+0
96=11
6444
3513
1911
PtrNet
100+0
7715
3214
2211
1240
NRAM
10010
9611
6013
3312
1542
ESBN
99:10
95:00
1412
11:0
1010
PANM
100
10040
84.33
5111
3311
Table 14: Reverse: accuracy (mean std. over 5 runs)
Mix Task
9(L.)
10(L+1)
20((L+1)*2
40((I++1)*4)
80((L++)**)
LSTM
100+0
96-10
53.0
33:10
22:10
Location Attention
100+0
92=10
56+1
45:0
3016
Content Attention
100+0
61+8
57+1
14+0
12+0
Hybrid Attention
100+0
98
5613
3440
2316
Transformer
100+00
18400
1543
1341
1140
NTM
100+0
95
65127
26-13
1341
DNC
100+0
91#4
5819
1913
11+1
Neural Stack
100+0
8713
50:5
14422
11:00
PtrNet
100+0
5943
5113
1341
11:0
NRAM
99110
8247
4816
1744
1011
ESBN
99-1
95:00
14+2
11:0
1040
PANM
10010
100+0
98-1
5410
54+1
Table 15: Mix: accuracy (mean + std. over runs)
29

---

Published in Transactions on Machine Learning Research (10/2024)
Drecall Task
9(L)
10(L+1)
20((L+1)* 22)
40((L+1))4)
80((++1)*))
LSTM
85+77
74=16
21+2
12+1
11+00
Location Attention
88:11
8241
3013
1942
1310
Content Attention
8842
8440
27
1741
1321
Hybrid Attention
69:25
66=24
28
1942
1321
Transformer
33+1
32++0
22+0
1441
12+1
NTM
8611
72188
22
15:0
1210
DNC
8940
8311
22
1442
11:0
Neural Stack
8514
7612
2311
1541
1321
PtrNet
65+14
4847
25
1441
12#1
NRAM
61:1
5944
21
1342
1111
ESBN
90:11
8611
22
11:1
1020
PANM
92:10
8910
4511
2210
1610
Table 16: Drecall: accuracy (mean + std. over 5 runs)
PSort Task
10(L)
11(L+1)
21((++1)*2)
41((L+1)*4)
81((L+1)*8)
LSTM
8712
8312
2813
16
12+1
Location Attention
6911
6613
4511
27122
20
Content Attention
97:0
96:0
5726
30
2215
Hybrid Attention
85+3
81+1
33+1
25+2
23+3
Transformer
71:99
4848
2113
1614
14:4
NTM
96122
95133
34118
1241
10:0
DNC
9510
92:22
2917
1121
1010
Neural Stack
92:12
7912
32+3
1342
11:1
PirNet
77:12
71122
43
24
1941
NRAM
8213
8042
5112
25
1311
ESBN
26114
2414
1312
11:1
10:1
PANM
97:00
97:1
86:12
3217
27:44
Table 17: PSort: accuracy (mean + std. over runs)
ID Sort Task
10(L.
11(L+1)
21([++1)*2
41((I+1)*4)
(1((L+1)*8)
LSTM
48410
40:5
2011
13411
1111
Location Attention
34+1
3241
20::0
1410
1210
Content Attention
98+1
56+1
28+22
16+0
12+0
Hybrid Attention
32+1
31+1
19+1
1410
1210
Transformer
3442
29-10
1940
1540
1240
NTM
40.23
32417
1614
1242
11:0
DNC
35+1
36+1
23:22
17+2
1340
Neural Stack
3313
32+1
1941
1341
1240
PtrNet
27+1
2411
15:0
1241
11:0
NRAM
3112
29+1
1410
1240
11:0
ESBN
47+18
42+12
18+0
1240
1040
PANM
10010
100+0
56=2
2510
1510
Table 18: ID) Sort: accuracy (mean + std. over 5 runs)
30

---

Published in Transactions on Machine Learning Research (10/2024)
Task
add sub
place value
U. TRM+ RPE
0.97 0.01
0.75 0.10
TRM RPE*
0.91 0.03
TRM RPE
0.91 0.04
0 =
TRM
0.89 = 0.01
0.12 = 0.07
TRM
0.86 0.01
0.05+00.05
U. TRM*
0.94 + 0.01
0.20 + 0.02
PANM TRM base (Ours)
0.91 - 0.01
0.15 = 0.02
PANM TRM + RPE base (Ours) 0.97 + 0.02
0.86 + 0.05
Table 19: Mathematics: mean + std accuracy over 5 runs. are numbers from Csordás et al. 2021 is our rerun to confirm the results, which, in some cases, could not match the reported numbers. means training crash reported in the original papers. We run PANM using the authors' codebase.
Algorithm PANM training. To simplify, the batch size and number of pointer heads are
Input: A dataset of sequence pairs D = (X,, Vette initial containing the parameters of the Encoderg, Pointer Unit PU and Controller Ctrlx, b representing the number of bits of the address space, Late being the maximum number of decoding sleps, and function measuing the length of a sequence.
Ouput: $², trained paramcters. for {Xi, Y.) ~ D do
/* Construct the memory
*/
M Encodera
/* Sample base address. During testing, ag can be set to 0 Uniform ((0, 139)
*/
/* Generate the address do A(j] = (an + f) mod 2
*/
for j == 0, 0, .M..
end
/* Decode with pointers
for t = 0, Ldec do
Use PU, and A to compute pf using Eq. 3
Use M and pt to compute pointer values "ff (Mode 1) and "pf (Mode 2) (see 2.3.2
10
Use Ctr and pointer values to compute pa (ye X,, 24) (see 2.3.3
11
of argmax,, y, Pa (v (X,, 2))
12
if o is EOS then break
13
14
end
end 15
/* Compute cross-entropy loss
L E. X
Use L update & through backpropagation
end
31


Reviews:
[{"review_id": "I8eMxQhp1o", "reviewer": "Reviewer_GYiy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The manuscript is generally well written and easy to follow, and is accompanied by an extensive appendix featuring additional experiments and ablation studies\n- Well justified choice of different symbolic reasoning tasks, as well as the baselines.\n- Significant improvements over baselines, particularly drastic on test sequences that are significantly longer than the ones observed in the training data, clearly addressing the main stated objective of the work.\n\nWeaknesses:\n- While the focus on test sequences longer than the ones observed during training is definitely welcome, it would also be interesting to see performance on test sequences that are shorter than the largest length observed during training. It would be interesting to see if significant performance improvements emerge on those kinds of sequences as well, as in most practical settings, most test sequences would not be longer than the longest train sequence, and a positive result there would point to an even more wide-ranging benefit of using the proposed approach. \n- While I understand the focus on demonstrating the improvements to generalization on a wide range of settings as opposed to outperforming the state-of-the-art on a select few tasks, due to the fact that in the experiments the authors considered highly specific evaluation scenarios with customized train-test splits, it is somewhat difficult to gauge the usefulness of their approach in a more general setting, or compare their approach to other methods aimed at improving systematic generalization.  For instance, due to the previously mentioned focus on longer test sequences required a hand-crafted evaluation setup on multiple tasks, making direct comparison with other works more difficult.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose Pointer-Augmented Neural Memory, which can be integrated into a wide range of neural networks in order to improve generalisation on symbol processing tasks, improve performance of compositional learning and improve scalability and length extrapolation. Their approach is inspired by the use of pointers in computing and as a part of their model architecture, they train networks that learn to manipulate addresses (which refer to the elements of the encoded input sequence) in order to solve the task at hand. \n\nEach element in the input sequence is encoded and serves as data in the memory bank. Each element in the memory (representations of input sequence elements) is assigned a physical address in the address bank, starting from the base address which is uniformly sampled from the address space (such that all possible address ranges are observed during training, avoiding fitting the model based on the values of the addresses in themselves). These addresses are used in subsequent pointer arithmetic in the decoding process. This data in the memory bank, i.e. the encodings of the input sequence elements, is not further altered in the decoding process and is not used to compute the pointers, reflecting the stated principle of manipulating the pointers in a way that is decoupled from input data. \n\nIn each iteration, a GRU computes a new hidden state based on the previous hidden state and the previous pointer. The alignments between the new hidden state and the outputs of the neural network that transforms each of the entries of the dress bank into an equal-dimension space are softmaxed - resulting in a vector of weights for each element in the address bank. The weights are used for computing a weighted sum of the memory addresses in the bank - resulting in a new memory address which represents a combination of physical addresses and which is independent of the length of the sequence. The computed pointer is used in the subsequent iteration of the GRU.\n\nIn the Addressing Mode 1, the pointer obtained from the PU is dereferenced by using the same aforementioned weights to compute a weighted sum of entries in the memory data. While the aspects of Mode 1 are very similar to the classic attention mechanism in the sense that a weighted sum of the encoded input sequence elements is eventually computed, the fact that the weights are computed based on the corresponding addresses in a way that is decoupled from the input data, to the best of my knowledge, makes this approach novel. The pointer unit does not receive the input sequence as input, but learns to manipulate the addresses in a way that it can serve appropriate data for the decoding process. The authors argue that this results in particularly high performance on input sequences that are longer than those observed in training data, which is subsequently confirmed in the experiments.\n\nThe Addressing Mode 2 on the other hand, uses an attention mechanism with the data memory serving as both keys and values, while the pointers computed by the PU are transformed into a query vector using an additional neural network, resulting in an additional pointer value that captures the most relevant context from the data memory for the decoder, which has proven itself as a critical component in content-based tasks that require the token value as well. In each decoding iteration, the Mode 1 and Mode 2 pointer values are concatenated and finally given as input to the decoder.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- From the sentence \"$h^a_0$ is initialized as 0 and $1 ≤ n ≤ l(X)$, $\\varphi$ denotes the parameter of the PU...\" in 2.3.1., it is still not clear how $h^a_0$ is initialised - is it just an all-zero vector and the statement $1 ≤ n ≤ l(X)$ should have been separate?\n- $\\varphi $ denotes the parameter of the PU in 2.3.1. - I assume there are more than one parameters and that $\\varphi$ indicates the parameters of the PU in general?\n- Since the output of the PU ($p^a_t$) is a combination of addresses in the address bank, and not one of the addresses itself, I find the Figure 1 confusing. To my understanding, the $*p^a_t$ will not necessarily (or nearly ever) match any of the entries of the data memory, while in the figure, both the output of the PU and one of the entries in the data memory are indicated as $*p^a_t$ - I believe clarifying this would be important and I'd encourage the authors to potentially correct my understanding or otherwise address the potential confusion in what is otherwise a prominent figure in the paper\n- In Figure 3. - Where the authors show the perplexity on the Multi30K and report \"The best test perplexity over 2 settings for different number of Transformer’s layers\", the perplexity increases sharply already with the addition of the second layer. Given the degree of overfitting that is observed, I'd appreciate a comment on whether their choice of the Transformer architecture could have been in any way disadvantageous for the baseline, and whether simpler techniques like early stopping could have potentially been used to erase the performance gap to a degree.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Y6ApJK4NzL", "reviewer": "Reviewer_fkQp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strength\n\nTo me the biggest strength is the intuitiveness and simplicity of the idea. I love the inspiration in programming languages which really motivates the approach. Moreover, the training is extremely simple, which is in contrast to previous approaches that rely on using variational inference for training the memory module (as a posterior over the memory contents is inferred from the training data).\n\n# Weaknesses\n\nFor me the biggest weakness is in the writing. I absolutely love the motivation and sets up the approach perfectly, but it is a little hard to gather what is going on. For starters, I think a little more hand-holding could be done in the beginning of section 2.2; for instance, I think spelling out the copy-list example would be great. I also think the choice of notation isn't helping here. $p$ (and $p_E$) is a specific pointer but $p_b$ corresponds to the size of the address bank.\n\nNext, while the experimental results clearly demonstrate that PANM outperforms non-memory augmented neural networks, I think comparisons against other memory-augmented NNs are needed, especially given the motivation that the use of PANM specifically allows for the networks to generalize.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors introduce PANM, which is an external memory module that endows a neural network with memory, allowing for generalization. Crucially, the approach relies on the pointer and memory being separate, as opposed to previous works, where the pointer and memory are the same.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Notation changes to make paper more readable (required)\n2. Baselines against other memory-augmented NNs.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "TYCKX792yx", "reviewer": "Reviewer_2qxe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nInnovative Approach: Uses explicit physical addresses and pointer manipulation, improving length extrapolation and systematic generalization.\n\nVersatility: Effective across various models, from simple recurrent networks to large language models.\n\nPerformance Gains: Significant improvements in algorithmic reasoning, language recognition, compositional learning, and NLP tasks.\n\nRobust Memory Access: Implements two robust memory access modes for flexible and reliable data retrieval.\n\n\nWeaknesses:\n\nComplexity: Increased implementation and computational complexity due to explicit pointer operations and memory management.\n\nResource Intensive: Higher computational demands may limit applicability in resource-constrained environments.\n\nTask Specificity: While excellent for symbolic and length extrapolation tasks, the benefits for other task types may be less pronounced.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces Pointer-Augmented Neural Memory (PANM), a novel module that employs explicit physical addresses and pointer manipulation techniques to enhance neural networks' capabilities in handling longer sequences and complex symbolic tasks. This approach significantly improves the ability of models to generalize to sequences much longer than those seen during training. PANM demonstrates its versatility and effectiveness across various sequential models, ranging from simple recurrent networks to large language models like Llama2-7B, showing broad applicability.\n\nThe experimental results indicate substantial performance gains in critical tasks such as algorithmic reasoning, Dyck language recognition, compositional learning (e.g., SCAN, Mathematical Problems), question answering, and machine translation. PANM's robust memory access mechanisms, including pointer dereferencing and relational access, provide flexible and reliable data retrieval from memory, further supporting enhanced systematic generalization. These contributions collectively address key limitations in current neural network models, particularly in systematic generalization and symbolic processing.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Clarify Terminology: Ensure all technical terms are clearly defined when first introduced. For example, define \"<eos>\" (end of sequence) in Section 2.1 (and EOSs in the later sections) to avoid any confusion.\n\nProvide More Intuitive Examples: Can you include more intuitive, real-world examples to explain complex concepts, particularly in the sections describing the Pointer Unit and memory access modes (Sections 2.3.1 and 2.3.2)? This will benefit readers grasp the practical implications of the model.\n\nCompare Baselines More Thoroughly: In the Experimental Results section, provide a more detailed comparison with baseline models (e.g., a table comparing key performance metrics (accuracy, precision, recall) for PANM and baseline models across different tasks).\n\nExpand on Limitations: Offer a more detailed discussion on the limitations of PANM, such as computational complexity and potential challenges in implementation.", "broader_impact_concerns": "No.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses an important problem, takes a novel approach, and provides substantial evidence to substantiate its claims. The reviewers originally had some concerns regarding the experimental setup and the writing, but these have been largely addressed in the revision.
Decision: Accept (poster)