Original: Neural Stored-program Memory
Revised: Neural Stored-program Memory

Original: Anonymous authors

Original: Paper under double-blind review

Original: Abstract
Revised: Abstract

Original: Neural networks powered with external memory simulate computer behaviors.
Revised: Neural networks powered with external memory simulate computer behaviors.

Original: These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks.
Revised: These models, which use the memory to store data for a neural controller, can learn algorithms and other complex tasks.

Original: In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern computer architectures.
Revised: In this paper, we introduce a new memory to store weights for the controller, analogous to the stored-program memory in modern computer architectures.

Original: The proposed model, dubbed Neural Stored-program Memory, augments current memory-augmented neural networks, creating diﬀerentiable machines that can switch programs through time, adapt to variable contexts and thus resemble the Universal Turing Machine.
Revised: The proposed model, dubbed Neural Stored-program Memory, augments current memory-augmented neural networks, creating diﬀerentiable machines that can switch programs through time, adapt to variable contexts and thus resemble the Universal Turing Machine.

Original: A wide range of experiments demonstrate that the resulting machines not only excel in classical algorithmic problems, but also have potential for compositional, continual, few-shot learning and question-answering tasks.
Revised: A wide range of experiments demonstrate that the resulting machines not only excel in classical algorithmic problems, but also have potential for compositional, continual, few-shot learning and question-answering tasks.

Original:  Introduction
Revised:  Introduction

Original: Recurrent Neural Networks (RNNs) are Turing-complete (Siegelmann & Sontag, 1995).
Revised: Recurrent Neural Networks (RNNs) are Turing-complete (Siegelmann & Sontag, 1995).

Original: However, in practice RNNs struggle to learn simple procedures as they lack explicit memory (Graves et al., 2014; Mozer & Das, 1993).
Revised: However, in practice RNNs struggle to learn simple procedures as they lack explicit memory (Graves et al., 2014; Mozer & Das, 1993).

Original: These ﬁndings have sparked a new research direction called Memory Augmented Neural Networks (MANNs) that emulate modern computer behavior by detaching memorization from computation via memory and controller network, respectively.
Revised: These ﬁndings have sparked a new research direction called Memory Augmented Neural Networks (MANNs) that emulate modern computer behavior by detaching memorization from computation via memory and controller network, respectively.

Original: MANNs have demonstrated signiﬁcant improvements over memory-less RNNs in various sequential learning tasks (Graves et al., 2016; Le et al., 2018a; Sukhbaatar et al., 2015).
Revised: MANNs have demonstrated signiﬁcant improvements over memory-less RNNs in various sequential learning tasks (Graves et al., 2016; Le et al., 2018a; Sukhbaatar et al., 2015).

Original: Nonetheless, MANNs have barely simulated general-purpose computers.
Revised: Nonetheless, MANNs have barely simulated general-purpose computers.

Original: Current MANNs miss a key concept in computer design: stored-program memory.
Revised: Current MANNs miss a key concept in computer design: stored-program memory.

Original: The concept has emerged from the idea of Universal Turing Machine (UTM) (Turing, 1936) and  developed in the Von Neumann Architecture(VNA) (von Neumann, 1993).
Revised: The concept has emerged from the idea of Universal Turing Machine (UTM) (Turing, 1936) and further developed in Harvard Architecture (Broesch, 2009), Von Neumann Architecture (von Neumann, 1993).

Original: In UTM/VNA, both data and programs that manipulate the data are stored in memory.
Revised: In UTM, both data and programs that manipulate the data are stored in memory.

Original: A control unit then reads the programs from the memory and executes them with the data.
Revised: A control unit then reads the programs from the memory and executes them with the data.

Original: This mechanism allows ﬂexibility to perform universal computations.
Revised: This mechanism allows ﬂexibility to perform universal computations.

Original: Unfortunately, current MANNs such as Neural Turing Machine (NTM) (Graves et al., 2014), Diﬀerentiable Neural Computer (DNC) (Graves et al., 2016) and Least Recently Used Access (LRUA) (Santoro et al., 2016) only support memory for data and embed a single program into the controller network, which goes against the stored-program memory principle.
Revised: Unfortunately, current MANNs such as Neural Turing Machine (NTM) (Graves et al., 2014), Diﬀerentiable Neural Computer (DNC) (Graves et al., 2016) and Least Recently Used Access (LRUA) (Santoro et al., 2016) only support memory for data and embed a single program into the controller network, which goes against the stored-program memory principle.

Original: Our goal is to advance a step further towards UTM/VNA by coupling a MANN with an external program memory.
Revised: Our goal is to advance a step further towards UTM by coupling a MANN with an external program memory.

Original: The program memory co-exists with the data memory in the MANN, providing more ﬂexibility, reuseability and modularity in learning complicated tasks.
Revised: The program memory co-exists with the data memory in the MANN, providing more ﬂexibility, reuseability and modularity in learning complicated tasks.

Original: The program memory stores the weights of the MANN’s controller network, which are retrieved quickly via a key-value attention mechanism across timesteps yet updated slowly via backpropagation.
Revised: The program memory stores the weights of the MANN’s controller network, which are retrieved quickly via a key-value attention mechanism across timesteps yet updated slowly via backpropagation.

Original: By introducing a meta network to moderate the operations of the program memory, our model, henceforth referred to as Neural Stored-program Memory (NSM), can learn to switch the programs/weights in the controller network appropriately, adapting to diﬀerent functionalities aligning with diﬀerent parts of a sequential task, or diﬀerent tasks in continual and few-shot learning.
Revised: By introducing a meta network to moderate the operations of the program memory, our model, henceforth referred to as Neural Stored-program Memory (NSM), can learn to switch the programs/weights in the controller network appropriately, adapting to diﬀerent functionalities aligning with diﬀerent parts of a sequential task, or diﬀerent tasks in continual and few-shot learning.

Original: To validate our proposal, the NTM armed with NSM, namely Neural Universal Turing
Revised: To validate our proposal, the NTM armed with NSM, namely Neural Universal Turing

Original: Machine (NUTM), is tested on a variety of synthetic tasks including algorithmic tasks from
Revised: Machine (NUTM), is tested on a variety of synthetic tasks including algorithmic tasks from

Original: Graves et al.
Revised: Graves et al.

Original: (2014), composition of algorithmic tasks and continual procedure learning.
Revised: (2014), composition of algorithmic tasks and continual procedure learning.

Original: For these algorithmic problems, we demonstrate clear improvements of NUTM over NTM.
Revised: For these algorithmic problems, we demonstrate clear improvements of NUTM over NTM.

Original: Further, we investigate NUTM in few-shot learning by using LRUA as the MANN and achieve notably better results.
Revised: Further, we investigate NUTM in few-shot learning by using LRUA as the MANN and achieve notably better results.

Original: Finally, we expand NUTM application to linguistic problems by equipping NUTM with DNC core and achieve competitive performances against stateof-the-arts in the bAbI task (Weston et al., 2015).
Revised: Finally, we expand NUTM application to linguistic problems by equipping NUTM with DNC core and achieve competitive performances against stateof-the-arts in the bAbI task (Weston et al., 2015).

Original: Taken together, our study advances neural network simulation of Turing Machines to neural architecture for Universal Turing Machines.
Revised: Taken together, our study advances neural network simulation of Turing Machines to neural architecture for Universal Turing Machines.

Original: This develops a new class of MANNs that can store and query both the weights and data of their own controllers, thereby following the stored-program principle.
Revised: This develops a new class of MANNs that can store and query both the weights and data of their own controllers, thereby following the stored-program principle.

Original: A set of ﬁve diverse experiments demonstrate the computational universality of the approach.
Revised: A set of ﬁve diverse experiments demonstrate the computational universality of the approach.

Original:  Background
Revised:  Background

Original: In this section, we brieﬂy review MANN and its relations to Turing Machines.
Revised: In this section, we brieﬂy review MANN and its relations to Turing Machines.

Original: A MANN consists of a controller network and an external memory M ∈ R N × M , which is a collection of N M -dimensional vectors.
Revised: A MANN consists of a controller network and an external memory M ∈ R N × M , which is a collection of N M -dimensional vectors.

Original: The controller network is responsible for accessing the memory, updating its state and optionally producing output at each timestep.
Revised: The controller network is responsible for accessing the memory, updating its state and optionally producing output at each timestep.

Original: The ﬁrst two functions are executed by an interface network and a state network 1 , respectively.
Revised: The ﬁrst two functions are executed by an interface network and a state network 1 , respectively.

Original: Usually, the interface network is a Feedforward neural network whose input is c t - the output of the state network implemented as RNNs.
Revised: Usually, the interface network is a Feedforward neural network whose input is c t - the output of the state network implemented as RNNs.

Original: Let W c denote the weight of the interface network, then the state update and memory control are as follows,
Revised: Let W c denote the weight of the interface network, then the state update and memory control are as follows,

Original: [Equation]
Revised: [Equation]

Original: where x t and r t − 1 are data from current input and the previous memory read, respectively.
Revised: where x t and r t − 1 are data from current input and the previous memory read, respectively.

Original: The interface vector ξ t then is used to read from and write to the memory M .
Revised: The interface vector ξ t then is used to read from and write to the memory M .

Original: We use a generic notation memory ( ξ t , M ) to represent these memory operations that either update or retrieve read value r t from the memory.
Revised: We use a generic notation memory ( ξ t , M ) to represent these memory operations that either update or retrieve read value r t from the memory.

Original: To support multiple memory accesses per step, the interface network may produce multiple interfaces, also known as control heads.
Revised: To support multiple memory accesses per step, the interface network may produce multiple interfaces, also known as control heads.

Original: Readers are referred to  Graves et al.
Revised: Readers are referred to App. F and Graves et al.

Original: (2014; 2016); Santoro et al.
Revised: (2014; 2016); Santoro et al.

Original: for details of memory read/write examples.
Revised: for details of memory read/write examples.

Original: A deterministic one-tape Turing Machine can be deﬁned by 4-tuple ( Q, Γ , δ, q 0 ), in which Q is ﬁnite set of states, q 0 ∈ Q is an initial state, Γ is ﬁnite set of symbol stored in the tape (the data) and δ is the transition function (the program), δ : Q × Γ → Γ × {− 1 , 1 } × Q .
Revised: A deterministic one-tape Turing Machine can be deﬁned by 4-tuple ( Q, Γ , δ, q 0 ), in which Q is ﬁnite set of states, q 0 ∈ Q is an initial state, Γ is ﬁnite set of symbol stored in the tape (the data) and δ is the transition function (the program), δ : Q × Γ → Γ × {− 1 , 1 } × Q .

Original: At each step, the machine performs the transition function, which takes the current state and the read value from the tape as inputs and outputs actions including writing new values, moving tape head to new location (left/right) and jumping to another state.
Revised: At each step, the machine performs the transition function, which takes the current state and the read value from the tape as inputs and outputs actions including writing new values, moving tape head to new location (left/right) and jumping to another state.

Original: Roughly mapping to current MANNs, Q , Γ and δ map to the set of the controller states, the read values and the controller network, respectively.
Revised: Roughly mapping to current MANNs, Q , Γ and δ map to the set of the controller states, the read values and the controller network, respectively.

Original: Further, the function δ can be factorized into two sub functions: Q × Γ → Γ × {− 1 , 1 } and Q × Γ → Q , which correspond to the interface and state networks, respectively.
Revised: Further, the function δ can be factorized into two sub functions: Q × Γ → Γ × {− 1 , 1 } and Q × Γ → Q , which correspond to the interface and state networks, respectively.

Original: By encoding a Turing Machine into the tape, one can build a UTM that simulates the encoded machine (Turing, 1936).
Revised: By encoding a Turing Machine into the tape, one can build a UTM that simulates the encoded machine (Turing, 1936).

Original: The transition function δ u of the UTM queries the encoded
Revised: The transition function δ u of the UTM queries the encoded

Original: Turing Machine that solves the considering task.
Revised: Turing Machine that solves the considering task.

Original: Amongst 4 tuples, δ is the most important and hence uses most of the encoding bits.
Revised: Amongst 4 tuples, δ is the most important and hence uses most of the encoding bits.

Original: In other words, if we assume that the space of
Revised: In other words, if we assume that the space of

Original: Q , Γ and q 0 are shared amongst Turing Machines, we can simulate any Turing Machine by encoding only its transition function δ .
Revised: Q , Γ and q 0 are shared amongst Turing Machines, we can simulate any Turing Machine by encoding only its transition function δ .

Original: Translating to neural language, if we can store the controller network into a queriable memory and make use of it, we can build a Neural
Revised: Translating to neural language, if we can store the controller network into a queriable memory and make use of it, we can build a Neural

Original: Universal Turing Machine.
Revised: Universal Turing Machine.

Original: Using NSM is a simple way to achieve this goal, which we introduce in the subsequent section.
Revised: Using NSM is a simple way to achieve this goal, which we introduce in the subsequent section.

Original:  Methods 3. Neural Stored-program Memory
Revised:  Methods 3. Neural Stored-program Memory

Original: A Neural Stored-program Memory (NSM) is a key-value memory M p ∈ R P × ( K + S ) , whose values are the basis weights of another neural network − the programs.
Revised: A Neural Stored-program Memory (NSM) is a key-value memory M p ∈ R P × ( K + S ) , whose values are the basis weights of another neural network − the programs.

Original: P , K , and S are the number of programs, the key space dimension and the program size, respectively.
Revised: P , K , and S are the number of programs, the key space dimension and the program size, respectively.

Original: This concept is a hybrid between the traditional slow-weight and fast-weight (Hinton & Plaut, 1987).
Revised: This concept is a hybrid between the traditional slow-weight and fast-weight (Hinton & Plaut, 1987).

Original: Like slow-weight, the keys and values in NSM are updated gradually by backpropagation.
Revised: Like slow-weight, the keys and values in NSM are updated gradually by backpropagation.

Original: However, the values are dynamically interpolated to produce the working weight on-the-ﬂy during the processing of a sequence, which resembles fast-weight computation.
Revised: However, the values are dynamically interpolated to produce the working weight on-the-ﬂy during the processing of a sequence, which resembles fast-weight computation.

Original: Let us denote M p ( i ) .k
Revised: Let us denote M p ( i ) .k

Original: and M p ( i ) .v
Revised: and M p ( i ) .v

Original: as the key and the program of the i -th memory slot.
Revised: as the key and the program of the i -th memory slot.

Original: At timestep t, given a query key k pt , the working program is retrieved as follows,
Revised: At timestep t, given a query key k pt , the working program is retrieved as follows,

Original: [Equation]
Revised: [Equation]

Original: where D ( · ) is cosine similarity and β pt is the scalar program strength parameter.
Revised: where D ( · ) is cosine similarity and β pt is the scalar program strength parameter.

Original: The vector working program p t is then reshaped to its matrix form and ready to be used as the weight of other neural networks.
Revised: The vector working program p t is then reshaped to its matrix form and ready to be used as the weight of other neural networks.

Original: The key-value design is essential for convenient memory access as the size of the program stored in M p can be millions of dimensions and thus, direct content-based addressing as in Graves et al.
Revised: The key-value design is essential for convenient memory access as the size of the program stored in M p can be millions of dimensions and thus, direct content-based addressing as in Graves et al.

Original: (2014; 2016); Santoro et al.
Revised: (2014; 2016); Santoro et al.

Original: (2016) is infeasible.
Revised: (2016) is infeasible.

Original: More importantly, we can inject external control on the behavior of the memory by imposing constraints on the key space.
Revised: More importantly, we can inject external control on the behavior of the memory by imposing constraints on the key space.

Original: For example, program collapse will happen when the keys stored in the memory stay close to each other.
Revised: For example, program collapse will happen when the keys stored in the memory stay close to each other.

Original: When this happens, p t is a balanced mixture of all programs regardless of the query key and thus having multiple programs is useless.
Revised: When this happens, p t is a balanced mixture of all programs regardless of the query key and thus having multiple programs is useless.

Original: We can avoid this phenomenon by minimizing a regularization loss deﬁned as the following,
Revised: We can avoid this phenomenon by minimizing a regularization loss deﬁned as the following,

Original: [Equation]
Revised: [Equation]

Original: 3. Neural Universal Turing Machine
Revised: 3. Neural Universal Turing Machine

Original: It turns out that the combination of MANN and NSM approximates a Universal Turing
Revised: It turns out that the combination of MANN and NSM approximates a Universal Turing

Original: Machine (Sec. 2).
Revised: Machine (Sec. 2).

Original: At each timestep, the controller in MANN reads its state and memory to generate control signal to the memory via the interface network W c , then updates its state using the state network RNN .
Revised: At each timestep, the controller in MANN reads its state and memory to generate control signal to the memory via the interface network W c , then updates its state using the state network RNN .

Original: Since the parameters of RNN and W c represent the encoding of δ , we should store both into NSM to completely encode an MANN.
Revised: Since the parameters of RNN and W c represent the encoding of δ , we should store both into NSM to completely encode an MANN.

Original: For simplicity, in this paper, we only use NSM to store W c , which is equivalent to the Universal
Revised: For simplicity, in this paper, we only use NSM to store W c , which is equivalent to the Universal

Original: Turing Machine that can simulate any one-state Turing Machine.
Revised: Turing Machine that can simulate any one-state Turing Machine.

Original: In traditional MANN, W c is constant across timesteps and only updated slowly during training, typically through backpropagation.
Revised: In traditional MANN, W c is constant across timesteps and only updated slowly during training, typically through backpropagation.

Original: In our design, we compute W ct from NSM for every timestep and thus, we need a program interface network − the meta network P I − that generates an interface vector for the program memory: ξ p t =
Revised: In our design, we compute W ct from NSM for every timestep and thus, we need a program interface network − the meta network P I − that generates an interface vector for the program memory: ξ p t =

Original: P I ( c t ), where ξ pt =
Revised: P I ( c t ), where ξ pt =

Original: [ k p t , β pt ].
Revised: [ k p t , β pt ].

Original: Together with the RNN , P I simulates δ u of the UTM and is implemented as a Feedforward neural network.
Revised: Together with the RNN , P I simulates δ u of the UTM and is implemented as a Feedforward neural network.

Original: The procedure for computing W ct is executed by following Eqs.
Revised: The procedure for computing W ct is executed by following Eqs.

Original: (3)-(5), hereafter referred to as NSM ( ξ pt , M p ).
Revised: (3)-(5), hereafter referred to as NSM ( ξ pt , M p ).

Original: Figure 1 depicts the integration of NSM into MANN.
Revised: Figure 1 depicts the integration of NSM into MANN.

Original: In this implementation, key-value NSM oﬀers a more ﬂexible learning scheme than direct attention, in which the meta-network can generate the weight w pt directly without matching k pt with M p ( i ) .k .
Revised: In this implementation, key-value NSM oﬀers a more ﬂexible learning scheme than direct attention, in which the meta-network can generate the weight w pt directly without matching k pt with M p ( i ) .k .

Original: That is, only the meta-network learns the mapping from context c t to program.
Revised: That is, only the meta-network learns the mapping from context c t to program.

Original: When it falls into some local-minima (generating suboptimal w pt ), the metanetwork struggles to escape.
Revised: When it falls into some local-minima (generating suboptimal w pt ), the metanetwork struggles to escape.

Original: In our proposal, together with the meta-network, the memory keys are learnable.
Revised: In our proposal, together with the meta-network, the memory keys are learnable.

Original: When the memory keys are slowly updated, the meta-network will shift its query key generation to match the new memory keys and possibly escape from the local-minima.
Revised: When the memory keys are slowly updated, the meta-network will shift its query key generation to match the new memory keys and possibly escape from the local-minima.

Original: For the case of multi-head NTM, we implement one NSM per control head and name this model Neural Universal Turing Machine (NUTM).
Revised: For the case of multi-head NTM, we implement one NSM per control head and name this model Neural Universal Turing Machine (NUTM).

Original: One NSM per head is to ensure programs for one head do not interfere with other heads and thus, encourage functionality separation amongst heads.
Revised: One NSM per head is to ensure programs for one head do not interfere with other heads and thus, encourage functionality separation amongst heads.

Original: Each control head will read from (for read head) or write to (for write head) the data memory M via memory ( ξ t , M ) as described in Graves et al.
Revised: Each control head will read from (for read head) or write to (for write head) the data memory M via memory ( ξ t , M ) as described in Graves et al.

Original: It should be noted that using multiple heads is unlike using multiple controllers per head.
Revised: It should be noted that using multiple heads is unlike using multiple controllers per head.

Original: The former increases the number of accesses to the data memory at each timestep and employs a ﬁxed controller to compute multiple heads, which may improve capacity yet does not enable adaptability.
Revised: The former increases the number of accesses to the data memory at each timestep and employs a ﬁxed controller to compute multiple heads, which may improve capacity yet does not enable adaptability.

Original: On the contrary, the latter varies the property of each memory access across timesteps by switching the controllers and thus potential for adaptation.
Revised: On the contrary, the latter varies the property of each memory access across timesteps by switching the controllers and thus potential for adaptation.

Original: Other MANNs such as DNC (Graves et al., 2016) and LRUA (Santoro et al., 2016) can be armed with NSM in this manner.
Revised: Other MANNs such as DNC (Graves et al., 2016) and LRUA (Santoro et al., 2016) can be armed with NSM in this manner.

Original: We also employ the regularization loss l p to prevent the programs from collapsing, resulting in a ﬁnal loss as follows,
Revised: We also employ the regularization loss l p to prevent the programs from collapsing, resulting in a ﬁnal loss as follows,

Original: [Equation]
Revised: [Equation]

Original: where Loss pred is the prediction loss and η t is annealing factor, reducing as the training step increases.
Revised: where Loss pred is the prediction loss and η t is annealing factor, reducing as the training step increases.

Original: The details of NUTM operations are presented in Algorithm 1.
Revised: The details of NUTM operations are presented in Algorithm 1.

Original: 3. On the Benefit of NSM to MANN: An Explanation from Multilevel Modeling
Revised: 3. On the Benefit of NSM to MANN: An Explanation from Multilevel Modeling

Original: Learning to access memory is a multi-dimensional regression problem.
Revised: Learning to access memory is a multi-dimensional regression problem.

Original: Given the input c t , which is derived from the state h t of the controller, the aim is to generate a correct interface vector ξ t via optimizing the interface network.
Revised: Given the input c t , which is derived from the state h t of the controller, the aim is to generate a correct interface vector ξ t via optimizing the interface network.

Original: Instead of searching for one transformation that maps the whole space of c t to the optimal space of ξ t , NSM ﬁrst partitions the space 
Revised: Instead of searching for one transformation that maps the whole space of c t to the optimal space of ξ t , NSM ﬁrst partitions the space of c t into subspaces, then ﬁnds multiple transformations, each of which covers subspace of

Original: Require: a sequence x = { x t } Tt =1 , a data memory M and R program memories { M p,n } Rn = corresponding to R control heads 1: Initilize h 0 , r2: for t = 1 , T do 3: h t , c t = RNN ([ x t , r t − 1 ] , h t − 1 ) (cid:46) RNN can be replaced by GRU/LSTM 4: for n = 1 , R do 5:
Revised: Require: a sequence x = { x t } Tt =1 , a data memory M and R program memories { M p,n } Rn = corresponding to R control heads 1: Initilize h 0 , r2: for t = 1 , T do 3: h t , c t = RNN ([ x t , r t − 1 ] , h t − 1 ) (cid:46) RNN can be replaced by GRU/LSTM 4: for n = 1 , R do 5:

Original: Compute the program interface ξ pt,n ← P I ,n ( c t ) 6:
Revised: Compute the program interface ξ pt,n ← P I ,n ( c t ) 6:

Original: Compute the program W ct,n ← NSM (cid:0) ξ pt,n , M p,n (cid:1) 7:
Revised: Compute the program W ct,n ← NSM (cid:0) ξ pt,n , M p,n (cid:1) 7:

Original: Compute the data interface ξ t,n ← c t W ct,n 8:
Revised: Compute the data interface ξ t,n ← c t W ct,n 8:

Original: Read r t,n from memory M
Revised: Read r t,n from memory M

Original: (if read head) or update memory M (if write head) using memory n ( ξ t,n , M ) 9: end for 10: r t ← [ r t, 1 , ..., r t,R ] 11: end forof c t into subspaces, then ﬁnds multiple transformations, each of which covers subspace of c t .
Revised: (if read head) or update memory M (if write head) using memory n ( ξ t,n , M ) 9: end for 10: r t ← [ r t, 1 , ..., r t,R ] 11: end for c t .

Original: The program interface network P I is a meta learner that routes c t to the appropriate transformation, which then maps c t to the ξ t space.
Revised: The program interface network P I is a meta learner that routes c t to the appropriate transformation, which then maps c t to the ξ t space.

Original: This is analogous to multilevel regression in statistics (Andrew Gelman, 2006).
Revised: This is analogous to multilevel regression in statistics (Andrew Gelman, 2006).

Original: Practical studies have shown that multilevel regression is better than ordinary regression if the input is clustered (Cohen et al., 2014; Huang, 2018).
Revised: Practical studies have shown that multilevel regression is better than ordinary regression if the input is clustered (Cohen et al., 2014; Huang, 2018).

Original: RNNs have the capacity to learn to perform ﬁnite state computations (Casey, 1996;
Revised: RNNs have the capacity to learn to perform ﬁnite state computations (Casey, 1996;

Original: Tiˇno et al., 1998).
Revised: Tiˇno et al., 1998).

Original: The states of a RNN must be grouped into partitions representing the states of the generating automaton.
Revised: The states of a RNN must be grouped into partitions representing the states of the generating automaton.

Original: As Turing Machines are ﬁnite state automata augmented with an external memory tape, we expect MANN, if learnt well, will organize its state space clustered in a way to reﬂect the states of the emulated Turing Machine.
Revised: As Turing Machines are ﬁnite state automata augmented with an external memory tape, we expect MANN, if learnt well, will organize its state space clustered in a way to reﬂect the states of the emulated Turing Machine.

Original: That is, h t as well as c t should be clustered.
Revised: That is, h t as well as c t should be clustered.

Original: We realize that NSM helps NTM learn better clusterization over this space (see App.
Revised: We realize that NSM helps NTM learn better clusterization over this space (see App.

Original: A), thereby improving NTM’s performances.
Revised: A), thereby improving NTM’s performances.

Original:  Results 4. NTM Single Tasks
Revised:  Results 4. NTM Single Tasks

Original: In this section, we investigate the performance of NUTM on algorithmic tasks introduced in Graves et al.
Revised: In this section, we investigate the performance of NUTM on algorithmic tasks introduced in Graves et al.

Original: Copy, Repeat Copy, Associative Recall, Dynamic N-Grams and
Revised: Copy, Repeat Copy, Associative Recall, Dynamic N-Grams and

Original: [Table]

Original: Priority Sort.
Revised: Priority Sort.

Original: Besides these ﬁve NTM tasks, we add another task named Long Copy which  doubles the length of training sequences in the Copy task.
Revised: Besides these ﬁve NTM tasks, we add another task named Long Copy which [Table] doubles the length of training sequences in the Copy task.

Original: In these tasks, the model will be fed a sequence of input items and is required to infer a sequence of output items.
Revised: In these tasks, the model will be fed a sequence of input items and is required to infer a sequence of output items.

Original: Each item is represented by a binary vector.
Revised: Each item is represented by a binary vector.

Original: In the experiment, we compare two models: NTM 2 and NUTM with two programs.
Revised: In the experiment, we compare two models: NTM 2 and NUTM with two programs.

Original: Although the tasks are atomic, we argue that there should be at least two memory manipulation schemes across timesteps, one for encoding the inputs to the memory and another for decoding the output from the memory.
Revised: Although the tasks are atomic, we argue that there should be at least two memory manipulation schemes across timesteps, one for encoding the inputs to the memory and another for decoding the output from the memory.

Original: The two models are trained with cross-entropy objective function under the same setting as in Graves et al.
Revised: The two models are trained with cross-entropy objective function under the same setting as in Graves et al.

Original: For fair comparison, the controller hidden dimension of NUTM is set smaller to make the total number of parameters of NUTM equivalent to that of NTM.
Revised: For fair comparison, the controller hidden dimension of NUTM is set smaller to make the total number of parameters of NUTM equivalent to that of NTM.

Original: The number of memory heads for both models are always equal and set to the same value as in the original paper (details in App. C).
Revised: The number of memory heads for both models are always equal and set to the same value as in the original paper (details in App. C).

Original: We run each experiments ﬁve times and report the mean with error bars of training losses for
Revised: We run each experiments ﬁve times and report the mean with error bars of training losses for

Original: NTM tasks in Fig.
Revised: NTM tasks in Fig.

Original: 2 (a).
Revised: 2 (a).

Original: Except for the Copy task, which is too simple, other tasks observe convergence speed improvement of NUTM over that of NTM, thereby validating the beneﬁt of using two programs across timesteps even for the single task setting.
Revised: Except for the Copy task, which is too simple, other tasks observe convergence speed improvement of NUTM over that of NTM, thereby validating the beneﬁt of using two programs across timesteps even for the single task setting.

Original: NUTM requires fewer training samples to converge and it generalizes better to unseen sequences that are longer than training sequences.
Revised: NUTM requires fewer training samples to converge and it generalizes better to unseen sequences that are longer than training sequences.

Original: Table 1 reports the test results of the best models chosen after ﬁve runs and conﬁrms the outperformance of NUTM over NTM for generalization.
Revised: Table 1 reports the test results of the best models chosen after ﬁve runs and conﬁrms the outperformance of NUTM over NTM for generalization.

Original: To illustrate the program usage, we plot NUTM’s program distributions across timesteps for Repeat Copy and Priority Sort in Fig.
Revised: To illustrate the program usage, we plot NUTM’s program distributions across timesteps for Repeat Copy and Priority Sort in Fig.

Original: 3 (a) and (b), respectively.
Revised: 3 (a) and (b), respectively.

Original: We observe two program usage patterns corresponding to the encoding and decoding phases.For Repeat
Revised: Examining the read head for Repeat Copy, we observe two program usage patterns corresponding to the encoding and decoding phases.

Original: Copy, there is no reading in encoding and thus, NUTM assigns the “no-read” strategy mainly to the “orange program”.
Revised: As there is no reading in encoding, NUTM assigns the “no-read” strategy mainly to the “orange program”.

Original: In decoding, the sequential reading is mostly done by the “blue program” with some contributions from the “orange program” when resetting reading head.
Revised: In decoding, the sequential reading is mostly done by the “blue program” with some contributions from the “orange program” when resetting reading head.

Original: For Priority Sort, while the encoding “ﬁtting writing” (see Graves et al.
Revised: Similar behaviors can be found in the write head for Priority Sort. While the encoding “ﬁtting writing” (see Graves et al.

Original: (2014) for explanation on the strategy) is often executed by the “blue program”, the decoding writing is completely taken by the “orange” program (more visualizations in App. B).
Revised: (2014) for explanation on the strategy) is often executed by the “blue program”, the decoding writing is completely taken by the “orange” program (more visualizations in App. B).

Original: 4. Ablation study on Associative Recall
Revised: 4. Ablation study on Associative Recall

Original: In this section, we conduct an ablation study on Associative Recall (AR) to validate the beneﬁt of proposed components that constitute NSM.
Revised: In this section, we conduct an ablation study on Associative Recall (AR) to validate the beneﬁt of proposed components that constitute NSM.

Original: We run the task with three additional baselines:
Revised: We run the task with three additional baselines:

Original: NUTM using direct attention (DA), NUTM using key-value without regularization (KV), NUTM using ﬁxed, uniform program distribution (UP) and a vanilla NTM with 2 memory heads ( h = 2).
Revised: NUTM using direct attention (DA), NUTM using key-value without regularization (KV), NUTM using ﬁxed, uniform program distribution (UP) and a vanilla NTM with 2 memory heads ( h = 2).

Original: The meta-network P I in DA generates the attention weight w pt directly.
Revised: The meta-network P I in DA generates the attention weight w pt directly.

Original: The KV employs key-value attention yet excludes the regularization loss presented in
Revised: The KV employs key-value attention yet excludes the regularization loss presented in

Original: Eq.
Revised: Eq.

Original: The training curves over 5 runs are plotted in Fig.
Revised: The training curves over 5 runs are plotted in Fig.

Original: 2 (b).
Revised: 2 (b).

Original: The results demonstrate that DA exhibits fast yet shallow convergence.
Revised: The results demonstrate that DA exhibits fast yet shallow convergence.

Original: It tends to fall into local minima, which ﬁnally fails to reach zero loss.
Revised: It tends to fall into local minima, which ﬁnally fails to reach zero loss.

Original: Key-value attention helps NUTM converge completely with fewer iterations.
Revised: Key-value attention helps NUTM converge completely with fewer iterations.

Original: The performance is further improved with the proposed regularization loss.
Revised: The performance is further improved with the proposed regularization loss.

Original: UP underperforms NUTM as it lacks dynamic programs.
Revised: UP underperforms NUTM as it lacks dynamic programs.

Original: The NTM with 2 heads shows slightly better convergence compared to the NTM, yet obviously underperforms NUTM ( p = 2) with 1 head and fewer parameters.
Revised: The NTM with 2 heads shows slightly better convergence compared to the NTM, yet obviously underperforms NUTM ( p = 2) with 1 head and fewer parameters.

Original: This validates our argument on the diﬀerence between using multiple heads and multiple programs (Sec. 3.2).
Revised: This validates our argument on the diﬀerence between using multiple heads and multiple programs (Sec. 3.2).

Original: 4. NTM Sequencing Tasks
Revised: 4. NTM Sequencing Tasks

Original: In neuroscience, sequencing tasks test the ability to remember a series of tasks and switch tasks alternatively (Blumenfeld, 2010).
Revised: In neuroscience, sequencing tasks test the ability to remember a series of tasks and switch tasks alternatively (Blumenfeld, 2010).

Original: A dysfunctional brain may have diﬃculty in changing from one task to the next and get stuck in its preferred task (perseveration phenomenon).
Revised: A dysfunctional brain may have diﬃculty in changing from one task to the next and get stuck in its preferred task (perseveration phenomenon).

Original: To analyze this problem in NTM, we propose a new set of experiments in which a task is generated by sequencing a list of subtasks.
Revised: To analyze this problem in NTM, we propose a new set of experiments in which a task is generated by sequencing a list of subtasks.

Original: The set of subtasks is chosen from the NTM single tasks (excluding Dynamic N-grams for format discrepancy) and the order of subtasks in the sequence is dictated by an indicator vector put at the beginning of the sequence.
Revised: The set of subtasks is chosen from the NTM single tasks (excluding Dynamic N-grams for format discrepancy) and the order of subtasks in the sequence is dictated by an indicator vector put at the beginning of the sequence.

Original: Amongst possible combinations of subtasks, we choose { Copy, Repeat Copy } (C+RC), { Copy, Associative Recall } (C+AR), { Copy, Priority Sort } (C+PS) and all (C+RC+AC+PS) 3 .
Revised: Amongst possible combinations of subtasks, we choose { Copy, Repeat Copy } (C+RC), { Copy, Associative Recall } (C+AR), { Copy, Priority Sort } (C+PS) and all (C+RC+AC+PS) 3 .

Original: The learner observes the order indicator followed by a sequence of subtasks’ input items and is requested to consecutively produce the output items of each subtasks.
Revised: The learner observes the order indicator followed by a sequence of subtasks’ input items and is requested to consecutively produce the output items of each subtasks.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 4, some tasks such as Copy and Associative Recall, which are easy to solve if trained separately, become unsolvable by NTM when sequenced together.
Revised: 4, some tasks such as Copy and Associative Recall, which are easy to solve if trained separately, become unsolvable by NTM when sequenced together.

Original: One reason is NTM fails to change the memory access behavior (perseveration).
Revised: One reason is NTM fails to change the memory access behavior (perseveration).

Original: For examples, NTM keeps following repeat copy reading strategy for all timesteps in C+RC task (Fig. 

Revised: For examples, NTM keeps following repeat copy reading strategy for all timesteps in C+RC task (Fig.

Original: (d)).
Revised: (d)).

Original: Meanwhile, NUTM can learn to change program distribution when a new subtask appears in the sequence and thus ensure diﬀerent accessing strategy per subtask (Fig.
Revised: Meanwhile, NUTM can learn to change program distribution when a new subtask appears in the sequence and thus ensure diﬀerent accessing strategy per subtask (Fig.

Original: 3 (c)).
Revised: 3 (c)).

Original: 4. Continual Procedure Learning
Revised: 4. Continual Procedure Learning

Original: In continual learning, catastrophic forgetting happens when a neural network quickly forgets previously acquired skills upon learning new skills (French, 1999).
Revised: In continual learning, catastrophic forgetting happens when a neural network quickly forgets previously acquired skills upon learning new skills (French, 1999).

Original: In this section, we prove the versatility of NSM by showing that a naive application of NSM without much modiﬁcation can help NTM to mitigate catastrophic forgetting.
Revised: In this section, we prove the versatility of NSM by showing that a naive application of NSM without much modiﬁcation can help NTM to mitigate catastrophic forgetting.

Original: We design an experiment similar to the Split MNIST (Zenke et al., 2017) to investigate whether NSM can improve
Revised: We design an experiment similar to the Split MNIST (Zenke et al., 2017) to investigate whether NSM can improve

Original: NTM’s performance.
Revised: NTM’s performance.

Original: In our experiment, we let the models see the training data from the 4 tasks:
Revised: In our experiment, we let the models see the training data from the 4 tasks:

Original: Copy (C), Repeat Copy (RC), Associative Recall (AR) and Priority Sort (PS), consecutively in this order.
Revised: Copy (C), Repeat Copy (RC), Associative Recall (AR) and Priority Sort (PS), consecutively in this order.

Original: Each task is trained in 20,000 iterations with batch size 16 (see
Revised: Each task is trained in 20,000 iterations with batch size 16 (see

Original: App. C for task details).
Revised: App. C for task details).

Original: To encourage NUTM to spend exactly one program per task while freezing others, we force “hard” attention over the programs by replacing the softmax function in Eq.
Revised: To encourage NUTM to spend exactly one program per task while freezing others, we force “hard” attention over the programs by replacing the softmax function in Eq.

Original: 5 with the Gumbel-softmax (Jang et al., 2016).
Revised: 5 with the Gumbel-softmax (Jang et al., 2016).

Original: Also, to ignore catastrophic forgetting in the state network, we use Feedforward controllers in the two baselines.
Revised: Also, to ignore catastrophic forgetting in the state network, we use Feedforward controllers in the two baselines.

Original: After ﬁnishing one task, we evaluate the bit accuracy − measured by 1 − (bit error per sequence/total bits per sequence) over 4 tasks.
Revised: After ﬁnishing one task, we evaluate the bit accuracy − measured by 1 − (bit error per sequence/total bits per sequence) over 4 tasks.

Original: As shown in in Fig.
Revised: As shown in in Fig.

Original: 5, NUTM outperforms NTM by a moderate margin (10-40% per task).
Revised: 5, NUTM outperforms NTM by a moderate margin (10-40% per task).

Original: Although NUTM also experiences catastrophic forgetting, it somehow preserves some memories of previous tasks.
Revised: Although NUTM also experiences catastrophic forgetting, it somehow preserves some memories of previous tasks.

Original: Especially,
Revised: Especially,

Original: NUTM keeps performing perfectly on Copy even after it learns Repeat Copy.
Revised: NUTM keeps performing perfectly on Copy even after it learns Repeat Copy.

Original: For other dissimilar task transitions, the performance drops signiﬁcantly, which requires more eﬀort to bring NSM to continual learning.
Revised: For other dissimilar task transitions, the performance drops signiﬁcantly, which requires more eﬀort to bring NSM to continual learning.

Original: 4. Few-shot Learning
Revised: 4. Few-shot Learning

Original: Few-shot learning or meta learning tests the ability to rapidly adapt within a task while gradually capturing the way the task structure varies (Thrun, 1998).
Revised: Few-shot learning or meta learning tests the ability to rapidly adapt within a task while gradually capturing the way the task structure varies (Thrun, 1998).

Original: By storing sampleclass bindings, MANNs are capable of classifying new data after seeing only few samples (Santoro et al., 2016).
Revised: By storing sampleclass bindings, MANNs are capable of classifying new data after seeing only few samples (Santoro et al., 2016).

Original: As NSM gives ﬂexible memory controls, it makes MANN more adaptive to changes and thus perform better in this setting.
Revised: As NSM gives ﬂexible memory controls, it makes MANN more adaptive to changes and thus perform better in this setting.

Original: To verify that, we apply NSM to the LRUA memory and follow the experiments introduced in Santoro et al.
Revised: To verify that, we apply NSM to the LRUA memory and follow the experiments introduced in Santoro et al.

Original: (2016) , using the Omniglot dataset to measure few-shot classiﬁcation accuracy.
Revised: (2016) , using the Omniglot dataset to measure few-shot classiﬁcation accuracy.

Original: The dataset includes images of 1623 characters, with 20 examples of each character.
Revised: The dataset includes images of 1623 characters, with 20 examples of each character.

Original: During training, a sequence (episode) of images are randomly selected from C classes of characters in the training set (1200 characters), where C = 5 , 10 corresponding to sequence length of 50, 75, respectively.
Revised: During training, a sequence (episode) of images are randomly selected from C classes of characters in the training set (1200 characters), where C = 5 , 10 corresponding to sequence length of 50, 75, respectively.

Original: Each class is assigned a random label which shuﬄes between episodes and is revealed to the models after each prediction.
Revised: Each class is assigned a random label which shuﬄes between episodes and is revealed to the models after each prediction.

Original: After 100,000 episodes of training, the models are tested with unseen images from the testing set (423 characters).
Revised: After 100,000 episodes of training, the models are tested with unseen images from the testing set (423 characters).

Original: The two baselines are MANN and
Revised: The two baselines are MANN and

Original: NUTM (both use LRUA core).
Revised: NUTM (both use LRUA core).

Original: For NUTM, we only tune p and pick the best values: p =and p = 3 for 5 classes and 10 classes, respectively.[Table]
Revised: For NUTM, we only tune p and pick the best values: p =and p = 3 for 5 classes and 10 classes, respectively.

Original: [Table] Table 11 reports the classiﬁcation accuracy when the models see characters for the second, third and ﬁfth time.
Revised: Table 11 reports the classiﬁcation accuracy when the models see characters for the second, third and ﬁfth time.

Original: NUTM generally achieves better results than MANN, especially when the number of classes increases, demanding more adaptation within an episode.
Revised: NUTM generally achieves better results than MANN, especially when the number of classes increases, demanding more adaptation within an episode.

Original: For the  persistent memory mode, which demands fast forgetting old experiences in previous episodes,
Revised: For the [Table] [Table] persistent memory mode, which demands fast forgetting old experiences in previous episodes,

Original: NUTM outperforms MANN signiﬁcantly (10-20%) 4 .
Revised: NUTM outperforms MANN signiﬁcantly (10-20%) 4 .

Original: Readers are referred to App.
Revised: Readers are referred to App.

Original: D for more details on learning curves and more results of the models.
Revised: D for more details on learning curves and more results of the models.

Original: 4. Text Question Answering
Revised: 4. Text Question Answering

Original: Reading comprehension typically involves an iterative process of multiple actions such as reading the story, reading the question, outputting the answers and other implicit reasoning steps (Weston et al., 2015).
Revised: Reading comprehension typically involves an iterative process of multiple actions such as reading the story, reading the question, outputting the answers and other implicit reasoning steps (Weston et al., 2015).

Original: We apply NUTM to the question answering domain by replacing the NTM core with DNC (Graves et al., 2016).
Revised: We apply NUTM to the question answering domain by replacing the NTM core with DNC (Graves et al., 2016).

Original: Compared to NTM’s sequential addressing, dynamic memory addressing in DNC is more powerful and thus suitable for NSM integration to solve non-algorithmic problems such as question answering.
Revised: Compared to NTM’s sequential addressing, dynamic memory addressing in DNC is more powerful and thus suitable for NSM integration to solve non-algorithmic problems such as question answering.

Original: Following previous works of
Revised: Following previous works of

Original: DNC, we use bAbI dataset (Weston et al., 2015) to measure the performance of the NUTM with DNC core (three variants p = 1, p = 2 and p = 4).
Revised: DNC, we use bAbI dataset (Weston et al., 2015) to measure the performance of the NUTM with DNC core (three variants p = 1, p = 2 and p = 4).

Original: In the dataset, each story is followed by a series of questions and the network reads all word by word, then predicts the answers.
Revised: In the dataset, each story is followed by a series of questions and the network reads all word by word, then predicts the answers.

Original: Although synthetically generated, bAbI is a good benchmark that tests 20 aspects of natural language reasoning including complex skills such as induction and counting,
Revised: Although synthetically generated, bAbI is a good benchmark that tests 20 aspects of natural language reasoning including complex skills such as induction and counting,

Original: We found that increasing number of programs helps NUTM improve performance.
Revised: We found that increasing number of programs helps NUTM improve performance.

Original: In particular, NUTM with 4 programs, after 50 epochs jointly trained on all 20 question types, can achieve a mean test error rate of 3.3% and manages to solve 19/20 tasks (a task is considered solved if its error < 5%).
Revised: In particular, NUTM with 4 programs, after 50 epochs jointly trained on all 20 question types, can achieve a mean test error rate of 3.3% and manages to solve 19/20 tasks (a task is considered solved if its error < 5%).

Original: The mean and s.d. across 10 runs are also compared with other results reported by recent works (see Table 3).
Revised: The mean and s.d. across 10 runs are also compared with other results reported by recent works (see Table 3).

Original: Excluding baselines under diﬀerent setups, our result is the best reported mean result on bAbI that we are aware of.
Revised: Excluding baselines under diﬀerent setups, our result is the best reported mean result on bAbI that we are aware of.

Original: More details are described in App.
Revised: More details are described in App.

Original: E.
Revised: E.

Original:  Related Work
Revised:  Related Work

Original: Previous investigations into MANNs mostly revolve around memory access mechanisms.
Revised: Previous investigations into MANNs mostly revolve around memory access mechanisms.

Original: The works in Graves et al.
Revised: The works in Graves et al.

Original: (2014; 2016) introduce content-based, location-based and dynamic memory reading/writing.
Revised: (2014; 2016) introduce content-based, location-based and dynamic memory reading/writing.

Original: Further, Rae et al.
Revised: Further, Rae et al.

Original: (2016) scales to bigger memory by sparse access;
Revised: (2016) scales to bigger memory by sparse access;

Original: Le et al.
Revised: Le et al.

Original: (2019) optimizes memory operations with uniform writing; and MANNs with extra memory have been proposed (Le et al., 2018b).
Revised: (2019) optimizes memory operations with uniform writing; and MANNs with extra memory have been proposed (Le et al., 2018b).

Original: However, these works keep using memory for storing data rather than the weights of the network and thus parallel to our approach.
Revised: However, these works keep using memory for storing data rather than the weights of the network and thus parallel to our approach.

Original: Other DNC modiﬁcations (Csordas & Schmidhuber, 2019; Franke et al., 2018) are also orthogonal to our work.
Revised: Other DNC modiﬁcations (Csordas & Schmidhuber, 2019; Franke et al., 2018) are also orthogonal to our work.

Original: Another line of related work involves modularization of neural networks, which is designed for visual question answering.
Revised: Another line of related work involves modularization of neural networks, which is designed for visual question answering.

Original: In module networks (Andreas et al., 2016b;a), the modules are manually aligned with predeﬁned concepts and the order of execution is decided by the question.
Revised: In module networks (Andreas et al., 2016b;a), the modules are manually aligned with predeﬁned concepts and the order of execution is decided by the question.

Original: Although the module in these works resembles the program in NSM, our model is more generic and ﬂexible with soft-attention over programs and thus fully diﬀerentiable.
Revised: Although the module in these works resembles the program in NSM, our model is more generic and ﬂexible with soft-attention over programs and thus fully diﬀerentiable.

Original: Further, the motivation of NSM does not limit to a speciﬁc application.
Revised: Further, the motivation of NSM does not limit to a speciﬁc application.

Original: Rather, NSM aims to help MANN reach general-purpose computability.
Revised: Rather, NSM aims to help MANN reach general-purpose computability.

Original: If we view NSM network as a dynamic weight generator, the program in NSM can be linked to fast weight (Hinton & Plaut, 1987; Schmidhuber, 1993).
Revised: If we view NSM network as a dynamic weight generator, the program in NSM can be linked to fast weight (von der Malsburg, 1981; Hinton & Plaut, 1987; Schmidhuber, 1993b).

Original: These papers share the idea of using diﬀerent weights across timesteps to enable dynamic adaptation.
Revised: These papers share the idea of using diﬀerent weights across timesteps to enable dynamic adaptation.

Original: More recent works implement fast-weight using outer-products (Ba et al., 2016; Schlag & Schmidhuber, 2017).
Revised: Using outer-product is a common way to implement fast-weight (Schmidhuber, 1993a; Ba et al., 2016; Schlag & Schmidhuber, 2017).

Original: However, these fast weights are directly generated while our programs are interpolated from a set of slow weights.
Revised: These fast weights are directly generated and thus diﬀerent from our programs, which are interpolated from a set of slow weights.

Original: Tensor/Multiplicative RNN (Sutskever et al., 2011) and Hypernetwork (Ha et al., 2016) are also relevant related works.
Revised: Tensor/Multiplicative RNN (Sutskever et al., 2011) and Hypernetwork (Ha et al., 2016) are also relevant related works.

Original: These methods attempt to make the working weight of RNNs dependent on the input to enable quick adaption through time.
Revised: These methods attempt to make the working weight of RNNs dependent on the input to enable quick adaption through time.

Original: Nevertheless, they do not support modularity.
Revised: Nevertheless, they do not support modularity.

Original: In particular, Hypernetwork generates scaling factors for the single weight of the main RNN.
Revised: In particular, Hypernetwork generates scaling factors for the single weight of the main RNN.

Original: It does not aim to use multiple slow-weights (programs) and thus, diﬀerent from our approach.
Revised: It does not aim to use multiple slow-weights (programs) and thus, diﬀerent from our approach.

Original: Tensor RNN is closer to our idea when the authors propose to store M slow-weights, where M is the number of input dimension, which is acknowledged impractical.
Revised: Tensor RNN is closer to our idea when the authors propose to store M slow-weights, where M is the number of input dimension, which is acknowledged impractical.

Original: Unlike our approach, they do not use a meta-network to generate convex combinations amongst weights.
Revised: Unlike our approach, they do not use a meta-network to generate convex combinations amongst weights.

Original: Instead, they propose Multiplicative RNN that factorizes the working weight to product of three matrices, which looses modularity.
Revised: Instead, they propose Multiplicative RNN that factorizes the working weight to product of three matrices, which looses modularity.

Original: On the contrary, we explicitly model the working weight as an interpolation of multiple programs and use a meta-network to generate the coeﬃcients.
Revised: On the contrary, we explicitly model the working weight as an interpolation of multiple programs and use a meta-network to generate the coeﬃcients.

Original: This design facilitates modularity because each program is trained towards some functionality and can be switched or combined with each other to perform the current task.
Revised: This design facilitates modularity because each program is trained towards some functionality and can be switched or combined with each other to perform the current task.

Original: Last but not least, while the related works focus on improving RNN with fast-weight, we aim to reach a neural simulation of Universal Turing
Revised: Last but not least, while the related works focus on improving RNN with fast-weight, we aim to reach a neural simulation of Universal Turing

Original: Machine, in which fast-weight is a way to implement stored-program principle.
Revised: Machine, in which fast-weight is a way to implement stored-program principle.

Original:  Conclusions
Revised:  Conclusions

Original: This paper introduces the Neural Stored-program Memory (NSM), a new type of external memory for neural networks.
Revised: This paper introduces the Neural Stored-program Memory (NSM), a new type of external memory for neural networks.

Original: The memory, which takes inspirations from the stored-program memory in computer architecture, gives memory-augmented neural networks (MANNs) ﬂexibility to change their control programs through time while maintaining diﬀerentiability.
Revised: The memory, which takes inspirations from the stored-program memory in computer architecture, gives memory-augmented neural networks (MANNs) ﬂexibility to change their control programs through time while maintaining diﬀerentiability.

Original: The mechanism simulates modern computer behavior, potential making MANNs truly neural computers.
Revised: The mechanism simulates modern computer behavior, potential making MANNs truly neural computers.

Original: Our experiments demonstrated that when coupled with our model, the Neural
Revised: Our experiments demonstrated that when coupled with our model, the Neural

Original: Turing Machine learns algorithms better and adapts faster to new tasks at both sequence and sample levels.
Revised: Turing Machine learns algorithms better and adapts faster to new tasks at both sequence and sample levels.

Original: When used in few-shot learning, our method helps MANN as well.
Revised: When used in few-shot learning, our method helps MANN as well.

Original: We also applied the NSM to the Diﬀerentiable Neural Computer and observed a signiﬁcant improvement, reaching the state-of-the-arts in the bAbI task.
Revised: We also applied the NSM to the Diﬀerentiable Neural Computer and observed a signiﬁcant improvement, reaching the state-of-the-arts in the bAbI task.

Original: Although this paper limits to
Revised: Although this paper limits to

Original: MANN integration, other neural networks can also reap beneﬁts from our proposed model, which will be explored in future works.
Revised: MANN integration, other neural networks can also reap beneﬁts from our proposed model, which will be explored in future works.

Original: Appendix A Clustering on The Latent Space
Revised: Appendix A Clustering on The Latent Space

Original: As previously mentioned in Sec. 3.3, MANN should let its states form clusters to wellsimulate Turing Machine.
Revised: As previously mentioned in Sec. 3.3, MANN should let its states form clusters to wellsimulate Turing Machine.

Original: Fig.
Revised: Fig.

Original: 6 (a) and (c) show NTM actually organizes its c t space into clusters corresponding to processing states (e.g, encoding and decoding).
Revised: 6 (a) and (c) show NTM actually organizes its c t space into clusters corresponding to processing states (e.g, encoding and decoding).

Original: NUTM, which explicitly partitions this space, clearly learn better clusters of c t (see Fig.
Revised: NUTM, which explicitly partitions this space, clearly learn better clusters of c t (see Fig.

Original: 6 (b) and (d)).
Revised: 6 (b) and (d)).

Original: This contributes to NUTM’s outperformance over NTM.
Revised: This contributes to NUTM’s outperformance over NTM.

Original: B Program Usage Visualizations
Revised: B Program Usage Visualizations

Original: B.1 and B.2 visualize the best inferences of NUTM on test data from single and sequencing tasks.
Revised: B.1 and B.2 visualize the best inferences of NUTM on test data from single and sequencing tasks.

Original: Each plot starts with the input sequence and the predicted output sequence with error bits in the ﬁrst row 6 .
Revised: Each plot starts with the input sequence and the predicted output sequence with error bits in the ﬁrst row 6 .

Original: The second and fourth rows depict the read and write locations on data memory, respectively.
Revised: The second and fourth rows depict the read and write locations on data memory, respectively.

Original: The third and ﬁfth rows depict the program distribution of the read head and write head, respectively.
Revised: The third and ﬁfth rows depict the program distribution of the read head and write head, respectively.

Original: B.3 visualizes random failed predictions of NTM on sequencing tasks.
Revised: B.3 visualizes random failed predictions of NTM on sequencing tasks.

Original: The plots follow previous pattern  for the  rows.
Revised: The plots follow previous pattern except for the program distribution rows.

Original: B. Visualization on program distribution across timesteps (single tasks)
Revised: B. Visualization on program distribution across timesteps (single tasks)

Original: B. Visualization on program distribution across timesteps (sequencing tasks)
Revised: B. Visualization on program distribution across timesteps (sequencing tasks)

Original: B. Perseveration phenomenon in NTM (sequencing tasks)
Revised: B. Perseveration phenomenon in NTM (sequencing tasks)

Original: [Figure]

Original: C Details on Synthetic Tasks C. NTM single tasks
Revised: C Details on Synthetic Tasks C. NTM single tasks

Original: Tasks #Read/Write Head
Revised: Tasks #Read/Write Head

Original: [Table]
Revised: Controller Size

Original:  #Parameters
Revised: Memory Size #Parameters

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: Copy      63, 52,
Revised: Copy      63, 52,

Original: Repeat Copy      63, 52,
Revised: Repeat Copy      63, 52,

Original: Associative Recall      [Table] [Table]
Revised: Associative Recall      62, 51,

Original: Dynamic N-grams      [Table]
Revised: Dynamic N-grams      58, 48,

Original: [Table] [Table] Priority Sort      
Revised: Priority Sort      344, 302,

Original: [Table]
Revised: Long Copy      63, 52,

Original: [Table]
Revised: Tasks

Original: [Table]
Revised: [Table]

Original: Long Copy
Revised: Testing Copy

Revised: Sequence length range

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table] Sequencelength range: [1, 10]
Revised: Sequence

Original: [Table]
Revised: [Table]

Revised: Repeat Copy

Original:  range: [ [Table]
Revised: Sequence length range: [1, 10]

Original: [Table] range: [1, 10] # [Table] [Table]
Revised: Sequence length range: [10, 20] #Repeat range: [1, 10] #Repeat range: [10, 20]

Original: Associative Recall #Item range: [2, 6] #Item range: [6, 20]
Revised: Associative Recall #Item range: [2, 6] #Item range: [6, 20]

Original: Item 
Revised: Item length: Item length: Dynamic N

Original: [Table]
Revised: [Table]

Original: Item
Revised: grams

Original: [Table] [Table] Sequence [Table]
Revised: Sequence length:

Original: [Table]
Revised: Sequence length:

Original: Priority Sort #Item:#Item:#Sorted Item:#Sorted Item:
Revised: Priority Sort #Item:#Item:#Sorted Item:#Sorted Item:

Original: [Table] Sequence length range: [1, 40]
Revised: Long Copy Sequence length range: [1, 40]

Original: Sequence length
Revised: Sequence length:

Original: [Table]

Original: C. NTM sequencing tasks
Revised: C. NTM sequencing tasks

Original: Tasks #Read/Write Head
Revised: Tasks #Read/Write Head

Original: Controller Size
Revised: Controller Size

Original: [Table] [Table]
Revised: Memory Size #Parameters

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: NTM
Revised: NTM

Original: NUTM
Revised: NUTM

Original: C+RC      206, 153,
Revised: C+RC      206, 153,

Original: C+AR      206, 153,
Revised: C+AR      206, 153,

Original: C+PS
Revised: C+PS

Original: [Table]
Revised: [Table]

Original: C+RC+AR+
Revised: C+RC+AR+PS

Original: [Table]
Revised: [Table]

Original: Tasks
Revised: Tasks

Original: Training
Revised: Training

Original: Testing
Revised: Testing

Original: C+RC
Revised: C+RC

Original: Sequence length range: [1, 10]
Revised: Sequence length range: [

Original:  Sequence length range: [10, 20] #Repeat range: [1, 10] #Repeat range: [10, 15]
Revised: [Table] Sequence length range: [

Original: C+AR
Revised: [Table]

Original: Sequence length range: [1, 10]
Revised: [Table] range: [

Original: Sequence length range: [10, 20] #Item range: [2, 4] #Item range: [4, 6]
Revised: [Table]

Original: Item length: Item length:
Revised: [Table] range: [ [Table]

Original: C+PS
Revised: C+AR

Original: Sequence length range: [1, 10]
Revised: Sequence length range: [

Original: Sequence length range: [10, 20] #Item:#Item:#Sorted Item:#Sorted Item: C+RC+AR+PS
Revised: [Table]

Original: Sequence length range: [1, 10]
Revised: Sequence length range: [

Original: Sequence length range: [10, 20] #Repeat range: [1, 5] #Repeat:#Item range: [2, 4] #Item:
Revised: [Table]

Original: Item length: Item length
Revised: 20] #Item range: [

Original: [Table]
Revised: [Table]

Original: #Item:#Item:#Sorted Item:#Sorted Item: C. Continual procedure learning tasks
Revised: 4] #Item range: [

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: Item length: Item length

Original: [Table]
Revised: [Table]

Original: [Table] [Table] [Table] [Table]
Revised: C+PS

Original: [Table]
Revised: [Table]

Original: Item length 
Revised: length range: [

Original: [Table]
Revised: [Table]

Revised: Sequence length range: [

Original:  #Item 
Revised: [Table] 20] #Item [Table] [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: #Item
Revised: Item

Original: [Table]
Revised: [Table]

Original: #Sorted Item
Revised: C+RC+AR+PS Sequence length range: [ [Table] Sequence length range

Original: [Table]
Revised: [Table]

Revised: [Table]

Revised: [Table]

Original: #Sorted Item
Revised: [Table] C. Continual procedure learning tasks [Table] [Table]

Original: [Table]
Revised: [Table]

Original: D Details on Few-shot Learning Task
Revised: Details on Few-shot Learning Task

Original: We use similar hyper-parameters as in Santoro et al.
Revised: We use similar hyper-parameters as in Santoro et al.

Original: (2016) , which are reported in Tab.
Revised: (2016) , which are reported in Tab.

Original: Model
Revised: Model

Original: [Table]
Revised: [Table]

Original: MANN (LRUA)
Revised: MANN (LRUA)

Revised: [Table]

Original: NUTM (LRUA) NUTM (LRUA)
Revised: NUTM (LRUA) NUTM (LRUA)

Revised: [Table]

Original: Testing accuracy through time is listed below,
Revised: Testing accuracy through time is listed below,

Original: [Table]
Revised: [Table]

Original: E Details on bAbI Task
Revised: E Details on bAbI Task

Original: We train the models using RMSprop optimizer with ﬁxed learning rate of 10 − 4 and momentum of 0.9.
Revised: We train the models using RMSprop optimizer with ﬁxed learning rate of 10 − 4 and momentum of 0.9.

Original: The batch size is 32 and we adopt layer normalization (Lei Ba et al., 2016)
Revised: The batch size is 32 and we adopt layer normalization (Lei Ba et al., 2016)

Original: to
Revised: to

Original: DNC’s layers.
Revised: DNC’s layers.

Original: Following Franke et al.
Revised: Following Franke et al.

Original: (2018) practice, we also remove temporal linkage for faster training.
Revised: (2018) practice, we also remove temporal linkage for faster training.

Original: The details of hyper-parameters are listed in Table 12.
Revised: The details of hyper-parameters are listed in Table 12.

Original: Full NUTM ( p = 4) results are reported in Table 
Revised: Full NUTM ( p = 4) results are reported in Table 13.

Original: [Table]
Revised: [Table]

Revised: Read Head

Original: [Table]
Revised: [Table]

Revised: Head

Revised: [Table]

Revised: [Table]

Revised: F Example of memory operation function in NTM

Revised: In NTM, ξ t = { β t , k t , g t , s t , e t , v t } .

Revised: The memory addressing weight is initially computed by content-based attention,

Revised: [Equation]

Revised: Here, w ct ∈ R N is the content-based weight, β t is a strength scalar, and m is implemented as cosine similarity

Revised: [Equation]

Revised: In addition, NTM supports location-based addressing started with an interpolation between content-based weight and the previous weight

Revised: [Equation]

Revised: where g t is the interpolation gate that determines to use (or ignore) content-based addressing.

Revised: Then, NTM can shift the address to other rows by performing convolution shift modulo

Revised: R ,

Revised: [Equation]

Revised: [Equation]

Revised: where s t is the shift weighting.

Revised: To prevent the shifted weight from blurring, sharpening is applied

Revised: [Equation]

Revised: Then, the memory is updated as follows,

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: where e t ∈ R D and v t ∈ R D are erase vector and update vector, respectively.

Revised: The read value is computed using the same address weight as follows,

Revised: [Equation]

Original: F Others
Revised: G Others

Original: If we deliberately set the key dimension equal to the number of programs, we can even place an orthogonal basis constraint on the key space of NSM by minimizing the following loss,
Revised: If we deliberately set the key dimension equal to the number of programs, we can even place an orthogonal basis constraint on the key space of NSM by minimizing the following loss,

Original: [Equation]
Revised: [Equation]

Original: where M p .K
Revised: where M p .K

Original: and I denote the key part in NSM and the identity matrix, respectively.
Revised: and I denote the key part in NSM and the identity matrix, respectively.

Original: Direct attention is one special case of key-value attention when the memory keys form orthogonal basis.
Revised: Direct attention is one special case of key-value attention when the memory keys form orthogonal basis.

Original: When this happens, the generated key k pt plays a direct role as the attention weight w pt .
Revised: When this happens, the generated key k pt plays a direct role as the attention weight w pt .

Original: Thus, using key-value attention is more generic.
Revised: Thus, using key-value attention is more generic.

Original: For all tasks, η t is ﬁxed to 0 .
Revised: For all tasks, η t is ﬁxed to 0 .

Original: 1, reducing with decay rate of 0 . 9.
Revised: 1, reducing with decay rate of 0 . 9.

Original: Ablation study’s learning losses with mean and error bar are plotted in Fig.
Revised: Ablation study’s learning losses with mean and error bar are plotted in Fig.