Original: E MERGENT S YMBOLS THROUGH B INDING IN E XTERNAL M EMORY
Revised: E MERGENT S YMBOLS THROUGH B INDING IN E XTERNAL M EMORY

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: A key aspect of human intelligence is the ability to infer abstract rules directly from high-dimensional sensory data, and to do so given only a limited amount of training experience.
Revised: A key aspect of human intelligence is the ability to infer abstract rules directly from high-dimensional sensory data, and to do so given only a limited amount of training experience.

Original: Deep neural network algorithms have proven to be a powerful tool for learning directly from high-dimensional data, but currently lack this capacity for data-efﬁcient induction of abstract rules, leading some to argue that symbol-processing mechanisms will be necessary to account for this capacity.
Revised: Deep neural network algorithms have proven to be a powerful tool for learning directly from high-dimensional data, but currently lack this capacity for data-efﬁcient induction of abstract rules, leading some to argue that symbol-processing mechanisms will be necessary to account for this capacity.

Original: In this work, we take a step toward bridging this gap by introducing the Emergent
Revised: In this work, we take a step toward bridging this gap by introducing the Emergent

Original: Symbol Binding Network (ESBN), a recurrent network augmented with an external memory that enables a form of variable-binding and indirection.
Revised: Symbol Binding Network (ESBN), a recurrent network augmented with an external memory that enables a form of variable-binding and indirection.

Original: This binding mechanism allows symbol-like representations to emerge through the learning process without the need to explicitly incorporate symbol-processing machinery, enabling the ESBN to learn rules in a manner that is abstracted away from the particular entities to which those rules apply.
Revised: This binding mechanism allows symbol-like representations to emerge through the learning process without the need to explicitly incorporate symbol-processing machinery, enabling the ESBN to learn rules in a manner that is abstracted away from the particular entities to which those rules apply.

Original: Across a series of tasks, we show that this architecture displays nearly perfect generalization of learned rules to novel entities given only a limited number of training examples, and outperforms a number of other competitive neural network architectures.
Revised: Across a series of tasks, we show that this architecture displays nearly perfect generalization of learned rules to novel entities given only a limited number of training examples, and outperforms a number of other competitive neural network architectures.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Human intelligence is characterized by a remarkable capacity to detect the presence of simple, abstract rules that govern high-dimensional sensory data, such as images or sounds, and then apply these to novel data.
Revised: Human intelligence is characterized by a remarkable capacity to detect the presence of simple, abstract rules that govern high-dimensional sensory data, such as images or sounds and then apply these to novel data.

Original: This capacity has been extensively studied by psychologists in both the visual domain, in tasks such as Raven’s Progressive Matrices (Raven & Court, 1938), and the auditory domain, in tasks that employ novel, artiﬁcial languages (Marcus et al., 1999).
Revised: This capacity has been extensively studied by psychologists in both the visual domain, in tasks such as Raven’s Progressive Matrices (Raven & Court, 1938) and the auditory domain, in tasks that employ novel, artiﬁcial languages (Marcus et al., 1999).

Original: In recent years, deep neural network algorithms have reemerged as a powerful tool for learning directly from high-dimensional data, though many studies have now demonstrated that these models suffer from similar limitations as those faced by the earlier generation of neural networks: requiring enormous amounts of training data and tending to generalize poorly outside the distribution of those training data (Lake & Baroni, 2018; Barrett et al., 2018).
Revised: In recent years deep neural network algorithms have reemerged as a powerful tool for learning directly from high-dimensional data, though many studies have now demonstrated that these models suffer from similar limitations as those faced by the earlier generation of neural networks: requiring enormous amounts of training data and tending to generalize poorly outside the distribution of those training data (Lake & Baroni, 2018; Barrett et al., 2018).

Original: This stands in sharp contrast to the ability of human learners to infer abstract structure from a limited number of training examples and then systematically generalize that structure to problems involving novel entities.
Revised: This stands in sharp contrast to the ability of human learners to infer abstract structure from a limited number of training examples and then systematically generalize that structure to problems involving novel entities.

Original: It has long been argued that the human ability to generalize in this manner depends crucially on a capacity for variable-binding, that is, the ability to represent a problem in terms of abstract symbollike variables that are bound to concrete entities (Holyoak & Hummel, 2000; Marcus, 2001).
Revised: It has long been argued that the human ability to generalize in this manner depends crucially on a capacity for variable-binding, that is, the ability to represent a problem in terms of abstract symbollike variables that are bound to concrete entities (Holyoak & Hummel, 2000; Marcus, 2001).

Original: This in turn can be broken down into two components: 1) a mechanism for indirection , the ability to bind two representations together and then use one representation to refer to and retrieve the other (Kriete et al., 2013), and 2) a representational scheme whereby one of the bound representations codes for abstract variables, and the other codes for the values of those variables.
Revised: This in turn can be broken down into two components: 1) a mechanism for indirection , the ability to bind two representations together and then use one representation to refer to and retrieve the other (Kriete et al., 2013), and 2) a representational scheme whereby one of the bound representations codes for abstract variables, and the other codes for the values of those variables.

Original: In this work, we present a novel architecture designed around the goal of having a capacity for abstract variable-binding.
Revised: In this work, we present a novel architecture designed around the goal of having a capacity for abstract variable-binding.

Original: This is accomplished through two important design considerations.
Revised: This is accomplished through two important design considerations.

Original: First, the architecture possesses an explicit mechanism for indirection, in the form of a two-column external memory.
Revised: First, the architecture possesses an explicit mechanism for indirection, in the form of a two-column external memory.

Original: Second, the architecture is separated into two information-processing streams, one that maintains learned embeddings of concrete entities (in our case, images), and one in which a recurrent controller learns to represent and operate over task-relevant variables.
Revised: Second, the architecture is separated into two information-processing streams, one that maintains learned embeddings of concrete entities (in our case, images), and one in which a recurrent controller learns to represent and operate over task-relevant variables.

Original: These two streams only interact in the form of bindings in the external memory, allowing the controller to learn to perform tasks in a manner that is abstracted away from the particular entities involved.
Revised: These two streams only interact in the form of bindings in the external memory, allowing the controller to learn to perform tasks in a manner that is abstracted away from the particular entities involved.

Original: We refer to this architecture as the Emergent Symbol Binding Network (ESBN), due to the fact that this arrangement allows abstract, symbol-like representations to emerge during the learning process, without the need to incorporate symbolic machinery.
Revised: We refer to this architecture as the Emergent Symbol Binding Network (ESBN), due to the fact that this arrangement allows abstract, symbol-like representations to emerge during the learning process, without the need to incorporate symbolic machinery.

Original: We evaluate this architecture on a suite of tasks involving relationships among images that are governed by abstract rules.
Revised: We evaluate this architecture on a suite of tasks involving relationships among images that are governed by abstract rules.

Original: Across these tasks, we show that the ESBN is capable of learning abstract rules from a limited number of training examples and systematically generalizing these rules to novel entities.
Revised: Across these tasks, we show that the ESBN is capable of learning abstract rules from a limited number of training examples and systematically generalizing these rules to novel entities.

Original: By contrast, the other architectures that we evaluate are capable of learning these rules in some cases, but fail to generalize them successfully when trained on a limited number of problems involving a limited number of entities.
Revised: By contrast, the other architectures that we evaluate are capable of learning these rules in some cases, but fail to generalize them successfully when trained on a limited number of problems involving a limited number of entities.

Original: We conclude from these results that a capacity for variable-binding is a necessary component for human-like abstraction and generalization, and that the ESBN is a promising candidate for how to incorporate such a capacity into neural network algorithms.
Revised: We conclude from these results that a capacity for variable-binding is a necessary component for human-like abstraction and generalization, and that the ESBN is a promising candidate for how to incorporate such a capacity into neural network algorithms.

Original:  T ASKS
Revised:  T ASKS

Original: [Figure]
Revised: [Figure]

Original: We consider a series of tasks, each involving the application of an abstract rule to a set of images.
Revised: We consider a series of tasks, each involving the application of an abstract rule to a set of images.

Original: For all tasks, we employ the same set of n = 100 images, in which each image is a distinct Unicode character (the speciﬁc characters used are shown in A.6).
Revised: For all tasks, we employ the same set of n = 100 images, in which each image is a distinct Unicode character (the speciﬁc characters used are shown in A.7).

Original: We construct training sets in which m images are withheld (where 0 ≤ m ≤ n − o , and o is the minimum number of images necessary to create a problem in a given task) consisting of problems that employ only the remaining ( n − m ) images, and then test on problems that employ only the m withheld images, thus requiring generalization to novel entities.
Revised: We construct training sets in which m images are withheld (where 0 ≤ m ≤ n − o , and o is the minimum number of images necessary to create a problem in a given task) consisting of problems that employ only the remaining ( n − m ) images, and then test on problems that employ only the m withheld images, thus requiring generalization to novel entities.

Original: In the easiest generalization regime ( m = 0 ) the test set contains problems composed of the same entities as observed during training (though the exact order of these entities differs).
Revised: In the easiest generalization regime ( m = 0 ) the test set contains problems composed of the same entities as observed during training (though the exact order of these entities differs).

Original: In the most extreme generalization regime, we evaluate models that have only been trained on the minimum number of entities for a given task, and then must generalize what they learn to the majority of the n images in the complete set.
Revised: In the most extreme generalization regime, we evaluate models that have only been trained on the minimum number of entities for a given task, and then must generalize what they learn to the majority of the n images in the complete set.

Original: This regime poses an extremely challenging test of the ability to learn to perform these tasks from limited training experience, in a manner that is abstracted away from the speciﬁc entities observed during training.
Revised: This regime poses an extremely challenging test of the ability to learn to perform these tasks from limited training experience, in a manner that is abstracted away from the speciﬁc entities observed during training.

Original: The ﬁrst task that we study is a same/different discrimination task (Figure 1a).
Revised: The ﬁrst task that we study is a same/different discrimination task (Figure 1a).

Original: In this task, two images are presented, and the task is to determine whether they are the same or different.
Revised: In this task, two images are presented, and the task is to determine whether they are the same or different.

Original: Though this task may appear quite simple, it has been shown that the ability to generalize this simple rule to novel entities is actually a signiﬁcant challenge for deep neural networks (Kim et al., 2018), a pattern that we also observe in our results.
Revised: Though this task may appear quite simple, it has been shown that the ability to generalize this simple rule to novel entities is actually a signiﬁcant challenge for deep neural networks (Kim et al., 2018), a pattern that we also observe in our results.

Original: The second task that we consider is a relational match-to-sample (RMTS) task (Figure 1b), essentially a higher-order version of a same/different task.
Revised: The second task that we consider is a relational match-to-sample (RMTS) task (Figure 1b), essentially a higher-order version of a same/different task.

Original: In this task, a source pair of objects is compared to two target pairs.
Revised: In this task, a source pair of objects is compared to two target pairs.

Original: The task is to identify the target pair with the same relation as the source pair; e.g., if the source pair contains two of the same object, to identify the target pair that contains two of the same object.
Revised: The task is to identify the target pair with the same relation as the source pair; e.g., if the source pair contains two of the same object, to identify the target pair that contains two of the same object.

Original: It was initially believed that the ability to perform this task is not unique to humans (Premack, 1983), but it has now been shown that this ability depends on a visual entropy confound that arises from using large arrays of objects rather than simple pairs (Fagot et al., 2001).
Revised: It was initially believed that the ability to perform this task is not unique to humans (Premack, 1983), but it has now been shown that this ability depends on a visual entropy confound that arises from using large arrays of objects rather than simple pairs (Fagot et al., 2001).

Original: When the task is presented in a manner that does not allow this confound to be exploited (as is the case in our experiments), the ability to perform the task with novel entities appears to be unique to humans, and therefore is a good test of the human ability for abstract rule learning.
Revised: When the task is presented in a manner that does not allow this confound to be exploited (as is the case in our experiments), the ability to perform the task with novel entities appears to be unique to humans, and therefore is a good test of the human ability for abstract rule learning.

Original: Next we consider a task based on Raven’s Progressive Matrices (RPM; Raven & Court (1938)).
Revised: Next we consider a task based on Raven’s Progressive Matrices (RPM; Raven & Court (1938)).

Original: RPM is a commonly used visual problem-solving task, and is one of the most widely used tests of ﬂuid intelligence (Snow et al., 1984), the ability to reason and make inferences in a novel domain (as opposed to crystallized intelligence , the ability to solve familiar tasks).
Revised: RPM is a commonly used visual problem-solving task, and is one of the most widely used tests of ﬂuid intelligence (Snow et al., 1984), the ability to reason and make inferences in a novel domain (as opposed to crystallized intelligence , the ability to solve familiar tasks).

Original: In this task, a 3 × 3 array of ﬁgural elements is presented, in which the elements are governed by a simple rule, or set of rules, with the lower right element of the array left blank.
Revised: In this task, a 3 × 3 array of ﬁgural elements is presented, in which the elements are governed by a simple rule, or set of rules, with the lower right element of the array left blank.

Original: The task is to infer the rule that governs the elements in the array, and then use that rule to select from among 8 candidate completions.
Revised: The task is to infer the rule that governs the elements in the array, and then use that rule to select from among 8 candidate completions.

Original: Many of the rules that govern RPM problems are relations involving sets.
Revised: Many of the rules that govern RPM problems are relations involving sets.

Original: One such rule is sometimes referred to as distribution-of-three (Carpenter et al., 1990), according to which the same set of three elements (e.g. a triangle, square, and circle) will appear in each row, though the order doesn’t matter.
Revised: One such rule is sometimes referred to as distribution-of-three (Carpenter et al., 1990), according to which the same set of three elements (e.g. a triangle, square, and circle) will appear in each row, though the order doesn’t matter.

Original: The task in this case is simply to identify the set, determining which element is missing from the ﬁnal row, and locating this element among the choices.
Revised: The task in this case is simply to identify the set, determining which element is missing from the ﬁnal row, and locating this element among the choices.

Original: Though multiple RPM-inspired datasets have recently been proposed (Barrett et al., 2018; Zhang et al., 2019), in this work we choose to strip away unnecessary complexity, focusing on 2 × 3 arrays governed by a single rule (Figure 1c), in order to focus speciﬁcally on the capacity for generalization of an abstract rule to novel entities.
Revised: Though multiple RPM-inspired datasets have recently been proposed (Barrett et al., 2018; Zhang et al., 2019), in this work we choose to strip away unnecessary complexity, focusing on 2 × 3 arrays governed by a single rule (Figure 1c), in order to focus speciﬁcally on the capacity for generalization of an abstract rule to novel entities.

Original: We ﬁnd that, even in this simpliﬁed setting, this form of generalization is extremely challenging.
Revised: We ﬁnd that, even in this simpliﬁed setting, this form of generalization is extremely challenging.

Original: The ﬁnal task that we consider is a visual version of the identity rules task studied by Marcus et al.
Revised: The ﬁnal task that we consider is a visual version of the identity rules task studied by Marcus et al.

Original: In this task, an abstract pattern (e.g. ABA or ABB) must be inferred from a sequence of elements.
Revised: In this task, an abstract pattern (e.g. ABA or ABB) must be inferred from a sequence of elements.

Original: For instance, in the original study, the following sequence ‘ga ni ga, li na li, wo fe wo’ is governed by an ABA rule, whereas the sequence ‘ga ni ni, li na na, wo fe fe’ is governed by an
Revised: For instance, in the original study, the following sequence ‘ga ni ga, li na li, wo fe wo’ is governed by an ABA rule, whereas the sequence ‘ga ni ni, li na na, wo fe fe’ is governed by an

Original: ABB rule.
Revised: ABB rule.

Original: This study played an important role in debates concerning the presence of algebraic rulelike processes in human cognition, because it demonstrated that even 7-month-old human infants are capable of detecting this abstract regularity and generalizing it to novel entities, whereas neural networks tend to overﬁt to the speciﬁc entities involved and fail to generalize the rule.
Revised: This study played an important role in debates concerning the presence of algebraic rulelike processes in human cognition, because it demonstrated that even 7-month-old human infants are capable of detecting this abstract regularity and generalizing it to novel entities, whereas neural networks tend to overﬁt to the speciﬁc entities involved and fail to generalize the rule.

Original: In our implementation, we use visual images rather than sounds, and present the task as a 2 × 3 array (Figure 1d).
Revised: In our implementation, we use visual images rather than sounds, and present the task as a 2 × 3 array (Figure 1d).

Original: In this task, each problem is governed by either an ABA, ABB, or AAA rule.
Revised: In this task, each problem is governed by either an ABA, ABB, or AAA rule.

Original: The task is to determine which of these patterns is present in the ﬁrst row, and then to apply that pattern by selecting an element from a set of 4 choices to complete the second row.
Revised: The task is to determine which of these patterns is present in the ﬁrst row, and then to apply that pattern by selecting an element from a set of 4 choices to complete the second row.

Original: For all four tasks, we consider generalization regimes in which some number of images ( m ∈ { 0 , 50 , 85 , 95 } out of n = 100 ) are withheld from training.
Revised: For all four tasks, we consider generalization regimes in which some number of images ( m ∈ { 0 , 50 , 85 , 95 } out of n = 100 ) are withheld from training.

Original: For the same/different discrimination task, on which only two images are necessary to construct a problem, we also consider the case in which m = 98 (such that the training set consists of problems involving only n − m = 2 images, the minimum number necessary to construct the task).
Revised: For the same/different discrimination task, on which only two images are necessary to construct a problem, we also consider the case in which m = 98 (such that the training set consists of problems involving only n − m = 2 images, the minimum number necessary to construct the task).

Original: In most settings, we construct training sets consisting of 10 4 problems.
Revised: In most settings, we construct training sets consisting of 10 4 problems.

Original: This is a tiny fraction of all possible problems (on the order of 10 9 when the multiple choice options are considered) 

Revised: This is a tiny fraction of all possible problems (on the order of 10 9 when the multiple choice options are considered)

Original: Thus, even in the easiest generalization regime ( m = 0 ) this is an extremely small amount of training data relative to the size of the task space.
Revised: Thus, even in the easiest generalization regime ( m = 0 ) this is an extremely small amount of training data relative to the size of the task space.

Original: In the most extreme regimes, in which m ≥ 95 , it is only possible to construct a few hundred problems, resulting in even more limited training experience.
Revised: In the most extreme regimes, in which m ≥ 95 , it is only possible to construct a few hundred problems, resulting in even more limited training experience.

Original:  A PPROACH
Revised:  A PPROACH

Original: For each task, we treat the problem as a sequence of images x t =1 ... x t = T , with an associated target y .
Revised: For each task, we treat the problem as a sequence of images x t =1 ... x t = T , with an associated target y .

Original: In the same/different discrimination task, there are T = 2 images, and y is a binary target indicating whether the images are the same or different.
Revised: In the same/different discrimination task, there are T = 2 images, and y is a binary target indicating whether the images are the same or different.

Original: In the RMTS task, there are T = 6 images,  consisting of the source pair followed by two target pairs, and y is a binary target indicating which target pair matches the source pair.
Revised: In the RMTS task, there are T = 6 images, 1 Except for the same/different task, as detailed in A.2. consisting of the source pair followed by two target pairs, and y is a binary target indicating which target pair matches the source pair.

Original: In both the distribution-of-three task and the identity rules task, there are T = 9 images, consisting of the three entries in the ﬁrst row, the two non-empty entries in the second row, and the four multiple-choice options, and y is a four-way classiﬁcation target, indicating which of the multiple-choice options is correct.
Revised: In both the distribution-of-three task and the identity rules task, there are T = 9 images, consisting of the three entries in the ﬁrst row, the two non-empty entries in the second row, and the four multiple-choice options, and y is a four-way classiﬁcation target, indicating which of the multiple-choice options is correct.

Original: All images are 32 × 32 grayscale images containing a single Unicode character.
Revised: All images are 32 × 32 grayscale images containing a single Unicode character.

Original: For each problem, we ﬁrst process each image independently by a shared encoder f e , generating image embeddings z t =1 ...
Revised: For each problem, we ﬁrst process each image independently by a shared encoder f e , generating image embeddings z t =1 ...

Original: z t = T , and then pass these embeddings to a sequential model component f s that generates a response (either through a sigmoid output layer for tasks with a binary target, or a softmax layer for tasks with a four-way classiﬁcation target).
Revised: z t = T , and then pass these embeddings to a sequential model component f s that generates a response (either through a sigmoid output layer for tasks with a binary target, or a softmax layer for tasks with a four-way classiﬁcation target).

Original: The sequential component is either the ESBN or one of a number of alternative architectures described below.
Revised: The sequential component is either the ESBN or one of a number of alternative architectures described below.

Original: We use the same encoder architecture f e (detailed in A.2) for all models.
Revised: We use the same encoder architecture f e (detailed in A.3) for all models.

Original: All components are trained end-to-end, including the encoder.
Revised: All components are trained end-to-end, including the encoder.

Original: 3. T EMPORAL CONTEXT NORMALIZATION
Revised: 3. T EMPORAL CONTEXT NORMALIZATION

Original: We use temporal context normalization (TCN), recently shown to improve out-of-distribution generalization in relational reasoning tasks (Webb et al., 2020).
Revised: We use temporal context normalization (TCN), recently shown to improve out-of-distribution generalization in relational reasoning tasks (Webb et al., 2020).

Original: TCN is similar to batch normalization, but, instead of normalizing over the batch dimension, normalizes over a task-relevant temporal window.
Revised: TCN is similar to batch normalization, but, instead of normalizing over the batch dimension, normalizes over a task-relevant temporal window.

Original: This has the effect of preserving information about the relations between the entities present within this window (e.g. the size of those entities relative to one another), resulting in better generalization of learned relations to novel contexts (i.e. out-of-distribution).
Revised: This has the effect of preserving information about the relations between the entities present within this window (e.g. the size of those entities relative to one another), resulting in better generalization of learned relations to novel contexts (i.e. out-of-distribution).

Original: We found that TCN signiﬁcantly improved generalization for all of the models on all of the tasks studied in the present work 2 .
Revised: We found that TCN signiﬁcantly improved generalization for all of the models on all of the tasks studied in the present work 2 .

Original: Therefore, the primary results we report all incorporate this technique ( A.4.1 includes a comparison of the performance of all models on all tasks with and without TCN).
Revised: Therefore, the primary results we report all incorporate this technique ( A.5.1 includes a comparison of the performance of all models on all tasks with and without TCN).

Original: Speciﬁcally, we applied TCN to the embeddings z t =1 ...
Revised: Speciﬁcally, we applied TCN to the embeddings z t =1 ...

Original: z t =
Revised: z t =

Original: T extracted by the encoder.
Revised: T extracted by the encoder.

Original: Webb et al.
Revised: Webb et al.

Original: (2020) also reported that it is sometimes useful to apply TCN separately to different components of a sequence.
Revised: (2020) also reported that it is sometimes useful to apply TCN separately to different components of a sequence.

Original: We found that this was the case for the RMTS task that we studied, in which we found it useful to apply TCN separately to the embeddings for the source pair and each target pair.
Revised: We found that this was the case for the RMTS task that we studied, in which we found it useful to apply TCN separately to the embeddings for the source pair and each target pair.

Original: For all of the other tasks that we studied, TCN was applied over the entire sequence for each problem
Revised: For all of the other tasks that we studied, TCN was applied over the entire sequence for each problem.

Original: . 3. E MERGENT S YMBOL B INDING N ETWORK
Revised: [Figure]

Original: The ESBN (Figure 2; Algorithm 1) uses an LSTM controller ( f s ) with a differentiable external memory that is explicitly separated into keys ( M k ) and values ( M v ).
Revised: The ESBN (Figure 2; Algorithm 1) uses an LSTM controller ( f s ) with a differentiable external memory that is explicitly separated into keys ( M k ) and values ( M v ).

Original: At each time step t , a key/value pair is written to memory.
Revised: At each time step t , a key/value pair is written to memory.

Original: The keys written to memory, k w t , are generated by an output layer from the LSTM controller, and the values are the individual input embeddings, z t , of the input sequence, unmodiﬁed by the LSTM.
Revised: The keys written to memory, k w t , are generated by an output layer from the LSTM controller, and the values are the individual input embeddings, z t , of the input sequence, unmodiﬁed by the LSTM.

Original: Our hypothesis was that factoring the model into two separate information processing streams would allow the LSTM to learn how to represent abstract variables in the keys {+
 Except for the PrediNet on the same/different task, as detailed in A.5.1.+} it generates, which could then be explicitly bound to associated values (image embeddings) learned by the separate encoder network ( f e ), allowing the ESBN to employ a form of indirection.
Revised: Our hypothesis was that factoring the model into two separate information processing streams would allow the LSTM to learn how to represent abstract variables in the keys {+
 Except for the PrediNet on the same/different task, as detailed in A.5.1.+} it generates, which could then be explicitly bound to associated values (image embeddings) learned by the separate encoder network ( f e ), allowing the ESBN to employ a form of indirection.

Original: To retrieve keys from memory, similarity scores are computed by comparing (via a dot product) the image embedding z t to all of the values in memory M v t − 1 .
Revised: To retrieve keys from memory, similarity scores are computed by comparing (via a dot product) the image embedding z t to all of the values in memory M v t − 1 .

Original: These similarity scores are passed through a softmax nonlinearity to generate weights w k t , and passed through a sigmoid nonlinearity (with learned gain and bias parameters, γ and β ) to generate conﬁdence values c k t (one weight and conﬁdence value per entry in memory).
Revised: These similarity scores are passed through a softmax nonlinearity to generate weights w k t , and passed through a sigmoid nonlinearity (with learned gain and bias parameters, γ and β ) to generate conﬁdence values c k t (one weight and conﬁdence value per entry in memory).

Original: The weights are used to compute 1) a weighted sum of all keys in memory M k t − 1 , and 2) a weighted sum of all associated conﬁdence values c k t .
Revised: The weights are used to compute 1) a weighted sum of all keys in memory M k t − 1 , and 2) a weighted sum of all associated conﬁdence values c k t .

Original: Finally, the retrieved key and associated conﬁdence value are concatenated and multiplied by a learned sigmoidal gate g t to form k r t , the input to the LSTM controller at the next time step.
Revised: Finally, the retrieved key and associated conﬁdence value are concatenated and multiplied by a learned sigmoidal gate g t to form k r t , the input to the LSTM controller at the next time step.

Original:  k r t =0 ← 0 ; h t =0 ← 0 ; M k t =

Revised: Algorithm k r t =0 ← 0 ; h t =0 ← 0 ; M k t =

Original: ← {} ; M v t =

Revised: ← {} ; M v t =

Original: ← {} ; for t in 1 ...
Revised: ← {} ; for t in 1 . . .

Original: T do z t ← f e ( x t ) ; ˆ y t , g t , k w t , h t ← f s ( h t − 1 , k r t − 1 ) ; if t is 1 then k r t ← 0 ; else w k t ← softmax( M v t − 1 · z t ); c k t ← σ ( γ ( M v t − 1 · z t ) + β ); k r t ← g t t − 1 (cid:88) i =
Revised: T do z t ← f e ( x t ) ; ˆ y t , g t , k w t , h t ← f s ( h t − 1 , k r t − 1 ) ; if t is 1 then k r t ← 0 ; else w k t ← softmax( M v t − 1 · z t ); c k t ← σ ( γ ( M v t − 1 · z t ) + β ); k r t ← g t t − 1 (cid:88) i =

Original: w k t ( i )( M k t − 1 ( i ) (cid:107) c
Revised: w k t ( i )( M k t − 1 ( i ) (cid:107) c

Original: k t ( i )) ; end
Revised: k t ( i )) ; end

Original: M k t ← { M k t − 1 , k w t } ;
Revised: M k t ← { M k t − 1 , k w t } ;

Original: M v t ← { M v t − 1 , z t } ; endreturn ˆ y t =
Revised: M v t ← { M v t − 1 , z t } ; endreturn ˆ y t =

Original: T
Revised: T

Original: 3. A LTERNATIVE A RCHITECTURES
Revised: 3. A LTERNATIVE A RCHITECTURES

Original: The simplest alternative architecture that we consider is an LSTM (Hochreiter & Schmidhuber, 1997) without external memory.
Revised: The simplest alternative architecture that we consider is an LSTM (Hochreiter & Schmidhuber, 1997) without external memory.

Original: We pass the low-dimensional embeddings z t =1 ...
Revised: We pass the low-dimensional embeddings z t =1 ...

Original: z t = T directly to the
Revised: z t = T directly to the

Original: LSTM, and generate a prediction ˆ y by passing the ﬁnal hidden state through an output layer.
Revised: LSTM, and generate a prediction ˆ y by passing the ﬁnal hidden state through an output layer.

Original: Next we consider two alternative external memory architectures: the Neural Turing Machine (NTM; Graves et al.
Revised: Next we consider two alternative external memory architectures: the Neural Turing Machine (NTM; Graves et al.

Original: and Metalearned Neural Memory (MNM; Munkhdalai et al. (2019)).
Revised: and Metalearned Neural Memory (MNM; Munkhdalai et al. (2019)).

Original: This comparison allows us to determine to what extent our results depend on the speciﬁc details of the ESBN’s external memory, and, in particular, the separation between its two informationprocessing pathways vs. the mere presence of an external memory.
Revised: This comparison allows us to determine to what extent our results depend on the speciﬁc details of the ESBN’s external memory, and, in particular, the separation between its two informationprocessing pathways vs. the mere presence of an external memory.

Original: Our NTM implementation consists of an LSTM controller (which takes image embeddings as input, and generates a prediction ˆ
Revised: Our NTM implementation consists of an LSTM controller (which takes image embeddings as input, and generates a prediction ˆ

Original: y as output) that interacts with an external memory using both content-based and location-based read/write mechanisms.
Revised: y as output) that interacts with an external memory using both content-based and location-based read/write mechanisms.

Original: Our MNM implementation employs the publicly available code from the original paper, modiﬁed so as to employ the same encoder architecture and TCN procedure as the other architectures that we test.
Revised: Our MNM implementation employs the publicly available code from the original paper, modiﬁed so as to employ the same encoder architecture and TCN procedure as the other architectures that we test.

Original: Just as with the ESBN, we allow both of these architectures an extra time step to process the information retrieved from memory following the ﬁnal input.
Revised: Just as with the ESBN, we allow both of these architectures an extra time step to process the information retrieved from memory following the ﬁnal input.

Original: We also consider the Relation Net (RN; Santoro et al.
Revised: We also consider the Relation Net (RN; Santoro et al.

Original: (2017)), an architecture that has proven to be an effective approach for a wide range of relational reasoning tasks.
Revised: (2017)), an architecture that has proven to be an effective approach for a wide range of relational reasoning tasks.

Original: In our implementation, we treat the low-dimensional image embeddings as individual ‘objects’ in the RN framework, using a shared MLP to process all pair-wise combinations of these embeddings, summing the outputs from this MLP, and then passing them to another MLP that generates the prediction ˆ y .
Revised: In our implementation, we treat the low-dimensional image embeddings as individual ‘objects’ in the RN framework, using a shared MLP to process all pair-wise combinations of these embeddings, summing the outputs from this MLP, and then passing them to another MLP that generates the prediction ˆ y .

Original: We also compare our model against the Transformer (Vaswani et al., 2017), an architecture originally developed in the domain of natural language processing, but has proven to be effective for a wide range of sequential data, and demonstrated a capacity for some degree of extrapolation (Saxton et al., 2019).
Revised: We also compare our model against the Transformer (Vaswani et al., 2017), an architecture originally developed in the domain of natural language processing, but has proven to be effective for a wide range of sequential data, and demonstrated a capacity for some degree of extrapolation (Saxton et al., 2019).

Original: After applying the transformer architecture to the sequence of image embeddings (allowing self-attention between these embeddings), we compute an average of the (transformed) embeddings, and then pass this to a small MLP that then generates the task output ˆ y .
Revised: After applying the transformer architecture to the sequence of image embeddings (allowing self-attention between these embeddings), we compute an average of the (transformed) embeddings, and then pass this to a small MLP that then generates the task output ˆ y .

Original: Finally, we consider the PrediNet (Shanahan et al., 2019).
Revised: Finally, we consider the PrediNet (Shanahan et al., 2019).

Original: PrediNet was designed with the goal of being ‘explicitly relational,’ and has been shown to be effective at generalizing learned relations to novel entities.
Revised: PrediNet was designed with the goal of being ‘explicitly relational,’ and has been shown to be effective at generalizing learned relations to novel entities.

Original: We apply the PrediNet’s multi-head attention over the 1D temporal sequence of image embeddings (as opposed to applying attention over a 2D image, as in the original work), and then pass the output of the PrediNet module to a small MLP that generates ˆ y .
Revised: We apply the PrediNet’s multi-head attention over the 1D temporal sequence of image embeddings (as opposed to applying attention over a 2D image, as in the original work), and then pass the output of the PrediNet module to a small MLP that generates ˆ y .

Original:  R ESULTS
Revised:  R ESULTS

Original: [Figure]
Revised: [Figure]

Original: Figure 3 shows the generalization results for all four tasks.
Revised: Figure 3 shows the generalization results for all four tasks.

Original: Our primary ﬁnding is that the ESBN displayed nearly perfect generalization ( ≥ 95% ) of the learned rule in all four tasks, even when trained on a very limited number of problems (just hundreds of problems, in the case of the most extreme generalization regimes) involving a limited number of entities (as few as just two entities, in the case of the same/different task), and tested on completely novel entities.
Revised: Our primary ﬁnding is that the ESBN displayed nearly perfect generalization ( ≥ 95% ) of the learned rule in all four tasks, even when trained on a very limited number of problems (just hundreds of problems, in the case of the most extreme generalization regimes) involving a limited number of entities (as few as just two entities, in the case of the same/different task), and tested on completely novel entities.

Original: Some of the alternative architectures that we evaluated showed a surprising capability to generalize to novel entities in some tasks as seen, for instance, in the generalization results for the Transformer and RN on the same/different and RMTS tasks (though we note that all architectures incorporate TCN, without which generalization is signiﬁcantly worse, as shown in A.4.1).
Revised: Some of the alternative architectures that we evaluated showed a surprising capability to generalize to novel entities in some tasks as seen, for instance, in the generalization results for the Transformer and RN on the same/different and RMTS tasks (though we note that all architectures incorporate TCN, without which generalization is signiﬁcantly worse, as shown in A.5.1).

Original: Nevertheless, none of these alternative architectures were able to generalize what they learned in the most extreme generalization regimes, whereas the ESBN performed comparably well across all regimes.
Revised: Nevertheless, none of these alternative architectures were able to generalize what they learned in the most extreme generalization regimes, whereas the ESBN performed comparably well across all regimes.

Original: Notably, the RN performed very poorly on the distribution-of-three and identity rules tasks, even in the easiest regime ( m = 0 ).
Revised: Notably, the RN performed very poorly on the distribution-of-three and identity rules tasks, even in the easiest regime ( m = 0 ).

Original: We speculate that this results from the fact that the RN is biased toward pair-wise relations, whereas these tasks are both based on a ternary relation.
Revised: We speculate that this results from the fact that the RN is biased toward pair-wise relations, whereas these tasks are both based on a ternary relation.

Original: It is possible to represent this ternary relation as a combination of pair-wise relations, but doing so requires a more complex strategy and therefore likely more training data.
Revised: It is possible to represent this ternary relation as a combination of pair-wise relations, but doing so requires a more complex strategy and therefore likely more training data.

Original: We include results in A.4.2 demonstrating that
Revised: We include results in A.5.2 demonstrating that

Original: the
Revised: the

Original: RN is capable of successfully generalizing in this task (though not in the most extreme regimes) when trained on an order of magnitude more data ( 10 5 instead of 10 4 examples).
Revised: RN is capable of successfully generalizing in this task (though not in the most extreme regimes) when trained on an order of magnitude more data ( 10 5 instead of 10 4 examples).

Original: We also present results for the Temporal Relation Network (Zhou et al., 2018), an RN variant that incorporates ternary relations via subsampling, though we ﬁnd that this doesn’t help as much as increasing the amount of training data.
Revised: We also present results for the Temporal Relation Network (Zhou et al., 2018), an RN variant that incorporates ternary relations via subsampling, though we ﬁnd that this doesn’t help as much as increasing the amount of training data.

Original: [Figure]
Revised: [Figure]

Original: In addition to requiring a very small amount of training data and generalizing systematically to novel entities, the ESBN also requires very little training time.
Revised: In addition to requiring a very small amount of training data and generalizing systematically to novel entities, the ESBN also requires very little training time.

Original: Figure 4 shows training accuracy time courses for the RMTS, distribution-of-three, and identity rules tasks for all models 3 .
Revised: Figure 4 shows training accuracy time courses for the RMTS, distribution-of-three, and identity rules tasks for all models 3 .

Original: The
Revised: The

Original: ESBN converged to nearly perfect training accuracy within 100 to 200 training updates on all four tasks, whereas the other models required thousands, or even tens of thousands of training updates to reach convergence 4 .
Revised: ESBN converged to nearly perfect training accuracy within 100 to 200 training updates on all four tasks, whereas the other models required thousands, or even tens of thousands of training updates to reach convergence 4 .

Original: We also performed some experiments to better understand how the ESBN operates, and why it was so effective.
Revised: We also performed some experiments to better understand how the ESBN operates, and why it was so effective.

Original: First, we tested whether the systematic generalization exhibited by the ESBN was dependent on the use of convolutional layers in the encoder, which naturally confer a signiﬁcant degree of generalization in tasks that involve shape recognition.
Revised: First, we tested whether the systematic generalization exhibited by the ESBN was dependent on the use of convolutional layers in the encoder, which naturally confer a signiﬁcant degree of generalization in tasks that involve shape recognition.

Original: We found that the ESBN generalized to novel entities comparably well when using either an MLP encoder or a random projection (see A.4.
Revised: We found that the ESBN generalized to novel entities comparably well when using either an MLP encoder or a random projection (see A.5.

Original: for details), suggesting that the ESBN is capable of generalizing learned rules to any arbitrary set of entities, regardless of how those entities are encoded.
Revised: for details), suggesting that the ESBN is capable of generalizing learned rules to any arbitrary set of entities, regardless of how those entities are encoded.

Original: For comparison, we also performed the same experiments with the Transformer (the best performing alternative architecture on our tasks) and found that, by contrast, its performance was signiﬁcantly impaired by the use of a random projection instead of a convolutional encoder.
Revised: For comparison, we also performed the same experiments with the Transformer (the best performing alternative architecture on our tasks) and found that, by contrast, its performance was signiﬁcantly impaired by the use of a random projection instead of a convolutional encoder.

Original: Second, we performed an ablation experiment on the conﬁdence value appended to retrieved memories.
Revised: Second, we performed an ablation experiment on the conﬁdence value appended to retrieved memories.

Original: We found that ablation of these conﬁdence values impaired the ESBN’s performance in both the same/different and RMTS tasks, but not the distribution-of-three or identity rules tasks (see A.4.
Revised: We found that ablation of these conﬁdence values impaired the ESBN’s performance in both the same/different and RMTS tasks, but not the distribution-of-three or identity rules tasks (see A.5.

Original: for details).
Revised: for details).

Original: A likely reason for this result is that the distribution-of-three and identity rules tasks only require retrieval of the best match from memory, whereas the same/different and RMTS tasks require a sense of how good of a match that memory is, which is exactly the information that the conﬁdence value conveys.
Revised: A likely reason for this result is that the distribution-of-three and identity rules tasks only require retrieval of the best match from memory, whereas the same/different and RMTS tasks require a sense of how good of a match that memory is, which is exactly the information that the conﬁdence value conveys.

Original: This dissociation mirrors the distinction sometimes made in cognitive psychology between recollection and familiarity (Yonelinas, 2001).
Revised: This dissociation mirrors the distinction sometimes made in cognitive psychology between recollection and familiarity (Yonelinas, 2001).

Original: Third, we performed an analysis of the key representations learned by the controller.
Revised: Third, we performed an analysis of the key representations learned by the controller.

Original: We hypothesized that the controller would learn to represent abstract variables in the keys that it writes to memory, and that these representations therefore shouldn’t vary based on the values to which they are bound.
Revised: We hypothesized that the controller would learn to represent abstract variables in the keys that it writes to memory, and that these representations therefore shouldn’t vary based on the values to which they are bound.

Original: This analysis revealed a high degree of overlap between the keys written during training and test (involving entirely different entities), suggesting that this was indeed the case (see A.5 for details).
Revised: This analysis revealed a high degree of overlap between the keys written during training and test (involving entirely different entities), suggesting that this was indeed the case (see A.6 for details).

Original: This ability to arbitrarily bind values to variables, without affecting the representations of those variables, is a key property of symbol-processing systems, and is likely the basis of the strong systematic generalization exhibited by the ESBN.
Revised: This ability to arbitrarily bind values to variables, without affecting the representations of those variables, is a key property of symbol-processing systems, and is likely the basis of the strong systematic generalization exhibited by the ESBN.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: There have been a number of proposals for augmenting neural networks with an external memory.
Revised: There have been a number of proposals for augmenting neural networks with an external memory.

Original: An inﬂuential early line of work, Complementary Learning Systems (McClelland et al., 1995), proposed that neural systems beneﬁt from having components that learn on different time scales, and argues that this combination allows neural networks both to learn general, abstract structure (using standard learning algorithms) and to rapidly encode arbitrary new items (using an external memory).
Revised: An inﬂuential early line of work, Complementary Learning Systems (McClelland et al., 1995), proposed that neural systems beneﬁt from having components that learn on different time scales, and argues that this combination allows neural networks both to learn general, abstract structure (using standard learning algorithms) and to rapidly encode arbitrary new items (using an external memory).

Original: In recent years, there have been a number of proposals for how to implement the latter efﬁciently, including Fast Weights (Ba et al., 2016a), the NTM and closely related Differentiable
Revised: In recent years, there have been a number of proposals for how to implement the latter efﬁciently, including Fast Weights (Ba et al., 2016a), the NTM and closely related Differentiable

Original: Neural Computer (Graves et al., 2016), and the Differentiable Neural Dictionary (DND; Pritzel et al.
Revised: Neural Computer (Graves et al., 2016), and the Differentiable Neural Dictionary (DND; Pritzel et al.

Original: Our external memory approach is most closely related to the DND, which also involves a two-column key/value memory.
Revised: Our external memory approach is most closely related to the DND, which also involves a two-column key/value memory.

Original: Variations on key/value memory have also been employed in other more recently proposed approaches, such as the Memory Recall Agent (Fortunato et al., 2019) and the Dual-Coding Episodic Memory (Hill et al., 2020), where it afforded various beneﬁts in terms of generalization.
Revised: Variations on key/value memory have also been employed in other more recently proposed approaches, such as the Memory Recall Agent (Fortunato et al., 2019) and the Dual-Coding Episodic Memory (Hill et al., 2020), where it afforded various beneﬁts in terms of generalization.

Original: One critical difference between our model and this previous work is that the ESBN’s controller is forced to interact with perceptual inputs only indirectly through its memory, a design decision that we argue is crucial to its ability to systematically generalize what it learns.
Revised: One critical difference between our model and this previous work is that the ESBN’s controller is forced to interact with perceptual inputs only indirectly through its memory, a design decision that we argue is crucial to its ability to systematically generalize what it learns.

Original: It is worth noting that architectures such as Fast Weights and the NTM are, in principle, capable of implementing variable-binding, though it is a separate question whether such a strategy will result from learning in any particular task.
Revised: It is worth noting that architectures such as Fast Weights and the NTM are, in principle, capable of implementing variable-binding, though it is a separate question whether such a strategy will result from learning in any particular task.

Original: Along these lines, a recent study from Chen et al.
Revised: Along these lines, a recent study from Chen et al.

Original: (2019) found that both of these architectures are capable of generalizing learned structure to novel entities when allowed a sufﬁciently dense sampling of the space of potential objects (the ‘objects’ in their study were randomly sampled 50-dimensional vectors).
Revised: (2019) found that both of these architectures are capable of generalizing learned structure to novel entities when allowed a sufﬁciently dense sampling of the space of potential objects (the ‘objects’ in their study were randomly sampled 50-dimensional vectors).

Original: This contrasts with our ﬁndings, in which the
Revised: This contrasts with our ﬁndings, in which the

Original: NTM performed poorly when trained on far fewer samples from a much higher-dimensional space (in the m = 95 regime).
Revised: NTM performed poorly when trained on far fewer samples from a much higher-dimensional space (in the m = 95 regime).

Original: This suggests that indirection and variable-binding, though possible in principle for architectures such as the NTM, do not emerge in practice when given only a limited amount of training experience, whereas this capacity is explicitly built into the ESBN.
Revised: This suggests that indirection and variable-binding, though possible in principle for architectures such as the NTM, do not emerge in practice when given only a limited amount of training experience, whereas this capacity is explicitly built into the ESBN.

Original: At a high level, the idea of factoring a model into two distinct information processing streams, one that codes abstract task-relevant variables or roles and one that codes concrete entities, has been explored before.
Revised: At a high level, the idea of factoring a model into two distinct information processing streams, one that codes abstract task-relevant variables or roles and one that codes concrete entities, has been explored before.

Original: Kriete et al.
Revised: Kriete et al.

Original: (2013) proposed the PBWM Indirection model, in which one population of neurons acted as a pointer to another population of neurons by gating its activity, and showed that this model enabled a signiﬁcant degree of generalization to novel role/ﬁller bindings.
Revised: (2013) proposed the PBWM Indirection model, in which one population of neurons acted as a pointer to another population of neurons by gating its activity, and showed that this model enabled a signiﬁcant degree of generalization to novel role/ﬁller bindings.

Original: Whittington et al.
Revised: Whittington et al.

Original: (2019) proposed the Tolman-Eichenbaum machine, a model that is capable of learning abstract relational structure (such as 2D spatial maps), and showed that this model captured a number of phenomena relating to grid cells and place cells.
Revised: (2019) proposed the Tolman-Eichenbaum machine, a model that is capable of learning abstract relational structure (such as 2D spatial maps), and showed that this model captured a number of phenomena relating to grid cells and place cells.

Original: Russin et al.
Revised: Russin et al.

Original: (2019) proposed Syntactic Attention, an architecture involving separate pathways for processing syntax vs. semantics, and showed that this approach was capable of a signiﬁcant degree of compositional generalization on the challenging SCAN dataset.
Revised: (2019) proposed Syntactic Attention, an architecture involving separate pathways for processing syntax vs. semantics, and showed that this approach was capable of a signiﬁcant degree of compositional generalization on the challenging SCAN dataset.

Original: Relative to this previous work, our central contribution is the development of a simple model that can learn abstract rules directly from high-dimensional data (images), exploiting this same high-level idea to enable nearly perfect generalization of those rules to novel entities.
Revised: Relative to this previous work, our central contribution is the development of a simple model that can learn abstract rules directly from high-dimensional data (images), exploiting this same high-level idea to enable nearly perfect generalization of those rules to novel entities.

Original: There has been extensive modeling work focusing on some of the tasks that we study.
Revised: There has been extensive modeling work focusing on some of the tasks that we study.

Original: The recent development of two datasets modeled after Raven’s Progressive Matrices, Procedurally Generated
Revised: The recent development of two datasets modeled after Raven’s Progressive Matrices, Procedurally Generated

Original: Matrices (Barrett et al., 2018), and RAVEN (Zhang et al., 2019), has spurred the development of models that are capable of solving RPM-like problems (Jahrens & Martinetz, 2020; Wu et al., 2020).
Revised: Matrices (Barrett et al., 2018), and RAVEN (Zhang et al., 2019), has spurred the development of models that are capable of solving RPM-like problems (Jahrens & Martinetz, 2020; Wu et al., 2020).

Original: However, these models typically require very large training sets (on the order of 10 6 training examples), and largely fail to generalize outside of the speciﬁc conditions under which they are trained, whereas the ESBN exhibits the ability to learn rapidly and generalize out-of-distribution.
Revised: However, these models typically require very large training sets (on the order of 10 6 training examples), and largely fail to generalize outside of the speciﬁc conditions under which they are trained, whereas the ESBN exhibits the ability to learn rapidly and generalize out-of-distribution.

Original: There have also been a number of models proposed to account for the human ability to rapidly learn identity rules (Alhama & Zuidema, 2019).
Revised: There have also been a number of models proposed to account for the human ability to rapidly learn identity rules (Alhama & Zuidema, 2019).

Original: Though some of these models achieved signiﬁcant generalization of learned identity rules to novel entities, they did so mostly through the inclusion of highly task-speciﬁc mechanisms.
Revised: Though some of these models achieved signiﬁcant generalization of learned identity rules to novel entities, they did so mostly through the inclusion of highly task-speciﬁc mechanisms.

Original: By contrast, our aim in the present work was to present a general approach that could be applied to a wider range of tasks.
Revised: By contrast, our aim in the present work was to present a general approach that could be applied to a wider range of tasks.

Original: Finally, there have been a number of recent proposals for so-called ‘neurosymbolic’ models, incorporating elements from both the neural network and symbolic modeling frameworks (Mao et al., 2019; Nye et al., 2020).
Revised: Finally, there have been a number of recent proposals for so-called ‘neurosymbolic’ models, incorporating elements from both the neural network and symbolic modeling frameworks (Mao et al., 2019; Nye et al., 2020).

Original: Though we have emphasized the notion of ‘emergent symbols’ in the present work, we stress that this is quite distinct from neurosymbolic modeling efforts since we do not explicitly incorporate any symbolic machinery into the ESBN.
Revised: Though we have emphasized the notion of ‘emergent symbols’ in the present work, we stress that this is quite distinct from neurosymbolic modeling efforts since we do not explicitly incorporate any symbolic machinery into the ESBN.

Original: Instead, our approach was to show how the functional equivalent of symbols can emerge in a neural network model with an appropriate architecture and binding mechanism.
Revised: Instead, our approach was to show how the functional equivalent of symbols can emerge in a neural network model with an appropriate architecture and binding mechanism.

Original:  D ISCUSSION 6. L IMITATIONS AND FUTURE WORK
Revised:  D ISCUSSION 6. L IMITATIONS AND FUTURE WORK

Original: One open question is whether the strict division between the two information processing streams in the ESBN is necessary, and whether it limits the sorts of relations and rules that it can learn.
Revised: One open question is whether the strict division between the two information processing streams in the ESBN is necessary, and whether it limits the sorts of relations and rules that it can learn.

Original: In future work, it may be desirable to soften this division, for instance by encouraging it in a regularization term, rather than strictly enforcing it architecturally.
Revised: In future work, it may be desirable to soften this division, for instance by encouraging it in a regularization term, rather than strictly enforcing it architecturally.

Original: A second limitation is that the tasks we study are not as complex as other similar tasks that have recently been studied, such as the two recently proposed RPM-like benchmarks (Barrett et al., 2018;
Revised: A second limitation is that the tasks we study are not as complex as other similar tasks that have recently been studied, such as the two recently proposed RPM-like benchmarks (Barrett et al., 2018;

Original: Zhang et al., 2019).
Revised: Zhang et al., 2019).

Original: In the present work, we intentionally stripped away some of this complexity in order to make progress on the issue of out-of-distribution generalization.
Revised: In the present work, we intentionally stripped away some of this complexity in order to make progress on the issue of out-of-distribution generalization.

Original: Extending the ESBN to more complex tasks will likely require the incorporation of visual attention mechanisms to enable selective sequential processing of individual elements within a scene.
Revised: Extending the ESBN to more complex tasks will likely require the incorporation of visual attention mechanisms to enable selective sequential processing of individual elements within a scene.

Original: There are many recently proposed approaches for doing this (Gregor et al., 2015; Locatello et al., 2020).
Revised: There are many recently proposed approaches for doing this (Gregor et al., 2015; Locatello et al., 2020).

Original: In future work, we look forward to extending the ESBN in this manner and testing it on more complex tasks.
Revised: In future work, we look forward to extending the ESBN in this manner and testing it on more complex tasks.

Original: 6. R ELATION TO WORK IN NEUROSCIENCE
Revised: 6. R ELATION TO WORK IN NEUROSCIENCE

Original: It is worth considering how the present work relates to pre-existing theories of how the brain might implement variable-binding.
Revised: It is worth considering how the present work relates to pre-existing theories of how the brain might implement variable-binding.

Original: Classic proposals for variable-binding in neural systems emphasize dynamic binding of representations, either by computing the tensor product between those representations (Smolensky, 1990), or by establishing synchronous activation between two pools of units (Hummel & Holyoak, 1997).
Revised: Classic proposals for variable-binding in neural systems emphasize dynamic binding of representations, either by computing the tensor product between those representations (Smolensky, 1990), or by establishing synchronous activation between two pools of units (Hummel & Holyoak, 1997).

Original: An alternative proposal is that variable-binding is accomplished via semi-permanent synaptic changes in the hippocampus, relying on the same mechanism that plays a central role in episodic memory (Cer & O’Reilly, 2006).
Revised: An alternative proposal is that variable-binding is accomplished via semi-permanent synaptic changes in the hippocampus, relying on the same mechanism that plays a central role in episodic memory (Cer & O’Reilly, 2006).

Original: This approach relies on contextual information and retrieval processes to prevent potential interference between conﬂicting memories, rather than explicit unbinding mechanisms.
Revised: This approach relies on contextual information and retrieval processes to prevent potential interference between conﬂicting memories, rather than explicit unbinding mechanisms.

Original: Our model is more in line with the latter account, since it does not possess an unbinding mechanism.
Revised: Our model is more in line with the latter account, since it does not possess an unbinding mechanism.

Original: As such, our model can be seen as part of a recent trend toward the reinterpretation of putatively working memory functions in terms of episodic memory (Beukers et al., 2020).
Revised: As such, our model can be seen as part of a recent trend toward the reinterpretation of putatively working memory functions in terms of episodic memory (Beukers et al., 2020).

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work, we have presented a model of abstract rule learning based on a novel architecture, the
Revised: In this work, we have presented a model of abstract rule learning based on a novel architecture, the

Original: ESBN, and shown that this model is capable of rapidly learning abstract rules directly from images given only a small amount of training experience, and then successfully generalizing those rules to novel entities.
Revised: ESBN, and shown that this model is capable of rapidly learning abstract rules directly from images given only a small amount of training experience, and then successfully generalizing those rules to novel entities.

Original: Key to the model’s performance is its separation into two streams that only interact through indirection, allowing the ESBN to learn tasks in a manner that is abstracted away from the speciﬁc entities involved, and resulting in the emergence of symbol-like representations.
Revised: Key to the model’s performance is its separation into two streams that only interact through indirection, allowing the ESBN to learn tasks in a manner that is abstracted away from the speciﬁc entities involved, and resulting in the emergence of symbol-like representations.

Original: We believe that these results suggest that such a variable-binding capacity is an essential ingredient for achieving human-like abstraction and generalization, and hope that the ESBN will be a useful tool for doing so. 
Revised: We believe that these results suggest that such a variable-binding capacity is an essential ingredient for achieving human-like abstraction and generalization, and hope that the ESBN will be a useful tool for doing so. A CKNOWLEDGMENTS

Revised: We would like to thank Zachary Dulberg, Steven Frankland, Randall O’Reilly, Alexander Petrov, and Simon Segert for their helpful feedback and discussions.

Original: A A PPENDIX A. D ATASET GENERATION
Revised: A A PPENDIX A. C ODE AVAILABILITY

Revised: All code, including code for dataset generation, model implementation, training and evaluation is available on GitHub.

Revised: A. D ATASET GENERATION

Original: In this section we provide details on the dataset generation process for all tasks.
Revised: In this section we provide details on the dataset generation process for all tasks.

Original: In all of our simulations, a dataset was generated from scratch (according to the procedures described below) at the beginning of each training run, such that different runs involved different datasets, though the statistics were the same across these datasets.
Revised: In all of our simulations a dataset was generated from scratch (according to the procedures described below) at the beginning of each training run such that different runs involved different datasets though the statistics were the same across these datasets.

Original: We did this to prevent the possibility that our results would reﬂect biases present in a particular dataset.
Revised: We did this to prevent the possibility that our results would reﬂect biases present in a particular dataset.

Original: [Table] 
Revised: [Table] A.2. S AME / DIFFERENT DISCRIMINATION

Original: Given n = 100 total images, with m = 0 withheld during training, there are n 2 = 10 4 possible same/different problems.
Revised: Given n = 100 total images, with m = 0 withheld during training, there are n 2 = 10 4 possible same/different problems.

Original: To prevent the potential for networks to be biased by the fact that the overwhelming majority of these are ‘different’ problems, we created balanced datasets by including duplicates of the ‘same’ problems.
Revised: To prevent the potential for networks to be biased by the fact that the overwhelming majority of these are ‘different’ problems, we created balanced datasets by including duplicates of the ‘same’ problems.

Original: Speciﬁcally, we randomly sampled (with replacement) n ( n − 1) of the n unique ‘same’ trials and combined them with the n ( n − 1) unique ‘different’ trials, resulting in 2 n ( n − 1) = 19 , 800 total problems.
Revised: Speciﬁcally, we randomly sampled (with replacement) n ( n − 1) of the n unique ‘same’ trials and combined them with the n ( n − 1) unique ‘different’ trials, resulting in 2 n ( n − 1) = 19 , 800 total problems.

Original: We reserved 990 of these problems for test, yielding training sets including 18 , 810 problems (ensuring that duplicates of the same problem did not appear in both the training and test sets).
Revised: We reserved 990 of these problems for test, yielding training sets including 18 , 810 problems (ensuring that duplicates of the same problem did not appear in both the training and test sets).

Original: We followed a similar procedure for the other regimes, generating balanced datasets by duplicating the ‘same’ problems when necessary.
Revised: We followed a similar procedure for the other regimes, generating balanced datasets by duplicating the ‘same’ problems when necessary.

Original: These datasets incorporated either all of the problems that resulted from this procedure (given the n − m images available for training, or the m images available for test), or 10 , 000 problems, whichever was smaller.
Revised: These datasets incorporated either all of the problems that resulted from this procedure (given the n − m images available for training, or the m images available for test), or 10 , 000 problems, whichever was smaller.

Original: The exact size of each of these datasets is shown in Table 1.
Revised: The exact size of each of these datasets is shown in Table 1.

Original: A.1. RMTS
Revised: A.2. RMTS

Original: [Table]
Revised: [Table]

Original: For the RMTS task, we constructed balanced training and test sets ensuring that there were an equal number of problems with a ‘same’ vs. ‘different’ source pair.
Revised: For the RMTS task, we constructed balanced training and test sets ensuring that there were an equal number of problems with a ‘same’ vs. ‘different’ source pair.

Original: These datasets contained either 10 ,problems, or the minimum number of problems possible in a given regime, whichever was smaller (Table 2).
Revised: These datasets contained either 10 ,problems, or the minimum number of problems possible in a given regime, whichever was smaller (Table 2).

Original: For most regimes, 10 , 000 problems constitutes a tiny fraction of the full space of possible problems (ranging from 10 9 for the m = 0 regime to 10 5 for the training set in the m = 85 regime), and thus there was no need to duplicate problems to achieve balanced datasets.
Revised: For most regimes, 10 , 000 problems constitutes a tiny fraction of the full space of possible problems (ranging from 10 9 for the m = 0 regime to 10 5 for the training set in the m = 85 regime), and thus there was no need to duplicate problems to achieve balanced datasets.

Original: For the m =regime, there are only 480 possible training problems, which happen to include the same number of ‘same’ and ‘different’ trial types.
Revised: For the m =regime, there are only 480 possible training problems, which happen to include the same number of ‘same’ and ‘different’ trial types.

Original: A.1. D ISTRIBUTION - OF - THREE
Revised: A.2. D ISTRIBUTION - OF - THREE

Original: [Table]
Revised: [Table]

Original: For the distribution-of-three task, we generated problems by randomly selecting three of the available images in a given regime (either n − m images during training, or m images during test), and then randomly sampling two permutations of those images for the two rows (allowing the possibility that the same permutation appears in both rows) of the 2 × 3 matrix.
Revised: For the distribution-of-three task, we generated problems by randomly selecting three of the available images in a given regime (either n − m images during training, or m images during test), and then randomly sampling two permutations of those images for the two rows (allowing the possibility that the same permutation appears in both rows) of the 2 × 3 matrix.

Original: We then randomly selected a fourth image to appear with the other three as possible answers, and randomly permuted these four answer choices.
Revised: We then randomly selected a fourth image to appear with the other three as possible answers, and randomly permuted these four answer choices.

Original: When taking into account the identity of this fourth image, and the permutation of the answer choices, the number of unique distribution-of-three problems in the m = 0 regime is on the order of 10 10 .
Revised: When taking into account the identity of this fourth image, and the permutation of the answer choices, the number of unique distribution-of-three problems in the m = 0 regime is on the order of 10 10 .

Original: For most regimes, we randomly created training and test sets consisting of 10 , 000 randomly sampled problems.
Revised: For most regimes, we randomly created training and test sets consisting of 10 , 000 randomly sampled problems.

Original: For the m = 95 regime, the training set consisted of 360 problems (the total number of unique problems possible in this regime when not considering the identity and order of the answer choices, which were randomly selected).
Revised: For the m = 95 regime, the training set consisted of 360 problems (the total number of unique problems possible in this regime when not considering the identity and order of the answer choices, which were randomly selected).

Original: A.1. I DENTITY RULES
Revised: A.2. I DENTITY RULES

Original:  For the identity rules task, we constructed datasets with an approximately balanced (through uniform random sampling) number of ABA, ABB, and AAA problems.
Revised: [Table] For the identity rules task, we constructed datasets with an approximately balanced (through uniform random sampling) number of ABA, ABB, and AAA problems.

Original: These datasets consisted of either 10 , 000 problems, or the minimum number of possible problems in a given regime, whichever was smaller (Table 4).
Revised: These datasets consisted of either 10 , 000 problems, or the minimum number of possible problems in a given regime, whichever was smaller (Table 4).

Original: For the training set in the m = 95 regime, datasets consisting of 8 , 640 problems were constructed from the 7 , 200 possible unique problems in this regime, by duplicating the AAA problems to match the number of ABA/ABB problems.
Revised: For the training set in the m = 95 regime, datasets consisting of 8 , 640 problems were constructed from the 7 , 200 possible unique problems in this regime, by duplicating the AAA problems to match the number of ABA/ABB problems.

Original: For all other datasets, 10 , 000 problems constituted a small fraction of the total number of possible problems (ranging from 10 9 for the[Table] m = 0 regime to 10 6 for the training set in the m = 85 regime), and no duplication was necessary to achieve balanced problem types.
Revised: For all other datasets, 10 , 000 problems constituted a small fraction of the total number of possible problems (ranging from 10 9 for the m = 0 regime to 10 6 for the training set in the m = 85 regime), and no duplication was necessary to achieve balanced problem types.

Original: A. I MPLEMENTATION DETAILS FOR ALL MODELS A.2. E NCODER
Revised: A. I MPLEMENTATION DETAILS FOR ALL MODELS A.3. E NCODER

Original: We used the same feedforward encoder architecture to process each of the images in a sequence x t =1 ... x t = T , generating low-dimensional embeddings z t =1 ...
Revised: We used the same feedforward encoder architecture to process each of the images in a sequence x t =1 ... x t = T , generating low-dimensional embeddings z t =1 ...

Original: z t = T that were then passed to the core sequential component of each model (either the ESBN or one of the alternative architectures) 5 .
Revised: z t = T that were then passed to the core sequential component of each model (either the ESBN or one of the alternative architectures) 5 .

Original: This encoder consisted of three convolutional layers, each with 32 channels, a 4 × 4 kernel, and a stride of 2 , followed by two fully-connected layers with 256 units and 128 units respectively.
Revised: This encoder consisted of three convolutional layers, each with 32 channels, a 4 × 4 kernel, and a stride of 2 , followed by two fully-connected layers with 256 units and 128 units respectively.

Original: All layers used ReLU nonlinearities.
Revised: All layers used ReLU nonlinearities.

Original: All weights were initialized using a Kaiming normal distribution (He et al., 2015), and all biases were initialized to 0 .
Revised: All weights were initialized using a Kaiming normal distribution (He et al., 2015), and all biases were initialized to 0 .

Original: A.2. T ASK OUTPUT LAYER
Revised: A.3. T ASK OUTPUT LAYER

Original: All models had an output layer for generating ˆ y .
Revised: All models had an output layer for generating ˆ y .

Original: The number of units and the nonlinearity depended on the task.
Revised: The number of units and the nonlinearity depended on the task.

Original: For the same/different and RMTS tasks, the output layer had 1 unit and a sigmoid nonlinearity (producing a number between 0 and 1 to code for ‘same’ vs. ‘different’, or pair 1 vs. pair 2).
Revised: For the same/different and RMTS tasks, the output layer had 1 unit and a sigmoid nonlinearity (producing a number between 0 and 1 to code for ‘same’ vs. ‘different’, or pair 1 vs. pair 2).

Original: For the distribution-of-three and identity rules tasks, the output layer had 4 units and a softmax nonlinearity (to select 1 of the 4 answer choices).
Revised: For the distribution-of-three and identity rules tasks, the output layer had 4 units and a softmax nonlinearity (to select 1 of the 4 answer choices).

Original: The weights of the output layer were initialized using an Xavier normal distribution (Glorot & Bengio, 2010), and the biases were initialized to 0 .
Revised: The weights of the output layer were initialized using an Xavier normal distribution (Glorot & Bengio, 2010), and the biases were initialized to 0 .

Original: A.2. ESBN
Revised: A.3. ESBN

Original: The details of the ESBN’s operations are given in Algorithm 1.
Revised: The details of the ESBN’s operations are given in Algorithm 1.

Original: The LSTM controller had 1 layer with 512 units, and employed the standard tanh nonlinearities and sigmoidal gates.
Revised: The LSTM controller had 1 layer with 512 units, and employed the standard tanh nonlinearities and sigmoidal gates.

Original: The controller also had output layers for k w ( 256 units with a ReLU nonlinearity), g ( 1 unit with a sigmoid nonlinearity), and ˆ y .
Revised: The controller also had output layers for k w ( 256 units with a ReLU nonlinearity), g ( 1 unit with a sigmoid nonlinearity), and ˆ y .

Original: The input to the controller at each time step was k r , the key retrieved from memory at the previous time step (along with the associated conﬁdence value, c k ).
Revised: The input to the controller at each time step was k r , the key retrieved from memory at the previous time step (along with the associated conﬁdence value, c k ).

Original: At the beginning of each sequence, k r and the controller’s hidden state h were initialized to 0 .
Revised: At the beginning of each sequence, k r and the controller’s hidden state h were initialized to 0 .

Original: After processing a full sequence, the ESBN was allowed an additional time step for the controller to process the retrieved key associated with the ﬁnal input.
Revised: After processing a full sequence, the ESBN was allowed an additional time step for the controller to process the retrieved key associated with the ﬁnal input.

Original: After this additional time step, the ﬁnal hidden state of the LSTM was passed through the task output layer to generate the prediction ˆ y .
Revised: After this additional time step, the ﬁnal hidden state of the LSTM was passed through the task output layer to generate the prediction ˆ y .

Original: We note that it is also possible to retrieve values from memory (from M v ) using a similar procedure and then decode these values to make predictions in image space (Anonymous, 2020), but in the present work we focus only on the classiﬁcation component of the model.
Revised: We note that it is also possible to retrieve values from memory (from M v ) using a similar procedure and then decode these values to make predictions in image space (Sinha et al., 2020), but in the present work we focus only on the classiﬁcation component of the model.

Original: The input weights for the LSTM controller were initialized using an Xavier normal distribution with a gain value of 5 / 3 .
Revised: The input weights for the LSTM controller were initialized using an Xavier normal distribution with a gain value of 5 / 3 .

Original: The weights for the LSTM’s gates, as well as the weights for the output layer for the gate g , were initialized with an Xavier normal distribution (with a gain of 1 ).
Revised: The weights for the LSTM’s gates, as well as the weights for the output layer for the gate g , were initialized with an Xavier normal distribution (with a gain of 1 ).

Original: The weights for the output layer that produced k w were initialized using a Kaiming normal distribution.
Revised: The weights for the output layer that produced k w were initialized using a Kaiming normal distribution.

Original: All biases were initialized to 0 .
Revised: All biases were initialized to 0 .

Original: The parameters γ and β were initialized to 1 and 0 respectively.
Revised: The parameters γ and β were initialized to 1 and 0 respectively.

Original: A.2. LSTM
Revised: A.3. LSTM

Original: The LSTM architecture had 1 layer with 512 units.
Revised: The LSTM architecture had 1 layer with 512 units.

Original: Image embeddings were passed to the LSTM as a sequence, after which ˆ y was generated through a task output layer.
Revised: Image embeddings were passed to the LSTM as a sequence, after which ˆ y was generated through a task output layer.

Original: The LSTM’s hidden state was initialized to 0 at the beginning of each sequence.
Revised: The LSTM’s hidden state was initialized to 0 at the beginning of each sequence.

Original: The LSTM’s weights were initialized using the same scheme as the LSTM controller in the ESBN (using an Xavier normal distribution, with a gain of 5 / 3 for the input weights and 1 for the gates), and biases were initialized to 0 .
Revised: The LSTM’s weights were initialized using the same scheme as the LSTM controller in the ESBN (using an Xavier normal distribution, with a gain of 5 / 3 for the input weights and 1 for the gates), and biases were initialized to 0 .

Original: A.2. NTM
Revised: A.3. NTM

Original: The NTM had an LSTM controller ( 1 layer with 512 units).
Revised: The NTM had an LSTM controller ( 1 layer with 512 units).

Original: The LSTM’s hidden state was initialized to 0 at the beginning of each sequence, and the LSTM’s parameters were initialized in the same way as the LSTM architecture and the controller for the ESBN.
Revised: The LSTM’s hidden state was initialized to 0 at the beginning of each sequence, and the LSTM’s parameters were initialized in the same way as the LSTM architecture and the controller for the ESBN.

Original: The NTM had one write head and one read head.
Revised: The NTM had one write head and one read head.

Original: The read head had the following output layers:
Revised: The read head had the following output layers:

Original: read key: 256 units, tanh nonlinearity, weights initialized using an Xavier normal distribution with a gain of 5 / 3 . 2.
Revised: read key: 256 units, tanh nonlinearity, weights initialized using an Xavier normal distribution with a gain of 5 / 3 . 2.

Original: key strength: 1 unit, softplus nonlinearity, weights initialized using a Kaiming normal distribution.
Revised: key strength: 1 unit, softplus nonlinearity, weights initialized using a Kaiming normal distribution.

Original: interpolation gate: 1 unit, sigmoid nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 . 4.
Revised: interpolation gate: 1 unit, sigmoid nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 . 4.

Original: shift weights: 3 units (corresponding to the allowable shifts − 1 , 0 , and 1 ), softmax nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 .
Revised: shift weights: 3 units (corresponding to the allowable shifts − 1 , 0 , and 1 ), softmax nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 .

Original: The write head had the following output layers:
Revised: The write head had the following output layers:

Original: erase vector: 256 units, sigmoid nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 . 2. add vector: 256 units, tanh nonlinearity, weights initialized using an Xavier normal distribution with a gain of 5 / 3 . 3.
Revised: erase vector: 256 units, sigmoid nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 . 2. add vector: 256 units, tanh nonlinearity, weights initialized using an Xavier normal distribution with a gain of 5 / 3 . 3.

Original: write key: 256 units, tanh nonlinearity, weights initialized using an Xavier normal distribution with a gain of 5 / 3 . 4.
Revised: write key: 256 units, tanh nonlinearity, weights initialized using an Xavier normal distribution with a gain of 5 / 3 . 4.

Original: key strength: 1 unit, softplus nonlinearity, weights initialized using a Kaiming normal distribution.
Revised: key strength: 1 unit, softplus nonlinearity, weights initialized using a Kaiming normal distribution.

Original: interpolation gate: 1 unit, sigmoid nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 . 6.
Revised: interpolation gate: 1 unit, sigmoid nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 . 6.

Original: shift weights: 3 units (corresponding to the allowable shifts − 1 , 0 , and 1 ), softmax nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 .
Revised: shift weights: 3 units (corresponding to the allowable shifts − 1 , 0 , and 1 ), softmax nonlinearity, weights initialized using an Xavier normal distribution with a gain of 1 .

Original: All biases for these output layers were initialized to 0 .
Revised: All biases for these output layers were initialized to 0 .

Original: The NTM used these outputs to interact with its external memory, employing all of the location- and content-based mechanisms described in the original work (Graves et al., 2014).
Revised: The NTM used these outputs to interact with its external memory, employing all of the location- and content-based mechanisms described in the original work (Graves et al., 2014).

Original: Cosine similarity was used as a similarity measure for the content-based mechanisms.
Revised: Cosine similarity was used as a similarity measure for the content-based mechanisms.

Original: The memory matrix had 10 rows of size 256 .
Revised: The memory matrix had 10 rows of size 256 .

Original: The initial state of the memory at the beginning of each sequence was learned.
Revised: The initial state of the memory at the beginning of each sequence was learned.

Original: The learned initial state was initialized (at the beginning of training) using an Xavier normal distribution.
Revised: The learned initial state was initialized (at the beginning of training) using an Xavier normal distribution.

Original: The input to the LSTM controller at each time step consisted of the image embedding corresponding to that time step and the read vector from the previous time step.
Revised: The input to the LSTM controller at each time step consisted of the image embedding corresponding to that time step and the read vector from the previous time step.

Original: At the beginning of each sequence, the read vector, read weights, and write weights were initialized to 0 .
Revised: At the beginning of each sequence, the read vector, read weights, and write weights were initialized to 0 .

Original: Just as with the ESBN, the
Revised: Just as with the ESBN, the

Original: NTM was allowed an additional time step to process the read vector retrieved from memory after observing the ﬁnal image embedding, after which ˆ y was generated through an output layer from the
Revised: NTM was allowed an additional time step to process the read vector retrieved from memory after observing the ﬁnal image embedding, after which ˆ y was generated through an output layer from the

Original: LSTM controller.
Revised: LSTM controller.

Original: A.2. MNM
Revised: A.3. MNM

Original: We implemented the MNM using publicly available code released with the original publication (Munkhdalai et al., 2019).
Revised: We implemented the MNM using publicly available code released with the original publication (Munkhdalai et al., 2019).

Original: Speciﬁcally, we used the version of MNM that employs a learned local update (‘MNM-p’ in the original paper).
Revised: Speciﬁcally, we used the version of MNM that employs a learned local update (‘MNM-p’ in the original paper).

Original: Before passing the images in our tasks to the MNM model, we applied the same encoder and TCN procedure used for the other architectures that we tested.
Revised: Before passing the images in our tasks to the MNM model, we applied the same encoder and TCN procedure used for the other architectures that we tested.

Original: Other than this modiﬁcation, the original implementation, including all architectural hyperparameters, was unmodiﬁed.
Revised: Other than this modiﬁcation, the original implementation, including all architectural hyperparameters, was unmodiﬁed.

Original: A.2. RN
Revised: A.3. RN

Original: The RN implementation consisted of two MLPs.
Revised: The RN implementation consisted of two MLPs.

Original: The ﬁrst MLP (used to process all pair-wise combinations of image embeddings) had a hidden layer of size 512 and an output layer of size 256 .
Revised: The ﬁrst MLP (used to process all pair-wise combinations of image embeddings) had a hidden layer of size 512 and an output layer of size 256 .

Original: The outputs from the ﬁrst MLP were summed, and then passed to the second MLP, which had a hidden layer of size 256 and an output layer for generating ˆ y .
Revised: The outputs from the ﬁrst MLP were summed, and then passed to the second MLP, which had a hidden layer of size 256 and an output layer for generating ˆ y .

Original: All layers (except the output layer) used ReLU nonlinearities.
Revised: All layers (except the output layer) used ReLU nonlinearities.

Original: All weights were initialized using a Kaiming normal distribution (except the output layer, which was initialized according to the description in A.2.2), and all biases were initialized to 0 .
Revised: All weights were initialized using a Kaiming normal distribution (except the output layer, which was initialized according to the description in A.3.2), and all biases were initialized to 0 .

Original: Before passing the image embeddings to the ﬁrst MLP, they were appended with a tag (an integer from 0 to T − 1 ) indicating their position in the input sequence.
Revised: Before passing the image embeddings to the ﬁrst MLP, they were appended with a tag (an integer from 0 to T − 1 ) indicating their position in the input sequence.

Original: A.2. TRN
Revised: A.3. TRN

Original: The TRN employs two key design decisions intended to prevent the combinatorial explosion that would naturally result from the inclusion of n-ary relations:
Revised: The TRN employs two key design decisions intended to prevent the combinatorial explosion that would naturally result from the inclusion of n-ary relations:

Original: Only considering temporally ordered, non-redundant sets (whereas the original RN considers all possible pairs of objects, including both permutations of the same pair, and the pair of each object with itself).
Revised: Only considering temporally ordered, non-redundant sets (whereas the original RN considers all possible pairs of objects, including both permutations of the same pair, and the pair of each object with itself).

Original: Subsampling from these sets.
Revised: Subsampling from these sets.

Original: We found that it was computationally feasible to implement a TRN with ternary relations in our tasks by only using (1), without the need to subsample.
Revised: We found that it was computationally feasible to implement a TRN with ternary relations in our tasks by only using (1), without the need to subsample.

Original: Thus, our implementation considers all temporally ordered, non-redundant sets of two and three.
Revised: Thus, our implementation considers all temporally ordered, non-redundant sets of two and three.

Original: Each pair of image embeddings was processed by an MLP with a hidden layer of size 512 and an output layer of size 256 .
Revised: Each pair of image embeddings was processed by an MLP with a hidden layer of size 512 and an output layer of size 256 .

Original: The outputs of this MLP for all pairs were then summed, and processed by an additional fully-connected layer with 256 units, yielding a single vector representing all pairs.
Revised: The outputs of this MLP for all pairs were then summed, and processed by an additional fully-connected layer with 256 units, yielding a single vector representing all pairs.

Original: Each set of three image embeddings was processed by a separate MLP with the same hyperparameters (hidden layer of 512 units, output layer of 256 units).
Revised: Each set of three image embeddings was processed by a separate MLP with the same hyperparameters (hidden layer of 512 units, output layer of 256 units).

Original: The outputs of this MLP for all sets of three were then summed, and processed by a separate fully-connected layer with 256 units, yielding a single vector representing all sets of three.
Revised: The outputs of this MLP for all sets of three were then summed, and processed by a separate fully-connected layer with 256 units, yielding a single vector representing all sets of three.

Original: These two vectors, representing all pairs and sets of three, were then summed and passed to an additional fully-connected layer with 256 units, and then to the output layer to generate ˆ y .
Revised: These two vectors, representing all pairs and sets of three, were then summed and passed to an additional fully-connected layer with 256 units, and then to the output layer to generate ˆ y .

Original: All layers (except for the output layer) used ReLU nonlinearities.
Revised: All layers (except for the output layer) used ReLU nonlinearities.

Original: All weights in these layers were initialized using a Kaiming normal distribution, and all biases were initialized to 0 .
Revised: All weights in these layers were initialized using a Kaiming normal distribution, and all biases were initialized to 0 .

Original: Just as with the RN, we append the image embeddings with a tag indicating their position in the sequence before passing them to the ﬁrst MLP in the TRN.
Revised: Just as with the RN, we append the image embeddings with a tag indicating their position in the sequence before passing them to the ﬁrst MLP in the TRN.

Original: A.2. T RANSFORMER
Revised: A.3. T RANSFORMER

Original: The Transformer implementation consisted of a single Transformer encoder layer.
Revised: The Transformer implementation consisted of a single Transformer encoder layer.

Original: We also experimented with 2- and 3-layer Transformers
Revised: We also experimented with 2- and 3-layer Transformers

Original: but these did not generalize as well as the 1-layer Transformer in the tasks that we studied.
Revised: but these did not generalize as well as the 1-layer Transformer in the tasks that we studied.

Original: Positional encoding (as described by Vaswani et al.
Revised: Positional encoding (as described by Vaswani et al.

Original: (2017)) was applied to the sequence of image embeddings, which were then passed to the Transformer layer.
Revised: (2017)) was applied to the sequence of image embeddings, which were then passed to the Transformer layer.

Original: The self-attention layer had 8 heads.
Revised: The self-attention layer had 8 heads.

Original: The MLP had a single hidden layer with 512 units, and used
Revised: The MLP had a single hidden layer with 512 units, and used

Original: ReLU nonlinearities.
Revised: ReLU nonlinearities.

Original: Residual connections and layer normalization (Ba et al., 2016b) were used following both the self-attention layer and the MLP.
Revised: Residual connections and layer normalization (Ba et al., 2016b) were used following both the self-attention layer and the MLP.

Original: The self-attention weights (for generating the keys, queries, and values) were initialized using an Xavier normal distribution.
Revised: The self-attention weights (for generating the keys, queries, and values) were initialized using an Xavier normal distribution.

Original: The MLP weights were initialized using a Kaiming normal distribution.
Revised: The MLP weights were initialized using a Kaiming normal distribution.

Original: After applying the Transformer layer, the (transformed) embeddings were averaged and passed to an output MLP.
Revised: After applying the Transformer layer, the (transformed) embeddings were averaged and passed to an output MLP.

Original: The output MLP had a single hidden layer with 256 units, and an output layer for generating ˆ y .
Revised: The output MLP had a single hidden layer with 256 units, and an output layer for generating ˆ y .

Original: The hidden layer used ReLU nonlinearities, and the weights were initialized using a
Revised: The hidden layer used ReLU nonlinearities, and the weights were initialized using a

Original: Kaiming normal distribution.
Revised: Kaiming normal distribution.

Original: All biases were initialized to 0 .
Revised: All biases were initialized to 0 .

Original: A.2. P REDI N ET
Revised: A.3. P REDI N ET

Original: The PrediNet implementation was as close as possible to the model described in the original work (Shanahan et al., 2019), except that the multi-head attention was applied over the 1D tem- poral sequence of image embeddings, rather than over a 2D feature map (since there was no spatial component to the tasks that we studied).
Revised: The PrediNet implementation was as close as possible to the model described in the original work (Shanahan et al., 2019), except that the multi-head attention was applied over the 1D temporal sequence of image embeddings, rather than over a 2D feature map (since there was no spatial component to the tasks that we studied).

Original: Before being passed to the PrediNet module, the image embeddings were appended with a tag (an integer from 0 to T − 1 ) indicating their temporal position.
Revised: Before being passed to the PrediNet module, the image embeddings were appended with a tag (an integer from 0 to T − 1 ) indicating their temporal position.

Original: The PrediNet module used keys of size 16 , 32 heads, and 16 relations.
Revised: The PrediNet module used keys of size 16 , 32 heads, and 16 relations.

Original: All weights in the PrediNet module were initialized using an Xavier normal distribution.
Revised: All weights in the PrediNet module were initialized using an Xavier normal distribution.

Original: The output of all PrediNet heads was concatenated and passed to an output MLP.
Revised: The output of all PrediNet heads was concatenated and passed to an output MLP.

Original: This MLP had a single hidden layer with 8 units, and an output layer for generating ˆ y .
Revised: This MLP had a single hidden layer with 8 units, and an output layer for generating ˆ y .

Original: The hidden layer used ReLU nonlinearities, and the weights were initialized using a Kaiming normal distribution.
Revised: The hidden layer used ReLU nonlinearities, and the weights were initialized using a Kaiming normal distribution.

Original: All biases were initialized to 0 .
Revised: All biases were initialized to 0 .

Original: A. T RAINING DETAILS
Revised: A. T RAINING DETAILS

Original: [Table]
Revised: [Table]

Original: All models were trained with a batch size of 32 using the ADAM optimizer (Kingma & Ba, 2014).
Revised: All models were trained with a batch size of 32 using the ADAM optimizer (Kingma & Ba, 2014).

Original: The learning rate for all models trained with TCN was 5 e − 4 .
Revised: The learning rate for all models trained with TCN was 5 e − 4 .

Original: Some of the models failed to converge when trained without TCN, requiring a smaller learning rate of 5 e − 5 .
Revised: Some of the models failed to converge when trained without TCN, requiring a smaller learning rate of 5 e − 5 .

Original: The learning rates used for all models when trained without TCN are shown in Table 5.
Revised: The learning rates used for all models when trained without TCN are shown in Table 5.

Original: Because different generalization regimes (different values for m ) involved different training set sizes, and therefore involved fewer training updates per epoch, the number of training epochs required to reliably achieve convergence varied based on the regime.
Revised: Because different generalization regimes (different values for m ) involved different training set sizes, and therefore involved fewer training updates per epoch, the number of training epochs required to reliably achieve convergence varied based on the regime.

Original: The default number of training epochs for all tasks and regimes is shown in Table 6.
Revised: The default number of training epochs for all tasks and regimes is shown in Table 6.

Original: [Table]

Original: Some models required additional training on some tasks to reach convergence.
Revised: Some models required additional training on some tasks to reach convergence.

Original: The PrediNet and the RN required longer training on the distribution-of-three task (Table 7), and the PrediNet, RN, and Transformer required longer training on the identity rules task (Table 8).
Revised: The PrediNet and the RN required longer training on the distribution-of-three task (Table 7), and the PrediNet, RN, and Transformer required longer training on the identity rules task (Table 8).

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Table]
Revised: [Table]

Original: When training the RN on larger datasets for the distribution-of-three and identity rules tasks, the same learning rate and number of training epochs as used when training on smaller datasets was sufﬁcient to reach convergence.
Revised: When training the RN on larger datasets for the distribution-of-three and identity rules tasks, the same learning rate and number of training epochs as used when training on smaller datasets was sufﬁcient to reach convergence.

Original: A. S UPPLEMENTARY RESULTS A.4. R ESULTS WITH AND WITHOUT TCN
Revised: A. S UPPLEMENTARY RESULTS A.5. R ESULTS WITH AND WITHOUT TCN

Original: Tables 9 - 12 show the results for all models trained both with and without TCN.
Revised: Tables 9 - 12 show the results for all models trained both with and without TCN.

Original: With the exception of the PrediNet on the same/different task, every model beneﬁted on every task from the incorporation of TCN, in many cases substantially.
Revised: With the exception of the PrediNet on the same/different task, every model beneﬁted on every task from the incorporation of TCN, in many cases substantially.

Original: Results for models trained with TCN (indicated by ‘+
Revised: Results for models trained with TCN (indicated by ‘+

Original: TCN’) correspond to the results presented in Figure 3 (except for the results of the PrediNet on the same/different task, for which the version of the model trained without TCN is plotted in Figure 3).
Revised: TCN’) correspond to the results presented in Figure 3 (except for the results of the PrediNet on the same/different task, for which the version of the model trained without TCN is plotted in Figure 3).

Original: We note that, even with a lower learning rate of 5 e − 5 , some models failed to converge without TCN, such as the ESBN on the same/different task, or the RN on the RMTS task.
Revised: We note that, even with a lower learning rate of 5 e − 5 , some models failed to converge without TCN, such as the ESBN on the same/different task, or the RN on the RMTS task.

Original: It is possible that some of these models might have performed better if we had optimized them further by training for longer or trying different learning rates, but we opted not to do that since TCN was so effective across all of the models and tasks that we studied.[Table] [Table] ESBN +
Revised: It is possible that some of these models might have performed better if we had optimized them further by training for longer or trying different learning rates, but we opted not to do that since TCN was so effective across all of the models and tasks that we studied.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 53.2 ± 1.

Original: 50.5 ± 0.

Original: 52.3 ± 0. [Table] [Table] [Table]
Revised: [Table]

Original: A.4.

Original: [Table]
Revised: [Table]

Original:  P ERFORMANCE OF RN ON TERNARY RELATIONS
Revised: A.5. P ERFORMANCE OF RN ON TERNARY RELATIONS

Original: Table 13 shows the results for the RN (w/ TCN) on the distribution-of-three and identity rules tasks when trained on larger training sets ( 10 5 instead of 10 4 training examples).
Revised: Table 13 shows the results for the RN (w/ TCN) on the distribution-of-three and identity rules tasks when trained on larger training sets ( 10 5 instead of 10 4 training examples).

Original: These results show that with more training data, the RN, which is biased toward processing pair-wise relations, is able to learn these tasks (which are based on ternary relations) in a manner that enables some degree of generalization.
Revised: These results show that with more training data, the RN, which is biased toward processing pair-wise relations, is able to learn these tasks (which are based on ternary relations) in a manner that enables some degree of generalization.

Original: Note that these results do not include the m = 95 regime, because there are not enough images in that regime to create larger training sets than were originally used
Revised: Note that these results do not include the m = 95 regime, because there are not enough images in that regime to create larger training sets than were originally used.

Original: [Table]
Revised: [Table]

Original: [Table] We also tested the TRN, which incorporates ternary relations through subsampling [Table] on these tasks (with the standard training set size of 10 4 training examples).
Revised: We also tested the TRN, which incorporates ternary relations through subsampling, on these tasks (with the standard training set size of 10 4 training examples).

Original: Table 14 shows the results.
Revised: Table 14 shows the results.

Original: This yielded a slight improvement over the RN (when trained on 10 4 training examples) [Table] though not as much of an improvement as resulted from training the RN with a larger training set.
Revised: This yielded a slight improvement over the RN (when trained on 10 4 training examples), though not as much of an improvement as resulted from training the RN with a larger training set.

Original: This result may seem surprising given that the TRN explicitly incorporates ternary relations.
Revised: This result may seem surprising given that the TRN explicitly incorporates ternary relations.

Original: We note two possible explanations for this result:
Revised: We note two possible explanations for this result:

Original: The systematic comparison of every pair of objects, including permutations and comparisons of each object with itself, allows the RN to take advantage of a very powerful form of data augmentation, enforcing a certain degree of systematicity in the relations that it learns.
Revised: The systematic comparison of every pair of objects, including permutations and comparisons of each object with itself, allows the RN to take advantage of a very powerful form of data augmentation, enforcing a certain degree of systematicity in the relations that it learns.

Original: By only considering temporally ordered and non-redundant sets, the TRN is not able to take advantage of this to the same extent, and therefore might not learn relations that generalize as well.
Revised: By only considering temporally ordered and non-redundant sets, the TRN is not able to take advantage of this to the same extent, and therefore might not learn relations that generalize as well.

Original: The distribution-of-three and identity rules tasks both involve not only ternary sets, but the higher-order comparison of multiple pairs of ternary sets (the ﬁrst row vs. the combination of the second row with each candidate answer).
Revised: The distribution-of-three and identity rules tasks both involve not only ternary sets, but the higher -order comparison of multiple pairs of ternary sets (the ﬁrst row vs. the combination of the second row with each candidate answer).

Original: One could presumably engineer a solution to this problem within the RN framework, but we take it as a strength of the ESBN that no such special engineering is necessary in this case.
Revised: One could presumably engineer a solution to this problem within the RN framework, but we take it as a strength of the ESBN that no such special engineering is necessary in this case.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: A.5. T RAINING TIME COURSES FOR SAME / DIFFERENT TASK

Original: Figure 5 shows the training time courses for all models on the same/different task.
Revised: Figure 5 shows the training time courses for all models on the same/different task.

Original: Unlike the other three tasks we studied (for which training time courses are shown in Figure 4), all models were able to learn this task within a few hundred training updates (though all models except the ESBN failed to generalize in the most extreme regime).
Revised: Unlike the other three tasks we studied (for which training time courses are shown in Figure 4), all models were able to learn this task within a few hundred training updates (though all models except the ESBN failed to generalize in the most extreme regime).

Original: A.4. A LTERNATIVE ENCODER ARCHITECTURES
Revised: A.5. A LTERNATIVE ENCODER ARCHITECTURES

Original: In order to determine whether the systematic generalization exhibited by the ESBN depended to some extent on the convolutional layers in its encoder we performed experiments with two alternative encoder architectures: a multilayer perceptron (MLP) encoder and a random projection.
Revised: In order to determine whether the systematic generalization exhibited by the ESBN depended to some extent on the convolutional layers in its encoder, we performed experiments with two alternative encoder architectures: a multilayer perceptron (MLP) encoder, and a random projection.

Revised: [Figure]

Original: The MLP encoder consisted of 3 fully-connected layers, with 512 ,  and 128 units, each of which used ReLU nonlinearities.
Revised: The MLP encoder consisted of 3 fully-connected layers, with 512 , 256 , and 128 units, each of which used ReLU nonlinearities.

Original: All weights were initialized using a Kaiming normal distribution and all biases were set to 0 .
Revised: All weights were initialized using a Kaiming normal distribution, and all biases were set to 0 .

Original: The random projection encoder involved only a single, untrained, fully-connected layer that projected from the ﬂattened image to 128 units, followed by a ReLU nonlinearity.
Revised: The random projection encoder involved only a single, untrained, fully-connected layer that projected from the ﬂattened image to 128 units, followed by a ReLU nonlinearity.

Original: Weights were sampled from a Kaiming normal distribution, and biases were set to 0 .
Revised: Weights were sampled from a Kaiming normal distribution, and biases were set to 0 .

Original: Figure 6 and Tables 15 - 18 show the results for these experiments, along with the original version of the model (with a convolutional encoder) for comparison.
Revised: Figure 6 and Tables 15 - 18 show the results for these experiments, along with the original version of the model (with a convolutional encoder) for comparison.

Original: To enable a fair comparison with the original model, all experiments employed TCN.
Revised: To enable a fair comparison with the original model, all experiments employed TCN.

Original: The results show that the ESBN performed comparably well with all three of the encoder architectures.
Revised: The results show that the ESBN performed comparably well with all three of the encoder architectures.

Original: This was conﬁrmed by performing paired t-tests on the average test accuracy in each task/generalization condition (each combination[Figure] of task and value of m ) 
Revised: This was conﬁrmed by performing paired t-tests on the average test accuracy in each task/generalization condition (each combination of task and value of m ) for the MLP vs. convolutional encoder ( t = − 1 .

Original: for the MLP vs. convolutional encoder ( t = − 1 . 7 , p = 0 . 1 ) and for the random vs. convolutional encoder ( t = 1 . 6 , p = 0 . 13 ).
Revised: 7 , p = 0 . 1 ) and for the random vs. convolutional encoder ( t = 1 . 6 , p = 0 . 13 ).

Original: For comparison, we also performed experiments with these alternative encoders in the Transformer architecture.
Revised: For comparison, we also performed experiments with these alternative encoders in the Transformer architecture.

Original: These experiments revealed that, in contrast with the ESBN, the Transformer’s performance was signiﬁcantly impaired by the use of a random vs. convolutional encoder ( t = − 4 . 0 , p = 0 . 001 ), though it appeared to perform comparably well with an MLP vs. convolutional encoder ( t = − 1 . 6 , p = 0 . 14 ).[Table] Transformer
Revised: These experiments revealed that, in contrast with the ESBN, the Transformer’s performance was signiﬁcantly impaired by the use of a random vs. convolutional encoder ( t = − 4 . 0 , p = 0 . 001 ), though it appeared to perform comparably well with an MLP vs. convolutional encoder ( t = − 1 . 6 , p = 0 . 14 ).

Original: [Table]
Revised: [Table]

Original: conv)

Original: [Table]
Revised: [Table]

Original: Transformer

Original: [Table]

Original: [Table]

Original: Transformer

Original: [Table][Table] [Table] [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: A.4. C ONFIDENCE ABLATION EXPERIMENT
Revised: C ONFIDENCE ABLATION EXPERIMENT

Original: In order to determine the importance of the conﬁdence values appended to retrieved memories, we tested a version of the ESBN without these conﬁdence values.
Revised: In order to determine the importance of the conﬁdence values appended to retrieved memories [Table] we tested a version of the ESBN without these conﬁdence values.

Original: These results are shown in Tableand Figure 7.
Revised: These results are shown in Tableand Figure 7.

Original: The ablation of conﬁdence values prevented the ESBN from being able to perform the same/different task at all, and resulted in much slower training on the RMTS task.
Revised: The ablation of conﬁdence values prevented the ESBN from being able to perform the same/different task at all, and resulted in much slower training on the RMTS task.

Original: By contrast, ablation of conﬁdence values did not affect performance, either in terms of generalization or training time, for the distribution-of-three or identity rules tasks.
Revised: By contrast, ablation of conﬁdence values did not affect performance, either in terms of generalization or training time, for the distribution-of-three or identity rules tasks.

Original: This can be explained by the fact that these tasks only require the retrieval of the best match from memory, whereas the same/different and
Revised: This can be explained by the fact that these tasks only require the retrieval of the best match from memory, whereas the same/different and

Original: RMTS tasks require the model to know how good of a match the best match is, which is precisely the information conveyed by conﬁdence values. 
Revised: RMTS tasks require the model to know how good of a match the best match is, which is precisely the information conveyed by conﬁdence values. [Table] [Figure]

Original: It is also worth noting one potential alternative to an explicit, inbuilt conﬁdence value.
Revised: It is also worth noting one potential alternative to an explicit, inbuilt conﬁdence value.

Original: In our implementation, the ESBN’s memory is empty at the beginning of each sequence that it processes.
Revised: In our implementation, the ESBN’s memory is empty at the beginning of each sequence that it processes.

Original: However, when multiple entries are present in memory, as will generally be the case in realistic, temporally extended settings, the presentation of a previously unseen item will result in the retrieval of a mixture of (weakly matched) memories.
Revised: However, when multiple entries are present in memory, as will generally be the case in realistic, temporally extended settings, the presentation of a previously unseen item will result in the retrieval of a mixture of (weakly matched) memories.

Original: This mixed representation can therefore serve as a reliable cue for the degree to which the current percept matches a stored memory, obviating the need for an explicit conﬁdence value.
Revised: This mixed representation can therefore serve as a reliable cue for the degree to which the current percept matches a stored memory, obviating the need for an explicit conﬁdence value.

Original: To demonstrate this, we implemented a version of the ESBN that begins each sequence with a single, learned key/value entry stored in memory (initialized to 0 at the beginning of training).
Revised: To demonstrate this, we implemented a version of the ESBN that begins each sequence with a single, learned key/value entry stored in memory (initialized to 0 at the beginning of training).

Original: Table 20 shows that this approach allows the ESBN to learn and perfectly generalize on the same/different task.
Revised: Table 20 shows that this approach allows the ESBN to learn and perfectly generalize on the same/different task.

Original: Figure 8 shows that this approach allows the ESBN to retain the short training time of the original model on the RMTS task.[Equation] [Table] [Figure]
Revised: Figure 8 shows that this approach allows the ESBN to retain the short training time of the original model on the RMTS task.

Original: [Figure] [Figure] [Figure] [Figure]
Revised: [Table]

Original: RMTS

Original: [Figure]
Revised: [Figure]

Original: Identity rules

Original: [Figure] [Table] A. A NALYSIS OF LEARNED REPRESENTATIONS To better understand how the ESBN works, we performed an analysis of the representations that it learned on the distribution-of-three task.
Revised: To better understand how the ESBN works, we performed an analysis of the representations that it learned on the distribution-of-three task.

Original: Speciﬁcally, we performed an analysis of a network trained on the most difﬁcult generalization regime ( m = 95 ), by performing principal component analysis (PCA) on all key vectors written to and retrieved from memory for both the training and test sets, and visualizing these vectors along the ﬁrst two principal components.
Revised: Speciﬁcally, we performed an analysis of a network trained on the most difﬁcult generalization regime ( m = 95 ), by performing principal component analysis (PCA) on all key vectors written to and retrieved from memory for both the training and test sets, and visualizing these vectors along the ﬁrst two principal components.

Original: First, we looked at the keys that were written to memory ( k w ).
Revised: First, we looked at the keys that were written to memory ( k w ).

Original: We found that the keys for the ﬁrst three time steps were tightly clustered, whereas the keys for the subsequent time steps (4-9) were more diffuse (Figures 9a and 9b).
Revised: We found that the keys for the ﬁrst three time steps were tightly clustered, whereas the keys for the subsequent time steps (4-9) were more diffuse (Figures 9a and 9b).

Original: This makes sense because, in the distribution-of-three task, the
Revised: This makes sense because, in the distribution-of-three task, the

Original: ESBN only needs to be able to reliably retrieve what it wrote during the ﬁrst three time steps (when the objects in the ﬁrst row were presented).
Revised: ESBN only needs to be able to reliably retrieve what it wrote during the ﬁrst three time steps (when the objects in the ﬁrst row were presented).

Original: For time steps 4-9, the only important consideration is that the keys written to memory not overlap with those written during the ﬁrst three time steps, which also appears to be the case
Revised: For time steps 4-9, the only important consideration is that the keys written to memory not overlap with those written during the ﬁrst three time steps, which also appears to be the case.

Original: Second, we compared the keys written to memory for the ﬁrst three time steps in the training vs. test sets (Figure 9b).
Revised: Second, we compared the keys written to memory for the ﬁrst three time steps in the training vs. test sets (Figure 9b).

Original: This revealed that, for a given time step, the keys written to memory in the training vs. test sets were remarkably similar (so much so that they are completely overlapping for time steps 1 and 2).
Revised: This revealed that, for a given time step, the keys written to memory in the training vs. test sets were remarkably similar (so much so that they are completely overlapping for time steps 1 and 2).

Original: Third, we looked at the keys that were retrieved from memory following the second appearance of the objects that appeared on time steps 1-3.
Revised: Third, we looked at the keys that were retrieved from memory following the second appearance of the objects that appeared on time steps 1-3.

Original: We found that 1) these closely matched the distribution of keys written to memory during time steps 1-3, and 2) these were highly overlapping for the training vs. test sets (Figure 9c).[Figure]
Revised: We found that 1) these closely matched the distribution of keys written to memory during time steps 1-3, and 2) these were highly overlapping for the training vs. test sets (Figure 9c).

Original: Taken together, these results help to explain why the ESBN was so successful in this generalization regime, despite the very small degree of overlap between the distribution of training and test images.
Revised: Taken together, these results help to explain why the ESBN was so successful in this generalization regime, despite the very small degree of overlap between the distribution of training and test images.

Original: Because the ESBN’s controller was relatively isolated from the part of the model that deals with image embeddings, it was able to learn to encode abstract symbol-like representations (such as ‘ﬁrst image’, ‘second image’, and ‘third image’), that did not depend on the identity of the images.
Revised: Because the ESBN’s controller was relatively isolated from the part of the model that deals with image embeddings, it was able to learn to encode abstract symbol-like representations (such as ‘ﬁrst image’, ‘second image’, and ‘third image’), that did not depend on the identity of the images.

Original: Then, when queried with an image, was able to successfully retrieve the image’s corresponding abstract encoding, even when that image was quite different than those observed during training.
Revised: Then, when queried with an image, was able to successfully retrieve the image’s corresponding abstract encoding, even when that image was quite different than those observed during training.

Original: That is, the model learned representations to use as keys that could be used for binding and indirection in the same way that symbols are used in traditional computational architectures.[Figure]
Revised: That is, the model learned representations to use as keys that could be used for binding and indirection in the same way that symbols are used in traditional computational architectures.

Original: . U NICODE CHARACTERS
Revised: A. U NICODE CHARACTERS

Original: Figure 10 shows all 100 images that were used to construct the abstract rule learning tasks.[Figure]
Revised: Figure 10 shows all 100 images that were used to construct the abstract rule learning tasks.