Original: A N E XPLICITLY R ELATIONAL N EURAL N ETWORK A RCHITECTURE
Revised: A N E XPLICITLY R ELATIONAL N EURAL N ETWORK A RCHITECTURE

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: With a view to bridging the gap between deep learning and symbolic AI, we present a novel end-to-end neural network architecture that learns to form propositional representations with an explicitly relational structure from raw pixel data.
Revised: With a view to bridging the gap between deep learning and symbolic AI, we present a novel end-to-end neural network architecture that learns to form propositional representations with an explicitly relational structure from raw pixel data.

Original: In order to evaluate and analyse the architecture, we introduce a family of simple visual relational reasoning tasks of varying complexity.
Revised: In order to evaluate and analyse the architecture, we introduce a family of simple visual relational reasoning tasks of varying complexity.

Original: We show that the proposed architecture, when pre-trained on a curriculum of such tasks, learns to generate reusable representations that better facilitate subsequent learning on previously unseen tasks when compared to a number of baseline architectures.
Revised: We show that the proposed architecture, when pre-trained on a curriculum of such tasks, learns to generate reusable representations that better facilitate subsequent learning on previously unseen tasks when compared to a number of baseline architectures.

Original: The workings of a successfully trained model are visualised to shed some light on how the architecture functions
Revised: The workings of a successfully trained model are visualised to shed some light on how the architecture functions

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: When humans face novel problems, they are able to draw effectively on past experience with other problems that are superﬁcially very different, but that have similarities on a more abstract, structural level.
Revised: When humans face novel problems, they are able to draw effectively on past experience with other problems that are superﬁcially very different, but that have similarities on a more abstract, structural level.

Original: This ability is essential for lifelong, continual learning, and confers on humans a degree of data efﬁciency, powers of transfer learning, and a capacity for out-of-distribution generalisation that contemporary machine learning has yet to match (Garnelo et al., 2016;
Revised: This ability is essential for lifelong, continual learning, and confers on humans a degree of data efﬁciency, powers of transfer learning, and a capacity for out-of-distribution generalisation that contemporary machine learning has yet to match (Garnelo et al., 2016;

Original: Lake et al., 2017; Marcus, 2018;
Revised: Lake et al., 2017; Marcus, 2018;

Original: Smith, 2019).
Revised: Smith, 2019).

Original: A case may be made that all these issues are different facets of the same underlying challenge, namely the challenge of devising systems that learn to construct general-purpose, reusable representations (McCarthy, 1987; Bengio et al., 2013).
Revised: A case may be made that all these issues are different facets of the same underlying challenge, namely the challenge of devising systems that learn to construct general-purpose, reusable representations (McCarthy, 1987; Bengio et al., 2013).

Original: A representation is general-purpose and reusable to the extent that it contains information whose domain of application exceeds the context within which it was acquired.
Revised: A representation is general-purpose and reusable to the extent that it contains information whose domain of application exceeds the context within which it was acquired.

Original: Representations that are general-purpose and reusable improve data efﬁciency because a system that already knows how to build representations relevant to a novel task (despite its novelty) doesn’t have to learn that task from scratch.
Revised: Representations that are general-purpose and reusable improve data efﬁciency because a system that already knows how to build representations relevant to a novel task (despite its novelty) doesn’t have to learn that task from scratch.

Original: Ideally, a system that efﬁciently exploits general-purpose, reusable representations in this way should be the very same system that learned how to construct them in the ﬁrst place.
Revised: Ideally, a system that efﬁciently exploits general-purpose, reusable representations in this way should be the very same system that learned how to construct them in the ﬁrst place.

Original: Moreover, in learning to solve a novel task using such representations, we should expect the system to learn further representations that are themselves general-purpose and reusable.
Revised: Moreover, in learning to solve a novel task using such representations, we should expect the system to learn further representations that are themselves general-purpose and reusable.

Original: So, with the exception of the very ﬁrst representations the system learns, all learning in such a system would in effect be transfer learning, and the process of learning would be inherently cumulative, continual, and lifelong.
Revised: So, with the exception of the very ﬁrst representations the system learns, all learning in such a system would in effect be transfer learning, and the process of learning would be inherently cumulative, continual, and lifelong.

Original: One approach to building such a system is to take inspiration from the paradigm of classical, symbolic
Revised: One approach to building such a system is to take inspiration from the paradigm of classical, symbolic

Original: AI (Garnelo & Shanahan, 2019).
Revised: AI (Garnelo & Shanahan, 2019).

Original: Building on the mathematical foundations of ﬁrst-order predicate calculus, a typical symbolic AI system works by applying logic-like rules of inference to language-like propositional representations whose elements are objects and relations.
Revised: Building on the mathematical foundations of ﬁrst-order predicate calculus, a typical symbolic AI system works by applying logic-like rules of inference to language-like propositional representations whose elements are objects and relations.

Original: Thanks to their declarative character and compositional structure, these representations lend themselves naturally to generality and reusability.
Revised: Thanks to their declarative character and compositional structure, these representations lend themselves naturally to generality and reusability.

Original: However, in contrast to contemporary deep learning systems, the representations deployed in classical AI are not usually learned from data but hand-crafted (Harnad, 1990).
Revised: However, in contrast to contemporary deep learning systems, the representations deployed in classical AI are not usually learned from data but hand-crafted (Harnad, 1990).

Original: The aim of the present work is to get the best of both worlds with an end-to-end differentiable neural network architecture that builds in propositional, relational priors in much the same way that a convolutional network builds in spatial and locality priors.
Revised: The aim of the present work is to get the best of both worlds with an end-to-end differentiable neural network architecture that builds in propositional, relational priors in much the same way that a convolutional network builds in spatial and locality priors.

Original: The architecture introduced here builds on recent work with non-local network architectures that learn to discover and exploit relational information (Wang et al., 2018), notably relation nets (Santoro et al., 2017; Palm et al., 2018) and architectures based on multi-head attention (Vaswani et al., 2017;
Revised: The architecture introduced here builds on recent work with non-local network architectures that learn to discover and exploit relational information (Wang et al., 2018), notably relation nets (Santoro et al., 2017; Palm et al., 2018) and architectures based on multi-head attention (Vaswani et al., 2017;

Original: A
Revised: A

Original: B
Revised: B

Original: C
Revised: C

Original: Means C=AB
Revised: Means C=AB

Original: W K
Revised: W K

Original: K n x m
Revised: K n x m

Original: L
Revised: L

Original: W Q g
Revised: W Q g

Original: L K T mn flatten(L) flatten(L)
Revised: L K T mn flatten(L) flatten(L)

Original: mn x g
Revised: mn x g

Original: W Q
Revised: W Q

Original: L g x n
Revised: L g x n

Original: K T softmax(Q 1 K T )
Revised: K T softmax(Q 1 K T )

Original: softmax(Q 2 K T ) m x g n x g mn mn
Revised: softmax(Q 2 K T ) m x g n x g mn mn

Original: x g g g x n g g n x m n x m k heads j comparator
Revised: x g g g x n g g n x m n x m k heads j comparator

Original: W S m x j
Revised: W S m x j

Original: W S m j
Revised: W S m j

Original: E 1 E
Revised: E 1 E

Original: D j+
Revised: D j+

Original: R*
Revised: R*

Original: Q Q
Revised: Q Q

Original: P 1 P
Revised: P 1 P

Original: P 1 P 2 j
Revised: P 1 P 2 j

Original: Santoro et al., 2018; Zambaldi et al., 2019).
Revised: Santoro et al., 2018; Zambaldi et al., 2019).

Original: However, these architectures generate representations that lack explicit structure.
Revised: However, these architectures generate representations that lack explicit structure.

Original: There is, in general, no straightforward mapping from the parts of a representation to the usual elements of a symbolic medium such as predicate calculus: propositions, relations, and objects.
Revised: There is, in general, no straightforward mapping from the parts of a representation to the usual elements of a symbolic medium such as predicate calculus: propositions, relations, and objects.

Original: To the extent that these elements are present, they are smeared across the embedding vector, which makes representations hard to interpret and makes it more difﬁcult for downstream processing to take advantage of compositionality.
Revised: To the extent that these elements are present, they are smeared across the embedding vector, which makes representations hard to interpret and makes it more difﬁcult for downstream processing to take advantage of compositionality.

Original: Here we present an architecture, which we call a PrediNet, that learns representations whose parts map directly onto propositions, relations, and objects.
Revised: Here we present an architecture, which we call a PrediNet, that learns representations whose parts map directly onto propositions, relations, and objects.

Original: To build a sound, scientiﬁc understanding of the proposed architecture, and to facilitate a detailed comparison with other architectures, the present study focuses on simple tasks requiring relatively little data and computation.
Revised: To build a sound, scientiﬁc understanding of the proposed architecture, and to facilitate a detailed comparison with other architectures, the present study focuses on simple tasks requiring relatively little data and computation.

Original: We develop a family of small, simple visual datasets that can be combined into a variety of multi-task curricula and used to assess the extent to which an architecture learns representations that are general-purpose and reusable.
Revised: We develop a family of small, simple visual datasets that can be combined into a variety of multi-task curricula and used to assess the extent to which an architecture learns representations that are general-purpose and reusable.

Original: We report the results of a number of experiments using these datasets that demonstrate the potential of an explicitly relational network architecture to improve data efﬁciency and generalisation, to facilitate transfer, and to learn reusable representations.
Revised: We report the results of a number of experiments using these datasets that demonstrate the potential of an explicitly relational network architecture to improve data efﬁciency and generalisation, to facilitate transfer, and to learn reusable representations.

Original: The main contribution of the present paper is a novel architecture that learns to discover objects and relations in high-dimensional data, and to represent them in a form that is beneﬁcial for downstream processing.
Revised: The main contribution of the present paper is a novel architecture that learns to discover objects and relations in high-dimensional data, and to represent them in a form that is beneﬁcial for downstream processing.

Original: The PrediNet architecture does not itself carry out logical inference, but rather extracts relational structure from raw data that has the potential to be exploited by subsequent processing.
Revised: The PrediNet architecture does not itself carry out logical inference, but rather extracts relational structure from raw data that has the potential to be exploited by subsequent processing.

Original: Here, for the purpose of evaluation, we graft a simple multi-layer perceptron output module to the
Revised: Here, for the purpose of evaluation, we graft a simple multi-layer perceptron output module to the

Original: PrediNet and train it on a simple set of spatial reasoning problems.
Revised: PrediNet and train it on a simple set of spatial reasoning problems.

Original: The aim is to acquire a sufﬁcient scientiﬁc understanding of the architecture and its properties in this minimalist setting before applying it to more complex problems using more sophisticated forms of downstream inference.
Revised: The aim is to acquire a sufﬁcient scientiﬁc understanding of the architecture and its properties in this minimalist setting before applying it to more complex problems using more sophisticated forms of downstream inference.

Original:  T HE P REDI N ET A RCHITECTURE
Revised:  T HE P REDI N ET A RCHITECTURE

Original: The idea that propositions are the building blocks of knowledge dates back to the ancient Greeks, and provides the foundation for symbolic AI, via the 19 th century mathematical work of Boole and
Revised: The idea that propositions are the building blocks of knowledge dates back to the ancient Greeks, and provides the foundation for symbolic AI, via the 19 th century mathematical work of Boole and

Original: Frege (Russell & Norvig, 2009).
Revised: Frege (Russell & Norvig, 2009).

Original: An elementary proposition asserts that a relationship holds between a set of objects.
Revised: An elementary proposition asserts that a relationship holds between a set of objects.

Original: Propositions can be combined using logical connectives (and, or, not, etc), and can participate in inference processes such as deduction.
Revised: Propositions can be combined using logical connectives (and, or, not, etc), and can participate in inference processes such as deduction.

Original: The task of the PrediNet is to (learn to) transform high-dimensional data such as images into propositional representations that are useful for downstream processing.
Revised: The task of the PrediNet is to (learn to) transform high-dimensional data such as images into propositional representations that are useful for downstream processing.

Original: A PrediNet module (Fig. 1) can be thought of as a pipeline comprising three stages: attention , binding , and evaluation .
Revised: A PrediNet module (Fig. 1) can be thought of as a pipeline comprising three stages: attention , binding , and evaluation .

Original: The attention stage selects pairs of objects of interest, the binding stage instantiates the ﬁrst two arguments of a set of three-place predicates (relations) with selected object pairs, and the evaluation stage computes values for each predicate’s remaining (scalar) argument such that the resulting proposition is true.
Revised: The attention stage selects pairs of objects of interest, the binding stage instantiates the ﬁrst two arguments of a set of three-place predicates (relations) with selected object pairs, and the evaluation stage computes values for each predicate’s remaining (scalar) argument such that the resulting proposition is true.

Original: More precisely, a PrediNet module comprises k heads, each of which computes j relations between pairs of objects (Fig. 1).
Revised: More precisely, a PrediNet module comprises k heads, each of which computes j relations between pairs of objects (Fig. 1).

Original: The input to the PrediNet is a matrix, L , comprising n rows of feature vectors, where each feature vector has length m .
Revised: The input to the PrediNet is a matrix, L , comprising n rows of feature vectors, where each feature vector has length m .

Original: In the present work, L is computed by a convolutional neural network (CNN).
Revised: In the present work, L is computed by a convolutional neural network (CNN).

Original: The CNN outputs a feature map consisting of n feature vectors that tile the input image.
Revised: The CNN outputs a feature map consisting of n feature vectors that tile the input image.

Original: The last two elements of the feature vector are the xy co-ordinates of the associated patch in the image.
Revised: The last two elements of the feature vector are the xy co-ordinates of the associated patch in the image.

Original: So the length m of each feature vector corresponds to the number of ﬁlters in the ﬁnal
Revised: So the length m of each feature vector corresponds to the number of ﬁlters in the ﬁnal

Original: CNN layer plus 2 (for the co-ordinates), and the i th element of a feature vector (for i < m − 2 ) is the output of the i th ﬁlter.
Revised: CNN layer plus 2 (for the co-ordinates), and the i th element of a feature vector (for i < m − 2 ) is the output of the i th ﬁlter.

Original: For a given input L , each head h computes the same set of relations (using shared weights W S ) but selects a different pair of objects, using dot-product attention based on key-query matching (Vaswani et al., 2017).
Revised: For a given input L , each head h computes the same set of relations (using shared weights W S ) but selects a different pair of objects, using dot-product attention based on key-query matching (Vaswani et al., 2017).

Original: Each head computes a separate pair of queries Q h 1 and
Revised: Each head computes a separate pair of queries Q h 1 and

Original: Q h 2 (via W hQ 1 and W hQ 2 ), but the key space K (deﬁned by W K ) is shared between heads.
Revised: Q h 2 (via W hQ 1 and W hQ 2 ). But the key space K (deﬁned by W K ) is shared, so that the set of entities that are candidates for attention is consistent across heads.

Revised: The whole (ﬂattened) image is used to generate queries, allowing attention masks to depend on the image’s full (non-local) content.

Original: [Equation]
Revised: [Equation]

Original: Applying the resulting pair of attention masks directly to L yields a pair of objects E h 1 and E h 2 , each represented by a weighted sum of feature vectors.
Revised: Applying the resulting pair of attention masks directly to L yields a pair of objects E h 1 and E h 2 , each represented by a weighted sum of feature vectors.

Original: [Equation]
Revised: [Equation]

Original: All j relations between E h 1 and E h 2 are then evaluated.
Revised: All j relations between E h 1 and E h 2 are then evaluated.

Original: There are many ways to compute a relationship between a pair of objects represented as feature vectors.
Revised: There are many ways to compute a relationship between a pair of objects represented as feature vectors.

Original: We chose to compute the values of relations by taking vector differences, which has been shown to be effective in the context of relationally structured knowledge bases (Bordes et al., 2011; Socher et al., 2013).
Revised: We chose to compute the values of relations by taking vector differences, which has been shown to be effective in the context of relationally structured knowledge bases (Bordes et al., 2011; Socher et al., 2013).

Original: In the current architecture, E hand E h 2 are subject to a linear mapping (via W S ) into j 1D spaces, one per relation, and the resulting vector is passed through an element-wise comparator, yielding a vector of differences D h .
Revised: In the current architecture, E hand E h 2 are subject to a linear mapping (via W S ) into j 1D spaces, one per relation, and the resulting vector is passed through an element-wise comparator, yielding a vector of differences D h .

Original: [Equation]
Revised: [Equation]

Original: The last two elements of E h 1 and E h 2 (the positions P h 1 and P h 2 , respectively) are concatenated to the vector D h of differences to give the head’s output R h
Revised: The last two elements of E h 1 and E h 2 (the positions P h 1 and P h 2 , respectively) are concatenated to the vector D h of differences to give the head’s output R h

Original: =
Revised: =

Original: ( D h , P h 1 , P h 2 ) .
Revised: ( D h , P h 1 , P h 2 ) .

Original: Finally, the outputs of all k heads are concatenated, yielding the output of the PrediNet module, a vector R ∗ of length k
Revised: Finally, the outputs of all k heads are concatenated, yielding the output of the PrediNet module, a vector R ∗ of length k

Original: ( j + 4) .
Revised: ( j + 4) .

Original: In predicate calculus terms, the ﬁnal output of a PrediNet module with k heads and j relations represents the conjunction of elementary propositions
Revised: In predicate calculus terms, the ﬁnal output of a PrediNet module with k heads and j relations represents the conjunction of elementary propositions

Original: [Equation]
Revised: [Equation]

Original: where ψ i ( d hi , e h 1 , e h 2 ) asserts that d hi is the distance between objects e h 1 and e h 2 in the 1D space deﬁned by column i of the weight matrix W S , and the denotations of e h 1 and e h 2 are captured by the vectors
Revised: where ψ i ( d hi , e h 1 , e h 2 ) asserts that d hi is the distance between objects e h 1 and e h 2 in the 1D space deﬁned by column i of the weight matrix W S , and the denotations of e h 1 and e h 2 are captured by the vectors

Original: Q h 1 and Q h 2 respectively, given the key-space deﬁned by K .
Revised: Q h 1 and Q h 2 respectively, given the key-space deﬁned by K .

Original: Equation 1 supplies a semantics for the PrediNet’s ﬁnal output vector R ∗ that maps each of its elements onto a well-deﬁned logical formula, something that cannot be claimed for other architectures, such as the relation net or multi-head attention net.
Revised: Equation 1 supplies a semantics for the PrediNet’s ﬁnal output vector R ∗ that maps each of its elements onto a well-deﬁned logical formula, something that cannot be claimed for other architectures, such as the relation net or multi-head attention net.

Original: In the experiments reported here, only R ∗ is used for downstream processing, and this vector by itself doesn’t have the logical structure described by Equation 1.
Revised: In the experiments reported here, only R ∗ is used for downstream processing, and this vector by itself doesn’t have the logical structure described by Equation 1.

Original: However, the PrediNet module can easily be extended to deliver an additional output in explicitly propositional form, with a predicate-argument structure corresponding to the
Revised: However, the PrediNet module can easily be extended to deliver an additional output in explicitly propositional form, with a predicate-argument structure corresponding to the

Original: RHS of Equation 1.
Revised: RHS of Equation 1.

Original: In the present paper, the pared-down vector form facilitates our experimental investigation, but in its explicitly propositional form, the PrediNet’s output could be piped directly to (say) a Prolog interpreter (Fig.7), to an inductive logic programming system, to a statistical relational learning system, or indeed to another differentiable neural module.
Revised: In the present paper, the pared-down vector form facilitates our experimental investigation, but in its explicitly propositional form, the PrediNet’s output could be piped directly to (say) a Prolog interpreter (Fig.7), to an inductive logic programming system, to a statistical relational learning system, or indeed to another differentiable neural module.

Original:  D ATASETS AND T ASKS
Revised:  D ATASETS AND T ASKS

Original: It would be premature to apply the PrediNet architecture to rich, complex data before we have a basic understanding of its properties and its behaviour.
Revised: It would be premature to apply the PrediNet architecture to rich, complex data before we have a basic understanding of its properties and its behaviour.

Original: To facilitate in-depth scientiﬁc study, we need small, simple datasets that allow the operation of the architecture to be examined in detail and the fundamental premises of its design to be assessed.
Revised: To facilitate in-depth scientiﬁc study, we need small, simple datasets that allow the operation of the architecture to be examined in detail and the fundamental premises of its design to be assessed.

Original: Our experimental goals in the present paper are 1) to test the hypothesis that the PrediNet architecture learns representations that are general-purpose [Figure] [Figure] Held
Revised: Our experimental goals in the present paper are 1) to test the hypothesis that the PrediNet architecture learns representations that are general-purpose and reusable, and 2) insofar as this is true, to investigate why.

Original: [Figure]

Original: out

Original: [Figure]

Original: set 1: hexominoes

Original: Held-out object set 2: stripes 5 possible row patterns [Figure] (b) AAA
Revised: Existing datasets for relational reasoning tasks, such as CLEVR (Johnson et al.

Original: ABA AAB ABB ABC between
Revised: and sort-of-CLEVR (Santoro et al.

Original: False

Original: True

Original: True

Original: True

Original: False

Original: True occurs

Original: False

Original: True

Original: False same 2: False 4: True 3: False column patterns (multi-task) AAA ABB ABA
Revised: (2017)), were ruled out because they include confounding complexities, such as occlusion and shadows or language input, and/or because they don’t lend themselves to the ﬁne-grained task-level splits we required.

Original: ABA

Original: AAB

Original: AAB =

Original: = ≠

Original: False

Original: True

Original: True match rows

Original: [Figure]

Original: and reusable, and 2) insofar as this is true, to investigate why.

Original: To do this, we devised a  conﬁgurable family of simple classiﬁcation tasks that we collectively call the Relations Game.
Revised: Consequently, we devised a new conﬁgurable family of simple classiﬁcation tasks that we collectively call the Relations Game.

Original: A Relations Game task involves the presentation of an image containing a number of objects laid out on a 3 × 3 grid, and the aim (in most tasks) is to label the image as True or False according to whether a given relationship holds among the objects in the image.
Revised: A Relations Game task involves the presentation of an image containing a number of objects laid out on a 3 × 3 grid, and the aim (in most tasks) is to label the image as True or False according to whether a given relationship holds among the objects in the image.

Original: While the elementary propositions learned by the PrediNet only assert simple relationships between pairs of entities, Relations Game tasks generally involve learning compound relations involving multiple relationships among many objects.
Revised: While the elementary propositions learned by the PrediNet only assert simple relationships between pairs of entities, Relations Game tasks generally involve learning compound relations involving multiple relationships among many objects.

Original: The objects in question are drawn from either a training set or one of two held-out sets (Fig. 2a).
Revised: The objects in question are drawn from either a training set or one of two held-out sets (Fig. 2a).

Original: None of the shapes or colours in the training set occurs in either of the held-out sets.
Revised: None of the shapes or colours in the training set occurs in either of the held-out sets.

Original: The training object set contains 8 uniformly coloured pentominoes and their rotations and reﬂections ( shapes in all) with 25 possible colours.
Revised: The training object set contains 8 uniformly coloured pentominoes and their rotations and reﬂections ( shapes in all) with 25 possible colours.

Original: The ﬁrst held-out object set contains 8 uniformly coloured hexominoes and their rotations and reﬂections (46 shapes in all) with 25 possible colours, and the second held-out object set contains only squares, but with a striped pattern of held-out colours.
Revised: The ﬁrst held-out object set contains 8 uniformly coloured hexominoes and their rotations and reﬂections (46 shapes in all) with 25 possible colours, and the second held-out object set contains only squares, but with a striped pattern of held-out colours.

Original: Each Relations Game task is tied to a given relation.
Revised: Each Relations Game task is tied to a given relation.

Original: Even with such a simple setup, the number of deﬁnable relations among all possible combinations of objects is astronomical ( 2 ( n +1) 9 for n distinct objects), although only a few of them will make intuitive sense.
Revised: Even with such a simple setup, the number of deﬁnable relations among all possible combinations of objects is astronomical ( 2 ( n +1) 9 for n distinct objects), although only a few of them will make intuitive sense.

Original: For the present study, we deﬁned a handful of intuitively meaningful relations and generated corresponding labelled datasets comprising 50% positive and 50% negative examples.
Revised: For the present study, we deﬁned a handful of intuitively meaningful relations and generated corresponding labelled datasets comprising 50% positive and 50% negative examples.

Original: A selection is shown in Fig. 2c.
Revised: A selection is shown in Fig. 2c.

Original: The ‘between’ relation holds iff the image contains three objects in a line in which the outer two objects have the same shape, orientation, and colour.
Revised: The ‘between’ relation holds iff the image contains three objects in a line in which the outer two objects have the same shape, orientation, and colour.

Original: The ‘occurs’ relation holds iff there is an object in the bottom row of three objects that has the same shape, orientation, and colour as the (single) object in the top row.
Revised: The ‘occurs’ relation holds iff there is an object in the bottom row of three objects that has the same shape, orientation, and colour as the (single) object in the top row.

Original: The ‘same’ relation holds iff the image contains two objects of the same shape, orientation, and colour.
Revised: The ‘same’ relation holds iff the image contains two objects of the same shape, orientation, and colour.

Original: In each case, we balanced the set of negative examples to ensure that “tricky” images involving pairs of objects with the same colour but different shape or the same shape but different colour occur just as frequently as those with objects that differ in both colour and shape.[Figure]
Revised: In each case, we balanced the set of negative examples to ensure that “tricky” images involving pairs of objects with the same colour but different shape or the same shape but different colour occur just as frequently as those with objects that differ in both colour and shape.

Original:  E XPERIMENTAL SETUP
Revised:  E XPERIMENTAL SETUP

Original: At the top level, each architecture we consider in this paper comprises 1) a single convolutional input layer (CNN), 2) a central module (which might be a PrediNet or a baseline), and 3) a small output multi-layer perceptron (MLP) (Fig. 3).
Revised: At the top level, each architecture we consider in this paper comprises 1) a single convolutional input layer (CNN), 2) a central module (which might be a PrediNet or a baseline), and 3) a small output multi-layer perceptron (MLP) (Fig. 3).

Original: A pair of xy co-ordinates is appended to each CNN feature vector, denoting its position in convolved image space and, where applicable, a one-hot task identiﬁer is appended to the output of the central module.
Revised: A pair of xy co-ordinates is appended to each CNN feature vector, denoting its position in convolved image space and, where applicable, a one-hot task identiﬁer is appended to the output of the central module.

Original: For most tasks, the ﬁnal output of the MLP is a one-hot label denoting True or False.
Revised: For most tasks, the ﬁnal output of the MLP is a one-hot label denoting True or False.

Original: The PrediNet was evaluated by comparing it to several baselines: two MLP baselines (MLP1 and MLP2), a relation net baseline (Santoro et al., 2017) (RN), and a multi-head attention baseline (Vaswani et al., 2017; Zambaldi et al., 2019) (MHA).
Revised: The PrediNet was evaluated by comparing it to several baselines: two MLP baselines (MLP1 and MLP2), a relation net baseline (Santoro et al., 2017) (RN), and a multi-head attention baseline (Vaswani et al., 2017; Zambaldi et al., 2019) (MHA).

Original: To facilitate a fair comparison, the top-level schematic is identical for the PrediNet and for all baselines (Fig. 3).
Revised: To facilitate a fair comparison, the top-level schematic is identical for the PrediNet and for all baselines (Fig. 3).

Original: All use the same input CNN architecture and the same output MLP architecture, and differ only in the central module.
Revised: All use the same input CNN architecture and the same output MLP architecture, and differ only in the central module.

Original: In MLP1, the central module is a single fully-connected layer with ReLu activations, while in MLP2 it has two layers.
Revised: In MLP1, the central module is a single fully-connected layer with ReLu activations, while in MLP2 it has two layers.

Original: In RN, the central module computes the set of all possible pairs of feature vectors, each of which is passed through a 2-layer MLP; the resulting vectors are then aggregated by taking their element-wise means to yield the output vector.
Revised: In RN, the central module computes the set of all possible pairs of feature vectors, each of which is passed through a 2-layer MLP; the resulting vectors are then aggregated by taking their element-wise means to yield the output vector.

Original: Finally,
Revised: Finally,

Original: MHA comprises multiple heads, each of which generates mappings from the input feature vectors to sets of keys K , queries Q , and values V , and then computes softmax( QK (cid:62) ) V .
Revised: MHA comprises multiple heads, each of which generates mappings from the input feature vectors to sets of keys K , queries Q , and values V , and then computes softmax( QK (cid:62) ) V .

Original: Each head’s output is a weighted sum of the resulting vectors, and the output of the MHA central module is the concatenation of all its heads’ outputs.
Revised: Each head’s output is a weighted sum of the resulting vectors, and the output of the MHA central module is the concatenation of all its heads’ outputs.

Original: The PrediNet used here comprises k = 32 heads and j =relations (Fig. 1).
Revised: The PrediNet used here comprises k = 32 heads and j =relations (Fig. 1).

Original: All reported experiments were carried out using stochastic gradient descent, and all results shown are averages over 10 runs.
Revised: All reported experiments were carried out using stochastic gradient descent, and all results shown are averages over 10 runs.

Original: Further experimental details are given in the Supplementary
Revised: Further experimental details are given in the Supplementary

Original: Material, which also shows results for experiments with different numbers of heads and relations, and with the Adam optimiser, all of which present qualitatively similar results. 
Revised: Material, which also shows results for experiments with different numbers of heads and relations, and with the Adam optimiser, all of which present qualitatively similar results. [Figure] [Figure] AAA

Original: To assess the generality and reusability of the representations produced by the PrediNet, we adopted a four-stage experimental protocol wherein 1) the network is pre-trained on a curriculum of one or more tasks, 2) the weights in the input CNN and PrediNet are frozen while the weights in the output
Revised: ABA AAB ABB ABC between

Revised: False

Revised: True

Revised: True

Revised: True

Revised: False

Revised: True occurs

Revised: False

Revised: True

Revised: False same 2: False 4: True 3: False column patterns (multi-task)

Revised: AAA

Revised: ABB

Revised: ABA

Revised: ABA

Revised: AAB

Revised: AAB =

Revised: = ≠

Revised: False

Revised: True

Revised: True match

Revised: [Figure]

Revised: [Figure]

Revised: [Table]

Revised: To assess the generality and reusability of the representations produced by the PrediNet

Revised: [Table]

Revised: we adopted a four-stage experimental protocol wherein 1) the network is pre-trained on a curriculum of one or more tasks, 2) the weights in the input CNN and PrediNet are frozen while the weights in the output

Original: MLP are re-initialised with random values, and 3) the network is retrained on a new target task or set of tasks (Fig. 3).
Revised: MLP are re-initialised with random values, and 3) the network is retrained on a new target task or set of tasks (Fig. 3).

Original: In step 3, only the weights in the output MLP change, so the target task can only be learned to the extent that the PrediNet delivers re-usable representations to it, representations the PrediNet has learned to produce without exposure to the target task.
Revised: In step 3, only the weights in the output MLP change, so the target task can only be learned to the extent that the PrediNet delivers re-usable representations to it, representations the PrediNet has learned to produce without exposure to the target task.

Original: To assess this, we can compare the learning curves for the target task with and without pre-training.
Revised: To assess this, we can compare the learning curves for the target task with and without pre-training.

Original: We expect pre-training to improve data efﬁciency, so we should see accuracy increasing more quickly with pre-training than without it.
Revised: We expect pre-training to improve data efﬁciency, so we should see accuracy increasing more quickly with pre-training than without it.

Original: For evidence of transfer, and to conﬁrm the hypothesis of reusability, we are also interested in the ﬁnal performance on the target task after pre-training, given that the weights of the pre-trained input CNN and PrediNet are frozen.
Revised: For evidence of transfer, and to conﬁrm the hypothesis of reusability, we are also interested in the ﬁnal performance on the target task after pre-training, given that the weights of the pre-trained input CNN and PrediNet are frozen.

Original: This measure indicates how well a network has learned to form useful representations.
Revised: This measure indicates how well a network has learned to form useful representations.

Original: The more different the target task is from the pre-training curriculum, the more impressed we should be that the network is able to learn the target task.
Revised: The more different the target task is from the pre-training curriculum, the more impressed we should be that the network is able to learn the target task.

Original: [Table]

Original:  R ESULTS
Revised:  R ESULTS

Original: As a prelude to investigating the issues of generality and reusabilty [Table] we studied the data efﬁciency of the PrediNet architecture in a single-task Relations Game setting.
Revised: As a prelude to investigating the issues of generality and reusabilty, we studied the data efﬁciency of the PrediNet architecture in a single-task Relations Game setting.

Original: Results obtained on a selection of ﬁve tasks – ‘same’, ‘between’, ‘occurs’, ‘xoccurs’, and ‘colour / shape’ – are summarised in Table 1.
Revised: Results obtained on a selection of ﬁve tasks – ‘same’, ‘between’, ‘occurs’, ‘xoccurs’, and ‘colour / shape’ – are summarised in Table 1.

Original: The ﬁrst three tasks are as described in Fig.
Revised: The ﬁrst three tasks are as described in Fig.

Original: The ‘xoccurs’ relation is similar to occurs.
Revised: The ‘xoccurs’ relation is similar to occurs.

Original: It holds iff the object in the top row occurs in the bottom row and the other two objects in the bottom row are different.
Revised: It holds iff the object in the top row occurs in the bottom row and the other two objects in the bottom row are different.

Original: The ‘colour / shape’ task involves four labels, rather than the usual two: same-shape / same-colour; different-colour / same-shape; same-colour / different shape; different-colour / different shape.
Revised: The ‘colour / shape’ task involves four labels, rather than the usual two: same-shape / same-colour; different-colour / same-shape; same-colour / different shape; different-colour / different shape.

Original: In the dataset for this task, each image contains two objects randomly placed, and one of the four labels must be assigned appropriately.
Revised: In the dataset for this task, each image contains two objects randomly placed, and one of the four labels must be assigned appropriately.

Original: Table 1 shows the accuracy obtained by each of the ﬁve architectures after 100,000 batches when tested on the two held-out object sets.
Revised: Table 1 shows the accuracy obtained by each of the ﬁve architectures after 100,000 batches when tested on the two held-out object sets.

Original: The PrediNet is the only architecture that achieves over 90% accuracy on all tasks with both held-out object sets after 100,000 batches.
Revised: The PrediNet is the only architecture that achieves over 90% accuracy on all tasks with both held-out object sets after 100,000 batches.

Original: On the ‘xoccurs’ task, the PrediNet out-performs the baselines by more than 10%, and on the ‘colour / shape’ task (where chance is 25%), it out-performs all the baselines except MHA by 25% or more.
Revised: On the ‘xoccurs’ task, the PrediNet out-performs the baselines by more than 10%, and on the ‘colour / shape’ task (where chance is 25%), it out-performs all the baselines except MHA by 25% or more.

Original: Next, using the protocol outlined in Fig.
Revised: Next, using the protocol outlined in Fig.

Original: 3, we compared the PrediNet’s ability to learn re-usable representations with each of the baselines.
Revised: 3, we compared the PrediNet’s ability to learn re-usable representations with each of the baselines.

Original: We looked at a number of combinations of target tasks and pre-training curriculum tasks.
Revised: We looked at a number of combinations of target tasks and pre-training curriculum tasks.

Original: Fig.
Revised: Fig.

Original: 4 depicts our ﬁndings for one these combinations in detail, speciﬁcally three target tasks corresponding to three of the ﬁve possible column patterns (ABA, AAB, and ABB (Fig. 2d)), and a pre-training curriculum comprising the single ‘between’ task.
Revised: 4 depicts our ﬁndings for one these combinations in detail, speciﬁcally three target tasks corresponding to three of the ﬁve possible column patterns (ABA, AAB, and ABB (Fig. 2d)), and a pre-training curriculum comprising the single ‘between’ task.

Original: The plots present learning curves for each of the ﬁve architectures at each of the four stages of the experimental protocol.
Revised: The plots present learning curves for each of the ﬁve architectures at each of the four stages of the experimental protocol.

Original: In all cases, accuracy is shown for the ‘stripes’ held-out object set (not the training set).
Revised: In all cases, accuracy is shown for the ‘stripes’ held-out object set (not the training set).

Original: Of particular interest are the (green) curves corresponding to Stage 3 of the experimental protocol.
Revised: Of particular interest are the (green) curves corresponding to Stage 3 of the experimental protocol.

Original: These show how well each architecture learns the target task(s) after the central module has been pre-trained on the curriculum task(s) and its weights are frozen.
Revised: These show how well each architecture learns the target task(s) after the central module has been [Figure] [Figure] b

Original: The PrediNet learns faster than any of the baselines, and is the only one to achieve an accuracy of 90%.
Revised: [Figure] c [Figure] between d match rows

Revised: [Figure]

Revised: e

Revised: [Figure]

Revised: pre-trained on the curriculum task(s) and its weights are frozen.

Revised: The PrediNet learns faster than any of the baselines, and is the only one to achieve an accuracy of 90%.

Original: The rapid reusability of the representations learned by both the MHA baseline and the PrediNet is noteworthy because the ‘between’ relation by itself seems an unpromising curriculum for subsequently learning the AAB and
Revised: The rapid reusability of the representations learned by both the MHA baseline and the PrediNet is noteworthy because the ‘between’ relation by itself seems an unpromising curriculum for subsequently learning the AAB and

Original: ABB column patterns.
Revised: ABB column patterns.

Original: As the (red) curve for Stage 4 of the protocol shows, the reusability of the
Revised: As the (red) curve for Stage 4 of the protocol shows, the reusability of the

Original: PrediNet’s representations cannot be accounted for by the pre-training of the input CNN alone.
Revised: PrediNet’s representations cannot be accounted for by the pre-training of the input CNN alone.

Original: Fig.
Revised: Fig.

Original: 5 shows a larger range of target task / curriculum task combinations, concentrating exclusively on the Stage 3 learning curves.
Revised: 5 shows a larger range of target task / curriculum task combinations, concentrating exclusively on the Stage 3 learning curves.

Original: Here a more complete picture emerges.
Revised: Here a more complete picture emerges.

Original: In both Fig.
Revised: In both Fig.

Original: 5a and Fig.
Revised: 5a and Fig.

Original: 5d the target task is ‘match rows’ (Fig. 2d), but they differ in their pre-training curricula.
Revised: 5d the target task is ‘match rows’ (Fig. 2d), but they differ in their pre-training curricula.

Original: The curriculum for Fig.
Revised: The curriculum for Fig.

Original: 5d is three of the ﬁve row patterns (ABA, AAB, and ABB).
Revised: 5d is three of the ﬁve row patterns (ABA, AAB, and ABB).

Original: This is the only case where the
Revised: This is the only case where the

Original: PrediNet does not learn representations that are more useful for the target task than those of all the baselines, outperforming only two of the four.
Revised: PrediNet does not learn representations that are more useful for the target task than those of all the baselines, outperforming only two of the four.

Original: However, when the curriculum is the three analogous column patterns rather than row patterns, the performance of all four baselines collapses to chance, while the PrediNet does well, attaining similar performance as for the row-based curriculum (Fig. 5a).
Revised: However, when the curriculum is the three analogous column patterns rather than row patterns, the performance of all four baselines collapses to chance, while the PrediNet does well, attaining similar performance as for the row-based curriculum (Fig. 5a).

Original: This suggests the PrediNet is able to learn representations that are orientation invariant, which aids [Figure] [Figure] e
Revised: This suggests the PrediNet is able to learn representations that are orientation invariant, which aids transfer.

Original: [Figure]

Original: a

Original: [Figure]

Original: b

Original: [Figure]

Original: c

Original: [Figure]

Original: column

Original: [Figure]

Original: between d match rows

Original: [Figure]

Original: e

Original: [Figure]

Original: transfer.

Original: This hypothesis is supported by Fig.
Revised: This hypothesis is supported by Fig.

Original: 5e, where the target tasks are all ﬁve row patterns, while the curriculum is all ﬁve column patterns.
Revised: 5e, where the target tasks are all ﬁve row patterns, while the curriculum is all ﬁve column patterns.

Original: None of the baselines is able to learn reusable representations in this context; all remain at chance, whereas the PrediNet achieves 85% accuracy. 
Revised: None of the baselines is able to learn reusable representations in this context; all remain at chance, whereas the PrediNet achieves 85% accuracy. [Figure]

Original: To better understand the operation of the PrediNet, we carried out a number of visualisations.
Revised: To better understand the operation of the PrediNet, we carried out a number of visualisations.

Original: One way to ﬁnd out what the PrediNet’s heads learn to attend is to submit images to a trained network and, for each head h , apply the two attention masks softmax( Q h 1 K (cid:62) ) and softmax( Q h 2 K (cid:62) ) to each of the n feature vectors in the convolved image L .
Revised: One way to ﬁnd out what the PrediNet’s heads learn to attend is to submit images to a trained network and, for each head h , apply the two attention masks softmax( Q h 1 K (cid:62) ) and softmax( Q h 2 K (cid:62) ) to each of the n feature vectors in the convolved image L .

Original: The resulting matrix can then be plotted as a heat map to show how attention is distrubuted over the image.
Revised: The resulting matrix can then be plotted as a heat map to show how attention is distrubuted over the image.

Original: We did this for a number of networks trained in the single-task setting.
Revised: We did this for a number of networks trained in the single-task setting.

Original: Fig.
Revised: Fig.

Original: 6a shows two examples, and the Supplementary Material contains a more extensive selection.
Revised: 6a shows two examples, and the Supplementary Material contains a more extensive selection.

Original: As we might expect, most of the attention focuses on the centres of single objects, and many of the heads pick out pairs of distinct objects in various combinations.
Revised: As we might expect, most of the attention focuses on the centres of single objects, and many of the heads pick out pairs of distinct objects in various combinations.

Original: But some heads attend to halves or corners of objects.
Revised: But some heads attend to halves or corners of objects.

Original: Although most attention is focal, whether directed at object centres or object parts, some heads exhibit diffuse attention, which is possible thanks to the soft key-query matching mechanism.
Revised: Although most attention is focal, whether directed at object centres or object parts, some heads exhibit diffuse attention, which is possible thanks to the soft key-query matching mechanism.

Original: So the PrediNet can (but isn’t forced to) treat the background as a single entity, or to treat an identical pair of objects as a single entity.
Revised: So the PrediNet can (but isn’t forced to) treat the background as a single entity, or to treat an identical pair of objects as a single entity.

Original: To gain some insight into how the PrediNet encodes relations, we carried out principal component analysis (PCA) on each head of the central module’s output vectors for a number of trained networks, again in the single-task setting (Fig. 6b).We chose the four-label ‘colour / shape’ task to train on, and [Figure] [Figure]
Revised: To gain some insight into how the PrediNet encodes relations, we carried out principal component analysis (PCA) on each head of the central module’s output vectors for a number of trained networks, again in the single-task setting (Fig. 6b).

Original: Prolog

Original: [Figure]

Original: PrediNet

Original: [Figure]

Original: in Prolog form

Original: Prolog answers

Original: [Figure]

Original: R = r0, V = -0.15,

Original: X = ob_4 R = r0,

Original: V = -0.07, X = ob_

Original: R = r3, V = 0.06,

Original: X = ob_0 R = r4, V = -0.1, X = ob_ ob_ ob_ ob_ ob_ ob_ ob_ [Figure] mapped 10,000 example images onto the ﬁrst two principal components, colouring each with their ground-truth label.
Revised: We chose the four-label ‘colour / shape’ task to train on, and mapped 10,000 example images onto the ﬁrst two principal components, colouring each with their ground-truth label.

Original: We found that, for some heads, differences in colour and shape appear to align along separate axes (Fig. 6b).
Revised: We found that, for some heads, differences in colour and shape appear to align along separate axes (Fig. 6b).

Original: This contrasts with the MHA baseline, whose heads don’t seem to individually cluster the labels in a meaningful way.
Revised: This contrasts with the MHA baseline, whose heads don’t seem to individually cluster the labels in a meaningful way.

Original: For the other baselines, which lack the multi-head organisation of the PrediNet and the MHA network, the only option is to carry out PCA on the whole output vector of the central module.
Revised: For the other baselines, which lack the multi-head organisation of the PrediNet and the MHA network, the only option is to carry out PCA on the whole output vector of the central module.

Original: Doing this, however, does not produce interpretable results for any of the architectures (Fig.S8).
Revised: Doing this, however, does not produce interpretable results for any of the architectures (Fig.S8).

Original: We also identiﬁed the heads in the PrediNet that attended to both objects in the image and found that they overlapped almost entirely with those that meaningfully clustered the labels (Fig.S10).
Revised: We also identiﬁed the heads in the PrediNet that attended to both objects in the image and found that they overlapped almost entirely with those that meaningfully clustered the labels (Fig.S10).

Original: Finally, still using the ‘shape / colour task’, we carried out an ablation study, which showed that the PrediNet is signiﬁcantly more robust than the MHA network to pruning a random subset of heads at test time.
Revised: Finally, still using the ‘shape / colour task’, we carried out an ablation study, which showed that the PrediNet is signiﬁcantly more robust than the MHA network to pruning a random subset of heads at test time.

Original: Moreover, if pruned to leave only those heads that attended to the two objects, the performance of the full network could be captured with just a handful of heads (Fig. 6c).
Revised: Moreover, if pruned to leave only those heads that attended to the two objects, the performance of the full network could be captured with just a handful of heads (Fig. 6c).

Original: Taken together, these results are suggestive of something we might term relational disentangling in the PrediNet.
Revised: Taken together, these results are suggestive of something we might term relational disentangling in the PrediNet.

Original: Finally, to ﬂesh out the claim that the PrediNet generates explicitly relational representations according to the semantics of Equation 1, we extended the PrediNet module to generate an additional output in the form of a Prolog program (Fig. 7).
Revised: Finally, to ﬂesh out the claim that the PrediNet generates explicitly relational representations according to the semantics of Equation 1, we extended the PrediNet module to generate an additional output in the form of a Prolog program (Fig. 7).

Original: This involves assigning symbolic identiﬁers 1) to each of the
Revised: This involves assigning symbolic identiﬁers 1) to each of the

Revised: [Table]

Original:  PrediNet’s j relations, and 2) to every object picked out by its k heads via the attention masks they compute.
Revised: [Figure] PrediNet’s j relations, and 2) to every object picked out by its k heads via the attention masks they compute.

Original: Then the corresponding j × k propositions can be enumerated in Prolog syntax.
Revised: Then the corresponding j × k propositions can be enumerated in Prolog syntax.

Original: Assigning symbolic identiﬁers to the relations is trivial.
Revised: Assigning symbolic identiﬁers to the relations is trivial.

Original: But because attention masks can differ slightly even when they ostensibly pick out the same region of the input image, it’s necessary to cluster them before assigning symbolic identiﬁers to the corresponding objects.
Revised: But because attention masks can differ slightly even when they ostensibly pick out the same region of the input image, it’s necessary to cluster them before assigning symbolic identiﬁers to the corresponding objects.

Original: We used mean shift clustering for this.
Revised: We used mean shift clustering for this.

Original: Fig.
Revised: Fig.

Original: 7 presents a sample of the PrediNet’s output in Prolog form, along with an example of deductive inference carried out with this program.
Revised: 7 presents a sample of the PrediNet’s output in Prolog form, along with an example of deductive inference carried out with this program.

Original: The example shown is not intended to be especially meaningful; without further analysis, we lack any intuitive understanding of the relations the PrediNet has discovered.
Revised: The example shown is not intended to be especially meaningful; without further analysis, we lack any intuitive understanding of the relations the PrediNet has discovered.

Original: But it demonstrates that the representations the PrediNet produces can be understood in predicate calculus terms, and that symbolic deductive inference is one way (though not the only way) in which they might be deployed downstream.
Revised: But it demonstrates that the representations the PrediNet produces can be understood in predicate calculus terms, and that symbolic deductive inference is one way (though not the only way) in which they might be deployed downstream.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: The need for good representations has long been recognised in AI (McCarthy, 1987; Russell & Norvig, 2009), and is fundamental to deep learning (Bengio et al., 2013).
Revised: The need for good representations has long been recognised in AI (McCarthy, 1987; Russell & Norvig, 2009), and is fundamental to deep learning (Bengio et al., 2013).

Original: The importance of reusability and abstraction, especially in the context of transfer, is emphasised by Bengio, et al.
Revised: The importance of reusability and abstraction, especially in the context of transfer, is emphasised by Bengio, et al.

Original: (Bengio et al., 2013), who argue for feature sets that are “invariant to the irrelevant features and disentangle the relevant features”.
Revised: (Bengio et al., 2013), who argue for feature sets that are “invariant to the irrelevant features and disentangle the relevant features”.

Original: Our work here shares this motivation.
Revised: Our work here shares this motivation.

Original: Other work has looked at learning representations that are disentangled at the feature level (Higgins et al., 2017a; 2018).
Revised: Other work has looked at learning representations that are disentangled at the feature level (Higgins et al., 2017a; 2018).

Original: The novelty of the PrediNet is to incorporate architectural priors that favour representations that are disentangled at the relational and propositional levels.
Revised: The novelty of the PrediNet is to incorporate architectural priors that favour representations that are disentangled at the relational and propositional levels.

Original: Previous work with relation nets and multi-head attention nets has shown how non-local information can be extracted from raw pixel data and used to solve tasks that require relational reasoning.
Revised: Previous work with relation nets and multi-head attention nets has shown how non-local information can be extracted from raw pixel data and used to solve tasks that require relational reasoning.

Original: (Santoro et al., 2017; Palm et al., 2018; Santoro et al., 2018;
Revised: (Santoro et al., 2017; Palm et al., 2018; Santoro et al., 2018;

Original: Zambaldi et al., 2019)
Revised: Zambaldi et al., 2019)

Original: But unlike the PrediNet, these networks don’t produce representations with an explicitly relational, propositional structure.
Revised: But unlike the PrediNet, these networks don’t produce representations with an explicitly relational, propositional structure.

Original: By addressing the problem of acquiring structured representations, the PrediNet complements another thread of related work, which is concerned with learning how to carry out inference with structured representations, but which assumes the job of acquiring those representations is done elsewhere (Getoor & Taskar, 2007; Battaglia et al., 2016; Rocktäschel &
Revised: By addressing the problem of acquiring structured representations, the PrediNet complements another thread of related work, which is concerned with learning how to carry out inference with structured representations, but which assumes the job of acquiring those representations is done elsewhere (Getoor & Taskar, 2007; Battaglia et al., 2016; Rocktäschel &

Original: Riedel, 2017; Evans & Grefenstette, 2018).
Revised: Riedel, 2017; Evans & Grefenstette, 2018; Dong et al., 2019).

Original: In part, the present work is motivated by the conviction that curricula will be essential to lifelong, continual learning in a future generation of RL agents if they are to exhibit more general intelligence, just as they are for human children.
Revised: In part, the present work is motivated by the conviction that curricula will be essential to lifelong, continual learning in a future generation of RL agents if they are to exhibit more general intelligence, just as they are for human children.

Original: Curricular pre-training has a decade-long pedigree in deep learning (Bengio et al., 2009).
Revised: Curricular pre-training has a decade-long pedigree in deep learning (Bengio et al., 2009).

Original: Closely related to curriculum learning is the topic of transfer (Bengio, 2012), a hallmark of general intelligence and the subject of much recent attention (Higgins et al., 2017b;
Revised: Closely related to curriculum learning is the topic of transfer (Bengio, 2012), a hallmark of general intelligence and the subject of much recent attention (Higgins et al., 2017b;

Original: Kansky et al., 2017; Schwarz et al., 2018).
Revised: Kansky et al., 2017; Schwarz et al., 2018).

Original: The PrediNet exempliﬁes a different (though not incompatible) viewpoint on curriculum learning and transfer from that usually found in the neural network literature.
Revised: The PrediNet exempliﬁes a different (though not incompatible) viewpoint on curriculum learning and transfer from that usually found in the neural network literature.

Original: Rather than (or as well as) a means to guide the network, step by step, into a favourable portion of weight space, curriculum learning is here viewed in terms of the incremental accumulation of propositional knowledge.
Revised: Rather than (or as well as) a means to guide the network, step by step, into a favourable portion of weight space, curriculum learning is here viewed in terms of the incremental accumulation of propositional knowledge.

Original: This necessitates the development of a different style of architecture, one that supports the acquisition of propositional, relational representations, which also naturally subserve transfer.
Revised: This necessitates the development of a different style of architecture, one that supports the acquisition of propositional, relational representations, which also naturally subserve transfer.

Original: Asai, whose paper was published while the present work was in progress, describes an architecture with some similarities to the PrediNet, but also some notable differences (Asai, 2019).
Revised: Asai, whose paper was published while the present work was in progress, describes an architecture with some similarities to the PrediNet, but also some notable differences (Asai, 2019).

Original: For example,
Revised: For example,

Original: Asai’s architecture assumes an input representation in symbolic form where the objects have already been segmented.
Revised: Asai’s architecture assumes an input representation in symbolic form where the objects have already been segmented.

Original: By contrast, in the present architecture, the input CNN and the PrediNet’s dotproduct attention mechanism together learn what constitutes an object.
Revised: By contrast, in the present architecture, the input CNN and the PrediNet’s dotproduct attention mechanism together learn what constitutes an object.

Original:  C ONCLUSION AND F URTHER W ORK
Revised:  C ONCLUSION AND F URTHER W ORK

Original: We have presented a neural network architecture capable, in principle, of supporting predicate logic’s powers of abstraction without compromising the ideal of end-to-end learning, where the network itself discovers objects and relations in the raw data and thus avoids the symbol grounding problem entailed by symbolic AI’s practice of hand-crafting representations (Harnad, 1990).
Revised: We have presented a neural network architecture capable, in principle, of supporting predicate logic’s powers of abstraction without compromising the ideal of end-to-end learning, where the network itself discovers objects and relations in the raw data and thus avoids the symbol grounding problem entailed by symbolic AI’s practice of hand-crafting representations (Harnad, 1990).

Original: Our empirical results support the view that a network architecturally constrained to learn explicitly propositional, relational representations will have beneﬁcial data efﬁciency, generalisation, and transfer properties.
Revised: Our empirical results support the view that a network architecturally constrained to learn explicitly propositional, relational representations will have beneﬁcial data efﬁciency, generalisation, and transfer properties.

Revised: Although, the present experiments don’t use the fully propositional version of the

Revised: PrediNet output, the concatenated vector form inherits many of its beneﬁcial properties, notably a degree of compositionality.

Revised: In particular, one important respect in which the PrediNet differs from other network architectures is the extent to which it canalises information ﬂow; at the core of the network, information is organised into small chunks which are processed in parallel channels that limit the ways the chunks can interact.

Revised: We believe this pressures the network to learn representations where each separate chunk of information (such as a single value in the vector R ∗ ) has independent meaning and utility.

Revised: (We see evidence of this in the relational disentanglement of Fig. 6.)

Original:  But the ﬁndings reported here are just the ﬁrst foray into unexplored architectural territory, and much work needs to be done to gauge the architecture’s full potential.
Revised: The result is a representation whose component parts are amenable to recombination, and therefore re-use in a novel task. But the ﬁndings reported here are just the ﬁrst foray into unexplored architectural territory, and much work needs to be done to gauge the architecture’s full potential.

Original: The focus of the present paper is the acquisition of propositional representations rather than their use.
Revised: The focus of the present paper is the acquisition of propositional representations rather than their use.

Original: But thanks to the structural priors of its architecture, representations generated by a PrediNet module have a natural semantics compatible with predicate calculus (Equation 1), which makes them an ideal medium for logic-like downstream processes such as rule-based deduction, causal or counterfactual reasoning, and inference to the best explanation (abduction).
Revised: But thanks to the structural priors of its architecture, representations generated by a PrediNet module have a natural semantics compatible with predicate calculus (Equation 1), which makes them an ideal medium for logic-like downstream processes such as rule-based deduction, causal or counterfactual reasoning, and inference to the best explanation (abduction).

Original: One approach here would be to stack PrediNet modules and / or make them recurrent, enabling them to carry out the sort of iterated, sequential computations required for such processes (Palm et al., 2018; Dehghani et al., 2019).
Revised: One approach here would be to stack PrediNet modules and / or make them recurrent, enabling them to carry out the sort of iterated, sequential computations required for such processes (Palm et al., 2018; Dehghani et al., 2019).

Original: Another worthwhile direction for further research would be to develop reinforcement learning (RL) agents using the PrediNet architecture.
Revised: Another worthwhile direction for further research would be to develop reinforcement learning (RL) agents using the PrediNet architecture.

Original: One form of inference of particular interest in this context is model-based prediction, which can be used to endow an RL agent with look-ahead and planning abilities (Racanière et al., 2017; Zambaldi et al., 2019).
Revised: One form of inference of particular interest in this context is model-based prediction, which can be used to endow an RL agent with look-ahead and planning abilities (Racanière et al., 2017; Zambaldi et al., 2019).

Original: Our expectation is that RL agents in which explicitly propositional, relational representations underpin these capacities will manifest more of the beneﬁcial data efﬁciency, generalisation, and transfer properties suggested by the present results.
Revised: Our expectation is that RL agents in which explicitly propositional, relational representations underpin these capacities will manifest more of the beneﬁcial data efﬁciency, generalisation, and transfer properties suggested by the present results.

Original: As a stepping stone to such RL agents, the Relations Game family of datasets could be extended into the temporal domain, and multi-task curricula developed to encourage the acquisition of temporal, as well as spatial, abstractions
Revised: As a stepping stone to such RL agents, the Relations Game family of datasets could be extended into the temporal domain, and multi-task curricula developed to encourage the acquisition of temporal, as well as spatial, abstractions.

Original: S H YPERPARAMETERS
Revised: S H YPERPARAMETERS

Original: Table S2 shows the default hyperparameters used for the experiments reported in the main text.
Revised: Table S2 shows the default hyperparameters used for the experiments reported in the main text.

Original: S S UPPLEMENTARY A NALYSIS S2. D IMENSIONALITY REDUCTION ON INTERMEDIATE REPRESENTATIONS
Revised: S S UPPLEMENTARY A NALYSIS S2. D IMENSIONALITY REDUCTION ON INTERMEDIATE REPRESENTATIONS

Original: To qualitatively assess the nature of the representations produced by each architecture
Revised: To qualitatively assess the nature of the representations produced by each architecture

Original: [Table]
Revised: [Table]

Original: we performed a dimensionality reduction analysis on the outputs of the central module of each architecture trained on the ‘colour / shape’ task.
Revised: we performed a dimensionality reduction analysis on the outputs of the central module of each architecture trained on the ‘colour / shape’ task.

Original: After training, a batch of 10,000 images (pentominoes) was passed through
Revised: After training, a batch of 10,000 images (pentominoes) was passed through

Original: [Figure]
Revised: [Figure]

Original: the network and principal component analysis (PCA) was performed on the resulting representations, which were then projected onto the two largest principal components for visualisation.
Revised: the network and principal component analysis (PCA) was performed on the resulting representations, which were then projected onto the two largest principal components for visualisation.

Original: The projected representations were then colour-coded by the labels for the corresponding images (i.e. different/same shape, different/same colour).
Revised: The projected representations were then colour-coded by the labels for the corresponding images (i.e. different/same shape, different/same colour).

Original: PCA on the full representations (concatenating the head outputs in the case of the PrediNet and MHA models) did not yield any clear clustering of representations according to the labels for any of the models (Figure S8).
Revised: PCA on the full representations (concatenating the head outputs in the case of the PrediNet and MHA models) did not yield any clear clustering of representations according to the labels for any of the models (Figure S8).

Original: For the PrediNet and MHA models, we also ran separate PCAs on the output relations of each head in order to see how distributed / disentangled the representations were.
Revised: For the PrediNet and MHA models, we also ran separate PCAs on the output relations of each head in order to see how distributed / disentangled the representations were.

Original: While in the MHA model there was no evidence of clustering by label on any of the heads, reﬂecting a heavily distributed representation, there were several heads in the PrediNet architecture that individually clustered the different labels (Figure S9).
Revised: While in the MHA model there was no evidence of clustering by label on any of the heads, reﬂecting a heavily distributed representation, there were several heads in the PrediNet architecture that individually clustered the different labels (Figure S9).

Original: In some heads, colour and shape seemed to be projected along separate axes (e.g. heads 5, 26, and 27), while in others objects with different colours seemed to be organised in a hexagonal grid (e.g. heads 9 and 14).
Revised: In some heads, colour and shape seemed to be projected along separate axes (e.g. heads 5, 26, and 27), while in others objects with different colours seemed to be organised in a hexagonal grid (e.g. heads 9 and 14).

Original: We noted that the clustering was preserved (though slightly compressed in PC space) when the held-out set of images (hexominoes) was passed through the PrediNet and projected onto the same principal components derived using the training set.
Revised: We noted that the clustering was preserved (though slightly compressed in PC space) when the held-out set of images (hexominoes) was passed through the PrediNet and projected onto the same principal components derived using the training set.

Original: (In Section S2.2, we show that the PrediNet heads that seem to cluster the labels also attend to the two objects in the image rather than the background.)
Revised: (In Section S2.2, we show that the PrediNet heads that seem to cluster the labels also attend to the two objects in the image rather than the background.)

Original: S2. A TTENTION ANALYSIS
Revised: S2. A TTENTION ANALYSIS

Original: To assess the extent to which the various PrediNet heads attend to actual objects as opposed to the background, we produced a lower resolution content mask for each image (with the same resolution as the attention mask) containing 0 .
Revised: To assess the extent to which the various PrediNet heads attend to actual objects as opposed to the background, we produced a lower resolution content mask for each image (with the same resolution as the attention mask) containing 0 .

Original: 0 s at locations where there are no objects in the corresponding pixels of the full image, 1 . 

Revised: 0 s at locations where there are no objects in the corresponding pixels of the full image, 1 .

Original: s where more than 90% of the pixels contain an object, and 0 .
Revised: s where more than 90% of the pixels contain an object, and 0 .

Original: 5 s otherwise.
Revised: 5 s otherwise.

Original: By applying the attention mask to the content mask, and summing the resulting elements, we produced a scalar indicating whether the attention mask was selecting a region of the image with an object (value close to 1 . 0 ), or the background (value close to 0 . 0 ).
Revised: By applying the attention mask to the content mask, and summing the resulting elements, we produced a scalar indicating whether the attention mask was selecting a region of the image with an object (value close to 1 . 0 ), or the background (value close to 0 . 0 ).

Original: This was tested over 1000 images from the training set (Fig. S10), but similar results are obtained if the held-out set images are used instead.
Revised: This was tested over 1000 images from the training set (Fig. S10), but similar results are obtained if the held-out set images are used instead.

Original: The top plot in Fig.
Revised: The top plot in Fig.

Original: S10 shows that both attention masks of some heads consistently attend to objects, while others to a combination of object and background.
Revised: S10 shows that both attention masks of some heads consistently attend to objects, while others to a combination of object and background.

Original: Importantly, the heads for which the PCA meaningfully clusters the labels are also the ones in which both attention masks attend to objects (Fig. S9).
Revised: Importantly, the heads for which the PCA meaningfully clusters the labels are also the ones in which both attention masks attend to objects (Fig. S9).

Original: We additionally provide a similar analysis with a position mask, where each pixel in the mask contains a unique location index.
Revised: We additionally provide a similar analysis with a position mask, where each pixel in the mask contains a unique location index.

Original: The middle plot in Fig.
Revised: The middle plot in Fig.

Original: S10 shows that the attention masks in the majority of the heads do not consistently attend to speciﬁc locations.
Revised: S10 shows that the attention masks in the majority of the heads do not consistently attend to speciﬁc locations.

Original: Finally, the mean absolute values of the per-head input weights to the output MLP are shown in the bottom plot of the same ﬁgure.
Revised: Finally, the mean absolute values of the per-head input weights to the output MLP are shown in the bottom plot of the same ﬁgure.

Original: Interestingly, the heads that consistently attend only to objects have higher weighting than the rest.
Revised: Interestingly, the heads that consistently attend only to objects have higher weighting than the rest.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 Backgr.
Revised: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 Backgr.

Original: Obj.
Revised: Obj.

Original: P o s i t i o n I n d e x 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 

Revised: P o s i t i o n I n d e x 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

Original: Head number 0.
Revised: Head number 0.

Original: M e a n a b s o l u t e w e
Revised: M e a n a b s o l u t e w e

Original: i g h t v a l u e
Revised: i g h t v a l u e

Original: S E XPERIMENTAL V ARIATIONS
Revised: S E XPERIMENTAL V ARIATIONS

Original: Further experimental results are provided in this section, including variations in hyper-parameters.
Revised: Further experimental results are provided in this section, including variations in hyper-parameters.

Original: Fig.
Revised: Fig.

Original: S11 presents test accuracy curves for the ‘stripes’ object set, for which a summary is presented in Table 1 of the main text.
Revised: S11 presents test accuracy curves for the ‘stripes’ object set, for which a summary is presented in Table 1 of the main text.

Original: Fig.
Revised: Fig.

Original: S12 shows the results on the same experiment but using the Adam optimiser instead of SGD, with a learning rate of 10 − 4 .
Revised: S12 shows the results on the same experiment but using the Adam optimiser instead of SGD, with a learning rate of 10 − 4 .

Original: The TensorFlow default values for all other
Revised: The TensorFlow default values for all other

Original: Adam parameters were used.
Revised: Adam parameters were used.

Original: While other learning rate values were also tested, a value of 10 − 4 gave the best overall performance for all architectures.
Revised: While other learning rate values were also tested, a value of 10 − 4 gave the best overall performance for all architectures.

Original: Multi-task experiments were also performed using
Revised: Multi-task experiments were also performed using

Original: Adam with the same learning rate (Fig.
Revised: Adam with the same learning rate (Fig.

Original: S14 & S15), yielding an overall similar performance to SGD with a learning rate of 10 − 2 .
Revised: S14 & S15), yielding an overall similar performance to SGD with a learning rate of 10 − 2 .

Original: To assess the extent to which the number of heads and relations plays a role in the performance, we ran experiments with k = 64 heads and j = 16 relations (Fig.
Revised: To assess the extent to which the number of heads and relations plays a role in the performance, we ran experiments with k = 64 heads and j = 16 relations (Fig.

Original: S16 & S17), as well as k = 16 heads and j = 32 relations (Fig.
Revised: S16 & S17), as well as k = 16 heads and j = 32 relations (Fig.

Original: S18 & S19).
Revised: S18 & S19).

Original: The results indicate that having a greater number of heads leads to better performance than having a greater number of relations, because they provide more stability during training and, perhaps, a richer propositional representation.
Revised: The results indicate that having a greater number of heads leads to better performance than having a greater number of relations, because they provide more stability during training and, perhaps, a richer propositional representation.

Original: [Table]
Revised: [Table]

Original: [Figure] 
Revised: [Figure] [Table] [Figure] [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table] [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Revised: [Table]

Original: [Figure] [Table] 
Revised: [Figure] [Table] [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure] [Table] [Figure] [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table][Figure] [Table] [Figure]
Revised: [Table]

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table] [Figure] 
Revised: [Table] [Figure] [Table]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure][Table] [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure][Table] [Figure] [Table]
Revised: [Figure]

Original: [Figure][Table] [Figure] [Figure]
Revised: [Figure]

Original: [Table][Figure] [Figure] [Table]
Revised: [Table]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]