Original: F ACTORIZING D ECLARATIVE AND P ROCEDURAL 
Revised: F ACTORIZING D ECLARATIVE AND P ROCEDURAL K NOWLEDGE IN S TRUCTURED , D YNAMICAL E NVIRONMENTS

Original: K NOWLEDGE

Original: IN

Original: S TRUCTURED , D

Original: YNAMICAL

Original: E

Original: NVIRONMENTS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Modeling a structured, dynamic environment like a video game requires keeping track of the objects and their states ( declarative knowledge) as well as predicting how objects behave ( procedural knowledge).
Revised: Modeling a structured, dynamic environment like a video game requires keeping track of the objects and their states ( declarative knowledge) as well as predicting how objects behave ( procedural knowledge).

Original: Black-box models with a monolithic hidden state often fail to apply procedural knowledge consistently and uniformly, i.e., they lack systematicity .
Revised: Black-box models with a monolithic hidden state often fail to apply procedural knowledge consistently and uniformly, i.e., they lack systematicity .

Original: For example, in a video game, correct prediction of one enemy’s trajectory does not ensure correct prediction of another’s.
Revised: For example, in a video game, correct prediction of one enemy’s trajectory does not ensure correct prediction of another’s.

Original: We address this issue via an architecture that factorizes declarative and procedural knowledge and that imposes modularity within each form of knowledge.
Revised: We address this issue via an architecture that factorizes declarative and procedural knowledge and that imposes modularity within each form of knowledge.

Original: The architecture consists of active modules called object ﬁles that maintain the state of a single object and invoke passive external knowledge sources called schemata that prescribe state updates.
Revised: The architecture consists of active modules called object ﬁles that maintain the state of a single object and invoke passive external knowledge sources called schemata that prescribe state updates.

Original: To use a video game as an illustration, two enemies of the same type will share schemata but will have separate object ﬁles to encode their distinct state (e.g., health, position).
Revised: To use a video game as an illustration, two enemies of the same type will share schemata but will have separate object ﬁles to encode their distinct state (e.g., health, position).

Original: We propose to use attention to determine which object ﬁles to update, the selection of schemata, and the propagation of information between object ﬁles.
Revised: We propose to use attention to determine which object ﬁles to update, the selection of schemata, and the propagation of information between object ﬁles.

Original: The resulting architecture is a drop-in replacement conforming to the same input-output interface as normal recurrent networks (e.g., LSTM, GRU) yet achieves substantially better generalization on environments that have multiple object tokens of the same type, including a challenging intuitive physics benchmark.
Revised: The resulting architecture is a drop-in replacement conforming to the same input-output interface as normal recurrent networks (e.g., LSTM, GRU) yet achieves substantially better generalization on environments that have multiple object tokens of the same type, including a challenging intuitive physics benchmark.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: An intelligent agent that interacts with its world must not only perceive objects but must also remember its past experience with these objects.
Revised: An intelligent agent that interacts with its world must not only perceive objects but must also remember its past experience with these objects.

Original: The wicker chair in one’s living room is not just a chair, it is the chair which has an unsteady leg and easily tips.
Revised: The wicker chair in one’s living room is not just a chair, it is the chair which has an unsteady leg and easily tips.

Original: Your keys may not be visible, but you recall placing them on the ledge by the door.
Revised: Your keys may not be visible but you recall placing them on the ledge by the door.

Original: The annoying ﬂy buzzing in your left ear is the same ﬂy you saw earlier which landed on the table.
Revised: The annoying ﬂy buzzing in your left ear is the same ﬂy you saw earlier which landed on the table.

Original: Visual cognition requires a short-term memory that keeps track of an object’s location, properties, and history.
Revised: Visual cognition requires a short-term memory that keeps track of an object’s location, properties, and history.

Original: In the cognitive science literature, this particular form of state memory is often referred to as an object ﬁle (Kahneman et al., 1992), which we’ll abbreviate as OF .
Revised: In the cognitive science literature, this particular form of state memory is often referred to as an object ﬁle (Kahneman et al., 1992), which we’ll abbreviate as OF .

Original: An OF serves as a temporally persistent reference to an external object, permitting object constancy and permanence as the object and the viewer move in the world.
Revised: An OF serves as a temporally persistent reference to an external object, permitting object constancy and permanence as the object and the viewer move in the world.

Original: Complementary to information in the OF is abstract knowledge about the dynamics and behavior of an object.
Revised: Complementary to information in the OF is abstract knowledge about the dynamics and behavior of an object.

Original: We refer to this latter type of knowledge as a schema (plural schemata ), another term borrowed from the cognitive-science literature.
Revised: We refer to this latter type of knowledge as a schema (plural schemata ), another term borrowed from the cognitive-science literature.

Original: The combination of OF s and schemata is sufﬁcient to predict future states of object-structured environments, critical for planning and goal-seeking behavior.
Revised: The combination of OF s and schemata is sufﬁcient to predict future states of object-structured environments, critical for planning and goal-seeking behavior.

Original: To model a complex, structured visual environment, multiple OF s must be maintained in parallel.
Revised: To model a complex, structured visual environment, multiple OF s must be maintained in parallel.

Original: Consider scenes like a PacMan video-game screen in which the ghosts chase the PacMan, a public square or sports ﬁeld in which people interact with one another, or a pool table with rolling and colliding balls.
Revised: Consider scenes like a PacMan video-game screen in which the ghosts chase the PacMan, a public square or sports ﬁeld in which people interact with one another, or a pool table with rolling and colliding balls.

Original: In each of these environments, multiple instances of the same object class are present;all operate according to fundamentally similar dynamics. To ensure systematic modeling of the
Revised: In each of these environments, multiple instances of the same object class are present;

Original: [Figure]
Revised: [Figure]

Original:  environment, the same dynamics must be applied to multiple object instances.
Revised: all operate according to fundamentally similar dynamics. To ensure systematic modeling of the environment, the same dynamics must be applied to multiple object instances.

Original: Toward this goal, we propose a method of separately representing the state of an individual object—via an OF —and how its state evolves over time—via a schema.
Revised: Toward this goal, we propose a method of separately representing the state of an individual object—via an OF —and how its state evolves over time—via a schema.

Original: Object-oriented programming (OOP) provides a metaphor for thinking about the relationship between
Revised: Object-oriented programming (OOP) provides a metaphor for thinking about the relationship between

Original: OF s and schemata.
Revised: OF s and schemata.

Original: In OOP, each object is an instantiation of an object class and it has a self-contained collection of variables whose values are speciﬁc to that object and methods that operate on all instances of the same class.
Revised: In OOP, each object is an instantiation of an object class and it has a self-contained collection of variables whose values are speciﬁc to that object and methods that operate on all instances of the same class.

Original: The relation between objects and methods mirrors the relationship between our OF s and schemata.
Revised: The relation between objects and methods mirrors the relationship between our OF s and schemata.

Original: In both OOP and our view of visual cognition, a key principle is the encapsulation of knowledge: internal details of objects ( OF s) are hidden from other objects ( OF s), and methods (schemata) are accessible to all and only objects ( OF s) to which they are applicable.
Revised: In both OOP and our view of visual cognition, a key principle is the encapsulation of knowledge: internal details of objects ( OF s) are hidden from other objects ( OF s), and methods (schemata) are accessible to all and only objects ( OF s) to which they are applicable.

Original: The modularity of knowledge in OOP supports human programmers in writing code that is readily debugged, extended, and reused.
Revised: The modularity of knowledge in OOP supports human programmers in writing code that is readily debugged, extended, and reused.

Original: We conjecture that the corresponding modularity of OF s and schemata will lead to neural-network models with more efﬁcient learning and more robust generalization, thanks to appropriate disentangling and separation of concerns.
Revised: We conjecture that the corresponding modularity of OF s and schemata will lead to neural-network models with more efﬁcient learning and more robust generalization, thanks to appropriate disentangling and separation of concerns.

Original: Modularity is the guiding principle of the model we propose, which we call SCOFF , an acronym for schema / object-file factorization .
Revised: Modularity is the guiding principle of the model we propose, which we call SCOFF , an acronym for schema / object-file factorization .

Original: Like other neural net models with external memory (e.g., Mozer and Das, 1993; Graves et al., 2016; Sukhbaatar et al., 2015), SCOFF includes a set of slots which are each designed to contain an OF (Figure 2).
Revised: Like other neural net models with external memory (e.g., Mozer and Das, 1993; Graves et al., 2016; Sukhbaatar et al., 2015), SCOFF includes a set of slots which are each designed to contain an OF (Figure 2).

Original: In contrast to most previous external memory models, the slots are not passive contents waiting to be read or written by an active process, but are dynamic, modular elements that seek information in the environment that is relevant to the object they represent, and when critical information is observed, they update their states, possibly via information provided by other OF s. Event-based OOP is a good metaphor for this active process, where external events can trigger the action of objects.
Revised: In contrast to most previous external memory models, the slots are not passive contents waiting to be read or written by an active process, but are dynamic, modular elements that seek information in the environment that is relevant to the object they represent, and when critical information is observed, they update their states, possibly via information provided by other OF s. Event-based OOP is a good metaphor for this active process, where external events can trigger the action of objects.

Original: As Figure 2 suggests, there is a factorization of declarative knowledge—the location, properties, and history of an object, as contained in the OF s—and procedural knowledge—the rules of object behavior, as contained in the schemata.
Revised: As Figure 2 suggests, there is a factorization of declarative knowledge—the location, properties, and history of an object, as contained in the OF s—and procedural knowledge—the rules of object behavior, as contained in the schemata.

Original: Whereas declarative knowledge can change rapidly, procedural knowledge is more stable over time.
Revised: Whereas declarative knowledge can change rapidly, procedural knowledge is more stable over time.

Original: This factorization allows any schema to be applied to any OF as deemed appropriate.
Revised: This factorization allows any schema to be applied to any OF as deemed appropriate.

Original: The model design ensures systematicity in the operation of a schema, regardless of the slot to which an OF is assigned.
Revised: The model design ensures systematicity in the operation of a schema, regardless of the slot to which an OF is assigned.

Original: Similarly, an OF can access any applicable schema regardlessof which slot it sits in. Furthermore, a schema can be applied to multiple OF s at once, and multiple
Revised: Similarly, an OF can access any applicable schema regardless

Original: [Figure]
Revised: [Figure]

Original: Schemata are sets of parameters that specify the dynamics of objects.
Revised: Schemata are sets of parameters that specify the dynamics of objects.

Original: Object ﬁles ( OF s) are active modules that maintain the timevarying state of an object, seek information from the input, and select schemata for updating, and transmit information to other object ﬁles.
Revised: Object ﬁles ( OF s) are active modules that maintain the timevarying state of an object, seek information from the input, and select schemata for updating, and transmit information to other object ﬁles.

Original: Through spatial attention, OF s compete and select different regions of the input.
Revised: Through spatial attention, OF s compete and select different regions of the input.

Original:  schemata could be applied to an OF (e.g., Figure 1).
Revised: of which slot it sits in. Furthermore, a schema can be applied to multiple OF s at once, and multiple schemata could be applied to an OF (e.g., Figure 1).

Original: In OOP, systematicity is similarly achieved by virtue of the fact that the same method can be applied to any object instantiation and that multiple methods exist which can be applied to an object of the appropriate type.
Revised: In OOP, systematicity is similarly achieved by virtue of the fact that the same method can be applied to any object instantiation and that multiple methods exist which can be applied to an object of the appropriate type.

Original: Our key contribution is to demonstrate the feasibility and beneﬁt of factorizing declarative knowledge (the location, properties, and history of an object) and procedural knowledge (the way objects behave).
Revised: Our key contribution is to demonstrate the feasibility and beneﬁt of factorizing declarative knowledge (the location, properties, and history of an object) and procedural knowledge (the way objects behave).

Original: This factorization enforces not only an important form of systematicity, but also of exchangeability: the model behaves exactly the same regardless of the assignment of schemata to schemata-slots and the assignment of objects to OF -slots, i.e., the neural network operates on a set of objects and a set of schemata.
Revised: This factorization enforces not only an important form of systematicity, but also of exchangeability: the model behaves exactly the same regardless of the assignment of schemata to schemata-slots and the assignment of objects to OF -slots, i.e., the neural network operates on a set of objects and a set of schemata.

Original: With this factorization, we ﬁnd improved accuracy of next-state prediction models and improved interpretability of learned parameters.
Revised: With this factorization, we ﬁnd improved accuracy of next-state prediction models and improved interpretability of learned parameters.

Original:  T HE SCHEMATA / OBJECT - FILE FACTORIZATION ( SCOFF ) MODEL
Revised:  T HE SCHEMATA / OBJECT - FILE FACTORIZATION ( SCOFF ) MODEL

Original: SCOFF , shown in Figure 2, is an architectural backbone that supports the separation of procedural and declarative knowledge about dynamical entities (objects) in an input sequence.
Revised: SCOFF , shown in Figure 2, is an architectural backbone that supports the separation of procedural and declarative knowledge about dynamical entities (objects) in an input sequence.

Original: The input sequence { x 1 , . . . , x t , . . .
Revised: The input sequence { x 1 , . . . , x t , . . .

Original: , x T } , indexed by time step t is processed by a neural encoder (e.g., a fully convolutional net for images) to obtain a deep embedding, { z 1 , . . .
Revised: , x T } , indexed by time step t is processed by a neural encoder (e.g., a fully convolutional net for images) to obtain a deep embedding, { z 1 , . . .

Original: , z t , . . .
Revised: , z t , . . .

Original: , z T } , which then drives a network with n f OF s and n s schemata.
Revised: , z T } , which then drives a network with n f OF s and n s schemata.

Original: OF s are active processing components that maintain and update their internal state.
Revised: OF s are active processing components that maintain and update their internal state.

Original: Essentially, an
Revised: Essentially, an

Original: OF is a layer of GRU (Chung et al., 2014) or LSTM (Hochreiter and Schmidhuber, 1997) units with three additional bits of machinery, which we now describe.
Revised: OF is a layer of GRU (Chung et al., 2014) or LSTM (Hochreiter and Schmidhuber, 1997) units with three additional bits of machinery, which we now describe.

Original: Our earlier metaphor identifying OF s in SCOFF with objects in OOP is apropos in the sense that OF s are event driven .
Revised: Our earlier metaphor identifying OF s in SCOFF with objects in OOP is apropos in the sense that OF s are event driven .

Original: OF s operate in a temporal loop, continuously awaiting relevant input signals.
Revised: OF s operate in a temporal loop, continuously awaiting relevant input signals.

Original: Relevance is determined by an attention-based soft competition among OF s: the current input serves a key that is matched to a query generated from the state of each OF ; based on the goodness of match, each OF is provided with a value extracted from the input.
Revised: Relevance is determined by an attention-based soft competition among OF s: the current input serves a key that is matched to a query generated from the state of each OF ; based on the goodness of match, each OF is provided with a value extracted from the input.

Original: Each OF performs a one-step update of its state layer of GRU or LSTM units, conditioned on the input signal received.
Revised: Each OF performs a one-step update of its state layer of GRU or LSTM units, conditioned on the input signal received.

Original: The weight parameters needed to perform this update, which we will denote generically as θ , are not—as in a standard GRU or LSTM—internal to the layer but rather are provided externally.
Revised: The weight parameters needed to perform this update, which we will denote generically as θ , are not—as in a standard GRU or LSTM—internal to the layer but rather are provided externally.

Original: Each schema j is nothing more than a set of parameters θ j which can be plugged into this layer.
Revised: Each schema j is nothing more than a set of parameters θ j which can be plugged into this layer.

Original: SCOFF uses a key-value attention mechanism to perform Gumbel-based hard selection of the appropriate schema (parameters).
Revised: SCOFF uses a key-value attention mechanism to perform Gumbel-based hard selection of the appropriate schema (parameters).

Original: OF s may pass information to other OF s (analogous to arguments being passed to a method in OOP), again using a soft attention mechanism by which each OF queries all other OF s. Keys provided by the other OF s are matched to the query, and a soft selection of the best matching OF s determines the weighting of values transmitted by the other OF s.
Revised: OF s may pass information to other OF s (analogous to arguments being passed to a method in OOP), again using a soft attention mechanism by which each OF queries all other OF s. Keys provided by the other OF s are matched to the query, and a soft selection of the best matching OF s determines the weighting of values transmitted by the other OF s

Original: This operation cycle ensures that OF s can update their state in response to both the external input and the internal state comprised of all the OF s’ contents.
Revised: This operation cycle ensures that OF s can update their state in response to both the external input and the internal state comprised of all the OF s’ contents.

Original: This updating is an extra wrapper around the ordinary update that takes place in a GRU or LSTM layer.
Revised: This updating is an extra wrapper around the ordinary update that takes place in a GRU or LSTM layer.

Original: It provides additional ﬂexibility in that (1) The external input is routed to OF s contingent on their internal state, (2) OF s can switch their dynamics from one time step to the next conditioned on their internal state, (3) OF s are modular in that they do not communicate with one another except via state-dependent selective message passing.
Revised: It provides additional ﬂexibility in that (1) The external input is routed to OF s contingent on their internal state, (2) OF s can switch their dynamics from one time step to the next conditioned on their internal state, (3) OF s are modular in that they do not communicate with one another except via state-dependent selective message passing.

Original: OF s are placeholders in that a particular OF has no weight parameter speciﬁc to that OF .
Revised: OF s are placeholders in that a particular OF has no weight parameter speciﬁc to that OF .

Original: Instead, the parameters are provided from two sources: either the schemata or a pool of generic parameters shared by the n f OF s. This generic parameter pool is used to implement key-value attention over the input, the schemata, and communication among OF s.
Revised: Instead, the parameters are provided from two sources: either the schemata or a pool of generic parameters shared by the n f OF s. This generic parameter pool is used to implement key-value attention over the input, the schemata, and communication among OF s.

Original: The sharing of schemata ensures systematicity; the sharing of the generic parameter pool ensures exchangeability —model behavior is unaffected by the assignment of object instances to OF slots.
Revised: The sharing of schemata ensures systematicity; the sharing of the generic parameter pool ensures exchangeability —model behavior is unaffected by the assignment of object instances to OF slots.

Original: 2. SCOFF SPECIFICS
Revised: 2. SCOFF SPECIFICS

Original: Algorithm 1 provides a precise speciﬁcation of SCOFF broken into four steps.
Revised: Algorithm 1 provides a precise speciﬁcation of SCOFF broken into four steps.

Original: Step 1 is external to SCOFF and involves processing an image input to obtain a deep embedding. 
Revised: Step 1 is external to SCOFF and involves processing an image input to obtain a deep embedding. The processing is [Table] [Table]

Revised: [Table]

Revised: [Table]

Original: The processing is performed by a fully Convolutional Neural Net (CNN) that preserves positional information (typically
Revised: [Table] [Table] [Table] [Table]

Revised: k

Original: [Table]
Revised: [Table]

Revised: p

Revised: [Table]

Original: where
Revised: d e

Original: [Table]
Revised: [Table]

Original: k ∈
Revised: κ p =

Revised: [Table]

Revised: p

Revised: e

Original: [Table]
Revised: [Table]

Original: n
Revised: k

Revised: [Table]

Revised: p

Revised: [Table]

Revised: p v p

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: k

Revised: [Table]

Revised: h t −

Revised: [Table]

Revised: k

Original: [Table]
Revised: [Table]

Revised: W

Revised: [Table]

Revised: • i k

Revised: [Table]

Revised: j

Revised: [Table]

Revised: T k

Revised: [Table]

Revised: γ

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: h t

Revised: [Table]

Revised: k

Revised: [Table]

Revised: h t

Revised: [Table]

Revised: k

Revised: [Table]

Revised: i

Revised: [Table]

Original: } 64 × 64 in our simulations), and for each position p encodes the processed input c p and concatenates a learned encoding of position, e p to produce a position-speciﬁc hidden state.
Revised: [Table] [Table] [Table] performed by a fully Convolutional Neural Net (CNN) that preserves positional information (typically 64 × 64 in our simulations), and for each position p encodes the processed input c p and concatenates a learned encoding of position, e p to produce a position-speciﬁc hidden state.

Original: ( SCOFF can also handle non-visual inputs, such as vectors or discrete tokens; in this case the CNN is replaced by an MLP.)
Revised: ( SCOFF can also handle non-visual inputs, such as vectors or discrete tokens; in this case the CNN is replaced by an MLP.)

Original: The subsequent core steps are as follows.
Revised: The subsequent core steps are as follows.

Original: Step 2: Soft competition among OF s to select regions of the input to process.
Revised: Step 2: Soft competition among OF s to select regions of the input to process.

Original: The state of each
Revised: The state of each

Original: OF k , h t − 1 ,k , is used to form a query, q k , which determines the input positions that it will attend to.
Revised: OF k , h t − 1 ,k , is used to form a query, q k , which determines the input positions that it will attend to.

Original: The query is matched to a set of position-speciﬁc input keys, κ k,p for position p , producing a position-speciﬁc match score, s k,p .
Revised: The query is matched to a set of position-speciﬁc input keys, κ k,p for position p , producing a position-speciﬁc match score, s k,p .

Original: Soft position-speciﬁc competition among the OF s results in the
Revised: Soft position-speciﬁc competition among the OF s results in the

Original: OF s selecting complementary image regions to attend to.
Revised: OF s selecting complementary image regions to attend to.

Original: The contents of the attended positions are combined yielding an OF -speciﬁc input encoding, z k .
Revised: The contents of the attended positions are combined yielding an OF -speciﬁc input encoding, z k .

Original: Step 3: OF s pick the most relevant schema and update.
Revised: Step 3: OF s pick the most relevant schema and update.

Original: OF k picks one schema via attention as follows.
Revised: OF k picks one schema via attention as follows.

Original: OF k binds to each schema j , and then performs a hypothetical update, yielding (cid:101) h t,k,j .
Revised: OF k binds to each schema j , and then performs a hypothetical update, yielding (cid:101) h t,k,j .

Original: In experiments below, the OF state is maintained by a GRU layer, and schema j is a parameterization of the GRU, denoted GRU θ j , which determines the update.
Revised: In experiments below, the OF state is maintained by a GRU layer, and schema j is a parameterization of the GRU, denoted GRU θ j , which determines the update.

Original: The previous state of the OF , h t − 1 ,k serves as a query in key-value attention against a key derived from the hypothetical updated state, (cid:101) h t,k,j .
Revised: The previous state of the OF , h t − 1 ,k serves as a query in key-value attention against a key derived from the hypothetical updated state, (cid:101) h t,k,j .

Original: The schema i k corresponding to the best query-key match for OF k is used to update OF k ’s state.
Revised: The schema i k corresponding to the best query-key match for OF k is used to update OF k ’s state.

Original: Selection is based on the straight-through Gumbel-softmax method (Jang et al., 2016), which makes a hard choice during forward propagation and during backward propagation, it considers a softened version of the output to permit gradient propagation to non-selected schemata.
Revised: Selection is based on the straight-through Gumbel-softmax method (Jang et al., 2016), which makes a hard choice during forward propagation and during backward propagation, it considers a softened version of the output to permit gradient propagation to non-selected schemata.

Original: Step 4: Soft competition among OF s to transmit relevant information to each OF .
Revised: Step 4: Soft competition among OF s to transmit relevant information to each OF .

Original: This step allows for interactions among OF s. Each OF k queries other OF s for information relevant to its update as follows.
Revised: This step allows for interactions among OF s. Each OF k queries other OF s for information relevant to its update as follows.

Original: The OF ’s previous state, h t − 1 ,k , is used to form a query, (cid:98) q k , in an attention mechanism against a key derived from the new state of each other OF k (cid:48) , (cid:98) κ k (cid:48) , and softmax selection ( s k,k (cid:48) ) is used to obtain weighted information from other OF s, (cid:98) v (cid:48) k , into OF k ’s state.
Revised: The OF ’s previous state, h t − 1 ,k , is used to form a query, (cid:98) q k , in an attention mechanism against a key derived from the new state of each other OF k (cid:48) , (cid:98) κ k (cid:48) , and softmax selection ( s k,k (cid:48) ) is used to obtain weighted information from other OF s, (cid:98) v (cid:48) k , into OF k ’s state.

Original: Number of Parameters.
Revised: Number of Parameters.

Original: SCOFF can be used as a drop-in replacement for a LSTM/GRU layer.
Revised: SCOFF can be used as a drop-in replacement for a LSTM/GRU layer.

Original: There is a subtlety that must be considered for successful integration.
Revised: There is a subtlety that must be considered for successful integration.

Original: If the total size of the hidden state is kept the same, integrating SCOFF dramatically reduces the total number of recurrent parameters in the model because of its block-sparse structure.
Revised: If the total size of the hidden state is kept the same, integrating SCOFF dramatically reduces the total number of recurrent parameters in the model because of its block-sparse structure.

Original: The majority of SCOFF parameters are the schemata, { θ j | j ∈ { 1 , . . .
Revised: The majority of SCOFF parameters are the schemata, { θ j | j ∈ { 1 , . . .

Original: , n s }} .
Revised: , n s }} .

Original: The remaining parameters are those of query ( W q , (cid:102) W q , (cid:99) W q ), key ( W e , (cid:102) W e , (cid:99) W e ), and value W v , (cid:99) W v ) functions.
Revised: The remaining parameters are those of query ( W q , (cid:102) W q , (cid:99) W q ), key ( W e , (cid:102) W e , (cid:99) W e ), and value W v , (cid:99) W v ) functions.

Original: Note that these linear functions could be replaced by nonlinear functions.
Revised: Note that these linear functions could be replaced by nonlinear functions.

Original: Its also interesting to note that SCOFF actually has far fewer parameters than other modular architectures (e.g., RIMs, Recurrent Entity Networks, Neural Module Networks)
Revised: Its also interesting to note that SCOFF actually has far fewer parameters than other modular architectures (e.g., RIMs, Recurrent Entity Networks, Neural Module Networks)

Original: when n s < n f (which holds true for most of our experiments), because of the potential one-to-many mapping between schemata and OF s. Optionally, at Step 2 , during training instead of activating all the
Revised: when n s < n f (which holds true for most of our experiments), because of the potential one-to-many mapping between schemata and OF s. Optionally, at Step 2 , during training instead of activating all the

Original: OF s, we can use a sparse attention, to only activate a subset of OF s that are relevant at that time step t , and only update the state of the activated OF s. We note that this is not speciﬁc to the proposed method, but the performance of the method can be improved by only selectively activating the relevant OF s. 
Revised: OF s, we can use a sparse attention, to only activate a subset of OF s that are relevant at that time step t , and only update the state of the activated OF s. We note that this is not speciﬁc to the proposed method, but the performance of the method can be improved by only selectively activating the relevant OF s. We also note that the different schemata in the proposed method are only used to update the state of

Revised: OF sand hence is agnostic as to how one obtains a set of OF s. Any other method for entity extraction (Greff et al., 2019) can be used for obtaining a set of OF s. Similarly, other methods for decoding which preserves equivariance among different OF s(Watters et al., 2019) can be used for decoding the contents of different OF s.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: CNNs .
Revised: CNNs .

Original: SCOFF applies the same knowledge (schemata) to multiple OF s, yielding systematicity.
Revised: SCOFF applies the same knowledge (schemata) to multiple OF s, yielding systematicity.

Original: Similarly, a CNN is a highly restrictive instantiation of this same notion, where knowledge—in the form of a convolutional ﬁlter—is applied uniformly to every location in an image, yielding equivariance.
Revised: Similarly, a CNN is a highly restrictive instantiation of this same notion, where knowledge—in the form of a convolutional ﬁlter—is applied uniformly to every location in an image, yielding equivariance.

Original: SCOFF is a more ﬂexible architecture than a CNN in that OF s are deﬁned by abstract notion of objects not a physical patch of an image, and schemata are applied dynamically and ﬂexibly over time, not in a ﬁxed, rigid manner as the ﬁlters in a CNN.
Revised: SCOFF is a more ﬂexible architecture than a CNN in that OF s are deﬁned by abstract notion of objects not a physical patch of an image, and schemata are applied dynamically and ﬂexibly over time, not in a ﬁxed, rigid manner as the ﬁlters in a CNN.

Original: Memory Networks .
Revised: Memory Networks .

Original: A variety of existing models leverage an external slot-based, content-addressible memory (e.g., Graves et al., 2016; Sukhbaatar et al., 2015).
Revised: A variety of existing models leverage an external slot-based, content-addressible memory (e.g., Graves et al., 2016; Sukhbaatar et al., 2015).

Original: The memory slots are passive elements that are processed by a differentiable neural controller such as an LSTM.
Revised: The memory slots are passive elements that are processed by a differentiable neural controller such as an LSTM.

Original: Whereas traditional memory networks have many dumb memory cells and one smart controller, SCOFF has many smart memory cells—the OF s—which also act as local controllers.
Revised: Whereas traditional memory networks have many dumb memory cells and one smart controller, SCOFF has many smart memory cells—the OF s—which also act as local controllers.

Original: However, SCOFF shares the notion with memory networks that the same knowledge is applied systematically to every cell.
Revised: However, SCOFF shares the notion with memory networks that the same knowledge is applied systematically to every cell.

Original: Relational RNN (RMC) .
Revised: Relational RNN (RMC) .

Original: The RMC (Santoro et al., 2018) has a multi-head attention mechanism which allows it to share information between multiple memory locations.
Revised: The RMC (Santoro et al., 2018) has a multi-head attention mechanism which allows it to share information between multiple memory locations.

Original: RMC is like Memory
Revised: RMC is like Memory

Original: Networks in that dynamics are driven by a central controller: memory is used to condition the dynamics of an RNN.
Revised: Networks in that dynamics are driven by a central controller: memory is used to condition the dynamics of an RNN.

Original: Recurrent Entity Networks .
Revised: Recurrent Entity Networks .

Original: Henaff et al.
Revised: Henaff et al.

Original: (2016) describe a collection of recurrent modules that update independently and in parallel in response to each input in a sequence.
Revised: (2016) describe a collection of recurrent modules that update independently and in parallel in response to each input in a sequence.

Original: The module outputs are integrated to form a response.
Revised: The module outputs are integrated to form a response.

Original: It shares a modular architecture with SCOFF , but the modules have a ﬁxed function and do not directly communicate with one another.
Revised: It shares a modular architecture with SCOFF , but the modules have a ﬁxed function and do not directly communicate with one another.

Original: This earlier work focused on language not images.
Revised: This earlier work focused on language not images.

Original: Recurrent Independent Mechanisms (RIMs) .
Revised: Recurrent Independent Mechanisms (RIMs) .

Original: RIMs (Goyal et al., 2019) are a key inspiration for our work.
Revised: RIMs (Goyal et al., 2019) are a key inspiration for our work.

Original: RIMs are a modular neural architecture consisting of an ensemble of dynamical components which have sparse interactions through the bottleneck of attention.
Revised: RIMs are a modular neural architecture consisting of an ensemble of dynamical components which have sparse interactions through the bottleneck of attention.

Original: Each RIM module has at its core an LSTM layer (Hochreiter and Schmidhuber, 1997).
Revised: Each RIM module has at its core an LSTM layer (Hochreiter and Schmidhuber, 1997).

Original: A RIM module is much like our
Revised: A RIM module is much like our

Original: OF .
Revised: OF .

Original: Both are meant to be dynamical entities with a time-evolving state.
Revised: Both are meant to be dynamical entities with a time-evolving state.

Original: However, in contrast to OF s in SCOFF , RIM modules operate according to ﬁxed dynamics and each RIM module is specialized for a particular computation.
Revised: However, in contrast to OF s in SCOFF , RIM modules operate according to ﬁxed dynamics and each RIM module is specialized for a particular computation.

Original: RIM modules are thus not interchangeable.
Revised: RIM modules are thus not interchangeable.

Original: Neural Module Networks .
Revised: Neural Module Networks .

Original: A modular module network (Jacobs et al., 1991; Bottou and Gallinari, 1991; Ronco et al., 1997; Reed and De Freitas, 2015; Andreas et al., 2016; Rosenbaum et al., 2017;
Revised: A modular module network (Jacobs et al., 1991; Bottou and Gallinari, 1991; Ronco et al., 1997; Reed and De Freitas, 2015; Andreas et al., 2016; Rosenbaum et al., 2017;

Original: Fernando et al., 2017; Shazeer et al., 2017; Kirsch et al., 2018; Rosenbaum et al., 2019) has an architecture which is composed dynamically from several neural modules, where each module is meant to perform a distinct function.
Revised: Fernando et al., 2017; Shazeer et al., 2017; Kirsch et al., 2018; Rosenbaum et al., 2019) has an architecture which is composed dynamically from several neural modules, where each module is meant to perform a distinct function.

Original: Each module has a ﬁxed function, unlike our OF s, and modules are applied one at a time, in contrast to our OF s, which can update and be used for prediction in parallel.
Revised: Each module has a ﬁxed function, unlike our OF s, and modules are applied one at a time, in contrast to our OF s, which can update and be used for prediction in parallel.

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: SCOFF is a drop-in replacement for a standard LSTM or GRU layer, conforming to the same inputoutput interface.
Revised: SCOFF is a drop-in replacement for a standard LSTM or GRU layer, conforming to the same inputoutput interface.

Original: Because of their interchangeability, we compare SCOFF to LSTM and GRUs.
Revised: Because of their interchangeability, we compare SCOFF to LSTM and GRUs.

Original: We also compare SCOFF to two alternative modular architectures: RMC , a memory based relational recurrent model with attention between memory elements and hidden states (Santoro et al., 2018), and
Revised: We also compare SCOFF to two alternative modular architectures: RMC , a memory based relational recurrent model with attention between memory elements and hidden states (Santoro et al., 2018), and

Original: Recurrent Independent Mechanisms (RIMs) , a modular memory based on a single layered recurrent model with attention modulated input and communication between modules (Goyal et al., 2019).
Revised: Recurrent Independent Mechanisms (RIMs) , a modular memory based on a single layered recurrent model with attention modulated input and communication between modules (Goyal et al., 2019).

Original: In all simulations we present, the input is a video sequence, each frame of which is preprocessed by a CNN backbone.
Revised: In all simulations we present, the input is a video sequence, each frame of which is preprocessed by a CNN backbone.

Original: We consider two task types: video prediction and reinforcement learning (RL).
Revised: We consider two task types: video prediction and reinforcement learning (RL).

Original: For video prediction, the simulation output is a prediction of the next frame in the sequence.
Revised: For video prediction, the simulation output is a prediction of the next frame in the sequence.

Original: For RL, the output is an action choice.
Revised: For RL, the output is an action choice.

Original: In both cases, SCOFF ’s internal state is mapped to an output in the following manner.
Revised: In both cases, SCOFF ’s internal state is mapped to an output in the following manner.

Original: The state of each OF is remapped by a siamese network that transforms the state.
Revised: The state of each OF is remapped by a siamese network that transforms the state.

Original: (By siamese network, we mean that every OF is remapped by the same function.)
Revised: (By siamese network, we mean that every OF is remapped by the same function.)

Original: The transformed state is concatenated and used by an attention-based mechanism operating over the OF slots to ensure exchangeability of the OF s.
Revised: The transformed state is concatenated and used by an attention-based mechanism operating over the OF slots to ensure exchangeability of the OF s.

Original: In the case of video prediction, a deconvolutional backbone yields an image as output; in the case of reinforcement learning, the output is a distribution over actions.
Revised: In the case of video prediction, a deconvolutional backbone yields an image as output; in the case of reinforcement learning, the output is a distribution over actions.

Original: The heart of the model is a single SCOFF layer, consisting of n f OF s and n s schemata.
Revised: The heart of the model is a single SCOFF layer, consisting of n f OF s and n s schemata.

Original: Most simulation details are contained in section A of the Appendix, but we summarize some key points here.
Revised: Most simulation details are contained in section A of the Appendix, but we summarize some key points here.

Original: Unless otherwise indicated, we always train in an end-to-end fashing using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 0.0001 and momentum of 0.9.
Revised: Unless otherwise indicated, we always train in an end-to-end fashing using the Adam (Kingma and Ba, 2014) optimizer with a learning rate of 0.0001 and momentum of 0.9.

Original: As a default, we use n f = 6 and n s = 4 , except where we are speciﬁcally exploring the effects of manipulating these hyperparameters.
Revised: As a default, we use n f = 6 and n s = 4 , except where we are speciﬁcally exploring the effects of manipulating these hyperparameters.

Original: We include more experimental results in the Appendix, and we will release the code.
Revised: We include more experimental results in the Appendix, and we will release the code.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Our experiments address the following questions.
Revised: Our experiments address the following questions.

Original: (1) Does SCOFF successfully factorize knowledge into OF s and schemata?
Revised: (1) Does SCOFF successfully factorize knowledge into OF s and schemata?

Original: (2) Do the learned schemata have semantically meaningful interpretations?
Revised: (2) Do the learned schemata have semantically meaningful interpretations?

Original: (3) Is the factorization of knowledge into object ﬁles and schemata helpful in downstream tasks?
Revised: (3) Is the factorization of knowledge into object ﬁles and schemata helpful in downstream tasks?

Original: (4) Does SCOFF outperform state-of-the-art approaches, both modular and non-modular, which lack
Revised: (4) Does SCOFF outperform state-of-the-art approaches, both modular and non-modular, which lack

Original: SCOFF ’s strong inductive bias toward systematicity and knowledge factorization?
Revised: SCOFF ’s strong inductive bias toward systematicity and knowledge factorization?

Original: We begin with a series of experiments involving greyscale synthetic video sequences consisting of a single ball-like object moving over time according to switchable dynamics.
Revised: We begin with a series of experiments involving greyscale synthetic video sequences consisting of a single ball-like object moving over time according to switchable dynamics.

Original: We model this scenario with SCOFF having a single ( n f = 1 ) OF and the number of schemata matching the number of different types of dynamics.
Revised: We model this scenario with SCOFF having a single ( n f = 1 ) OF and the number of schemata matching the number of different types of dynamics.

Original: Single object with ﬁxed dynamics.
Revised: Single object with ﬁxed dynamics.

Original: To demonstrate that the proposed model is able to factorize knowledge, we consider video scenes in which a single object, starting in a random location, has dynamics that cause it to either (a) accelerate in a particular direction, (b) move at a constant velocity in a particular direction, or (c) take a random walk with a constant velocity.
Revised: To demonstrate that the proposed model is able to factorize knowledge, we consider video scenes in which a single object, starting in a random location, has dynamics that cause it to either (a) accelerate in a particular direction, (b) move at a constant velocity in a particular direction, or (c) take a random walk with a constant velocity.

Original: Following training, SCOFF can predict trajectories after being shown the ﬁrst few frames.
Revised: Following training, SCOFF can predict trajectories after being shown the ﬁrst few frames.

Original: It does so by activating a schema that has a one-to-one correspondence with the three types of dynamics (Figure 3, left panel), leading to interpretable semantics and a clean factorization of knowledge.
Revised: It does so by activating a schema that has a one-to-one correspondence with the three types of dynamics (Figure 3, left panel), leading to interpretable semantics and a clean factorization of knowledge.

Original: For additional details, refer to
Revised: For additional details, refer to

Original: Appendix C.
Revised: Appendix C.

Original: Single object with switching dynamics.
Revised: Single object with switching dynamics.

Original: In this experiment, video scenes contain a single object, starting in a random location, operating according to one of two dynamics: vertical and horizontal motion.
Revised: In this experiment, video scenes contain a single object, starting in a random location, operating according to one of two dynamics: vertical and horizontal motion.

Original: In contrast to the previous experiment, the object can switch dynamics in the course of a sequence.
Revised: In contrast to the previous experiment, the object can switch dynamics in the course of a sequence.

Original: The current dynamics are indicated by markers in the image (see Figure 4) which
Revised: The current dynamics are indicated by markers in the image (see Figure 4) which

Original: SCOFF must learn to detect in order to select the corresponding schema.

Original: In this experiment, we ﬁnd that SCOFF is able to learn a one-to-one correspondence with two types of dynamics, yielding high-precision prediction of the next frame, even on the ﬁrst frame after a switch.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: of SCOFF is ampliﬁed as the number of balls increases (4Balls versus 678Balls) and as predictions require greater look ahead (10 versus 30 frames).
Revised: SCOFF is ampliﬁed as the number of balls increases (4Balls versus 678Balls) and as predictions require greater look ahead (10 versus 30 frames).

Revised: SCOFF must learn to detect in order to select the corresponding schema.

Revised: In this experiment, we ﬁnd that SCOFF is able to learn a one-to-one correspondence with two types of dynamics, yielding high-precision prediction of the next frame, even on the ﬁrst frame after a switch.

Original: Single object with switching dynamics in an RL paradigm.
Revised: Single object with switching dynamics in an RL paradigm.

Original: In this experiment, we use the partiallyobservable GotoObjMaze environment from Chevalier-Boisvert et al.
Revised: In this experiment, we use the partiallyobservable GotoObjMaze environment from Chevalier-Boisvert et al.

Original: (2018), in which an agent must learn to navigate to navigate a 2D multi-room grid world to locate distinct objects such as a key.
Revised: (2018), in which an agent must learn to navigate to navigate a 2D multi-room grid world to locate distinct objects such as a key.

Original: The world consists of 6 × 6 square rooms that are randomly connected through doors to form a 3 × 3 maze of rooms.
Revised: The world consists of 6 × 6 square rooms that are randomly connected through doors to form a 3 × 3 maze of rooms.

Original: The agent’s view of the world is an egocentric 5 × 5 patch.
Revised: The agent’s view of the world is an egocentric 5 × 5 patch.

Original: Our experiment involves n s = 4 schemata whose activation pattern over time steps is shown in the right panel of
Revised: Our experiment involves n s = 4 schemata whose activation pattern over time steps is shown in the right panel of

Original: Figure 3.
Revised: Figure 3.

Original: The schema activation pattern is interpretable; for example, schema 4 is triggered when the ‘key’ is in the agent’s ﬁeld of view (see Appendix, Figure 13 and Section D).
Revised: The schema activation pattern is interpretable; for example, schema 4 is triggered when the ‘key’ is in the agent’s ﬁeld of view (see Appendix, Figure 13 and Section D).

Original: To obtain quantitative evaluations, we test transfer by increasing the room size to 10 × 10 during testing.
Revised: To obtain quantitative evaluations, we test transfer by increasing the room size to 10 × 10 during testing.

Original: SCOFF is able to successfully reach its goal on 82% of trials, whereas a GRU baseline succeeds only 56% of trials.
Revised: SCOFF is able to successfully reach its goal on 82% of trials, whereas a GRU baseline succeeds only 56% of trials.

Original: Even an overparameterized GRU, which matches SCOFF in number of free parameters, still succeeds less often—on 74% of trials.
Revised: Even an overparameterized GRU, which matches SCOFF in number of free parameters, still succeeds less often—on 74% of trials.

Original: All models are trained for an equal number of time steps.
Revised: All models are trained for an equal number of time steps.

Original: To summarize, the factorization of knowledge in SCOFF leads not only to better next-step prediction but also improve performance on a downstream control task.
Revised: To summarize, the factorization of knowledge in SCOFF leads not only to better next-step prediction but also improve performance on a downstream control task.

Original: Multiple objects with multiple dynamics.
Revised: Multiple objects with multiple dynamics.

Original: We now turn from single-object sequences to sequences involving multiple objects which operate according complex dynamics.
Revised: We now turn from single-object sequences to sequences involving multiple objects which operate according complex dynamics.

Original: We consider a bouncing-balls environment in which multiple balls move with billiard-ball dynamics (Van Steenkiste et al., 2018).
Revised: We consider a bouncing-balls environment in which multiple balls move with billiard-ball dynamics (Van Steenkiste et al., 2018).

Original: The dataset consists of 50,000 training examples and 10,000 test examples showing ∼ 50 framesof either 4 solid same-color balls bouncing in a conﬁned square geometry with different masses corresponding to their radii ( 4Balls ), 6-8 same-color balls bouncing in a conﬁned geometry ( 678Balls ), 3 same-color balls bouncing in a conﬁned geometry with a central occluder ( Curtain ), or balls of
Revised: The dataset consists of 50,000 training examples and 10,000 test examples showing ∼ 50 frames

Original: [Table]
Revised: [Table]

Original:  four different colors ( Colored 678Balls ).
Revised: of either 4 solid same-color balls bouncing in a conﬁned square geometry with different masses corresponding to their radii ( 4Balls ), 6-8 same-color balls bouncing in a conﬁned geometry ( 678Balls ), 3 same-color balls bouncing in a conﬁned geometry with a central occluder ( Curtain ), or balls of four different colors ( Colored 678Balls ).

Original: Although each ball has a distinct state (position, velocity, and possibly color), they share the same underlying dynamics of motion and collision.
Revised: Although each ball has a distinct state (position, velocity, and possibly color), they share the same underlying dynamics of motion and collision.

Original: We expected that SCOFF would learn to dissect these dynamics, e.g., by treating collisions, straight-line motion, interactions with the walls, and moving behind an occluder as distinct schemata.
Revised: We expected that SCOFF would learn to dissect these dynamics, e.g., by treating collisions, straight-line motion, interactions with the walls, and moving behind an occluder as distinct schemata.

Original: We use the encoder and decoder architecture of (Van Steenkiste et al., 2018) for all models.
Revised: We use the encoder and decoder architecture of (Van Steenkiste et al., 2018) for all models.

Original: SCOFF has n f = 4 for 4Balls and Curtain, and n f = 8 for 678Balls and Coloured678Balls, and n s = 4 for all simulations.
Revised: SCOFF has n f = 4 for 4Balls and Curtain, and n f = 8 for 678Balls and Coloured678Balls, and n s = 4 for all simulations.

Original: All models are trained for 100 epochs.
Revised: All models are trained for 100 epochs.

Original: As shown in Figure 5, SCOFF achieves dramatic improvements in successfullypredicting 10- and 30-step iterated predictions relative to GRUs, and RIMs. (For further details, see Appendix Section C.)
Revised: As shown in Figure 5, SCOFF achieves dramatic improvements in successfully details, see Appendix Section C.)

Original: We omitted RMC (Santoro et al., 2018) from Figures 5a-d because
Revised: We omitted RMC (Santoro et al., 2018) from Figures 5a-d because

Original: RIMs performs strictly better than RMC in these environments (Goyal et al., 2019), but have included
Revised: RIMs performs strictly better than RMC in these environments (Goyal et al., 2019), but have included

Original: RMC in the Appendix.
Revised: RMC in the Appendix.

Original: Increasing the number of schemata.
Revised: Increasing the number of schemata.

Original: Our previous experiment used n s = 4 for all simulations.
Revised: Our previous experiment used n s = 4 for all simulations.

Original: In order to study what happens if SCOFF has access to a large number of schemata, we perform an experiment with n s = 10 .
Revised: In order to study what happens if SCOFF has access to a large number of schemata, we perform an experiment with n s = 10 .

Original: For the curtain task, we found that only three schemata were being used, and performance was about the same as when training with n s = 3 or n s = 4 .
Revised: For the curtain task, we found that only three schemata were being used, and performance was about the same as when training with n s = 3 or n s = 4 .

Original: For 678Balls, we found that by the end of training only four schemas were being used, and performance is about the same as when training with n s = 4 .
Revised: For 678Balls, we found that by the end of training only four schemas were being used, and performance is about the same as when training with n s = 4 .

Original: Thus, providing SCOFF with excess resources leads to some not being used which is a waste of computation and memory but does not lead to suboptimal performance.
Revised: Thus, providing SCOFF with excess resources leads to some not being used which is a waste of computation and memory but does not lead to suboptimal performance.

Original: For example, one might have been concerned about overﬁtting with too many resources.
Revised: For example, one might have been concerned about overﬁtting with too many resources.

Original: However, with n s = 3 schemata, we never observe unused schemata, suggesting that the model does not have difﬁculty using the resources we provide to it.
Revised: However, with n s = 3 schemata, we never observe unused schemata, suggesting that the model does not have difﬁculty using the resources we provide to it.

Original: That is, there are no ‘dead’ schemata that fail to be trained due to local optima.
Revised: That is, there are no ‘dead’ schemata that fail to be trained due to local optima.

Original: Modeling physical laws in a multi-object environment.
Revised: Modeling physical laws in a multi-object environment.

Original: Modeling a physical system, such as objects in a world obeying laws of gravity and momentum, requires factorization of state (object position, velocity) and dynamics.
Revised: Modeling a physical system, such as objects in a world obeying laws of gravity and momentum, requires factorization of state (object position, velocity) and dynamics.

Original: All objects must obey the same laws while each object must maintain its distinct state.
Revised: All objects must obey the same laws while each object must maintain its distinct state.

Original: We used the Intuitive Physics Benchmark (Riochet et al., 2019) in which balls roll behind a brick wall such that they are brieﬂy occluded.
Revised: We used the Intuitive Physics Benchmark (Riochet et al., 2019) in which balls roll behind a brick wall such that they are brieﬂy occluded.

Original: The training data is constructed such that the examples are all physically realistic.
Revised: The training data is constructed such that the examples are all physically realistic.

Original: The test set consists of both sequences that behave according to the laws used to synthesize the training data and sequences that follow unrealistic physical laws.
Revised: The test set consists of both sequences that behave according to the laws used to synthesize the training data and sequences that follow unrealistic physical laws.

Original: We test three forms of unrealistic physics: balls disappearing behind the wall (O1 task), balls having their shape change for no reason (O2 task), and balls teleporting (O3 task).
Revised: We test three forms of unrealistic physics: balls disappearing behind the wall (O1 task), balls having their shape change for no reason (O2 task), and balls teleporting (O3 task).

Original: The Benchmark has three subsets of experiments, and we chose the challenging subset with signiﬁcant occlusions.
Revised: The Benchmark has three subsets of experiments, and we chose the challenging subset with signiﬁcant occlusions.

Original: We trained models to perform next state prediction on the training set and we use model likelihood (Riochet et al., 2019) to discriminate between realistic and unnatural test sequences.
Revised: We trained models to perform next state prediction on the training set and we use model likelihood (Riochet et al., 2019) to discriminate between realistic and unnatural test sequences.

Original: Further details are in
Revised: Further details are in

Original: Appendix E. As Table 1 indicates, SCOFF signiﬁcantly outperforms two competitors on all three tasks.
Revised: Appendix E. As Table 1 indicates, SCOFF signiﬁcantly outperforms two competitors on all three tasks.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: Understanding the visual world requires interpreting images in terms of distinct independent physical entities.
Revised: Understanding the visual world requires interpreting images in terms of distinct independent physical entities.

Original: These entities have persistent intrinsic properties, such as a color or velocity, and they have dynamics that transform the properties.
Revised: These entities have persistent intrinsic properties, such as a color or velocity, and they have dynamics that transform the properties.

Original: We explored a mechanism that is able to factorize declarative knowledge (the properties) and procedural knowledge (the dynamics).
Revised: We explored a mechanism that is able to factorize declarative knowledge (the properties) and procedural knowledge (the dynamics).

Original: Using attention, our SCOFF model learns this factorization into representations of entities— OF s—and representations of how they transform over time—schemata.
Revised: Using attention, our SCOFF model learns this factorization into representations of entities— OF s—and representations of how they transform over time—schemata.

Original: By applying the same schemata to multiple OF s, SCOFF achieves systematicity of prediction, resulting in signiﬁcantly improved generalization performance over state-of-the-art methods.
Revised: By applying the same schemata to multiple OF s, SCOFF achieves systematicity of prediction, resulting in signiﬁcantly improved generalization performance over state-of-the-art methods.

Original: It also addresses a fundamental issue in AI and cognitive science: the distinction between types and tokens .
Revised: It also addresses a fundamental issue in AI and cognitive science: the distinction between types and tokens .

Original: SCOFF is also interpretable, in that we can identify the binding between schemata and entity behavior.
Revised: SCOFF is also interpretable, in that we can identify the binding between schemata and entity behavior.

Original: The factorization of declarative and procedural knowledge has broad applicability to a wide variety of challenging deep learning prediction tasks.
Revised: The factorization of declarative and procedural knowledge has broad applicability to a wide variety of challenging deep learning prediction tasks

Original: A I MPLEMENTATION D ETAILS AND H YPERPARAMETERS
Revised: A I MPLEMENTATION D ETAILS AND H YPERPARAMETERS

Original: The model setup consists of three main components: an encoder, the process of interaction between object ﬁles and schemata followed by a decoder.
Revised: The model setup consists of three main components: an encoder, the process of interaction between object ﬁles and schemata followed by a decoder.

Original: The images are ﬁrst processed by an encoder, which is parameterized as a CNN.
Revised: The images are ﬁrst processed by an encoder, which is parameterized as a CNN.

Original: For experiments containing multiple entities, we use a spatial attention (i.e., output of the CNN preserves the spatial information) such that different OF s can attend to different spatial regions.
Revised: For experiments containing multiple entities, we use a spatial attention (i.e., output of the CNN preserves the spatial information) such that different OF s can attend to different spatial regions.

Original: Resources Used.
Revised: Resources Used.

Original: It takes about 2 days to train the proposed model on bouncing ball task forepochs on V100 (32G).
Revised: It takes about 2 days to train the proposed model on bouncing ball task forepochs on V100 (32G).

Original: We did not do any hyper-parameter search speciﬁc to a particular dataset (i.e 4Balls or 678Balls or Curtain Task).
Revised: We did not do any hyper-parameter search speciﬁc to a particular dataset (i.e 4Balls or 678Balls or Curtain Task).

Original: We ran the proposed model for different number of schemata (i.e 2/4/6).
Revised: We ran the proposed model for different number of schemata (i.e 2/4/6).

Original: Similarly, it takes about 3 days to run for 20M steps for the Reinforcement learning task.
Revised: Similarly, it takes about 3 days to run for 20M steps for the Reinforcement learning task.

Original: B A DDING T ASK
Revised: B A DDING T ASK

Original: We analyzed the proposed method on the adding task.
Revised: We analyzed the proposed method on the adding task.

Original: This is a standard task for investigating recurrent models (Hochreiter and Schmidhuber, 1997).
Revised: This is a standard task for investigating recurrent models (Hochreiter and Schmidhuber, 1997).

Original: The input consists of two co-occuring sequences: 1) N numbers ( a 0 · · · a N − 1 ) sampled independently from U [0 , 1] , 2) an index i 0 in the ﬁrst half of the sequence, and an index i 1 in the second half of the sequence together encoder as a one hot sequences.
Revised: The input consists of two co-occuring sequences: 1) N numbers ( a 0 · · · a N − 1 ) sampled independently from U [0 , 1] , 2) an index i 0 in the ﬁrst half of the sequence, and an index i 1 in the second half of the sequence together encoder as a one hot sequences.

Original: The target output is a i 

Revised: The target output is a i

Original: + a i 1 .
Revised: + a i 1 .

Original: As shown in ﬁgure 6 (a), we can clearly observe the factorisation of procedural knowledge into two schemata effectively, one schema is triggered when an operand is encountered and the other when non-operand is encountered.
Revised: As shown in ﬁgure 6 (a), we can clearly observe the factorisation of procedural knowledge into two schemata effectively, one schema is triggered when an operand is encountered and the other when non-operand is encountered.

Original: Generalization Result:
Revised: Generalization Result:

Original: For demonstrating the generalization capability of SCOFF , we consider a scenario where the models are trained to add a mixture of two and four numbers from sequences of length 50.
Revised: For demonstrating the generalization capability of SCOFF , we consider a scenario where the models are trained to add a mixture of two and four numbers from sequences of length 50.

Original: They are evaluated on adding variable number (2-10) of numbers on 200 length sequences.
Revised: They are evaluated on adding variable number (2-10) of numbers on 200 length sequences.

Original: As shown in Table 1, we note better generalization when using SCOFF .
Revised: As shown in Table 1, we note better generalization when using SCOFF .

Original: The dataset consists of 50, training sequences and 20,000 testing sequences for each different number of numbers to add.
Revised: The dataset consists of 50, training sequences and 20,000 testing sequences for each different number of numbers to add.

Original: All the models are trained for 100 epochs with a learning rate of 0.001 using the Adam optimizer.
Revised: All the models are trained for 100 epochs with a learning rate of 0.001 using the Adam optimizer.

Original: We use 300 as the hidden dimension for both the LSTM baseline and the LSTM’s in RIMS, SCOFF .
Revised: We use 300 as the hidden dimension for both the LSTM baseline and the LSTM’s in RIMS, SCOFF .

Original: Table 2 lists the different hyperparameters used for training SCOFF .
Revised: Table 2 lists the different hyperparameters used for training SCOFF .

Original: [Table]
Revised: [Table]

Original: C B OUNCING B ALL
Revised: C B OUNCING B ALL

Original: The dataset consists of 50,000 training examples and 10,000 test examples showing ∼ 50 frames of either 4 solid balls bouncing in a conﬁned square geometry ( 4Balls ), 6-8 balls bouncing in a conﬁned geometry ( 678Balls ), 3 balls bouncing in a conﬁned geometry with an occluded region ( Curtain )
Revised: The dataset consists of 50,000 training examples and 10,000 test examples showing ∼ 50 frames of either 4 solid balls bouncing in a conﬁned square geometry ( 4Balls ), 6-8 balls bouncing in a conﬁned geometry ( 678Balls ), 3 balls bouncing in a conﬁned geometry with an occluded region ( Curtain )

Original: [Table]
Revised: [Table]

Original: or balls of different colors ( Colored 678Balls ).
Revised: or balls of different colors ( Colored 678Balls ).

Original: The balls have distinct states (and hence distinct object ﬁles) but share underlying procedures (schema), which we aim to capture using SCOFF .
Revised: The balls have distinct states (and hence distinct object ﬁles) but share underlying procedures (schema), which we aim to capture using SCOFF .

Original: We trained baselines as well as proposed model for about 100 epochs. .
Revised: We trained baselines as well as proposed model for about 100 epochs. .

Original: We also provide the rollouts predicted by the models in the Figures 8, 10, 9, 11, 12.
Revised: We also provide the rollouts predicted by the models in the Figures 8, 10, 9, 11, 12.

Original: As part of future work, We also provide a scenario where instead of activating all the OF only a subset of the OF that are most relevant to the input based on attention scores are activated.
Revised: As part of future work, We also provide a scenario where instead of activating all the OF only a subset of the OF that are most relevant to the input based on attention scores are activated.

Original: We denote the original model as SCOFF -ab, and the ablation of only activating a subset of the OF as SCOFF .
Revised: We denote the original model as SCOFF -ab, and the ablation of only activating a subset of the OF as SCOFF .

Original: Comparison to the RMC baseline is shown in ﬁg. 7.
Revised: Comparison to the RMC baseline is shown in ﬁg. 7.

Original: D B ABY AI: R EINFORCEMENT L EARNING
Revised: D B ABY AI: R EINFORCEMENT L EARNING

Original: We use the GotoObjMaze environment (i.e MiniGrid-GotoObjMaze-v0) from (Chevalier-Boisvert et al., 2018).
Revised: We use the GotoObjMaze environment (i.e MiniGrid-GotoObjMaze-v0) from (Chevalier-Boisvert et al., 2018).

Original: Here, the agent has to navigate to an object, the object may be in another room.
Revised: Here, the agent has to navigate to an object, the object may be in another room.

Original: We use exactly the same RL setup as in (Chevalier-Boisvert et al., 2018) except we extend the setup in
Revised: We use exactly the same RL setup as in (Chevalier-Boisvert et al., 2018) except we extend the setup in

Original: BabyAI to only apply RGB images of the world rather than symbolic state representations, and hence making the task much more difﬁcult.
Revised: BabyAI to only apply RGB images of the world rather than symbolic state representations, and hence making the task much more difﬁcult.

Original: Hyper-parameters for this task are listed in Tab. 5.
Revised: Hyper-parameters for this task are listed in Tab. 5.

Original: In this environment, the agent is expected to navigate a 3x3 maze of 6x6 rooms, randomly interconnected by doors to ﬁnd an object like "key".
Revised: In this environment, the agent is expected to navigate a 3x3 maze of 6x6 rooms, randomly interconnected by doors to ﬁnd an object like "key".

Original: Here we use only one object ﬁle, but different number
Revised: Here we use only one object ﬁle, but different number

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Revised: schemas (4 in this example

Revised: If we

Original: E I NTUITIVE P HYSICS
Revised: E I NTUITIVE P HYSICS

Original: We use the similar training setup as (Riochet et al., 2019).
Revised: We use the similar training setup as (Riochet et al., 2019).

Original: Hyper-parameters related to the proposed method are listed in Tab. 6.
Revised: Hyper-parameters related to the proposed method are listed in Tab. 6.

Original: F S WITCHING D YNAMICS
Revised: F S WITCHING D YNAMICS

Original: We consider a scenario where a ball follows one of the two dynamics, horizontal and vertical oscillations at a given time.
Revised: We consider a scenario where a ball follows one of the two dynamics, horizontal and vertical oscillations at a given time.

Original: We limit the number of switches between the dynamics followed to be one.
Revised: We limit the number of switches between the dynamics followed to be one.

Original: An example ground truth trajectory is given in ﬁgure 14.
Revised: An example ground truth trajectory is given in ﬁgure 14.

Original: Here we run another experiment, where we have more number of OF s as compared to number of entities or objects, and then we investigate if we are still able to have factorization of different dynamics in different schemata.
Revised: Here we run another experiment, where we have more number of OF s as compared to number of entities or objects, and then we investigate if we are still able to have factorization of different dynamics in different schemata.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: F. E XPERIMENT S ETUP
Revised: F. E XPERIMENT S ETUP

Original: The dataset consists of 10,000 trajectories of 51 length with the switching between dynamics happening at the middle of the trajectory.
Revised: The dataset consists of 10,000 trajectories of 51 length with the switching between dynamics happening at the middle of the trajectory.

Original: [Table]
Revised: [Table]

Original: We use the same architecture for encoder as well as decoder as in (Van Steenkiste et al., 2018).
Revised: We use the same architecture for encoder as well as decoder as in (Van Steenkiste et al., 2018).

Original: At each time step, we give the last ﬁve frames stacked across the channels as input to the encoder.
Revised: At each time step, we give the last ﬁve frames stacked across the channels as input to the encoder.

Original: [Table]
Revised: [Table]

Original: F. T HE FACTORISATION OF DYNAMICS INTO SCHEMATA
Revised: F. T HE FACTORISATION OF DYNAMICS INTO SCHEMATA

Original: As shown in ﬁgure 15, SCOFF is effective in factorising the two different dynamics into two different schemata, even if the order of dynamics followed is different.
Revised: As shown in ﬁgure 15, SCOFF is effective in factorising the two different dynamics into two different schemata, even if the order of dynamics followed is different.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: G P SEUDOCODE FOR SCOFF A LGORITHM
Revised: G P SEUDOCODE FOR SCOFF A LGORITHM

Original:  we defined pos of positions  incoming input) num_ object files current layer dim the  number of hidden units in each object file bsz the batch size def scoff_
Revised: #Assume we have defined: pos_in (number of positions in the incoming input), num_OF_out (number of object files on the current layer), of_dim (the number of hidden units in each object file), bsz (the batch size). #Step fully convolutional net. #Step def scoff_

Original: : inp = self
Revised: (inp, h): inp = self.

Revised: CNN(inp)

Revised: att_inp = objectfile_input_selection(inp, h) h_next,schema_indices = schemata_selective_update(att_inp, h) h_next

Original: (inp
Revised: = h_next + objectfile_communication(h_next, schema_indices) return h_next def objectfile_input_selection(inp, h): inp = inp.reshape((bsz, pos_in, OF_dim)) att_inp,_ = self.inp_attn(h.reshape((bsz, num_OF_out, OF_dim)), inp, inp) att_inp = att_inp.reshape((bsz, of_dim*num_OF_out)) return att_inp def schemata_selective_update(inp_to_objfile, h): h_all =

Original:  #We batch the rest of these operations over the object files.
Revised: torch.zeros((bsz*num_OF, num_schemata, OF_dim)) #We batch the rest of these operations over the object files.

Original:  for j in range (num_schemata): h_all[:,j] = self.gru_lst[j](inp_to_objfile, h) key[:, j] = self.key(h_all[:,j]) q = self.query(h) h_sel, ind = gumbel_softmax(torch.bmm(key, q.unsqueeze(1))) return h_sel.reshape((bsz,num_OF,-1), ind.reshape((bzs,num_OF,-1))
Revised: inp_to_objfile = inp_to_objfile.reshape((bsz*num_OF, OF_dim)) h = h.reshape((bsz*num_OF, OF_dim)) for j in range (num_schemata): h_all[:,j] = self.gru_lst[j](inp_to_objfile, h) key[:, j] = self.key(h_all[:,j]) q = self.query(h) h_sel, ind = gumbel_softmax(torch.bmm(key, q.unsqueeze(1))) return h_sel.reshape((bsz,num_OF,-1), ind.reshape((bzs,num_OF,-1))

Original: def objectfile_communication(h, schema_indices): h = h.reshape((bsz, num_OF_out, of_dim))
Revised: def objectfile_communication(h, schema_indices): h = h.reshape((bsz, num_OF_out, of_dim))

Original: q = self.comm_query(h)[:,schema_indices] k = self.comm_key(h)[:,schema_indices] v = self.comm_value(h)[:,schema_indices] h_att,_ = self.comm_attn(q, k, v) h_att
Revised: q = self.comm_query(h)[:,schema_indices] k = self.comm_key(h)[:,schema_indices] v = self.comm_value(h)[:,schema_indices] h_att,_ = self.comm_attn(q, k, v) h_att

Original: = h_att.reshape((bsz, num_OF_out*of_dim)) return h_att
Revised: = h_att.reshape((bsz, num_OF_out*of_dim)) return h_att

Original: We show how the attention scores from attention are used to re-weight the input to the different object ﬁles (Step 2 in the box).
Revised: We show how the attention scores from attention are used to re-weight the input to the different object ﬁles (Step 2 in the box).

Original: Then different object ﬁles select particular schemata (Step 3 in algorithm), and then different object ﬁles transmit relevant information (Step 4 in algorithm).
Revised: Then different object ﬁles select particular schemata (Step 3 in algorithm), and then different object ﬁles transmit relevant information (Step 4 in algorithm).