  Visual environments are structured, consisting of distinct objects or
entities. These entities have properties -- both visible and latent -- that
determine the manner in which they interact with one another. To partition
images into entities, deep-learning researchers have proposed structural
inductive biases such as slot-based architectures. To model interactions among
entities, equivariant graph neural nets (GNNs) are used, but these are not
particularly well suited to the task for two reasons. First, GNNs do not
predispose interactions to be sparse, as relationships among independent
entities are likely to be. Second, GNNs do not factorize knowledge about
interactions in an entity-conditional manner. As an alternative, we take
inspiration from cognitive science and resurrect a classic approach, production
systems, which consist of a set of rule templates that are applied by binding
placeholder variables in the rules to specific entities. Rules are scored on
their match to entities, and the best fitting rules are applied to update
entity properties. In a series of experiments, we demonstrate that this
architecture achieves a flexible, dynamic flow of control and serves to
factorize entity-specific and rule-based information. This disentangling of
knowledge achieves robust future-state prediction in rich visual environments,
outperforming state-of-the-art methods using GNNs, and allows for the
extrapolation from simple (few object) environments to more complex
environments.
