Paper:
Published as a conference paper at ICLR 2018 RELATIONAL NEURAL EXPECTATION MAXIMIZATION: UNSUPERVISED DISCOVERY OF OBJECTS AND THEIR INTERACTIONS
Common-sense physical reasoning is an essential ingredient for any intelligent agent operating in the real-world. For example, it can be used to simulate the environment, or to infer the state of parts of the world that are currently unobserved. In order to match real-world conditions this causal knowledge must be learned without access to supervised data. To address this problem we present a novel method that learns to discover objects and model their physical interactions from raw visual images in a purely unsupervised fashion. It incorporates prior knowledge about the compositional nature of human perception to factor interactions between object-pairs and learn efficiently. On videos of bouncing balls we show the superior modelling capabilities of our method compared to other unsupervised neural approaches that do not incorporate such prior knowledge. We demonstrate its ability to handle occlusion and show that it can extrapolate learned knowledge to scenes with different numbers of objects.

Review:
Summary
---
This work applies a representaion learning technique that segments entities to learn simple 2d intuitive physics without per-entity supervision. It adds a relational mechanism to Neural Expectation Maximization and shows that this mechanism provides a better simulation of bouncing balls in a synthetic environment.
Neural Expectation Maximization (NEM) decomposes an image into K latent variables (vectors of reals) theta_k. A decoder network reconstructs K images from each of these latent variables and these K images are combined into a single reconstruction using pixel-wise mixture components that place more weight on pixels that match the ground truth. An encoder network f_enc() then updates the latent variables to better explain the reconstructions they produced.
The neural nets are learned so that the latent variables reconstruct the image well when used by the mixture model and match a prior otherwise. Previously NEM has been shown to learn variables which represent individual objects (simple shapes) in a compositional manner, using one variable per object.
Other recent neural models can learn to simulate simple 2d physics environments (balls bouncing around in a 2d plane). That work supervises the representation for each entity (ball) explicitly using states (e.g. position and velocity of balls) which are known from the physics simulator used to generate the training data. The key feature of these models is the use of a pairwise embedding of an object and its neighbors (message passing) to predict the object's next state in the simulation.
This paper paper combines the two methods to create Relational Neural Expectation Maximization (R-NEM), allowing direct interaction at inference time between the latent variables that encode a scene. The encoder network from NEM can be seen as a recurrent network which takes one latent variable theta_k at time t and some input x to produce the next latent variable theta_k at time t+1. R-NEM adds a relational module which computes an embedding used as a third input to the recurrent encoder. Like previous relational models, this one uses a pairwise embedding of the object being updated (object k) and its neighbors. Unlike previous neural physics models, R-NEM uses a soft attention mechanism to determine which objects are neighbors and which are not. Also unlike previous neural models, this method does not require per-object supervision.
Experiments show that R-NEM learns compositional representations that support intuitive physics more effectively than ablative baselines. These experiments
show:
1) R-NEM reconstructs images more accurately than baselines (RNN/LSTM) and NEM (without object interaction).
2) R-NEM is trained with 4 objects per image. It does a bit worse at reconstructing images with 6-8 objects per image, but still performs better than baselines.
3) A version of R-NEM without neighborhood attention in the relation module matches the performance of R-NEM using 4 objects and performs worse than R-NEM at 6-8 objects.
4) R-NEM learns representations which factorize into one latent variable per object as measured by the Adjusted Rand Index, which compares NEM's pixel clustering to a ground truth clustering with one cluster per object.
5) Qualitative and quantitative results show that R-NEM can simulate 2d ball physics for many time steps more effectively than an RNN and while only suffering gradual divergence from the ground truth simulation.
Qualitative results show that the attentional mechanism attends to objects which are close to the context object together, acting like the heuristic neighborhood mechanism from previous work.
Follow up experiments extend the basic setup significantly. One experiment shows that R-NEM demonstrates object permanence by correctly tracking a collision when one of the objects is completely occluded. Another experiment applies the method to the Space Invaders Atari game, showing that it treats columns of aliens as entities. This representation aligns with the game's goal.
Strengths
---
The paper presents a clear, convincing, and well illustrated story.
Weaknesses
---
* RNN-EM BCE results are missing from the simulation plot (right of figure 4).
Minor comments/concerns:
* 2nd paragraph in section 4: Are parameters shared between these 3 MLPs (enc,emb,eff)? I guess not, but this is ambiguous.
* When R-NEM is tested against 6-8 balls is K set to the number of balls plus 1? How does performance vary with the number of objects?
* Previous methods report performance across simulations of a variety of physical phenomena (e.g., see "Visual Interaction Networks"). It seems that supervision isn't needed for bouncing ball physics, but I wonder if this is the case for other kinds of phenomena (e.g., springs in the VIN paper). Can this method eliminate the need for per-entity supervision in this domain?
* A follow up to the previous comment: Could a supervised baseline that uses per-entity state supervision and neural message passsing (like the NPE from Chang et. al.) be included?
* It's a bit hard to qualitatively judge the quality of the simulations without videos to look at. Could videos of simulations be uploaded (e.g., via anonymous google drive folder as in "Visual Interaction Networks")?
* This uses a neural message passing mechanism like those of Chang et. al. and Battaglia et. al. It would be nice to see a citation to neural message passing outside of the physics simulation domain (e.g. to "Neural Message Passing for Quantum Chemistry" by Gilmer et. al. in ICML17).
* Some work uses neighborhood attention coefficients for neural message passing. It would be nice to see a citation included.
    * See "Neighborhood Attention" in "One-Shot Imitation Learning" by Duan et. al. in NIPS17
    * Also see "Programmable Agents" by Denil et. al.
Final Evaluation
---
This paper clearly advances the body of work on neural intuitive physics by incorporating NEM entity representation to allow for less supervision. Alternatively, it adds a message passing mechanism to the NEM entity representation technique. These are moderately novel contributions and there are only minor weaknesses, so this is a clear accept.