  Neural networks have achieved success in a wide array of perceptual tasks,
but it is often stated that they are incapable of solving tasks that require
higher-level reasoning. Two new task domains, CLEVRER and CATER, have recently
been developed to focus on reasoning, as opposed to perception, in the context
of spatio-temporal interactions between objects. Initial experiments on these
domains found that neuro-symbolic approaches, which couple a logic engine and
language parser with a neural perceptual front-end, substantially outperform
fully-learned distributed networks, a finding that was taken to support the
above thesis. Here, we show on the contrary that a fully-learned neural network
with the right inductive biases can perform substantially better than all
previous neural-symbolic models on both of these tasks, particularly on
questions that most emphasize reasoning over perception. Our model makes
critical use of both self-attention and learned "soft" object-centric
representations, as well as BERT-style semi-supervised predictive losses. These
flexible biases allow our model to surpass the previous neuro-symbolic
state-of-the-art using less than 60% of available labelled data. Together,
these results refute the neuro-symbolic thesis laid out by previous work
involving these datasets, and they provide evidence that neural networks can
indeed learn to reason effectively about the causal, dynamic structure of
physical events.
