Attention over Learned Object Embeddings Enables Complex Visual Reasoning
Authors: David Ding
Felix Hill
Adam Santoro
Malcolm Reynolds
Matthew Botvinick
Venue: NeurIPS 2021 Oral (2021)
Keywords: Spatio-temporal reasoning
self-attention
transformers
object attention
visual question answering

Abstract: Neural networks have achieved success in a wide array of perceptual tasks but often fail at tasks involving both perception and higher-level reasoning. On these more challenging tasks, bespoke approaches (such as modular symbolic components, independent dynamics models or semantic parsers) targeted towards that specific type of task have typically performed better. The downside to these targeted approaches, however, is that they can be more brittle than general-purpose neural networks, requiring significant modification or even redesign according to the particular task at hand. Here, we propose a more general neural-network-based approach to dynamic visual reasoning problems that obtains state-of-the-art performance on three different domains, in each case outperforming bespoke modular approaches tailored specifically to the task. Our method relies on learned object-centric representations, self-attention and self-supervised dynamics learning, and all three elements together are required for strong performance to emerge. The success of this combination suggests that there may be no need to trade off flexibility for performance on problems involving spatio-temporal or causal-style reasoning. With the right soft biases and learning objectives in a neural network we may be able to attain the best of both worlds.    


Attention over learned object embeddings enables complex visual reasoning
David Ding
Felix Hill
Adam Santoro
Malcolm Reynolds
Matt Botvinick
DeepMind
London, United Kingdom
(fding, felixhill, adamsantoro, mareynolds, botvinick (egpa/iiick@@oolll....oooooo @google.com com
Abstract
Neural networks have achieved success in a wide array of perceptual tasks but often fail tasks involving both perception and higher-level reasoning. On these more challenging tasks, bespoke approaches (such as modular symbolic components, in- dependent dynamics models semantic parsers) largeted towards that specific type of task have typically performed better. The downside to these targeted approaches, however, is that they can be more brittle than general-purpont neural networks, requiring sigmificant modification or even redesign according to the particular task at hand. Here, we propose more general netrolkn based approach dynamic visual reasoning problems that obtains state-of-the performance on three different domains, in each case outperforming bespoke modular approaches tailored specifically to the task. Our method relies on learned object-centric rep- resentations, self-attention and self-supervised dynamics learning, and all three elements together required for strong performance this combination suggests that there may no need to trade off flexibility for performance problems involving spatio-temporal or causal-style reasoning. With the right soft biases and learning objectives neural network we may be able attain the best of both worlds.
emerge. The success
Introduction
Despite the popularity of artificial neural networks, body of recent work has focused on their limitations as models cognition and reasoning. Experiments with dynamical reasoning datasets such as CLEVRER [11. CATER 11), and ACRE 44 show that neural networks can fail to adequately reason about the spatio-temporal, compositional or causal structure of visual scenes. On CLEVRER, where models must answer questions about the dynamics of colliding objects, previous experiments show that neural networks can adequately describe the vidco, but fail when asked to predict, explain, consider counterfactual possibilities. Similarly, CATER, object-tracking task, models have trouble tracking the movement of objects when they are hidden container. Finally, ACRE, dataset testing for causal inference, popular models only learned correlations between visual scenes and not the deeper causal logic.
Failures such these reasoning (rather than perception) problems have motivated the adoption of pipeline-style approaches that combine a general purpose neural network (such as a convolutional block) with a task-specific module that builds in the core logic of the task. For example, on CLEVRER the NS-DR method [41] applies a hand-coded symbolic logic engine (that has the core logic of CLEVRER built-in) to the outputs of "perceptual" neural front-end, achieving better results than neural network baselines, particularly on counterfactual and explanatory problems. One
Model Code:
object_attente attentionn for Berph__oo_rrrrreee reasoning
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

limitation of these pipeline approaches, however, is that they are typically created with single problem or problem domain mind, and may not apply out-of-the-box other related problems. For example, to apply NS-DR to CATER, the entire symbolic module needs to be rewritten to handle the new interactions and task logic of CATER: the custom logic to handle collisions and object removal must be replaced with new custom logic to handle occlusions and grid-resolution, and these changes require further modifications to the perceptual front-end to output data in a new format. This brittleness is not exclusive to symbolic approaches. While Hungarian-matthhn
between object embeddings may be well-suited for object-tracking tasks 45.. it is not obvious how it would help for causal inference tasks.
Here, we describe more general neural-network- bebser approach to visual spatio-temporal reasoning problems, which does not rely on task-specific integration of modular components. In place of these components, our model relies on three key aspects:
Self-attention to effectively integrate information over time
Soft-discretization of the input at the most informative level of abstraction above pixels and local features, and below entire trames-correspooete
approximately to 'objects' Self-supervised learning, i.c. requiring the model to infer masked out objects, to extract more information about dynamics from each sample.
While many past models have applied each individual ingredient separately (including on the tasks study), we show that it is the combination of all three ingredients in the right way that allows model to succeed.
The resulting model, which we call Aloe (Attention over Learned Object Embeddings), outperforms both pipeline and neural seerralnetworr-baaee approaches on three different task domains designed to test physical and dynamical reasoning from pixel inputs. We highlight our key results here:
CLEVRER (explanatory, predictive, and counterfactual reasoning): Aloe achieves significantly higher accuracy than both more task-specific, modular approaches, and previous neural network methods on all question types. On counterfactual questions, thought to be most challenging for neural-only architectures, we achieve % vs 46 % accuracy for more specialised methods.
CATER (object-permancet perrmanceer Aloe achieves accuracy exceeding or matching other current models. Notably, the strongest alternative models were expressly designed for abject-tracking whereas our architecture is applicable without modification to other reasoning tasks as well.
ACRE (causal-inference "beyond the simple strategy of inducing causal relationships by covariation" " 44 Overall, Aloe achieves 94% vs the 67% accuracy achieved by the top neuro-symbolic model. On the most challenging tasks, we achieve, for baockward-blocking inference, 94.48 % (vs 16.06% by the best modular, neuro suurolyymbolic systems), and, for "screen- off" inference, 98.97% % (vs 0.00% % by a CNN-BERT baseline).
As we have emphasized, the previous best performing models for each task all contain task-specific design elements, whereas Aloe can be applied to all the tasks without modification. On CLEVRER, we also show that Aloe matches the performance of the previous best models with 40% less training data, which demonstrates that our approach is data-efficient as well as performant.
2 Methods
guiding motivation for the design of Aloe is the converging evidence for the value self-attention mechanisms operating on finite sequences of discrete entities. Written language is inherently discrete and hence is well-suited to self-attention-base approaches. In other domains, such as raw audio or vision, it is less clear how to leverage self-attention. We hypothesize that the application of self-attentionn-art based models to visual tasks could benefit from an approximate 'discretization process, and determining the right level of discretization is important choice that can significantly affect model performance.
At the finest level, data could simply be discretized into pixels (as is already the case for most machine- processed visual data). Pixels are too fine- mme-grained for many applications, however-for one, the memory required to support self-attention across all pixels is prohibitive. Partly for this reason,
2

---

Self-supervised learning
Supervised learning yes
Linear
Lineer softmax
Transformer + Position Encoding
A
MONet
Embedding lookup
will the red triangle
the green square <<->>
Figure 1: A schematic of the model architecture. See the main text for details.
coarser representations, such the downsampled "hyper-pixel" outputs of convolutional network, are often used instead (e.g. [27 43).. In the case of vidcos, previous work considered even coarser discretization schemes, such as frame or subclip level representations (35).
The neuroscience literature, however, suggests that biological visual systems infer and exploit the existence of objects, rather than spatial or temporal blocks with artificial boundaries [
30 32
Because objects are the atomic units of physical interactions, it makes sense to discretize on the level of objects. Numerous object segmentation algorithms have been proposed [95] 29). We chose to use MONet, an unsupervised object segmentation algorithm (2]. Because MONet is unsupervised, can train it directly in our domain of interest without the need for object segmentation labels. We emphasize that our choice of MONet is an implementation detail, and in Appendix [B] we show that our framework of attention over learned object embeddings also works with other spiee-seemmnntttion schemes. We also do not need to place strong demands on the object segmentation algorithm, c.g. for it produce aligned output or have a built-in dynamics model.
To segment each frame into object representations, MONet uses a recurrent attention network to obtain set of No "object attention masks" (N, is fixed parameter). Each attention mask represents the probability that any given pixel belongs to that mask's object. The pixels assigned to the mask are encoded into latent variables with means pes IRd, where indexes the object slot and the frame. These means are used as the object embeddings in Aloe. More details are provided in Appendix A
The self-attention component is transformer model 337 operating Rd: the object representations μąa for all t and i, a trainable vector CLS e Rd used to generate classification results (analogous to the CLS token in BERT [9), and (for CLEVRER) the embedded words w, from the question (and choice for multiple choice questions). For the object representations H4i and word embeddings w,, append two-dinensional one-hot vector (4: and w, indicate whether the input is word or an object. Because the transformer is shared between the modalities, information can flow between objects and words to solve the task,
sequence of vectors in
show in Section 3.1
We pass this sequence of vectors through a transformer with Ny layers. All inputs are first projected (via a linear layer and ReL.U activation) to R xd, where N₂ is the number of self-attention heads. We add a relative sinusoidal positional encoding at each layer of the transformer to give the model knowledge of the word and frame order 77. The transformed value of C'LS is passed through an MLP (with one hidden layer of size NH) to generate the final answer. A schema of our architecture shown in Figure T
Note that in the model presented above (which we call global attention). the transformer sees no distinction between objects of different frames (other than through the position encoding). Another intuitive choice, which we call hierarchical atlention, is to have one transformer acting on the objects of each frame independently, and another transformer acting on the concatenated outputs of the first transformer (this temporal division of input data commonly used, e.g. in (55). In pseudo-code, global attention can be expressed as
3

---

a) Single object per frame
c) Object prediction
e) Object infill
Estimaation
Context (C-2)
Mask (M-22)
Estiimation step
Context (C=1)
Mask (Mail)
Fatimetion
Maak
Context
stops
step
b) Multiple objects per frame
Scene prediction
Scene infill
Targens
Inpurt
Estimation stepa
Contect
Mask (M:2)
Estimation
(C=1)
(Mal)
Estimation
(C=2)
step
stap
Figure 2: Different masking schemes for self-supervised learning applied to Aloe.
out transformer (reshape (objects, [B, F N, D1) and hiearchical attention as
out transformerl (reshape (objects, [B F, N, D1))
out transformer2 (reshape (out, IB, E, N DI))
We study the importance global attention (objects as the atomic entities) vs hierarchical attention (objects, and subsequently frames as the atomic entities). The comparison is shown in Table
2.1 Self-supervised learning
We explored whether self-supervised learning could improve the performance of Aloe beyond the benefits conveyed by object-level representation, i.e. in ways that support the model's s interpretation of scene dynamics rather than just via improved perception of static observations. Our approach is inspired by the loss used in BERT [9. where transformer model is trained to predict certain words that are masked from the input. our case, we mask object embedldnngs, and train the model to infer the content of the masked object representations using its knowledge of unmasked objects.
Concretely, during training, we multiply each MONet latent 14 by a masking indicator, my {0, 1]. Let U4: be the transformed value of mailt after passing through the transformer. We expect the transformer to understand the underlying dynamics of the video, so that the masked slot pei could be predicted from M/g. To guide the transformer in learning effective representations capable of this type of dynamics prediction, we add an auxiliary loss:
auxiliary loss
Σ Tul (f(pl)) t,
where is learned linear mapping to R&, l loss function, and {0, 1) are one-hot indicator variables identifying the prediction targets (not necessarily just the masked out entries, since the prediction targets could be subset of the masked out entries). We propagate gradients only to the parameters of f and the transformer and not to the learned word and CLS embeddings. This auxiliary loss is added the main classification loss with weighting X, and both losses are minimized simultaneously by the optimizer. We do not pretrain the model with only the auxiliary loss.
We tested two different loss functions for 1, an L2 loss and a contrastive loss (formulas given in Appendix A.2)). and six different masking schemes (settings of m,; and Tr.). as illustrated Figure This exploration was motivated by the observation that video inputs at adjacent timesteps are highly correlated in a way that adjacent words are not. We thus hypothesized that BERT-style prediction of adjacent words might not be optimal. A different masking strategy, in which prediction targets

---

separated from the context by more than a single timestep, may stimulate capacity in the network to acquire knowledge that permits context-based unrolls and better long-horizon predictions.
The simplest approach would be to set mai = uniformly at random across t and i, fixing the expected proportion of the me set to 1 (schema b in Figure 2 The targets would simply be the unmasked slots, Tts = 1 Mas. One potential problem with this approach is that multiple objects could be masked out in a single frame. MONet can unpredictably switch object-to-slot assignments multiple times in a single video. If multiple slots are masked out, the transformer cannot determine with certainty which missing object to assign to each slot. Thus, the auxiliary loss could penalize the model even if it predicted all the objects correctly. To avoid this problem, we also try constraining the mask such that exactly one slot masked out per frame (schema a).
To pose harder prediction challenges, we can add a buffer between the context (where m.; = 1) and the infilling targets (where In = 1). For & in this buffer zone, both max and тыi (schemas c-f). We choose a single cutoff 7 randomly, and we set Ma = 0 for t < T and myi = 1 for t 2 T. In the presence of this buffer, we compared prediction (where the context strictly before the targets; schema c, d) versus infilling (where the context surrounds the targets; schema f) We also compared setting the targets as individual objects (schema c, e) versus targets all objects in the scene (schema f. We visually inspect the efficacy of this self-supervised loss in encouraging better representations (beyond improvements of scores on tasks) in Appendix D
3 Experiments
We tested Aloe on three datasets, CLEVRER 41,, CATER 12], and ACRE [44]. For each dataset, pretrained a MONet model on individual frames. More training details and a table of hyperparameters given in Appendix AA.] these hyperparameters were obtained through hyperparameter sweep. All error bars are standard deviations computed over at least random seeds.
3.1 CLEVRER
CLEVRER features videos of CLEVR objects [2]] that move and collide with each other. For each video, several questions are posed to test the model's understanding of the scene. Unlike most other visual question answering datasets, which test for only descriptive understanding ("what happened?"), CLEVRER poses other more complex questions, including explanatory questions ("why did something happen?"), predictive questions ("what will happen next?"), and counterfactual questions ("hhat would happen in a unseen circumstance?")
We compare Aloe to state-of-the-art models reported in the literature: MAC (V+) and NS-DR [1 as well as the DCL model 6 (simultaneous to our work). MAC (V+) (based on the MAC network (2 is an end-to-end network augmented with object information and trained using ground truth labels for object segmentation masks and features (c.g. color, shape). NS-DR and DCL are hybrid models that apply a symbolic logic engine to outputs of various neural networks. The neural networks are used to detect objects, predict dynamics, and parse the question into a program, and the symbolic executor runs the parsed program to obtain the final output. NS-DR is trained using ground truth labels and ground truth parsed programs, while DCL requires only the ground truth parsed programs.
Table shows the result of Aloe compared to these models. Across all categories, Aloe significantly outperforms the previous best models. Moreover, compared to the other models, Aloe does not use any labeled data other than the correct answer for the questions, nor does it require pretraining on any other dataset. Aloe also was not specifically designed for this task, and it straightforwardly generalizes to other tasks as well, such CATER [12] and ACRE [44). We provide a few sample model classifications on a randomly selected set of vidcos and questions in Appendix E. and detailed analysis of counterfactual questions in Appendix C These examples suggest qualitatively that, for most instances where the model was incorrect, humans would plausibly furnish the same answer.
Attention analysis (More analyses are given in Appendix DD We analyzed the cross-modal attention between question- words and the MONet objects. For each word, we determined the object that attended to that word with highest weight (for one head in the last layer). the visualization below, the bounding boxes show the objects found by MONet, and each word is colored according to the
5

---

Model
Descriptive
Explanatory 22.3
Predictive 42.9
Counterfactual
MAC (V+)
86,4
25.1
NS-DR
88.1
79.6
68.7
42.2
DCL
90.7
82.8
82.0
46.5
Aloe
94.0 0.4
96.0 0.6
87.5 + 3.0
75.6 3.8
Aloe self-attention MLP
45.4
16.0
27.7
9.9
Aloe object-repr. + ResNet
74.9
66.1
58.3
32.4
Aloe global hierarchical attn.
80.6
87.4
73.5
55.1
Aloe self-supervised loss
91.0
92.8
82.8
68.7
Table 1: Performance (per question accuracy) on CLEVRER of Aloe compared to results from
literature and to ablations: 1) MLP instead of self-attention; 2) ResNet superpixels instead of MONet objects; 3) hierarchical frame-level and intra-frame attention instead of global cross-frame object attention; 4) auxiliary loss.
object that attended to it with highest weight (black represents MONet slot without any objects). We observe that generally, objects attend heavily to the words that describe them.
Q: If the cylinder is removed, which event will not happen?
1. The brown object collides with the green object.
2. The yellow object and the metal cube collide.
3. The yellow cube collides with the green object.
We also looked at the objects that were most heavily attended upon in determining the final answer. The image below illustrates the attention weights for the CLS token attending each object (for one head in the last layer), when the model tasked with assessing the first choice of the question above. The bounding boxes show the two most heavily attended upon objects for one transformer head. We observe that this head focuses on the green and brown objects (asked about in choice 1), but switches its focus to the cyan cylinder when it looks like the cylinder might collide with the cubes and change the outcome.
Model ablation T shows the contributions of various components of Aloe. First, self-attention necessary for solving this problem. For comparison, replace Aloe's transformer with four fully connected layers with 2048 units per layer"] We find that an MLP is unable to answer non-descriptive questions effectively, despite using more parameters (20M 15M parameters).
Second, we verify that an object-based discretization scheme essential to the performance of Aloe. We compare with a version of the architecture where the MONet object representations Pea are replaced with ResNet hyperpixels as in Zambaldi et al. [43]. Concretely, we flatten the output of the final convolutional layer of the ResNet to obtain sequence of feature vectors that is fed into the transformer as the discrete entities. To match MONet's pretraining regimen, we pretrain the ResNet on CLEVR (1 by training Aloe model (using a ResNet instead of MONet) the CLEVR task and initializing the ResNet used in the CLEVRER task with these pre-trained weights. We find that an object level representation, such one output by MONet, greatly outperforms the locality- -ware but object-agnostic ResNet representation.
We also observe the importance of global attention between all objects across all frames, compared to a hierarchical attention model where objects within frame could attend to each other but frames could only attend to each other atomic entity. We hypothesize that global attention may be
We also tried a bidirectional LSTM, which achieved even lower performance. This may be because the structure of inputs requires the learning of long- range dependencies.

---

Descriptive
Explanatory
Predictive
Counterfactual
1.0
Auxiiary Less
0.0
With Auxliary Loss
Accuracy
0.6
No Auailiary Loss With Ausillary Loss
0.2
112 No Auellaary Loss
No Auxiliary Loss With Auxilaayy Loss
Auxiliary
0.0
0.2
0.4 0.5
0.8
1.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4 0.5 n.8 1.0
0.2
0.4
0.6
0.8
1.0
Proportion of dataset
Propartion of dataset
Proportion of dataset
Praportion of dataset Top 5 Acc Moving
Top Acc
Top Acc Moving
Top 5 Acc
1.0
No Ausiliary loss Auxillary
No Auxilary Loss
With Auxilary Loss
Acturacy
0.6
0.4
0.2
No Auxiliary Loss With Auxiliary Loss
No Auxilaary Less WNihh Auxiliary Loss
0.0
0.8
0 Proportion of dataset
Proportion of dataset
Proportion dataset
Proportion dataset
Figure 3: Accuracy with/without auxiliary loss for different proportions of CLEVRER (row 1) and CATER (row 2) training data. We also show comparisons with previous and concurrent work. For CLEVRER, the lighter yellow bar represents the best neurosymbolic model DCL, and the darker yellow bar represents the previous best distributed model, MAC (V+). For CATER, the lighter yellow bar represents Hopper and the darker yellow bar represents R3D+NL, the best published results for the moving camera dataset.
important because with hierarchical attention, objects in different frames can only attend to each other at the "frame" granularity. A cube attending to cube in a different frame would then gather information about the other non-cube objects, muddling the resulting representation.
Finally, we see that an auxiliary self-supervised loss improves the performance of the model by between 4 and 6 percentage points, with the greatest improvement on the counterfactual questions.
Self-supervision strategies We compared the various masking schemes and loss functions for our auxiliary loss; a detailed figure is provided in Appendix[A (Figure 4 We find that for all question types in CLEVRER, an L2 loss performs better than contrastive loss, and among the masking schemes, masking one object per frame is the most effective. This particular result runs counter to our hypothesis that predictions or infilling in which the target is temporally removed from the context could encourage the model to learn more about scene dynamics and object interactions than (BERT-style) local predictions of adjacent targets. Of course, there may be other settings or loss functions that reveal the benefits of non-local prediction or constrastive losses; we leave this investigation to future work.
Data efficiency We investigated how model performance varies as function of the number of labelled (question-answer) pairs learns from. To so, we train models on N% of the videos and their associated labeled data. We evaluate the effect of including the auxiliary self-supervieed loss (applied to the entire datasct, not just the labelled portion) in this low data regime. This scenario, where unlabeled data is plentiful while labeled data is scarce, occurs frequently in practice, since collecting labeled data is much more expensive than collecting unlabeled data.
Figure 3 shows that our best model reaches the approximate level of the previous state-of of-the-art approaches using only 50%-60% of the data. The self-suuervised auxiliary loss makes particular improvement to performance in low-data regimes. For instance, when trained on only 50% of the available labelled data, self-supervised learning enables the model to reach a performance of 37% on counterfactual questions (compared to 25% by MAC (V+) and 42% by NS-DR on the full dataset), while without self-supervision, the model only reaches a performance of 13% (compared to the 10% achieved by answering randomly (1)).
7

---

Model
Top (S)
Top 5 (S)
L1 (S)
Top (M)
Top 5 (M)
L1 (M)
R3D LSTM
60.2
81.8
1.2
28.6
63.3
1.7
R3D NL LSTM
46.2
69.9
1.5
38.6
70.2
1.5
OPNet
74.8
0.54
Hopper
73.2
93.8
0.85
Aloe (no auxiliary)
60.5
84.5
0.90
46.8
75.1
1.3
Aloe
70.6
93.0
0.53
56.6
87.0
0.82
Aloe (with L1 loss)
74.0 = 0.3
94.0 + 0.4
0.44 + 0.01
59.7 = 0.5
90.1 + 0.6
0.69 = 0.01
Table 2: Performance CATER of Aloe compared to the best results from literature. We report top 1 accuracy, top accuracy, and LI distance between the predicted grid cell and true grid cell. The labels (S) and (M) refer to static and moving cameras.
3.2 CATER
In a second experiment, we tested Aloe on CATER, a widely-used object-tracking dataset 12
45). In CATER, objects from the CLEVR dataset [2]] and potentially occlude other objects, and the goal is to predict the location of a target object (called the snitch) in the final frame. Because the snitch could be occluded by multiple objects that could move in the meantime, successful model must be sensitive to notions of object permanence. CATER also includes a moving camera variant, which introduces additional complexities for the model.
Concretely, CATER is setup as a classification challenge. Objects are located in an zyz coordinate system, where and range from -3 to 3. The zy plane is divided into 6 by 6 grid, and the task is to predict the grid index the snitch in the final frame. For Aloe, we use a classification loss (cross entropy over the 36 possible grid indices) and an L1 loss (LI distance between predicted grid cell and the true grid cell).
Table 2 shows Aloe compared to state oftte-of- models in the literature on both static and moving camera videos. R3D and R3D NI. are the strongest two models evaluated by Girdhar and Ramanan [12. OPNet, or the Object Permanence Network [31], is an architecture with inductive biases designed for object tracking tasks; was trained with extra supervised labels, namely the bounding boxes for all objects (including occluded ones). Hopper is a multi-hop transformer model developed simultaneously with this work [45]. One key component of Hopper is Hungarian matching between objects of different frames, strong inductive bias for object tracking.
We train Aloe simultaneously on both static and moving camera videos. Aloe outperforms the R3D models for both static and moving cameras. We also ran Aloe with an additional auxiliary loss consisting of the L.1 distance between the predicted cell and the actual cell. With this additional loss, we get comparable results in the moving camera case as the R3D models for the static camera case. Morcover, we achieve comparable accuracy as OPNet for accuracy and LI distance, despite requiring less supervision to train. Appendix E.2 gives few sample outputs from Aloe; in particular we note that it is able to find the target object in several cases where the object was occluded, demonstrating that Aloe is able to do some level of object tracking. Finally,we find that auxiliary self-supervised loss helps the model perform well in the low data regime for CATER as well, as shown in Figure
3.3 ACRE
Finally, we measured Aloe's performance on ACRE, a causal induction dataset inspired by the Blicket task from developmental psychology [13 ACRE is divided into a set of problems. In each problem, certain objects are chosen to be "Blickets", and this assignment changes across problems. Each problem presents context of six images the model, where different objects are placed Blicket machine that lights up if one of those objects is Blicket. The model is asked whether an unseen combination of objects will light up the Blicket machine. Besides "yes" and "no", third possible answer is "undetermined" which is the case if it is impossible to determine for certain the objects will light up the machine. Correct inference goes beyond mere correlation: even every context scene involving object A has lit-up machine, A's Blicketness is still uncertain if each of those scenes can potentially be explained by another object (deduction of A's Blicketness backward blockedd.
8

---

Model
(C)
D.R.
L.D.
S.O.
B.B.
AS)
D.R.
L.D.
S.O.
B.B
CNN-BERT
43.79
54.07
46.88
40.57
28.79
39.93
55.97
68.25
0.00
45.59
NS-OPT
69.04
92.5
76.05
88.33
13.48
67.44
94.73
88.38
82.76
16.06
Aloe 91.76
97.14
90.8
96.8
78.81
93.90
97.18
71.24
98.97
94,48
Table 3: Performance on ACRE of Aloe compared to the best results from Zhang et al. [44] split across inference type (D.ReDirect, I.D=Indirect, S.O-Screen-Offf, B. B.Br-askwards Blocking) and generalization type (C=Compositional, S=Systematic).
Inference problems in ACRE are categorized by reasoning type: reasoning from direct evidence (one of the context frames show the query objects machine), reasoning from indirect evidence (Blicketness must be deduced by combining evidence from several frames), screened-of off reasoning (presence of non-Blickets do not matter if single Blicket is present), and sackward-blocker reasoning (Blicketness cannot be deduced due to confounding variables). Please see Zhang et al. 44 for a more detailed discussion of these reasoning types.
Table Aloe performance compared to a CNN-BERT baseline and NS-OPT, a neuro-symbolic model introduced in Zhang et al. A4 Aloe outperforms all extant models for almost all reasoning types and train-test splits. We did not need to do any tuning to apply our model to ACREnsettings from CATER yielded the reported results the first attempt. Contrary to widely-held opinions that neural networks cannot generalize, Aloe generalizes contain different visual features (compositional split) or different numbers of activated machines in the context (systematic split). Moreover, Aloe achieved by far the best performance on the backward- blocking task, which requires the model to "go beyond the simple covariation strategy to discover the hidden causal relations" [44], dispelling the notion that neural networks can only find correlation. Comparison with NS-OPT (which uses object representations) and CNN-BERT (which uses attention) shows that neither object representations nor attention alone is sufficient for the task; combining these two ideas, done in Aloe for instance, essential for this complex reasoning task as well.
scenarios where the training and test sets
4 Related work
Self-attention for reasoning Various studies have shown that transformers (37 can manipulate symbolic data in manner traditionally associated with symbolic computation. For example, in Lample and Charton (23, a transformer model learned to do symbolic integration and solve ordinary differential equations symbolically, tasks traditionally reserved for symbolic computer algebra systems. Similarly, in Hahn et al. [17), transformer model learned to solve formulas in propositional logic and demonstrated some degree of generalization to out of distribution formulas. Finally, Brown et al. [ showed that a transformer trained for language modeling can also do simple analogical reasoning tasks without explicit training. Although these models do not necessarily beat carefully tuned symbolic algorithms in all cases (especially on out of distribution data), they are an important motivation for our proposed recipe for attaining strong reasoning capabilities from self-attention-ber based models on visually grounded tasks.
Object representations A wide body of research points to the importance of object segmentation
and representation learning (see e.g. Garnelo and Shanahan [I] for a discussion). Various methods
have been proposed for object detection and feature extraction 2 1O, 15 19 25
26
29
Past
research have also investigated using object based representations in downstream tasks
g
28
Self-supervised learning Another line of research concerns learning good representations through self-supervised learning, with an unsupervised auxiliary loss encourage the discovery of better representations. These better representations could lead to improved performance on supervised tasks, especially when labeled data is scaree. In Devlin et al. [9, for instance, an auxiliary infill loss allows the BERT model to benefit from pretraining on a large corpus of unlabeled data. Our approach to object-centric self-supervised learning is heavily inspired by the BERT infilling loss. Other studies have shown similar benefits to auxiliary learning in vision as well 4 16 18] These works apply various forms of contrastive losses to predict scene dynamics, and the better representations that result carry downstream benefits to supervised and reinforcement learning tasks.

---

Vision and language in self-attention models Recently, many works have emerged on applying
transformer models visual and multimodal data, for static images [24 27 33 36] and videos
These approaches combine the output of convolutional networks with language in various ways using self-attention. While these previous works focused on popular visual question answering tasks, which typically consist of descriptive questions only [41, we focus on understanding deeper causal dynamics of videos. Together with these works, we provide more evidence that self-attention between visual and language elements enables good performance on a diverse set of tasks.
34
35 43
In addition, while the use of object representations for discretization in tasks involving static images is becoming more popular, the right way to discretize videos is less clear. We provide strong evidence in the form of ablation studies for architectural decisions that we claim are essential for higher reasoning for this type of data: visual elements should correspond to physical objects in the videos and inter-frame attention between sub-frame entities (as opposed inter-frame attention of entire frames) is crucial. We also demonstrate the success of using unsupervised object segmentation methods as opposed the supervised methods used past work.
Conclusion
We have presented Aloe, model that obtains sate-of-theear performance on three different task domains involving spatiotemporal reasoning about objects. In each of these tasks, previous state-of- the-art results were established by models with modular, task-specific components. Aloe, by contrast, a unified solution to all three domains. Its flexibility comes from a reliance on only soft biases and learning objectives: self-attention over learned object embeddings and self-supervised learning of dynamics. We believe the simplicity of this approach is its strength, and hope that this fact, together with the provided code, makes it easy for others adopt and apply to arbitrary spatio-temporal reasoning problems.
On many of these spatio-tempord reasoning problems, previous state-of-the thr achieved by neuro-sy symoolim models 66 [ 40 41 44.. Compared to neuro-symbolic models, Aloe can more easily be adapted other tasks. Indeed, the symbolic components of neuro-symbolic models often task-specific and not straightforwardly applicable to other tasks. Neuro-symbolic models do have few advantages, however. First, they often easier to interpret. Despite the insights that can be gleaned from Aloe's attention weights, these soft computations are harder to interpret than the explicit symbolic computation found in neuro-symbolic models. Moreover, neuro-symbolic models can be structured in a more modular fashion, which can enable effective generalization to sub-tasks of the task which the model trained (6.
Aloe also has some important limitations. limitation is mainly due to the lack of real-world datasets that test for higher-order spatiotemporal reasoning, although we are excited that new datasets such as Traffic QA will be released soon (38). Second, while the domains where Aloe is applied have been widely adopted and well-received by the research community, it remains possible that they do not evaluate the capacities that they aim to evaluate because of hidden biases other factors. Regardless, we hope that this work stimulates the design and development of more challenging tasks that more closely approximate the ultimate goal of human super-human-level visual, spatiotemporal and causal reasoning. Finally, from ethical point of view, our model may share the common drawback of deep-learning models in perpetuating biases found in the training data, especially when applied to real world data. Development of causal reasoning models could also invite problematic applications involving automated assignment of blame.
First, it has only been applied to synthetic datasets. This
10

---

References
[1] Tom B Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005. 1 /4/65, 2020.
[2] Chris P. Burgess, Loic Matthey, Nick Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, and Alex Lerchner. MONet: Unsupervised scene decomposition and representation. arXiv preprint arXiv: 2019.
[3] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In ECCV, 2020.
[4] Ting Chen, Simon Korblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. arXiv preprint arXiv:2002.05709, 2020.
[5] Zhe Chen. Object-based attention: A tutorial review. Attention, Perception, & Psychophysics, 74:784-802, 2012. URL https: //dois::/ddi..o org. 3755//1...77888 3758 S131///s1144-4111
0322-22
[6] Zhenfang Chen, Jiayuan Mao, Jiajun Wu, Kwan-Yee Kenneth Wong, Joshua B. Tenenbaum, and Chuang Gan. Grounding physical object and event concepts through dynamic visual reasoning. In International Conference Learning Representations, 2021. URL https 7/arxiv. wttp:////p/....1aaee..
pdd
[7] Zihang Dai, Zhilin Yang, Yiming Yang, William W Cohen, Jaime Carbonell, Quoc V Le, and Ruslan Salakhutdinov. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv preprint arXiv:1901.02866 2019.
[8] Mikyas T. Desta, Larry Chen, and Tomasz Kornuta. Object-based reasoning in VQA. CoRR, abs/1801.09978, 18, 2018. BRthptt:///axii wttpaxiww..rr/b////.1..... 09718
[9] Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint 1rXiv:8810004805, 2018.
[10] Yilun Du, Kevin Smith, Tomer Ulman, Joshua Tenenbaum, and Jiajun Wu. Unsupervised discovery of 3d physical objects from video. arXiv preprint arXiv:2007.122 12344.22388...... 2020.
[11] Marta Garnelo and Murray Shanahan. Reconciling deep learning with symbolic artificial intelligence: representing objects and relations. Current Opinion in Behavioral Sciences, 29: 17-23, 2019.
[12] Rohit Girdhar and Deva Ramanan. CATER: A diagnostic dataset for Compositional Actions and T'Epporal Reasoning. In ICLR, 2020.
[13] Alison Gopnik and David Sobel. Detecting blickets: how young children use information about novel causal powers in categorization and induction. Child Dev, pages 1205-22 22, 2000. doi: 11111/167--824400222
[14) Anirudh Goyal, Aniket Didolkar, Alex Lamb, Kartikeya Badola, Nan Rosemary Ke, Nasim Rahaman, Jonathan Binas, Charles Blundell, Michael Mozer, and Yoshua Bengio. Coordination among neural modules through a shared global workspace. CoRR, abs/2103.01197, 2021. URL
https: /tap::///rxiii wrg. attrg//bs/2113.... 11111.111999. 01197
[15] Klaus Greff, Raphaël Lopez Kaufman, Rishabh Kabra, Nick Watters, Christopher Burgess, Daniel Zoran, Loie Matthey, Matthew Botvinick, and Alexander Lerchner. Multi-object rep- resentation learning with iterative variational inference. CoRR, abs/1 000/000030050 2019. URL.
http::/arxivv www..voorrrgaaae... abs 1903. 00450
[16] Karol Gregor, Danilo Jimenez Rezende, Frederic Besse, Yan Wu, Hamza Merzic, and Aaron van den Oord. Shaping belief states with generative environment models for rl. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alché-Bue, E. Fox, and R. Garnett, editors, Advances Neural Information Processing Systems 32, pages 13475-13487. Curran Associates, Inc., 2019. http:http:///peeeeeee nips cappe/ witeees--ttth--eeetttteee ieteerrtttvveeeeee ve-envi wew.eeviromeeeteeeeeeeeeeeeeeeee rorment model s- for
pdf

---

[17] Christopher Hahn, Frederik Schmitt, Jens U. Kreber, Markus N. Rabe, and Bernd Finkbeiner. Transformers generalize the semantics of logics. arXiv preprint arXiv:2003.04218, 2020.
[18] Tengda Han, Weidi Xie, and Andrew Zisserman. Video representation learning by dense predictive coding. In Workshop on Large Scale Holistic Video Understanding. ICCV, 2019.
[19] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross B. Girshick. Mask R-CNN. CoRR, abs/170,.06870, 2017. URL hattp://arxiv.oot abs/1703. 06870
[20] Drew A Hudson and Christopher D Manning. Compositional attention networks for machine reasoning. 2018.
[21] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. CoRR, abs/1612.06899 2016. URL [tltt:///aaxxv http://awxxvv..aaaaaaaa wwt./r/////..... httgg//b/11100 1612
06890
[22] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
[23] Guillaume Lample and François Charton. Deep learning for symbolic mathematics. Interna- tional Conference Learning Representations, 2020. URL RRLLLttt////ppeeeeee.e
Forum?id-SSe2Yelll
[24] Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsich, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprini meo:::908.355 2019.
[25] Zhixuan Lin, Yi-Fu Wu, Skand Vishwanath Peri, Weihao Sun, Gautam Singh, Fei Deng. Jindong Jiang, and Sungjin Ahn. SPACE: Unsupervised object-oriented scene representation via spatial attention and decomposition. In International Conference on Learning Representations, 2020.
URL httt::tt:///ptpt / net/ www.rooruuooooooooooo forumm?dd=rk1033yY
[26] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf1. Object-Centrie Learning with Slot Attention. In Advances in Neural Information Processing Systems (NIPS), 2020.
[27] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agmostie visiolinguistic representations for vision-and-lange ann-uang tasks. arXiv preprint arXiv:1908. 02265, 2019.
[28] David Raposo, Adam Santoro, David G. T. Barrett, Razvan Pascanu, Timothy P. Lillicrap, and Peter W. Battaglia. Discovering objects and their relations from entangled scene representations. CoRR, abs/1702.05068,, 2017. URL /ttl::t:///aa...orro.
wrwia. org. abs 702.0022 05068
[29] Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real- time object detection with region proposal networks. CoRR, Abs/1506.014997 2015, URL
org/ abs/ 1506. 01497
[30] Pieter R. Roelfsema, Victor A. Lamme, and Henk Spekreijse. Object-based attention in the primary visual cortex of the macaque monkey. Nature, 395:376-381, 1998. doi: 10.1038/26475.
[31] Aviv Shamsian, Ofri Kleinfeld, Amir Globerson, and Gal Chechik. Learning object permanence from video. arXiv preprint arXiv:2003. 10469, 2020.
[32] Elizabeth Spelke. Core knowledge. The American psychologist, 55:1233-43, 12 2000. doi: 100107/0000300665555..0 1...133
[33] Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. V1-bert: Pre- training of generic risual-linguisii representations. In International Conference on Learning Representations, 2020. URL httpts////peeeeeiieee wwwp:/rrntttoteeeeeee Syg.euuaayddSSyXPPayYY..
[34] Chen Sun, Fabien Baradel, Kevin Murphy, and Cordelia Schmid. Contrastive bidirectional transformer for temporal representation learning. CoRR, abs/1906.05743, 2019. http
7/arxiv org/ abs 1906.04 05743
12

---

[35] Chen Sun, Austin Myers, Carl Vondrick, Kevin Murphy, and Cordelia Schmid. Videobert: A joint model for video and language representation learning, In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
[36] Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from transformers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, 2019.
[37] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998-6000 6008, 2017.
[38] Li Xu, He Huang, and Jun Liu. Sutd-trafficqa: question answering benchmark and an efficient network for video reasoning over traffic events. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021.
[39] Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Joshua B Tenenbaum. Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding. In Advances in Neural Information Processing Systems (NIPS), 2018.
[40] Kexin Yi, Jiajun Wu, Chuang Gan, Antonio Torralba, Pushmeet Kohli, and Joshua B. Tenenbaum. Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. In Advances in Neural Information Processing Systems, pages 1039-1050, 2018.
[41] Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and
Joshua B. Tenenbaum. CLEVRER: Collision events for video representation and rea- soning. In International Conference on Learning Representations, 2020. URL https //openreview net fow.mmid----x
[42] Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, and Cho-Jui Hsieh. Large batch optimization for deep learning: Training BERT 76 minutes. Technical Report (CB/EECS-20019-0 EECS Department, University of California, Berkeley, Jun 2019. URL http: htt::://wwwwwwww eecs www.eeeeeeedeeuu.s...ee SechRpts/2019/EBSe 2019 9 103.html
[43] Vinicius Zambaldi, David Raposo, Adam Santoro, Victor Bapst, Yujia Li, Igor Babuschkin, Karl Tuyls, David Reichert, Timothy Lillicrap, Edward Lockhart, Murray Shanahan, Victoria Langston, Razvan Pascanu, Matthew Botvinick, Oriol Vinyals, and Peter Battaglia. Deep reinforcement learning with relational inductive biases. In International Conference Learning Representations, 2019. URL induucivve wwwp:nrwwwwwwweweeeeeeeeeeeeeeee net wwwe/eoouuueee oo
[44] Chi Zhang, Baoxiong Jia, Mark Edmonds, Song-Chun Zhu, and Yixin Zhu. Acre: Abstract causal reasoning beyond covariation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[45] Honglu Zhou, Asim Kadav, Farley Lai, Alexandru Niculescu-Mizit, Martin Renqiang Min, Mubbasir Kapadia, and Hans Peter Graf. Hopper: Multi-hop transformer for spatiotemporal reasoning. In International Conference Learning Representations, 2021. URL https 7/arxiv. www.oorgaaae. 3org/abs/2103.1055
13


Reviews:
[{"review_id": "Uthy-nsiNy4", "reviewer": "Reviewer_cuzz", "summary": "This paper builds an end-to-end system for visual reasoning. The videos are first parsed to object-centric representations using an unsupervised object detection method (MONet). After that, they use a transformer on top of the object-centric representations to model the spatio-temporal relationship. The features are then combined with the question embedding and output the answer. The learning process is driven by two losses: 1) self-supervised prediction (the authors examine different forms of such supervision); 2) a supervised-learning loss such as the answer to questions. The proposed method achieves impressive improvement over previous state-of-the-art.", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors propose a simple framework that can achieve impressive performance on many visual reasoning benchmarks. The proposed method holds the potential to be a simple baseline for future research in this direction because of its simplicity, elegance, and good performance. The model use transformer to simultaneously process object information and language information, which leads to significant performance improvement and makes a lot of sense.\n\nIndeed, as the authors mentioned in the submission history, there exists criticism about the visual reasoning benchmark itself (such as CLEVERER). However, the contribution and originality of this work should not be omitted. One thing I want to mention is that the current leading neural-symbolic methods in this dataset (such as [5] and [37]) usually imposes human priors specifically for this simple environment (such as it only contains blocks and balls). However, an important point is that the proposed method does not have any human priors thus it holds the potential to be applied to real-world. Maybe the only component that cannot easily generalize to real-world scenario is the MONet which people find it hard to apply to real-world videos. However, with the advanced supervised object detection techniques, it’s still highly possible towards that goal, as also shown in [A].\n\nIn conclusion, I think it’s a good paper in this field and can potentially help the neural-symbolic methods to get new improvements.\n\n[A] Learning Long-term Visual Dynamics with Region Proposal Interaction Networks. ICLR 2021\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I didn't find any potential negative societal impact of this work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5.0 hours"}, "extra_text": {}}, {"review_id": "R4Wa505Z8do", "reviewer": "Reviewer_1FAX", "summary": "The authors have proposed a model for spatiotemporal reasoning problems using self-attention and self-supervised learning in this work. For object representation, an unsupervised method, MONet (Burgess et al., 2019) is used where recurrent attention mechanisms are used to obtain N attention masks. This self-supervision along with unsupervised auxiliary loss helps in learning better representations. Experiments were performed with datasets like CLEVERER, CATER and ACRE after pretraining the model on individual frames. Through various experiments, authors have demonstrated the importance of self-attention, object-based discretization and auxiliary loss in their model. Overall, the model can perform higher-level reasoning with its simplified architecture. \n", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is clearly written and easy to follow. Wide ranges of experiments were performed to cover various aspects of reasoning using different datasets and have clearly shown an edge over previous SOTA models in a data-efficient way. Their ablation studies show the importance of different modules in their model. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I feel that the model has certain limitations like:\n1. Prior knowledge about the number of objects (N0) is required.\n2. Pre-training is required for MONet on each dataset.\n\nI am unable to understand how the authors have set the hyperparameters like transformer heads/layers, embedding size etc for their models. I am also not sure about the statement made in line 186; Was there any study showing that humans and models would have given incorrect responses for questions? Can you explain global vs hierarchical attention which is not very much clear from the paper? Finally, do you have a comparison of the total number of trainable parameters in ALOE vs other baseline models which might also affect the performance.\n\nMinor:\nhttps://arxiv.org/pdf/2006.15055.pdf [slot attention] reference is missing which I believe has a similar approach of finding the object-centric representation in an unsupervised manner. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "EeIaI11WuIT", "reviewer": "Reviewer_WrDi", "summary": "A transformer achieves state of the art in three video+text reasoning tasks. A pre-trained unsupervised method obtains object representations per frame; object representations across frames, word representations (question+options) and a special token is fed to a transformer and a prediction is made using a linear layer on top of the transformed special token. Question answers provide supervision and masking objects provide self-supervision. Ablation experiments are performed: they show the usefulness of object-centric representations and, albeit less so, self-supervision. Model attention and errors seem sensible. It shows that a fully end-to-end network can solve causal reasoning tasks; which until now required symbolic modules such as program generation and execution engines or physical engines.", "questions": "", "limitations": "", "rating": 9, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality\nModels (unsupervised object representation, self-supervision, transformers) and datasets are current; the novelty of the paper is in combining them to show that “simple” architectures can perform visual reasoning/causal tasks without  explicit need for a symbolic reasoning engine. \n\nQuality\nGood paper with well-thought, thorough experiments and positive results. \n\nAblation experiments, error bounds, and qualitative evaluations provide assurance on the validity of results and claims made. Provided hyperparameters, training details and code will help replicability.\n\nBy design, you allow objects to attend to words and (in multi-option questions) words from one option to attend to words from other options which seems counterintuitive, did you ever try to use an encoder-decoder architecture (word-to-word, object-to-object and word-to-object attention), sending the questions as query in the decoder (or perhaps question+each_option as a query)? This is also a standard transformer setup and the extra structure may benefit the model.\n\nManaging the information flow during masking becomes troublesome when you have rolling (frame-dependent) masks (c-f in Fig. 2). For instance in scheme c, even though in the first transformer layer representation slot i cannot access the masked slots at i, i-1 and i -2, in the second layer it will be able to access all representations, most of which have access to i, i-1 and i-2 thus bypassing the masking (I don’t see how one can mask the second layer to avoid this leaking of information.). This should not be a problem for a, b; maybe avoiding the allocation of informational capacity to this shortcutting is why a, b masking performs better.\n\nIn Tab. 1, the comparison of object-centric representations with representations from a ResNet trained from scratch is unfair; a ResNet-like architecture pre-trained with self-supervision (e.g., a contrastive loss) in the CLEVRER data would be better, or even a standard VAE encoder (perhaps with some disentanglement regularization to encourage object separation). In any case, using an object-centric representation seems more principled.\n\nAlthough the paper advances the point of connectionist models being enough to solve reasoning tasks, in fairness, you could still point to some of the advantages that symbolic and neuro-symbolic models have over purely learned models: modularity, interpretability, human-computer interaction, etc. or contrast them more in the discussion.\n \nClarity\nWell-written, readable and consistent figures.\n\nFirst paragraphs in 3.2 and 3.3 describing the datasets are hard to understand without already being familiar with the datasets (which the paper relies on). I don’t think the authors can do much more as the responsibility falls on the readers but it does still take away from the manuscript. For instance, it is hard to contextualize what the results of Table 3 mean without a very good understanding of ACRE; observation holds for the other datasets too, though they are better known. Whether it is worth to present an example (or just pointing to the appendix examples) or to guide readers through the significance of results is up to the authors. I, of course, understand finding a good balance is hard.\n\nGood explanation of the model architecture (Sec. 2). References to the appendix for extra details are welcomed.\n\nFor object collision and tracking datasets, object position (inside the frame) is very important. Perhaps, it is worth explicitly mentioning how this xy position is provided to the transformer; as I understand, in the distributed Monet representation (and not, for instance, provided in two separate dimensions or using the mask somehow). Regarding Monet, it is also unclear how you select the number of object slots and whether you discard the extra “empty” slots (and how).\n\nFig. 1 conveys a lot of information about the model architecture. I can point a number of small details (none too important): 1) arrows and SSL schematic are hard to see in print, perhaps you can decrease the transparency (or pick a different color), 2) you say you use a MLP on top of the CLS class for predictions (l. 120) but the figure says linear+softmax, 3) If model details are not in the caption, perhaps you could point directly to “see Methods/section 2 for details” rather than “see the main text” and 4) I wonder how important is it to have the “+position encoding” in the main transformer block, having it so front and center makes it look more important than it is and other details (such as the extra dimensions pointing whether the representation is a word or object) are not in the diagram either.\n\nIt is odd that half a page and a figure is devoted to explaining the different masking strategies and the results are not presented in the main text. Effect of the supervised loss is minor (as shown by ablation experiments) and it does not seem like the effect of different masking is that significant anyway (Fig. 4 in Appendix); maybe you just present the one-object per-frame masking in the main, bump the rest to appendix and more results from CLEVR (like the counterfactual analyses) can be upgraded to main. I do not feel strongly about this, though, it just feels discordant to spend a lot of space describing the method in main to then not show the results.\n\nHow do you do the object-to-slot matching when masking more than one object per frame? is it in the appendix?\n\nMissing info: What are the error bounds in Tab. 1, Tab.2 and Fig. 3? stddev, stderr, ci? and how many samples are used?\n\nFigure 3 could have bigger font sizes and thicker lines; it is relatively hard to read. The light/dark yellow bars are also hard to see (and one dissapears behind the other sometimes); perhaps using two side-by-side bars would help (or have them as horizontal lines in the plot). Also, consider adding a y axis at the very right of the plot (or turn grids on) as it is hard to see, for instance, how high the accuracy on counterfactuals goes in the last column.\n\nIt is unclear whether the Aloe results in Tab 1 (middle row) use infill or one object per frame masking? You say “infill” improves 4-6% (l. 217) but then say one object per frame is the most effective (l.222).\n\nWhich transformer layer is used for the attention visualizations in l.189?\n\nThe non-captioned figures in l.193 and l.200 are visually off-putting; I guess you were constrained on space; putting them together in the same figure could be as economical and underline the fact that the two images are a set (i.e., the question in image 1 is answered given the video in image 2).\n\nFor the CATER dataset, it is not fully clear what is the supervision signal for the transformer (from the original paper I surmised it is to predict the executed actions, is this correct?).  From the text, it seems like the supervision would be the distance between predicted and target snitch position (no mention of questions), which makes the addition of an L1 loss confusing (as it will be redundant).\n\nTypo (l.285): representations\n\nNice literature review. I am curious to know why is it near the bottom and not between intro and methods where it would usually be. The self-supervised learning section is somehow light (for instance, word2vec used infilling long before BERT and other SSL methods have been used before contrastive learning), but it may be enough.\n \n(Appendix) May be worth stating that code is written in Tensorflow and whether implementation is TPU specific? It will be nice to state which hyperparameters were searched, some more details about word embeddings (pretrained, vocabulary treatment, etc.) and how the positional encoding was done (I presume all slots in one frame have the same encoding, and then words start encoding at the start of the sinusoidal series).\n\nSignificance\nPaper with good results that allow it to make bold claims: that end-to-end networks can model reasoning and causal tasks (at least the ones tested) without need for symbolic modules. It will be a good reference/baseline for future work in this subfield that is growing in popularity. Its applicability to realistic images as noted in the manuscript is left unadressed. Addressing some minor details will make the paper stronger but in its current state, acceptance as poster is still merited.\n\n\nUpdate:\nBumped the rating from 7 to 9 (top 15% of papers) after the authors improved the manuscript using reviewers' input.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Good job pointing the limitation of using only synthetic datasets. I would go even further saying it has only being applied to the same syntethic environment (object identities, colors, shapes, size, material, background, illumination, etc.): all CLEVR-like. Can you venture a prediction, perhaps from experience, about whether it will transfer to other synthetic or real environments?\n\nThe one-sentence ethical statement seems haphazard; picking data biases is surely a pitfall of any learning method but methods like the one proposed here that eases observation/tracking of activity in video have more present concerns: facilitating population tracking, for instance, or (through better causal modelling) artificial manipulation of public opinion. Training big models, with the usual disregard for the amount or source of energy used, could also be brought up.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "9"}, "extra_text": {}}, {"review_id": "8OaUI9FkUD", "reviewer": "Reviewer_Un95", "summary": "The paper proposes to combine self-attention and object-centric representations to be able to solve complex reasoning tasks. The author(s) also propose a self-supervised loss based on masking of the objects' representations to implicitly learn the dynamics, further enhancing the performance on downstream tasks. The results of the proposed method are very impressive, especially on the challenging counterfactuals reasoning task.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n\n1) The proposed model seems to be fairly simple with minimal inductive bias as compared to neuro-symbolic (NS) methods that assume the parser to be available. It is quite impressive that such a model can outperform NS based methods by a significant margin (especially on the harder causal reasoning tasks).\n2) I'd also like to appreciate the author(s) efforts to share the model's code during the review process.\n\nQuestions:\n\n3) Despite knowing the underlying dynamics explicitly, why do neuro-symbolic methods (like DCL/NS-DR) fail with such a large margin as compared to Aloe (~30% points)? Is this because of self-attention that the model can capture the long-term dependencies to reason, which MLP and even LSTM (from the footnote on Pg. 6) cannot?\n\n4) Also, why is DCL/NS-DR not as sample efficient as Aloe given that DCL has enough (rather high) inductive bias of having a parser as well as ground-truth annotations for objects to learn from? For this case, we can exclude the counterfactual reasoning and only focus on the other three experiments. In Fig. 3, having DCL plots for different proportions for the dataset would be beneficial. (I don't think it is required for both CATER and CLEVERER datasets; just one should suffice)\n\n\n5) One key aspect that NS methods claim is that of generalization to newer scenarios. For example authors in the DCL paper propose CLEVERER-grounding and CLEVERER-Retrieval tasks. I'm wondering how Aloe would generalize to such new tasks (need not be specifically these tasks).\n\n\nMinor details:\n\n6) Please include some details in the caption of Figure 1. Typically every figure should be stand-alone and should not require the reader to refer to the text.\n\n7) Fig. 4 in the appendix shows results with 50% of CLEVERER data. Does the trend remain similar or does the gap reduce when trained with 100% of data (no need to show the results but I'm sure that an experiment with the complete data would've been run first)?\n\nJustification for rating: Overall, the paper is well written and the proposed method though simple, is quite effective. My major concerns are with sample-efficiency and generalization compared to Neuro-Symbolic methods. However, I lean towards a weak accept of the paper as it does have some important claims regarding using a more general approach with less inductive bias for reasoning tasks. \nMy final rating however, would be based on the author(s) rebuttal as well as other reviewers' comments.\n\n------\nEdit (post-rebuttal) : All my concerns have been addressed satisfactorily, hence I lean towards the acceptance of the paper and am increasing my rating to 7 (accept). I thank the author(s) for their rebuttal.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "NA", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "9 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper received 4 strong accepts. The reviewers have lauded the work because of extensive experiments and clear cutting results which are pushing the SOTA. The approach is likely to be of broad interest because this "connectionist" approach is pitted again the "symbolic" one in an area where so-called neuro-symbolic models have been dominating. The AC thus recommends an oral.
Decision: Accept (Oral)