Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: The authors have proposed a model for spatiotemporal reasoning problems using self-attention and self-supervised learning in this work. For object representation, an unsupervised method, MONet (Burgess et al., 2019) is used where recurrent attention mechanisms are used to obtain N attention masks. This self-supervision along with unsupervised auxiliary loss helps in learning better representations. Experiments were performed with datasets like CLEVERER, CATER and ACRE after pretraining the model on individual frames. Through various experiments, authors have demonstrated the importance of self-attention, object-based discretization and auxiliary loss in their model. Overall, the model can perform higher-level reasoning with its simplified architecture. 

main_review: The paper is clearly written and easy to follow. Wide ranges of experiments were performed to cover various aspects of reasoning using different datasets and have clearly shown an edge over previous SOTA models in a data-efficient way. Their ablation studies show the importance of different modules in their model. 


limitations_and_societal_impact: I feel that the model has certain limitations like:
1. Prior knowledge about the number of objects (N0) is required.
2. Pre-training is required for MONet on each dataset.

I am unable to understand how the authors have set the hyperparameters like transformer heads/layers, embedding size etc for their models. I am also not sure about the statement made in line 186; Was there any study showing that humans and models would have given incorrect responses for questions? Can you explain global vs hierarchical attention which is not very much clear from the paper? Finally, do you have a comparison of the total number of trainable parameters in ALOE vs other baseline models which might also affect the performance.

Minor:
https://arxiv.org/pdf/2006.15055.pdf [slot attention] reference is missing which I believe has a similar approach of finding the object-centric representation in an unsupervised manner.
review_title: null

final_score: rating: 9: Top 15% of accepted NeurIPS papers, strong accept
review_content: summary: A transformer achieves state of the art in three video+text reasoning tasks. A pre-trained unsupervised method obtains object representations per frame; object representations across frames, word representations (question+options) and a special token is fed to a transformer and a prediction is made using a linear layer on top of the transformed special token. Question answers provide supervision and masking objects provide self-supervision. Ablation experiments are performed: they show the usefulness of object-centric representations and, albeit less so, self-supervision. Model attention and errors seem sensible. It shows that a fully end-to-end network can solve causal reasoning tasks; which until now required symbolic modules such as program generation and execution engines or physical engines.
main_review: Originality
Models (unsupervised object representation, self-supervision, transformers) and datasets are current; the novelty of the paper is in combining them to show that “simple” architectures can perform visual reasoning/causal tasks without  explicit need for a symbolic reasoning engine. 

Quality
Good paper with well-thought, thorough experiments and positive results. 

Ablation experiments, error bounds, and qualitative evaluations provide assurance on the validity of results and claims made. Provided hyperparameters, training details and code will help replicability.

By design, you allow objects to attend to words and (in multi-option questions) words from one option to attend to words from other options which seems counterintuitive, did you ever try to use an encoder-decoder architecture (word-to-word, object-to-object and word-to-object attention), sending the questions as query in the decoder (or perhaps question+each_option as a query)? This is also a standard transformer setup and the extra structure may benefit the model.

Managing the information flow during masking becomes troublesome when you have rolling (frame-dependent) masks (c-f in Fig. 2). For instance in scheme c, even though in the first transformer layer representation slot i cannot access the masked slots at i, i-1 and i -2, in the second layer it will be able to access all representations, most of which have access to i, i-1 and i-2 thus bypassing the masking (I don’t see how one can mask the second layer to avoid this leaking of information.). This should not be a problem for a, b; maybe avoiding the allocation of informational capacity to this shortcutting is why a, b masking performs better.

In Tab. 1, the comparison of object-centric representations with representations from a ResNet trained from scratch is unfair; a ResNet-like architecture pre-trained with self-supervision (e.g., a contrastive loss) in the CLEVRER data would be better, or even a standard VAE encoder (perhaps with some disentanglement regularization to encourage object separation). In any case, using an object-centric representation seems more principled.

Although the paper advances the point of connectionist models being enough to solve reasoning tasks, in fairness, you could still point to some of the advantages that symbolic and neuro-symbolic models have over purely learned models: modularity, interpretability, human-computer interaction, etc. or contrast them more in the discussion.
 
Clarity
Well-written, readable and consistent figures.

First paragraphs in 3.2 and 3.3 describing the datasets are hard to understand without already being familiar with the datasets (which the paper relies on). I don’t think the authors can do much more as the responsibility falls on the readers but it does still take away from the manuscript. For instance, it is hard to contextualize what the results of Table 3 mean without a very good understanding of ACRE; observation holds for the other datasets too, though they are better known. Whether it is worth to present an example (or just pointing to the appendix examples) or to guide readers through the significance of results is up to the authors. I, of course, understand finding a good balance is hard.

Good explanation of the model architecture (Sec. 2). References to the appendix for extra details are welcomed.

For object collision and tracking datasets, object position (inside the frame) is very important. Perhaps, it is worth explicitly mentioning how this xy position is provided to the transformer; as I understand, in the distributed Monet representation (and not, for instance, provided in two separate dimensions or using the mask somehow). Regarding Monet, it is also unclear how you select the number of object slots and whether you discard the extra “empty” slots (and how).

Fig. 1 conveys a lot of information about the model architecture. I can point a number of small details (none too important): 1) arrows and SSL schematic are hard to see in print, perhaps you can decrease the transparency (or pick a different color), 2) you say you use a MLP on top of the CLS class for predictions (l. 120) but the figure says linear+softmax, 3) If model details are not in the caption, perhaps you could point directly to “see Methods/section 2 for details” rather than “see the main text” and 4) I wonder how important is it to have the “+position encoding” in the main transformer block, having it so front and center makes it look more important than it is and other details (such as the extra dimensions pointing whether the representation is a word or object) are not in the diagram either.

It is odd that half a page and a figure is devoted to explaining the different masking strategies and the results are not presented in the main text. Effect of the supervised loss is minor (as shown by ablation experiments) and it does not seem like the effect of different masking is that significant anyway (Fig. 4 in Appendix); maybe you just present the one-object per-frame masking in the main, bump the rest to appendix and more results from CLEVR (like the counterfactual analyses) can be upgraded to main. I do not feel strongly about this, though, it just feels discordant to spend a lot of space describing the method in main to then not show the results.

How do you do the object-to-slot matching when masking more than one object per frame? is it in the appendix?

Missing info: What are the error bounds in Tab. 1, Tab.2 and Fig. 3? stddev, stderr, ci? and how many samples are used?

Figure 3 could have bigger font sizes and thicker lines; it is relatively hard to read. The light/dark yellow bars are also hard to see (and one dissapears behind the other sometimes); perhaps using two side-by-side bars would help (or have them as horizontal lines in the plot). Also, consider adding a y axis at the very right of the plot (or turn grids on) as it is hard to see, for instance, how high the accuracy on counterfactuals goes in the last column.

It is unclear whether the Aloe results in Tab 1 (middle row) use infill or one object per frame masking? You say “infill” improves 4-6% (l. 217) but then say one object per frame is the most effective (l.222).

Which transformer layer is used for the attention visualizations in l.189?

The non-captioned figures in l.193 and l.200 are visually off-putting; I guess you were constrained on space; putting them together in the same figure could be as economical and underline the fact that the two images are a set (i.e., the question in image 1 is answered given the video in image 2).

For the CATER dataset, it is not fully clear what is the supervision signal for the transformer (from the original paper I surmised it is to predict the executed actions, is this correct?).  From the text, it seems like the supervision would be the distance between predicted and target snitch position (no mention of questions), which makes the addition of an L1 loss confusing (as it will be redundant).

Typo (l.285): representations

Nice literature review. I am curious to know why is it near the bottom and not between intro and methods where it would usually be. The self-supervised learning section is somehow light (for instance, word2vec used infilling long before BERT and other SSL methods have been used before contrastive learning), but it may be enough.
 
(Appendix) May be worth stating that code is written in Tensorflow and whether implementation is TPU specific? It will be nice to state which hyperparameters were searched, some more details about word embeddings (pretrained, vocabulary treatment, etc.) and how the positional encoding was done (I presume all slots in one frame have the same encoding, and then words start encoding at the start of the sinusoidal series).

Significance
Paper with good results that allow it to make bold claims: that end-to-end networks can model reasoning and causal tasks (at least the ones tested) without need for symbolic modules. It will be a good reference/baseline for future work in this subfield that is growing in popularity. Its applicability to realistic images as noted in the manuscript is left unadressed. Addressing some minor details will make the paper stronger but in its current state, acceptance as poster is still merited.


Update:
Bumped the rating from 7 to 9 (top 15% of papers) after the authors improved the manuscript using reviewers' input.
limitations_and_societal_impact: Good job pointing the limitation of using only synthetic datasets. I would go even further saying it has only being applied to the same syntethic environment (object identities, colors, shapes, size, material, background, illumination, etc.): all CLEVR-like. Can you venture a prediction, perhaps from experience, about whether it will transfer to other synthetic or real environments?

The one-sentence ethical statement seems haphazard; picking data biases is surely a pitfall of any learning method but methods like the one proposed here that eases observation/tracking of activity in video have more present concerns: facilitating population tracking, for instance, or (through better causal modelling) artificial manipulation of public opinion. Training big models, with the usual disregard for the amount or source of energy used, could also be brought up.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper builds an end-to-end system for visual reasoning. The videos are first parsed to object-centric representations using an unsupervised object detection method (MONet). After that, they use a transformer on top of the object-centric representations to model the spatio-temporal relationship. The features are then combined with the question embedding and output the answer. The learning process is driven by two losses: 1) self-supervised prediction (the authors examine different forms of such supervision); 2) a supervised-learning loss such as the answer to questions. The proposed method achieves impressive improvement over previous state-of-the-art.
main_review: The authors propose a simple framework that can achieve impressive performance on many visual reasoning benchmarks. The proposed method holds the potential to be a simple baseline for future research in this direction because of its simplicity, elegance, and good performance. The model use transformer to simultaneously process object information and language information, which leads to significant performance improvement and makes a lot of sense.

Indeed, as the authors mentioned in the submission history, there exists criticism about the visual reasoning benchmark itself (such as CLEVERER). However, the contribution and originality of this work should not be omitted. One thing I want to mention is that the current leading neural-symbolic methods in this dataset (such as [5] and [37]) usually imposes human priors specifically for this simple environment (such as it only contains blocks and balls). However, an important point is that the proposed method does not have any human priors thus it holds the potential to be applied to real-world. Maybe the only component that cannot easily generalize to real-world scenario is the MONet which people find it hard to apply to real-world videos. However, with the advanced supervised object detection techniques, it’s still highly possible towards that goal, as also shown in [A].

In conclusion, I think it’s a good paper in this field and can potentially help the neural-symbolic methods to get new improvements.

[A] Learning Long-term Visual Dynamics with Region Proposal Interaction Networks. ICLR 2021

limitations_and_societal_impact: I didn't find any potential negative societal impact of this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes to combine self-attention and object-centric representations to be able to solve complex reasoning tasks. The author(s) also propose a self-supervised loss based on masking of the objects' representations to implicitly learn the dynamics, further enhancing the performance on downstream tasks. The results of the proposed method are very impressive, especially on the challenging counterfactuals reasoning task.

main_review: Strengths:

1) The proposed model seems to be fairly simple with minimal inductive bias as compared to neuro-symbolic (NS) methods that assume the parser to be available. It is quite impressive that such a model can outperform NS based methods by a significant margin (especially on the harder causal reasoning tasks).
2) I'd also like to appreciate the author(s) efforts to share the model's code during the review process.

Questions:

3) Despite knowing the underlying dynamics explicitly, why do neuro-symbolic methods (like DCL/NS-DR) fail with such a large margin as compared to Aloe (~30% points)? Is this because of self-attention that the model can capture the long-term dependencies to reason, which MLP and even LSTM (from the footnote on Pg. 6) cannot?

4) Also, why is DCL/NS-DR not as sample efficient as Aloe given that DCL has enough (rather high) inductive bias of having a parser as well as ground-truth annotations for objects to learn from? For this case, we can exclude the counterfactual reasoning and only focus on the other three experiments. In Fig. 3, having DCL plots for different proportions for the dataset would be beneficial. (I don't think it is required for both CATER and CLEVERER datasets; just one should suffice)


5) One key aspect that NS methods claim is that of generalization to newer scenarios. For example authors in the DCL paper propose CLEVERER-grounding and CLEVERER-Retrieval tasks. I'm wondering how Aloe would generalize to such new tasks (need not be specifically these tasks).


Minor details:

6) Please include some details in the caption of Figure 1. Typically every figure should be stand-alone and should not require the reader to refer to the text.

7) Fig. 4 in the appendix shows results with 50% of CLEVERER data. Does the trend remain similar or does the gap reduce when trained with 100% of data (no need to show the results but I'm sure that an experiment with the complete data would've been run first)?

Justification for rating: Overall, the paper is well written and the proposed method though simple, is quite effective. My major concerns are with sample-efficiency and generalization compared to Neuro-Symbolic methods. However, I lean towards a weak accept of the paper as it does have some important claims regarding using a more general approach with less inductive bias for reasoning tasks. 
My final rating however, would be based on the author(s) rebuttal as well as other reviewers' comments.

------
Edit (post-rebuttal) : All my concerns have been addressed satisfactorily, hence I lean towards the acceptance of the paper and am increasing my rating to 7 (accept). I thank the author(s) for their rebuttal.
limitations_and_societal_impact: NA
review_title: null

Meta-review: This paper received 4 strong accepts. The reviewers have lauded the work because of extensive experiments and clear cutting results which are pushing the SOTA. The approach is likely to be of broad interest because this "connectionist" approach is pitted again the "symbolic" one in an area where so-called neuro-symbolic models have been dominating. The AC thus recommends an oral.
Decision: Accept (Oral)