Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to combine the slot-based model with the gaussian mixture model (GMM) to improve the object-centric model. It explicitly represents the slot as the clustering center and uses the distance between slots to learn the mixture model. The experiments show certain improvements compared with some previous models.
strength_and_weaknesses: # Strength
1. The SMM model integrates the learnable slots into the GMM model by replacing the clustering assignment and center update with learnable functions, such design learns the mixture model in an end-to-end manner.
2. The paper conducts some experiments to compare with other object-centric models, demonstrating its efficacy in terms of performance.

# Weakness
1. The SMM model learns to update the slots with a density function and distribute the learned density using a mechanism similar to slot competition. Such design is basically the composition of slot attention and GMM with incremental contribution, the novelty is not sound.
2. The experimental results are not sufficient to convince me that SMM is a promising model that has certain significant advantages over the slot-attention model or GMM. It achieves SOTA performances on set prediction, which is not sufficient for evaluating object-centric learning, direct metrics for object-centric learning such as ARI and PSNR should be used to evaluate.
3. Table 2 does not contain any comparison with other methods, same as Figure1 and Figure2.
4. There is no ablation or further analyses for the modules in SMM, makes its performance mistery and hard to explain.
 
clarity,_quality,_novelty_and_reproducibility: # Clarity and Quality
Middle. The paper is not well-written, with some clear grammar errors, typography issues, and implausible references. For example,  in *abstract*: entities -> entity, in *intro*: centir -> centric. The writing is rough, and some sentences are long and not easy to understand. It seems the paper is not finished entirely.



summary_of_the_review: Overall, I think the paper is not ready and the contributions are incremental, detailed justification are list in *weakness*. I suggest the author to submit it to next venue with sufficient experiments and analyses. The SMM model might deserve further investigation for its potentials in object-centric learning and broader domains.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper contributed a generalised slot-based approach for object centric representations as a Gaussian Mixture Model. To do so, this enables a set property prediction task. Finally, the representation of Gaussian Mixture Model can generalise to a variety of different object-orientated tasks. In their study they did note the gap of transfer learning not being extensively studied in this context. 

strength_and_weaknesses: This paper follows a clear logical flow moving from one concept to the next.

This paper is missing key vidualisation elements that make it difficult to interpret. For example, the image generation figures could have benefitted from a data visualisation technique that showed what was gained and generated universally between the different objects.
clarity,_quality,_novelty_and_reproducibility: The clarity is lacking in certain regards, however, overall the paper was succinct and focussed.

The novelty is somewhat lacking in that their approach might be unique, but the outcomes are similar, if not identical, to other approaches.
summary_of_the_review: Overall, the paper was well written, and although certain technical elements were missing from the paper, the overall quality is in line with the review.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes an algorithm to extract object-centric representations from feature maps inspired by Slot Attention. The main insight is to   represent slots using a mixture of Gaussians. This comes from the observation that Slot Attention can be viewed as performing soft k-means clustering on features maps, and mixtures of Gaussians being another formulation to a similar problem. The authors show that their method is competitive with Slot Attention and other methods to extract object-centric representations.
strength_and_weaknesses: **Strenghts**:

[+] Good quantitative results compared to Slot Attention in the reported benchmarks.

**Weaknesses**:

[-] Soundness: while the idea of using Gaussian mixture models for clustering features is interesting, in practice the authors perform a very different operation according to algorithm 1. Even though it is claimed that the model performs expectation-maximization, from the algorithm it does not seem like it. Additionally, the slots are represented as gaussians with a diagonal covariance, but it is not clear if the authors use the mean to represent the slots (making the variance redundant) or if they sample and then how would this sampling step impacts the gradient computation. In fact, the authors do not explictily mention what is the optimization objective and algorithm in the main document.

[-] Quality and clarity: the document is missing details that make it hard to assess its soundness as discussed in the previous point. Furthermore, the clarity of the document could be improved, with many typos and missing information.

[-] Qualitative results: the qualitative results hardly show a difference between the presented model and Slot Attention (making me doubt the quantitative claims). For example, for Figure 1 it is hard to see a qualitative difference. Furthermore, the results in Figure 3 are not informative, as they clearly show random segmentations of the input images (instead of object-centric segmentations) with poor quality reconstructions.
clarity,_quality,_novelty_and_reproducibility: * Clarity: the paper could be more clear, as there are many typos and there are missing details. Citations do not follow the proper format (they are missing parenthesis in most cases).

* Quality: while the motivation behind the paper is clear, the presented algorithm is unclear and the evaluation is inconclusive.

* Originality: the motivation for the paper is novel enough.

* Reproducibility: Given the doubts about the quantitative results in the experiment section and the missing details on the method, I believe this paper would be hard to reproduce.

Example Typo - in the introduction section "in such object-centir" -> "in such object-centric"
summary_of_the_review: While the paper has a clear and interesting motivation, the algorithm presented in the paper is unclear and has technical flaws. At the same time, the results are unconvincing, and therefore I argue for the rejection of the paper in its current state.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a variant of slot attention resembling the expectation-maximization (EM) algorithm for learning Gaussian mixture models: Slots are represented by a set of means and isotropic variances, and they are inferred using EM update rules combined with some learned mappings. It is shown that the resulting component, named slot mixture model (SMM), yields improved set prediction results on CLEVR, as well as improved reconstruction quality on multi object image datasets preprocessed via a discrete variational autoencoder (dVAE).
strength_and_weaknesses: Strengths:
 - The paper addresses the interesting and relevant problem of improving understanding and performance of the slot attention algorithm.
 - The connection to GMMs is interesting and generally sensible.
 - The set prediction results on CLEVR are promising, especially at lower distance thresholds.

Weaknesses:
 - The paper's contribution is quite incremental, especially since the connection to soft k-means was already discussed in the slot attention paper.
 - The centerpiece of the GMM, the likelihood model, is abstracted away into the learned function $f_\theta(x, \mu, \Sigma)$, which is never defined in detail. This not only hurts reproducability, but also makes it difficult to understand to what degree the resulting system still resembles a GMM.
 - In general, the motivating connection to GMMs is underexplored. The proposed method appears to be a somewhat ad-hoc combination of the GMM formulas and learned attention/GRU components as in slot attention. It is unclear which properties of either system transfer to SMM. It would at least be useful to include experimental results for a "pure GMM" version of the model, similar to what was done in the Appendix of the slot attention paper for k-means.
 - The image reconstruction experiment is not convincing. The motivation behind object-centric models like slot attention or SLATE is to learn compositional representations by introducing representation bottlenecks, typically at the cost of reducing reconstruction quality. Exclusively measuring reconstruction quality (and related metrics), without also evaluating the usefulness of the learned representation, therefore misses the point - such improvements could be easily achieved by removing bottlenecks. Given the unconvincing segmentations visible in Fig. 3, it seems plausible that the improvements in reconstruction quality came at the expense of the models primary purpose, the learning of compositional representations.
- It is unclear why the two-stage model SLATE is chosen as a baseline, as the dVAE preprocessing seems to introduce addtional complications making the results harding to interpret.
- The clarity of the writing could be significantly improved (see below).

clarity,_quality,_novelty_and_reproducibility: The overall presentation is quite terse, with little explanation regarding the motivation for the method and its design choices. The argument that SMM's improvements over slot attention stem from it taking into account distances between slots and input features could use some elaboration - which distance metric is meant by this, and why can't slot attention simulate similar behavior?

The text leans heavily on the original slot attention paper, and would be difficult to follow without having read it first. Adding a small section introducing the problem statement, and slot attention, would make the paper more self contained. In general, the writing would benefit from another editing pass, as it contains a number of errors.

Some implementation details are missing, most importantly the definition of $f_\theta(x, \mu, \Sigma)$, but also the architecture of the MLP updating $\mu$. This impairs the paper's reproducability. While these definitions should be included in any case, this issue could also be alleviated if the authors choose to publish their submitted code.



summary_of_the_review: Overall, the proposed a slot attention variant shows promising set prediction results. However, the presentation, analysis, and representation learning evaluation could all be significantly improved. As a result, I do not think the paper is ready for publication in its current form.

----

Post rebuttal update: The authors have introduced major changes to the paper, which partially address my concerns. I think the vanilla clustering and object discovery experiments are both significant improvements, although the latter is somewhat basic and only conducted on one dataset. As expressed in my response to the authors, I still have some concerns regarding clarity and the setup of the SLATE experiment. Given this, as well as the magnitude of the changes introduced to the manuscript during the review process, I still think the paper is not quite ready for publication. I have increased my score from 3 to 5.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper addresses the problem of object-centric representation learning for images using a slot-based neural network. The paper presents a variant of the Slot Attention [1] mechanism that resembles a Gaussian mixture model (GMM) as opposed to a soft k-means clustering algorithm (used in the original Slot Attention mechanism). The method is validated on supervised object detection and property prediction (framed as set prediction), and unsupervised object discovery.

I agree with the reviewers that the investigation of the Slot Attention mechanism under a Gaussian mixture model perspective is interesting. While the paper generally shows quantitative improvements over the original Slot Attention model, these improvements are, however, primarily focused on the supervised set prediction setting of Slot Attention on a very simple task (object property prediction on CLEVR). The field of set-based object detection (and property prediction) has significantly advanced since Slot Attention was published, primarily with methods that build on top of Detection Transformers (DETR) [2]. While these results are promising (as pointed out by reviewer *LBbF*), an improvement over Slot Attention on a synthetic supervised task like the one investigated here is not sufficient to carry the weight of a full conference paper (as highlighted by, e.g., reviewer *oeon*). Other weaknesses of the paper pointed out by the reviewers include the insufficient clarity of writing, concerns about the claimed relation to GMMs (i.e. correctness of the algorithm), and the significance of the remaining experimental evaluation.

During the discussion period, the authors have added additional experiments (incl. on object discovery) and improved the writing, which alleviate some of these concerns. Overall, however, I agree with the reviewers that this paper does not yet meet the bar for acceptance.

[1] Locatello et al., Object-Centric Learning with Slot Attention (NeurIPS 2020)
[2] Carion et al., End-to-End Object Detection with Transformers (ECCV 2020)
justification_for_why_not_higher_score: To meet the bar for acceptance and to justify the added complexity of this model over Slot Attention, the authors should consider providing stronger empirical evidence demonstrating the benefits of the model, going beyond simple synthetic tasks. The paper would also heavily benefit from a stronger focus on self-supervised tasks as opposed to supervised set prediction (for which Slot Attention is no longer a strong baseline) and general improvements in quality and clarity of writing / exposition.
Decision: Reject