Object-Centric Learning with Slot Mixture Models
Authors: Daniil Kirilenko
Alexey Kovalev
Aleksandr Panov
Venue: Submitted to ICLR 2023 (2023)
Keywords: object-centric task
gaussian mixture model
slot attention

Abstract: Object-centric architectures usually apply some differentiable module on the whole feature map to decompose it into sets of entities representations called slots. Some of these methods structurally resemble clustering algorithms, where the center of the cluster in latent space serves as slot representation. Slot Attention is an example of such a method as a learnable analog of the soft k-Means algorithm. In our work, we use the learnable clustering method based on Gaussian Mixture Model, unlike other approaches we represent slots not only as centers of clusters but we also use information about the distance between clusters and assigned vectors, which leads to more expressive slots representations. Our experiments demonstrate that using this approach instead of Slot Attention improves performance in different scenarios achieving state-of-the-art performance in the set property prediction task.

Under review as a conference paper at ICLR 2023
OBIECT-CENTRIC LEARNING WITH SLOT MIXTURE MODELS
Anonymous authors Paper under double-blind review
ABSTRACT
Object-centric architectures usually apply differentiable module the whole feature map to decompose it into sets of entity representations called slots. Some of these methods structurally resemble clustering algorithms, where the center of the cluster in latent space serves as slot representation. Slot Attention is an example of such method as learnable analog of the soft K-Means algorithm. In our work, we use the learnable clustering method based on Gaussian Mixture Model, unlike other approaches we represent slots not only as centers of clusters but we also use information about the distance between clusters and assigned vectors, which leads to more expressive slot representations, Our experiments demonstrate that using this approach instead of Slot Attention improves performance in different object- centric scenarios, achieving the state-of-the-arr performance in the set property prediction task.
INTRODUCTION
In recent years, interest in object-centric representations has greatly increased (Greff et al. (2019)
Burgess et al. (2019): Li al. (2020); Engelcke et al. (2020) 2021 Locatello et al. (2020)) ). Such
representations have the potential to improve the generalization ability of machine learning methods in many domains, such as reinforcement learning (Keramati et al. 2018): Watters et al. (2019a);
Kulkarni et al. (2019); Berner et al.] (0019;; Sun et (2019)), scene representation and generation (El-Nouby al. (2019); Matsumori al. (2021); Kulkarni al. (2019)). reasoning (Yang al.)
(2020)), object-centric visual tasks (Groth al. 2018a); Yi et al. (2020): Singh et al. (2021b)). and
planning (Migimatsu Bohg (2020).
Automatic segmentation of objects on the scene and the formation of structured latent space can be carried out in various ways (Greff et al. (2020)): augmentation features with grouping information, the use of tensor product representation, engaging ideas of attractor dynamics, etc. However, the most effective method for learning object-centric representations is Slot Attention Locatello et al. (2020). Slot Attention maps the input feature vector received from the convolutional encoder to fixed number of output feature vectors, which are called Slots. As result of training, each object is assigned corresponding slot. If the number of slots is greater than the number of objects, then some of the slots remain empty (do not contain objects). This approach showed significant results in such object-centric tasks property prediction and object discovery.
The iterative slot-based approach can be considered as variant of the soft K-Means algorithm (Bauckhage (2015b), where the key/value/query projections are replaced with the identity function and updating via the recurrent neural network is excluded (Locatello et al. (2020)) In our work, we propose another version of the generalization, when the K-Means algorithm considered as Gaussian Mixture Model (Bauckhage| (2015a)). We represent slots not only as centers of clusters, we also use information about distance between clusters and assigned vectors, which leads to more expressive slot representations. Representing slots in this way improves the quality of the model object-centric problems, achieving the state-of-the-e art results in the set property prediction task, compared to highly specialized models (Zhang al. (2019b)), and also improves the generalization ability of the model for image reconstruction task.
The paper is structured follows. In Section 2 provide background information about Slot Attention module and Mixture Models. We also describe the process their training. In Section introduce Slot Mixture Module the modification of Slot Attention Module which provides

---

Under review as a conference paper at ICLR 2023
more expressive slot representations. In Section 4.1 by extensive experiments show that the proposed Slot Mixture Module riches the state-of-ther the--arrrt performance the set property prediction task on the CLEVR dataset Johnssn et al. (2017) and outperforms even highly specialized models
Zhang et al. 2019b). In Section 4.2 we provide experimental results for the image reconstruction
task on four datasets: three with synthetic images (CLEVR-Mirror (inghh et al. (2021a), ShapeStacks
Groth et al. (2018b), ClevrTex Karazija et al. (202I)) and one with real life images COCO-2017
(Lin et (2014)) The proposed Slot Mixture Module improves reconstruction performance. In we demonstrate that Slot Mixture Module outperforms original Slot Attention on the Object Discovery task on the CLEVRIO dataset. In Section 4.4 we compare K-Means and Gaussian Mixture Model clustering approaches on the Set Property Prediction task and show that Gaussian Mixture Model clustering a better choice for object-centric learning. give short overview of we discuss the obtained results, advantages, and limitations
Section 4 .3
related works in Section S In Section 6
of our work.
The main contributions of our paper follows:
1. We proposed generalization of slot-based approach for object-centric representations Gaussian Mixture Model (Section 3).
2. Such representation allows state-of-the-art performance in the set property prediction task, even in comparison with specialized models (Section 4.1), which are not aimed at building disentangled representations.
3. The slot representations as a Gaussian Mixture improve the generalization ability of the model in other object-oriented tasks (Section 4.2)
4. The Slot Mixture Module shows a much faster convergence on the Object Discovery task compare to the Original Slot Attention (Section 4.3
BACKGROUND
2.1 SLOT ATTENTION
Slot Attention (SA) module (Locatello et al. (2020)) is an iterative attention mechanism that is designed to map a distributed feature map set of K slots. Randomly initialized slots from Gaussian distribution with trainable parameters are used to get g projections of slots. Feature map vectors with corresponding projections
as k and vectors. Dot-product attention between q and vectors with the softmax across dimension implies competition between slots for explaining parts of the input. Attention coefficients are used assign v vectors to slots via weighted mean.
Mis
M
k ((nppts) E NxK, attn;.j =
Σ/ Mi."
W Ei= attn:.
attm;
updates = T (inputs) E RxD
Gated Recurrent Unit (GRU) (Cho et al. 2014g) is used for addition slots update. It takes slot representations before update iteration as hidden state and updated slots as inputs. The important property of Slot Attention is that it has permutation invariance with respect to input vectors of the feature map and permutation equivariance with respect to slots. These properties make the Slot Attention module suitable for operating with sets and object-centric representations.
Technically, Slot Attention is a learnable analogue of K-Means clustering algorithm with an additional trainable GRU update step and dot product (with trainable g, k, projections instead of Euclidean distance as the measure of similarity between the input vectors and cluster centroids. At the same time, K- K-Means clustering be considered simplified version of the Gaussian Mixture Model.
2

---

Under review as a conference paper at ICLR 2023
2.2 MIXTURE MODELS
Mixture Models (MM) is a class of parametric probabilistic models, in which it is assumed that each z; from some observations X {I1, .... ZN R NxD is sampled from the mixture distribution with K mixture components and prior mixture weights π E R
N
~ p(z =
Σ Tep(a:(02). P(X = II p(a: 00)
Σ
TA =1
These models can seen the models with latent variables 24,6 E {21,..,, that indicate which component came from. The problem is to find such K groups of component parameters O₂ and component assignments each sample æ; that will maximize the likelihood of the model P(X (0) The Expectation Maximization (EM) algorithm is an iterative algorithm that addresses this problem and includes two general steps. The Expectation (E) step: evaluate the expected value of the complete likelihood P(X, O with respect to the conditional distribution of P(Z
Q(0", "000," πT -Epczix.o)loog P(X, Z(0"),, P(X, ZI0")
Ta x. (k)] (=L=E)))
where 7(*) an indicator function.
The Maximization (M) step: find 0" nt" that maximize Q(0", 200""=="'0" 00, π):
0, π = (=-,a x nT).
One of the most widely used models of this kind is Gaussian Mixture Model (GMM), where each mixture component is modeled as a Gaussian distribution parameterized with its mean values and covariance matrix, which is diagonal in the simplest case: P(x, 0x) N(x; pg, Ex), En diag(a)) In this case, EM algorithm is reduced to the following calculations.
step:
p(=))p(x;; (2)
TxN(< pAk, Ex)
p(zx
Rm pl ZR ((
K k-
Me Ex)
7r,i-
M step:
Yhus N
iml
En Ziml
Ma, p2) - pN
Tk
Zim1 This
Liml
The key difference between the Gaussian Mixture Model and K-Means clustering is that GMM considers not only the centers of clusters, but also the distance between clusters and assigned vectors with the prior probabilities of each cluster.
3 SLOT MIXTURE MODULE
For the purposes of the object-centric learning propose modified Gaussian Mixture Model approach and call it the Slot Mixture Module (SMM). This module uses GMM E and M steps (Section 2.2 map feature map vectors from the convolutional neural network (CNN) encoder the set of slot representations, where slots are concatenation of mean values and diagonal covariance matrix. This set of slots is further used in the downstream task. We also use the same additional neural network update step for the mean values before updating the covariance values:
AA RNN(input=yu) hidden=px), Me MLP((1)) + pkk-
3

---

Under review as a conference paper at ICLR 2023
These two steps serve the needs of the downstream task by linking the external and internal models. The internal model (E and M steps in SMM) tries to update its parameters f., Σ so that the input vectors are assigned to slots with the maximum likelihood. while external model takes these parameters as input. The full pseudocode
presented in the Algorithm RNDD x IR x + R K stands for a log-Gaussian density function with additional h and projections for input and slot vectors.
| A function fo(æ, μa Ediag)
Slot Mixture Module can be seen as an extension of the Slot Attention Module with the following
key differences: (1) SMM updates not only the mean values, but also the covariance values and prior probabilities, (2) the Gaussian density function is used instead of the dot-produe attention, and (3) slots are considered only mean values of the cluster, but the concatenation of mean and covariance values.
Algorithm 1:
The Slot Mixture Module pseudocode. initialized uniform categorical distribution, μr and Edigg are initialized from Gaussian distributions with trainable parameters.
Input: I E NR flattened CNN feature map with added positional embeddings;
μ., Edig E RRKxD π R.K SMM initialization parameters.
Output: slots 1R22
-MLPP(LayrNNommzz)
for f 0...7 do
logits = fo(æ, nr, Edigg)
gammas SoftMax((ogits log n, dim=1)
gammas gammas.mean((iin=))
14" = WiigheetMeentveeghssaaannnns values=a)
GRU(input=4L", hidi(deeptt""",hhhdddep==p)) hidden=ju)
+= MLP(LLyerNNrr(44)
Σ diog = WeigheeMMeanteeiiltsggmmnn, values=(x 2)
10 slots = concat([ı,, Eiing
return slots
4 EXPERIMENTS
Since our model can be seen as an extension of the Slot Attention, we use it as our main competitor. In each experiment, we trained two versions of the same model: the one with the Slot Attention module and another one with our SMM, keeping the same training conditions. We also make use of the technique from (Chang et al. (2022) detaching slots from the gradient computational graph the last iteration of the algorithm in every experiment.
With equal dimensionality of input vectors, SMM implies twice the dimensionality of the slots in comparison with SA (SA represents slots with μ., while SMM uses concatenation of μ., Ediag). ). To ensure a fair comparison of approaches, we added matrix multiplication after the SMM module, reducing the dimensionality by a factor of two.
4.1 SET PROPERTY PREDICTION
Neural networks for sets are involved various applications across many data modalities (Carion et al. (2000) [Ahhioptass et al. (2017): [Simonovsky & Komodakis (2018) Fujita et al. (2019). Set Property Prediction is supervised task that requires the model predict an unordered set of vectors representing the properties of objects from the input image. Sets of predicted and target vectors are matched using a Hungarian algorithm (Kuhn (955)) and the learning signal is provided by Huber Loss (Zhang al. (2019a)) between matched vectors. Slot Mixture Module is suitable for operating with sets as it preserves permutation equivariance regarding mixture components (slots) and initializes them randomly. The scheme of the model is shown in the Fig. 1
4

---

Under review as a conference paper at ICLR 2023
Preprecessing
Skx Ansigement
Prop, Prediction daject Marching
Meantdemea
Ii's Sv-
sumfiir
Mehha
Manching
Recader
NUP
Figure 1: Architecture the set property prediction model. Input image is encoded into set of vectors with added positional embeddings via CNN encoder. These vectors assigned to slots with our Slot Mixture Module. Then slots representations are passed to the MLP to predict object properties. Ground truth and predicted sets matched using the Hungarian algorithm.
Setup We use the CLEVR dataset with rescaled images to a resolution of 128 x 128 as the data source. All modules except SA/SMM are the same as the (Locatello et al. (2020)) Each model is trained with Adam optimizer (Kingma & Ba (2055) for 1.5 x 10" iterations with OneCycleLR (Smith & Topin (2019)) learning rate scheduler at a maximum learning rate of 4 x 10 4 we use batch size of 512. SA/SMM number of iterations is set to 5 during training and to 7 during evaluation.
The number of slots is equal to 10 since CLEVR images contain 10 fewer objects.
Results Quantitative results of the experiments presented in Table I We compute Average Precision (AP) with certain distance threshold (co means we do not use distance threshold). A predicted vector of properties and coordinates is correct if there is object with the same properties within the threshold distance. The lower the distance threshold, the more difficult the task. Our experiments demonstrate it possible to improve significantly the original Slot Attention performance via detaching slots before the last iteration of slots refinement and rescaling coordinates to wider range of values. With these modifications, the original Slot Attention performance is still worse than the current state-of-te -f-too--tttttee art model iDSPN, while our SMM confidently outperforms iDSPN.
Table 1: Set prediction performance on the CLEVR dataset (AP in %, mean + std for 4 seeds in our experiments). Slot Attention (Locatello al. (2020) and iDSPN (Zhang et al. (2019b)) results are from the original papers. SA* is the Slot Attention model that trained with the same conditions as our SMM: detached slots at the last iteration, rescaled coordinates to the range of [-1, 1].
Model
AP
AP1
0.5
APo.25
APo.125
APo.0625
SA
94.3 + 1.1
86.7 + 1.4
56.0 + 3.6
10.8 + 1.7
0.9 + 0.2
SA*
97.1 + 0.7
94.5 +.7
88.3
3.2
62.5 = 5.4
23.6 1.4
4.6 4 0.3
iDSPN
98.8 0.5
0.6
98.2 2 0.6
95.8 0.7
76.9 + 2.5
32.3 + 3.9
SMM (ours)
99.4 + 0.2
99.3
1.33-00 0.2
98.8
98.4 + 0.7
92.1
1.2
47.3 + 2.5
4.2
IMAGE RECONSTRUCTION USING TRANSFORMER
For comparison an unsupervised image-to-ing to--tiioiimeeeee- task, we use the SLATE model (Singh al.) 2021a replacing the Slot Attention module with our SMM. Unlike the pixel-mixture decoders, SLATE uses an Image GPT (Chen et al. (2020)) decoder conditioned on slot representations to reconstruct in the autoregressive manner the discrete visual tokens from a discrete VAE (dVAE) (Im) ct al. (2077) treating pre-computed slot representations from a dVAE output as query vectors and latent code-vectors of the image as key/value vectors. SLATE demonstrates an impressive ability to capture complex interactions between the slots and pixels of the synthetic images, but exhibits poor performance for real-world data. dVAE encoder, decoder, and latent discrete tokens receiving training signals from MSE between the input image and the reconstructed one with dVAE only. SA/SMM modules and Image GPT trained with cross-entropy using compressed image into
5

---

Under review as a conference paper at ICLR 2023
dVAE tokens target distribution, these gradients blocked from the rest of the model (i.e. dVAE), but both parts of the system are trained simultaneously.
Setup We consider the following datasets: LLEVR-MiNON (Singh et al. (202la)), ClevrTex (araaz ija et al. (202I)) ShapeStacks (Groth al. 20188)), and COCO-2017 (Lin et al. 2014 CLEVR- Mirror is an extension of the standard CLEVR dataset, which requires capturing global relations between local components due the presence of mirror, ShapeStacks tests the ability of the model to describe complex local interactions (multiple different objects stacked on each other), Clevr examines the capabilities of the model in the textural-rich scenes. For ShapeStacks, ClevrTex and COCO we used images rescaled to the resolution of 96 96, CLEVR-Mirror images are rescaled to 64 x 64. Training conditions with hyperparameters corresponding to a certain dataset are taken from (Singh et al. (2021a)), except that we use batch size equal to 64 and 2.5 10x training iterations for all experiments.
Results Table shows the metrics of the image reconstruction performance for the test-parts of different datasets. We use mean squared error (MSE) and Frechet Inception Distance (FID) (Heusel et al. (2017)) computed with the Py PyTocch-Ignite library (Fomin et al. (2020) as measure of quality of the generated images. We also evaluate cross-entropy between tokens from discrete VAE (as
one-hot distributions) and predicted distributions by Image GPT that are conditioned by certain slot representations, as MSE and FID can be limited by the discrete VAE.
Table 2: Reconstruction performance using Image GPT decoder with different conditions.
FID
MSE
Cross-Entropy SMM
Data
SA
SMM
SA
SMM
SA
CLEVR-Mirror
35.4
34.8
4.5
4.3
0.82
0.20
ShapeStacks
56.6
50.4
102.2
67.3
88.3
66
ClevrTex
116
113
358
344
566
517
COCO
129
122
1354
938
540
479
Fig. 2 2indicates the advantage of SMM in terms of cross-entropy during training across all the datasets we use. But as can be seen from the Table this advantage does not always translate into much better image quality metrics of generated images due to dVAE limitations. The largest increase in quality was obtained for the ShapeStacks dataset. Examples of the images from ShapeStacks, ClevrTex and CLEVR-Mirror datasets and their reconstructions are presented in the Fig. 3 The model trained with SMM is much more likely to reconstruct the correct order of objects from the original image, as can be seen in examples from the random batch of 64 images from ShapeStacks dataset (see Fig.
Even though we were able to improve the quality of real-world object-centric image generation, the final quality in general quite poor and attention maps barely reflect human object-centered vision (see Fig. 5). Scaling and extending such visual models to handle a wide range of complex real-world images an area of the future research.
4.3 OBJECT DISCOVERY
Another unsupervised object-centric image-to-iman to-image task is the Object Discovery. In this task, each slot representation is decoded into the 4-channel image using a Spatial Broadcast decoder (Watters et al. 2019b)). The resulting reconstruction is estimated mixture of decoded slots, where the

---

Under review as a conference paper at ICLR 2023
CLEvR-Miroo
CLEVR-TeK
ShapeStacks
COCO
200
Figure 2: Validation cross-entropy during training for different datasets. Our experiments show that using SMM module instead of SA improves validation performance of the autoregressive transformer by about 10 percent during training. The result maintained for all the datasets that we use.
ShapeStacks
ClevrTex
CLEVR-Mirror
Figure Examples image generation with Image GPT conditioned different slot representations. Images in the blue borders from the model with the Slot Attention module, and images in green borders are generated using slots from the Slot Mixture Module. Red color stands for input images.
first three channels responsible for the reconstructed RGB image and the fourth channel is for the weights the mixture component.
Setup To compare with the Slot Attention module, consider the training setup from the original work (Locatello et al. (2020)) for the CLEVR10 dataset but with the decreased number of training steps (300k instead of 500k).
Results Table shows similarity between ground truth segmentation masks objects (excluded background) and mixture coefficients estimated via the Adjusted Rand Index (ARI) score. Fig 6 demonstrates a much faster convergence of the model that uses SMM instead of the Slot Attention, which results in higher ARI score.
4.4 COMPARING VANILLA CLUSTERING
Slot Attention and Slot Mixture modules be reduced K-Means and Gaussian Mixture Model clustering approaches by removing GRU/MLP updates, trainable lt, u projections and LayerNorm layers (Ba al. (2016)). Table shows results of training set prediction model for CLEVR dataset
7

---

Under review as a conference paper at ICLR 2023
Source image
SMM
reconstruction
SA reconstruction
Figure 4: Examples of all the qualitatively incorrectly generated images from the random batch of 64 samples. In 6 cases reconstruction using Slot Attention gave a wrong order objects (blue circle) lost object (red circle), in the remaining samples both modules gave incorrect reconstruction (green circle).
Figure 5: Examples COCO image generations with Image GPT conditioned to slot representations from SMM. The first and the second columns show the input images and the end-to-end dVAE reconstruction correspondingly. The third column shows the generated image and other columns demonstrate the corresponding attention maps from slots to the input image.
using these vanilla clustering methods. Our experiments demonstrate that GMM clustering is better choice for object-centric learning, even without trainable layers.
MSE
Table 3: Adjusted Rand Index (ARI) score be- tween ground truth masks of objects and mixture coefficients for CLEVR1O after 300k iterations of training.
Model
ARI
SA
85.9
304
2006
2504
2006
2005
2006
SMM
91.3
Figure Mean Squared Error (MSE) between input and reconstructed images for Object Discov- ery task during training.
RELATED WORKS
Set prediction Neural network models for sets are applied to different machine learning tasks, such as point cloud generation (Achlioptas et al. (2017)). object detection (Carion et al. (2020)), speaker diarization (Fujita et al. (2019)) and molecule generation (Simonovsky & Komodakis (2018)). Albeit the set structure is suitable in many cases, traditional deep learning models are not inherently suitable for representing sets. There
approaches that built to reflect the unordered nature of
8

---

Under review as a conference paper at ICLR 2023
Table 4: Average Precision with different distance thresholds for set property prediction task on the CLEVR dataset after 100k iterations of training.
Model
APoc
AP1
APo.5
APp.25
AP
K-Means
81.7
49.1
7.2
1.4
0.2
GMM
88.6
53.3
9.2
2.3
0.5
sets, c.g., the Deep Set Prediction Network (DSPN) (Zhang et al. (2019a)) reflects permutation symmetry by running an inner gradient descent loop that changes a set encode more similarly to the input. An improved version of DSPN - iDSPN (Zhang et al. (2019b)) with approximate implicit differentiation provides better optimizations with faster convergence and state-of-the the- performance on the CLEVR dataset. Such models as Slot Attention and TSPN (Kosiorek al. (2020) use set-equivariant self-attention layers to represent the structure of sets.
Object-centric representation The discovery of objects in a scene in unsupervised manner is crucial aspect of representation learning, especially for object-centric tasks. In recent years, many approaches have been presented to solve this problem (Greff' et al. (2019); Burgess al. 2019) Engelcke et al. (2000 2021) and others). Such methods as IODINE (Greff et al. (2019)). 2019)) and GENESIS (Engelcke et al. (2020) are built upon the Variational Autoencoder (VAE) framework (Kingma & Welling (2014); Rezende et al. (2014)). MONET uses the attention network that generates masks and conditions VAE on these masks. IODINE models an image as a spatial Gaussian mixture model to jointly infer the object representation and segmentation. Compared to MONET and IODINE, GENESIS explicitly models dependencies between scene components that allow the sampling of novel scenes. MONET, IODINE, and GENESIS use multiple steps to encode and decode an image, while Slot Attention (and its sequential extension for video (Kipf al. (2021)) uses one step but performs an iterative procedure inside this step. The useful property of Slot Attention is that it produces the set of output vectors (slots) with permutation symmetry. Slots group input information and could be used in unsupervised tasks (object discovery) and supervised tasks (set prediction). GENESIS-v2 (Engelcke et al. (202T)), a development of the GENESIS model, uses attention masks similarly to (Locatello et (2020))
MONET (Burgess
CONCLUSION AND DISCUSSION
In this paper, we propose a new slot-based approach for object-centric representations - the Slot Mixture Module. Our module is generalization of the K-Means clustering algorithm to the Gaussian Mixture Model (Bauckhage) (2015a)). Unlike other approaches, we represent slots not only as centers of clusters, but we also use information about the distance between clusters and assigned vectors, which leads to more expressive slot representations. We have demonstrated on the CLEVR dataset (Johnson et al. (2017)) that using this module achieves the best results in the set property prediction task, even compared to highly specialized models. Also, the use of Slot Mixture Module shows considerable results in the image reconstruction task. On synthetic datasets, such as CLEVR-Mirror (Singh et al. (2021a)), ShapeStacks (Grooth et al. (20185)) ClevrTTex (Karazija et al.] (2021). )). we achieved improved reconstruction performance. However, is worth noting that modern object- centric models still do not perform well enough on real-life images, such as in COCO-17 (Lin et al. (2014)) The generalization of such methods real data is an important scientific problem. We also show that Slot Mixture Module outperforms original Slot Attention on the Object Discovery task on the CLEVR1O dataset.
Also, in models using the slot representation, the number of slots is hyperparameter, the same as the number of clusters for the K-Means clustering algorithm. In object-centric methods, this parameter is usually chosen equal to the maximum number of objects in the image plus one (assuming one slot is reserved for the background). This value varies from image to image, making it difficult to transfer from one dataset to another. It should be noted that, in general, the issue of transfer learning for clot-based models has not been sufficiently studied. Another limitation of modern slot- based models is that they work with small number of slots (about 10), which means small number of objects in the image. Therefore, the issue of scalability of such models to a larger number of objects is considered direction for further research.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Panos Achlioptas, Olga Diamanti, Toannis Mitliagkas, and Leonidas J. Guibas. Representation learning and adversarial generation of 3d point clouds. ArXiv, abs/1777.0/22292 2017.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv: 1607000600 2016.
Christian Bauckhage. Lecture notes on data science: k-means clustering is gaussian mixture modeling. 2015a.
Christian Bauckhage. Lecture notes on data science: Soft k-means clustering. 2015b.
Christopher Berner, Greg Brockman, Brooke Chan, Vicki Cheung, Przemyslaw Debiak, Christy Dennison, David Farhi, Quirin Fischer, Shariq Hashme, Christopher Hesse, Rafal Józefowicz, Scott Gray, Catherine Olsson, Jakub W. Pachocki, Michael Petrov, Henrique Pondé de Oliveira Pinto, Jonathan Raiman, Tim Salimans, Jeremy Schlatter, Jonas Schneider, Szymon Sidor, Ilya Sutskever, Jie Tang, Filip Wolski, and Susan Zhang. Dota with large scale deep reinforcement learning. ArXiv, abs/1912.06650 2019.
Christopher Burgess, Loie Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matthew M. Botvinick, and Alexander Lerchner. Monet: Unsupervised scene decomposition and representation. ArXiv, abs/1901. 1390, 2019.
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End object detection with transformers. ArXiv, 1287,,, 2020.
Michael Chang, Thomas L. Griffiths, and Sergey Levine. Object representations as fixed points: Training iterative inference algorithms with implicit differentiation. In ICLR2022 Workshop on the Elements of Reasoning: Objects, Structure and Causality, 2022. URL httpss://ppn net fiet/forum? dd
Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1691-1703. PMLR, 13-18 Jul 2020. URL http::::::// WRRpttsss//ppccedii.
m1 mlr.press/vll swteesssvv19ie
wtwp1///chheneseee wttppess//19/ccen20000 html
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN mncodrr-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724-1734, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/v1/D14-1111 URL htt.ttss//acllaamhoooog
org/DD4-1179
Alaaeldin El-Nouby, Shikhar Sharma, Hannes Schulz, Devon Hjelm, Layla El Asri, Samira Ebrahimi Kahou, Yoshua Bengio, and Graham W.Taylor. Tell, draw, and repeat: Generating and modifying images based continual linguistic instruction. 2019 IEEE/CVF International Conference Computer Vision (ICCV), pp. 10303-10311, 2019.
Martin Engelcke, Adam R. Kosiorek, Oiwi Parker Jones, and Ingmar Posner. Genesis: Generative scene inference and sampling with object-centric latent representations. ArXiv, 0ts//997..1005. 2020.
Martin Engelcke, Oiwi Parker Jones, and Ingmar Posner. Genesis-v2: Inferring unordered object representations without iterative refinement. In NeurlPS, 2021.
V. Fomin, J. Anmol, S. Desroziers, J. Kriss, and A. Tejani. High-level library to help with training 2020.
neural networks in pytorch. httts:///ithhbe wttp:/ww.w.ooo/ttoo//////oe/e//a http://///////tto...oooooo//o///
Yusuke Fujita, Naoyuki Kanda, Shota Horiguchi, Kenji Nagamatsu, and Shinji Watanabe. End-to-end neural speaker diarization with permutation-free objectives. In INTERSPEECH,
2019. 2019.
10

---

Under review as a conference paper at ICLR 2023
Klaus Greff, Raphael Lopez Kaufman, Rishabh Kabra, Nicholas Watters, Christopher P. Burgess, Daniel Zoran, Loie Matthey, Matthew M. Botvinick, and Alexander Lerchner. Multi-object representation learning with iterative variational inference. In ICML, 2019.
Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber. On the binding problem in artificial neural networks. ArXiv, abs/2012.05206 2020.
Oliver Groth, Fabian B. Fuchs, Ingmar Posner, and Andrea Vedaldi. Shapestacks: Learning vision- based physical intuition for generalised object stacking. ArXiv, abs/1804.08018 2018a
Oliver Groth, Fabian B. Fuchs, Ingmar Posner, and Andrea Vedaldi. Shapestacks: Learning vision- based physical intuition for generalised object stacking, 2018b. BtRLhtpps:// /ttp::::/axx.vvoor///oo abs 1804. 08018
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS' 17, pp. 6629 6640, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Daniel Jiwoong Im, Sungjin Ahn, Roland Memisevic, and Yoshua Bengio. Denoising criterion for variational auto-encoding framework. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI' 17, pp. 2059-2065. AAAI Press, 2017.
Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In CVPR, 2017.
Laurynas Karazija, Iro Laina, and Christian Rupprecht. Clevrtex: texture-rich benchmark for unsupervised multi-object segmentation. CoRR, abs/2111.10265. 2021. URL httts:://tttx:: exq/abs/240 2211 10265
Ramtin Keramati, Jay Whang, Patrick Cho, and Emma Brunskill. Fast exploration with simplified models and approximately optimistic planning in model based reinforcement learning. arXiv: Artificial Intelligence, 2018.
Diederik Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. http:
//arxiv. abs 1112. 6980
Diederik P. Kingma and Max Welling, Auto-encoding variational bayes. CoRR, abs/1312.6114, 2014.
Thomas Kipf, Gamaleldin F. Elsayed. Aravindh Mahendran, Austin Stone, Sara Sabour, Georg Heigold, Rico Jonschkowski, Alexey Dosovitskiy, and Klaus Greff. Conditional object-centric learning from video. ArXiv, abs/21 111111.12000 12.594 2021.
Adam R. Kosiorek, Hyunjik Kim, and Danilo Jimenez Rezende. Conditional set generation with transformers. ArXiv, abs/2006. 16841, 2020.
H. W. Kuhn. The hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1-2):3399, 1955. doi:
Tejas D. Kulkarni, Ankush Gupta, Catalin Ionescu, Sebastian Borgeaud, Malcolm Reynolds, Andrew Zisserman, and Volodymyr Mnih. Unsupervised learning of object keypoints for perception and control. ArXiv, abs/19066 1188, 2019.
Nanbo Li, Cian Eastwood, and Robert Fisher. Learning object-centric representations of multi-object scenes from multiple views. Advances in Neural Information Processing Systems, 33:5656-5666 2020.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zimicck. Microsoft coco: Common objects in context. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (eds.), Computer Vision - ECCV 2014, pp. 740-755, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10602-1.

---

Under review as a conference paper at ICLR 2023
Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. arXiv preprint (rXXv:2006.150555 2020.
Shoya Matsumori, Kosuke Shingyouchi, Yukikoko Abe, Yosuke Fukuchi, Komei Sugiura, and Michita Imai. Unified questioner transformer for descriptive question generation in goal-oriented visual dialogue. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 1878-1887, 2021.
Toki Migimatsu and Jeannette Bohg. Object-centric task and motion planning in dynamic environ- ments. IEEE Robotics and Automation Letters, 5:844-851, 2020.
Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic back- proeppppppagtiionn and variational inference in deep latent gaussian models. ArXiv, abs/1400 4082, 2014.
Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders. In ICANN, 2018.
Gautam Singh, Fei Deng, and Sungjin Ahn. Illiterate dall-e learns to compose, 2021a. URL htttss://arxie orgg/aa/////o/ 11405
Gautam Singh, Fei Deng, and Sungjin Ahn. Illiterate dall-e learns to compose. ArXiv, abs/2110.11400 2021b.
Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications, volume 11006, pp. 369-386. SPIE, 2019.
Chen Sun, Per Karlsson, Jiajun Wu, Joshua B. Tenenbaum, and Kevin P. Murphy. Stochastic prediction of multi-agent interactions from partial observations. ArXiv, abs/1902.00641 2019.
Nicholas Watters, Loic Matthey, Matko Bosnjak, Christopher Burgess, and Alexander Lerchner. Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration. ArXív, abs/1905.00 90.009975, 2019a.
Nicholas Watters, Loic Matthey, Christopher Burgess, and Alexander Lerchner. Spatial broad- cast decoder: A simple architecture for learning disentangled representations in vaes. ArXiv, 0/5/1901.070111 2019b.
Jianwei Yang. Jiayuan Mao, Jiajun Wu, Devi Parikh, David Cox, Joshua Tenenbaum, and Chuang Gan. Object-centric diagnosis of visual reasoning. ArXiv, 1015/2012.11587, 2020.
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum. Clevrer: Collision events for video representation and reasoning. ArXiv, abs/1910.01442, 2020.
Yan Zhang, Jonathon Hare, and Adam Prnnet-Bennett. Deep set prediction networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc Buc. E. Fox, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019a. URL htttps::/prrceeeiiigg meerrrr httpsss..nr.......ppppppee. wwtpcccpppcppcccctccccc /201 9/file/ PaapedddddPape...pe Paper.pdf pdf
Yan Zhang, Jonathon Hare, and Adam Prugel- Deep set prediction networks. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue, E. Fox, and R. Gameet (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Asso- ciates, Inc., 2019b. URL sttp://uUUuuttsssp/////eeeii http:///g.....ooo..../////////// 6e (er3ggeuhhaae2222eeebbleeee//222 Pww.eeaappprrpp... pdf
12

---

Under review as a conference paper at ICLR 2023
A ARCHITECTURE DETAILS
Tables S 6 describe hyperparameters for our experiments. In the case using SMM instead of SA we use additional dimensionality reduction for slots via trainable matrix multiplication.
Table 5: Architecture of the CNN encoder for the experiments CLEVR dataset for set property prediction and object discovery tasks. Set prediction uses stride of in the layers with 8, while object discovery model uses stride of I in these layers.
Layer
Channels
Activation
Params
Conv2D x5
64
ReLU
stride:
Conv2D 5 x5
64
ReLU
stride: 1/2*
Conv2D 5 x5
64
ReLU
stride: 1/2*
Conv2D 5 x5
64
ReLU
stride:
Position Embedding
absolute
Flatten
dims: w,
LayerNorm
Linear
64
ReLU
Linear
64
Table 6: Spatial broadcast decoder for object discovery task.
Layer
Channels/Size
Activation
Params
Spatial Broadcast
8 x
Position Embedding
absolute
Cony'Toan TwrTssosspse2 5 x5
64
ReLU
stride:
Conv Transpose2D 5 x5
64
ReLU
stride:
Cony Transpose2D 5 x5
64
ReLU
stride:
Cony'Taanspont Transpose21 5 x5
64
ReLU
stride:
Cony Cony'rranpooe2D 5 x5
64
ReLU
stride:
Conv SonyTranspose2D x3
stride:
Split Channels
RGB (3), mask (1)
Softmax masks (slots dim)
Combine components
Table Hyperparameters used for our experiments with SLATE architecture.
Module
Parameter
Value
Image Size
96
Encoded Tokens
576
dVAE
Vocab
4096
dVAE
Temp. Cooldown
1.0 to 0.1
dVAE
Temp. Cooldown Steps
30000
dVAE
LR (no warmup)
0.0003
Transformer
Layers
8
Transformer
Heads
8
Transformer
Hidden Dim.
192
SA/SMM
Num. slots
12
SA/SMM
Iterations
7
SA/SMM
Slot dim.
192
13


Reviews:
[{"review_id": "b7YX-z-oSWS", "reviewer": "Reviewer_oeon", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an algorithm to extract object-centric representations from feature maps inspired by Slot Attention. The main insight is to   represent slots using a mixture of Gaussians. This comes from the observation that Slot Attention can be viewed as performing soft k-means clustering on features maps, and mixtures of Gaussians being another formulation to a similar problem. The authors show that their method is competitive with Slot Attention and other methods to extract object-centric representations.", "summary_of_the_review": "While the paper has a clear and interesting motivation, the algorithm presented in the paper is unclear and has technical flaws. At the same time, the results are unconvincing, and therefore I argue for the rejection of the paper in its current state.", "strength_and_weaknesses": "**Strenghts**:\n\n[+] Good quantitative results compared to Slot Attention in the reported benchmarks.\n\n**Weaknesses**:\n\n[-] Soundness: while the idea of using Gaussian mixture models for clustering features is interesting, in practice the authors perform a very different operation according to algorithm 1. Even though it is claimed that the model performs expectation-maximization, from the algorithm it does not seem like it. Additionally, the slots are represented as gaussians with a diagonal covariance, but it is not clear if the authors use the mean to represent the slots (making the variance redundant) or if they sample and then how would this sampling step impacts the gradient computation. In fact, the authors do not explictily mention what is the optimization objective and algorithm in the main document.\n\n[-] Quality and clarity: the document is missing details that make it hard to assess its soundness as discussed in the previous point. Furthermore, the clarity of the document could be improved, with many typos and missing information.\n\n[-] Qualitative results: the qualitative results hardly show a difference between the presented model and Slot Attention (making me doubt the quantitative claims). For example, for Figure 1 it is hard to see a qualitative difference. Furthermore, the results in Figure 3 are not informative, as they clearly show random segmentations of the input images (instead of object-centric segmentations) with poor quality reconstructions.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "WLEC-cUc7E", "reviewer": "Reviewer_wLsE", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to combine the slot-based model with the gaussian mixture model (GMM) to improve the object-centric model. It explicitly represents the slot as the clustering center and uses the distance between slots to learn the mixture model. The experiments show certain improvements compared with some previous models.", "summary_of_the_review": "Overall, I think the paper is not ready and the contributions are incremental, detailed justification are list in *weakness*. I suggest the author to submit it to next venue with sufficient experiments and analyses. The SMM model might deserve further investigation for its potentials in object-centric learning and broader domains.", "strength_and_weaknesses": "# Strength\n1. The SMM model integrates the learnable slots into the GMM model by replacing the clustering assignment and center update with learnable functions, such design learns the mixture model in an end-to-end manner.\n2. The paper conducts some experiments to compare with other object-centric models, demonstrating its efficacy in terms of performance.\n\n# Weakness\n1. The SMM model learns to update the slots with a density function and distribute the learned density using a mechanism similar to slot competition. Such design is basically the composition of slot attention and GMM with incremental contribution, the novelty is not sound.\n2. The experimental results are not sufficient to convince me that SMM is a promising model that has certain significant advantages over the slot-attention model or GMM. It achieves SOTA performances on set prediction, which is not sufficient for evaluating object-centric learning, direct metrics for object-centric learning such as ARI and PSNR should be used to evaluate.\n3. Table 2 does not contain any comparison with other methods, same as Figure1 and Figure2.\n4. There is no ablation or further analyses for the modules in SMM, makes its performance mistery and hard to explain.\n ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "9ub_sjVwdbU", "reviewer": "Reviewer_LBbF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a variant of slot attention resembling the expectation-maximization (EM) algorithm for learning Gaussian mixture models: Slots are represented by a set of means and isotropic variances, and they are inferred using EM update rules combined with some learned mappings. It is shown that the resulting component, named slot mixture model (SMM), yields improved set prediction results on CLEVR, as well as improved reconstruction quality on multi object image datasets preprocessed via a discrete variational autoencoder (dVAE).", "summary_of_the_review": "Overall, the proposed a slot attention variant shows promising set prediction results. However, the presentation, analysis, and representation learning evaluation could all be significantly improved. As a result, I do not think the paper is ready for publication in its current form.\n\n----\n\nPost rebuttal update: The authors have introduced major changes to the paper, which partially address my concerns. I think the vanilla clustering and object discovery experiments are both significant improvements, although the latter is somewhat basic and only conducted on one dataset. As expressed in my response to the authors, I still have some concerns regarding clarity and the setup of the SLATE experiment. Given this, as well as the magnitude of the changes introduced to the manuscript during the review process, I still think the paper is not quite ready for publication. I have increased my score from 3 to 5.", "strength_and_weaknesses": "Strengths:\n - The paper addresses the interesting and relevant problem of improving understanding and performance of the slot attention algorithm.\n - The connection to GMMs is interesting and generally sensible.\n - The set prediction results on CLEVR are promising, especially at lower distance thresholds.\n\nWeaknesses:\n - The paper's contribution is quite incremental, especially since the connection to soft k-means was already discussed in the slot attention paper.\n - The centerpiece of the GMM, the likelihood model, is abstracted away into the learned function $f_\\theta(x, \\mu, \\Sigma)$, which is never defined in detail. This not only hurts reproducability, but also makes it difficult to understand to what degree the resulting system still resembles a GMM.\n - In general, the motivating connection to GMMs is underexplored. The proposed method appears to be a somewhat ad-hoc combination of the GMM formulas and learned attention/GRU components as in slot attention. It is unclear which properties of either system transfer to SMM. It would at least be useful to include experimental results for a \"pure GMM\" version of the model, similar to what was done in the Appendix of the slot attention paper for k-means.\n - The image reconstruction experiment is not convincing. The motivation behind object-centric models like slot attention or SLATE is to learn compositional representations by introducing representation bottlenecks, typically at the cost of reducing reconstruction quality. Exclusively measuring reconstruction quality (and related metrics), without also evaluating the usefulness of the learned representation, therefore misses the point - such improvements could be easily achieved by removing bottlenecks. Given the unconvincing segmentations visible in Fig. 3, it seems plausible that the improvements in reconstruction quality came at the expense of the models primary purpose, the learning of compositional representations.\n- It is unclear why the two-stage model SLATE is chosen as a baseline, as the dVAE preprocessing seems to introduce addtional complications making the results harding to interpret.\n- The clarity of the writing could be significantly improved (see below).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "0KPiBJP7lr", "reviewer": "Reviewer_koSa", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper contributed a generalised slot-based approach for object centric representations as a Gaussian Mixture Model. To do so, this enables a set property prediction task. Finally, the representation of Gaussian Mixture Model can generalise to a variety of different object-orientated tasks. In their study they did note the gap of transfer learning not being extensively studied in this context. \n", "summary_of_the_review": "Overall, the paper was well written, and although certain technical elements were missing from the paper, the overall quality is in line with the review.", "strength_and_weaknesses": "This paper follows a clear logical flow moving from one concept to the next.\n\nThis paper is missing key vidualisation elements that make it difficult to interpret. For example, the image generation figures could have benefitted from a data visualisation technique that showed what was gained and generated universally between the different objects.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject