Title: Multi-Object Representation Learning via Feature Connectivity and Object-Centric Regularization
Venue: NeurIPS 2023
Accepted: yes
Keywords: Object-Centric Learning

Multi-Object Representation Learning
Abstract: Discovering object-centric representations from images has the potential to greatly improve the robustness, sample efficiency and interpretability of machine learning algorithms. Current works on multi-object images typically follow a generative approach that optimizes for input reconstruction and fail to scale to real-world datasets despite significant increases in model capacity. We address this limitation by proposing a novel method that leverages feature connectivity to cluster neighboring pixels likely to belong to the same object. We further design two object-centric regularization terms to refine object representations in the latent space, enabling our approach to scale to complex real-world images. Experimental results on simulated, real-world, complex texture and common object images demonstrate a substantial improvement in the quality of discovered objects compared to state-of-the-art methods, as well as the sample efficiency and generalizability of our approach. We also show that the discovered object-centric representations can accurately predict key object properties in downstream tasks, highlighting the potential of our method to advance the field of multi-object representation learning.

Multi-Object Representation Learning via Feature Connectivity and Object-Centric Regularization

Abstract

Discovering object-centric representations from images has the potential to greatly improve the robustness, sample efficiency and interpretability of machine learning algorithms. Current works on multi-object images typically follow a generative approach that optimizes for input reconstruction and fail to scale to real-world datasets despite significant increases in model capacity. We address this limitation by proposing a novel method that leverages feature connectivity to cluster neighboring pixels likely to belong to the same object. We further design two object-centric regularization terms to refine object representations in the latent space, enabling our approach to scale to complex real-world images. Experimental results on simulated, real-world, complex texture and common object images demonstrate a substantial improvement in the quality of discovered objects compared to state-of-the-art methods, as well as the sample efficiency and generalizability of our approach. We also show that the discovered object-centric representations can accurately predict key object properties in downstream tasks, highlighting the potential of our method to advance the field of multi-object representation learning.

1 Introduction

Human understanding of the world relies on objects as compositional building blocks [24], and emulating this through object-centric representations can improve robustness, sample efficiency, generalization to out-of-domain distributions, and interpretability of machine learning algorithms [18, 9]. Recent work utilizes a generative approach, optimizing pixel-based reconstruction to learn object-centric representations [8, 17, 33, 4, 35, 12, 13, 11, 45, 46, 51]. This approach has limitations as it prioritizes pixel accuracy over object discovery and functional feature extraction [30, 11]. This may lead to the failure of discovering objects [35], or obtaining useful object features, such as position, shape, or boundaries between overlapping objects [23]. Additionally, pixel-based reconstruction tends to waste model capacity on less important visual features, such as complex backgrounds [25], making scaling these methods to real-world images a challenge.

To address the fundamental limitations of pixel-based reconstruction, we propose a framework that leverages feature connectivity and design two object-centric regularization terms to directly refine object representations, ensuring sufficient separation and high disentanglement between dimensions. Our method utilizes visual connectedness principles [38], where similar pixels that are connected should belong to the same object, to guide object discovery. The two regularization terms promote disentangled representations and prevent sub-optimal clustering.

We demonstrate that our approach outperforms state-of-the-art methods in discovering multiple objects from simulated, real-world, complex texture and common object images in a fine-grained manner without supervision. The proposed solution attains sample efficiency and is generalizable to out-of-domain images. The learned object representations also accurately predict key object

properties in downstream tasks. Our contributions include: (1) a framework that leverages feature connectivity for fine-grained object discovery, (2) introduction of object-centric regularization terms as an alternative to pixel-based reconstruction loss, (3) experimental validation of our solution's superior performance, and (4) demonstration of the usefulness of discovered object representations in downstream tasks.

2 Related Work

Numerous works have demonstrated remarkable success in segmenting real-world images. Most of these works focus on semantic segmentation and object detection by utilizing supervised signals [19, 6]. In contrast, unsupervised approaches like SLIC [1] employ a modified k-means algorithm to cluster pixels into superpixels, similar to Felzenszwalb's algorithm [15] that relies on hand-crafted features for clustering. However, these methods do not focus on learning useful representations for the segmented components.

Various unsupervised methods for learning object-centric representations have been proposed, and can be categorized into three main approaches: spatial attention, sequential attention, and iterative attention. Spatial attention approach utilizes spatial transformer networks [21] to crop out rectangular regions from an image and extract object attributes such as position and scale [14, 31, 8, 33]. They rely on a fixed-size sampling grid which may not be suitable for scenes with varying object sizes, and may compromise training when the sampling grid does not overlap with any object.

The sequential attention approach uses RNN-based models such as MONet [5] and GENESIS [12] to sequentially attend to different regions in an image. These methods employ a deterministic network to perform the attention process, which allows them to capture and represent objects in the scene. However, these methods may neglect smaller objects as they tend to produce a weaker signal during the attention process. This can lead to incomplete or biased representations of scenes with objects of varying sizes. To overcome this, GENESIS-V2 [13] uses a stochastic stick-breaking process to perform attention randomly.

In the iterative attention approach, a set of object representations is randomly initialized and then iteratively refined to bind these objects to different regions of an image. IODINE [17] is a model that can discover objects with disentangled representations. However, it requires long training times and many samples. Slot Attention [35] introduces competition among the object representations by utilizing cross-attention along the object dimension. While this method is fast, versatile and can be extended to handle videos [29], it may fail to discover objects when the training set is diverse, and the resulting representations are highly entangled.

EfficientMORL [11], SLATE [45], SysBinder [46] and BO-QSA [22] are recent developments in the iterative attention approach aimed at addressing some limitations of earlier methods like Slot Attention. EfficientMORL presents an hierarchical variational autoencoder and a lightweight iterative refinement network to increase efficiency without sacrificing representation quality. SLATE increases the non-linear interaction between the slots in Slot Attention with an autoregressive decoder that is conditioned on the slots, resulting in improved reconstructions and object-centric representations. SysBinder enhances the slots of Slot Attention with factor representations called block-slots, which provides within-slot disentanglement between learned factors. BO-QSA initializes the slots of Slot Attention as learnable embeddings instead of sampling from a Gaussian distribution and uses bi-level optimization, resulting in more stable training. Despite the advancements, one drawback remains: the number of clusters are fixed a priori which limits the applicability in real-world scenarios where the number of objects or clusters is not known beforehand.

3 Methodology

Our proposed method OC-Net is designed to extract objects in an image without relying on labeled data or specifying the number of objects present in the image. By not requiring the number of objects to be specified beforehand, OC-Net can generalize better to real-world scenes with varying numbers of objects and handle more complex scenarios.

Figure 1 shows the main components of OC-Net. The main idea behind OC-Net is to learn pixel embeddings that can be clustered to discover objects and their respective object masks and object

1 ×1 Conv

Pixel Embeddings P

Extracting

Input

LayerNorm&

Position Enc.

Entanglement

Object Mask m1

Loss Lent

Computing

feature

Sampling

connectivity

Clustering

Averaging

Object

Representation Z1

Object O1

Extracting

Separation

Entanglement

Loss Lsep

Computing

Object Mask m2

Loss Lent

feature

Sampling

connectivity

Clustering

Averaging

Object

Object O2

Extracting

Separation

Loss Lsep

Entanglement

Computing

Object Mask mM

Loss Lent

feature

Sampling

connectivity

Clustering

Averaging

Object

Object OM

Representation ZM

representations. This is achieved by passing the input image through a 1 × 1 convolutional layer to obtain a set of N pixel embeddings P = { p 1 , . . . , p N } of D dimensions each. We leverage on feature connectivity and iteratively cluster the embeddings of neighbouring pixels based on the likelihood that they belong to the same object. The output is a set of objects O = { O 1 , . . . , O M } where each object is a set of pixel embeddings. We derive the object mask m j of each object by setting the pixel corresponding to the embedding in O j :

where m j [ i ] denotes the i th pixel value and i ∈ { 1 , . . . , N } .

With this, we obtain the matrix of object representations Z = [ z 1 , . . . , z M ] where each z j is the sum of extracted mask information and the average of the pixel embeddings in O j :

where z j [ d ] denotes the d th value of vector z j , d ∈ { 1 , . . . , D } , A is the mask transformation matrix.

3.1 Object Discovery

The object discovery process iteratively clusters the pixel embeddings based on their feature connectivity and similarity. LayerNorm [2] is applied to normalize all pixel embeddings, and positional encodings are added to the pixel embeddings. The neighbors of a pixel embedding p are the set of embeddings of the 8 neighbours in the input image. We use Dijkstra's algorithm to compute the shortest distance of a sampled pixel embedding to all other embeddings as follows.

Let U be the set of pixel embeddings that have not been assigned to an object yet. We uniformly sample a pixel embedding p i ∈ U . The distance from p i to itself is set to zero, and the distance to all the other pixels is set to infinity. We select an unvisited pixel embedding p m that has the minimum distance to p i . Let p k be the embedding of a neighbour of the pixel corresponding to p m . We compute the distance between a pair of neighbouring pixels as the similarity between their corresponding embeddings given by:

where p m [ d ] denotes the d th value of the embedding p m .

If the distance between p i and p k is shorter through p m , we update the shortest distance accordingly. This ensures that we consider the most efficient path between pixel embeddings, leading to better object discovery. We mark p m as visited and consider it to be part of the same object as p i according to a threshold. The process is repeated for the next unvisited pixel embedding until all pixels have been visited.

3.2 Object-Centric Regularization

We design two object-centric regularization terms L sep and L ent to improve the quality of the learned object representations for downstream generalization and object discovery. Given the matrix of object representations Z = [ z 1 , . . . , z M ] corresponding to the object training samples { ( x i , y i ) } i M =1 , we quantify downstream generalization performance with the prediction error:

where W is the minimum-norm solution of the downstream predictor.

Since labels y i are unknown in the unsupervised setting, δ Z cannot be directly minimized. Theorem 3.1 below shows that we can minimize an upper bound of δ Z by using the projection matrix P Z . Proof of the theorem is provided in the supplementary material.

Theorem 3.1. Let Y be the matrix of training sample labels and P Z be the projection matrix of Z :

where I is the identity matrix, ( . ) † is the pseudoinverse and Σ Z = ZZ /latticetop is the unnormalized covariance matrix of Z . Let || . || F be the Frobenius norm. Then, the following relation holds:

Since || Y || F in Equation 6 is unknown but fixed, we can minimize δ Z by minimizing || P Z || F . From the definition of P Z in Equation 5, || P Z || F is minimized when the rank of Σ Z is maximized [44, 3]. We achieve this by maximizing the diagonal entries of Σ Z with a separation term L sep while simultaneously minimizing its off-diagonal entries with an entanglement term L ent , in effect regularizing Σ Z to be a diagonal matrix with a maximum number of nonzero entries.

Maximizing the diagonal entries of Σ Z via L sep consequently maximizes the distance between object representations in the latent space. This encourages the model to learn distinct and non-overlapping object representations. Expanding the representation space also ensures that objects with varying features and properties can be accurately represented and distinguished from one another, enhancing both downstream generalization and fine-grained object discovery. We define L sep as:

where σ d is the variance of the d th dimension across the vectors z 1 , · · · , z M and τ is a small constant to maintain numerical stability.

The entanglement term L ent minimizes the off-diagonal entries of Σ Z and consequently minimizes the correlation between dimensions in the latent space Z , thereby achieves more disentangled object representations. Such representations are easier to manipulate and analyze, as each dimension captures a distinct object property, such as position, scale, or color. L ent is defined as follows:

/negationslash

4 Performance Study

We conduct experiments to evaluate the performance of OC-Net in terms of quality, sample efficiency and generalizability. We use a diverse range of datasets to demonstrate its effectiveness across various scenarios:

Simulated datasets Multi-dSprites [23] and Tetrominoes-NM. The former consists of multiple oval, heart or square-shaped sprites with some occlusions, while the latter is a subset of the original Tetrominoes dataset [23] where images whose ground truth segmentation requires knowledge of the object shapes are filtered out.

Real-world multi-object datasets SVHN [36] and IDRiD [40]. SVHN consists of street view images of house numbers while IDRiD is the Indian Diabetic Retinopathy Image Segmentation Dataset.

Complex texture datasets CLEVRTEX [26] and CLEVRTEX-OOD. CLEVRTEX features scenes with diverse shapes, textures and photo-mapped materials while CLEVRTEX-OOD is the CLEVRTEX out-of-distrbution test set with 25 new materials and 4 new shapes.

Common object datasets Flowers [37], Birds [48] and COCO [50]. The Flowers dataset features 17 diverse flower classes with large variations viewpoint, scale, illumination and background. Birds is the most widely-used CUB-200-2011 dataset for fine-grained visual categorization. COCO is the variant of the Microsoft Common Objects in Context dataset used for large-scale object segmentation [32].

Table 1 shows the dataset characteristics. Following [35, 11], we use the first 60K samples in MultidSprites, Tetrominoes-NM and SVHN for training and hold out the next 320 samples for testing. For IDRiD, we split this dataset into 54 images for training and 27 images for testing. For CLEVRTEX, we use the first 40K samples for training and last 5K samples for testing. For CLEVRTEX-OOD, we use 10K samples for testing. For Flowers, we use the first 6K samples for training and last 1K samples for testing. For Birds, we use the first 10K samples for training and last 1K samples for testing. For COCO, we use the first 10K samples for training and last 2K samples for testing.

We compare OC-Net with SLIC [1], Felzenszwalb [11], Slot Attention [35], EfficientMORL [11], GENESIS-V2 [13], SLATE [45], SysBinder [46] and BO-QSA [22]. We train OC-Net for 1000 iterations with a batch size of 64 using Adam [28] with a learning rate of 1 × 10 -3 . We carried out an initial experiment to choose the clustering threshold. The results show that the value can range from 0.2 to 2.0 without affecting the performance of OC-Net. As such, we set the threshold to /epsilon1 = 0 . 7 so that two pixels will belong to the same object if their normalized feature similarity is more than 50%. If a pixel is assigned to multiple objects, we assign it to the mask of the first object in that list and ignore its membership in other objects. Training on 64-by-64 images from Multi-dSprites on a single V100 GPU with 32GB of RAM takes about 10 minutes.

For all methods, we set the maximum number of foreground objects to 6 and 4 for Multi-dSprites and Tetrominoes respectively. Training is carried out for 300,000 iterations with a batch size of 64, using the Adam optimizer with a base learning rate of 4 × 10 -4 . We set the size of the latent space to be D = 64 for all models. For SVHN and COCO, the number of objects is set to 6. For IDRiD, the number of objects is set to 20 and we train them for 100,000 iterations. For CLEVRTEX and CLEVRTEX-OOD, the number of objects is set to 11. For Flowers and Birds, the number of objects is set to 2.

We use the Adjusted Rand Index (ARI) to measure the quality of objects discovered [20]. The ARI is a measure of similarity between two data clusterings that takes into account the permutation-invariant nature of the predicted segmentation masks and their corresponding ground-truth masks. We also

use the Dice similarity coefficient, along with the Intersection-over-Union (IoU) between the best matching object masks X and Y as follows:

where X the set of object pixels extracted and Y is the set of annotated object pixels in the ground truth. We compute the mean Dice and the mean IoU scores, denoted as mDice and mIoU respectively, by averaging the individual Dice and IoU scores across all matches. For the complex textures dataset, background discovery is included in the computation of the scores.

4.1 Experiments on Quality of Discovered Objects

We first evaluate the ability of OC-Net to discover objects from images with multiple objects. Table 2(a) shows the average ARI, mDice and mIoU scores based on the discovered foreground objects in the simulated datasets after 3 runs. We observe that OC-Net outperforms all other methods by a large margin in Multi-dSprites, and even achieves perfect score for all Tetrominoes-NM test samples.

Table 2(b) shows the results on the real-world multi-object image datasets. For SVHN, OC-Net outperforms all methods, even when the ground truth is provided in the form of bounding boxes. This implies that we need to expand the discovered object masks into their corresponding bounding boxes which are often rough fits, and is the reason for the close difference in mDice and mIoU scores between OC-Net and BO-QSA. For IDRiD, which contains multiple small objects, OC-Net significantly improves the ARI scores and more than doubles the mDice and mIoU scores over all methods, demonstrating its robustness in challenging object discovery tasks.

Table 2(c) shows the results on CLEVRTEX and CLEVRTEX-OOD, which contains complex textured objects and backgrounds. Here, OC-Net again shows superior performance in all metrics, illustrating its capability to effectively segment complex objects. Although a general decrease in performance is observed across all methods in the CLEVRTEX-OOD test set, likely due to a change in data distribution, OC-Net's performance drop is slight and it still outperforms the closest baseline.

Finally, Table 2(d) shows the results on Flowers, Birds and COCO common object datasets. Here, OC-Net again shows superior performance in all metrics, illustrating its capability to effectively segment commonly seen natural objects. Notably, OC-Net outperforms all other methods by a large margin in COCO, demonstrating its robustness in handling objects with highly varied appearances.

Figure 2 visualizes the objects discovered by the various methods for sample images. OC-Net is able to identify large and small objects in Multi-dSprites even when these objects are significantly occluded. Moreover, in the Tetrominoes-NM dataset, despite the presence of shadow effects that often confuse existing methods, OC-Net still manages to separate each tile. For SVHN, only OC-Net is able to segment the character objects out in a fine-grained manner. EfficientMORL tend to group all the characters together while the other methods segment the objects in a coarse-grained manner. For IDRiD, OC-Net is able to segment out the optic disc and small lesions which other methods fail to discover. For CLEVRTEX and CLEVRTEX-OOD, OC-Net is able to segment out the various objects from the complex-textured backgrounds in a fine-grained manner. Finally, for Flowers, Birds, and COCO, only OC-Net is able to segment out the complex-shaped and multi-part objects from the backgrounds in a fine-grained manner.

4.2 Experiments on Sample Efficiency

One obstacle to unsupervised object discovery is the availability of a sufficiently large number of suitable training samples. Sample efficiency refers to a model's ability to learn effectively from a relatively small number of examples. Figure 3 shows the mIoU scores as we decrease the number of training samples in Multi-dSprites, SVHN and CLEVRTEX. OC-Net is able to achieve near-optimal performance even with a significantly smaller training set (1,000 samples) compared to all the other methods. The high sample efficiency of OC-Net reduces the need for large, potentially costly or difficult-to-obtain datasets. This makes OC-Net a more practical solution for real-world applications.

(a) Simulated datasets

(b) Real-world datasets

(c) Complex textures dataset

(d) Common objects datasets

(a) Multi-dSprites

(b) Tetrominoes-NM

(c) SVHN

(d) IDRiD

(e) CLEVRTEX

(f) CLEVRTEX-OOD

(g) Flowers

(h) Birds

Image

Ground Truth

SLIC

Felzenszwalb

Slot Attention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

Image

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

1

Image

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

5c0

L

5cc

0

Image

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

Image

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

Image

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

Image

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

Image

Ground Truth

SLIC

Felzenszwalb

SlotAttention

EfficientMORL

GENESIS-V2

SLATE

SysBinder

BO-QSA

OC-Net

(i) COCO

4.3 Experiments on Model Generalizability

Ideally, an unsupervised object discovery model should be trained to understand common visual appearances such as the difference between foreground vs background, so as to discover objects from

(a) Multi-dSprites

(b) SVHN

(c) CLEVRTEX

100

50

40

80

40

30

60

30

mloU

mloU

SlotAttention

mloU

20

EfficientMORL

40

20

GENESIS-V2

SLATE

10

20

10

SysBinder

BO-QSA

OC-Net

0

0

0

60K

50K

40K

30K

20K

10K

0

60K

50K

40K

30K

20K

10K

0

40K

30K

20K

10K

0

#TrainingSamples

out-of-domain images. In this set of experiments, we compare the generalization ability of OC-Net with the baselines by training the models on Multi-dSprites and testing them on the other datasets.

Table 3 shows the results. For Tetrominoes-NM, there is a decrease in performance for all methods while OC-Net still obtains perfect score. For SVHN, CLEVRTEX and CLEVRTEX-OOD, the performance of all models decrease due to the shift in data distribution. However, OC-Net experiences the smallest drop in performance and still significantly surpasses the best performing method. For IDRiD, the methods show improvement in performance. One possible reason is that the larger training set in Multi-dSprites enables the circular shape of the optic disc to be better segmented. Despite this, OC-Net remains the top performer.

4.4 Ablation Studies

Next, we examine the effect of feature connectivity and regularization terms L sep and L ent on the performance of OC-Net. We implemented three variants of OC-Net: (a) w/o connectivity. Here, we do not require that a path should exist between i and k when computing dist [ i, k ] ; (b) w/o L ent . This network is trained without the entanglement regularization term; (c) w/o L sep . The separation regularization term is not used in the training of OC-Net.

Table 4 shows the mIoU scores for all the datasets. Without feature connectivity, we observe a drop in performance across all datasets since it is common for images to have multiple identical objects, that is, same color and shape. As such, OC-Net w/o connectivity tend to cluster these blocks as a single object.

Removing the entanglement term (OC-Net w/o L ent ) leads to a slight decrease in the performance as the object representations may still be separated even when the dimensions are entangled. The largest performance drop in all datasets is seen when the object representation separation term is removed (OC-Net w/o L sep ), indicating the importance of having a well-separated object representation space for effective object discovery.

5 Prediction based on Learned Object Representation

One characteristic of an effective object-centric representation is its ability to encode object properties such as color, position and shape [42]. In this section, we show that the learned object representations are disentangled and can be used to predict the values of these properties.

Given the representations and their corresponding ground truth values of a target object property, we employ them as features to train a gradient boosted tree (GBT) [16]. To evaluate how well the properties of unseen objects are predicted by the GBT, we use the coefficient of determination R 2 [49]. We perform an experiment using the learned object representations from the simulated datasets to predict the properties of object such as color, position and shape. We use the mIoU score to match the discovered object to the ground truth object. We split the 320 test images equally into two sets, one for training the GBT model for each object property, and the other for evaluation.

Table 5 shows the average R 2 scores of the GBT models on the evaluation set. The GBT models trained with OC-Net representations achieved the highest R 2 scores compared to the models trained using the representations from other methods. This suggests that the object representations learned by OC-Net are effective in encoding the object properties.

6 Conclusion

In this work, we have described a framework called OC-Net that learns object-centric representations in a fine-grained manner without supervision. OC-Net leverages on feature connectivity and two new regularization terms to learn disentangled representations and to ensure the representations of different objects are well-separated. From the results of experiments conducted on simulated, real-world, complex texture and common object images, we have demonstrated the superior quality of the object representations over current state-of-the-art. Moreover, we have highlighted the sample efficiency and generalizability of OC-Net. Finally, we have shown how the discovered object representations can be used to predict object properties in a downstream task, indicating its potential use for other computer vision applications where samples and ground truth labels are limited.

There are still obstacles that have to be overcome for successful application of our framework to the full visual complexity of the real world. A natural next step would be to extend OC-Net to handle real-world scenes containing objects with more complex part-whole hierarchies. It is also promising to explore explicit representation of the discovered objects into a dictionary of prototypes to better handle occlusion between objects. Lastly, real-world scenes with multiple objects is still of higher visual complexity than the datasets considered here and reliably bridging this gap is an open problem.

Acknowledgments and Disclosure of Funding

This research is supported by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: AISG-GC-2019-001-2A).

References

Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Süsstrunk, S. Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence , 34(11):2274-2282, 2012. doi: 10.1109/TPAMI.2012.120.

Ba, J. L., Kiros, J. R., and Hinton, G. E. Layer normalization. Advances in NIPS 2016 Deep Learning Symposium , 2016.

Bardes, A., Ponce, J., and LeCun, Y. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In The Tenth International Conference on Learning Representations, ICLR , 2022.

Bear, D., Fan, C., Mrowca, D., Li, Y., Alter, S., Nayebi, A., Schwartz, J., Fei-Fei, L. F., Wu, J., Tenenbaum, J., et al. Learning physical graph representations from visual scenes. Advances in Neural Information Processing Systems , 33:6027-6039, 2020.

Burgess, C. P., Matthey, L., Watters, N., Kabra, R., Higgins, I., Botvinick, M. M., and Lerchner, A. Monet: Unsupervised scene decomposition and representation. CoRR , 1901.11390, 2019.

Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers. In European conference on computer vision , pp. 213-229. Springer, 2020.

Caron, M., Touvron, H., Misra, I., Jégou, H., Mairal, J., Bojanowski, P., and Joulin, A. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision , pp. 9650-9660, 2021.

Crawford, E. and Pineau, J. Spatially invariant unsupervised object detection with convolutional neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pp. 3412-3420, 2019.

Dittadi, A., Papa, S. S., Vita, M. D., Schölkopf, B., Winther, O., and Locatello, F. Generalization and robustness implications in object-centric learning. In International Conference on Machine Learning, ICML , 2022.

Eastwood, C. and Williams, C. K. I. A framework for the quantitative evaluation of disentangled representations. In International Conference on Learning Representations , 2018.

Emami, P., He, P., Ranka, S., and Rangarajan, A. Efficient iterative amortized inference for learning symmetric and disentangled multi-object representations. In International Conference on Machine Learning , pp. 2970-2981, 2021.

Engelcke, M., Kosiorek, A. R., Jones, O. P., and Posner, I. GENESIS: generative scene inference and sampling with object-centric latent representations. In 8th International Conference on Learning Representations, ICLR 2020 , 2020.

Engelcke, M., Parker Jones, O., and Posner, I. Genesis-v2: Inferring unordered object representations without iterative refinement. Advances in Neural Information Processing Systems , 34: 8085-8094, 2021.

Eslami, S., Heess, N., Weber, T., Tassa, Y., Szepesvari, D., Hinton, G. E., et al. Attend, infer, repeat: Fast scene understanding with generative models. Advances in Neural Information Processing Systems , 29, 2016.

Felzenszwalb, P. F. and Huttenlocher, D. P. Efficient graph-based image segmentation. International journal of computer vision , 59(2):167-181, 2004.

Friedman, J. H. Greedy function approximation: a gradient boosting machine. Annals of statistics , pp. 1189-1232, 2001.

Greff, K., Kaufman, R. L., Kabra, R., Watters, N., Burgess, C., Zoran, D., Matthey, L., Botvinick, M., and Lerchner, A. Multi-object representation learning with iterative variational inference. In International Conference on Machine Learning , pp. 2424-2433, 2019.

Greff, K., Van Steenkiste, S., and Schmidhuber, J. On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208 , 2020.

He, K., Gkioxari, G., Dollár, P., and Girshick, R. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision , pp. 2961-2969, 2017.

Hubert, L. and Arabie, P. Comparing partitions. Journal of classification , 2(1):193-218, 1985.

Jaderberg, M., Simonyan, K., Zisserman, A., et al. Spatial transformer networks. Advances in neural information processing systems , 28, 2015.

Jia, B., Liu, Y ., and Huang, S. Improving object-centric learning with query optimization. In The Eleventh International Conference on Learning Representations , 2023.

Kabra, R., Burgess, C., Matthey, L., Kaufman, R. L., Greff, K., Reynolds, M., and Lerchner, A. Multi-object datasets. https://github.com/deepmind/multi-object-datasets/, 2019.

Kahneman, D., Treisman, A., and Gibbs, B. J. The reviewing of object files: Object-specific integration of information. Cognitive psychology , 24(2):175-219, 1992.

Karazija, L., Laina, I., and Rupprecht, C. Clevrtex: A texture-rich benchmark for unsupervised multi-object segmentation. In Proceedings of the Neural Information Processing Systems , 2021.

Karazija, L., Laina, I., and Rupprecht, C. Clevrtex: A texture-rich benchmark for unsupervised multi-object segmentation. In Vanschoren, J. and Yeung, S. (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks . Curran, 2021.

Kim, J., Choi, J., Choi, H.-J., and Kim, S. J. Shepherding slots to objects: Towards stable and robust object-centric learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 19198-19207, 2023.

Kingma Diederik, P. and Adam, J. B. A method for stochastic optimization. International Conference on Learning Representations ICLR , 2015.

Kipf, T., Elsayed, G. F., Mahendran, A., Stone, A., Sabour, S., Heigold, G., Jonschkowski, R., Dosovitskiy, A., and Greff, K. Conditional Object-Centric Learning from Video. In International Conference on Learning Representations (ICLR) , 2022.

Kipf, T. N., van der Pol, E., and Welling, M. Contrastive learning of structured world models. In 8th International Conference on Learning Representations, ICLR , 2020.

Kosiorek, A., Kim, H., Teh, Y. W., and Posner, I. Sequential attend, infer, repeat: Generative modelling of moving objects. Advances in Neural Information Processing Systems , 31, 2018.

Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 , pp. 740-755. Springer, 2014.

Lin, Z., Wu, Y., Peri, S. V., Sun, W., Singh, G., Deng, F., Jiang, J., and Ahn, S. SPACE: unsupervised object-oriented scene representation via spatial attention and decomposition. In 8th International Conference on Learning Representations, ICLR 2020 , 2020.

Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B., and Bachem, O. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning , pp. 4114-4124. PMLR, 2019.

Locatello, F., Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., Dosovitskiy, A., and Kipf, T. Object-centric learning with slot attention. Advances in Neural Information Processing Systems , 33:11525-11538, 2020.

Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., and Ng, A. Y. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning , 2011.

Nilsback, M.-E. and Zisserman, A. Delving deeper into the whorl of flower segmentation. Image and Vision Computing , 28(6):1049-1062, 2010.

Palmer, S. and Rock, I. Rethinking perceptual organization: The role of uniform connectedness. Psychonomic bulletin & review , 1(1):29-55, 1994.

Paszke, A. e. a. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems , 2019.

Porwal, P., Pachade, S., Kamble, R., Kokare, M., Deshmukh, G., Sahasrabuddhe, V., and Meriaudeau, F. Indian diabetic retinopathy image dataset (idrid), 2018.

Rezende, D. J. and Viola, F. Taming vaes. ArXiv , abs/1810.00597, 2018.

Schölkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., and Bengio, Y. Toward causal representation learning. Proceedings of the IEEE , 109(5):612-634, 2021. doi: 10.1109/JPROC.2021.3058954.

Shi, J. and Malik, J. Normalized cuts and image segmentation. IEEE Transactions on pattern analysis and machine intelligence , 22(8):888-905, 2000.

Shwartz-Ziv, R., Balestriero, R., Kawaguchi, K., Rudner, T. G., and LeCun, Y. An information-theoretic perspective on variance-invariance-covariance regularization. arXiv preprint arXiv:2303.00633 , 2023.

Singh, G., Deng, F., and Ahn, S. Illiterate dall-e learns to compose. In International Conference on Learning Representations , 2021.

Singh, G., Kim, Y., and Ahn, S. Neural systematic binder. In The Eleventh International Conference on Learning Representations , 2023.

Wang, X., Girdhar, R., Yu, S. X., and Misra, I. Cut and learn for unsupervised object detection and instance segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 3124-3134, 2023.

Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P. Caltechucsd birds 200. 2010.

Wright, S. Correlation and causation. Journal of agricultural research , 1921.

Yang, Y. and Yang, B. Promising or elusive? unsupervised object segmentation from real-world single images. Advances in Neural Information Processing Systems , 35:4722-4735, 2022.

Yuan, J., Chen, T., Li, B., and Xue, X. Compositional scene representation learning via reconstruction: A survey. arXiv preprint arXiv:2202.07135 , 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a novel approach towards an unsupervised object-centric representation learning framework. Diverging from previous methodologies that prioritize an input reconstruction scheme, this method emphasizes feature connectivity to cluster neighboring pixels, employing two object-centric regularization losses. The underlying insight for these two losses is the learning of a distinct feature of each object while repelling the object feature between objects by constraining the covariance matrix of the predicted label. To validate their approach, the authors execute their experiments on a variety of image types, ranging from simulated and real-world images to those with complex textures. The results reveal a marked performance improvement when compared to alternative methods, demonstrating the efficacy of the proposed framework.\n\nSTRENGTHS:\n1. The innovative approach proposed in this paper provides food for thought. It suggests that the customary input reconstruction paradigm utilized in prior methods can be substituted with the two object-centric regularization terms introduced in this study. The basic concept underpinning these two loss functions is simple, yet it proves to be highly efficacious.\n\n2. The experimental outcomes presented in the paper are encouraging. The proposed framework demonstrates significant improvement when compared with baseline methods. Moreover, the authors effectively showcase the applicability of the proposed design even when the scale of the dataset diminishes. An ablation study further underscores the relevance and importance of the proposed design in this context.\n\nWEAKNESSES:\n1. The scalability of this method to real-world scenes of higher complexity remains a point of ambiguity. Further testing and discussion are necessary to ascertain the effectiveness of this approach in more complex, real-world scenarios.\n2. Here is one typo: Line 113, to be.\n\nQUESTIONS:\n1. Beyond the L2 distance employed for feature embeddings, exploring the impact of other choices for feature embeddings on model performance and threshold selection could be beneficial. Various methods such as cosine similarity or Mahalanobis distance, among others, could potentially impact the model performance and threshold choice differently.\n\n2. The mask transformation matrix A's initialization strategy is an area that requires further investigation. An exploration into whether a more intricate network structure could enhance the performance would provide more insight into the system's capabilities and potential limitations. This exploration could open up new possibilities for performance optimization.\n\nLIMITATIONS:\nSee weaknesses and questions.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel approach to object-centric (multi-object) representation learning. In contrast to conventional sequential or iterative attention-based methods, the proposed method employs a pixel-wise shortest distance-based clustering technique. Moreover, the paper emphasizes the use of regularization-based representation learning as an alternative to traditional reconstruction-based approaches. The inclusion of theoretical proofs in the appendix, combined with extensive empirical studies, serves to substantiate the paper's claims and highlight the superior and robust performance achieved by the proposed methodology.\n\nSTRENGTHS:\n1. The paper is well-written with clear structures, logical flow, and informative figures that effectively convey the research findings.\n1. The proposed method introduces novel concepts, such as feature connectivity and regularization-based training losses, which lead to impressive results, not only just high score but also in terms of the sample efficiency, on six diverse datasets. These datasets encompass synthetic (simulated), real-world, and complex textures, and the evaluation covers two fundamental tasks in object-centric learning: object discovery and property prediction.\n1. The paper offers theoretical support through a mathematical proof presented in the appendix. This proof establishes an upper bound for the downstream generalization error, enhancing our understanding of the proposed method's performance and its capacity to generalize effectively in downstream tasks.\n\nWEAKNESSES:\n1. The primary concern raised by the reviewer revolves around the scalability of the proposed method to other datasets.\n   - The reviewer questions the difference between using the original RGB input image (with proper normalization) and the feature map generated by the single 1x1 convolution layer where the single 1x1 convolution layer will act as an injective function from a pixel to a representation vector. It is important to address this aspect and provide a clear distinction between the two approaches.\n   - The minimal receptive field (1x1) of the encoder (a single-layer 1x1 convolution) limits the ability to consider local or global context during clustering. This issue is not adequately resolved by the empirical studies since the datasets used lack objects with diverse colors or shapes.\n   - The reviewer acknowledges that the positional encoding helps to compensate for this limitation. However, there is a concern about the naive utilization of positional encoding and its ability to capture complex object shapes. Additionally, there is a worry that the resulting clusters might primarily focus on object colors without incorporating positional information.\n   - In contrast to the reviewer's concerns, the authors claim in section G.1 that the 1x1 convolution contributes to fine-grained object-centric learning. This conflict needs to be addressed and supported by the authors' analysis and empirical evidence.\n   - The reviewer suggests conducting empirical studies on additional datasets with more complex object shapes and color combinations, such as PTR [1], MSN [2], and MOVi [3], as demonstrated in previous works [4,5].\n   - Additionally, considering the results obtained by other encoders used in previous methods, such as Slot Attention [6], would further support the authors' claims.\n\n\n2. Another concern pertains to the rigidity of the feature aggregation process due to the use of a hard clustering algorithm. This rigidity may hinder the proposed method's ability to perform well on downstream tasks and more complex real-world datasets.\n\n[1] Hong, Yining, et al., \"PTR: A Benchmark for Part-based Conceptual, Relational, and Physical Reasoning\", NeurIPS 2021. \n\n[2] Stelzner, Karl, et al., \"Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentation\", ICLR 2022.\n\n[3] Greff, Klaus, et al., \"Kubric: A Scalable Dataset Generator\", CVPR 2022.\n\n[4] Biza, Ondrej, et al., \"Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames\", ICML 2023.\n\n[5] Kim, Jinwoo, et al., \"Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning\", CVPR 2023.\n\n[6] Locatello, Francesco, et al., \"Object-Centric Learning with Slot Attention\", NeurIPS 2020.\n\nQUESTIONS:\n1. In terms of the training schedule, it is necessary to provide a clear description regarding any distinction between \"steps\" (L157) and \"iterations\" (L165). This will help readers better understand the experimental details and avoid any potential confusion.\n1. In the figures depicting qualitative results (Figure 2, 6-13), there seems to be a discrepancy in how the authors classify backgrounds and visualize them as black. While the majority of backgrounds are depicted as black, there are cases where backgrounds are shown in different colors. This raises a question about whether there is any meaningful distinction between black-colored backgrounds and those with different colors. Clarification is needed to understand the criteria used for classifying and representing backgrounds in the visualizations.\n1. The paper utilizes a simplistic approach for positional encoding, which differs from other possible options such as the soft positional embedding used in Slot Attention [1] or fixed sinusoidal positional encoding. Since the chosen method deviates from the conventional approaches, it is important to provide a clear explanation regarding the design and rationale behind the positional encoding used in the paper. This will help readers understand the unique choices made and the implications of the selected approach for the overall methodology.\n\n\n[1] Locatello, Francesco, et al., \"Object-Centric Learning with Slot Attention\", NeurIPS 2020.\n\nLIMITATIONS:\n1. To facilitate a comprehensive and fair comparison, it is crucial to include an analysis of the model's speed in the paper. Given that the proposed method involves iterative updates of pixel distances, which may pose challenges for parallelization, there could potentially be a significant speed sacrifice compared to other methods. However, the paper currently lacks any analysis or discussion regarding the speed performance of the proposed method. \n1. To enhance the persuasiveness of the paper, it would be beneficial to include some more experiments, such as a) recent studies such as ISA [1] and SLASH [2] for the object discovery task, and b) property prediction results over the complex datasets such as CLEVRTEX, in the final version.\n1. The explanation regarding the mask transformation matrix (denoted as A) lacks clarity. It is crucial to provide additional details regarding the initialization and training of the matrix, going beyond the information provided in section A.4. Furthermore, the purpose and necessity of the matrix should be clearly articulated, supported by a thorough description of the procedural intricacies and empirical studies demonstrating its effectiveness.\n\n\n[1] Biza, Ondrej, et al., \"Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames\", ICML 2023.\n\n[2] Kim, Jinwoo, et al., \"Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning\", CVPR 2023.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a method for learning pixel representations for multi-object segmentation. The method works by projecting pixels using $1 \\times 1$ convolution into a feature space. The graph is defined on top of pixels using 8-connectivity. A modified Dijkstra's algorithm assigns pixels to objects based on a distance threshold, where the distance between pixels is measured as the Euclidean distance between normalised feature embeddings. Object representation is formed by averaging the pixel embeddings and adding a learnable mask projection. Two losses are used to learn the pixel embeddings, which aim to increase the diagonal and minimise the off-diagonal entries of the object-feature covariance matrix. The method is evaluated on a series of 2D sprite datasets, SVHN and IDRiD datasets and CLEVRTEX, where the method shows strong segmentation performance.\n\nSTRENGTHS:\nThe presented formulation is quite simple. This leads to short training time and high-sample efficiency.\n\nThe method shows strong performance on benchmarks to the point of saturating or \"solving\" simpler cases.\n\nThe writing is clear.\n\nWEAKNESSES:\nThe description of the method is lacking critical information and details. \n - In particular, it is not clear how final masks are obtained. Description of the method in Algorithm 1 suggests that the object sets $\\mathcal{O}_c$ are not disjoint and may contain the same pixels as other objects (a consequence of revisiting all pixels starting from the empty set (Lines 11 and 24 of the algorithm)). How are the masks obtained in that case, e.g. for evaluation?\n - It is also not clear why the object discovery procedure converges, as it relies upon having learned an already appropriate pixel embedding to always guarantee that the condition in line 21 of the algorithm can be satisfied. What happens if that is not the case?\n\nAs there is no information exchange between pixels in the same object, it is not clear whether semantic or instance information can be learned and extracted. This is somewhat evidenced by the model not performing well on Tetrominoes in the case where the model is required to learn object shapes. As evaluation is limited to simpler or simulated datasets, it is not clear whether this would scale to complex real-world data. Given performance on simpler datasets, it would interesting to show the method working on Birds, COCO, and Flowers in comparison to prior work [21, A].\n\nThere are related works that explore image segmentation by modelling a graph on top of learned or hand-crafted features [B, C, D]. It would be beneficial to discuss and compare to such works, e.g. MaskCut method from [C].\n\nComparison to prior work is limited to older methods, whereas newer methods are not compared against. In particular, a comparison to [21, A, E] is needed given the strong performance on some of the same datasets. Furthermore, methods that were previously shown to obtain good results on CLEVRTEX are not included [E, F, G] in comparisons. Additionally, the object property prediction is limited to simple, nearly monochrome datasets whereas prior work [41] have considered more complex cases in CLEVRTEX already. \n\n\n### References\n\n[A] Seitzer et al. \"Bridging the Gap to Real-World Object-Centric Learning\"\n\n[B] Wang et al. \"Self-supervised Transformers for Unsupervised Object Discovery using Normalized Cut\"\n\n[C] Want et al. \"Cut and learn for unsupervised object detection and instance segmentation\"\n\n[D] Shi and Malik \"Normalized cuts and image segmentation\"\n\n[E] Sauvalle and Fortelle \"Unsupervised multi-object segmentation using attention and soft-argmax\"\n\n[F] Jiang and Ahn \"Generative Neurosymbolic Machines\"\n\n[G] Monnier et al. \"Unsupervised layered image decomposition into object prototypes\"\n\nQUESTIONS:\nIs it correct to say that the LayerNorm is applied _without_ learned affine transformation as outputs of this layer (or functions thereof) are not included in the loss calculation? If it is, how is it trained as the membership of pixel to object set does not appear to be differentiable? \n\nHow is the Figure 2 visualisation obtained? It seems that the object discovery procedure requires pixels to be connected, however, there are several \"floating\" pixels coloured the same colour as the objects.\n\nOn L161, $\\epsilon$ is described as being set such that the normalised similarity should be 50%. How are the similarities normalised?\n\nLIMITATIONS:\nThere seems to be an unstated assumption that objects need to be up to a certain size. I.e. the threshold implicitly defines a maximum distance between the sampled initial pixel and the furthest pixel that can be included. For example, considering an image of solid color, the resulting pixels should have the same features (close to zero due to LayerNorm) save for the added positional embedding. Thus, only pixels $\\epsilon$ from the starting pixel will be added to the object set $\\mathcal{O}_c$.\n\nIt also seems that the assignment of the pixels to a particular object might be heavily dependent on the sampled initial pixel. That is, a pixel j might be included if the initial sampled pixel i is close to it, but not if it is further away as it cannot satisfy the condition on Line 21 of Algorithm 1 due to too many hops. It will then become a part of a different object. Or conversely, pixels of two different objects from a boundary would be included in the same object if one was sampled as initial pixels. Training would further enforce this. Considering the situation above, if the initial sampled pixel was in the corner, would this not potentially create several components?\n\nSimilarly, could it be that the model cannot handle an object split into two regions by another object, such as e.g. a dog behind a fence post?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper discusses a novel method for learning object-centric representations from images, the new approach uses feature connectivity to group together pixels likely to be part of the same object and employs two object-centric regularization terms for refining the representations. \n\nThe method has been tested on various types of images and has significant improvements in object discovery quality, sample efficiency, and generalizability compared to prior work. It has the potential to improve many downstream tasks in predicting object properties. But it also has limitations to handle objects with complex part-whole hierarchies and object occlusions.\n\nSTRENGTHS:\n1. Contributed a novel method for learning object-centric representations from images, the Object Discovery process and Object-Centric Regularization are carefully designed for object-centric learning.\n\n2. Comprehensive mathematical proof and reasoning for the proposed embeddings and regularization terms.\n\n3. Comprehensive comparison between the proposed methods and prior work, and on various datasets.\n\n4. Promising results on sample efficiency, accuracy, and generalizability compared to prior work.\n\n5. Paper writing is clear and succinct.\n\nWEAKNESSES:\n1. Lack of example on incorporating the method to an actual downstream task, and showing the advantage of the method using the proposed terms.\n\n2. Lack of results on more challenging datasets with complex scenes, object types, object shapes, and object occlusions.\n\n3. Designed for object-centric, but as mentioned in the limitations, it has limitations for objects with complex part-whole hierarchies.\n\nQUESTIONS:\nHow does the proposed object entanglement compare to the attention mechanism?\n\nLIMITATIONS:\nThe authors mentioned the method has limitations to handle objects with complex part-whole hierarchies and object occlusions. It would still be interesting to see how the method works on more realistic images, such as a picture of a complex kitchen scene, with part-whole complex cabinetry and a human holding objects and having big object occlusions.", "TITLE": "Official Review", "RECOMMENDATION": 8, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}