Original: Promising or Elusive? Unsupervised Object Segmentation from Real-world Single Images
Revised: Promising or Elusive? Unsupervised Object Segmentation from Real-world Single Images

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: In this paper, we study the problem of unsupervised object segmentation fromsingle images.
Revised: In this paper, we study the problem of unsupervised object segmentation fromsingle images.

Original: We do not introduce a new algorithm, but systematically investigate the effectiveness of existing unsupervised models on challenging real-worldimages.
Revised: We do not introduce a new algorithm, but systematically investigate the effectiveness of existing unsupervised models on challenging real-worldimages.

Original: We firstly introduce four complexity factors to quantitatively measurethe distributions of object- and scene-level biases in appearance and geometry fordatasets with human annotations.
Revised: We firstly introduce four complexity factors to quantitatively measurethe distributions of object- and scene-level biases in appearance and geometry fordatasets with human annotations.

Original: With the aid of these factors, we empiricallyfind that, not surprisingly, existing unsupervised models catastrophically fail tosegment generic objects in real-world images, although they can easily achieveexcellent performance on numerous simple synthetic datasets, due to the vast gapin objectness biases between synthetic and real images.
Revised: With the aid of these factors, we empiricallyfind that, not surprisingly, existing unsupervised models catastrophically fail tosegment generic objects in real-world images, although they can easily achieveexcellent performance on numerous simple synthetic datasets, due to the vast gapin objectness biases between synthetic and real images.

Original: By conducting extensiveexperiments on multiple groups of ablated real-world datasets, we ultimately findthat the key factors underlying the colossal failure of existing unsupervised modelson real-world images is the challenging distributions of object- and scene-levelbiases in appearance and geometry.
Revised: By conducting extensiveexperiments on multiple groups of ablated real-world datasets, we ultimately findthat the key factors underlying the colossal failure of existing unsupervised modelson real-world images is the challenging distributions of object- and scene-levelbiases in appearance and geometry.

Original: Because of this, the inductive biases introducedin existing unsupervised models can hardly capture the diverse object distributions.
Revised: Because of this, the inductive biases introducedin existing unsupervised models can hardly capture the diverse object distributions.

Original: Our research results suggest that future work should exploit more explicitobjectness biases in the network design.
Revised: Our research results suggest that future work should exploit more explicitobjectness biases in the network design.

Original:  Introduction
Revised:  Introduction

Original: The capability of automatically identifying individual objects from complex visual observationsis a central aspect of human intelligence [42].
Revised: The capability of automatically identifying individual objects from complex visual observationsis a central aspect of human intelligence [52].

Original: It serves as the key building block for higher-levelcognition tasks such as planning and reasoning [25].
Revised: It serves as the key building block for higher-levelcognition tasks such as planning and reasoning [28].

Original: In last years, a plethora of models have beenproposed to segment objects from single static images in an unsupervised fashion: from the early
Revised: In last years, a plethora of models have beenproposed to segment objects from single static images in an unsupervised fashion: from the early

Original: AIR [19] and MONet [7] to the recent SPACE [34], SlotAtt [35], GENESIS-V2 [17], etc.
Revised: AIR [22] and MONet [8] to the recent SPACE [39], SlotAtt [40], GENESIS-V2 [20] etc .

Original: Theyjointly learn to represent and segment multiple objects from a single image, without needing anyhuman annotations in training.
Revised: Theyjointly learn to represent and segment multiple objects from a single image, without needing anyhuman annotations in training.

Original: This process is often called perceptual grouping/binding or objectcentric learning.
Revised: This process is often called perceptual grouping/binding or objectcentric learning.

Original: These methods and their variants have achieved impressive segmentation results onnumerous synthetic scene datasets such as dSprites [37] and CLEVR [29].
Revised: These methods and their variants have achieved impressive segmentation results onnumerous synthetic scene datasets such as dSprites [42] and CLEVR [33].

Original: Such advances come withgreat expectations that the unsupervised techniques would likely close the gap with fully-supervisedmethods for real-world visual understanding.
Revised: Such advances come withgreat expectations that the unsupervised techniques would likely close the gap with fully-supervisedmethods for real-world visual understanding.

Original: However, few work has systematically investigatedthe true potential of the emerging unsupervised object segmentation models on complex real-worldimages such as COCO dataset [33].
Revised: However, few work has systematically investigatedthe true potential of the emerging unsupervised object segmentation models on complex real-worldimages such as COCO dataset [38].

Original: This naturally raises an essential question:
Revised: This naturally raises an essential question:

Original: Is it promising or even possible to segment generic objects from real-world single images using(existing) unsupervised methods?
Revised: Is it promising or even possible to segment generic objects from real-world single images using(existing) unsupervised methods?

Original: What is an object?
Revised: What is an object?

Original: To answer the above question involves another fundamental question: whatis an object?
Revised: To answer the above question involves another fundamental question: whatis an object?

Original: Exactly 100 years ago in Gestalt psychology, Wertheimer [48] first introduced a set of Gestalt principles such as proximity, similarity and continuation to heuristically define visual
Revised: Exactly 100 years ago in Gestalt psychology, Wertheimer [60] first introduced a setof Gestalt principles such as proximity, similarity and continuation to heuristically define visualdata as objects.

Revised: However, these factors are highly subjective, whilst the real-world generic objects

Original: [Figure]
Revised: [Figure]

Original: data as objects. However, these factors arehighly subjective, whilst the real-world generic objectsare far more complex with extremely diverse appearances and shapes.
Revised: are far more complex with extremely diverse appearances and shapes.

Original: Therefore, it is practicallyimpossible to quantitatively define what is an object, i.e., the objectness, from visual inputs (e.g., aset of image pixels).
Revised: Therefore, it is practicallyimpossible to quantitatively define what is an object, i . e ., the objectness, from visual inputs ( e . g ., aset of image pixels).

Original: Nevertheless, to thoroughly understand whether unsupervised methods can trulylearn objectness akin to the psychological process of humans, it is vital to investigate the underlyingfactors that potentially facilitate or otherwise hinder the ability of unsupervised models.
Revised: Nevertheless, to thoroughly understand whether unsupervised methods can trulylearn objectness akin to the psychological process of humans, it is vital to investigate the underlyingfactors that potentially facilitate or otherwise hinder the ability of unsupervised models.

Original: In this regard,by drawing on Gestalt principles, we instead define a series of new factors to quantitatively measurethe complexity of objects and scenes in Section 2.
Revised: In this regard,by drawing on Gestalt principles, we instead define a series of new factors to quantitatively measurethe complexity of objects and scenes in Section 2.

Original: By taking into account both the appearance andgeometry of objects and scenes, our complexity factors explicitly assess the difficulty of segmentinga specific object.
Revised: By taking into account both the appearance andgeometry of objects and scenes, our complexity factors explicitly assess the difficulty of segmentinga specific object.

Original: For example, a chair with colorful textures tend to have higher complexity thana single-color ball for unsupervised methods.
Revised: For example, a chair with colorful textures tend to have higher complexity thana single-color ball for unsupervised methods.

Original: With the aid of these factors, we extensively studywhether and how existing unsupervised models can discover objects in Section 4.
Revised: With the aid of these factors, we extensively studywhether and how existing unsupervised models can discover objects in Section 4.

Original: What is the problem of unsupervised object segmentation from single images?
Revised: What is the problem of unsupervised object segmentation from single images?

Original: A large numberof models [50] aim to tackle the problem of unsupervised object segmentation from single images.
Revised: A large numberof models [62] aim to tackle the problem of unsupervised object segmentation from single images.

Original: They share several key problem settings: 1) all training images do not have any human annotations;2)
Revised: They share several key problem settings: 1) all training images do not have any human annotations;2)

Original: every single image has multiple objects; 3) each image is treated as a static data point withoutany dynamic or temporal information; 4) all models are trained from scratch without requiring anypretrained networks on additional datasets.
Revised: every single image has multiple objects; 3) each image is treated as a static data point withoutany dynamic or temporal information; 4) all models are trained from scratch without requiring anypretrained networks on additional datasets.

Original: Ultimately, the goal of these models is to segment allindividual objects as accurate as the ground truth human annotations.
Revised: Ultimately, the goal of these models is to segment allindividual objects as accurate as the ground truth human annotations.

Original: In this paper, we regard thesesettings as the basic and necessary part of unsupervised object segmentation from single images, andempirically evaluate how successfully the existing models can exhibit on real-world images.
Revised: In this paper, we regard thesesettings as the basic and necessary part of unsupervised object segmentation from single images, andempirically evaluate how successfully the existing models can exhibit on real-world images.

Original: Our contributions and findings.
Revised: Contributions and findings.

Original: This paper addresses the essential question regarding the potentialof unsupervised segmentation of generic objects from real-world single images.
Revised: This paper addresses the essential question regarding the potential ofunsupervised segmentation of generic objects from real-world single images

Original: Our key contributionsand findings can be summarized as follows:
Revised: Our contributions are:

Original: • We firstly introduce 4 complexity factors to quantitatively measure the difficulty of objects andscenes.
Revised: • We firstly introduce 4 complexity factors to quantitatively measure the difficulty of objects andscenes.

Original: These factors are key to investigate the true potential of existing unsupervised models.•
Revised: These factors are key to investigate the true potential of existing unsupervised models.

Original:  We extensively evaluate current unsupervised approaches in a large-scale experimental study. 
Revised: • We extensively evaluate current unsupervised approaches in a large-scale experimental study. We

Original: Weimplement 4 representative methods and train more than 130 models on 6 curated datasets fromscratch.
Revised: implement 4 representative methods and train more than 130 models on 6 curated datasets fromscratch.

Original: These datasets and pretrained models will be released for the community.• We analyze our experimental results 
Revised: The datasets, code and pretrained models are available at https://github.com/xxx .• We analyze our experimental results and find that: 1) existing unsupervised object segmentationmodels cannot discover generic objects from single real-world images, although they can achieve outstanding performance on synthetic datasets, as qualitatively illustrated in Figure 1; 2)

Original: and
Revised: the

Original: find that 1) existing unsupervised object segmentation models cannot discover generic objects from singlereal-world images, although they can achieve outstanding performance on synthetic datasets, asqualitatively illustrated in Figure 1;2) the challenging distributions of both object- and scene-level biases in appearance and geometryfrom real-world images are the key factors incurring the failure of existing models;3)
Revised: challenging distributions of both object- and scene-level biases in appearance and geometry from real-world images are the key factors incurring the failure of existing models; 3) the inductive biases

Original: the inductive biases introduced in existing unsupervised models are fundamentally not matchedwith the objectness biases exhibited in real-world images, and therefore fail to discover the realobjectness.
Revised: introduced in existing unsupervised models are fundamentally not matched with the objectnessbiases exhibited in real-world images and therefore fail to discover the real objectness. Related Work.

Revised: Recently, ClevrTex [35] and the concurrent work [43] also study unsupervised objectsegmentation on single images.

Original:  Scope of this research.
Revised: Through evaluation on (complex) synthetic datasets only, both worksfocus on benchmarking the effectiveness of particular network designs of baselines. By comparison,our paper aims to explore what and how the objectness distribution gaps between synthetic andreal-world images incur the failure of existing models. The recent work [59] which investigates videoobject discovery is orthogonal to our work as the motion signals do not exist in single images. Scope of this research.

Original: This paper does not investigate unsupervised object discovery on saliencymaps [47],
Revised: This paper does not investigate unsupervised object discovery on saliencymaps [58],

Original: static multi-views or dynamic videos [50].
Revised: static multi-views or dynamic videos [62].

Original: Recent methods [9, 26] requiring pretrainedmodels on monolithic object images such as ImageNet [39] are not evaluated as well.  
Revised: Recent methods [10, 30] requiring pretrainedmodels on monolithic object images such as ImageNet [47] are not evaluated as well.  Complexity Factors

Original: [Figure]
Revised: [Figure]

Original: Object #
Revised: Object #

Original: Object # Object #
Revised: Object # Object #

Original: Scene # Scene #Scene #
Revised: Scene # Scene #

Original:  As illustrated in the top row of Figure 2, an individual object,represented by a set of color pixels painted within a mask, canvary significantly given different types of appearance and geometric shape.
Revised: [Figure] As illustrated in the top row of Figure 2, an individual object,represented by a set of color pixels painted within a mask,can vary significantly given different types of appearance andgeometric shape.

Original: A specific scene, represented by a set of objectsplaced within an image, can also differ vastly given differenttypes of relative appearance and geometric layout between objects, as illustrated in the bottom row.
Revised: A specific scene, represented by a set ofobjects placed within an image, can also differ vastly givendifferent types of relative appearance and geometric layoutbetween objects, as illustrated in the bottom row.

Original: Unarguably, such variationand complexity of appearance and geometry in both object leveland scene level directly affects human’s ability to precisely separate all objects.
Revised: Unarguably,such variation and complexity of appearance and geometry inboth object level and scene level directly affects human’s abilityto precisely separate all objects.

Original: Naturally, the performance of unsupervisedsegmentation models are also expected to be influenced by thevariation.
Revised: Naturally, the performanceof unsupervised segmentation models are also expected to beinfluenced by the variation.

Original: In this regard, we carefully define the following twogroups of factors to quantitatively describe the complexity ofdifferent datasets.
Revised: In this regard, we carefully definethe following two groups of factors to quantitatively describethe complexity of different datasets.

Original: 2.1 Object-level Complexity Factors
Revised: 2.1 Object-level Complexity Factors

Original: As to a specific object, all its information can be described by appearance and geometry.
Revised: As to a specific object, all its information can be described by appearance and geometry.

Original: Thereforewe define the below two factors to measure the complexity of appearance and geometry respectively.
Revised: Thereforewe define the below two factors to measure the complexity of appearance and geometry respectively.

Original: Notably, both factors are nicely invariant to the object scale.
Revised: Notably, both factors are nicely invariant to the object scale.

Original: • Object Color Gradient: This factor aims to calculate how frequently the appearance changes
Revised: • Object Color Gradient: This factor aims to calculate how frequently the appearance changes

Original: within the object mask.
Revised: within the object mask.

Original: In particular, given the RGB image and mask of an object, we firstlyconvert RGB into grayscale and then apply Sobel filter [41] to compute the gradients horizontally
Revised: In particular, given the RGB image and mask of an object, we firstly convert RGB into grayscale and then apply Sobel filter [51] to compute the gradients horizontally

Original: and vertically for each pixel within the mask.
Revised: and vertically for each pixel within the mask.

Original: The final gradient value is obtained by averaging outall object pixels.
Revised: The final gradient value is obtained by averaging outall object pixels.

Original: Note that, the object boundary pixels are removed to avoid the interference of background.
Revised: Note that, the object boundary pixels are removed to avoid the interference ofbackground.

Original: Numerically, the higher this factor is, the more complex texture and/or lighting effect the object has, and therefore it is likely harder to segment.
Revised: Numerically, the higher this factor is, the more complex texture and/or lighting effectthe object has, and therefore it is likely harder to segment.

Original: • Object Shape Concavity: This factor is designed to evaluate how irregular the object boundary is.
Revised: • Object Shape Concavity: This factor is designed to evaluate how irregular the object boundaryis.

Original: Particularly, given an object (binary) mask, denoted as M obj ∈ R H × W , we firstly find the
Revised: Particularly, given an object (binary) mask, denoted as M obj ∈ R H × W , we firstly find the

Original: smallest convex polygon mask ( M cvx ∈ R H × W ) that surrounds the object mask using 
Revised: smallest convex polygon mask ( M cvx ∈ R H × W ) that surrounds the object mask using an existing

Original: an existingalgorithm [16], and then the object shape concavity value is computed as: 1 − (cid:80) M obj / (cid:80) M cvx .
Revised: algorithm [19], and then the object shape concavity value is computed as: 1 − (cid:80) M obj / (cid:80) M cvx .

Original: Clearly, the higher this factor is, the more irregular object shape is, and segmentation is more tricky.
Revised: Clearly, the higher this factor is, the more irregular object shape is, and segmentation is more tricky.

Original: 2.2 Scene-level Complexity Factors
Revised: 2.2 Scene-level Complexity Factors

Original: As to a specific image, in addition to the object-level complexity, the spatial and appearance relationships between all objects can also incur extra difficulty for segmentation.
Revised: As to a specific image, in addition to the object-level complexity, the spatial and appearance relationships between all objects can also incur extra difficulty for segmentation.

Original: We define the following twofactors to quantify the complexity of relative appearance and geometry between objects in an image.
Revised: We define the following twofactors to quantify the complexity of relative appearance and geometry between objects in an image

Original: • Inter-object Color Similarity: This factor intends to assess the appearance similarity between all
Revised: • Inter-object Color Similarity: This factor intends to assess the appearance similarity between all

Original: objects in the same image.
Revised: objects in the same image.

Original: Specifically, we firstly calculate the average color for each object, and
Revised: Specifically, we firstly calculate the average color for each object, and

Original: then compute the pair-wise Euclidean distances of object colors, obtaining a K × K matrix where
Revised: then compute the pair-wise Euclidean distances of object colors, obtaining a K × K matrix where

Original: K represents the object number.
Revised: K represents the object number.

Original: The final inter-object color similarity is calculated by averaging
Revised: The final inter-object color similarity is calculated by averaging

Original: the matrix excluding diagonal entries.
Revised: the matrix excluding diagonal entries.

Original: Intuitively, the higher this factor is, the more similar all
Revised: Intuitively, the higher this factor is, the more similar all

Original: objects appear to be, the less distinctive each object is, and it is harder to separate each object.
Revised: objects appear to be, the less distinctive each object is, and it is harder to separate each object.

Original: • Inter-object Shape Variation: This factor aims to measure the relative geometry diversity between
Revised: • Inter-object Shape Variation: This factor aims to measure the relative geometry diversity between

Original: all objects in the image.
Revised: all objects in the image.

Original: We firstly calculate the diagonal length of each object bounding box, and
Revised: We firstly calculate the diagonal length of each object bounding box, and

Original: then compute the pair-wise absolute differences for all object diagonal lengths, obtaining a K × K matrix.
Revised: then compute the pair-wise absolute differences for all object diagonal lengths, obtaining a K × Kmatrix.

Original: The final inter-object shape variation is the average of the matrix excluding diagonal entries.
Revised: The final inter-object shape variation is the average of the matrix excluding diagonal entries.

Original: If this factor is higher, it means that objects within the image have more diverse and imbalanced
Revised: If this factor is higher it means that objects within the image have more diverse and imbalanced

Original: sizes, and therefore segmenting both gigantic and tiny objects is likely more challenging.
Revised: sizes, and therefore segmenting both gigantic and tiny objects is likely more challenging.

Original: By capturing the appearance and geometry in both object and scene levels, the four factors aredesigned to quantify the complexity of objects and images.
Revised: By capturing the appearance and geometry in both object and scene levels, the four factors aredesigned to quantify the complexity of objects and images.

Revised: For illustration, Figure 3 shows sampleimages for the four factors at different values.

Original:  In fact, they are chosen from more than10 candidates because they areempirically more suitable to differentiate the gaps between synthetic
Revised: The higher the values, the more complex the objectsand scenes. In fact, these factors are carefully selected from more than 10 candidates because they are

Original: [Table]
Revised: [Figure]

Original:  and real-world scenes, and they eventually serve as key indicators to diagnose existing unsupervisedmodels in Section 4.
Revised: empirically more suitable to differentiate the gaps between synthetic and real-world images, and theyeventually serve as key indicators to diagnose existing unsupervised models in Section 4.

Original: Calculation details of the four factors and other candidates are in appendix
Revised: Calculationdetails of the four factors and other candidates are in appendix.

Original: [Table]

Original:  Experimental Design
Revised:  Experimental Design

Original: 3.1 Considered Methods
Revised: 3.1 Considered Methods

Original: A range of works have explored unsupervised object segmentation in recent years.
Revised: A range of works have explored unsupervised object segmentation in recent years.

Original: They are typicallyformulated as (variational) autoencoders (AE/VAE) [31] or generative adversarial networks (GAN)[21].
Revised: They are typicallyformulated as (variational) autoencoders (AE/VAE) [36] or generative adversarial networks (GAN)[24].

Original: GAN based models [11, 3, 6, 44, 5, 46, 1] are usually limited to identifying a single foregroundobject and can hardly discover multiple objects due to the training instabilities, therefore not considered in this paper.
Revised: GAN based models [13, 3, 7, 55, 5, 57, 1] are usually limited to identifying a single foregroundobject and can hardly discover multiple objects due to the training instabilities, therefore not considered in this paper.

Original: As shown in Table 1, the majority of existing models are based on AE/VAE andcan be generally divided into two groups according to the object representation:
Revised: As shown in Table 1, the majority of existing models are based on AE/VAE andcan be generally divided into two groups according to the object representation:

Original: • Factor-based models : Each object is represented by explicit factors such as size, position, ap-
Revised: • Factor-based models : Each object is represented by explicit factors such as size, position, ap-

Original: pearance, etc, and the whole image is a spatial organization of multiple objects.
Revised: pearance, etc ., and the whole image is a spatial organization of multiple objects.

Original: Basically, such
Revised: Basically, such

Original: representation explicitly enforces objects to be bounded within particular regions.
Revised: representation explicitly enforces objects to be bounded within particular regions.

Original: • Layer-based models : Each object is represented by an image layer, i
Revised: • Layer-based models : Each object is represented by an image layer, i

Original: e., a binary mask, and the
Revised: e ., a binary mask, and the

Original: whole image is a spatial mixture of multiple object layers.
Revised: whole image is a spatial mixture of multiple object layers.

Original: Intuitively, this representation does not
Revised: Intuitively, this representation does not

Original: have strict spatial constrains, and instead is more flexible to cluster similar pixels as objects
Revised: have strict spatial constrains, and instead is more flexible to cluster similar pixels as objects.

Original: In order to decompose the input images into objects, these approaches introduce different typesof network architecture, loss functions, and regularization terms as inductive biases.
Revised: In order to decompose the input images into objects, these approaches introduce different typesof network architecture, loss functions, and regularization terms as inductive biases.

Original: These biasesbroadly include: 1) variational encoding which encourages the disentanglement of latent variables; 2)iterative inference which likely ends up with better scene representations over occlusions; 3) objectrelationship regularization such as depth estimation and autoregressive prior which aims at capturingthe dependency of multiple objects; and many other biases.
Revised: These biasesbroadly include: 1) variational encoding which encourages the disentanglement of latent variables; 2)iterative inference which likely ends up with better scene representations over occlusions; 3) objectrelationship regularization such as depth estimation and autoregressive prior which aims at capturingthe dependency of multiple objects; and many other biases.

Original: With different combinations of thesebiases, many methods have shown outstanding performance in synthetic datasets.
Revised: With different combinations of thesebiases, many methods have shown outstanding performance in synthetic datasets.

Original: Among them, wecarefully select 4 representative models for our investigation: 1) AIR [19], 2) MONet [7], 3) IODINE[22], and 4) SlotAtt[35].
Revised: Among them, weselect 4 representative models for our investigation: 1) AIR [22], 2) MONet [8], 3) IODINE [25],and 4) SlotAtt

Revised: We also add the fully-supervised Mask R-CNN [29] as an additional baseline forcomprehensive comparison.

Original: Implementation details are provided in appendix.
Revised: Implementation details are provided in appendix.

Original: 3.2 Considered Datasets
Revised: 3.2 Considered Datasets

Original: We consider two groups of datasets for extensive benchmarking and analysis: 1) three commonly-usedsynthetic datasets: dSprites [37], Tetris [30] and CLEVR [29], 2) three real-world datasets: YCB [8],
Revised: We consider two groups of datasets for extensive benchmarking and analysis: 1) three commonly-usedsynthetic datasets: dSprites [42], Tetris [34] and CLEVR [33], 2) three real-world datasets: YCB [9],

Original: ScanNet [14], and COCO [33], representing the small-scale, indoor- and outdoor-level real scenesrespectively.
Revised: ScanNet [17], and COCO [38], representing the small-scale, indoor- and outdoor-level real scenesrespectively.

Original: Naturally, objects and scenes in different datasets tend to have very different types ofbiases.
Revised: Naturally, objects and scenes in different datasets tend to have very different types ofbiases.

Original: For example, the objects in dSprites tend to have the single-color bias, while COCO does not.
Revised: For example, the objects in dSprites tend to have the single-color bias, while COCO does not.

Original: Generally, the object-level biases can be divided as: 1) appearance biases including different texturesand lighting effects, and 2) geometry biases including the object shape and occlusions.
Revised: Generally, the object-level biases can be divided as: 1) appearance biases including different texturesand lighting effects, and 2) geometry biases including the object shape and occlusions.

Original: Similarly,the scene-level biases include: 1) appearance biases such as the color similarity between all objects,and 2) geometry biases such as the diversity of all object shapes.
Revised: Similarly,the scene-level biases include: 1) appearance biases such as the color similarity between all objects,and 2) geometry biases such as the diversity of all object shapes.

Original: In fact, our complexity factorsintroduced in Section 2 are designed to well capture these biases.
Revised: In fact, our complexity factorsintroduced in Section 2 are designed to well capture these biases.

Original: Table 2 qualitatively summarizes the biases of selected datasets.
Revised: Table 2 qualitatively summarizesthe biases of selected datasets.

Original: We may hypothesize that the large gaps of biases between syntheticand real-world datasets would have a huge impact on the effectiveness of existing models. 
Revised: We may hypothesize that the large gaps of biases between syntheticand real-world datasets would have a huge impact on the effectiveness of existing models. [Table]

Original: [Table]
Revised: [Table]

Original: To guarantee the fairness and consistency of all experiments, we carefully prepare all six datasetsusing the following same protocols.
Revised: To guarantee the fairness and consistency of all experiments, we carefully prepare all six datasetsusing the following same protocols.

Original: Preparation details for each dataset are provided in appendix.
Revised: Preparation details for each dataset are provided in appendix.

Original: • All images are rerendered or cropped with the same resolution of 128 × 128 .
Revised: • All images are rerendered or cropped with the same resolution of 128 × 128 .

Original: • Each image has about 2 to 6 solid objects with a blank background.
Revised: • Each image has about 2 to 6 solid objects with a blank background.

Original: • Each dataset has about 10000 images for training, 2000 images for testing.
Revised: • Each dataset has about 10000 images for training, 2000 images for testing.

Original: 3.3 Considered Metrics
Revised: 3.3 Considered Metrics

Original: Having the six representative datasets and four existing unsupervised methods at hand, we choosethe following metrics to evaluate the object segmentation performance: 1) AP score which is widelyused for object detection and segmentation [20], 2) PQ score which is used to measure non-overlappanoptic segmentation [32], and 3) Precision and Recall scores.
Revised: Having the six representative datasets and four existing unsupervised methods at hand, we choosethe following metrics to evaluate the object segmentation performance: 1) AP score which is widelyused for object detection and segmentation [23], 2) PQ score which is used to measure non-overlappanoptic segmentation [37], and 3) Precision and Recall scores.

Original: A predicted mask is consideredcorrect if its IoU against a ground truth mask is above 0.5.
Revised: A predicted mask is consideredcorrect if its IoU against a ground truth mask is above 0.5.

Original: All objects are treated as a single class.
Revised: All objects are treated as a single class.

Original: Theblank background is not taken into account for fair comparison.
Revised: Theblank background is not taken into account for fair comparison.

Original: To compute AP, we simply treat themean value of the soft object mask as the object confidence score.
Revised: To compute AP, we simply treat themean value of the soft object mask as the object confidence score.

Original: Note that, the alternative metrics
Revised: Note that, the alternative metrics

Original: ARI [38] and segmentation covering (SC) [4] are not considered as they can be easily saturated.
Revised: ARI [45] and segmentation covering (SC) [4] are not considered as they can be easily saturated.

Original:  Key Experimental Results
Revised:  Key Experimental Results

Original: 4.1 Can current unsupervised models succeed on real-world datasets?
Revised: 4.1 Can current unsupervised models succeed on real-world datasets?

Original: First of all, we evaluate the selected four representative baselines on our six datasets separately.
Revised: First of all, we evaluate all baselines on our six datasets separately.

Original: Inparticular, we train each model from scratch on each dataset separately.
Revised: In particular, we train each modelfrom scratch on each dataset separately.

Original: For fair evaluation, wecarefully tune the hyperparameters of each model on every dataset and fully optimize the networksuntil convergence.
Revised: For fair evaluation, we carefully tune the hyperparameters ofeach model on every dataset and fully optimize the networks until convergence.

Original: Table 3 compares the quantitative results of all models.
Revised: Figure 4 comparesthe quantitative results.

Original: It can be seen thatall methods demonstrate satisfactoryobject segmentation results on the three synthetic datasets,especially the recent strong baselines IODINE and SlotAtt.
Revised: It can be seen that all methods demonstrate satisfactory segmentation resultson synthetic datasets, especially the recent strong baselines IODINE and SlotAtt.

Original: However, not surprisingly, all methodsfail catastrophically on the three real-world datasets.
Revised: However, notsurprisingly, all unsupervised methods fail catastrophically on the three real-world datasets.

Original: [Table]
Revised: [Figure]

Original: Preliminary Diagnosis: In order to diagnose the colossal failure, we hypothesize that it is becauseof huge gaps in objectness biases between two types of datasets.
Revised: Preliminary Diagnosis: In order to diagnose the colossal failure, we hypothesize that it is because ofthe huge gaps in objectness biases between two types of datasets.

Original: In this regard, we quantitativelycompute the distributions of our four complexity factors on the six datasets.In particular, the two [Figure] d S p r
Revised: In this regard, we quantitativelycompute the distributions of our four complexity factors on the six datasets.

Original: i t e s T e t

Original: r

Original: i s C L E V R Y C B S c a n N e t C O C O 0.

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: d S p r

Original: i t e s T e t r i

Original: [Figure]

Original: C L E V R Y C B S c a n N e t C O C O

Original: [Figure]

Original: d S p r i [Figure] object-level factors, i .
Revised: In particular, the twoobject-level factors, i .

Original: e . Object Color Gradient and Object Shape Concavity, are computed foreach object of the six training splits.
Revised: e ., Object Color Gradient and Object Shape Concavity, are computed for eachobject of the six training splits.

Original: The two scene-level factors, i .
Revised: The two scene-level factors, i .

Original: e . Inter-object Color Similarityand Inter-object Shape Variation, are computed for each image of the six training splits.
Revised: e ., Inter-object Color Similarity and Inter-object Shape Variation, are computed for each image of the six training splits.

Original: From thedistributions shown in  Figure 3, we can see that, 1) for the two object-level factors, the three syntheticdatasets tend to have extremely lower scores than the real-world datasets, which means that thesynthetic objects are more likely have uniform colors and convex shapes; 2) for the two scene-levelfactors, the images in synthetic datasets tend to include less similar objects in terms of color, whichmeans that multiple objects in real-world scenes are less distinctive in appearance.
Revised: From the distributions shown in the top row of Figure 5, we can see that, 1) for the two object-levelfactors (Subfigs 1-a and 1-c), the three synthetic datasets tend to have extremely lower scores thanthe real-world datasets, which means that the synthetic objects are more likely have uniform colorsand convex shapes; 2) for the two scene-level factors (Subfigs 1-b and 1-d), the images in syntheticdatasets tend to include less similar objects in terms of color, which means that multiple objects inreal-world scenes are less distinctive in appearance.

Original: In addition, themultiple objects in synthetic scenes tend to have similar sizes, whereas real-world scenes usuallyhave diverse object scales in single images.
Revised: In addition, the multiple objects in syntheticscenes tend to have similar sizes, whereas real-world scenes usually have diverse object scales insingle images.

Original: To validate whether these distribution biases are the truereasons incurring the failure, we conduct extensive experiments in Sections 4.2, 4.3 and 4.4. 
Revised: To validate whether these distribution biases are the true reasons incurring the failure,we conduct extensive experiments in Sections 4.2, 4.3 and 4.4. [Figure]

Revised: (1-a) (1-b) (2-a) (2-b) (3-a) (3-b) (1-c) (1-d) (2-c) (2-d) (3-c) (3-d)

Original: 4.2 How do object-level factors affect current models?
Revised: 4.2 How do object-level factors affect current models?

Original: In this section, we aim to verify to what extent the distributions of object-level factors affect thesegmentation performance.
Revised: In this section, we aim to verify to what extent the distributions of object-level factors affect thesegmentation performance.

Original: In particular, we conduct the following three ablative experiments.
Revised: In particular, we conduct the following three ablative experiments

Original: • Ablation of Object Color Gradient : For each object of the three real-world datasets, we only replaceall pixel colors by its average color rgb value within each object mask, without touching the objectshapes.
Revised: • Ablation of Object Color Gradient : For each object of the three real-world datasets, we only replaceall pixel colors by its average color rgb value within each object mask, without touching the objectshapes.

Original: In this way, the color gradients of each object are totally erased, thus removing the potentialimpact of Object Color Gradient.
Revised: In this way, the c olor gradients of each object are totally erased, thus removing the potentialimpact of Object Color Gradient.

Original: The three ablated datasets are: YCB-S / ScanNet-S / COCO-S.
Revised: The three ablated datasets are: YCB-C / ScanNet-C / COCO-C.•

Original: • Ablation of Object Shape Concavity : For each object in the real-world datasets, we find the smallestconvex hull [16] for its object mask and then fill the empty pixels by shifting the original objectpixels.
Revised: Ablation of Object Shape Concavity : For each object in the real-world datasets, we find the smallestconvex hull [19] for its object mask and then fill the empty pixels by shifting the original objectpixels.

Original: Basically, this ablation aims to only reduce the irregularity of object shapes, yet retaining the distributions of color gradients.
Revised: Basically, this ablation aims to only reduce the irregularity of object s hapes, yet retainingthe distributions of color gradients.

Original: The ablated datasets are: YCB-C / ScanNet-C / COCO-C.
Revised: The ablated datasets are: YCB-S / ScanNet-S / COCO-S. [Figure] SlotAtt

Original:  • Ablation of both Object Color Gradient and Shape Concavity : We simply combine the above two
Revised: [Figure] • Ablation of both Object Color Gradient and Shape Concavity : We simply combine the above two

Original: ablation for each real-world object, getting datasets: YCB-S+C / ScanNet-S+C / COCO-S+C.
Revised: ablation for each real-world object , getting datasets: YCB-C+S / ScanNet-C+S / COCO-C+S.

Original: In  Figure 4(a), we calculate the new distributions of both ObjectColor Gradient and Inter-object Color
Revised: In the second row of Figure 5 (Subfigs 2-a and 2-b), we calculate the new distributions of both Object

Original:  Similarity on the ablated datasets YCB-S/ScanNet-S/COCO-S. It can be seen that the object-levelgradients become all zeros, even simpler than the three synthetic datasets.
Revised: Color Gradient and Inter-object Color Similarity on the ablated datasets YCB-C/ScanNet-C/COCO-C. It can be seen that the object-level gradients become all zeros in Subfig 2-a, even simpler than thethree synthetic datasets.

Original: Yet, the distributions of Inter-object Color Similarity are almost the same as the original YCB/ScanNet/COCO.
Revised: Yet, the distributions of Inter-object Color Similarity are almost the same asthe original YCB/ScanNet/COCO, i .

Original:  Similarly, 
Revised: e ., Subfig 2-b is similar to Subfig 1-b. Similarly, the second row of of Figure 5 (Subfigs 2-c and 2-d) shows that the distributions of

Original: Figure 4(b) shows that the distributions of Object Shape Concavity of ablated datasets now becomesimilar to the synthetic datasets, while the distributions of Inter-object Shape Variation keep the same.
Revised: Object Shape Concavity of ablated datasets now become similar to the synthetic datasets, while thedistributions of Inter-object Shape Variation keep the same, i . e ., Subfig 2-d is similar to Subfig 1-d.

Original: Note that, for the ablation of both Object Color Gradient and Shape Concavity, the distributions willbe the same as shown in  Figure 4.
Revised: Note that, for the ablation of both Object Color Gradient and Shape Concavity, the distributions willbe the same as shown in second row of Figure 5.

Original: Having the three groups of object-level ablated real-world datasets,we then evaluate segmentation performance of the four unsupervised models separately.
Revised: Having the three groups of object-level ablatedreal-world datasets, we then evaluate segmentation performance of the baselines separately.

Original:  Analysis: As shown in Figure 5(a) and (c), we can see that once the pixels of real-world objects arereplaced by its mean color, i .
Revised: Brief Analysis: As shown in Figure 6, we can see that: 1) Once the pixels of real-world objects arereplaced by its mean color, i .

Original: e . without any color gradients, the object segmentation performance hasbeen significantly improved for almost all methods. 
Revised: e ., without any color gradients, the object segmentation performance hasbeen significantly improved for almost all methods. 2)

Original: From Figure 5(b), we can see that reducing theirregularity of real-world objects can also improve the object segmentation, although not significantly.
Revised: Reducing the irregularity of real-world objectscan also improve the object segmentation, although not significantly.

Original: Basically, these results show that exiting methods are more likely to learn the objectness representedby uniform colors and/or regular objects.However, comparing with Table 3, the results of current [Figure] AIR
Revised: Overall, these results showthat existing methods are more likely to learn the objectness represented by uniform colors and/orregular objects.

Original: MONet

Original: IODINE

Original: SlotAtt

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt YCB-S

Original: ScanNet-S

Original: COCO-S

Original: AP PQ Precision Recall

Original: [Figure]

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt YCB-C

Original: ScanNet-C

Original: COCO-C

Original: [Figure]

Original: Recall

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt

Original: AIR

Original: MONet

Original: IODINE

Original: [Figure]

Original: AIR

Original: MONet IODINE [Figure] ablated datasets in Figure 5 still lag behind the synthetic datasets.
Revised: However, comparing with Figure 4, the results of current ablated datasets in Figure6 still lag behind the synthetic datasets.

Original: This means that there must besome other factors that also potentially affect the object segmentation of existing models.
Revised: This means that there must be some other factors that alsopotentially affect the object segmentation of existing models.

Original: 4.3 How do scene-level factors affect current models?
Revised: 4.3 How do scene-level factors affect current models?

Original: In this section, we turn to investigate to what extent the distributions of scene-level factors affect thesegmentation performance.
Revised: In this section, we turn to investigate to what extent the distributions of scene-level factors affect thesegmentation performance.

Original: Particularly, we conduct the following three ablative experiments
Revised: Particularly, we conduct the following three ablative experiments.

Original: • Ablation of Inter-object Color Similarity : In each image of the three real-world datasets, we replaceall object textures by a set of new distinctive textures from the existing DTD database [12].
Revised: • Ablation of Inter-object Color Similarity : In each image of the three real-world datasets, we replace all object t extures by a set of new distinctive textures from the existing DTD database [15].

Original: In thisway, the multiple objects look more distinctive in appearance, while the per-object texture gradientsare roughly preserved.
Revised: In this way, the multiple objects look more distinctive in appearance, while the per-object texture gradientsare roughly preserved.

Original: The ablated datasets are denoted: YCB-T / ScanNet-T / COCO-T.•
Revised: The ablated datasets are denoted: YCB-T / ScanNet-T / COCO-T.

Original:  Ablation of Inter-object Shape Variation : In each image of the real-world datasets, we normalizethe scales of multiple objects by shrinking or expanding the diagonal length of their boundingboxes, such that the new object sizes tend to be uniform.
Revised: • Ablation of Inter-object Shape Variation : In each image of the real-world datasets, we normalizethe scales of multiple objects by shrinking or expanding the diagonal length of their boundingboxes, such that the new object sizes tend to be u niform.

Original: For each object, its shape and texture arelinearly scaled up or down.
Revised: For each object, its shape and texture arelinearly scaled up or down.

Original: Basically, this aims to remove the diversity of object sizes within singleimages.
Revised: Basically, this aims to remove the diversity of object sizes within singleimages.

Original: The ablated datasets are denoted as: YCB-U / ScanNet-U / COCO-U. • Ablation of both Inter-object Color Similarity and Shape Variation : We simply combine the above two ablation strategies for each real-world image.
Revised: The ablated datasets are denoted as: YCB-U / ScanNet-U / COCO-U.• Ablation of both Inter-object Color Similarity and Shape Variation : We simply combine the abovetwo ablation strategies for each real-world image.

Original: Ablation details are in appendix.
Revised: Ablation details are in appendix.

Original: The ablateddatasets are denoted as: YCB-T+U / ScanNet-T+U / COCO-T+U. In Figure 6(a), we calculate the new distributions of both  Inter-object Color Similarityand Object Color Gradient on the ablated datasets: YCB-T/ScanNet-T/COCO-T. It can be seen that the distributions of scene-level color similarity become nearly the same as thethree synthetic datasets, whilethe distributions of object-level color gradients are still  as high as the original real-world datasets 
Revised: The ablateddatasets are denoted as: YCB-T+U / ScanNet-T+U / COCO-T+U.In the third row of Figure 5 (Subfigs 3-a and 3-b), we calculate the new distributions of both ObjectColor Gradient and Inter-object Color Similarity on the ablated datasets YCB-T/ScanNet-T/COCO-T.It can be seen that the distributions of scene-level color similarity become more similar to syntheticdatasets in Subfig 3-b, while the distributions of object color gradients are still similar as originalreal-world datasets YCB/ScanNet/COCO, i .

Original: YCB/ScanNet/COCO.
Revised: e ., Subfig 3-a is similar to Subfig 1-a.

Original: Similarly, Figure 6(b) shows that the distributions of Inter-object Shape Variation of ablated datasets now become similar to the synthetic datasets, whereas the distributions of Object Shape Concavity are still the same as original real-world datasets YCB/ScanNet/COCO. Note [Figure]
Revised: Similarly the third row of Figure 5 shows that the distributions of Inter-object Shape Variation of ablated datasets now become similar to the synthetic datasets in Subfig 3-d, whereas the distributions of Object Shape Concavity are still the same as original real-world datasets YCB/ScanNet/COCO,

Original: ) Ablation of Inter-object Color Similarity

Original: [Figure] AIR MONet IODINE
Revised: i

Original: SlotAtt

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt AIR MONet IODINE
Revised: e .

Original: SlotAtt YCB-T ScanNet-T COCO-T AP
Revised: , Subfig 3-c is similar to Subfig 1-c. Note that, for the ablation of both Inter-object Color Similarity and Shape Variation, the distributions will be the same as the third row of Figure 5.

Original: PQ

Original: Precision

Original: Recall a AIR MONet IODINE
Revised: Having thesethree

Original: SlotAtt

Original: AIR MONet IODINE SlotAtt
Revised: groups

Original: AIR

Original: MONet IODINE SlotAtt YCB-U ScanNet-U
Revised: of scene-level ablated real-world datasets, we then separately evaluate each baseline

Original: COCO-U

Original: AP PQ Precision Recall
Revised: Brief Analysis: As

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt

Original: AIR

Original: MONet

Original: IODINE

Original: SlotAtt AIR MONet IODINE
Revised: shown

Original: SlotAtt YCB-T+U

Original: ScanNet-T+U

Original: COCO-T+U

Original: AP

Original: PQ Precision Recall that, for the ablation of both Inter-object Color Similarity and Shape Variation, the distributions willbe the same as shown inFigure 6. Having these three groups of scene-level of ablated real-worlddatasets, we then separately evaluate the four unsupervised models.
Revised: in

Original: Analysis: As shown in Figure7(a) and (c), we can see that once the textures of real-world imagesare replaced by more distinctive textures, i .e .
Revised: Figure we can see that : 1) Once the textures of real-world imagesare replaced by more distinctive textures, i .

Original:  with lower similarity between object appearance, theobject segmentation performance has been surprisingly boosted remarkablyalmost for  all methods.
Revised: e ., with lower similarity between object appearance, theobject segmentation performance has been surprisingly boosted remarkably for almost all methods.2)

Original: From Figure 7(b), we can see that normalizing object sizes over images can also reasonably improvethe object segmentation performance.
Revised: Normalizing object sizes over images can also reasonably improve the object segmentationperformance.

Original: These results clearly show that existing unsupervised modelssignificantly favor objectness with distinctive appearance in single images.
Revised: 3) Overall, these results clearly show that existing unsupervised models significantlyfavor objectness with distinctive appearance in single images.

Original: However, compared with Table 3, the results on current scene-level ablated datasets are still inferior to the synthetic datasets,meaning that the scene-level factors alone are not enough to explain the performance gap.
Revised: However, compared with Figure 4, theresults on current scene-level ablated datasets are still inferior to the synthetic datasets, meaning thatthe scene-level factors alone are not enough to explain the performance gap.

Original: 4.4 How do object- and scene-level factors jointly affect current models?
Revised: 4.4 How do object- and scene-level factors jointly affect current models?

Original: In this section, we aim to study how the object- and scene-level factors jointly affect the segmentationperformance.
Revised: In this section, we aim to study how the object- and scene-level factors jointly affect the segmentationperformance.

Original: In particular, we conduct the following three ablative experiments
Revised: In particular, we conduct the following three ablative experiments.

Original: • Ablation of Object Color Gradient, Object Shape Concavity and Inter-object Color Similarity : Ineach image of the three real-world datasets, we replace the object color by averaging all pixels ofthe distinctive texture, and also replace the object shape with a simple convex hull.
Revised: • Ablation of Object Color Gradient, Object Shape Concavity and Inter-object Color Similarity : Ineach image of the three real-world datasets, we replace the object color by averaging all pixels ofthe distinctive texture, and also replace the object shape with a simple convex hull.

Original: The ablateddatasets are denoted: YCB-S+C+T / ScanNet-S+C+T / COCO-S+C+T.•
Revised: The ablateddatasets are denoted: YCB-C+S+T / ScanNet-C+S+T / COCO-C+S+T.• Ablation of Object Color Gradient, Object Shape Concavity and Inter-object Shape Variation :

Original: Ablation of Object Color Gradient, Object Shape Concavity and Inter-object Shape Variation :In each image of the three real-world datasets, we replace the object color by averaging its owntexture, and modify the object shape as convex hull following by size normalization.
Revised: In each image of the three real-world datasets, we replace the object color by averaging its owntexture, and modify the object shape as convex hull following by size normalization.

Original: The ablateddatasets are denoted as: YCB-S+C+U / ScanNet-S+C+U / COCO-S+C+U.•
Revised: The ablateddatasets are denoted as: YCB-C+S+U / ScanNet-C+S+U / COCO-C+S+U.•

Original: Ablation of all four factors : We aggressively combine all four ablation strategies and we getdatasets: YCB-S+C+T+U / ScanNet-S+C+T+U / COCO-S+C+T+U.
Revised: Ablation of all four factors : We aggressively combine all four ablation strategies and we getdatasets: YCB-C+S+T+U / ScanNet-C+S+T+U / COCO-C+S+T+U.

Original: Analysis Since these ablations areactually conducted independently, the new distributions ofthe four complexity factors on current jointly ablated datasets are exactly the same shown in Figures4/6 respectively.
Revised: Since these ablations are conducted independently, the new distributions of four complexity factorson current jointly ablated datasets are exactly the same as the second and third rows of Figure 5.

Original:  Figure 8 shows the results of four unsupervised models on the three groups ofobject- and scene-level ablated datasets. From Figure 8 (a) and (b), we can see that combiningthe two object-level factors and either of scene-level factors to ablate the real-world datasets, thesegmentation performance can be significantly improved as we expected.
Revised: Brief Analysis: As shown in Figure 6, we can see that: 1) Combining the two object-level factorsand either of scene-level factors to ablate the real-world datasets, the segmentation performance canbe improved as we expected, especially for IODINE and SlotAtt. If the challenging real-worldobjects and images are ablated in both object- and scene-level, the segmentation performance ofall unsupervised models achieves the same level with three synthetic datasets as shown in Figure 4.3) Overall, these three groups of experiments demonstrate that the colossal failure of unsupervisedmodels on real-world images involves both object- and scene-level dataset biases.

Revised: 4.5 Why do current unsupervised models fail on real-world datasets?

Original: From Figure 8 (c), we cansee that if the challenging real-world objects and images are ablated in both object- and scene-level,the segmentation performance of all unsupervised models achieves the same level with three syntheticdatasets as shown in Table 3.
Revised: As demonstrated in Sections 4.2/4.3/4.4, once the complexity factors are removed from the challenging real-world datasets, existing unsupervised models can perform as excellent as on the syntheticdatasets, as qualitatively illustrated in Figure 7. From this, we can safely conclude that the inductive

Original: Clearly, these three groups of experiments demonstrate that the colossalfailure of unsupervised models on real-world images involves both object- and scene-level datasetbiases. Existing unsupervised models cannot learn the challenging objectness  in real-world images. 
Revised: [Figure] biases designed in existing unsupervised models are far from able to match with and fully capture thetrue and complex objectness biases exhibited in real-world images. Nevertheless, from Figure 6, eachbaseline tends to favor different objectness biases.

Original:  AIR 
Revised: In particular, • AIR [22]: As a factor based model, AIR has a strong spatial-locality bias. Despite its poor seg-

Original: MONet
Revised: mentation performance across all datasets, there is a notable improvement when inter-object shape variation is ablated from real-world datasets (U / T+U / C+S+U / C+S+T+U). More convinc- ingly, even when all other three factors are ablated (C+S+T), it can be hardly improved.

Original: IODINE
Revised: These

Original: SlotAtt AIR
Revised: observations show that object shape variation is a significant factor for AIR.

Original:  MONet IODINE SlotAtt
Revised: • MONet [8]: MONet is more sensitive to color-related factors than shape-related factors.

Original: AIR
Revised: The abla-

Original: MONet IODINE
Revised: tions of object color gradient and inter-object color similarity significantly improve its performance,

Original: SlotAtt YCB-S+C+T
Revised: while ablations of object shape concavity and inter-object shape variation make little differences.

Original: ScanNet-S+C+T COCO-S+C+T AP PQ Precision Recall (a) Results for
Revised: For the two color-related factors, the scene-level one is more important than the object-level factor. From this, we can see that MONet has a strong dependency on color.

Original: Color Similarity. AIR
Revised: Similar colors tend to be

Original: MONet
Revised: grouped together while different colors are separated apart.

Original: IODINE
Revised: Furthermore, the ablation on object color gradient alleviates over-segmentation whereas the ablation on inter-object color similarity alleviates under-segmentation.

Original: SlotAtt
Revised: We conjecture that under-segmentation can be a more severe issue

Original: AIR MONet IODINE SlotAtt
Revised: for MONet on real-world datasets, leading to a larger sensitivity on the scene-level color factor.

Original: AIR MONet IODINE SlotAtt YCB-S+C+U
Revised: • IODINE [25]: IODINE also has a heavy dependency on both object- and scene-level color-relatedfactors.

Original: ScanNet-S+C+U COCO-S+C+U AP
Revised: However, different from MONet, the ablation on object color gradient brings better performance than inter-object color similarity.

Original: PQ Precision
Revised: We speculate it is because the regularization on shape latentalleviates under-segmentation by bias towards more regular shapes.

Original: Recall (b) Results AIR MONet IODINE
Revised: In this way, over-segmentationis the key issue, making the object color gradient a dominant factor.

Original:  SlotAtt
Revised: • SlotAtt

Original: AIR MONet IODINE SlotAtt 
Revised: [40]: The ablations on all four factors increase the performance of SlotAtt at different levels,among which object- and scene-level color-related factors are more significant.

Original: AIR MONet IODINE
Revised: We conjecture that

Original: SlotAtt [Figure] ScanNet-S+C+T+U COCO-S+C+T+U
Revised: it is because the feature embeddings used by Slot Attention module are learnt from both pixel

Original: AP PQ Precision Recall

Original: [Figure] ) Results for the Ablation of all object- and scene-level factors. [Figure]
Revised: colors and coordinates, which contributes to its sensitivity to both shape and color factors

Original:  Conclusions
Revised:  Conclusions

Original: In this paper, we systematically show thatthe existing unsupervised methods are practically impossibleto segment generic objects from single real-world images, and investigate the underlying factors thatincur the catastrophic failure.
Revised: We systematically show that existing unsupervised methods are practically impossible to segmentgeneric objects from single real-world images, and investigate the underlying factors that incur thecatastrophic failure.

Original: In particular, we perform a large-scale experimental study with fourrepresentative unsupervised methods, four evaluation metrics on six datasets.

Original: With the aid of ourcarefully designed four object- and scene-level complexity factors, we conduct extensive experimentson multiple groups of ablated real-world objects and images, and safely conclude that the challengingdistributions of boththe object- and scene-level biases in appearance and geometry of real-worlddatasets are the key factors that current unsupervised models fail to segment real objects.
Revised: With the aid of our carefully designed four object- and scene-level complexityfactors, we conduct extensive experiments on multiple groups of ablated real-world objects andimages, and safely conclude that the distributions of both object- and scene-level biases in appearanceand geometry of real-world datasets are particularly diverse and indiscriminative, such that currentunsupervised models cannot segment real objects.

Original: Oncethese factors are removed from the real-world datasets, existing models can perform as excellentas on the synthetic datasets, as qualitatively illustrated in Figure 9.

Original: Fundamentally, this means thatthe inductive biases designed in existing unsupervised models are far from able to match with andcapture the true objectness biases exhibited in real-world images. From this, we urge the communityto exploit more explicit and reliable objectness biasesin future unsupervised models, such as objectmotions or pretrained features from single-object-dominant datasets. Broader Impact: This paper does not introduce a new algorithm, but extensively investigates thelimit of existing unsupervised object segmentation models on real-world images.
Revised: Based on this finding, we suggest two maindirections for future study: 1) To exploit more discriminative objectness biases such as object motionswhich expressively describe the ownership of visual pixels as recently explored in [54, 12, 6]. 2) Toleverage pretrained features from single-object-dominant datasets which explicitly regard each imageas an object as recently studied in [10, 30], although such settings may not be purely unsupervised.

Original:  One limitation ofthis work is the lack of study about foreground-background segmentation. However, we believe thatour work can still have a positive impact for researchers to design more practically useful models.
Revised: Broader Impact: One limitation of this work is the lack of study about foreground-backgroundsegmentation, but our work can still have a positive impact for researchers to design more practicallyuseful models.

Original: Apotential negative impact could be using our findings to attackan existing model.
Revised: A potential negative impact could be using our findings to attack existing models.