Original: MOVE: Unsupervised Movable Object Segmentation and Detection
Revised: MOVE: Unsupervised Movable Object Segmentation and Detection

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: We introduce MOVE, a novel method to segment objects without any form ofsupervision.
Revised: We introduce MOVE, a novel method to segment objects without any form of supervision.

Original: MOVE exploits the fact that foreground objects can be shifted locallyrelative to their initial position and result in realistic (undistorted) new images.
Revised: MOVE exploits the fact that foreground objects can be shifted locally relative to their initial position and result in realistic (undistorted) new images.

Original: This property allows us to train a segmentation model that achieves state of theart (SotA) performance on several evaluation datasets for unsupervised salientobject detection and segmentation.
Revised: This property allows us to train a segmentation model on a dataset of images without annotation and to achieve state of the art (SotA) performance on several evaluation datasets for unsupervised salient object detection and segmentation.

Original: In unsupervised single object discovery, MOVEgives an average CorLoc improvement of 4.5% over the SotA and in unsupervisedclass-agnostic object detection it gives a relative AP improvement of 30% onaverage.
Revised: In unsupervised single object discovery, MOVE gives an average CorLoc improvement of 7.2% over the SotA, and in unsupervised class-agnostic object detection it gives a relative AP improvement of 53% on average.

Original: Our approach is built on top of self-supervised features (from DINO), aninpainting network (based on the Masked AutoEncoder) and adversarial trainingwith a projected discriminator
Revised: Our approach is built on top of selfsupervised features (e.g. from DINO or MAE), an inpainting network (based on the Masked AutoEncoder) and adversarial training with a projected discriminator.

Original:  Introduction
Revised:  Introduction

Original: Image segmentation and object detection are today mature and essential components in vision-basedsystems with applications in a wide range of fields including automotive [1], agriculture [2], andmedicine [3], just to name a few.
Revised: Image segmentation and object detection are today mature and essential components in vision-based systems with applications in a wide range of fields including automotive [1], agriculture [2], and medicine [3], just to name a few.

Original: A major challenge in building and deploying such components at scale is that they require costly andtime-consuming human annotation.
Revised: A major challenge in building and deploying such components at scale is that they require costly and time-consuming human annotation.

Original: This has motivated efforts in self-supervised learning (SSL) [4–6].
Revised: This has motivated efforts in self-supervised learning (SSL) [4–6].

Original: The aim of SSL is to learn general-purpose image representations from large unlabeled datasets thatcan be fine-tuned to different downstream tasks with small annotated datasets.
Revised: The aim of SSL is to learn general-purpose image representations from large unlabeled datasets that can be fine-tuned to different downstream tasks with small annotated datasets.

Original: While SSL methodshave also been fine-tuned for image segmentation since their inception, it is only with the recent stateof the art (SotA) methods, such as DINO [4] and Dense Contrastive Learning [6], that a clear andstrong link to object segmentation has been observed.
Revised: While SSL methods have also been fine-tuned for image segmentation since their inception it is only with the recent state of the art (SotA) methods, such as DINO [4] and Dense Contrastive Learning [6], that a clear and strong link to object segmentation has been observed.

Original: This has led to several methods for salientobject detection built on top of SSL features [7–12].
Revised: This has led to several methods for salient object detection built on top of SSL features [7–12].

Original: Most prior work based on SSL features defines some form of clustering by either using attentionmaps [7–9] or similarity graphs [10–12].
Revised: Most prior work based on SSL features defines some form of clustering by either using attention maps [7–9] or similarity graphs [10–12].

Original: In this work, we take a quite different direction.
Revised: In this work, we take a quite different direction.

Original: Ratherthan directly clustering the features, we train a network to map them to a segmentation mask.
Revised: Rather than directly clustering the features, we train a network to map them to a segmentation mask.

Original: Assupervision signal we use the movability of objects, i.e. , whether they can be locally shifted in arealistic manner.
Revised: As supervision signal we use the movability of objects, i.e. , whether they can be locally shifted in a realistic manner.

Original: We call our method MOVE.
Revised: We call our method MOVE.

Original: This property holds for objects in the foreground,as they occlude all other objects in the scene.
Revised: This property holds for objects in the foreground, as they occlude all other objects in the scene.

Original: This basic idea has already been exploited in priorwork with relative success [13–18].
Revised: This basic idea has already been exploited in prior work with relative success [13–18].

Original: Nonetheless, here we introduce a novel formulation basedon movability that yields a significant performance boost across several datasets for salient objectdetection.
Revised: Nonetheless, here we introduce a novel formulation based on movability that yields a significant performance boost across several datasets for salient object detection.

Original: In our approach, it is not necessary to move objects  to other images [13, 14] and thus we do nothave to handle the context mismatch.
Revised: In our approach, it is not necessary to move objects far from their initial location or to other images [14, 16] and thus we do not have to handle the context mismatch.

Original: It is also not necessary to employgenerative models  [15, 16],which can be challenging to train.
Revised: It is also not necessary to employ models to generate entire scenes [15, 19], which can be challenging to train.

Original: Our working principle exploits observations also made by [17, 18].[Figure]
Revised: Our working principle exploits observations also made by [17, 18].

Original: They point out that the correct mask maximizes the inpainting error both for the background and theforeground.
Revised: They point out that the correct mask maximizes the inpainting error both for the background and the foreground.

Original: However, rather than using the  reconstruction error as a supervision signal, we rely onthe detection of artifacts generated through shifting, which we find to provide a stronger guidance.
Revised: However, rather than using the [Figure] reconstruction error as a supervision signal, we rely on the detection of artifacts generated through shifting, which we find to provide a stronger guidance.

Original: We propose an adversarial training scheme, where we train a segmenter network against a discriminator.

Original: The segmenter predicts the segmentation mask  of the foreground object (Figure 1 (b)) from asingle image (Figure 1 (a)).
Revised: Suppose that, given a single image (Figure 1 (a)), we predict a segmentation mask (one of the 3 cases in Figure 1 (b)).

Original: Given the mask, we can separate the foreground from the background inthe input image, locally shift it and then compose it with the background to synthesize a new image(see Figure 1 (d)). However, due to the shift, regions in the background that were previously occludedby the foreground object become exposed. Thus, to fill in the missing pixels, we also employ aninpainter network before adding the foreground (see Figure 1 (c)).Finally, the discriminator aims todistinguish these composite shifted images from real ones.
Revised: With the mask we can remove the object and inpaint the background (Figure 1 (c)).

Original: In Figure 1, we consider several scenariosfor the predicted mask. Suppose that the segmentation mask is correct (Figure 1 (b) top) and wefeed the corresponding segmented background to the inpainter. Because the inpainter does not haveany information about the  foreground object, it can only complete the patterns from the background,i.e. , it can obtain a counterfactual image had the object not been there (Figure 1 (c) top).
Revised: Then, we can also extract the foreground object, randomly shift it locally, and paste it on top of the inpainted background (Figure 1 (d)). When the mask does not accurately follow the outline of a foreground object (e.g., as in the middle and bottom rows in Figure 1), we can see duplication artifacts (of the foreground or of the background).

Original: In thiscase, the discriminator should not be able to tell the difference between the composite shifted image(Figure 1 (d) top) and the real one (Figure 1 (a)). If the mask is instead incorrect (Figure 1 (b) middleand bottom), i.e. , it includes parts of the background or it does not include all of the foreground object,the composite shifted image will contain repeated and mismatching patterns (Figure 1 (d), (e) middleand bottom). In fact, in either case the inpainting model would complete the missing pixels in thebackground and predict foreground and/or background patterns that are in the incorrect foregroundtoo (Figure 1 (e) middle and bottom). To encourage the discriminator to focus on detecting suchrepeated patterns, we also feed composite images without any shift (Figure 1 (e) top) as examples of“real” images.
Revised: We exploit these artifacts as supervision signal to detect the correct segmentation mask.

Original: As inpainter we use a publicly available Masked AutoEncoder (MAE) [19] trained with an adversarialloss.
Revised: As inpainter we use a publicly available Masked AutoEncoder (MAE) [20] trained with an adversarial loss.

Original:  Our segmenter uses a pre-trained DINO ViT as backbone [4].
Revised: 1 Our segmenter uses a pre-trained SSL ViT as backbone (e.g., DINO [4] or the MAE encoder [20]).

Original: We then train a neural networkhead based on an upsampling Convolutional Neural Network (CNN).
Revised: We then train a neural network head based on an upsampling Convolutional Neural Network (CNN).

Original: Following [12], we also further[Figure] refine the segmenter by training a second segmentation network (SelfMask [12]) with supervisionfrom pseudo-masks generated by the trained segmenter.
Revised: Following [12], we also further refine the segmenter by training a second segmentation network (SelfMask [12]) with supervision from pseudo-masks generated by our trained segmenter.

Original: Even without these further refinements MOVE shows a remarkable performance on a wide rangeof datasets and tasks.
Revised: Even without these further refinements MOVE shows a remarkable performance on a wide range of datasets and tasks.

Original: In particular, in unsupervised single object discovery on VOC07, VOC12 and COCO20K it improves the SotA CorLoc between 3 . 2 % and 6 %, and in unsupervised class agnosticobject detection on COCOval2017 it improves the AP 50 by 5 % (a relative improvement of 41 %), the
Revised: In particular, in unsupervised single object discovery on VOC07, VOC12 and COCO20K it improves the SotA CorLoc between 6 . 1 % and 9 . 3 %, and in unsupervised class agnostic object detection on COCOval2017 it improves the AP 50 by 6 .

Original:  AP 75 by1 . 2  % (relative 29 %) and the AP by 1 . 8 % (relative 33 %).
Revised: 8 % (a relative improvement of 56 %), the AP 75 by 2 . 3 % (relative 55 %) and the AP by 2 . 7 % (relative 49 %).

Original:  Method
Revised:  Method

Original: Following the recent trend of methods for unsupervised object segmentation [7–12, 20], we build ourmethod on top of SSL features, and, in particular, DINO features [4].
Revised: Our objective is to train a segmenter to map a real image x ∈ R H × W × 3 , with H the height and W the width of the image, to a mask m ∈ R H × W of the foreground, such that we can synthesize a realistic image for any small shifts of the foreground. The mask allows to cut out the foreground from x and to move it arbitrarily by some δ ∈ R 2 shift (see Figure 2, top-left).

Revised: However, when the shifted foreground is copied back onto the background, missing pixels remain exposed.

Revised: Thus, we inpaint the background with a frozen pre-trained MAE 2 and obtain ˆ b (see Figure 3).

Revised: Moreover, there is a difference between the texture of ˆ b , which is generated from a neural network, and the texture of the cut out foreground from x , which is a real image.

Revised: To ensure more similarity between these two textures, we synthesize ˆ x δ by extracting the foreground from the autoencoding (AE) of the input image x shifted by δ , which we call ˇ x δ , and by pasting it onto the background ˆ b .

Revised: We enforce the realism of the synthesized images ˆ x δ by using adversarial training, i.e., by training the segmenter against a discriminator that distinguishes two sets of real (Figure 2, right hand side) from two sets of fake images (Figure 2 left hand side).

Revised: The synthetic real image ˆ x δ =0 is obtained by composing a zero-shifted foreground with the inpainted background; the second real image ˇ x is instead simply the AE of x .

Revised: The two fake images are obtained by composing a δ -shifted foreground with either the inpainted background ˆ b or ˇ x , and obtain ˆ x δ and ˜ x δ respectively.

Revised: We introduce all the above synthetic images so that the discriminator pays attention only to artifacts due to incorrect masks from the segmenter.

Original: Thus, as a backbone, we adoptthe Vision Transformer (ViT) architecture [21].
Revised: Ideally, the segmenter should generate masks such that the fake image ˆ x δ looks as realistic as ˇ x for any small δ . However, the discriminator might distinguish these two images because of the background inpainting artifacts and not because of the artifacts due to an incorrect segmentation (which are exposed by random shifts). To avoid this undesired behavior, we also introduce the real image ˆ x δ =0 . This image has no segmentation artifacts for any mask, but has the same background inpanting artifacts as the fake images (although there is no shift in ˆ x δ =0 , the background inpainting creates artifacts beyond the boundaries of the segmentation mask).

Original: Following the notation in [10], we split an imagex ∈ R
Revised: Finally, to guide the discriminator to detect repeated patterns (as those caused by incorrect masks, see Figure 1), we also add a fake image ˜ x δ , where the background has the original foreground. The segmenter is trained only through the backpropagation from ˆ x δ .

Original: H
Revised: The details of the segmentation network, the inpainting network and the adversarial training are explained in the following sections. 2.1 Segmenter Following the recent trend of methods for unsupervised object segmentation [7–12, 21], we build our method on top of SSL features, and, in particular, DINO [4] or MAE [20] features. Thus, as a backbone, we adopt the Vision Transformer (ViT) architecture [22].

Original:  × W × 3 in tiles of size P × P pixels, for a total of N = HW/P 2 tiles (and we assume that H and W are such that H/P and W/P are integers).
Revised: Following the notation in [10], we split an image x ∈ R H × W × 3 in tiles of size P × P pixels, for a total of N = HW/P 2 tiles (and we assume that H and W are such that H/P and W/P are integers).

Original: Each tile is then mapped through a trainablelinear layer to an embedding of size d and an additional CLS token is included in the input set (see Figure 2 left).
Revised: Each tile is then mapped through a trainable linear layer to an embedding of size d and an additional CLS token is included in the input set (see Figure 3 left).

Original: The segmenter network is a CNN that takes DINO features as input, upsamples them and then outputsa mask for the original input image.
Revised: The segmenter network is a CNN that takes SSL features as input (e.g., from a pre-trained DINO or MAE encoder), upsamples them and then outputs a mask for the original input image.

Original: The final output is generated by using a sigmoid to ensure thatthe mask values are always between 0 and 1 .
Revised: The final output is generated by using a sigmoid to ensure that the mask values are always between 0 and 1 .

Original: We also encourage the support of the predicted masktolie between a minimum and a maximum size by using
Revised: We also ensure a minimum size of the support of the predicted mask by using

Original: [Equation]
Revised: [Equation]

Original: [Figure] where n is the number of images in the training dataset, m ( i ) is the predicted mask from image x
Revised: (1) where n is the number of images in the training dataset, m ( i ) is the predicted mask from image x

Original: ( i ) ,p is a pixel location within the image domain, and θ min and θ max are thresholds for the minimumandmaximum mask coverage percentage respectively (in the range [0 , 1] , where 0 implies that the maskis empty and 1 implies that the mask covers the whole image).
Revised: ( i ) , p is a pixel location within the image domain, and θ min is a threshold for the minimum mask coverage percentage respectively (in the range [0 , 1] , where 0 implies that the mask is empty and 1 implies that the mask covers the whole image).

Original: Since masks should only take binaryvalues to clearly indicate a segment, we use a loss that encourages m ( i ) to take either 0 or 1 values
Revised: Since masks should only take binary values to clearly indicate a segment, we use a loss that encourages m ( i ) to take either 0 or 1 values

Original: [Equation]
Revised: [Equation]

Original: The main task of MOVE is to predict a segmentation mask that can be used to synthesize a realisticimage, where the foreground object is shifted  (see Figure 1 (e) top). Figure 3 shows the main stepsused to synthesize a composite image where the foreground is shifted.
Revised: The main task of MOVE is to predict a segmentation mask that can be used to synthesize a realistic image, where the foreground object is shifted on top of the inpainted background (see Figure 1 (e) top and Figure 2 top left).

Original: The most challenging partis dealing with the background. When the shifted foreground is copied back onto the background,missing pixels remain exposed. To avoid this issue, we inpaint the background with a pre-trainednetwork. As shown in Figure 2 on the right, we usea MAE [19], which is based on a ViT architectureand has been pre-trained in an adversarial fashion (as opposed to the standard training with an MSEloss) to output more realistic-looking details.
Revised: Figure 3 shows how we use the predicted high resolution mask for inpainting with MAE.

Original: Because MAE performs inpainting by masking orretaining entire tiles, it is necessary to also split the segmentation mask into a grid of tiles of P  × Ppixels and to map each tile to a single scalar between 0 and 1 .
Revised: Since MAE performs inpainting by masking or retaining entire patches of P ′ × P ′ pixels , it is necessary to also split the segmentation mask into a grid of tiles of P ′ × P ′ pixels and to map each tile to a single scalar between 0 and 1 .

Original: We do that by using a max poolingoperation within each tile and obtain a low-res mask ˆ m .
Revised: We do that by using a max pooling operation within each tile and obtain a low-res mask ˆ m , such that 1 − ˆ m does not contain any part of the predicted mask.

Original: Toavoid full masks ˆ m and to regularize thepredicted mask m , all three losses L min , Lmax , L bin are also computed on max pool downsampledmasks (at scales 1/2, 1/4, 1/8, and 1/16 of the original image resolution; for more details see thesupplementary material).
Revised: To regularize the predicted mask m , the mask losses L min , L bin are also computed on max pool ˆ m and average pool downsampled masks (at a scale 1 /P ′ of the original image resolution; for more details see the supplementary material).

Original: Then, we feed the entire set of image tiles to the MAE encoder and obtainembeddings ξ 1 , . . . , ξ N .
Revised: Then, we feed the entire set of image tiles to the MAE encoder and obtain embeddings ξ 1 , . . . , ξ N .

Original: Next, for j = 1 , . . .
Revised: Next, for j = 1 , . . .

Original: , N , we compute the convex combination between theembeddings ξ j and the learned MSK (masked) token from MAE by using the low res mask ˆ m asˆ ξ j = ˆ m [ j ] · ξ MSK + (1 − ˆ m [ j ]) · ξ j .
Revised: , N , we compute the convex combination between the embeddings ξ j and the learned MSK (masked) token from MAE by using the low res mask ˆ m as ˆ ξ j = ˆ m [ j ] · ξ MSK + (1 − ˆ m [ j ]) · ξ j .

Original: Finally, we feed the new embeddings ˆ ξ j in the MAE decoder andreassemble the output tiles back into the inpainted background image ˆ b (see Figure 2 bottom-right).
Revised: Finally, we feed the new embeddings ˆ ξ j in the MAE decoder and reassemble the output tiles back into the inpainted background image ˆ b (see Figure 3 bottom-right).

Original: Notice that we feed all the tiles as input to obtain a differentiable mapping that we can backpropagateon because we found that  the embeddings ξ j do not store significant information about their neighbors.
Revised: Notice that we feed all the tiles as input to obtain a differentiable mapping that we can backpropagate on. Interestingly, we found that when no tile is masked at the input of the MAE encoder, the embeddings ξ j do not store significant information about their neighbors (see the supplementary material).

Original: This is in contrast to the typical use of MAE, where only the subset of “visible” tiles is fed as inputto the encoder.
Revised: This is in contrast to the typical use of MAE, where only the subset of “visible” tiles is fed as input to the encoder.

Original: However, such tile selection operation would make the inpainting not differentiable.
Revised: However, such tile selection operation would make the inpainting not differentiable.

Original: The complete pipeline just described allows the synthesis of images, where the foreground isshifted relative to its original position. We obtain ˆ x δ [ p ] = m δ [ p ] x [ p + δ ] + (1 − m δ [ p ])ˆ b [ p ] , wherem δ
Revised: Figure 2 shows how we create the images used in the adversarial training. First, we mask the input image with the predicted mask and compose with the inpainted background image, obtaining

Original: [ p ] = m
Revised: [Equation]

Original:  [ p + δ ] , δ ∈ [ − ∆ W, ∆ W ] × [ − ∆ H, ∆ H ] is a 2D shift, with ∆ the maximum shift range(relative to the image size).
Revised: where m δ [ p ] = m [ p + δ ] , δ ∈ [ − ∆ W, ∆ W ] × [ − ∆ H, ∆ H ] is a 2D shift, with ∆ the maximum shift range (relative to the image size).

Original: When the mask m corresponds to an object, for a limited range of shifts,the composite image ˆ x δ still looks realistic. Thus, it is possible to train the segmenter adversarially toa discriminator D that aims at distinguishing the  composite image from a real one.
Revised: To make the inpainting artifacts in the no-shift composite image ˆ x δ =0 more comparable to those in the shifted composite image, we define the background inpainting region as the union between the predicted mask and its shifted version (see Figure 3).

Original: Despite the impressive performance of MAE, the inpainted backgrounds ˆ b have visible artifacts(typically blurriness) and, as a consequence, so do the composite images. This means that even whenthe predicted mask is correct, the discriminator could easily distinguish the original image from the composite one due to its unavoidable artifacts. Thus,we also produce composite images with δ = 0(no-shift) and add them to the pool of real images. To make the inpainting artifacts in the no-shiftcomposite image comparable to those in the shifted composite image, we define the backgroundinpainting region as the union between the predicted mask and its shifted version (see Figure 2).
Revised: Thus,

Original: Thus,ˆ m = maxpool P (1 − (1 − m ) ⊙ (1 − m δ )) . In this way, the discriminator will be forced to focuson differences between synthesized images other than the reconstruction and inpainting artifacts of MAE. Moreover, even when an empty mask is used in MAE, the reconstructed image will not beidentical to the input image.
Revised: [Equation]

Original: Thus, to ensure that both the foreground and the background undergothe same processing, we also feed the shiftedimage to MAE before extracting the foreground andcomposing it (see Figure 3 top). The discriminator also needs a reference of what real images withoutinpainting artifacts are. To this aim we also use images ˇ x , which are real images x autoencodedthrough MAE.
Revised: To improve the discriminator’s ability to focus on repeated patterns artifacts, we additionally create fake images with a predicted shifted foreground pasted on top of the autoencoded image, obtaining ˜ x δ = ˇ x δ ⊙ m δ + ˇ x ⊙ (1 − m δ ) .

Original: The adversarial loss for the discriminator can be written as
Revised: The adversarial loss for the discriminator can be written as

Original: [Equation]
Revised: [Equation]

Original: where IE denotes the expectation, the “real” images set is R =
Revised: where IE denotes the expectation, the “real” images set is R =

Original: { ˇ x ( [Equation] ,...,n (cid:83) { ˆ x [Equation]
Revised: { ˇ x ( i ) } i =1 ,...,n (cid:83) { ˆ x ( i ) δ

Original: [Equation] = [Equation] } i =1 ,...,n andthe synthetic images set is S
Revised: =0 } i =1 ,...,n and the synthetic images set is S

Original: =
Revised: =

Original: { ˆ x ( i ) δ } i =1 ,...,n  , with uniform random samples δ ∼ U 2 ([ − ∆ W, ∆ W ] ×[ − ∆ H, ∆ H ]) .
Revised: { ˆ x ( i ) δ } i =1 ,...,n (cid:83) { ˜ x ( i ) δ } i =1 ,...,n , with uniform random samples δ ∼ U 2 ([ − ∆ W, ∆ W ] × [ − ∆ H, ∆ H ]) .

Original: To speed up the convergence, we also use the projected discriminator method [22].
Revised: To speed up the convergence, we also use the projected discriminator method [23].

Original: For the segmenter, we use instead the standard loss 
Revised: For the segmenter, we use instead the standard loss computed on the composite shifted images

Original: [Equation]
Revised: [Equation]

Original: Finally, with λ min , λmax , λ bin nonnegative hyperparameters, our optimization is the adversarialminimization
Revised: Finally, with λ min , λ bin nonnegative hyperparameters, our optimization is the adversarial minimization [Equation] subject to

Original: [Equation]
Revised: [Equation]

Original:  Implementation
Revised:  Implementation

Original: We use a self-supervised DINO [4] ViT-S/8 transformer pre-trained on ImageNet [23] as a featureextractor in all our experiments.
Revised: Except for the ablation studies, in all our experiments we use a self-supervised DINO [4] ViT-S/ transformer pre-trained on ImageNet [24] as an SSL feature extractor.

Original: We take the output of the penultimate transformer block of DINO asthe feature tokens with P = 8 and feed them to the segmenter.
Revised: We take the output of the penultimate transformer block of DINO as the feature tokens with P = 8 and feed them to the segmenter.

Original: Our segmenter is a small upsamplingconvolutional neural network.
Revised: Our segmenter is a small upsampling convolutional neural network.

Original: It assembles the DINO features into a grid of size H/P × W/P andprocesses them with 3 upsampling blocks, so that the output matches the input image resolution.
Revised: It assembles the DINO features into a grid of size H/P × W/P and processes them with 3 upsampling blocks, so that the output matches the input image resolution.

Original: Each upsampling block first performs a 2 × 2 nearest upsampling, followed by a 3 × 3 convolutionallayer with padding, batch normalization  and a LeakyReLU activation function.
Revised: Each upsampling block first performs a 2 ×nearest upsampling, followed by a 3 × 3 convolutional layer with padding, batch normalization [25] and a LeakyReLU activation function (see the supplementary material for details).

Original: We add an additionalblock without upsampling and a linear projection to 1 channel, representing the mask.
Revised: We add an additional block without upsampling and a linear projection to 1 channel, representing the mask.

Original: Our inpaintingnetwork is a ViT-L/16 transformer pre-trained on ImageNet as a Masked Autoencoder (MAE)
Revised: Our inpainting network is a ViT-L/16 transformer pre-trained on ImageNet as a Masked Autoencoder [Table] [Table] [Table]

Original: [19]with an adversarial loss to increase the details of the reconstructed images.
Revised: [Table] [Table] (MAE) [20] with an adversarial loss to increase the details of the reconstructed images.

Original: For the discriminator weuse the Projected Discriminator [22] in its standard setting, but we exclude CutOut differentiableaugmentation.
Revised: For the discriminator we use the Projected Discriminator [23] in its standard setting [Table] but we only use color differentiable augmentation.

Original: For the training we use random resized crops of size 224 with a scale in range (0 . 9 , 1)and aspect ratio (3 / 4 , 4 / 3) .
Revised: For the training we use random resized crops of size 224 with a scale in range (0 . 9 , 1) and aspect ratio (3 / 4 , 4 / 3) .

Original: We set the minimum mask area θ min = 0 .
Revised: We set the minimum mask area θ min = 0 .

Original: 05 and maximum mask areaθ max = 0 . We set the minimumand maximum mask loss coefficients λ min = λ max = 100 andthe{+
 and we linearly ramp up the+} binarization loss coefficient λ bin = 5 . 
Revised: 05 , the minimum loss coefficient λ min = {+
 and we linearly ramp up the+} binarization loss coefficient λ bin from 0 to 12 . 5 over the first 2500 segmenter iterations.

Original: We use the shift range ∆ = 1 / 8 .
Revised: We use the shift range ∆ = 1 / 8 .

Original: We train the segmenterby alternatively minimizing the discriminator loss and the segmenter losses.
Revised: We train the segmenter by alternatively minimizing the discriminator loss and the segmenter losses.

Original: Both are trained witha learning rate of 0 . 0002 and an Adam  optimizer with betas = (0 , 0 . 99) for the discriminator and(0 . 9 , 0 . 95) for the segmenter.
Revised: Both are trained with a learning rate of 0 . 0002 and an Adam [26] optimizer with betas = (0 , 0 . 99) for the discriminator and (0 . 9 , 0 . 95) for the segmenter.

Original: We implemented our experiments in PyTorch [24].
Revised: We implemented our experiments in PyTorch [27].

Original: We train our modelfor 160 epochs with a batch size of 32 on a single NVIDIA GeForce 3090Ti GPU with 24GB ofmemory.
Revised: We train our model for 80 epochs with a batch size of 32 on a single NVIDIA GeForce 3090Ti GPU with 24GB of memory.

Original:  Experiments 
Revised:  Experiments 4.1 Unsupervised saliency segmentation

Original: 4.1 Unsupervised saliency segmentation

Original: Datasets.
Revised: Datasets.

Original: We train our main model using the train split of the DUTS dataset (DUTS-TR) [26],containing 10 , 553 images of scenes and objects of varying sizes and appearances.
Revised: We train our main model using the train split of the DUTS dataset (DUTS-TR) [29], containing 10 , 553 images of scenes and objects of varying sizes and appearances.

Original: We emphasize thatwe only use the images without the corresponding ground truth.
Revised: We emphasize that we only use the images without the corresponding ground truth.

Original: For comparison, we evaluate our

Original: [Table]

Original: LOST [10] + Bilateral .

Original: [Table]

Original: TokenCut [11] + Bilateral

Original: [Table]

Original: MOVE

Original: [Table]

Original: Bilateral

Original: [Table]

Original: SelfMask on pseudo

Original: [Table]

Original: [Table]

Original: on

Original: [Table]

Original: Bilateral

Original: [Table] on [Table] more examples and a see the approach on three saliency detection datasets: the test set of DUTS ( 5 , 019 images), DUT-OMRON[25] ( 5 , 168 images) and ECSSD [27] ( 1 , 000 images).
Revised: For comparison, we evaluate our approach on three saliency detection datasets: the test set of DUTS ( 5 , 019 images), DUT-OMRON [28] ( 5 , 168 images) and ECSSD [30] ( 1 , 000 images).

Original: We report three standard metrics: pixel maskaccuracy (Acc), intersection over union (IoU), max F β , where F β =
Revised: We report three standard metrics: pixel mask accuracy (Acc), intersection over union (IoU), max F β , where F β =

Original: (1+ β 2 ) Precision × Recall β 2 Precision + Recall for β = 0 .
Revised: (1+ β 2 ) Precision × Recall β 2 Precision + Recall for β = 0 .

Original: 3 ;the max F β is the score for the single optimal threshold on a whole dataset.
Revised: 3 ; the max F β is the score for the single optimal threshold on a whole dataset.

Revised: Additionally, we report the IoU on the test split [37] of CUB-200-2011 (CUB-Birds) [38] dataset.

Original: Evaluation.
Revised: Evaluation.

Original: We train our segmenter in an adversarial manner as specified in sections 2 and 3 andevaluate it on the test datasets.
Revised: We train our segmenter in an adversarial manner as specified in sections 2 and 3 and evaluate it on the test datasets.

Original: We compare with other methods in Table 1.
Revised: We compare with other methods in Table 1.

Original: Note that without anytype of post-processing of our predicted masks, we surpass all other methods by a significant margin.
Revised: Note that without any type of post-processing of our predicted masks, we surpass all other methods by a significant margin.

Original: We also follow [11, 12] and further refine our masks with a bilateral solver [34].
Revised: We also follow [11, 12] and further refine our masks with a bilateral solver [39]. [Table]

Original: Since the bilateralsolver only marginally improves or even decreases the quality of our segmentation, we conclude thatour predicted masks are already very accurate.
Revised: Since the bilateral solver only marginally improves or even decreases the quality of our segmentation, we conclude that our predicted masks are already very accurate.

Original: Using the bilateral solver might also inadvertentlydiscard correct, but fragmented segmentations, as we show in the supplementary material.
Revised: Using the bilateral solver might also inadvertently discard correct, but fragmented segmentations, as we show in the supplementary material.

Original: Next,we extract the predicted unsupervised masks from the DUTS-TR dataset and use them as pseudoground-truth to train a class-agnostic segmenter.
Revised: Next, we extract the predicted unsupervised masks from the DUTS-TR dataset and use them as pseudo ground-truth to train a class-agnostic segmenter.

Original: We use the same architecture (a MaskFormer [35])and training scheme as SelfMask [12].
Revised: We use the same architecture (a MaskFormer [40]) and training scheme as SelfMask [12].

Original: We then evaluate again on the saliency prediction datasets.
Revised: We then evaluate again on the saliency prediction datasets.

Original: Without additional pre-processing our method surpasses or is on par with the SotA across all metrics[Table] [Figure] and datasets.
Revised: Without additional pre-processing our method surpasses or is on par with the SotA across all metrics and datasets.

Original: While additional processing with the bilateral solver seems to benefit SelfMask [12], itmostly hurts the performance of our method.
Revised: While additional processing with the bilateral solver seems to benefit SelfMask [12], it mostly hurts the performance of our method.

Original: Figure 4 shows qualitative results of our method.
Revised: Figure 4 shows qualitative results of our method.

Revised: Finally, we evaluate our method on the test set of CUB-Birds dataset.

Revised: Additionally, we train our model on the train split of CUBBirds dataset and run the same evaluation.

Revised: We present the comparison with other methods in Table 2 and show that we achieve SotA performance.

Original: 4.2 Single-object discovery
Revised: 4.2 Single-object discovery

Original: Datasets.
Revised: Datasets.

Original: We evaluate our trained model (see section 4.1) on 3 typical single-object discoverybenchmarks: the train split of COCO20K [38, 39] and the trainval splits of VOC07 [36] and VOC12[37].
Revised: We evaluate our trained model (see section 4.1) on 3 typical single-object discovery benchmarks: the train split of COCO20K [41, 42] and the trainval splits of VOC07 [43] and VOC [44].

Original: Following [10, 11, 39, 45–49], we report the Correct Localization metric (CorLoc), i.e. , thepercentage of images, where the IoU > 0 .
Revised: Following [10, 11, 42, 45–49], we report the Correct Localization metric (CorLoc), i.e. , the percentage of images, where the IoU > 0 .

Original: 5 of a predicted single bounding box with at least one ofthe ground truth ones.
Revised: 5 of a predicted single bounding box with at least one of the ground truth ones.

Original: Evaluation.
Revised: Evaluation.

Original: Since our method tends to produce a single segmentation mask for multiple objects in thescene, we separate the objects by detecting connected components via OpenCV [50].
Revised: Since our method tends to produce a single segmentation mask for multiple objects in the scene, we separate the objects by detecting connected components via OpenCV [50].

Original: We then convertthe separate masks to bounding boxes and choose the biggest one as our prediction for the givenimage.
Revised: We then convert the separate masks to bounding boxes and choose the biggest one as our prediction for the given image.

Original: In Table 2, we compare MOVE with other unsupervised methods and we show that just byusing processed masks from our method we achieve SotA results on all three datasets, outperformingeven methods that used their bounding boxes to train a Class Agnostic Detector (CAD).
Revised: In Table 3, we compare MOVE with other unsupervised methods and we show that just by using processed masks from our method we achieve SotA results on all three datasets, outperforming even methods that used their bounding boxes to train a Class Agnostic Detector (CAD).

Original: We showqualitative results for object detection in Figure 5.
Revised: We show qualitative results for object detection in Figure 5.

Original: We also follow the practice of [10, 11] and useour predicted bounding boxes as pseudo-ground truth for training the CAD on each of the evaluationdatasets.
Revised: We also follow the practice of [10, 11] and use our predicted bounding boxes as pseudo-ground truth for training the CAD on each of the evaluation datasets.

Original: To train the detector, we use either the largest or all the bounding boxes  that we obtainedfrom the connected components analysis and after filtering those that have an area smaller than 1%of the image.
Revised: To train the detector, we use either the largest or all the bounding boxes ( Multi ) that we obtained from the connected components analysis and after filtering those that have an area smaller than 1% of the image.

Original: For the evaluation we take the bounding box with the highest prediction confidence, asdone in [10, 11].
Revised: For the evaluation we take the bounding box with the highest prediction confidence, as done in [10, 11].

Original: We use the exact same architecture and training scheme as our competitors for a faircomparison.
Revised: We use the exact same architecture and training scheme as our competitors for a fair comparison.

Original: Training with a single bounding boxslightly improves the  performance of our method,while training with multiple ones gives it a significant additional boost.
Revised: Training with a single bounding box improves the [Table] [Table] performance of our method, while training with multiple ones gives it a significant additional boost.

Original: [Table]

Original: [Table]
Revised: [Table]

Original: Unsupervised class-agnostic object detection.
Revised: Unsupervised class-agnostic object detection.

Original: We evaluate our unsupervised object detectionmodel trained on COCO20K with CAD post-training and compare it with SotA on unsupervised class-agnostic object detection. 
Revised: We evaluate our unsupervised object detection model trained on COCO20K with CAD post-training and compare it with SotA on unsupervised class-agnostic object detection. In Table 4, we evaluate MOVE on

Original: We evaluate on COCOval2017 and report Average Precision (AP and Average Recall (AR), as in [8].
Revised: COCOval2017 and report Average Precision (AP) and Average Recall (AR), as in [8].

Original: MOVE yields a remarkable relative improvement over the AP SotA of 30 % on average
Revised: MOVE yields a remarkable relative improvement over the AP SotA of 50 % on average.

Original: 4.3 Ablation study
Revised: 4.3 Ablation study

Original: We perform ablation experiments on the validation split (500 images) of HKU-IS [52] to validate therelative importance of the components of our segmentation approach.
Revised: We perform ablation experiments on the validation split (500 images) of HKU-IS [57] to validate the relative importance of the components of our segmentation approach.

Original: For the ablation we train eachmodel for 100 epochs on DUTS-TR.
Revised: For the ablation we train each model for 80 epochs on DUTS-TR.

Original: We report the IoU in Table 4.
Revised: We report the IoU in Table 5.

Original: Our baseline model trained with 3different seeds gives a mean IoU 0 .
Revised: Our baseline model trained withdifferent seeds gives a mean IoU 0 .

Original: 782 with std = 0 .
Revised: 818 with std = 0 .

Original: Thus we only report results for a single runin all experiments. 
Revised: Thus we only report results for a single run in all experiments. [Table]

Original: Mask losses.
Revised: Mask losses.

Original: We validate the importance of the mask losses: minimum maskarea, maximum maskarea, binarization and losses on downsampled max-pooled masks. We find that the minimum area ormaximum area losses do not affect the performance much.
Revised: We validate the importance of the mask losses: minimum mask

Original: The binarization loss is instead essential: Removing it results in mostly uniform small-valued masks, for which no inpainting is done. Thesame holds for the mask loss terms on themax-pooled downsampled versions of the masks.
Revised: We find that the minimum area loss is necessary for our method to work, otherwise there is no incentive to produce anything other than empty masks. Removing the binarization loss or mask losses on the downsampled masks makes the masks noisier, which negatively affects the results.

Original: Shift range.
Revised: Shift range.

Original: We evaluate different ranges of the random shift δ .
Revised: We evaluate different ranges of the random shift δ .

Original: A small range ∆ = 1 / 16 seems tobe too challenging for the discriminator to detect inconsistencies.
Revised: A small range ∆ = 1 / 16 makes it more challenging for the discriminator to detect inconsistencies at the border of objects.

Original: Larger shifts may cause objects togo out of the image boundaries ( ∆ = 3 / 16 , 4 / 16 ) and thus reduce the feedback at the object boundaryto the segmenter.
Revised: Larger shifts may cause objects to go out of the image boundaries ( ∆ = 3 / 16 , 4 / 16 ) and thus reduce the feedback at the object boundary to the segmenter.

Original:  There is no incentive to produce any meaningful masks for ∆ = 0 . We also trya boundary-aware (BA) shift, where we limit the shifts so that when the object is partly out of theimage domain, we do not expose the clipped edge(s).
Revised: For ∆ = 0 (no-shift) the only possible discriminator inputs are composed images without a shift as fake and autoencoded images as real. There is no incentive to produce any meaningful masks in this case.

Original: In this case the segmenter sometimes outputsartifacts on the borders to limit the shift range and this hurts the performance.

Original: Discriminator inputs.
Revised: Discriminator inputs.

Original: In our baseline model, we feed both composed images with no-shift andreal images autoencoded with MAE as real samples  for the discriminator training.
Revised: In our baseline model, we feed both composed images with no-shift and real images autoencoded with MAE as real samples and composed images with a shift and autoencoded images with copy-pasting of a predicted masked object as fake samples for the discriminator training.

Original: We test the casedisc.
Revised: We test the case DISC .

Original: real , where we feed only real images without autoencoding.
Revised: REAL x + COMP . W / O SHIFT , where we feed to the discriminator real images without autoencoding.

Original: In this case, the discriminatorcan detect  artifacts of MAE and has no incentive to look for image inconsistencies resulting fromshifts.
Revised: In this case, the discriminator can detect the artifacts of MAE instead of focusing on inconsistencies resulting from an incorrect mask.

Original: In disc.
Revised: In DISC .

Original: mae we only feed the autoencoded images as real.
Revised: REAL x ae we only feed the autoencoded images as real.

Original: Here, the discriminator can focuson the mismatch from the inpainting artifacts and encourages the segmenter to output empty masks,where no inpainting is done.
Revised: Here, the discriminator can focus on the mismatch from the inpainting artifacts and encourages the segmenter to output empty masks, where no inpainting is done.

Original: If we only feed the composite non-shifted images ( disc.
Revised: If we only feed the composite non-shifted images ( DISC .

Original: w/ shift ), theartifacts resulting from an incorrect masks cannot be fixed because there is no reference of what thereal images look like. 
Revised: REAL COMP W / O SHIFT ) , the artifacts resulting from an incorrect masks cannot be fixed, because there is no reference of what the real images look like. In DISC . FAKE INPUTS :

Revised: COMPOSED we only feed the composed image as fake to the discriminator and omit the real image with a copy-pasted predicted masked object, which slightly degrades the performance.

Original: Non-differentiable inpainter.
Revised: Non-differentiable inpainter.

Original: We evaluate the use of hard thresholded  masks as input to thebackground inpainter.
Revised: We evaluate the use of hard thresholded downsampled masks as input to the background inpainter.

Original: In this case the only feedback for the masks comes from the composition ofthe images.
Revised: In this case the only feedback for the masks comes from the composition of the images.

Original: We find it to be insufficient for the segmenter to learn any meaningful masks.
Revised: We find it to be insufficient for the segmenter to learn any meaningful masks.

Original: Inpainter model.
Revised: Inpainter model.

Original: Finally, we substitute the MAE trained with a GAN loss with a MAE that wastrained only to reconstruct missing patches with a Mean Squared Error (MSE) loss.
Revised: We substitute the MAE trained with a GAN loss with a MAE that was trained only to reconstruct missing patches with a Mean Squared Error (MSE) loss.

Original: In this case, boththe inpainted regions and  the autoencoded images are of poor quality.
Revised: Since this model was trained to only reconstruct the missing patches and not the entire image, we construct the inpainted background by composing the inpainted part with the real image: ˆ m up = upsample 16 ( ˆ m ) ; ˆ b := x ⊙ (1 − ˆ m up ) + ˆ b ⊙ ˆ m up . Consequently, we do not use autoencoding when creating the discriminator inputs. We find this model to perform competitively. Feature extractor.

Original: Even in this case  the masks areaccurate.
Revised: We train the model using the features provided by MAE encoder instead of a separate DINO model. In this case we adapted the segmenter architecture and added one more upsampling block, since MAE takes patches of size P = 16 (instead DINO has P = 8 ). We find that with these features we are able to train a competitive segmenter. ImageNet100 dataset.

Original: It suggests that a realistic inpainter is important, but not crucial, to our approach, since weare feeding the autoencoded and composed no-shifted images as real samples to the discriminator.
Revised: We train our model on the ImageNet100 dataset [58], with 131 , 689 images from 100 randomly selected ImageNet [24] classes. Since this dataset is much bigger than DUTS-TR, we adapt our segmenter by adding an additional convolutional layer in each upsampling block (see section 3) and train the model for 8 epochs. The results are comparable to the DUTS-TR dataset.

Original:  Prior Work
Revised:  Prior Work

Original: In the past decade, research on object segmentation and detection has seen remarkable progresswhen full supervision is available [53–55].
Revised: In the past decade, research on object segmentation and detection has seen remarkable progress when full supervision is available [59–61].

Original: To limit the cost of annotation several methods exploreddifferent forms of weak supervision [56, 57] or ways to avoid labeling altogether [13, 14, 58, 59].
Revised: To limit the cost of annotation several methods explored different forms of weak supervision [62, 63] or ways to avoid labeling altogether [13, 14, 64, 65].

Original: MOVE falls in the latter category.
Revised: MOVE falls in the latter category.

Original: Therefore, we focus our review of prior work on unsupervisedmethods for object segmentation and the related task of object detection.
Revised: Therefore, we focus our review of prior work on unsupervised methods for object segmentation and the related task of object detection.

Original: Unsupervised Object Detection and Category Discovery.
Revised: Unsupervised Object Detection and Category Discovery.

Original: Unsupervised object detection and category discovery are extremely challenging tasks that have recently seen a surge of efforts [51, 60, 61]thanks to the capabilities of modern deep learning models.
Revised: Unsupervised object detection and category discovery are extremely challenging tasks that have recently seen a surge of efforts [56, 66, 67] thanks to the capabilities of modern deep learning models.

Original: Recently, features based on deep learninghave shown significant progress in object detection [51], even with just some noisy (unsupervised)guidance [40].
Revised: Recently, features based on deep learning have shown significant progress in object detection [56], even with just some noisy (unsupervised) guidance [51].

Original: More in general, one limitation of unsupervised object detection is that it only providesa coarse localization of the objects.
Revised: More in general, one limitation of unsupervised object detection is that it only provides a coarse localization of the objects.

Original: As we have shown with MOVE, it is possible to obtain muchmore information without supervision.
Revised: As we have shown with MOVE, it is possible to obtain much more information without supervision.

Original: Unsupervised Object Segmentation.
Revised: Unsupervised Object Segmentation.

Original: Object segmentation can be formulated as a pixel-wise imagepartitioning task [58, 59, 62, 63] or through the generation of layered models from which a segmenteris trained as a byproduct [8, 31, 64, 65].
Revised: Object segmentation can be formulated as a pixel-wise image partitioning task [64, 65, 68, 69] or through the generation of layered models from which a segmenter is trained as a byproduct [8, 34, 70, 71].

Original: The use of SSL features has spawned several methods withsignificant performance on real datasets, which we discuss in the following paragraphs.
Revised: The use of SSL features has spawned several methods with significant performance on real datasets, which we discuss in the following paragraphs.

Original: SSL-Based Methods.
Revised: SSL-Based Methods.

Original: Due to the success of SSL methods and the emergence of segmentationcapabilities, several recent methods for unsupervised object segmentation have been built on top of
Revised: Due to the success of SSL methods and the emergence of segmentation capabilities, several recent methods for unsupervised object segmentation have been built on top of

Original: SSL features.
Revised: SSL features.

Original: In particular, SelfMask [12] proposes a clustering approach that can use multiple SSLfeatures and evaluates all possible combinations of DINO [4], SwAV [66] and MOCOV2 [67].
Revised: In particular, SelfMask [12] proposes a clustering approach that can use multiple SSL features and evaluates all possible combinations of DINO [4], SwAV [72] and MOCOV2 [73].

Original: Theyfind that combining features from all three SSL methods yields the best results for segmentation.
Revised: They find that combining features from all three SSL methods yields the best results for segmentation.

Original: FreeSOLO [8] instead finds that DenseCL features [6] work best.
Revised: FreeSOLO [8] instead finds that DenseCL features [6] work best.

Original: More in general, some methods usea weak (unsupervised) guidance and losses robust to the coarse pseudo-labels [8], but the majority isbased on directly clustering SSL features [7, 9–11, 20, 68].
Revised: More in general, some methods use a weak (unsupervised) guidance and losses robust to the coarse pseudo-labels [8], but the majority is based on directly clustering SSL features [7, 9–11, 21, 74].

Original: In contrast to these methods, we showthat movability can provide a robust supervision signal.
Revised: In contrast to these methods, we show that movability can provide a robust supervision signal.

Original: Generative Methods.
Revised: Generative Methods.

Original: A wide range of methods also exploits generative models to create layeredimage representations [15, 17, 69–73].
Revised: A wide range of methods also exploits generative models to create layered image representations [15, 17, 19, 75–78].

Original: A general scheme is to train a network to generate a background, a foreground and its mask.
Revised: A general scheme is to train a network to generate a background, a foreground and its mask.

Original: These components can then be combined to generate an imageand then, in a second stage, one can train a segmenter that learns to map a synthetic image to itscorresponding foreground mask.
Revised: These components can then be combined to generate an image and then, in a second stage, one can train a segmenter that learns to map a synthetic image to its corresponding foreground mask.

Original: In alternative, the segmenter could be built during the training ofthe generative model as a byproduct.
Revised: In alternative, the segmenter could be built during the training of the generative model as a byproduct.

Original: Some methods rely on the assumption that a dataset of onlybackgrounds is available [14, 74].
Revised: Some methods rely on the assumption that a dataset of only backgrounds is available [14, 79].

Original: The use of shifts to define segments has also been used before[13, 15, 16].
Revised: The use of shifts to define segments has also been used before [13, 15, 16].

Original: However, MOVE does not require the training of a generative model, which can be achallenge on its own.
Revised: However, MOVE does not require the training of a generative model, which can be a challenge on its own.

Original:  Limitations and Societal Impact
Revised:  Limitations and Societal Impact

Original: Limitations.
Revised: Limitations.

Original: As mentioned in the introduction, movability alone may not suffice in identifyingan object unambiguously.
Revised: As mentioned in the introduction, movability alone may not suffice in identifying an object unambiguously.

Original: In fact, the method can segment any combination of multiple objects.
Revised: In fact, the method can segment any combination of multiple objects.

Original: To address this we use a post-processing algorithm to find connected components, but there is noguarantee that all objects have been segmented.
Revised: To address this we use a post-processing algorithm to find connected components, but there is no guarantee that all objects have been segmented.

Original: Another issue is that shifts would not expose artifactswhen the background is uniform ( e.g. , looking at the sky, underwater, with macrophotography).
Revised: Another issue is that shifts would not expose artifacts when the background is uniform ( e.g. , looking at the sky, underwater, with macrophotography).

Original: Societal Impact.
Revised: Societal Impact.

Original: An important aspect that is relevant to unsupervised learning methods in general isthe potential to become biased if the training datasets are unbalanced.
Revised: An important aspect that is relevant to unsupervised learning methods in general is the potential to become biased if the training datasets are unbalanced.

Original: This clearly has a potentiallynegative impact on the segmentation of categories that are underrepresented and thus this work shouldbe integrated with mechanisms to take the dataset imbalance into account.
Revised: This clearly has a potentially negative impact on the segmentation of categories that are underrepresented and thus this work should be integrated with mechanisms to take the dataset imbalance into account.

Original:  Conclusions
Revised:  Conclusions

Original: We have introduced MOVE, a novel self-supervised method for object segmentation that exploits thesynthesis of images where objects are randomly shifted.
Revised: We have introduced MOVE, a novel self-supervised method for object segmentation that exploits the synthesis of images where objects are randomly shifted.

Original: MOVE improves the SotA in object saliency segmentation, unsupervised single object discovery and unsupervised class agnostic object detection by significant margins.
Revised: MOVE improves the SotA in object saliency segmentation, unsupervised single object discovery, and unsupervised class agnostic object detection by significant margins.

Original: Our ablations show that movability is a strong supervision signal that can be robustly exploited as a pseudo-task for self-supervised object segmentation.
Revised: Our ablations show that movability is a strong supervision signal that can be robustly exploited as a pseudo-task for self-supervised object segmentation.

Original: We believe that our approach can be further scaled by exploring different architectures and larger datasets 
Revised: We believe that our approach can be further scaled by exploring different architectures and larger datasets Checklist

Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Revised: [Yes] (b) Did you describe the limitations of your work?

Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Revised: [Yes] 2.

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Revised: [N/A] 3.

Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [No] We use publicly available datasets.

Revised: We plan on sharing the code and the instructions at a later time.

Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Revised: [Yes] We trained our baseline model with 3 different seeds and reported mean and std IoU on the ablation dataset.

Revised: We do not include it for all the experiments due to time and cost constraints.

Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Revised: [Yes] 4.

Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes] (b) Did you mention the license of the assets?

Revised: [No] (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [No] (d) Did you discuss whether

Revised: and how consent was obtained from people whose data you’re using/curating?

Revised: [No] We use datasets publicly available for research (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Revised: [No] 5.

Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Revised: [N/A]