Original: MOVE: Unsupervised Movable Object Segmentation and Detection
Revised: MOVE: Unsupervised Movable Object Segmentation and Detection

Original: Abstract
Revised: Abstract

Original: We introduce MOVE, a novel method to segment objects without any form of supervision.
Revised: We introduce MOVE, a novel method to segment objects without any form of supervision.

Original: MOVE exploits the fact that foreground objects can be shifted locally relative to their initial position and result in realistic (undistorted) new images.
Revised: MOVE exploits the fact that foreground objects can be shifted locally relative to their initial position and result in realistic (undistorted) new images.

Original: This property allows us to train a segmentation model on a dataset of images without annotation and to achieve state of the art (SotA) performance on several evaluation datasets for unsupervised salient object detection and segmentation.
Revised: This property allows us to train a segmentation model on a dataset of images without annotation and to achieve state of the art (SotA) performance on several evaluation datasets for unsupervised salient object detection and segmentation.

Original: In unsupervised single object discovery, MOVE gives an average CorLoc improvement of 7.2% over the SotA, and in unsupervised class-agnostic object detection it gives a relative AP improvement of 53% on average.
Revised: In unsupervised single object discovery, MOVE gives an average CorLoc improvement of 7.2% over the SotA, and in unsupervised class-agnostic object detection it gives a relative AP improvement of 53% on average.

Original: Our approach is built on top of self-supervised features (e.g. from DINO or MAE), an inpainting network (based on the Masked AutoEncoder) and adversarial training.
Revised: Our approach is built on top of self-supervised features (e.g. from DINO or MAE), an inpainting network (based on the Masked AutoEncoder) and adversarial training.

Original:  Introduction
Revised:  Introduction

Original: Image segmentation and object detection are today mature and essential components in vision-based systems with applications in a wide range of fields including automotive [1], agriculture [2], and medicine [3], just to name a few.
Revised: Image segmentation and object detection are today mature and essential components in vision-based systems with applications in a wide range of fields including automotive [1], agriculture [2], and medicine [3], just to name a few.

Original: A major challenge in building and deploying such components at scale is that they require costly and time-consuming human annotation.
Revised: A major challenge in building and deploying such components at scale is that they require costly and time-consuming human annotation.

Original: This has motivated efforts in self-supervised learning (SSL) [4–6].
Revised: This has motivated efforts in self-supervised learning (SSL) [4–6].

Original: The aim of SSL is to learn general-purpose image representations from large unlabeled datasets that can be fine-tuned to different downstream tasks with small annotated datasets.
Revised: The aim of SSL is to learn general-purpose image representations from large unlabeled datasets that can be fine-tuned to different downstream tasks with small annotated datasets.

Original: While SSL methods have also been fine-tuned for image segmentation since their inception, it is only with the recent state of the art (SotA) methods, such as DINO [4] and
Revised: While SSL methods have also been fine-tuned for image segmentation since their inception, it is only with the recent state of the art (SotA) methods, such as DINO [4] and

Original: Dense Contrastive Learning [6], that a clear and strong link to object segmentation has been observed.
Revised: Dense Contrastive Learning [6], that a clear and strong link to object segmentation has been observed.

Original: This has led to several methods for salient object detection built on top of SSL features [7–12].
Revised: This has led to several methods for salient object detection built on top of SSL features [7–12].

Original: Most prior work based on SSL features defines some form of clustering by either using attention maps [7–9] or similarity graphs [10–12].
Revised: Most prior work based on SSL features defines some form of clustering by either using attention maps [7–9] or similarity graphs [10–12].

Original: In this work, we take a quite different direction.
Revised: In this work, we take a quite different direction.

Original: Rather than directly clustering the features, we train a network to map them to a segmentation mask.
Revised: Rather than directly clustering the features, we train a network to map them to a segmentation mask.

Original: As supervision signal we use the movability of objects, i.e. , whether they can be locally shifted in a realistic manner.
Revised: As supervision signal we use the movability of objects, i.e. , whether they can be locally shifted in a realistic manner.

Original: We call our method MOVE.
Revised: We call our method MOVE.

Original: This property holds for objects in the foreground, as they occlude all other objects in the scene.
Revised: This property holds for objects in the foreground, as they occlude all other objects in the scene.

Original: This basic idea has already been exploited in prior work with relative success [13–18].
Revised: This basic idea has already been exploited in prior work with relative success [13–19].

Original: Nonetheless, here we introduce a novel formulation based on movability that yields a significant performance boost across several datasets for salient object detection.
Revised: Nonetheless, here we introduce a novel formulation based on movability that yields a significant performance boost across several datasets for salient object detection.

Original: In our approach, it is not necessary to move objects far from their initial location or to other images [14, 16] and thus we do not have to handle the context mismatch.
Revised: In our approach, it is not necessary to move objects far from their initial location or to other images [14, 16] and thus we do not have to handle the context mismatch.

Original: It is also not necessary to employ models to generate entire scenes [15, 19], which can be challenging to train.
Revised: It is also not necessary to employ models to generate entire scenes [15, 20], which can be challenging to train.

Original: Our working principle exploits observations also made by [17, 18].
Revised: Our working principle exploits observations also made by [17–19].

Original: They point out that the correct mask maximizes the inpainting error both for the background and the foreground.
Revised: They point out that the correct mask maximizes the inpainting error both for the background and the foreground.

Original: However, rather than using the
Revised: However, rather than using the

Original: [Figure]
Revised: [Figure]

Original: reconstruction error as a supervision signal, we rely on the detection of artifacts generated through shifting, which we find to provide a stronger guidance.
Revised: reconstruction error as a supervision signal, we rely on the detection of artifacts generated through shifting, which we find to provide a stronger guidance.

Original: Suppose that, given a single image (Figure 1 (a)), we predict a segmentation mask (one of the 3 cases in Figure 1 (b)).
Revised: Suppose that, given a single image (Figure 1 (a)), we predict a segmentation mask (one of the 3 cases in Figure 1 (b)).

Original: With the mask we can remove the object and inpaint the background (Figure 1 (c)).
Revised: With the mask we can remove the object and inpaint the background (Figure 1 (c)).

Original: Then, we can also extract the foreground object, randomly shift it locally, and paste it on top of the inpainted background (Figure 1 (d)).
Revised: Then, we can also extract the foreground object, randomly shift it locally, and paste it on top of the inpainted background (Figure 1 (d)).

Original: When the mask does not accurately follow the outline of a foreground object (e.g., as in the middle and bottom rows in Figure 1), we can see duplication artifacts (of the foreground or of the background).
Revised: When the mask does not accurately follow the outline of a foreground object (e.g., as in the middle and bottom rows in Figure 1), we can see duplication artifacts (of the foreground or of the background).

Original: We exploit these artifacts as supervision signal to detect the correct segmentation mask.
Revised: We exploit these artifacts as supervision signal to detect the correct segmentation mask.

Original: As inpainter we use a publicly available
Revised: As inpainter we use a publicly available

Original: Masked AutoEncoder (MAE) [20] trained with an adversarial loss.
Revised: Masked AutoEncoder (MAE) [21] trained with an adversarial loss.

Original: 1 Our segmenter uses a pre-trained
Revised: 1 Our segmenter uses a pre-trained

Original: SSL ViT as backbone (e.g., DINO [4] or the MAE encoder [20]).
Revised: SSL ViT as backbone (e.g., DINO [4] or the MAE encoder [21]).

Original: We then train a neural network head based on an upsampling Convolutional Neural Network (CNN).
Revised: We then train a neural network head based on an upsampling Convolutional Neural Network (CNN).

Original: Following [12], we also further refine the segmenter by training a second segmentation network (SelfMask [12]) with supervision from pseudo-masks generated by our trained segmenter.
Revised: Following [12], we also further refine the segmenter by training a second segmentation network (SelfMask [12]) with supervision from pseudo-masks generated by our trained segmenter.

Original: Even without these further refinements MOVE shows a remarkable performance on a wide range of datasets and tasks.
Revised: Even without these further refinements MOVE shows a remarkable performance on a wide range of datasets and tasks.

Original: In particular, in unsupervised single object discovery on VOC07, VOC12 and COCO20K it improves the SotA CorLoc between 6 . 1 % and 9 .
Revised: In particular, in unsupervised single object discovery on VOC07, VOC12 and COCO20K it improves the SotA CorLoc between 6 . 1 % and 9 .

Original: 3 %, and in unsupervised class agnostic object detection on COCOval2017 it improves the AP 50 by 6 .
Revised: 3 %, and in unsupervised class agnostic object detection on COCOval2017 it improves the AP 50 by 6 .

Original: 8 % (a relative improvement of 56 %), the AP 75 by 2 .
Revised: 8 % (a relative improvement of 56 %), the AP 75 by 2 .

Original: 3 % (relative 55 %) and the AP by 2 .
Revised: 3 % (relative 55 %) and the AP by 2 .

Original: 7 % (relative 49 %).
Revised: 7 % (relative 49 %).

Original:  Method
Revised:  Method

Original: Our objective is to train a segmenter to map a real image x ∈ R H
Revised: Our objective is to train a segmenter to map a real image x ∈ R H

Original: × W × 3 , with H the height and W the width of the image, to a mask m ∈
Revised: × W × 3 , with H the height and W the width of the image, to a mask m ∈

Original: R H × W of the foreground, such that we can synthesize a realistic image for any small shifts of the foreground.
Revised: R H × W of the foreground, such that we can synthesize a realistic image for any small shifts of the foreground.

Original: The mask allows to cut out the foreground from x and to move it arbitrarily by some δ ∈ R 2 shift (see Figure 2, top-left).
Revised: The mask allows to cut out the foreground from x and to move it arbitrarily by some δ ∈ R 2 shift (see Figure 2, top-left).

Original: However, when the shifted foreground is copied back onto the background, missing pixels remain exposed.
Revised: However, when the shifted foreground is copied back onto the background, missing pixels remain exposed.

Original: Thus, we inpaint the background with a frozen pre-trained MAE 2 and obtain ˆ b (see Figure 3).
Revised: Thus, we inpaint the background with a frozen pre-trained MAE 2 and obtain ˆ b (see Figure 3).

Original: Moreover, there is a difference between the texture of ˆ b , which is generated from a neural network, and the texture of the cut out foreground from x , which is a real image.
Revised: Moreover, there is a difference between the texture of ˆ b , which is generated from a neural network, and the texture of the cut out foreground from x , which is a real image.

Original: To ensure more similarity between these two textures, we synthesize ˆ x δ by extracting the foreground from the autoencoding (AE) of the input image x shifted by δ , which we call ˇ x δ , and by pasting it onto the background ˆ b .
Revised: To ensure more similarity between these two textures, we synthesize ˆ x δ by extracting the foreground from the autoencoding (AE) of the input image x shifted by δ , which we call ˇ x δ , and by pasting it onto the background ˆ b .

Original: We enforce the realism of the synthesized images ˆ x δ by using adversarial training, i.e., by training the segmenter against a discriminator that distinguishes two sets of real (Figure 2, right hand side) from two sets of fake images (Figure 2 left hand side).
Revised: We enforce the realism of the synthesized images ˆ x δ by using adversarial training, i.e., by training the segmenter against a discriminator that distinguishes two sets of real (Figure 2, right hand side) from two sets of fake images (Figure 2 left hand side).

Original: The synthetic real image ˆ x δ =0 is obtained by composing a zero-shifted foreground with the inpainted background; the second real image ˇ x is instead simply the AE of x .
Revised: The synthetic real image ˆ x δ =0 is obtained by composing a zero-shifted foreground with the inpainted background; the second real image ˇ x is instead simply the AE of x .

Original: The two fake images are obtained by composing a δ -shifted foreground with either the inpainted background ˆ b or ˇ x , and obtain ˆ x δ and ˜ x δ respectively.
Revised: The two fake images are obtained by composing a δ -shifted foreground with either the inpainted background ˆ b or ˇ x , and obtain ˆ x δ and ˜ x δ respectively.

Original: We introduce all the above synthetic images so that the discriminator pays attention only to artifacts due to incorrect masks from the segmenter.
Revised: We introduce all the above synthetic images so that the discriminator pays attention only to artifacts due to incorrect masks from the segmenter.

Original: Ideally, the segmenter should generate masks such that the fake image ˆ x δ looks as realistic as ˇ x for any small δ .
Revised: Ideally, the segmenter should generate masks such that the fake image ˆ x δ looks as realistic as ˇ x for any small δ .

Original: However, the discriminator might distinguish these two images because of the background inpainting artifacts and not because of the artifacts due to an incorrect segmentation (which are exposed by random shifts).
Revised: However, the discriminator might distinguish these two images because of the background inpainting artifacts and not because of the artifacts due to an incorrect segmentation (which are exposed by random shifts).

Original: To avoid this undesired behavior, we also introduce the real image ˆ x δ =0 .
Revised: To avoid this undesired behavior, we also introduce the real image ˆ x δ =0 .

Original: This image has no segmentation artifacts for any mask, but has the same background inpanting artifacts as the fake images (although there is no shift in ˆ x δ =0 , the background inpainting creates artifacts beyond the boundaries of the segmentation mask).
Revised: This image has no segmentation artifacts for any mask, but has the same background inpanting artifacts as the fake images (although there is no shift in ˆ x δ =0 , the background inpainting creates artifacts beyond the boundaries of the segmentation mask).

Original: Finally, to guide the discriminator to detect repeated patterns (as those caused by incorrect masks, see Figure 1), we also add a fake image ˜ x δ , where the background has the original foreground.
Revised: Finally, to guide the discriminator to detect repeated patterns (as those caused by incorrect masks, see Figure 1), we also add a fake image ˜ x δ , where the background has the original foreground.

Original: The segmenter is trained only through the backpropagation from ˆ x δ .
Revised: The segmenter is trained only through the backpropagation from ˆ x δ .

Original: The details of the segmentation network, the inpainting network and the adversarial training are explained in the following sections.
Revised: The details of the segmentation network, the inpainting network and the adversarial training are explained in the following sections.

Original: 2.1 Segmenter
Revised: 2.1 Segmenter

Original: Following the recent trend of methods for unsupervised object segmentation [7–12, 21], we build our method on top of SSL features, and, in particular, DINO [4] or MAE [20] features.
Revised: Following the recent trend of methods for unsupervised object segmentation [7–12, 22], we build our method on top of SSL features, and, in particular, DINO [4] or MAE [21] features.

Original: Thus, as a backbone, we adopt the Vision Transformer (ViT) architecture [22].
Revised: Thus, as a backbone, we adopt the Vision Transformer (ViT) architecture [23].

Original: Following the notation in [10], we split an image x ∈ R H × W × 3 in tiles of size P × P pixels, for a total of N = HW/P 2 tiles (and we assume that H and W are such that H/P and W/P are integers).
Revised: Following the notation in [10], we split an image x ∈ R H × W × 3 in tiles of size P × P pixels, for a total of N = HW/P 2 tiles (and we assume that H and W are such that H/P and W/P are integers).

Original: Each tile is then mapped through a trainable linear layer to an embedding of size d and an additional CLS token is included in the input set (see Figure 3 left).
Revised: Each tile is then mapped through a trainable linear layer to an embedding of size d and an additional CLS token is included in the input set (see Figure 3 left).

Original: The segmenter network is a CNN that takes SSL features as input (e.g., from a pre-trained DINO or MAE encoder), upsamples them and then outputs a mask for the original input image.
Revised: The segmenter network is a CNN that takes SSL features as input (e.g., from a pre-trained DINO or MAE encoder), upsamples them and then outputs a mask for the original input image.

Original: The final output is generated by using a sigmoid to ensure that the mask values are always between 0 and 1 .
Revised: The final output is generated by using a sigmoid to ensure that the mask values are always between 0 and 1 .

Original: We also ensure a minimum size of the support of the predicted mask by using
Revised: We also ensure a minimum size of the support of the predicted mask by using

Original: [Equation]
Revised: [Equation]

Original: (1) where n is the number of images in the training dataset, m ( i ) is the predicted mask from image x
Revised: (1) where n is the number of images in the training dataset, m ( i ) is the predicted mask from image x

Original: ( i ) , p is a pixel location within the image domain, and θ min is a threshold for the minimum mask coverage percentage respectively (in the range [0 , 1] , where 0 implies that the mask is empty and 1 implies that the mask covers the whole image).
Revised: ( i ) , p is a pixel location within the image domain, and θ min is a threshold for the minimum mask coverage percentage respectively (in the range [0 , 1] , where 0 implies that the mask is empty and 1 implies that the mask covers the whole image).

Original: Since masks should only take binary values to clearly indicate a segment, we use a loss that encourages m ( i ) to take either 0 or 1 values
Revised: Since masks should only take binary values to clearly indicate a segment, we use a loss that encourages m ( i ) to take either 0 or 1 values

Original: [Equation]
Revised: [Equation]

Original: 2.2 Differentiable inpainting
Revised: 2.2 Differentiable inpainting

Original: The main task of MOVE is to predict a segmentation mask that can be used to synthesize a realistic image, where the foreground object is shifted on top of the inpainted background (see Figure 1 (e) top and Figure 2 top left).
Revised: The main task of MOVE is to predict a segmentation mask that can be used to synthesize a realistic image, where the foreground object is shifted on top of the inpainted background (see Figure 1 (e) top and Figure 2 top left).

Original: Figure 3 shows how we use the predicted high resolution mask for inpainting with MAE.
Revised: Figure 3 shows how we use the predicted high resolution mask for inpainting with MAE.

Original: Since MAE performs inpainting by masking or retaining entire patches of P ′ ×
Revised: Since MAE performs inpainting by masking or retaining entire patches of P ′ ×

Original: P ′ pixels
Revised: P ′ pixels

Original: , it is necessary to also split the segmentation mask into a grid of tiles of P ′ ×
Revised: , it is necessary to also split the segmentation mask into a grid of tiles of P ′ ×

Original: P ′ pixels and to map each tile to a single scalar between 0 and 1 .
Revised: P ′ pixels and to map each tile to a single scalar between 0 and 1 .

Original: We do that by using a max pooling operation within each tile and obtain a low-res mask ˆ m , such that 1 − ˆ m does not contain any part of the predicted mask.
Revised: We do that by using a max pooling operation within each tile and obtain a low-res mask ˆ m , such that 1 − ˆ m does not contain any part of the predicted mask.

Original: To regularize the predicted mask m , the mask losses L min , L bin are also computed on max pool ˆ m and average pool downsampled masks (at a scale 1 /P ′ of the original image resolution; for more details see the supplementary material).
Revised: To regularize the predicted mask m , the mask losses L min , L bin are also computed on max pool ˆ m and average pool downsampled masks (at a scale 1 /P ′ of the original image resolution; for more details see the supplementary material).

Original: Then, we feed the entire set of image tiles to the
Revised: Then, we feed the entire set of image tiles to the

Original: MAE encoder and obtain embeddings ξ 1 , . .
Revised: MAE encoder and obtain embeddings ξ 1 , . .

Original: . , ξ N .
Revised: . , ξ N .

Original: Next, for j = 1 , . . .
Revised: Next, for j = 1 , . . .

Original: , N , we compute the convex combination between the embeddings ξ j and the learned MSK (masked) token from MAE by using the low res mask ˆ m as ˆ ξ j = ˆ m [ j ] · ξ MSK + (1 − ˆ m [ j ]) · ξ j .
Revised: , N , we compute the convex combination between the embeddings ξ j and the learned MSK (masked) token from MAE by using the low res mask ˆ m as ˆ ξ j = ˆ m [ j ] · ξ MSK + (1 − ˆ m [ j ]) · ξ j .

Original: Finally, we feed the new embeddings ˆ ξ j in the MAE decoder and reassemble the output tiles back into the inpainted background image ˆ b (see
Revised: Finally, we feed the new embeddings ˆ ξ j in the MAE decoder and reassemble the output tiles back into the inpainted background image ˆ b (see

Original: Figure 3 bottom-right).
Revised: Figure 3 bottom-right).

Original: Notice that we feed all the tiles as input to obtain a differentiable mapping that we can backpropagate on.
Revised: Notice that we feed all the tiles as input to obtain a differentiable mapping that we can backpropagate on.

Original: Interestingly, we found that when no tile is masked at the input of the MAE encoder, the embeddings ξ j do not store significant information about their neighbors (see the supplementary material).
Revised: Interestingly, we found that when no tile is masked at the input of the MAE encoder, the embeddings ξ j do not store significant information about their neighbors (see the supplementary material).

Original: This is in contrast to the typical use of MAE, where only the subset of “visible” tiles is fed as input to the encoder.
Revised: This is in contrast to the typical use of MAE, where only the subset of “visible” tiles is fed as input to the encoder.

Original: However, such tile selection operation would make the inpainting not differentiable.
Revised: However, such tile selection operation would make the inpainting not differentiable.

Original: 2.3 Adversarial training
Revised: 2.3 Adversarial training

Original: Figure 2 shows how we create the images used in the adversarial training.
Revised: Figure 2 shows how we create the images used in the adversarial training.

Original: First, we mask the input image with the predicted mask and compose with the inpainted background image, obtaining
Revised: First, we mask the input image with the predicted mask and compose with the inpainted background image, obtaining

Original: [Equation]
Revised: [Equation]

Original: where m δ [ p ] = m
Revised: where m δ [ p ] = m

Original: [ p + δ ] , δ ∈ [ − ∆ W, ∆ W ] × [ − ∆ H, ∆ H ] is a 2D shift, with ∆ the maximum shift range (relative to the image size).
Revised: [ p + δ ] , δ ∈ [ − ∆ W, ∆ W ] × [ − ∆ H, ∆ H ] is a 2D shift, with ∆ the maximum shift range (relative to the image size).

Original: To make the inpainting artifacts in the no-shift composite image ˆ x δ =0 more comparable to those in the shifted composite image, we define the background inpainting region as the union between the predicted mask and its shifted version (see Figure 3).
Revised: To make the inpainting artifacts in the no-shift composite image ˆ x δ =0 more comparable to those in the shifted composite image, we define the background inpainting region as the union between the predicted mask and its shifted version (see Figure 3).

Original: Thus,
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: To improve the discriminator’s ability to focus on repeated patterns artifacts, we additionally create fake images with a predicted shifted foreground pasted on top of the autoencoded image, obtaining ˜ x δ =
Revised: To improve the discriminator’s ability to focus on repeated patterns artifacts, we additionally create fake images with a predicted shifted foreground pasted on top of the autoencoded image, obtaining ˜ x δ =

Original: ˇ x δ ⊙ m δ + ˇ x ⊙ (1 − m δ ) .
Revised: ˇ x δ ⊙ m δ + ˇ x ⊙ (1 − m δ ) .

Original: The adversarial loss for the discriminator can be written as
Revised: The adversarial loss for the discriminator can be written as

Original: [Equation]
Revised: [Equation]

Original: where samples for “real” images x R are the set { ˇ x ( i ) } i =1 ,...,n (cid:83) { ˆ x ( i ) δ =0 } i =1 ,...,n and samples for synthetic images x S are the set { ˆ x ( i ) δ } i =1 ,...,n (cid:83) { ˜ x ( i ) δ } i =1 ,...,n , with uniform random samples δ ∼
Revised: where samples for “real” images x R are the set { ˇ x ( i ) } i =1 ,...,n (cid:83) { ˆ x ( i ) δ =0 } i =1 ,...,n and samples for synthetic images x S are the set { ˆ x ( i ) δ } i =1 ,...,n (cid:83) { ˜ x ( i ) δ } i =1 ,...,n , with uniform random samples δ ∼

Original: U 2 ([ − ∆ W, ∆ W ] × [ − ∆ H, ∆ H ]) and IE denotes the expectation.
Revised: U 2 ([ − ∆ W, ∆ W ] × [ − ∆ H, ∆ H ]) and IE denotes the expectation.

Original: To speed up the convergence, we also use the projected discriminator method [23].
Revised: To speed up the convergence, we also use the projected discriminator method [24].

Original: For the segmenter, we use instead the standard loss computed on the composite shifted images
Revised: For the segmenter, we use instead the standard loss computed on the composite shifted images

Original: [Equation]
Revised: [Equation]

Original: Finally, with λ min , λ bin nonnegative hyperparameters, our optimization is the adversarial minimization
Revised: Finally, with λ min , λ bin nonnegative hyperparameters, our optimization is the adversarial minimization

Original: [Equation]
Revised: [Equation]

Original: subjectto
Revised: subject

Original: [Equation]
Revised: [Equation]

Original:  Implementation
Revised:  Implementation

Original: Except for the ablation studies, in all our experiments we use a self-supervised DINO [4] ViT-S/ transformer pre-trained on ImageNet [24] as an SSL feature extractor.
Revised: Except for the ablation studies, in all our experiments we use a self-supervised DINO [4] ViT-S/ transformer pre-trained on ImageNet [25] as an SSL feature extractor.

Original: We take the output of the penultimate transformer block of DINO as the feature tokens with P = 8 and feed them to the segmenter.
Revised: We take the output of the penultimate transformer block of DINO as the feature tokens with P = 8 and feed them to the segmenter.

Original: Our segmenter is a small upsampling convolutional neural network.
Revised: Our segmenter is a small upsampling convolutional neural network.

Original: It assembles the
Revised: It assembles the

Original: DINO features into a grid of size H/P × W/P and processes them with 3 upsampling blocks, so that the output matches the input image resolution.
Revised: DINO features into a grid of size H/P × W/P and processes them with 3 upsampling blocks, so that the output matches the input image resolution.

Original: Each upsampling block first performs a 2 ×nearest upsampling, followed by a 3 × 3 convolutional layer with padding, batch normalization [25] and a LeakyReLU activation function (see the supplementary material for details).
Revised: Each upsampling block first performs a 2 ×nearest upsampling, followed by a 3 × 3 convolutional layer with padding, batch normalization [26] and a LeakyReLU activation function (see the supplementary material for details).

Original: We add an additional block without upsampling and a linear projection to 1 channel, representing the mask.
Revised: We add an additional block without upsampling and a linear projection to 1 channel, representing the mask.

Original: Our inpainting network is a ViT-L/16 transformer pre-trained on ImageNet as a Masked Autoencoder
Revised: Our inpainting network is a ViT-L/16 transformer pre-trained on ImageNet as a Masked Autoencoder

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: (MAE)
Revised: (MAE)

Original: [20] with an adversarial loss to increase the details of the reconstructed images.
Revised: [21] with an adversarial loss to increase the details of the reconstructed images.

Original: For the discriminator we use the Projected Discriminator [23] in its standard setting
Revised: For the discriminator we use the Projected Discriminator [24] in its standard setting

Original: [Table]
Revised: [Table]

Original: but we only use color differentiable augmentation.
Revised: but we only use color differentiable augmentation.

Original: For the training we use random resized crops of size 224 with a scale in range (0 . 9 , 1) and aspect ratio (3 / 4 , 4 / 3) .
Revised: For the training we use random resized crops of size 224 with a scale in range (0 . 9 , 1) and aspect ratio (3 / 4 , 4 / 3) .

Original: We set the minimum mask area θ min = 0 .
Revised: We set the minimum mask area θ min = 0 .

Original: 05 , the minimum loss coefficient λ min = 

Revised: 05 , the minimum loss coefficient λ min =

Original: and we linearly ramp up the binarization loss coefficient λ bin from 0 to 12 .
Revised: and we linearly ramp up the binarization loss coefficient λ bin from 0 to 12 .

Original: 5 over the first 2500 segmenter iterations.
Revised: 5 over the first 2500 segmenter iterations.

Original: We use the shift range ∆ = 1 / 8 .
Revised: We use the shift range ∆ = 1 / 8 .

Original: We train the segmenter by alternatively minimizing the discriminator loss and the segmenter losses.
Revised: We train the segmenter by alternatively minimizing the discriminator loss and the segmenter losses.

Original: Both are trained with a learning rate of 0 . 0002 and an Adam [26] optimizer with betas = (0 , 0 . 99) for the discriminator and (0 . 9 , 0 . 95) for the segmenter.
Revised: Both are trained with a learning rate of 0 . 0002 and an Adam [27] optimizer with betas = (0 , 0 . 99) for the discriminator and (0 . 9 , 0 . 95) for the segmenter.

Original: We implemented our experiments in PyTorch [27].
Revised: We implemented our experiments in PyTorch [28].

Original: We train our model for 80 epochs with a batch size of 32 on a single NVIDIA GeForce 3090Ti GPU with 24GB of memory.
Revised: We train our model for 80 epochs with a batch size of 32 on a single NVIDIA GeForce 3090Ti GPU with 24GB of memory.

Original:  Experiments 4.1 Unsupervised saliency segmentation
Revised:  Experiments 4.1 Unsupervised saliency segmentation

Original: Datasets.
Revised: Datasets.

Original: We train our main model using the train split of the DUTS dataset (DUTS-TR) [29], containing 10 , 553 images of scenes and objects of varying sizes and appearances.
Revised: We train our main model using the train split of the DUTS dataset (DUTS-TR) [30], containing 10 , 553 images of scenes and objects of varying sizes and appearances.

Original: We emphasize that we only use the images without the corresponding ground truth.
Revised: We emphasize that we only use the images without the corresponding ground truth.

Original: For comparison, we evaluate our approach on three saliency detection datasets: the test set of DUTS ( 5 , 019 images), DUT-OMRON [28] ( 5 , 168 images) and ECSSD [30] ( 1 , 000 images).
Revised: For comparison, we evaluate our approach on three saliency detection datasets: the test set of DUTS ( 5 , 019 images), DUT-OMRON [29] ( 5 , 168 images) and ECSSD [31] ( 1 , 000 images).

Original: We report three standard metrics: pixel mask accuracy (Acc), intersection over union (IoU), max F β , where F β =
Revised: We report three standard metrics: pixel mask accuracy (Acc), intersection over union (IoU), max F β , where F β =

Original: (1+ β 2 ) Precision × Recall β 2 Precision + Recall for β = 0 .
Revised: (1+ β 2 ) Precision × Recall β 2 Precision + Recall for β = 0 .

Original: 3 ; the max F β is the score for the single optimal threshold on a whole dataset.
Revised: 3 ; the max F β is the score for the single optimal threshold on a whole dataset.

Original: Additionally, we report the IoU on the test split [37] of CUB-200-2011 (CUB-Birds) [38] dataset.
Revised: Additionally, we report the IoU on the test split [38] of CUB-200-2011 (CUB-Birds) [39] dataset.

Original: Evaluation.
Revised: Evaluation.

Original: We train our segmenter in an adversarial manner as specified in sections 2 and 3 and evaluate it on the test datasets.
Revised: We train our segmenter in an adversarial manner as specified in sections 2 and 3 and evaluate it on the test datasets.

Original: We compare with other methods in Table 1.
Revised: We compare with other methods in Table 1.

Original: Note that without any type of post-processing of our predicted masks, we surpass all other methods by a significant margin.
Revised: Note that without any type of post-processing of our predicted masks, we surpass all other methods by a significant margin.

Original: We also follow [11, 12] and further refine our masks with a bilateral solver [39].
Revised: We also follow [11, 12] and further refine our masks with a bilateral solver [40].

Original: [Table]
Revised: [Table]

Original: Since the bilateral solver only marginally improves or even decreases the quality of our segmentation, we conclude that our predicted masks are already very accurate.
Revised: Since the bilateral solver only marginally improves or even decreases the quality of our segmentation, we conclude that our predicted masks are already very accurate.

Original: Using the bilateral solver might also inadvertently discard correct, but fragmented segmentations, as we show in the supplementary material.
Revised: Using the bilateral solver might also inadvertently discard correct, but fragmented segmentations, as we show in the supplementary material.

Original: Next, we extract the predicted unsupervised masks from the
Revised: Next, we extract the predicted unsupervised masks from the

Original: DUTS-TR dataset and use them as pseudo ground-truth to train a class-agnostic segmenter.
Revised: DUTS-TR dataset and use them as pseudo ground-truth to train a class-agnostic segmenter.

Original: We use the same architecture (a MaskFormer [40]) and training scheme as SelfMask [12].
Revised: We use the same architecture (a MaskFormer [41]) and training scheme as SelfMask [12].

Original: We then evaluate again on the saliency prediction datasets.
Revised: We then evaluate again on the saliency prediction datasets.

Original: Without additional pre-processing our method surpasses or is on par with the SotA across all metrics and datasets.
Revised: Without additional pre-processing our method surpasses or is on par with the SotA across all metrics and datasets.

Original: While additional processing with the bilateral solver seems to benefit
Revised: While additional processing with the bilateral solver seems to benefit

Original: SelfMask [12], it mostly hurts the performance of our method.
Revised: SelfMask [12], it mostly hurts the performance of our method.

Original: Figure 4 shows qualitative results of our method.
Revised: Figure 4 shows qualitative results of our method.

Original: Finally, we evaluate our method on the test set of CUB-Birds dataset.
Revised: Finally, we evaluate our method on the test set of CUB-Birds dataset.

Original: Additionally, we train our model on the train split of CUBBirds dataset and run the same evaluation.
Revised: Additionally, we train our model on the train split of CUBBirds dataset and run the same evaluation.

Original: We present the comparison with other methods in Table 2 and show that we achieve SotA performance.
Revised: We present the comparison with other methods in Table 2 and show that we achieve SotA performance.

Original: 4.2 Single-object discovery
Revised: 4.2 Single-object discovery

Original: Datasets.
Revised: Datasets.

Original: We evaluate our trained model (see section 4.1) on 3 typical single-object discovery benchmarks: the train split of COCO20K [41, 42] and the trainval splits of VOC07 [43] and VOC [44].
Revised: We evaluate our trained model (see section 4.1) on 3 typical single-object discovery benchmarks: the train split of COCO20K [42, 43] and the trainval splits of VOC07 [44] and VOC [45].

Original: Following [10, 11, 42, 45–49], we report the Correct Localization metric (CorLoc), i.e. , the percentage of images, where the IoU > 0 .
Revised: Following [10, 11, 43, 46–50], we report the Correct Localization metric (CorLoc), i.e. , the percentage of images, where the IoU > 0 .

Original: 5 of a predicted single bounding box with at least one of the ground truth ones.
Revised: 5 of a predicted single bounding box with at least one of the ground truth ones.

Original: Evaluation.
Revised: Evaluation.

Original: Since our method tends to produce a single segmentation mask for multiple objects in the scene, we separate the objects by detecting connected components via OpenCV [50].
Revised: Since our method tends to produce a single segmentation mask for multiple objects in the scene, we separate the objects by detecting connected components via OpenCV [51].

Original: We then convert the separate masks to bounding boxes and choose the biggest one as our prediction for the given image.
Revised: We then convert the separate masks to bounding boxes and choose the biggest one as our prediction for the given image.

Original: In Table 3, we compare MOVE with other unsupervised methods and we show that just by using processed masks from our method we achieve SotA results on all three datasets, outperforming even methods that used their bounding boxes to train a Class Agnostic Detector (CAD).
Revised: In Table 3, we compare MOVE with other unsupervised methods and we show that just by using processed masks from our method we achieve SotA results on all three datasets, outperforming even methods that used their bounding boxes to train a Class Agnostic Detector (CAD).

Original: We show qualitative results for object detection in Figure 5.
Revised: We show qualitative results for object detection in Figure 5.

Original: We also follow the practice of [10, 11] and use our predicted bounding boxes as pseudo-ground truth for training the CAD on each of the evaluation datasets.
Revised: We also follow the practice of [10, 11] and use our predicted bounding boxes as pseudo-ground truth for training the CAD on each of the evaluation datasets.

Original: To train the detector, we use either the largest or all the bounding boxes ( Multi ) that we obtained from the connected components analysis and after filtering those that have an area smaller than 1% of the image.
Revised: To train the detector, we use either the largest or all the bounding boxes ( Multi ) that we obtained from the connected components analysis and after filtering those that have an area smaller than 1% of the image.

Original: For the evaluation we take the bounding box with the highest prediction confidence, as done in [10, 11].
Revised: For the evaluation we take the bounding box with the highest prediction confidence, as done in [10, 11].

Original: We use the exact same architecture and training scheme as our competitors for a fair comparison.
Revised: We use the exact same architecture and training scheme as our competitors for a fair comparison.

Original: Training with a single bounding box improves the
Revised: Training with a single bounding box improves the

Original: [Table]
Revised: [Table]

Original: [Table]

Original: performance of our method, while training with multiple ones gives it a significant additional boost.
Revised: performance of our method, while training with multiple ones gives it a significant additional boost.

Original: [Table]
Revised: [Table]

Original: Unsupervised class-agnostic object detection.
Revised: Unsupervised class-agnostic object detection.

Original: We evaluate our unsupervised object detection model trained on COCO20K with
Revised: We evaluate our unsupervised object detection model trained on COCO20K with

Original: CAD post-training and compare it with SotA on unsupervised class-agnostic object detection.
Revised: CAD post-training and compare it with SotA on unsupervised class-agnostic object detection.

Original: In Table 4, we evaluate MOVE on
Revised: In Table 4, we evaluate MOVE on

Original: COCOval2017 and report Average Precision (AP) and Average Recall (AR), as in [8].
Revised: COCOval2017 and report Average Precision (AP) and Average Recall (AR), as in [8].

Original: MOVE yields a remarkable relative improvement over the AP SotA of 50 % on average.
Revised: MOVE yields a remarkable relative improvement over the AP SotA of 50 % on average.

Original: 4.3 Ablation study
Revised: 4.3 Ablation study

Original: We perform ablation experiments on the validation split (500 images) of HKU-IS [57] to validate the relative importance of the components of our segmentation approach.
Revised: We perform ablation experiments on the validation split (500 images) of HKU-IS [58] to validate the relative importance of the components of our segmentation approach.

Original: For the ablation we train each model for 80 epochs on DUTS-TR.
Revised: For the ablation we train each model for 80 epochs on DUTS-TR.

Original: We report the IoU in Table 5.
Revised: We report the IoU in Table 5.

Original: Our baseline model trained withdifferent seeds gives a mean IoU 0 .
Revised: Our baseline model trained withdifferent seeds gives a mean IoU 0 .

Original: 818 with std = 0 .
Revised: 818 with std = 0 .

Original: Thus we only report results for a single run in all experiments.
Revised: Thus we only report results for a single run in all experiments.

Original: [Table]
Revised: [Table]

Original: Mask losses.
Revised: Mask losses.

Original: We validate the importance of the mask losses: minimum mask
Revised: We validate the importance of the mask losses: minimum mask

Original: We find that the minimum area loss is necessary for our method to work, otherwise there is no incentive to produce anything other than empty masks.
Revised: We find that the minimum area loss is necessary for our method to work, otherwise there is no incentive to produce anything other than empty masks.

Original: Removing the binarization loss or mask losses on the downsampled masks makes the masks noisier, which negatively affects the results.
Revised: Removing the binarization loss or mask losses on the downsampled masks makes the masks noisier, which negatively affects the results.

Original: Shift range.
Revised: Shift range.

Original: We evaluate different ranges of the random shift δ .
Revised: We evaluate different ranges of the random shift δ .

Original: A small range ∆ = 1 / 16 makes it more challenging for the discriminator to detect inconsistencies at the border of objects.
Revised: A small range ∆ = 1 / 16 makes it more challenging for the discriminator to detect inconsistencies at the border of objects.

Original: Larger shifts may cause objects to go out of the image boundaries ( ∆ = 3 / 16 , 4 / 16 ) and thus reduce the feedback at the object boundary to the segmenter.
Revised: Larger shifts may cause objects to go out of the image boundaries ( ∆ = 3 / 16 , 4 / 16 ) and thus reduce the feedback at the object boundary to the segmenter.

Original: For ∆ = 0 (no-shift) the only possible discriminator inputs are composed images without a shift as fake and autoencoded images as real.
Revised: For ∆ = 0 (no-shift) the only possible discriminator inputs are composed images without a shift as fake and autoencoded images as real.

Original: There is no incentive to produce any meaningful masks in this case.
Revised: There is no incentive to produce any meaningful masks in this case.

Original: Discriminator inputs.
Revised: Discriminator inputs.

Original: In our baseline model, we feed both composed images with no-shift and real images autoencoded with MAE as real samples and composed images with a shift and autoencoded images with copy-pasting of a predicted masked object as fake samples for the discriminator training.
Revised: In our baseline model, we feed both composed images with no-shift and real images autoencoded with MAE as real samples and composed images with a shift and autoencoded images with copy-pasting of a predicted masked object as fake samples for the discriminator training.

Original: We test the case DISC .
Revised: We test the case DISC .

Original: REAL x + COMP .
Revised: REAL x + COMP .

Original: W / O
Revised: W / O

Original: SHIFT , where we feed to the discriminator real images without autoencoding.
Revised: SHIFT , where we feed to the discriminator real images without autoencoding.

Original: In this case, the discriminator can detect the artifacts of MAE instead of focusing on inconsistencies resulting from an incorrect mask.
Revised: In this case, the discriminator can detect the artifacts of MAE instead of focusing on inconsistencies resulting from an incorrect mask.

Original: In DISC .
Revised: In DISC .

Original: REAL x ae we only feed the autoencoded images as real.
Revised: REAL x ae we only feed the autoencoded images as real.

Original: Here, the discriminator can focus on the mismatch from the inpainting artifacts and encourages the segmenter to output empty masks, where no inpainting is done.
Revised: Here, the discriminator can focus on the mismatch from the inpainting artifacts and encourages the segmenter to output empty masks, where no inpainting is done.

Original: If we only feed the composite non-shifted images ( DISC .
Revised: If we only feed the composite non-shifted images ( DISC .

Original: REAL COMP W / O SHIFT )
Revised: REAL COMP W / O SHIFT )

Original: , the artifacts resulting from an incorrect masks cannot be fixed, because there is no reference of what the real images look like.
Revised: , the artifacts resulting from an incorrect masks cannot be fixed, because there is no reference of what the real images look like.

Original: In DISC .
Revised: In DISC .

Original: FAKE INPUTS :
Revised: FAKE INPUTS :

Original: COMPOSED we only feed the composed image as fake to the discriminator and omit the real image with a copy-pasted predicted masked object, which slightly degrades the performance.
Revised: COMPOSED we only feed the composed image as fake to the discriminator and omit the real image with a copy-pasted predicted masked object, which slightly degrades the performance.

Original: Non-differentiable inpainter.
Revised: Non-differentiable inpainter.

Original: We evaluate the use of hard thresholded downsampled masks as input to the background inpainter.
Revised: We evaluate the use of hard thresholded downsampled masks as input to the background inpainter.

Original: In this case the only feedback for the masks comes from the composition of the images.
Revised: In this case the only feedback for the masks comes from the composition of the images.

Original: We find it to be insufficient for the segmenter to learn any meaningful masks.
Revised: We find it to be insufficient for the segmenter to learn any meaningful masks.

Original: Inpainter model.
Revised: Inpainter model.

Original: We substitute the MAE trained with a GAN loss with a MAE that was trained only to reconstruct missing patches with a Mean Squared Error (MSE) loss.
Revised: We substitute the MAE trained with a GAN loss with a MAE that was trained only to reconstruct missing patches with a Mean Squared Error (MSE) loss.

Original: Since this model was trained to only reconstruct the missing patches and not the entire image, we construct the inpainted background by composing the inpainted part with the real image: ˆ m up = upsample 16 ( ˆ m ) ; ˆ b := x ⊙ (1 − ˆ m up ) + ˆ b ⊙ ˆ m up .
Revised: Since this model was trained to only reconstruct the missing patches and not the entire image, we construct the inpainted background by composing the inpainted part with the real image: ˆ m up = upsample 16 ( ˆ m ) ; ˆ b := x ⊙ (1 − ˆ m up ) + ˆ b ⊙ ˆ m up .

Original: Consequently, we do not use autoencoding when creating the discriminator inputs.
Revised: Consequently, we do not use autoencoding when creating the discriminator inputs.

Original: We find this model to perform competitively.
Revised: We find this model to perform competitively.

Original: Feature extractor.
Revised: Feature extractor.

Original: We train the model using the features provided by MAE encoder instead of a separate DINO model.
Revised: We train the model using the features provided by MAE encoder instead of a separate DINO model.

Original: In this case we adapted the segmenter architecture and added one more upsampling block, since MAE takes patches of size P = 16 (instead DINO has P = 8 ).
Revised: In this case we adapted the segmenter architecture and added one more upsampling block, since MAE takes patches of size P = 16 (instead DINO has P = 8 ).

Original: We find that with these features we are able to train a competitive segmenter.
Revised: We find that with these features we are able to train a competitive segmenter.

Original: ImageNet100 dataset.
Revised: ImageNet100 dataset.

Original: We train our model on the ImageNet100 dataset [58], with 131 , 689 images from 100 randomly selected ImageNet [24] classes.
Revised: We train our model on the ImageNet100 dataset [59], with 131 , 689 images from 100 randomly selected ImageNet [25] classes.

Original: Since this dataset is much bigger than DUTS-TR, we adapt our segmenter by adding an additional convolutional layer in each upsampling block (see section 3) and train the model for 8 epochs.
Revised: Since this dataset is much bigger than DUTS-TR, we adapt our segmenter by adding an additional convolutional layer in each upsampling block (see section 3) and train the model for 8 epochs.

Original: The results are comparable to the DUTS-TR dataset.
Revised: The results are comparable to the DUTS-TR dataset.

Original:  Prior Work
Revised:  Prior Work

Original: In the past decade, research on object segmentation and detection has seen remarkable progress when full supervision is available [59–61].
Revised: In the past decade, research on object segmentation and detection has seen remarkable progress when full supervision is available [60–62].

Original: To limit the cost of annotation several methods explored different forms of weak supervision [62, 63] or ways to avoid labeling altogether [13, 14, 64, 65].
Revised: To limit the cost of annotation several methods explored different forms of weak supervision [63, 64] or ways to avoid labeling altogether [13, 14, 65, 66].

Original: MOVE falls in the latter category.
Revised: MOVE falls in the latter category.

Original: Therefore, we focus our review of prior work on unsupervised methods for object segmentation and the related task of object detection.
Revised: Therefore, we focus our review of prior work on unsupervised methods for object segmentation and the related task of object detection.

Original: Unsupervised Object Detection and Category Discovery.
Revised: Unsupervised Object Detection and Category Discovery.

Original: Unsupervised object detection and category discovery are extremely challenging tasks that have recently seen a surge of efforts [56, 66, 67] thanks to the capabilities of modern deep learning models.
Revised: Unsupervised object detection and category discovery are extremely challenging tasks that have recently seen a surge of efforts [57, 67, 68] thanks to the capabilities of modern deep learning models.

Original: Recently, features based on deep learning have shown significant progress in object detection [56], even with just some noisy (unsupervised) guidance [51].
Revised: Recently, features based on deep learning have shown significant progress in object detection [57], even with just some noisy (unsupervised) guidance [52].

Original: More in general, one limitation of unsupervised object detection is that it only provides a coarse localization of the objects.
Revised: More in general, one limitation of unsupervised object detection is that it only provides a coarse localization of the objects.

Original: As we have shown with MOVE, it is possible to obtain much more information without supervision.
Revised: As we have shown with MOVE, it is possible to obtain much more information without supervision.

Original: Unsupervised Object Segmentation.
Revised: Unsupervised Object Segmentation.

Original: Object segmentation can be formulated as a pixel-wise image partitioning task [64, 65, 68, 69] or through the generation of layered models from which a segmenter is trained as a byproduct [8, 34, 70, 71].
Revised: Object segmentation can be formulated as a pixel-wise image partitioning task [65, 66, 69, 70] or through the generation of layered models from which a segmenter is trained as a byproduct [8, 35, 71, 72].

Original: The use of SSL features has spawned several methods with significant performance on real datasets, which we discuss in the following paragraphs.
Revised: The use of SSL features has spawned several methods with significant performance on real datasets, which we discuss in the following paragraphs.

Original: SSL-Based Methods.
Revised: SSL-Based Methods.

Original: Due to the success of SSL methods and the emergence of segmentation capabilities, several recent methods for unsupervised object segmentation have been built on top of
Revised: Due to the success of SSL methods and the emergence of segmentation capabilities, several recent methods for unsupervised object segmentation have been built on top of

Original: SSL features.
Revised: SSL features.

Original: In particular, SelfMask [12] proposes a clustering approach that can use multiple SSL features and evaluates all possible combinations of DINO [4], SwAV [72] and MOCOV2 [73].
Revised: In particular, SelfMask [12] proposes a clustering approach that can use multiple SSL features and evaluates all possible combinations of DINO [4], SwAV [73] and MOCOV2 [74].

Original: They find that combining features from all three SSL methods yields the best results for segmentation.
Revised: They find that combining features from all three SSL methods yields the best results for segmentation.

Original: FreeSOLO [8] instead finds that DenseCL features [6] work best.
Revised: FreeSOLO [8] instead finds that DenseCL features [6] work best.

Original: More in general, some methods use a weak (unsupervised) guidance and losses robust to the coarse pseudo-labels [8], but the majority is based on directly clustering SSL features [7, 9–11, 21, 74].
Revised: More in general, some methods use a weak (unsupervised) guidance and losses robust to the coarse pseudo-labels [8], but the majority is based on directly clustering SSL features [7, 9–11, 22, 75].

Original: In contrast to these methods, we show that movability can provide a robust supervision signal.
Revised: In contrast to these methods, we show that movability can provide a robust supervision signal.

Original: Generative Methods.
Revised: Generative Methods.

Original: A wide range of methods also exploits generative models to create layered image representations [15, 17, 19, 75–78].
Revised: A wide range of methods also exploits generative models to create layered image representations [15, 17, 20, 76–79].

Original: A general scheme is to train a network to generate a background, a foreground and its mask.
Revised: A general scheme is to train a network to generate a background, a foreground and its mask.

Original: These components can then be combined to generate an image and then, in a second stage, one can train a segmenter that learns to map a synthetic image to its corresponding foreground mask.
Revised: These components can then be combined to generate an image and then, in a second stage, one can train a segmenter that learns to map a synthetic image to its corresponding foreground mask.

Original: In alternative, the segmenter could be built during the training of the generative model as a byproduct.
Revised: In alternative, the segmenter could be built during the training of the generative model as a byproduct.

Original: Some methods rely on the assumption that a dataset of only backgrounds is available [14, 79].
Revised: Some methods rely on the assumption that a dataset of only backgrounds is available [14, 80].

Original: The use of shifts to define segments has also been used before [13, 15, 16].
Revised: The use of shifts to define segments has also been used before [13, 15, 16].

Original: However, MOVE does not require the training of a generative model, which can be a challenge on its own.
Revised: However, MOVE does not require the training of a generative model, which can be a challenge on its own.

Original:  Limitations and Societal Impact
Revised:  Limitations and Societal Impact

Original: Limitations.
Revised: Limitations.

Original: As mentioned in the introduction, movability alone may not suffice in identifying an object unambiguously.
Revised: As mentioned in the introduction, movability alone may not suffice in identifying an object unambiguously.

Original: In fact, the method can segment any combination of multiple objects.
Revised: In fact, the method can segment any combination of multiple objects.

Original: To address this we use a post-processing algorithm to find connected components, but there is no guarantee that all objects have been segmented.
Revised: To address this we use a post-processing algorithm to find connected components, but there is no guarantee that all objects have been segmented.

Original: Another issue is that shifts would not expose artifacts when the background is uniform ( e.g. , looking at the sky, underwater, with macrophotography).
Revised: Another issue is that shifts would not expose artifacts when the background is uniform ( e.g. , looking at the sky, underwater, with macrophotography).

Original: Societal Impact.
Revised: Societal Impact.

Original: An important aspect that is relevant to unsupervised learning methods in general is the potential to become biased if the training datasets are unbalanced.
Revised: An important aspect that is relevant to unsupervised learning methods in general is the potential to become biased if the training datasets are unbalanced.

Original: This clearly has a potentially negative impact on the segmentation of categories that are underrepresented and thus this work should be integrated with mechanisms to take the dataset imbalance into account.
Revised: This clearly has a potentially negative impact on the segmentation of categories that are underrepresented and thus this work should be integrated with mechanisms to take the dataset imbalance into account.

Original:  Conclusions
Revised:  Conclusions

Original: We have introduced MOVE, a novel self-supervised method for object segmentation that exploits the synthesis of images where objects are randomly shifted.
Revised: We have introduced MOVE, a novel self-supervised method for object segmentation that exploits the synthesis of images where objects are randomly shifted.

Original: MOVE improves the SotA in object saliency segmentation, unsupervised single object discovery, and unsupervised class agnostic object detection by significant margins.
Revised: MOVE improves the SotA in object saliency segmentation, unsupervised single object discovery, and unsupervised class agnostic object detection by significant margins.

Original: Our ablations show that movability is a strong supervision signal that can be robustly exploited as a pseudo-task for self-supervised object segmentation.
Revised: Our ablations show that movability is a strong supervision signal that can be robustly exploited as a pseudo-task for self-supervised object segmentation.

Original: We believe that our approach can be further scaled by exploring different architectures and larger datasets
Revised: We believe that our approach can be further scaled by exploring different architectures and larger datasets

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] We use publicly available datasets.
Revised: [No] We use publicly available datasets.

Original: We plan on sharing the code and the instructions at a later time.
Revised: We plan on sharing the code and the instructions at a later time.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] We trained our baseline model with 3 different seeds and reported mean and std IoU on the ablation dataset.
Revised: [Yes] We trained our baseline model with 3 different seeds and reported mean and std IoU on the ablation dataset.

Original: We do not include it for all the experiments due to time and cost constraints.
Revised: We do not include it for all the experiments due to time and cost constraints.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 4.
Revised: [Yes] 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [No] (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [No] (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether
Revised: [No] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [No] We use datasets publicly available for research (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [No] We use datasets publicly available for research (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No] 5.
Revised: [No] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]