Original: Y OU O NLY N EED A DVERSARIAL S UPERVISION FOR S EMANTIC I MAGE S YNTHESIS
Revised: Y OU O NLY N EED A DVERSARIAL S UPERVISION FOR S EMANTIC I MAGE S YNTHESIS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Despite their recent successes, GAN models for semantic image synthesis still suffer from poor image quality when trained with only adversarial supervision.
Revised: Despite their recent successes, GAN models for semantic image synthesis still suffer from poor image quality when trained with only adversarial supervision.

Original: Historically, additionally employing the VGG-based perceptual loss has helped to overcome this issue, signiﬁcantly improving the synthesis quality, but at the same time limiting the progress of GAN models for semantic image synthesis.
Revised: Historically, additionally employing the VGG-based perceptual loss has helped to overcome this issue, signiﬁcantly improving the synthesis quality, but at the same time limiting the progress of GAN models for semantic image synthesis.

Original: In this work, we propose a novel, simpliﬁed GAN model, which needs only adversarial supervision to achieve high quality results.
Revised: In this work, we propose a novel, simpliﬁed GAN model, which needs only adversarial supervision to achieve high quality results.

Original: We re-design the discriminator as a semantic segmentation network, directly using the given semantic label maps as the ground truth for training.
Revised: We re-design the discriminator as a semantic segmentation network, directly using the given semantic label maps as the ground truth for training.

Original: By providing stronger supervision to the discriminator as well as to the generator through spatially- and semantically-aware discriminator feedback, we are able to synthesize images of higher ﬁdelity with better alignment to their input label maps, making the use of the perceptual loss superﬂuous.
Revised: By providing stronger supervision to the discriminator as well as to the generator through spatially- and semantically-aware discriminator feedback, we are able to synthesize images of higher ﬁdelity with better alignment to their input label maps, making the use of the perceptual loss superﬂuous.

Original: Moreover, we enable high-quality multi-modal image synthesis through global and local sampling of a 3D noise tensor injected into the generator, which allows complete or partial image change.
Revised: Moreover, we enable high-quality multi-modal image synthesis through global and local sampling of a 3D noise tensor injected into the generator, which allows complete or partial image change.

Original: We show that images synthesized by our model are more diverse and follow the color and texture distributions of real images more closely.
Revised: We show that images synthesized by our model are more diverse and follow the color and texture distributions of real images more closely.

Original: We achieve an average improvement of 6 FID and 5 mIoU points over the state of the art across different datasets using only adversarial supervision.
Revised: We achieve an average improvement of 6 FID and 5 mIoU points over the state of the art across different datasets using only adversarial supervision.

Original: [Figure]
Revised: [Figure]

Revised: SPADE (Park et al., 2019)

Revised: Our model (OASIS), sampled with different noise label map with VGG w/o VGG w/o VGG the VGG-based perceptual loss to improve the quality of generated images.

Revised: In contrast, our model can synthesize diverse and high-quality images while only using an adversarial loss, without any external supervision.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Conditional generative adversarial networks (GANs) (Mirza & Osindero, 2014) synthesize images conditioned on class labels (Zhang et al., 2019; Brock et al., 2019), text (Reed et al., 2016; Zhang et al., 2018a), other images (Isola et al., 2017; Huang et al., 2018), or semantic label maps (Wang et al., 2018; Park et al., 2019).
Revised: Conditional generative adversarial networks (GANs) (Mirza & Osindero, 2014) synthesize images conditioned on class labels (Zhang et al., 2019; Brock et al., 2019), text (Reed et al., 2016; Zhang et al., 2018a), other images (Isola et al., 2017; Huang et al., 2018), or semantic label maps (Wang et al., 2018; Park et al., 2019).

Original: In this work, we focus on the latter, addressing semantic image synthesis.
Revised: In this work, we focus on the latter, addressing semantic image synthesis.

Original: Semantic image synthesis enables rendering of realistic images from user-speciﬁed layouts, Label map OASIS OASIS
Revised: Semantic image synthesis enables rendering of realistic images from user-speciﬁed layouts, without the use of an intricate graphic engine.

Original: OASIS

Original: OASIS

Original: OASIS

Original: G l ob a l ( t h e w ho l e s ce n e )

Original: L o ca l ( b e d )

Original: L o ca l ( s h a p e ) without the use of an intricate graphic engine.

Original: Therefore, its applications range widely from content creation and image editing to generating training data that needs to adhere to speciﬁc semantic requirements (Wang et al., 2018; Chen & Koltun, 2017).
Revised: Therefore, its applications range widely from content creation and image editing to generating training data that needs to adhere to speciﬁc semantic requirements (Wang et al., 2018; Chen & Koltun, 2017).

Original: Despite the recent progress on stabilizing
Revised: Despite the recent progress on stabilizing

Original: GANs (Gulrajani et al., 2017; Miyato et al., 2018) and developing their architectures (Zhang et al., 2019; Karras et al., 2019), state-of-the-art GAN-based semantic image synthesis models (Park et al., 2019; Liu et al., 2019) still greatly suffer from training instabilities and poor image quality when trained only with adversarial supervision (see Fig. 1).
Revised: GANs (Gulrajani et al., 2017; Miyato et al., 2018; Zhang & Khoreva, 2019) and developing their architectures (Zhang et al., 2019; Karras et al., 2019), state-of-the-art GAN-based semantic image synthesis models (Park et al., 2019; Liu et al., 2019) still greatly suffer from training instabilities and poor image quality when trained only with adversarial supervision (see Fig. 1).

Original: An established practice to overcome this issue is to employ a perceptual loss (Wang et al., 2018) to train the generator, in addition to the discriminator loss.
Revised: An established practice to overcome this issue is to employ a perceptual loss (Wang et al., 2018) to train the generator, in addition to the discriminator loss.

Original: The perceptual loss aims to match intermediate features of synthetic and real images, that are estimated via an external perception network.
Revised: The perceptual loss aims to match intermediate features of synthetic and real images, that are estimated via an external perception network.

Original: A popular choice for such a network is VGG (Simonyan & Zisserman, 2015), pre-trained on ImageNet (Deng et al., 2009).
Revised: A popular choice for such a network is VGG (Simonyan & Zisserman, 2015), pre-trained on ImageNet (Deng et al., 2009).

Original: Although the perceptual loss substantially improves the accuracy of previous methods, it comes with the computational overhead introduced by utilizing an extra network for training.
Revised: Although the perceptual loss substantially improves the accuracy of previous methods, it comes with the computational overhead introduced by utilizing an extra network for training.

Original: Moreover, it usually dominates over the adversarial loss during training, which can have a negative impact on the diversity and quality of generated images, as we show in our experiments.
Revised: Moreover, it usually dominates over the adversarial loss during training, which can have a negative impact on the diversity and quality of generated images, as we show in our experiments.

Original: Therefore, in this work we propose a novel, simpliﬁed model that achieves state-of-the-art results without requiring a perceptual loss.
Revised: Therefore, in this work we propose a novel, simpliﬁed model that achieves state-of-the-art results without requiring a perceptual loss.

Original: A fundamental question for GAN-based semantic image synthesis models is how to design the discriminator to efﬁciently utilize information from the given semantic label maps.
Revised: A fundamental question for GAN-based semantic image synthesis models is how to design the discriminator to efﬁciently utilize information from the given semantic label maps.

Original: Conventional methods (Park et al., 2019; Wang et al., 2018;
Revised: Conventional methods (Park et al., 2019; Wang et al., 2018;

Original: Liu et al., 2019; Isola et al., 2017) adopt a multi-scale classiﬁcation network, taking the label map as input along with the image, and making a global image-level real/fake decision.
Revised: Liu et al., 2019; Isola et al., 2017) adopt a multi-scale classiﬁcation network, taking the label map as input along with the image, and making a global image-level real/fake decision.

Original: Such a discriminator has limited representation power, as it is not incentivized to learn high-ﬁdelity pixel-level details of the images and their precise alignment with the input semantic label maps.
Revised: Such a discriminator has limited representation power, as it is not incentivized to learn high-ﬁdelity pixel-level details of the images and their precise alignment with the input semantic label maps.

Original: To mitigate this issue, we propose an alternative architecture for the discriminator, re-designing it as an encoder-decoder semantic segmentation network (Ronneberger et al., 2015), and directly exploiting the given semantic label maps as ground truth via a ( N + 1 )
Revised: To mitigate this issue, we propose an alternative architecture for the discriminator, re-designing it as an encoder-decoder semantic segmentation network (Ronneberger et al., 2015), and directly exploiting the given semantic label maps as ground truth via a ( N + 1 )

Original: -class cross-entropy loss (see Fig. 3).
Revised: -class cross-entropy loss (see Fig. 3).

Original: This new discriminator provides semantically-aware pixel-level feedback to the generator, partitioning the image into segments belonging to one of the N real semantic classes or the fake class.
Revised: This new discriminator provides semantically-aware pixel-level feedback to the generator, partitioning the image into segments belonging to one of the N real semantic classes or the fake class.

Original: Enabled by the discriminator per-pixel response, we further introduce a LabelMix regularization, which fosters the discriminator to focus more on the semantic and structural differences of real and synthetic images.
Revised: Enabled by the discriminator per-pixel response, we further introduce a LabelMix regularization, which fosters the discriminator to focus more on the semantic and structural differences of real and synthetic images.

Original: The proposed changes lead to a much stronger discriminator, that maintains a powerful semantic representation of objects, giving more meaningful feedback to the generator, and thus making the perceptual loss supervision superﬂuous (see Fig. 1).
Revised: The proposed changes lead to a much stronger discriminator, that maintains a powerful semantic representation of objects, giving more meaningful feedback to the generator, and thus making the perceptual loss supervision superﬂuous (see Fig. 1).

Original: Next, we propose to enable multi-modal synthesis of the generator via 3D noise sampling.
Revised: Next, we propose to enable multi-modal synthesis of the generator via 3D noise sampling.

Original: Previously, directly using 1D noise as input was not successful for semantic image synthesis, as the generator tended to mostly ignore it or synthesized images of poor quality (Isola et al., 2017; Wang et al., 2018).
Revised: Previously, directly using 1D noise as input was not successful for semantic image synthesis, as the generator tended to mostly ignore it or synthesized images of poor quality (Isola et al., 2017; Wang et al., 2018).

Original: Thus, prior work (Wang et al., 2018; Park et al., 2019) resorted to using an image encoder to produce multi-modal outputs.
Revised: Thus, prior work (Wang et al., 2018; Park et al., 2019) resorted to using an image encoder to produce multi-modal outputs.

Original: In this work, we propose a lighter solution.
Revised: In this work, we propose a lighter solution.

Original: Empowered by our stronger discriminator, the generator can effectively synthesize different images by simply re-sampling a 3 D noise tensor, which is used not only as the input but also combined with intermediate features via conditional normalization at every layer.
Revised: Empowered by our stronger discriminator, the generator can effectively synthesize different images by simply re-sampling a 3 D noise tensor, which is used not only as the input but also combined with intermediate features via conditional normalization at every layer.

Original: Such noise is spatially sensitive, so we can re-sample it both globally (channel-wise) and locally (pixel-wise), allowing to change not only the appearance of the whole scene, but also of speciﬁc semantic classes or any chosen areas (see Fig.
Revised: Such noise is spatially sensitive, so we can re-sample it both globally (channel-wise) and locally (pixel-wise), allowing to change not only the appearance of the whole scene, but also of speciﬁc semantic classes or any chosen areas (see Fig.

Original: We call our model OASIS, as it needs o nly a dversarial supervision for s emantic i mage s ynthesis.
Revised: We call our model OASIS, as it needs o nly a dversarial supervision for s emantic i mage s ynthesis.

Original: In summary, our main contributions are: (1) We propose a novel segmentation-based discriminator architecture, that gives more powerful feedback to the generator and eliminates the necessity of the perceptual loss supervision.
Revised: In summary, our main contributions are: (1) We propose a novel segmentation-based discriminator architecture, that gives more powerful feedback to the generator and eliminates the necessity of the perceptual loss supervision.

Original: (2) We present a simple 3D noise sampling scheme, notably increasing the diversity of multi-modal synthesis and enabling complete or partial change of the generated image.
Revised: (2) We present a simple 3D noise sampling scheme, notably increasing the diversity of multi-modal synthesis and enabling complete or partial change of the generated image.

Original: (3) With the OASIS model, we achieve high quality results on the ADE20K, Cityscapes and
Revised: (3) With the OASIS model, we achieve high quality results on the ADE20K, Cityscapes and

Original: COCO-stuff datasets, on average improving the state of the art by 6 FID and 5 mIoU points, while relying only on adversarial supervision.
Revised: COCO-stuff datasets, on average improving the state of the art by 6 FID and 5 mIoU points, while Label map OASIS OASIS

Original: We show that images synthesized by OASIS exhibit much higher diversity and more closely follow the color and texture distributions of real images.
Revised: OASIS OASIS OASIS G l ob a l ( t h e w ho l e s ce n e )

Revised: L o ca l ( b e d )

Revised: L o ca l ( s h a p e )

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Semantic image synthesis.
Revised: Semantic image synthesis.

Original: Pix2pix (Isola et al., 2017) ﬁrst proposed to use conditional
Revised: Pix2pix (Isola et al., 2017) ﬁrst proposed to use conditional

Original: GANs (Mirza & Osindero, 2014) for semantic image synthesis, adopting an encoder-decoder generator which takes semantic label maps as input, and employing a PatchGAN discriminator.
Revised: GANs (Mirza & Osindero, 2014) for semantic image synthesis, adopting an encoder-decoder generator which takes semantic label maps as input, and employing a PatchGAN discriminator.

Original: Since then, various generator and discriminator modiﬁcations have been introduced (Wang et al., 2018;
Revised: Since then, various generator and discriminator modiﬁcations have been introduced (Wang et al., 2018;

Original: Park et al., 2019; Liu et al., 2019; Tang et al., 2020c;b; Ntavelis et al., 2020).
Revised: Park et al., 2019; Liu et al., 2019; Tang et al., 2020c;b; Ntavelis et al., 2020).

Original: Besides GANs, (Chen & Koltun, 2017) proposed to use a cascaded reﬁnement network (CRN) for high-resolution semantic image synthesis, and SIMS (Qi et al., 2018) extended it with a non-parametric component, serving as a memory bank of source material to assist the synthesis.
Revised: Besides GANs, Chen & Koltun (2017) proposed to use a cascaded reﬁnement network (CRN) for high-resolution semantic image synthesis, and SIMS (Qi et al., 2018) extended it with a non-parametric component, serving as a memory bank of source material to assist the synthesis.

Original: Further, (Li et al., 2019) employed implicit maximum likelihood estimation of Li & Malik (2018) to increase the variety of the CRN model.
Revised: Further, Li et al. (2019) employed implicit maximum likelihood estimation (Li & Malik, 2018) to increase the variety of the CRN model.

Original: However, these approaches still underperform in comparison to state-of-the-art GAN models.
Revised: However, these approaches still underperform in comparison to state-of-the-art GAN models.

Original: Therefore, next we focus on the recent GAN architectures for semantic image synthesis.
Revised: Therefore, next we focus on the recent GAN architectures for semantic image synthesis.

Original: Discriminator architectures.
Revised: Discriminator architectures.

Original: Pix2pix (Isola et al., 2017), Pix2pixHD (Wang et al., 2018) and
Revised: Pix2pix (Isola et al., 2017), Pix2pixHD (Wang et al., 2018) and

Original: SPADE (Park et al., 2019) all employed a multi-scale PatchGAN discriminator, that takes an image and its semantic label map as input.
Revised: SPADE (Park et al., 2019) all employed a multi-scale PatchGAN discriminator, that takes an image and its semantic label map as input.

Original: CC-FPSE (Liu et al., 2019) proposed a feature-pyramid discriminator, embedding both images and label maps into a joint feature map, and then consecutively upsampling it in order to classify it as real/fake at multiple scales.
Revised: CC-FPSE (Liu et al., 2019) proposed a feature-pyramid discriminator, embedding both images and label maps into a joint feature map, and then consecutively upsampling it in order to classify it as real/fake at multiple scales.

Original: LGGAN (Tang et al., 2020c) introduced a classiﬁcation-based feature learning module to learn more discriminative and class-speciﬁc features.
Revised: LGGAN (Tang et al., 2020c) introduced a classiﬁcation-based feature learning module to learn more discriminative and class-speciﬁc features.

Original: In this work, we propose to use a pixel-wise semantic segmentation network as a discriminator instead of multi-scale image classiﬁers as in the above approaches, and to directly exploit the semantic label maps for its supervision.
Revised: In this work, we propose to use a pixel-wise semantic segmentation network as a discriminator instead of multi-scale image classiﬁers as in the above approaches, and to directly exploit the semantic label maps for its supervision.

Original: Segmentation-based discriminators have been shown to improve semantic segmentation (Souly et al., 2017) and unconditional image synthesis Sch¨onfeld et al. (2020), but to the best of our knowledge have not been explored for semantic image synthesis and our work is the ﬁrst to apply adversarial semantic segmentation loss for this task.
Revised: Segmentation-based discriminators have been shown to improve semantic segmentation (Souly et al., 2017) and unconditional image synthesis (Sch ¨ onfeld et al., 2020), but to the best of our knowledge have not been explored for semantic image synthesis and our work is the ﬁrst to apply adversarial semantic segmentation loss for this task.

Original: Generator architectures.
Revised: Generator architectures.

Original: Conventionally, the semantic label map is provided to the image generation pipeline via an encoder (Isola et al., 2017;
Revised: Conventionally, the semantic label map is provided to the image generation pipeline via an encoder (Isola et al., 2017;

Original: Wang et al., 2018; Tang et al., 2020c;b; Ntavelis et al., 2020).
Revised: Wang et al., 2018; Tang et al., 2020c;b; Ntavelis et al., 2020).

Original: However, it is shown to be suboptimal at preserving the semantic information until the later stages of image generation.
Revised: However, it is shown to be suboptimal at preserving the semantic information until the later stages of image generation.

Original: Therefore, SPADE introduced a spatially-adaptive normalization layer that directly modulates the label map onto the generator’s hidden layer outputs at variousscales.
Revised: Therefore, SPADE introduced a spatially-adaptive normalization layer that directly modulates the label map onto the generator’s hidden layer outputs at various

Original: Alternatively, CC-FPSE proposed to use spatially-varying convolution kernels conditioned on the label map.
Revised: Alternatively, CC-FPSE proposed to use spatially-varying convolution kernels conditioned on the label map.

Original: Struggling with generating diverse images from noise, both Pix2pixHD and SPADE resorted to having an image encoder in the generator design to enable multi-modal synthesis.
Revised: Struggling with generating diverse images from noise, both Pix2pixHD and SPADE resorted to having an image encoder in the generator design to enable multi-modal synthesis.

Original: The generator then combines the extracted image style with the label map to reconstruct the original image.
Revised: The generator then combines the extracted image style with the label map to reconstruct the original image.

Original: By alternating the style vector, one can generate multiple outputs conditioned on the same label map.
Revised: By alternating the style vector, one can generate multiple outputs conditioned on the same label map.

Original: However, using an image encoder is a resource demanding solution.
Revised: However, using an image encoder is a resource demanding solution.

Original: In this work, we enable multi-modal synthesis directly through sampling of a 3D noise tensor injected at every layer of the network.
Revised: In this work, we enable multi-modal synthesis directly through sampling of a 3D noise tensor injected at every layer of the network.

Original: Different from structured noise injection of Alharbi & Wonka (2020) and class-speciﬁc latent codes of Zhu et al.
Revised: Differently from structured noise injection of Alharbi & Wonka (2020) and class-speciﬁc latent codes of Zhu et al.

Original: , we inject  3D noise along with label maps and adjust it to image resolution,enabling also  re-sampling of selected semantic segments (see Fig. 2).
Revised: (2020), we inject the 3D noise along with label maps and adjust it to image resolution, also enabling re-sampling of selected semantic segments (see Fig. 2).

Original: losses
Revised: Perceptual losses. Gatys et al. ; Gatys et al. ; Johnson et al.

Revised: (2016) and Bruna et al.

Original: (2016) were pioneers at exploiting perceptual losses to produce high-quality images for superresolution and style transfer using convolutional networks.
Revised: (2016) were pioneers at exploiting perceptual losses to produce high-quality images for superresolution and style transfer using convolutional networks.

Original: For semantic image synthesis, the VGGbased perceptual loss was ﬁrst introduced by CRN, and later adopted by Pix2pixHD.
Revised: For semantic image synthesis, the VGGbased perceptual loss was ﬁrst introduced by CRN, and later adopted by Pix2pixHD.

Original: Since then, it has become a default for training the generator (Park et al., 2019; Liu et al., 2019; Tan et al., 2020; Tang et al., 2020a).
Revised: Since then, it has become a default for training the generator (Park et al., 2019; Liu et al., 2019; Tan et al., 2020; Tang et al., 2020a).

Original: As the perceptual loss is based on a VGG network pre-trained on ImageNet (Deng et al., 2009), methods relying on it are constrained by the ImageNet domain and the representational power of VGG.
Revised: As the perceptual loss is based on a VGG network pre-trained on ImageNet (Deng et al., 2009), methods relying on it are constrained by the ImageNet domain and the representational power of VGG.

Original: With the recent progress on GAN training, e.g. by architecture designs and regularization techniques, the actual necessity of the perceptual loss requires a reassessment.
Revised: With the recent progress on GAN training, e.g. by architecture designs and regularization techniques, the actual necessity of the perceptual loss requires a reassessment.

Original: We experimentally show that such loss imposes unnecessary constraints on the generator, signiﬁcantly limiting sample diversity.
Revised: We experimentally show that such loss imposes unnecessary constraints on the generator, signiﬁcantly limiting sample diversity.

Original: While our model, trained without the VGG loss, achieves improved image diversity while not compromising image quality.
Revised: While our model, trained without the VGG loss, achieves improved image diversity while not compromising image quality.

Original:  OASIS MODEL
Revised:  OASIS MODEL

Original: In this section, we present our OASIS model, which, in contrast to other semantic image synthesis methods, needs only adversarial supervision for generator training.
Revised: In this section, we present our OASIS model, which, in contrast to other semantic image synthesis methods, needs only adversarial supervision for generator training.

Original: Using SPADE as a starting point (Sec. 3.1), we ﬁrst propose to re-design the discriminator as a semantic segmentation network, directly using the given semantic label maps as ground truth (Sec. 3.2).
Revised: Using SPADE as a starting point (Sec. 3.1), we ﬁrst propose to re-design the discriminator as a semantic segmentation network, directly using the given semantic label maps as ground truth (Sec. 3.2).

Original: Empowered by spatiallyand semantically-aware feedback of the new discriminator, we next re-design the SPADE generator, enabling its effective multi-modal synthesis via 3D noise sampling (Sec. 3.3).
Revised: Empowered by spatiallyand semantically-aware feedback of the new discriminator, we next re-design the SPADE generator, enabling its effective multi-modal synthesis via 3D noise sampling (Sec. 3.3).

Original: 3. T HE SPADE BASELINE
Revised: 3. T HE SPADE BASELINE

Original: We choose SPADE as our baseline as it is a state-of-the-art model and a relatively simple representative of conventional semantic image synthesis models.
Revised: We choose SPADE as our baseline as it is a state-of-the-art model and a relatively simple representative of conventional semantic image synthesis models.

Original: As depicted in Fig.
Revised: As depicted in Fig.

Original: 3, the discriminator of
Revised: 3, the discriminator of

Original: SPADE largely follows the PatchGAN multi-scale discriminator (Isola et al., 2017), adopting two image classiﬁcation networks operating at different resolutions.
Revised: SPADE largely follows the PatchGAN multi-scale discriminator (Isola et al., 2017), adopting two image classiﬁcation networks operating at different resolutions.

Original: Both of them take the channel-wise concatenation of the semantic label map and the real/synthesized image as input, and produce true/- fake classiﬁcation scores.
Revised: Both of them take the channel-wise concatenation of the semantic label map and the real/synthesized image as input, and produce true/- fake classiﬁcation scores.

Original: On the generator side, SPADE adopts spatially-adaptive normalization layers to effectively integrate the semantic label map into the synthesis process from low to high scales.
Revised: On the generator side, SPADE adopts spatially-adaptive normalization layers to effectively integrate the semantic label map into the synthesis process from low to high scales.

Original: Additionally, the image encoder is used to extract the style vector from the reference image and then combine it with a 1 D noise vector for multi-modal synthesis.
Revised: Additionally, the image encoder is used to extract the style vector from the reference image and then combine it with a 1 D noise vector for multi-modal synthesis.

Original: The training loss of SPADE consists of three terms, namely, an adversarial loss, a feature matching loss and the VGG-based perceptual loss: L = max G min D L adv + λ fm L fm
Revised: The training loss of SPADE consists of three terms, namely, an adversarial loss, a feature matching loss and the VGG-based perceptual loss: L = max G min D L adv + λ fm L fm

Original: + λ vgg L vgg .
Revised: + λ vgg L vgg .

Original: Overall, SPADE is a resource demanding model at both training and test time, i.e. with two PatchGAN discriminators, an image encoder in addition to the generator, and the VGG loss.
Revised: Overall, SPADE is a resource demanding model at both training and test time, i.e., with two PatchGAN discriminators, an image encoder in addition to the generator, and the VGG loss.

Original: In the following, we revisit its architecture and introduce a simpler and more efﬁcient model that offers better performance with less complexity.
Revised: In the following, we revisit its architecture and introduce a simpler and more efﬁcient model that offers better performance with less complexity.

Original: 3. OASIS DISCRIMINATOR
Revised: 3. OASIS DISCRIMINATOR

Original: For the generator to learn to synthesize images that are well aligned with the input semantic label maps, we need a powerful discriminator that coherently captures discriminative semantic features at different image scales.
Revised: For the generator to learn to synthesize images that are well aligned with the input semantic label maps, we need a powerful discriminator that coherently captures discriminative semantic features at different image scales.

Original: While classiﬁcation-based discriminators, such as PatchGAN, take label maps as input concatenated to images, they can afford to ignore them and make the decision solely on image patch realism.
Revised: While classiﬁcation-based discriminators, such as PatchGAN, take label maps as input concatenated to images, they can afford to ignore them and make the decision solely on image patch realism.

Original: Thus, we propose to cast the discriminator task as a multi-class semantic[Figure] segmentation problem to directly utilize label maps for supervision, and accordingly alter its architecture to an encoder-decoder segmentation network (see Fig. 3).
Revised: Thus, we propose to cast the discriminator task as a multi-class semantic segmentation problem to directly utilize label maps for supervision, and accordingly alter its architecture to an encoder-decoder segmentation network (see Fig. 3).

Original: Encoder-decoder networks have proven to be effective for semantic segmentation Badrinarayanan et al. ; Chen et al.
Revised: Encoder-decoder networks have proven to be effective for semantic segmentation (Badrinarayanan et al., 2016; Chen et al., 2018).

Original: Thus, we build our discriminator architecture upon U-Net Ronneberger et al. (2015), which consists of the encoder and decoder connected by skip connections.
Revised: Thus, we build our discriminator architecture upon U-Net (Ronneberger et al., 2015), which consists of the encoder and decoder connected by skip connections.

Original: This discriminator architecture is multiscale through its design, integrating information over up- and down-sampling pathways and through the encoder-decoder skip connections.
Revised: This discriminator architecture is multi-scale through its design, integrating information over up- and down-sampling pathways and through the encoder-decoder skip connections.

Original: For details on the architecture see App.
Revised: For details on the architecture see App.

Original: C.1.
Revised: C.1.

Original: The segmentation task of the discriminator is formulated to predict the per-pixel class label of the real images, using the given semantic label maps as ground truth.
Revised: The segmentation task of the discriminator is formulated to predict the per-pixel class label of the real images, using the given semantic label maps as ground truth.

Original: In addition to the N semantic classes from the label maps, all pixels of the fake images are categorized as one extra class.
Revised: In addition to the N semantic classes from the label maps, all pixels of the fake images are categorized as one extra class.

Original: Overall, we have N +1 classes in the semantic segmentation problem, and thus propose to use a ( N + 1 )-class cross-entropy loss for training.
Revised: Overall, we have N +1 classes in the semantic segmentation problem, and thus propose to use a ( N + 1 )-class cross-entropy loss for training.

Original: Considering that the N semantic classes are usually imbalanced and that the per-pixel size of objects varies for different semantic classes, we weight each class by its inverse per-pixel frequency, giving rare semantic classes more weight.
Revised: Considering that the N semantic classes are usually imbalanced and that the per-pixel size of objects varies for different semantic classes, we weight each class by its inverse per-pixel frequency, giving rare semantic classes more weight.

Original: In doing so, the contributions of each semantic class are equally balanced, and, thus, the generator is also encouraged to adequately synthesize less-represented classes.
Revised: In doing so, the contributions of each semantic class are equally balanced, and, thus, the generator is also encouraged to adequately synthesize less-represented classes.

Original: Mathematically, the new discriminator loss is expressed as:
Revised: Mathematically, the new discriminator loss is expressed as:

Original: [Equation]
Revised: [Equation]

Original: where x denotes the real image; ( z, t ) is the noise-label map pair used by the generator G to synthesize a fake image; and the discriminator D maps the real or fake image into a per-pixel ( N + 1 )-class prediction probability.
Revised: where x denotes the real image; ( z, t ) is the noise-label map pair used by the generator G to synthesize a fake image; and the discriminator D maps the real or fake image into a per-pixel ( N + 1 )-class prediction probability.

Original: The ground truth label map t has three dimensions, where the ﬁrst two correspond to the spatial position ( i, j ) ∈ H × W , and the third one is a one-hot vector encoding the class c ∈ { 1 , .., N +1 } .
Revised: The ground truth label map t has three dimensions, where the ﬁrst two correspond to the spatial position ( i, j ) ∈ H × W , and the third one is a one-hot vector encoding the class c ∈ { 1 , .., N +1 } .

Original: The class balancing weight α c is the inverse of the per-pixel class frequency
Revised: The class balancing weight α c is the inverse of the per-pixel class frequency

Original: [Equation]
Revised: [Equation]

Original: LabelMix regularization.
Revised: LabelMix regularization.

Original: In order to encourage our discriminator to focus on differences in content and structure between the fake and the real classes, we propose a LabelMix regularization.
Revised: In order to encourage our discriminator to focus on differences in content and structure between the fake and the real classes, we propose a LabelMix regularization.

Original: Based on the semantic layout, we generate a binary mask M to mix a pair ( x, ˆ x ) of real and fake images conditioned on the same label map: LabelMix( x, ˆ x, M )
Revised: Based on the semantic layout, we generate a binary mask M to mix a pair ( x, ˆ x ) of real and fake images conditioned on the same label map: LabelMix( x, ˆ x, M )

Original: =
Revised: =

Original: M (cid:12) x
Revised: M (cid:12) x

Original: + (1 − M ) (cid:12) ˆ x , as visualized in Fig.
Revised: + (1 − M ) (cid:12) ˆ x , as visualized in Fig.

Original: Given the mixed image, we further train the discriminator to be equivariant under the LabelMix operation.
Revised: Given the mixed image, we further train the discriminator to be equivariant under the LabelMix operation.

Original: This is achieved by adding a consistency loss term L cons to Eq. 1:
Revised: This is achieved by adding a consistency loss term L cons to Eq. 1:

Original: [Equation]
Revised: [Equation]

Original: where D logits are the logits attained before the last softmax activation layer, and (cid:107) · (cid:107) is the Lnorm.
Revised: where D logits are the logits attained before the last softmax activation layer, and (cid:107) · (cid:107) is the Lnorm.

Original: This consistency loss compares the output of the discriminator on the LabelMix image with the LabelMix of its outputs, penalizing the discriminator for inconsistent predictions.
Revised: This consistency loss compares the output of the discriminator on the LabelMix image with the LabelMix of its outputs, penalizing the discriminator for inconsistent predictions.

Original: LabelMix is different to CutMix (Yun et al., 2019), which randomly samples the binary mask M .
Revised: LabelMix is different to CutMix (Yun et al., 2019), which randomly samples the binary mask M .

Original: A random mask will introduce inconsistency between the pixel-level classes and the scene layout provided by the label map.
Revised: A random mask will introduce inconsistency between the pixel-level classes and the scene layout provided by the label map.

Original: For an object with the semantic class c , it will contain pixels from both real and fake images, resulting in two labels, i.e. c and N +1 .
Revised: For an object with the semantic class c , it will contain pixels from both real and fake images, resulting in two labels, i.e. c and N +1 .

Original: To avoid such inconsistency, the mask of LabelMix is generated according to the label map, providing natural borders between semantic regions, see
Revised: To avoid such inconsistency, the mask of LabelMix is generated according to the label map, providing natural borders between semantic regions, see

Original: Fig.
Revised: Fig.

Original: (Mask M ).
Revised: (Mask M ).

Original: Under LabelMix regularization, the generator is encouraged to respect the natural semantic boundaries, improving pixel-level realism while also considering the class segment shapes. 
Revised: Under LabelMix regularization, the generator is encouraged to respect the natural semantic boundaries, improving pixel-level realism while also considering the class segment shapes. [Figure]

Original: Other variants.
Revised: Other variants.

Original: Besides the proposed ( N + 1 )-class cross entropy loss, there are other ways to train the segmentation-based discriminator with the label map.
Revised: Besides the proposed ( N + 1 )-class cross entropy loss, there are other ways to train the segmentation-based discriminator with the label map.

Original: One can concatenate the label map to the input image, analogous to SPADE.
Revised: One can concatenate the label map to the input image, analogous to SPADE.

Original: Another option is to use projection, by taking the inner product between the last linear layer output and the embedded label map, analogous to class-label[Figure] OASIS conditional GANs (Miyato & Koyama, 2018).
Revised: Another option is to use projection, by taking the inner product between the last linear layer output and the embedded label map, analogous to class-label conditional GANs (Miyato & Koyama, 2018).

Original: For both alternatives, the training loss is pixel-level real/fake binary cross-entropy (Sch¨onfeld et al., 2020).
Revised: For both alternatives, the training loss is pixel-level real/fake binary cross-entropy (Sch ¨ onfeld et al., 2020).

Original: From the label map encoding perspective, these two variants use labels map as input (concatenated to image or at last linear layer), propagating it forward through the network.
Revised: From the label map encoding perspective, these two variants use labels map as input (concatenated to image or at last linear layer), propagating it forward through the network.

Original:  ( N + 1 ) -setting uses the label map for loss computation, so it is propagated backward via gradient updates.
Revised: The ( N + 1 )-setting uses the label map for loss computation, so it is propagated backward via gradient updates.

Original: Backward propagation ensures that the discriminator learns semantic-aware features, in contrast to forward propagation, where the label map alignment is not as strongly enforced.
Revised: Backward propagation ensures that the discriminator learns semantic-aware features, in contrast to forward propagation, where the label map alignment is not as strongly enforced.

Original: Performance comparison of the label map encodings is shown in Table 5.
Revised: Performance comparison of the label map encodings is shown in Table 5.

Original: 3. OASIS GENERATOR
Revised: 3. OASIS GENERATOR

Original: To stay in line with the OASIS discriminator design, the training loss for the generator is changed to
Revised: To stay in line with the OASIS discriminator design, the training loss for the generator is changed to

Original: [Equation]
Revised: [Equation]

Original: which is a direct outcome of the non-saturation trick (Goodfellow et al., 2014) to Eq.
Revised: which is a direct outcome of the non-saturation trick (Goodfellow et al., 2014) to Eq.

Original: We next re-design the generator to enable multi-modal synthesis through noise sampling.
Revised: We next re-design the generator to enable multi-modal synthesis through noise sampling.

Original: SPADE is deterministic in its default setup, but can be trained with an extra image encoder to generate multi-modal outputs.
Revised: SPADE is deterministic in its default setup, but can be trained with an extra image encoder to generate multi-modal outputs.

Original: We introduce a simpler version, that enables synthesis of diverse outputs directly from input noise.
Revised: We introduce a simpler version, that enables synthesis of diverse outputs directly from input noise.

Original: For this, we construct a noise tensor of size 64 × H × W , matching the spatial dimensions of the label map H × W .
Revised: For this, we construct a noise tensor of size 64 × H × W , matching the spatial dimensions of the label map H × W .

Original: Channel-wise concatenation of the noise and label map forms a 3 D tensor used as input to the generator and also as a conditioning at every spatially-adaptive normalization layer.
Revised: Channel-wise concatenation of the noise and label map forms a 3 D tensor used as input to the generator and also as a conditioning at every spatially-adaptive normalization layer.

Original: In doing so, intermediate feature maps are conditioned on both the semantic labels and the noise (see Fig. 3).
Revised: In doing so, intermediate feature maps are conditioned on both the semantic labels and the noise (see Fig. 3).

Original: With such a design, the generator produces diverse, noise-dependent images.
Revised: With such a design, the generator produces diverse, noise-dependent images.

Original: As the 3 D noise is channel- and pixel-wise sensitive, at test time, one can sample the noise globally, per-channel, and locally, per-segment or per-pixel, for controlled synthesis of the whole scene or of speciﬁc semantic objects.
Revised: As the 3 D noise is channel- and pixel-wise sensitive, at test time, one can sample the noise globally, per-channel, and locally, per-segment or per-pixel, for controlled synthesis of the whole scene or of speciﬁc semantic objects.

Original: For example, when generating a scene of a bedroom, one can re-sample the noise locally and change the appearance of the bed alone (see Fig. 2).
Revised: For example, when generating a scene of a bedroom, one can re-sample the noise locally and change the appearance of the bed alone (see Fig. 2).

Original: Note that for simplicity during training we sample the 3 D noise tensor globally, i.e. per-channel, replicating each channel value spatially along the height and width of the tensor.
Revised: Note that for simplicity during training we sample the 3 D noise tensor globally, i.e. per-channel, replicating each channel value spatially along the height and width of the tensor.

Original: We analyse alternative ways of sampling 3 D noise during training in App.
Revised: We analyse alternative ways of sampling 3 D noise during training in App.

Original: A.7.
Revised: A.7.

Original: Using image styles via an encoder, as in SPADE, is also possible in our setting, by concatenating encoder features to 3 D noise.
Revised: Using image styles via an encoder, as in SPADE, is also possible in our setting, by replacing noise with encoder features.

Original: Lastly, to further reduce the complexity, we remove the ﬁrst residual block in the generator, reducing the number of parameters from 96 M to 72 M (see App. C.2) without a noticeable performance loss (see Table 3). 
Revised: Lastly, to further reduce the complexity, we remove the ﬁrst residual block in the generator, reducing the number of parameters from 96 M to 72 M (see App. C.2) without a noticeable performance loss (see Table 3). [Figure]

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: We conduct experiments on three challenging datasets: ADE20K
Revised: We conduct experiments on three challenging datasets: ADE20K

Original: (Zhou et al., 2017), COCO-stuff (Caesar et al., 2018) and Cityscapes (Cordts et al., 2016).
Revised: (Zhou et al., 2017), COCO-stuff (Caesar et al., 2018) and Cityscapes (Cordts et al., 2016).

Original: Following Qi et al.
Revised: Following Qi et al.

Original: (2018), we also evaluate OASIS on ADE20K-outdoors, a subset of ADE20K containing outdoor scenes.
Revised: (2018), we also evaluate OASIS on ADE20K-outdoors, a subset of ADE20K containing outdoor scenes.

Original: We follow the experimental setting of Park et al.
Revised: We follow the experimental setting of Park et al.

Original: We did not use the GAN feature matching loss for
Revised: We did not use the GAN feature matching loss for

Original: OASIS, as we did not observe any improvement with it (see App. A.5), and used the VGG loss only for ablations with λ VGG = 10.
Revised: OASIS, as we did not observe any improvement with it (see App. A.5), and used the VGG loss only for ablations with λ VGG = 10.

Original: We did not experience any training instabilities and, thus, did not employ any extra stabilization techniques.
Revised: We did not experience any training instabilities and, thus, did not employ any extra stabilization techniques.

Original: All our models use an exponential moving average (EMA) of the generator weights with 0 .
Revised: All our models use an exponential moving average (EMA) of the generator weights with 0 .

Original: 9999 decay.
Revised: 9999 decay.

Original: For further training details refer to App.
Revised: For further training details refer to App.

Original: C.3.
Revised: C.3.

Original: [Table] [Table] Following prior work (Isola et al., 2017; Wang et al., 2018; Park et al., 2019; Liu et al., 2019), we evaluate models quantitatively on the validation set using the Fr ´ echet Inception Distance (FID) (Heusel et al., 2017) and mean Intersection-over-Union (mIoU).
Revised: Following prior work (Isola et al., 2017; Wang et al., 2018; Park et al., 2019; Liu et al., 2019), we evaluate models quantitatively on the validation set using the Fr´echet Inception Distance (FID) (Heusel et al., 2017) and mean Intersection-over-Union (mIoU).

Original: FID is known to be sensitive to both quality and diversity and has been shown to be well aligned with human judgement (Heusel et al., 2017).
Revised: FID is known to be sensitive to both quality and diversity and has been shown to be well aligned with human judgement (Heusel et al., 2017).

Revised: We show additional evaluation of quality and diversity with ”improved precision and recall” in App.

Original: While mIoU is used to assess the alignment of the generated image with the ground truth label map, computed via a pre-trained semantic segmentation network.
Revised: A.9. Mean IoU is used to assess the alignment of the generated image with the ground truth label map, computed via a pre-trained semantic segmentation network.

Original: We use UperNet101 (Xiao et al., 2018) for ADE20K, multi-scale DRN-D-

Revised: We use UperNet101 (Xiao et al., 2018) for ADE20K, multi-scale DRN-D-

Original: (Yu et al., 2017) for Cityscapes, and DeepLabV (Chen et al., 2015) for COCO-Stuff.
Revised: (Yu et al., 2017) for Cityscapes, and DeepLabV2 (Chen et al., 2015) for COCO-Stuff.

Original: We additionally propose to compare color and texture statistics between generated and real images on ADE20K to better understand how the perceptual loss inﬂuences performance.
Revised: We additionally propose to compare color and texture statistics between generated and real images on ADE20K to better understand how the perceptual loss inﬂuences performance.

Original: For this, we compute color histograms in LAB space and‘ measure the earth mover’s distance between the real and generated sets (Rubner et al., 2000).
Revised: For this, we compute color histograms in LAB space and measure the earth mover’s distance between the real and generated sets (Rubner et al., 2000).

Original: We measure the texture similarity to the real data as the χ 2 -distance between Local Binary Patterns histograms (Ojala et al., 1996).
Revised: We measure the texture similarity to the real data as the χ 2 -distance between Local Binary Patterns histograms (Ojala et al., 1996).

Original: As different classes have different color and texture distributions, we aggregate histogram distances separately per class and then take the mean.
Revised: As different classes have different color and texture distributions, we aggregate histogram distances separately per class and then take the mean.

Original: Lower values for the texture and color distances indicate a closer similarity to real data.
Revised: Lower values for the texture and color distances indicate a closer similarity to real data.

Original: 4. M AIN RESULTS
Revised: 4. M AIN RESULTS

Original: We use SPADE as our baseline, using the authors’ implementation 1 .
Revised: We use SPADE as our baseline, using the authors’ implementation 1 .

Original: For a fair comparison, we train this model without the feature matching loss and using EMA (Yazici et al., 2018) at test phase, which  we further refer to as SPADE+.
Revised: For a fair comparison, we train this model without the feature matching loss and using EMA (Yaz et al., 2018) at test phase, which [Table] [Table] we further refer to as SPADE+.

Original: We found that the feature matching loss has a negligible impact (see
Revised: We found that the feature matching loss has a negligible impact (see

Original: App. A.5), while EMA signiﬁcantly increases the performance for all metrics (see Table 1).
Revised: App. A.5), while EMA signiﬁcantly increases the performance for all metrics (see Table 1).

Original: OASIS outperforms the current state of the art on all datasets with an average improvement of
Revised: OASIS outperforms the current state of the art on all datasets with an average improvement of

Original: FID and 5 mIoU points (Table 1).
Revised: FID and 5 mIoU points (Table 1).

Original: Importantly, OASIS achieved the improvement via adversarial supervision alone.
Revised: Importantly, OASIS achieved the improvement via adversarial supervision alone.

Original: On the contrary, SPADE+ does not produce images of high visual quality without the perceptual loss, and struggles to learn the color and texture distribution of real images (Fig. 6).
Revised: On the contrary, SPADE+ does not produce images of high visual quality without the perceptual loss, and struggles to learn the color and texture distribution of real images (Fig. 6).

Original: A strong discriminator is the key factor for good performance: without a rich training signal from the discriminator, the SPADE+ generator has to learn through minimizing the VGG loss.
Revised: A strong discriminator is the key factor for good performance: without a rich training signal from the discriminator, the SPADE+ generator has to learn through minimizing the VGG loss.

Original: With the stronger OASIS discriminator, the perceptual loss does not overtake the generator supervision (see
Revised: With the stronger OASIS discriminator, the perceptual loss does not overtake the generator supervision (see

Original: App. A.2), allowing to produce images with the color and texture distribution closer to the real data.
Revised: App. A.2), allowing to produce images with the color and texture distribution closer to the real data.

Original: Fig.
Revised: Fig.

Original: 5 shows a qualitative comparison of our results to previous models.
Revised: 5 shows a qualitative comparison of our results to previous models.

Original: Our approach noticeably improves image quality, synthesizing ﬁner textures and more natural colors.
Revised: Our approach noticeably improves image quality, synthesizing ﬁner textures and more natural colors.

Original: With the powerful feedback from the discriminator, OASIS is able to learn the appearance of small or rarely occurring semantic classes (which is reﬂected in the per-class IoU scores presented in App. A.3), producing plausible results even for complex scenes with rare classes and reducing unnatural artifacts.
Revised: With the powerful feedback from the discriminator, OASIS is able to learn the appearance of small or rarely occurring semantic classes (which is reﬂected in the per-class IoU scores presented in App. A.3), producing plausible results even for complex scenes with rare classes and reducing unnatural artifacts.

Original: Multi-modal image synthesis.
Revised: Multi-modal image synthesis.

Original: In contrast to previous work, OASIS can produce diverse images by directly re-sampling input 3D noise.
Revised: In contrast to previous work, OASIS can produce diverse images by directly re-sampling input 3D noise.

Original: As 3D noise modulates features directly at every layer of the generator at different scales, matching their resolution, it affects both global and local characteristics of the image.
Revised: As 3D noise modulates features directly at every layer of the generator at different scales, matching their resolution, it affects both global and local characteristics of the image.

Original: Thus, the noise can be sampled globally, varying the whole image, or locally, resulting in the selected object change while preserving the rest of the scene (see Fig. 2).
Revised: Thus, the noise can be sampled globally, varying the whole image, or locally, resulting in the selected object change while preserving the rest of the scene (see Fig. 2).

Original: To measure the variation in the multi-modal generation, we evaluate MS-SSIM (Wang et al., 2003) and LPIPS (Zhang et al., 2018b) between images generated from the same label map.
Revised: To measure the variation in the multi-modal generation, we evaluate MS-SSIM (Wang et al., 2003) and LPIPS (Zhang et al., 2018b) between images generated from the same label map.

Original: We generate 20 images and compute the mean pairwise scores, and then average over all label maps.
Revised: We generate 20 images and compute the mean pairwise scores, and then average over all label maps.

Original: The lower the MS-SSIM and the higher the LPIPS scores, the more diverse the generated images are.
Revised: The lower the MS-SSIM and the higher the LPIPS scores, the more diverse the generated images are.

Original: To assess the effect of the perceptual loss and the noise sampling on diversity, we train SPADE+ with 3D noise or the image encoder, and with or without the perceptual loss.
Revised: To assess the effect of the perceptual loss and the noise sampling on diversity, we train SPADE+ with 3D noise or the image encoder, and with or without the perceptual loss.

Original: Table 2 shows that OASIS, without perceptual loss, improves over SPADE+ with the image encoder, both in terms of image diversity (MS-SSIM, LPIPS) and quality (mean FID, mIoU across 20 realizations).
Revised: Table 2 shows that OASIS, without perceptual loss, improves over SPADE+ with the image encoder, both in terms of image diversity (MS-SSIM, LPIPS) and quality (mean FID, mIoU across 20 realizations).

Original: Using 3D noise further increases diversity for SPADE+.
Revised: Using 3D noise further increases diversity for SPADE+.

Original: However, a strong quality-diversity tradeoff exists for SPADE+:
Revised: However, a strong quality-diversity tradeoff exists for SPADE+:

Original: 3D noise improves diversity at the cost of quality, and the perceptual loss improves quality at the cost of diversity.
Revised: 3D noise improves diversity at the cost of quality, and the perceptual loss improves quality at the cost of diversity.

Original: For OASIS, the VGG loss also reduces diversity but does not noticeably affect quality.
Revised: For OASIS, the VGG loss also reduces diversity but does not noticeably affect quality.

Original: Note that in our experiments LabelMix does not notably affect diversity (see App. A.1).
Revised: Note that in our experiments LabelMix does not notably affect diversity (see App. A.1).

Original: 4. A BLATIONS
Revised: 4. A BLATIONS

Original: [Table]

Original: We conduct ablations on ADE20K to evaluate our proposed changes.
Revised: We conduct ablations on ADE20K to evaluate our proposed changes.

Original: The main ablation shows the impact of our new discriminator, lighter generator, LabelMix and 3D noise.
Revised: The main ablation shows the impact of our new discriminator, lighter generator, LabelMix and 3D noise.

Original: Further ablations are concerned with architecture changes and the label map encodings in the discriminator, where for fair comparison we use settings without 3D noise and LabelMix.
Revised: Further ablations are concerned with architecture changes and the label map encodings in the discriminator, where for fair comparison we use no 3D noise and LabelMix.

Original: [Table]
Revised: [Table]

Original: Main ablation.
Revised: Main ablation.

Original: Table 3 shows that SPADE+ scores low on the image quality metrics without the perceptual loss.
Revised: Table 3 shows that SPADE+ scores low on the image quality metrics without the perceptual loss.

Original: Replacing the SPADE+ discriminator with the OASIS discriminator, while keeping the generator ﬁxed, improves FID and mIoU by more than 30 points.
Revised: Replacing the SPADE+ discriminator with the OASIS discriminator, while keeping the generator ﬁxed, improves FID and mIoU by more than 30 points.

Original: Changing the SPADE+ generator to the lighter OASIS generator leads to a negligible degradation of 0 .
Revised: Changing the SPADE+ generator to the lighter OASIS generator leads to a negligible degradation of 0 .

Original: 3 in FID and 0 .
Revised: 3 in FID and 0 .

Original: 5 in mIoU. With LabelMix FID improves further by ∼ 1 point (more ablations on LabelMix in App.
Revised: 5 in mIoU. With LabelMix FID improves further by ∼ 1 point (more ablations on LabelMix in App.

Original: A.4).
Revised: A.4).

Original: Adding 3D noise improves FID but degrades mIoU, as diversity complicates the task of the pre-trained semantic segmentation network used to compute the score.
Revised: Adding 3D noise improves FID but degrades mIoU, as diversity complicates the task of the pre-trained semantic segmentation network used to compute the score.

Original: For OASIS the perceptual loss deteriorates FID by more than 2 points, but improves mIoU. Overall, without the perceptual loss the new discriminator is the key to the performance boost over SPADE+.
Revised: For OASIS the perceptual loss deteriorates FID by more than 2 points, but improves mIoU. Overall, without the perceptual loss the new discriminator is the key to the performance boost over SPADE+.

Original: [Table]
Revised: [Table]

Original: Ablation on the discriminator architecture.
Revised: Ablation on the discriminator architecture.

Original: We train the OASIS generator with three alternative discriminators: the original multi-scale PatchGAN consisting of two networks, a single-scale PatchGAN, and a ResNet-based discriminator, corresponding to the encoder of the U-Net shaped OASIS discriminator.
Revised: We train the OASIS generator with three alternative discriminators: the original multi-scale PatchGAN consisting of two networks, a single-scale PatchGAN, and a ResNet-based discriminator, corresponding to the encoder of the U-Net shaped OASIS discriminator.

Original: Table 4 shows that the alternative discriminators only perform well with perceptual supervision, while the OASIS discriminator achieves superior performance independent of it.
Revised: Table 4 shows that the alternative discriminators only perform well with perceptual supervision, while the OASIS discriminator achieves superior performance independent of it.

Original: The single-scale discriminators even collapse without the perceptual loss (red in Table 4).
Revised: The single-scale discriminators even collapse without the perceptual loss (highlighted in red in Table 4) [Table] [Table]

Original: Ablation on the label map encoding.
Revised: Ablation on the label map encoding.

Original: We study four different label map encodings: input concatenation, as in SPADE, projection conditioned on the label map (Miyato & Koyama, 2018), employing label maps as ground truth for the N + 1 segmentation loss, or for the class-balanced N + 1 loss (Sec.
Revised: We study four different label map encodings: input concatenation, as in SPADE, projection conditioned on the label map (Miyato & Koyama, 2018), employing label maps as ground truth for the N + 1 segmentation loss, or for the class-balanced N + 1 loss (see Sec.

Original: As shown in Table 5, input concatenation is not sufﬁcient without additional perceptual loss supervision, leading to training collapse.
Revised: As shown in Table 5, input concatenation is not sufﬁcient without additional perceptual loss supervision, leading to training collapse.

Original: Without perceptual loss, the N + 1 loss outperforms the input concatenation and the projection in both  FID and mIoU.
Revised: Without perceptual loss, the N + 1 loss outperforms the input concatenation and the projection in both the FID and mIoU metrics.

Original: The class balancing noticeably improves mIoU due to better supervision for rarely occurring  classes.
Revised: The class balancing noticeably improves mIoU due to better supervision for rarely occurring semantic classes.

Original: More ablations can be found in App. A.
Revised: More ablations can be found in App. A.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this work we propose OASIS, a semantic image synthesis model that only relies on adversarial supervision to achieve high ﬁdelity image synthesis.
Revised: In this work we propose OASIS, a semantic image synthesis model that only relies on adversarial supervision to achieve high ﬁdelity image synthesis.

Original: In contrast to previous work, our model eliminates the need for a perceptual loss, which often imposes extra constraints on image quality and diversity.
Revised: In contrast to previous work, our model eliminates the need for a perceptual loss, which often imposes extra constraints on image quality and diversity.

Original: This is achieved via detailed spatial and semantic-aware supervision from our novel segmentation-based discriminator, which uses semantic label maps as ground truth for training.
Revised: This is achieved via detailed spatial and semantic-aware supervision from our novel segmentation-based discriminator, which uses semantic label maps as ground truth for training.

Original: With this powerful discriminator, OASIS can easily generate diverse multi-modal outputs by re-sampling the 3D noise, both globally and locally, allowing to change the appearance of the whole scene and of individual objects.
Revised: With this powerful discriminator, OASIS can easily generate diverse multi-modal outputs by re-sampling the 3D noise, both globally and locally, allowing to change the appearance of the whole scene and of individual objects.

Original: OASIS signiﬁcantly improves over the state of the art in terms of image quality and diversity, while being simpler and more lightweight than previous methods. 
Revised: OASIS signiﬁcantly improves over the state of the art in terms of image quality and diversity, while being simpler and more lightweight than previous methods. A CKNOWLEDGEMENT

Revised: J¨urgen Gall has been supported by the Deutsche Forschungsgemeinschaft (DFG, German Research

Revised: Foundation) under Germany’s Excellence Strategy - EXC 2070 -390732324.

Original: A PPENDIX
Revised: A PPENDIX

Original: This supplementary material to the main paper is structured as follows:
Revised: This supplementary material to the main paper is structured as follows:

Original: A Additional quantitative results.
Revised: A Additional quantitative results.

Original: A.1 Main ablation study on two datasets.
Revised: A.1 Main ablation study on two datasets.

Original: A.

Revised: A.

Original: The inﬂuence of the perceptual loss on training dynamics.
Revised: The inﬂuence of the perceptual loss on training dynamics.

Original: A.3 Per-class IoU scores across different datasets.
Revised: A.3 Per-class IoU scores across different datasets.

Original: A.4 Comparing LabelMix and CutMix for consistency regularization.
Revised: A.4 Comparing LabelMix and CutMix for consistency regularization.

Original: A.

Revised: A.

Original: Ablation on the Feature Matching loss.
Revised: Ablation on the Feature Matching loss.

Original: A.6 Ablation on using multiple OASIS discriminators.
Revised: A.6 Ablation on using multiple OASIS discriminators.

Original: A.7 Ablation on noise sampling strategies during training.
Revised: A.7 Ablation on noise sampling strategies during training.

Original: A.8 Additional experiments on COCO-stuff.
Revised: A.8 Additional experiments on COCO-stuff.

Original: A.9Additional image quality metrics.
Revised: A.9Additional image quality metrics.

Original: B: Additional qualitative results.
Revised: B: Additional qualitative results.

Original: B.1 Visual comparison of OASIS to other works.
Revised: B.1 Visual comparison of OASIS to other works.

Original: B.2 Multi-modal synthesis results for different label maps.
Revised: B.2 Multi-modal synthesis results for different label maps.

Original: B.3 Interpolations between multi-modal images in latent space.
Revised: B.3 Interpolations between multi-modal images in latent space.

Original: [-B.
 Repurposing the trained discriminator as an image segmenter.-]
Revised: [-B.
 Repurposing the trained discriminator as an image segmenter.-]B.4 Application to unlabelled data.

Original: B.5 Additional visual LabelMix examples.
Revised: B.5 Additional visual LabelMix examples.

Original: C:
Revised: C:

Original: A detailed description of the OASIS architecture and its training details.
Revised: A detailed description of the OASIS architecture and its training details.

Original: C.1 Discriminator architecture.
Revised: C.1 Discriminator architecture.

Original: C.2 Generator architecture.
Revised: C.2 Generator architecture.

Original: C.3 Learning objective and training details.
Revised: C.3 Learning objective and training details.

Original: A Q UANTITATIVE RESULTS A. S UMMARIZED MAIN ABLATION OVER TWO DATASETS
Revised: A Q UANTITATIVE RESULTS A. S UMMARIZED MAIN ABLATION OVER TWO DATASETS

Original: [Table]
Revised: [Table]

Original: +
Revised: +

Original: [Table]
Revised: [Table]

Original: + OASIS D, G
Revised: + OASIS D, G

Original: [Table]
Revised: [Table]

Revised: + 3D noise

Revised: [Table]

Original: + LabelMix
Revised: + LabelMix

Original: [Table]
Revised: [Table]

Original: +

Original: [Table] [Table]
Revised: [Table] [Table]

Original: In Table A we present a summarized version of our ablations for the ADE20K and Cityscapes dataset.
Revised: In Table A we present a summarized version of our ablations for the ADE20K and Cityscapes dataset.

Original: The following observations can be made:
Revised: The following observations can be made:

Original: (1) Looking at the 2nd row of Table A, we see that the main performance gain comes from the discriminator design (major) (OASIS D,G).
Revised: (1) Looking at the 2nd row of Table A, we see that the main performance gain comes from the discriminator design (major) (OASIS D,G).

Original: The OASIS generator is a lighter version of the SPADE generator, which does not result in a performance improvement (Table 3), but has signiﬁcantly less parameters.
Revised: The OASIS generator is a lighter version of the SPADE generator, which does not result in a performance improvement (Table 3), but has signiﬁcantly less parameters.

Original: A second source of improvement is LabelMix.
Revised: A second source of improvement is LabelMix.

Original: The mIoU can drop when  noise is added, as diversity complicates the task of the pre-trained semantic segmentation network that is used to compute the mIoU score.
Revised: The mIoU can drop when 3D noise is added, as diversity complicates the task of the pre-trained semantic segmentation network that is used to compute the mIoU score.

Original: Note that the purpose of noise is not to improve the image quality (FID) but to improve diversity (MS-SSIM).
Revised: Note that the purpose of noise is not to improve the image quality (FID) but to improve diversity (MS-SSIM).

Original: In effect, the FID numbers in Table A are slightly different from Table 1 which was evaluated without noise for a fair comparison with the other methods that also do not employ noise.

Original: The perceptual loss can hurt performance and diversity by biasing the generator towards ImageNet, hence the target distribution cannot be fully recreated.
Revised: The perceptual loss can hurt performance and diversity by biasing the generator towards ImageNet, as in this case the target distribution is more difﬁcult to recreate fully.

Original: By punishing diversity, the perceptual loss  helps to raise the mIoU.
Revised: By punishing diversity, the perceptual loss encourages generating images with more standard semantic features This facilitates the task of external pretrained segmenters, and consequently helps to raise the mIoU metric.

Original: A. T HE INFLUENCE OF VGG ON TRAINING DYNAMICS
Revised: A. T HE INFLUENCE OF VGG ON TRAINING DYNAMICS

Original: Table 1 and Figure 1 illustrate, that performance of SPADE+ strongly depends on the perceptual loss.
Revised: Table 1 and Figure 1 illustrate that performance of SPADE+ strongly depends on the perceptual loss.

Original: In contrast, OASIS achieves high quality without this loss (Table 1).
Revised: In contrast, OASIS achieves high quality without this loss (Table 1).

Original: We ﬁnd the explanation in the fact, that  SPADE+ Patch-GAN discriminator does not provide  strong training signal for the generator.
Revised: We ﬁnd the explanation in the fact, that the SPADE+ Patch-GAN discriminator does not provide a strong training signal for the generator.

Original: At the absence of strong supervision from the discriminator, the generator resorts to learning mostly from the VGG loss.
Revised: At the absence of strong supervision from the discriminator, the generator resorts to learning mostly from the VGG loss.

Original: The loss curves in Fig.
Revised: The loss curves in Fig.

Original: A support this ﬁnding: throughout the training the SPADE+ model focuses on minimizing the VGG loss, keeping the adversarial generator loss more or less constant.
Revised: A support this ﬁnding: throughout the training the SPADE+ model focuses on minimizing the VGG loss, keeping the adversarial generator loss more or less constant.

Original: In contrast, OASIS signiﬁcantly improves adversarial generator loss during training, learning to fool the segmentation-based OASIS discriminator.
Revised: In contrast, OASIS signiﬁcantly improves adversarial generator loss during training, learning to fool the segmentation-based OASIS discriminator.

Original: That indicates a better adversarial balance, when the generator learns semantically meaningful features that the segmenter judges as real.
Revised: That indicates a better adversarial balance, when the generator learns semantically meaningful features that the segmenter judges as real.

Original: The difference in scales of G loss for models comes from different objectives, since
Revised: The difference in scales of G loss for models comes from different objectives, since

Original: SPADE+ optimizes binary cross entropy, and OASIS minimizes multi-class cross entropy with N +classes.
Revised: SPADE+ optimizes binary cross entropy, and OASIS minimizes multi-class cross entropy with N +classes.

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: As seen in Table 1 in the main paper, OASIS signiﬁcantly outperforms previous approaches in mIoU. We found that the improvement comes mainly from the better IoU scores achieved for lessrepresented semantic classes.
Revised: As seen in Table 1 in the main paper, OASIS signiﬁcantly outperforms previous approaches in mIoU. We found that the improvement comes mainly from the better IoU scores achieved for lessrepresented semantic classes.

Original: To illustrate the gain, we report per-class IoU scores on ADE20k,
Revised: To illustrate the gain, we report per-class IoU scores on ADE20k,

Original: COCO-Stuff and Cityscapes in Tables B, C and D.
Revised: COCO-Stuff and Cityscapes in Tables B, C and D.

Original: For visualization purposes, we sorted the semantic classes of all datasets, ordering by their pixel-wise frequency in the training images.
Revised: For visualization purposes, we sorted the semantic classes of all datasets, ordering by their pixel-wise frequency in the training images.

Original: Taking ADE20k as an example,the Table B highlights that the relative gain in mIoU is especially high for the group of less-represented semantic classes, that cover less than 3% of all the images.
Revised: Taking ADE20k as an example, Table B highlights that the relative gain in mIoU is especially high for the group of less-represented semantic classes, that cover less than 3% of all the images.

Original: For these rare classes, the relative gain over the baseline exceeds 40% .
Revised: For these rare classes the relative gain over the baseline exceeds 40% .

Original: We found that the gain majorly comes from the per-class balancing applied in the OASIS loss function.
Revised: We found that the gain majorly comes from the per-class balancing applied in the OASIS loss function.

Original: In order to illustrate this effect, we train OASIS without the proposed balancing.
Revised: In order to illustrate this effect, we train OASIS without the proposed balancing.

Original: Table B reveals,that this baseline reaches a bit higher score for frequent classes, but shows worse performance for the rarely occurring ones.
Revised: Table B reveals, this baseline reaches a bit higher score for frequent classes, but shows worse performance for the rarely occurring ones.

Original: This is expected, as the balancing down-weights the objects met frequently while up-weights infrequent classes.
Revised: This is expected, as the balancing down-weights the objects met frequently while up-weights infrequent classes.

Original: We thus conclude that the balancing draws the attention of the discriminator to rarely occurring semantic classes, which results in a much higher quality of the generation.
Revised: We thus conclude that the balancing draws the attention of the discriminator to rarely occurring semantic classes, which results in a much higher quality of the generation.

Original: A. A BLATION ON L ABEL M IX
Revised: A. A BLATION ON L ABEL M IX

Original: Consistency regularization for the segmentation output of the discriminator requires a method of generating binary masks.
Revised: Consistency regularization for the segmentation output of the discriminator requires a method of generating binary masks.

Original: Therefore, we compare the effectiveness of CutMix (Yun et al., 2019) and our proposed LabelMix.
Revised: Therefore, we compare the effectiveness of CutMix (Yun et al., 2019) and our proposed LabelMix.

Original: Both methods produce binary masks, but only LabelMix respects the boundaries between semantic classes in the label map.
Revised: Both methods produce binary masks, but only LabelMix respects the boundaries between semantic classes in the label map.

Original: Table E compares the FID and mIoU scores of
Revised: Table E compares the FID and mIoU scores of

Original: OASIS trained with both methods on the Cityscapes dataset.
Revised: OASIS trained with both methods on the Cityscapes dataset.

Original: It can be seen that LabelMix improves both FID ( 51 . 5 vs. 47 .
Revised: It can be seen that LabelMix improves both FID ( 51 . 5 vs. 47 .

Original: 7 ) and mIoU ( 66 . 3 vs. 69 . 3 ), in comparison to OASIS without consistency regularization.
Revised: 7 ) and mIoU ( 66 . 3 vs. 69 . 3 ), in comparison to OASIS without consistency regularization.

Original: CutMix-based consistency regularization only improves the mIoU ( 66 .
Revised: CutMix-based consistency regularization only improves the mIoU ( 66 .

Original: 3 vs. 67 . 4 ),
Revised: 3 vs. 67 . 4 ),

Original: [Table]
Revised: [Table]

Original: [Table]

Original:  denotes the 
Revised: Bold denotes the best performance.

Original: [Table]
Revised: [Table]

Original: performance

Original: [Table] [Table]
Revised: [Table] [Table]

Original: but not as much as LabelMix ( 69 . 3 ).
Revised: but not as much as LabelMix ( 69 . 3 ).

Original: We suspect that since the images are already partitioned through the label map, an additional partition through CutMix results in a dense patchwork of areas that differ by semantic class and real-fake class identity.
Revised: We suspect that since the images are already partitioned through the label map, an additional partition through CutMix results in a dense patchwork of areas that differ by semantic class and real-fake class identity.

Original: This may introduce additional label noise during training for the discriminator.
Revised: This may introduce additional label noise during training for the discriminator.

Original: To avoid such inconsistency between semantic classes and real-fake identity, the mask of LabelMix is generated according to the label map, providing natural borders between semantic regions, so that the real and fake objects are placed side-by-side without interfering each other.
Revised: To avoid such inconsistency between semantic classes and real-fake identity, the mask of LabelMix is generated according to the label map, providing natural borders between semantic regions, so that the real and fake objects are placed side-by-side without interfering each other.

Original: Under LabelMix regularization, the generator is encouraged to respect the natural semantic class boundaries, improving pixel-level realism while also considering the class segment shapes.
Revised: Under LabelMix regularization, the generator is encouraged to respect the natural semantic class boundaries, improving pixel-level realism while also considering the class segment shapes.

Original: A. A BLATION ON FEATURE MATCHING LOSS
Revised: A. A BLATION ON FEATURE MATCHING LOSS

Original: We measure the effect of the feature matching loss (FM) in the absence and presence of the perceptual loss (VGG).
Revised: We measure the effect of the feature matching loss (FM) in the absence and presence of the perceptual loss (VGG).

Original: Table F and G present the results for OASIS on Cityscapes and SPADE+ on
Revised: Table F and G present the results for OASIS on Cityscapes and SPADE+ on

Original: ADE20K.
Revised: ADE20K.

Original: For both SPADE+ and OASIS we observe that the feature matching loss does only affect the FID notably when no perceptual loss is used.
Revised: For both SPADE+ and OASIS we observe that the feature matching loss does only affect the FID notably when no perceptual loss is used.

Original: In the case where no perceptual loss is used, we observe that the feature matching prolongs the time until SPADE+ collapses, resulting in a better FID score ( 49 .
Revised: In the case where no perceptual loss is used, we observe that the feature matching prolongs the time until SPADE+ collapses, resulting in a better FID score ( 49 .

Original: 7 vs 60 . 7 ).
Revised: 7 vs 60 . 7 ).

Original: Consequently, the mIoU also improves.
Revised: Consequently, the mIoU also improves.

Original: Hence, the role of the FM loss in the training of SPADE+ is to stabilize the training through additional self-supervision.
Revised: Hence, the role of the FM loss in the training of SPADE+ is to stabilize the training through additional self-supervision.

Original: This observation is in line with the general observation that
Revised: This observation is in line with the general observation that

Original: SPADE and other semantic image synthesis models require the help of additional losses because the adversarial supervision through the discriminator is not strong enough.
Revised: SPADE and other semantic image synthesis models require the help of additional losses because the adversarial supervision through the discriminator is not strong enough.

Original: In comparison, we did not observe any training collapses in OASIS, despite not using any extra losses.
Revised: In comparison, we did not observe any training collapses in OASIS, despite not using any extra losses.

Original: For OASIS, the feature matching loss results in a worse FID (by 0 .
Revised: For OASIS, the feature matching loss results in a worse FID (by 0 .

Original: 8 points) in the absence of the perceptual loss.
Revised: 8 points) in the absence of the perceptual loss.

Original: We also observe a degradation of 1 . 1 mIoU points through the FM loss, in the case where the perceptual supervision is present.
Revised: We also observe a degradation of 1 . 1 mIoU points through the FM loss, in the case where the perceptual supervision is present.

Original: This indicates that the FM loss negatively affects the strong supervision from the semantic segmentation adversarial loss of OASIS.
Revised: This indicates that the FM loss negatively affects the strong supervision from the semantic segmentation adversarial loss of OASIS.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A. A BLATION ON USING MORE THAN ONE OASIS DISCRIMINATOR
Revised: A. A BLATION ON USING MORE THAN ONE OASIS DISCRIMINATOR

Original: A major difference between SPADE and OASIS is that OASIS employs only one discriminator, while SPADE uses two PatchGAN discriminators at different scales.
Revised: A major difference between SPADE and OASIS is that OASIS employs only one discriminator, while SPADE uses two PatchGAN discriminators at different scales.

Original: Naturally, the question arises how OASIS performs with two discriminators at different scales, as in SPADE.
Revised: Naturally, the question arises how OASIS performs with two discriminators at different scales, as in SPADE.

Original: For this, Table H presents the FID and mIoU performance of OASIS with two discriminators operating at scales 1 and 0.5 on Cityscapes.
Revised: For this, Table H presents the FID and mIoU performance of OASIS with two discriminators operating at scales 1 and 0.5 on Cityscapes.

Original: One can see that an additional discriminator at scale 0.5 does not improve performance, but slightly worsens it.
Revised: One can see that an additional discriminator at scale 0.5 does not improve performance, but slightly worsens it.

Original: The reason that no performance gain is visible is that the OASIS discriminator already encodes multi-scale information through its U-Net structure: skip connections between encoder, decoder and individual blocks integrate information at all scales.
Revised: The reason that no performance gain is visible is that the OASIS discriminator already encodes multi-scale information through its U-Net structure: skip connections between encoder, decoder and individual blocks integrate information at all scales.

Original: In contrast,
Revised: In contrast,

Original: SPADE requires two discriminators to capture information at 
Revised: SPADE requires two discriminators to capture information at different

Original: [Table] [Table]
Revised: [Table] [Table]

Original:  A BLATION ON NOISE SAMPLING STRATEGIES DURING TRAINING
Revised: A. A BLATION ON NOISE SAMPLING STRATEGIES DURING TRAINING

Original: Our 3D noise can contain the same sampled vector for each pixel, or different vectors for different regions.
Revised: Our 3D noise can contain the same sampled vector for each pixel, or different vectors for different regions.

Original: This allows for different noise sampling schemes during training.
Revised: This allows for different noise sampling schemes during training.

Original: Table I shows the effect of using different methods of sampling 3D noise for different locations during training: Image-level sampling creates one global 1D noise vector and replicates it along the height and width of the label map to create a 3D noise tensor.
Revised: Table I shows the effect of using different methods of sampling 3D noise for different locations during training: Image-level sampling creates one global 1D noise vector and replicates it along the height and width of the label map to create a 3D noise tensor.

Original: Region-level sampling relies on generating one 1D noise vector per label, and stacking them in 3D to match the height and width of the label map.
Revised: Region-level sampling relies on generating one 1D noise vector per label, and stacking them in 3D to match the height and width of the label map.

Original: Pixel-level sampling creates different noise for every spatial position, with no replication taking place.
Revised: Pixel-level sampling creates different noise for every spatial position, with no replication taking place.

Original: Mix switches between image-level and region-level sampling via a coin ﬂip decision at every training step.
Revised: Mix switches between image-level and region-level sampling via a coin ﬂip decision at every training step.

Original: With no obvious winner in performance, we choose the simplest scheme (image-level) for our experiments.
Revised: With no obvious winner in performance, we choose the simplest scheme (image-level) for our experiments.

Original: By choosing image-level sampling for training, we thus generate a single 1D latent noise vector of size 64, broadcast it to 64xHxW and concatenate with the label map (NxHxW).
Revised: By choosing image-level sampling for training, we thus generate a single 1D latent noise vector of size 64, broadcast it to 64xHxW and concatenate with the label map (NxHxW).

Original: This new composite tensor is used as input to the 1st generator layer and at all SPADE-norm layers.
Revised: This new composite tensor is used as input to the 1st generator layer and at all SPADE-norm layers.

Original: The noise is not ignored for the following reasons:
Revised: The noise is not ignored for the following reasons:

Original: (1) The noise modulates the activations directly at every layer, so it is very hard to ignore.
Revised: (1) The noise modulates the activations directly at every layer, so it is very hard to ignore.

Original: Here, it is important to emphasize how the noise is used:
Revised: Here, it is important to emphasize how the noise is used:

Original: For SPADE it was observed that label maps are paid more attention to if they are used for location-sensitive conditional batch normalization (CBN).
Revised: For SPADE it was observed that label maps are paid more attention to if they are used for location-sensitive conditional batch normalization (CBN).

Original: Analogously, we observe that the noise is also paid more attention to when it is injected via CBN.
Revised: Analogously, we observe that the noise is also paid more attention to when it is injected via CBN.

Original: Like label maps, which are 3D tensors of stacked one-hot vectors, we stack noise vectors into a 3D tensor of the same dimensions.
Revised: Like label maps, which are 3D tensors of stacked one-hot vectors, we stack noise vectors into a 3D tensor of the same dimensions.

Original: Thus, similarly that SPADE is spatially sensitive to labels, OASIS is spatially sensitive to both labels and noise.
Revised: Thus, in the same way that SPADE is spatially sensitive to labels, OASIS is spatially sensitive to both labels and noise.

Original: The 3D broadcasting strategy provides a spatially uniform signal making it easy to embed semantic meaning into the latent code (see interpolations, Fig.
Revised: The 3D broadcasting strategy provides a spatially uniform signal making it easy to embed semantic meaning into the latent code (see interpolations, Fig.

Original: I , J).
Revised: I , J).

Original: As noise modulates features at different scales in the generator, matching their resolution, it affects both global and local characteristics.
Revised: As noise modulates features at different scales in the generator, matching their resolution, it affects both global and local characteristics.

Original: This is why a generator trained with image-level noise can perform region-level manipulation at inference (Fig.
Revised: This is why a generator trained with image-level noise can perform region-level manipulation at inference (Fig.

Original: F, H).
Revised: F, H).

Original: However, more evolved spatial noise sampling schemes can be explored in the future.
Revised: However, more evolved spatial noise sampling schemes can be explored in the future.

Original: [Table]
Revised: [Table]

Original: A. A DDITIONAL EXPERIMENTS ON COCO- STUFF
Revised: A DDITIONAL EXPERIMENTS ON COCO- STUFF

Original: [Table]
Revised: [Table]

Original: We performed all our extensive ablations on
Revised: We performed all our extensive ablations on

Original: ADE20K and Cityscapes, due to their shorter training time.
Revised: ADE20K and Cityscapes, due to their shorter training time.

Original: Training on ADE20K and Cityscapes takes circa 10 days on 4 Tesla V100 GPUs while training on COCO-stuff can stretch to 4 weeks.
Revised: Training on ADE20K and Cityscapes takes circa 10 days on 4 Tesla V100 GPUs while training on COCO-stuff can stretch to 4 weeks.

Original: Therefore, we only executed essential experiments on COCOstuff.
Revised: Therefore, we only executed essential experiments on COCOstuff.

Original: We compare the results of these experiments in Table J. For SPADE+, it can be seen that without the external perceptual supervision of VGG, training collapses (with FID 99.1 at the best checkpoint before collapse).
Revised: We compare the results of these experiments in Table J. For SPADE+, it can be seen that without the external perceptual supervision of VGG, training collapses (with FID 99.1 at the best checkpoint before collapse).

Original: In contrast, for OASIS image quality is better without VGG (16.7 vs 18.0 FID).
Revised: In contrast, for OASIS image quality is better without VGG (16.7 vs 18.0 FID).

Original: When 3D noise is added to OASIS, sampling of multi-modal images is enabled (0.61 vs 1.0 MSSSIM), with very similar performance in synthesis quality (17.0 vs 16.7 FID) and slightly worse mIoU (44.1 vs 45.5 mIoU) due to the increased variation of generated samples, as the semantic segmentation task of the pre-trained segmentation network becomes harder.
Revised: When 3D noise is added to OASIS, sampling of multi-modal images is enabled (0.61 vs 1.0 MSSSIM), with very similar performance in synthesis quality (17.0 vs 16.7 FID) and slightly worse mIoU (44.1 vs 45.5 mIoU) due to the increased variation of generated samples, as the semantic segmentation task of the pre-trained segmentation network becomes harder.

Original: A. A DDITIONAL EVALUATION METRICS
Revised: A. A DDITIONAL EVALUATION METRICS

Original: Currently, the FID score is the most widely adopted metric for quantifying image quality of GAN models.
Revised: Currently, the FID score is the most widely adopted metric for quantifying image quality of GAN models.

Original: However, it is often argued that the FID score does not adequately disentangle synthesis quality and diversity (Kynk¨a¨anniemi et al., 2019).
Revised: However, it is often argued that the FID score does not adequately disentangle synthesis quality and diversity (Kynk¨a¨anniemi et al., 2019).

Original: Recently, a series of metrics have been proposed to address this issue by measuring scores related to the concepts of precision and recall (Ravuri &
Revised: Recently, a series of metrics have been proposed to address this issue by measuring scores related to the concepts of precision and recall (Ravuri &

Original: Vinyals, 2019; Shmelkov et al., 2018; Sajjadi et al., 2018; Kynk¨a¨anniemi et al., 2019).
Revised: Vinyals, 2019; Shmelkov et al., 2018; Sajjadi et al., 2018; Kynk¨a¨anniemi et al., 2019).

Original: Here, we have a closer look at the ”improved precision and recall” score proposed by (Kynk¨a¨anniemi et al., 2019), where precision is the probability that a generated image falls into the estimated support of the real image distribution, and recall is the probability that a real image falls into the estimated support of the generator distribution.
Revised: Here, we have a closer look at the ”improved precision and recall” score proposed by (Kynk¨a¨anniemi et al., 2019), where precision is the probability that a generated image falls into the estimated support of the real image distribution, and recall is the probability that a real image falls into the estimated support of the generator distribution.

Original: Precision and recall can be interpreted as sample quality and diversity.
Revised: Precision and recall can be interpreted as sample quality and diversity.

Original: Table K presents a comparison of precision (P) and recall R) between SPADE+ and OASIS.
Revised: Table K presents a comparison of precision (P) and recall R) between SPADE+ and OASIS.

Original: It can be seen that OASIS outperforms SPADE+ both in terms of image quality and variety.
Revised: It can be seen that OASIS outperforms SPADE+ both in terms of image quality and variety.

Original: [Table]
Revised: [Table]

Original: B Q UALITATIVE RESULTS B. C OMPARISON TO OTHER METHODS
Revised: B Q UALITATIVE RESULTS B. C OMPARISON TO OTHER METHODS

Original: In this section we present a visual comparison between OASIS and other semantic image synthesis methods.
Revised: In this section we present a visual comparison between OASIS and other semantic image synthesis methods.

Original: Firstly, we show images generated by SPADE (Park et al., 2019), CC-FPSE (Liu et al., 2019) and OASIS on ADE20k, COCO-Stuff, and Cityscapes (in Figures B, C, and D, respectively).
Revised: Firstly, we show images generated by SPADE (Park et al., 2019), CC-FPSE (Liu et al., 2019) and OASIS on ADE20k, COCO-Stuff, and Cityscapes (in Figures B, C, and D, respectively).

Original: A further comparison for SPADE, SPADE+ and OASIS is presented in
Revised: A further comparison for SPADE, SPADE+ and OASIS is presented in

Original: Figure E. We observed that
Revised: Figure E. We observed that

Original: OASIS often produces more visually plausible images than the previous methods.
Revised: OASIS often produces more visually plausible images than the previous methods.

Original: Our method commonly produces ﬁner textures, especially for complex and large semantic objects, e.g building facades, mountains, water.
Revised: Our method commonly produces ﬁner textures, especially for complex and large semantic objects, e.g building facades, mountains, water.

Original: We also note that OASIS usually generates brighter and more diverse colors, compared to other methods.
Revised: We also note that OASIS usually generates brighter and more diverse colors, compared to other methods.

Original: As we showed in Section 4 in the main paper, the diversity in colors partially comes from the fact that the feature space of the OASIS generator is not constrained by the VGG loss.
Revised: As we showed in Section 4 in the main paper, the diversity in colors partially comes from the fact that the feature space of the OASIS generator is not constrained by the VGG loss.

Original: We observed that images, generated by SPADE and CC-FPSE, typically have closer colors, while
Revised: We observed that images, generated by SPADE and CC-FPSE, typically have closer colors, while

Original: OASIS frequently generates objects with completely different color tones.
Revised: OASIS frequently generates objects with completely different color tones.

Original: This also forms one of the failure modes of our approach, when the colors of objects fall out of distribution and seem unnatural (see Figure G).
Revised: This also forms one of the failure modes of our approach, when the colors of objects fall out of distribution and seem unnatural (see Figure G).

Original: B. M ULTI - MODAL IMAGE SYNTHESIS
Revised: B. M ULTI - MODAL IMAGE SYNTHESIS

Original: Multi-modal image synthesis for a given label map is easy for OASIS: we simply re-sample noise like in a conventional unconditional GAN model.
Revised: Multi-modal image synthesis for a given label map is easy for OASIS: we simply re-sample noise like in a conventional unconditional GAN model.

Original: Since OASIS employs a 3D noise tensor (64channels × height × width), the noise can be re-sampled entirely (”globally”) or only for speciﬁc regions in the 2D image plane (”locally”).
Revised: Since OASIS employs a 3D noise tensor (64channels × height × width), the noise can be re-sampled entirely (”globally”) or only for speciﬁc regions in the 2D image plane (”locally”).

Original: For our visualizations, we replicate a single 64-dimensional noise vector along the spatial dimensions for global sampling.
Revised: For our visualizations, we replicate a single 64-dimensional noise vector along the spatial dimensions for global sampling.

Original: For local sampling, we re-sample a new noise vector and use it to replace the global noise vector at every spatial position within a restricted area of interest.
Revised: For local sampling, we re-sample a new noise vector and use it to replace the global noise vector at every spatial position within a restricted area of interest.

Original: The results are shown in Figure
Revised: The results are shown in Figure

Original: F. The generated images are diverse and of high quality.
Revised: F. The generated images are diverse and of high quality.

Original: We observe different degrees of variety for different object classes.
Revised: We observe different degrees of variety for different object classes.

Original: For example, buildings change drastically in appearance and often change their spatial orientation with respect to the road.
Revised: For example, buildings change drastically in appearance and often change their spatial orientation with respect to the road.

Original: On the other side, many common objects (like tables) vary in color, texture, and illumination, but do not change shapes as they are restricted by the ﬁne details of the region that is outlined for them in the label map.
Revised: On the other side, many common objects (like tables) vary in color, texture, and illumination, but do not change shapes as they are restricted by the ﬁne details of the region that is outlined for them in the label map.

Original: Local noise re-sampling does not have to be restricted to only semantic class areas: in Figure H we sample a different noise vector for the left and right half of the image, as well as for arbitrarily shaped regions.
Revised: Local noise re-sampling does not have to be restricted to only semantic class areas: in Figure H we sample a different noise vector for the left and right half of the image, as well as for arbitrarily shaped regions.

Original: In effect, the two areas can differ substantially.
Revised: In effect, the two areas can differ substantially.

Original: However, often a bridging element is found between two areas, such as clouds extending partly from one region to the other region of the image.
Revised: However, often a bridging element is found between two areas, such as clouds extending partly from one region to the other region of the image.

Original: B. L ATENT SPACE INTERPOLATIONS
Revised: B. L ATENT SPACE INTERPOLATIONS

Original: In Figure I we present images that are the results of linear interpolations in the latent space (see
Revised: In Figure I we present images that are the results of linear interpolations in the latent space (see

Original: Fig.
Revised: Fig.

Original: I), using an OASIS model trained on the ADE20K dataset.
Revised: I), using an OASIS model trained on the ADE20K dataset.

Original: To generate the images, we sample two noise vectors z ∈ R 64 and interpolate them with three intermediate points.
Revised: To generate the images, we sample two noise vectors z ∈ R 64 and interpolate them with three intermediate points.

Original: The images are synthesized for these ﬁve different noise inputs while the label map is held ﬁxed.
Revised: The images are synthesized for these ﬁve different noise inputs while the label map is held ﬁxed.

Original: Note that in Figure
Revised: Note that in Figure

Original: I we only vary the noise globally , not locally (See Section 3.3 in the main paper).
Revised: I we only vary the noise globally , not locally (See Section 3.3 in the main paper).

Original: In contrast, Figure
Revised: In contrast, Figure

Original: J shows local interpolations.
Revised: J shows local interpolations.

Original: For this, we only re-sample the 3D noise in the area corresponding to a single semantic class.
Revised: For this, we only re-sample the 3D noise in the area corresponding to a single semantic class.

Original: The effect is that only the appearance of the selected semantic class varies while the rest of the image remains ﬁxed.
Revised: The effect is that only the appearance of the selected semantic class varies while the rest of the image remains ﬁxed.

Original: It can be observed that strong changes in a local area can slightly affect the surroundings if the local area is also very big.
Revised: It can be observed that strong changes in a local area can slightly affect the surroundings if the local area is also very big.

Original: As such, the clouds are slightly different in the ﬁrst and last panel of the mountain row and tree row in Figure J.
Revised: As such, the clouds are slightly different in the ﬁrst and last panel of the mountain row and tree row in Figure J.

Original: We see from Figure I and J that the trajectories in the latent space are smooth and semantically meaningful.
Revised: We see from Figure I and J that the trajectories in the latent space are smooth and semantically meaningful.

Original: For example, we observe transitions from winter to summer, day to night, green trees to leaﬂess trees, shiny parquet to matt carpet, as well as smooth transitions between buildings with different architectural styles.
Revised: For example, we observe transitions from winter to summer, day to night, green trees to leaﬂess trees, shiny parquet to matt carpet, as well as smooth transitions between buildings with different architectural styles.

Original: B. R EPURPOSING THE TRAINED DISCRIMINATOR AS AN IMAGE SEGMENTER
Revised: B. A PPLICATION TO UNLABELLED DATA

Original: OASIS has the unique property that it issimultaneously trained to bea label-to-image generator and an image segmenter.
Revised: OASIS has a unique property that its discriminator is trained to be an image segmenter.

Original: For this reason, we measure the performance of the discriminator as a segmenter. The performance is measured as the mean intersection over union (mIoU) between the ground truth label map and thesegmentation. We measure a mIoU of 40 . 0 for OASIS on ADE20K.{+
+}
Revised: We observed that it shows good performance on this task, reaching the mIoU of 40 . {+
+}

Original:  For comparison, the state of the artin image segmentation on ADE20K is a mIoU of 46.91 by
Revised: on ADE20K validation set. For comparison, current state of the art on ADE20K is a mIoU of 46 .

Original:  ResNeST (Zhang et al., 2020).
Revised: 91 , achieved by ResNeST (Zhang et al., 2020).

Original: The good segmentation  allows OASIS to be used as a scene autoencoder: given an unseen image without a label map, OASIS can infer a label map via the discriminator and subsequently feed it through the generator  to generate a scene with the same layout but different textures.
Revised: Such a good segmentation performance allows OASIS to be applied to unlabelled images: given an unseen image without a ground truth annotation, OASIS can predict a label map via the discriminator. Subsequently feeding this prediction to the generator allows to synthesize a scene with the same layout but different style.

Original: This property is shown in Figure
Revised: This property is shown in Fig.

Original: K. 
Revised: K. Due to the good segmentation performance, the recreated scenes closely follow the ground truth label map of the original image.

Original: The recreated scenes are diverse thanks to the fact that OASISis sensitive to noise.
Revised: The high sensitivity of OASIS to the 3D noise enforces good variability, so the recreations are different from each other.

Revised: We believe that creating multiple versions of one image while retaining the layout can be useful for data-augmentation.

Original: B. L ABEL M IX
Revised: B. L ABEL M IX

Original: Figure L shows additional visual examples of LabelMix regularization, as described in Section 3.
Revised: Figure L shows additional visual examples of LabelMix regularization, as described in Section 3.

Original: in the main paper.
Revised: in the main paper.

Original: It can be seen that the discriminator prediction on the mixed images often differs from the mix of individual predictions on real and fake images.
Revised: It can be seen that the discriminator prediction on the mixed images often differs from the mix of individual predictions on real and fake images.

Original: In particular, regions that are classiﬁed as real in the latter are classiﬁed as fake when the images are mixed.
Revised: In particular, regions that are classiﬁed as real in the latter are classiﬁed as fake when the images are mixed.

Original: This means that the discriminator takes the global context into account for local predictions and thereby often bases the prediction on arbitrary details that should not affect the real-fake class identity.
Revised: This means that the discriminator takes the global context into account for local predictions and thereby often bases the prediction on arbitrary details that should not affect the real-fake class identity.

Original: In return, the consistency regularization helps to minimize the difference between these two predictions.
Revised: In return, the consistency regularization helps to minimize the difference between these two predictions.

Original: Label map
Revised: Label map

Original: Ground truth
Revised: Ground truth

Original: SPADE
Revised: SPADE

Original: CC-FPSE
Revised: CC-FPSE

Original: OASIS
Revised: OASIS

Original: Label map
Revised: Label map

Original: Ground [Figure]
Revised: Ground truth

Original: SPADE
Revised: SPADE

Original: CC-FPSE
Revised: CC-FPSE

Original: OASIS
Revised: OASIS

Original: Label map
Revised: Label map

Original: Ground truth
Revised: Ground truth

Original: Pix2pixHD Wang et al.
Revised: Pix2pixHD Wang et al.

Original: SPADE Park et al.
Revised: SPADE Park et al.

Original: CC-FPSE Liu et al.
Revised: CC-FPSE Liu et al.

Original: OASIS
Revised: OASIS

Original: Label map
Revised: Label map

Original: Ground truth
Revised: Ground truth

Original: Pix2pixHD Wang et al.
Revised: Pix2pixHD Wang et al.

Original: SPADE Park et al.
Revised: SPADE Park et al.

Original: CC-FPSE Liu et al.
Revised: CC-FPSE Liu et al.

Original: OASIS
Revised: OASIS

Original: Label map
Revised: Label map

Original: Ground truth
Revised: Ground truth

Original: Pix2pixHD Wang et al.
Revised: Pix2pixHD Wang et al.

Original: SPADE Park et al.
Revised: SPADE Park et al.

Original: CC-FPSE Liu et al.
Revised: CC-FPSE Liu et al.

Original: OASIS
Revised: OASIS

Original: Label map
Revised: Label map

Original: Ground truth
Revised: Ground truth

Original: Pix2pixHD Wang et al.
Revised: Pix2pixHD Wang et al.

Original: SPADE Park et al.
Revised: SPADE Park et al.

Original: CC-FPSE Liu et al.
Revised: CC-FPSE Liu et al.

Original: OASIS
Revised: OASIS

Original: [Figure]
Revised: [Figure]

Original: Label map
Revised: Label map

Original: OASIS OASIS
Revised: OASIS OASIS

Original: OASIS OASIS
Revised: OASIS OASIS

Original: OASIS
Revised: OASIS

Original: G l ob a l
Revised: G l ob a l

Original: L o ca l
Revised: L o ca l

Original: G l ob a l
Revised: G l ob a l

Original: L o ca l
Revised: L o ca l

Original: G l ob a l
Revised: G l ob a l

Original: L o ca l
Revised: L o ca l

Original: G l ob a l
Revised: G l ob a l

Original: L o ca l
Revised: L o ca l

Revised: Label map

Revised: Ground

Original: [Figure]
Revised: [Figure]

Revised: SPADE

Revised: CC

Original: [Figure]
Revised: [Figure]

Original:  ky
Revised: OASIS [Figure] n S ky

Original: T r ee
Revised: T r ee

Original: W a t e r
Revised: W a t e r

Original: [Figure]
Revised: [Figure]

Original: i
Revised: i

Original: [Figure]
Revised: [Figure]

Original: Label map

Original: Ground truth Inferred Recreation Recreation
Revised: [Figure]

Original: Recreation

Original: [Figure]
Revised: [Figure]

Original: C A RCHITECTURAL AND TRAINING DETAILS
Revised: C A RCHITECTURAL AND TRAINING DETAILS

Original: The architecture of OASIS builds upon SPADE Park et al.
Revised: The architecture of OASIS builds upon SPADE Park et al.

Original: In the following, we describe in detail our proposed changes to the discriminator and the generator.
Revised: In the following, we describe in detail our proposed changes to the discriminator and the generator.

Original: C. D ISCRIMINATOR ARCHITECTURE
Revised: C. D ISCRIMINATOR ARCHITECTURE

Original: This OASIS discriminator follows a U-Net architecture and is built from ResNet blocks, inspired in their design by Brock et al.
Revised: This OASIS discriminator follows a U-Net architecture and is built from ResNet blocks, inspired in their design by Brock et al.

Original: The architecture of the OASIS discriminator is outlined in Table L.
Revised: The architecture of the OASIS discriminator is outlined in Table L.

Original: It has in total 22M learnable parameters and is bigger than the multi-scale PatchGAN discriminator (5.5M) used by SPADE Park et al.
Revised: It has in total 22M learnable parameters and is bigger than the multi-scale PatchGAN discriminator (5.5M) used by SPADE Park et al.

Original: The increased capacity of the OASIS discriminator allows it to learn a more powerful representation and provide more informative feedback to the generator.
Revised: The increased capacity of the OASIS discriminator allows it to learn a more powerful representation and provide more informative feedback to the generator.

Original: Operation
Revised: [Table] [Table] [Table] ResBlock

Original: [Table]
Revised: [Table]

Original: ResBlock-Down image (3,256,256) down(128,128,128)
Revised: down(

Original: ResBlock-Down down(128,128,128) down(128,64,64)
Revised: [Table] [Table] 128) down( [Table]

Original:  ResBlock-Down down(128,64,64) down(256,32,32)
Revised: [Table] ResBlock-Down down(128,64,64) down(256,32,32)

Original: ResBlock-Down down(256,32,32) down(256,16,16)
Revised: ResBlock-Down down(256,32,32) down( [Table]

Revised: [Table]

Original: ResBlock-Down down(256,16,16) down(512,8,8)
Revised: ResBlock-Down down( [Table] [Table] 16) down(512,8,8)

Original: ResBlock-Down down(512,8,8)
Revised: ResBlock-Down down(512,8,8)

Original: down(512,4,4)
Revised: down(512,4,4)

Original: ResBlock-Up down(512,4,4) up(512,8,8)
Revised: ResBlock-Up down(512,4,4) up(512,8,8)

Original: ResBlock-Up cat(up 1, down 5) (1024,8,8) up(256,16,16)
Revised: ResBlock-Up cat(up 1, down 5) (1024,8,8) up(256,16,16)

Original: ResBlock-Up cat(up 2, down 4) (512,16,16) up(256,32,32)
Revised: ResBlock-Up cat(up 2, down 4) (512,16,16) up(256,32,32)

Original: ResBlock-Up cat(up 3, down 3) (512,32,32) up(128,64,64)ResBlock
Revised: ResBlock-Up cat(up 3, down 3) (512,32,32) up(128,64,64)

Original: [Table] Up cat(up 4, down 2) (256,64,64) up(128,128,128)
Revised: ResBlock-Up cat(up 4, down 2) (256,64,64) up(128,128,128)

Original: ResBlock-Up cat(up 5, down 1) (256,128,128) up(64,256,256)
Revised: ResBlock-Up cat(up 5, down 1) (256,128,128) up(64,256,256)

Original: Conv2D up(64,256,256) out (N+1,256,256)
Revised: Conv2D up(64,256,256) out (N+1,256,256)

Original: C. G ENERATOR ARCHITECTURE
Revised: C. G ENERATOR ARCHITECTURE

Original: The generator architecture is built from SPADE ResNet blocks and includes a concatenation of 3D noise with the label map along the channel dimension as an option.
Revised: The generator architecture is built from SPADE ResNet blocks and includes a concatenation of 3D noise with the label map along the channel dimension as an option.

Original: The generator can be either trained directly on the label maps or with 3D noise concatenated to the label maps.
Revised: The generator can be either trained directly on the label maps or with 3D noise concatenated to the label maps.

Original: The latter option is shown in Table M.
Revised: The latter option is shown in Table M.

Original: OASIS generator drops the ﬁrst residual block used in Park et al.
Revised: OASIS generator drops the ﬁrst residual block used in Park et al.

Original: (2019), which decreases the number of learnable parameters from 96M to 72M. The optional 3D noise injection brings additionally 2M parameters.
Revised: (2019), which decreases the number of learnable parameters from 96M to 72M. The optional 3D noise injection brings additionally 2M parameters.

Original: This sampling scheme is ﬁve times lighter than the image encoder used by SPADE (10M).
Revised: This sampling scheme is ﬁve times lighter than the image encoder used by SPADE (10M).

Original: C. L EARNING OBJECTIVE AND TRAINING DETAILS
Revised: C. L EARNING OBJECTIVE AND TRAINING DETAILS

Original: Learning objective.
Revised: Learning objective.

Original: We train our model with ( N + 1 )-class cross entropy as an adversarial loss.
Revised: We train our model with ( N + 1 )-class cross entropy as an adversarial loss.

Original: Additionally, the discriminator is regularized with the proposed LabelMix consistency regularization.
Revised: Additionally, the discriminator is regularized with the proposed LabelMix consistency regularization.

Original: The full OASIS learning objective thus takes the following form:
Revised: The full OASIS learning objective thus takes the following form:

Original: [Equation]
Revised: [Equation]

Revised: OASIS D

Revised: [Equation]

Revised: i

Revised: [Equation]

Revised: i

Revised: [Equation]

Revised: [Equation]

Revised: t )

Revised: [Equation]

Revised: N

Revised: [Equation]

Revised: 

Revised: [Equation]

Revised: +

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Original: where x denotes the real image and ( z, t ) is the noise-label map.
Revised: where x denotes the real image and ( z, t ) is the noise-label map.

Original: [Table]
Revised: [Table]

Original: Our objective function is different from SPADE.
Revised: Our objective function is different from SPADE.

Original: Their model uses hinge adversarial loss and adds the VGG perceptual loss and a feature matching loss to train the generator.
Revised: Their model uses hinge adversarial loss and adds the VGG perceptual loss and a feature matching loss to train the generator.

Original: For an easier comparison, we provide the objective function of SPADE:
Revised: For an easier comparison, we provide the objective function of SPADE:

Original: [Equation]
Revised: [Equation]

Revised: ) − D ( i ) k

Revised: ( t, G ( z, t ))

Original: {+(cid:107) 
+}
Revised: {+(cid:107) 
+}

Revised: + + λ VGG E

Revised: ( z,t,x )

Revised: N (cid:88) i

Revised: = (cid:107) F ( i ) ( x ) − F

Revised: ( i ) ( G ( z, t ))

Revised: (cid:107) 1 ,

Revised: L SPADE D =

Revised: [Equation]

Revised: t

Revised: [Equation]

Revised: [Equation]

Revised: [Equation]

Revised: E ( z

Revised: [Equation]

Revised: [min

Revised: [Equation]

Original:  where F is the pre-trained VGG network.
Revised: ] , where F is the pre-trained VGG network.

Original: Training details.
Revised: Training details.

Original: We follow the experimental setting of Park et al.
Revised: We follow the experimental setting of (Park et al., 2019).

Original: The image resolution is set to 256x256 for ADE20K and COCO-Stuff and 256x512 for Cityscapes.
Revised: The image resolution is set to 256x256 for ADE20K and COCO-Stuff and 256x512 for Cityscapes.

Original: The Adam Kingma & Ba (2015) optimizer was used with momentums β = (0 , 0 . 999) and constant learning rates (0 . 0001 , 0 . 0004) for G and D .
Revised: The Adam (Kingma & Ba, 2015) optimizer was used with momentums β = (0 , 0 . 999) and constant learning rates (0 . 0001 , 0 . 0004) for G and D .

Original: We did not apply the GAN feature matching loss, and used the
Revised: We did not apply the GAN feature matching loss, and used the

Original: VGG perceptual loss only for ablations with λ VGG = 10.
Revised: VGG perceptual loss only for ablations with λ VGG = 10.

Original: The coefﬁcient for LabelMix λ LM was set to 5 for ADE20k and Cityscapes, and to 10 for COCO-Stuff.
Revised: The coefﬁcient for LabelMix λ LM was set to 5 for ADE20k and Cityscapes, and to 10 for COCO-Stuff.

Original: All our models use an exponential moving average (EMA) of the generator weights with 0 .
Revised: All our models use an exponential moving average (EMA) of the generator weights with 0 .

Original: 9999 decay Brock et al.
Revised: 9999 decay (Brock et al., 2019).

Original: All the experiments were run on 4 Tesla V100 GPUs, with a batch size of 20 for Cityscapes, and 32 for
Revised: All the experiments were run on 4 Tesla V100 GPUs, with a batch size of 20 for Cityscapes, and 32 for

Original: ADE20k and COCO-Stuff.
Revised: ADE20k and COCO-Stuff.

Original: The training epochs are 200 on ADE20K and Cityscapes, and 100 for the larger COCO-Stuff dataset.
Revised: The training epochs are 200 on ADE20K and Cityscapes, and 100 for the larger COCO-Stuff dataset.

Original: On average, a complete forward-backward pass with batch sizeon Ade20k takes around 0.95ms per training image.
Revised: On average, a complete forward-backward pass with batch sizeon Ade20k takes around 0.95ms per training image.