Original: R OBUST C ONDITIONAL G ENERATIVE A DVERSARIAL N ETWORKS
Revised: R OBUST C ONDITIONAL G ENERATIVE A DVERSARIAL N ETWORKS

Original: A BSTRACT
Revised: A BSTRACT

Original: Conditional generative adversarial networks (cGAN) have led to large improvements in the task of conditional image generation, which lies at the heart of computer vision.
Revised: Conditional generative adversarial networks (cGAN) have led to large improvements in the task of conditional image generation, which lies at the heart of computer vision.

Original: The major focus so far has been on performance improvement, while there has been little effort in making cGAN more robust to noise or leveraging structure in the output space of the model.
Revised: The major focus so far has been on performance improvement, while there has been little effort in making cGAN more robust to noise.

Original: Theend-to-end regression (of the generator) might lead to arbitrarily large errors in the output, which is unsuitable forthe application of such networks to real-world systems.
Revised: The regression (of the generator) might lead to arbitrarily large errors in the output, which makes cGAN unreliable for real-world applications.

Original: In this work, we introduce a novel conditional GAN model, called RoCGAN , which adds implicit constraints to address the issue.
Revised: In this work, we introduce a novel conditional GAN model, called RoCGAN , which leverages structure in the target space of the model to address the issue.

Original: Our model augments the generator with an unsupervised pathway, which promotes the outputs of the generator to span the target manifold even in the presence of large amounts of noise.
Revised: Our model augments the generator with an unsupervised pathway, which promotes the outputs of the generator to span the target manifold even in the presence of intense noise.

Original: We prove that RoCGAN share similar theoretical properties as GAN and experimentally verify that our model outperforms existing state-of-the-art cGAN
Revised: We prove that RoCGAN share similar theoretical properties as GAN and experimentally verify that our model outperforms existing state-of-the-art cGAN

Original: architectures by a large margin in a variety of domains including images from natural scenes and faces.
Revised: architectures by a large margin in a variety of domains including images from natural scenes and faces.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Image-to-image translation and more generally conditional image generation lie at the heart of computer vision.
Revised: Image-to-image translation and more generally conditional image generation lie at the heart of computer vision.

Original: Conditional Generative Adversarial Networks (cGAN) (Mirza & Osindero, 2014) have become a dominant approach in the ﬁeld, e.g. in dense 1 regression (Isola et al., 2017; Pathak et al., 2016; Ledig et al., 2017; Bousmalis et al., 2016; 
Revised: Conditional Generative Adversarial Networks (cGAN) (Mirza & Osindero, 2014) have become a dominant approach in the ﬁeld, e.g. in dense 1 regression (Isola et al., 2017; Pathak et al., 2016; Ledig et al., 2017; Bousmalis et al., 2016; Liu et al., 2017; Miyato & Koyama, 2018;

Original: Liu et al., 2017; Miyato & Koyama, 2018; Yu et al., 2018; Tulyakov et al., 2018).
Revised: Yu et al., 2018; Tulyakov et al., 2018).

Original: They accept a source signal as input, e.g. prior information in the form of an image or text, and map it to the target signal (image).
Revised: They accept a source signal as input, e.g. prior information in the form of an image or text, and map it to the target signal (image).

Original: Ideally, the output should be a sample from the target manifold, however the mapping of cGAN does not constrain the output and thus it can be arbitrarily off the target manifold (Vidal et al., 2017).
Revised: The mapping of cGAN does not constrain the output to the target manifold, thus the output can be arbitrarily off the target manifold (Vidal et al., 2017).

Original: This is a critical problem both for academic and commercial applications.
Revised: This is a critical problem both for academic and commercial applications.

Original: If we aim to utilize cGAN or similar methods as a production technology, they need to be reliable and have performance guarantees under large amount of  noise.
Revised: To utilize cGAN or similar methods as a production technology, we need to study their generalization even in the face of intense noise.

Original: Similarly to regression, classiﬁcation also suffers from sensitivity to noise and lack of output constraints.
Revised: Similarly to regression, classiﬁcation also suffers from sensitivity to noise and lack of output constraints.

Original: One notable line of research consists in complementing supervision with unsupervised learning modules.
Revised: One notable line of research consists in complementing supervision with unsupervised learning modules.

Original: The unsupervised module forms a new pathway that can be fed to either the same, or different data samples.
Revised: The unsupervised module forms a new pathway that is trained with the same, or different data samples.

Original: The unsupervised pathway enables the network to explore the structure that is not present in the labelled training set, while implicitly constraining the output.
Revised: The unsupervised pathway enables the network to explore the structure that is not present in the labelled training set, while implicitly constraining the output.

Original: The addition of the unsupervised modules is only required during the training stage and results in no additional computational cost during inference.
Revised: The addition of the unsupervised module is only required during the training stage and results in no additional computational cost during inference.

Original: Rasmus et al.
Revised: Rasmus et al.

Original: (2015) and Zhang et al.
Revised: (2015) and Zhang et al.

Original: modiﬁed the original bottom-up  network to include top-down  modules during training.
Revised: modiﬁed the original bottom-up (encoder) network to include top-down (decoder) modules during training.

Original: However, in dense regression both bottom-up and top-down modules exist by default, and such methods are thus not trivial to extend to regression tasks.
Revised: However, in dense regression both bottom-up and top-down modules exist by default, and such methods are thus not trivial to extend to regression tasks.

Original: Motivated by the combination of supervised and unsupervised pathways, we propose a novel conditional GAN which includes implicit constraints in the latent subspaces.
Revised: Motivated by the combination of supervised and unsupervised pathways, we propose a novel conditional GAN which includes implicit constraints in the latent subspaces.

Original: We coin this new model ‘ Robust Conditional GAN ’ ( RoCGAN ).
Revised: We coin this new model ‘ Robust Conditional GAN ’ ( RoCGAN ).

Original: In the original cGAN the generator accepts a source signal and maps it to the target domain.
Revised: In the original cGAN the generator accepts a source signal and maps it to the target domain.

Original: In our work, we (implicitly) constrain the decoder to generate samples that span only the target manifold.
Revised: In our work, we (implicitly) constrain the decoder to generate samples that span only the target manifold.

Original: To that end, we replace the original generator, i.e. encoder-decoder, with a two pathway module; depicted in Fig. 
Revised: We replace the original generator, i.e. encoder-decoder, with a two pathway module (see Fig. 1).

Original: The ﬁrst pathway, similarly to the cGAN generator, performs regression while the second is an autoencoder in the target domain (unsupervised pathway).
Revised: The ﬁrst pathway, similarly to the cGAN generator, performs regression while the second is an autoencoder in the target domain (unsupervised pathway).

Original: The two pathways share a similar network structure, i.e. each one includes an encoder-decoder network.
Revised: The two pathways share a similar network structure, i.e. each one includes an encoder-decoder network.

Original: The weights of the two decoders are shared which promotes the latent representations of the two pathways to be semantically similar.
Revised: The weights of the two decoders are shared which promotes the latent representations of the two pathways to be semantically similar.

Original: Intuitively, this can be thought of as constraining the output of our dense regression to span the target subspace.
Revised: Intuitively, this can be thought of as constraining the output of our dense regression to span the target subspace.

Original: The unsupervised pathway enables the utilization of all the samples in the target domain even in the absence of a corresponding input sample.
Revised: The unsupervised pathway enables the utilization of all the samples in the target domain even in the absence of a corresponding input sample.

Original: During inference, the unsupervised pathway is no longer required, therefore the testing complexity remains the same as in cGAN.
Revised: During inference, the unsupervised pathway is no longer required, therefore the testing complexity remains the same as in cGAN.

Original: [Figure]
Revised: [Figure]

Original: In the following sections, we introduce our novel RoCGAN and study their (theoretical) properties.
Revised: In the following sections, we introduce our novel RoCGAN and study their (theoretical) properties.

Original: We prove that RoCGAN share similar theoretical properties with the original GAN, i.e. convergence and optimal discriminator.
Revised: We prove that RoCGAN share similar theoretical properties with the original GAN, i e. convergence and optimal discriminator.

Original: An experiment with synthetic data is designed to visualize the target subspaces and assess our intuition.
Revised: An experiment with synthetic data is designed to visualize the target subspaces and assess our intuition.

Original: We experimentally scrutinize the sensitivity of the hyper-parameters and evaluate our model in the face of intense noise.
Revised: We experimentally scrutinize the sensitivity of the hyper-parameters and evaluate our model in the face of intense noise.

Original: Moreover, thorough experimentation with both images from natural scenes and human faces is conducted in  different tasks to evaluate the model.
Revised: Moreover, thorough experimentation with both images from natural scenes and human faces is conducted in two different tasks.

Original: We compare our model with both the state-of-the-art cGAN and the recent method of Rick Chang et al.
Revised: We compare our model with both the state-of-the-art cGAN and the recent method of Rick Chang et al.

Original: The experimental results demonstrate that RoCGAN outperform the baseline by a large margin in all cases.
Revised: The experimental results demonstrate that RoCGAN outperform the baseline by a large margin in all cases.

Original: Our contributions can be summarized as following:
Revised: Our contributions can be summarized as following:

Original: • We introduce RoCGAN that leverages structure in the target space.
Revised: • We introduce RoCGAN that leverages structure in the target space.

Original: The goal is to promote robustness in dense regression tasks.
Revised: The goal is to promote robustness in dense regression tasks.

Original: • We scrutinize the model performance under (extreme) noise and adversarial perturbations.
Revised: • We scrutinize the model performance under (extreme) noise and adversarial perturbations.

Original: To the authors’ knowledge, this robustness analysis has not been studied previously for dense regression.
Revised: To the authors’ knowledge, this robustness analysis has not been studied previously for dense regression.

Original: • We conduct a thorough experimental analysis for  different cases/tasks.
Revised: • We conduct a thorough experimental analysis for two different tasks.

Original: We outline how RoCGAN can be used in a semi-supervised learning task,or how it performs with lateral connections from encoder to decoder.
Revised: We outline how RoCGAN can be used in a semi-supervised learning task, how it performs with lateral connections from encoder to decoder.

Original: Notation : Given a set of N samples, s ( n ) denotes the n th conditional label, e.g. a prior image; y ( n ) denotes the respective target image.
Revised: Notation : Given a set of N samples, s ( n ) denotes the n th conditional label, e.g. a prior image; y ( n ) denotes the respective target image.

Original: Unless explicitly mentioned otherwise || · || will declare an (cid:96)norm.
Revised: Unless explicitly mentioned otherwise || · || will declare an (cid:96)norm.

Original: The symbols L ∗ deﬁne loss terms, while λ ∗ denote regularization hyper-parameters optimized on the validation set.
Revised: The symbols L ∗ deﬁne loss terms, while λ ∗ denote regularization hyper- parameters optimized on the validation set.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Conditional image generation is a popular task in computer vision, dominated by approaches similar to cGAN.
Revised: Conditional image generation is a popular task in computer vision, dominated by approaches similar to cGAN.

Original: Apart from cGAN, the method by Isola et al.
Revised: Apart from cGAN, the method by Isola et al.

Original: (2017), widely known as ‘pix2pix’, is the main alternative.
Revised: (2017), widely known as ‘pix2pix’, is the main alternative.

Original: Pix2pix includes three modiﬁcations over the baseline cGAN: i) lateral skip connections between the encoder and the decoder network are added in the generator, ii) the discriminator accepts pairs of source/gt and source/model output images, iii) additional content loss terms are added.
Revised: Pix2pix includes three modiﬁcations over the baseline cGAN: i) lateral skip connections between the encoder and the decoder network are added in the generator, ii) the discriminator accepts pairs of source/gt and source/model output images, iii) additional content loss terms are added.

Original: The authors demonstrate how those performance related modiﬁcations can lead to an improved visual outcome.
Revised: The authors demonstrate how those performance related modiﬁcations can lead to an improved visual outcome.

Original: Despite the improved performance, the problem with the additional guarantees remains the same.
Revised: Despite the improved performance, the problem with the additional guarantees remains the same.

Original: That is we do not have any direct supervision in the process, since both the latent subspace and the projection are learned; the only supervision is provided by the ground-truth (gt) signal in the generator’s output.
Revised: That is we do not have any direct supervision in the process, since both the latent subspace and the projection are learned; the only supervision is provided by the ground-truth (gt) signal in the generator’s output.

Original: Adding regularization terms in the loss function can impose stronger supervision, thus restricting the output.
Revised: Adding regularization terms in the loss function can impose stronger supervision, thus restricting the output.

Original: The most frequent regularization term is feature matching, e.g. perceptual loss (Ledig et al., 2017; Johnson et al., 2016), or embeddings for faces (Schroff et al., 2015).
Revised: The most frequent regularization term is feature matching, e.g. perceptual loss (Ledig et al., 2017; Johnson et al., 2016), or embeddings for faces (Schroff et al., 2015).

Original: Feature matching minimizes the distance between the projection of generated and ground-truth signals.
Revised: Feature matching minimizes the distance between the projection of generated and ground-truth signals.

Original: However, the pre-deﬁned feature space is restrictive.
Revised: However, the pre-deﬁned feature space is restrictive.

Original: The method introduced by Salimans et al.
Revised: The method introduced by Salimans et al.

Original: (2016) performs feature matching in the discriminator; the motivation lies in matching the low-dimensional distributions created by the discriminator layers.
Revised: (2016) performs feature matching in the discriminator; the motivation lies in matching the low-dimensional distributions created by the discriminator layers.

Original: Matching the discriminator’s features has demonstrated empirical success.
Revised: Matching the discriminator’s features has demonstrated empirical success.

Original: However, this does not affect the generator and its latent subspaces directly.
Revised: However, this does not affect the generator and its latent subspaces directly.

Original: A new line of research that correlates with our goals is that of adversarial attacks (Szegedy et al., 2014; Yuan et al., 2017; Samangouei et al., 2018).
Revised: A new line of research that correlates with our goals is that of adversarial attacks (Szegedy et al., 2014; Yuan et al., 2017; Samangouei et al., 2018).

Original: It is observed that perturbing input samples with a small amount of noise, often imperceptible to the human eye, can lead to severe classiﬁcation errors.
Revised: It is observed that perturbing input samples with a small amount of noise, often imperceptible to the human eye, can lead to severe classiﬁcation errors.

Original: There are several techniques to ‘defend’ against such attacks.
Revised: There are several techniques to ‘defend’ against such attacks.

Original: A recent example is the Fortiﬁed networks of Lamb et al.
Revised: A recent example is the Fortiﬁed networks of Lamb et al.

Original: (2018) which uses Denoising Autoencoders (Vincent et al., 2008) to ensure that the input samples do not fall off the target manifold.
Revised: (2018) which uses Denoising Autoencoders (Vincent et al., 2008) to ensure that the input samples do not fall off the target manifold.

Original: Kumar et al.
Revised: Kumar et al.

Original: (2017) estimate the tangent space to the target manifold and use that to insert invariances to the discriminator for classiﬁcation purposes.
Revised: (2017) estimate the tangent space to the target manifold and use that to insert invariances to the discriminator for classiﬁcation purposes.

Original: Even though RoCGAN share similarities with those methods, the scope is different since a) the output of our method is high-dimensional 2 and b) adversarial examples are not extended to dense regression 3 .
Revised: Even though RoCGAN share similarities with those methods, the scope is different since a) the output of our method is high-dimensional 2 and b) adversarial examples are not extended to dense regression 3 .

Original: Except for the study of adversarial attacks, combining supervised and unsupervised learning has been used for enhancing the classiﬁcation performance.
Revised: Except for the study of adversarial attacks, combining supervised and unsupervised learning has been used for enhancing the classiﬁcation performance.

Original: In the Ladder network, Rasmus et al.
Revised: In the Ladder network, Rasmus et al.

Original: (2015) modify a typical bottom-up network for classiﬁcation by adding a decoder and lateral connections between the encoder and the decoder.
Revised: (2015) modify a typical bottom-up network for classiﬁcation by adding a decoder and lateral connections between the encoder and the decoder.

Original: During training they utilize the augmented network as two pathways: i) labelled input samples are fed to the initial bottom-up module, ii) input samples are corrupted with noise and fed to the encoder-decoder with the lateral connections.
Revised: During training they utilize the augmented network as two pathways: i) labelled input samples are fed to the initial bottom-up module, ii) input samples are corrupted with noise and fed to the encoder-decoder with the lateral connections.

Original: The latter pathway is an autoencoder; the idea is that it can strengthen the resilience of the network to samples outside the input manifold, while it improves the classiﬁcation performance.
Revised: The latter pathway is an autoencoder; the idea is that it can strengthen the resilience of the network to samples outside the input manifold, while it improves the classiﬁcation performance.

Original: Our core goal consists in constraining the model’s output.
Revised: Our core goal consists in constraining the model’s output.

Original: Aside from deep learning approaches, such constraints in manifolds were typically tackled with component analysis.
Revised: Aside from deep learning approaches, such constraints in manifolds were typically tackled with component analysis.

Original: Canonical correlation analysis (Hotelling, 1936) has been extensively used for ﬁnding common subspaces that maximally correlate the data (Panagakis et al., 2016).
Revised: Canonical correlation analysis (Hotelling, 1936) has been extensively used for ﬁnding common subspaces that maximally correlate the data (Panagakis et al., 2016).

Original: The recent work of Murdock et al.
Revised: The recent work of Murdock et al.

Original: (2018) combines the expressiveness of neural networks with the theoretical guarantees of classic component analysis.
Revised: (2018) combines the expressiveness of neural networks with the theoretical guarantees of classic component analysis.

Original:  M ETHOD
Revised:  M ETHOD

Original: In this section, we elucidate our proposed RoCGAN.
Revised: In this section, we elucidate our proposed RoCGAN.

Original: To make the paper self-contained we ﬁrst review the original conditional GAN model (sec. 3.1), before introducing RoCGAN (sec. 3.2).
Revised: To make the paper self-contained we ﬁrst review the original conditional GAN model (sec. 3.1), before introducing RoCGAN (sec. 3.2).

Original: Sequentially, we pose the modiﬁcations required in case of shortcut connections from the encoder to the decoder (sec. 3.3).
Revised: Sequentially, we pose the modiﬁcations required in case of shortcut connections from the encoder to the decoder (sec. 3.3).

Original: In sec.
Revised: In sec.

Original: 3.4 we assess the intuition behind our model with synthetic data.
Revised: 3.4 we assess the intuition behind our model with synthetic data.

Original: The core idea in RoCGAN is to leverage structure in the output space of the model.
Revised: The core idea in RoCGAN is to leverage structure in the output space of the model.

Original: We achieve that by replacing the single pathway in the generator with two pathways.
Revised: We achieve that by replacing the single pathway in the generator with two pathways.

Original: In the appendix, we study the theoretical properties of our method and prove that RoCGAN share the same properties as the original
Revised: In the appendix, we study the theoretical properties of our method and prove that RoCGAN share the same properties as the original

Original: GAN (Goodfellow et al., 2014).
Revised: GAN (Goodfellow et al., 2014).

Original: 3. C ONDITIONAL GAN
Revised: 3. C ONDITIONAL GAN

Original: GAN consist of a generator and a discriminator module commonly optimized with alternating gradient descent methods.
Revised: GAN consist of a generator and a discriminator module commonly optimized with alternating gradient descent methods.

Original: The generator samples z from a prior distribution p z , e.g. uniform, and tries to model the target distribution p d ; the discriminator D tries to distinguish between the samples generated from the model and the target (ground-truth) distributions.
Revised: The generator samples z from a prior distribution p z , e.g. uniform, and tries to model the target distribution p d ; the discriminator D tries to distinguish between the samples generated from the model and the target (ground-truth) distributions.

Original: Conditional GAN (cGAN) (Mirza & Osindero, 2014) extend the formulation by providing the generator with additional labels.
Revised: Conditional GAN (cGAN) (Mirza & Osindero, 2014) extend the formulation by providing the generator with additional labels.

Original: In cGAN the generator G typically takes the form of an encoder-decoder network, where the encoder projects the label into a low-dimensional latent subspace and the decoder performs the opposite mapping, i.e. from low-dimensional to high-dimensional subspace.
Revised: In cGAN the generator G typically takes the form of an encoder-decoder network, where the encoder projects the label into a low-dimensional latent subspace and the decoder performs the opposite mapping, i.e. from low-dimensional to high-dimensional subspace.

Original: If we denote s the conditioning label and y a sample from the target distribution, the adversarial loss is expressed as:
Revised: If we denote s the conditioning label and y a sample from the target distribution, the adversarial loss is expressed as:

Original: [Equation]
Revised: [Equation]

Original: ) by solving the following min-max problem:
Revised: ) by solving the following min-max problem:

Original: [Equation]
Revised: [Equation]

Original: where w G , w D denote the generator’s and the discriminator’s parameters respectively.
Revised: where w G , w D denote the generator’s and the discriminator’s parameters respectively.

Original: To simplify the notation, we drop the dependencies on the parameters and the noise z in the rest of the paper.
Revised: To simplify the notation, we drop the dependencies on the parameters and the noise z in the rest of the paper.

Original: The works of Salimans et al.
Revised: The works of Salimans et al.

Original: (2016) and Isola et al.
Revised: (2016) and Isola et al.

Original: (2017) demonstrate that auxiliary loss terms, i.e. feature matching and content loss, improve the ﬁnal outcome, hence we consider those as part of the vanilla cGAN.
Revised: (2017) demonstrate that auxiliary loss terms, i.e. feature matching and content loss, improve the ﬁnal outcome, hence we consider those as part of the vanilla cGAN.

Original: The feature matching loss (Salimans et al., 2016) is:
Revised: The feature matching loss (Salimans et al., 2016) is:

Original: [Equation]
Revised: [Equation]

Original: where π () extracts the features from the penultimate layer of the discriminator.
Revised: where π () extracts the features from the penultimate layer of the discriminator.

Original: The ﬁnal loss function for the cGAN is the following:
Revised: The ﬁnal loss function for the cGAN is the following:

Original: [Equation]
Revised: [Equation]

Original: where λ c , λ π are hyper-parameters to balance the loss terms.
Revised: where λ c , λ π are hyper-parameters to balance the loss terms.

Original: 3. R O CGAN
Revised: 3. R O CGAN

Original: Just like cGAN, RoCGAN consist of a generator and a discriminator.
Revised: Just like cGAN, RoCGAN consist of a generator and a discriminator.

Original: The generator of RoCGAN includes two pathways instead of the single pathway of the original cGAN.
Revised: The generator of RoCGAN includes two pathways instead of the single pathway of the original cGAN.

Original: The ﬁrst pathway, referred as reg pathway henceforth, performs a similar regression as its counterpart in cGAN; it accepts a sample from the source domain and maps it to the target domain.
Revised: The ﬁrst pathway, referred as reg pathway henceforth, performs a similar regression as its counterpart in cGAN; it accepts a sample from the source domain and maps it to the target domain.

Original: We introduce an additional unsupervised pathway, named AE pathway .
Revised: We introduce an additional unsupervised pathway, named AE pathway .

Original: AE pathway works as an autoencoder in the target domain.
Revised: AE pathway works as an autoencoder in the target domain.

Original: Both pathways consist of similar encoder-decoder networks 4 .
Revised: Both pathways consist of similar encoder-decoder networks 4 .

Original: By sharing the weights of their decoders, we promote the regression outputs to span the target manifold and not induce arbitrarily
Revised: By sharing the weights of their decoders, we promote the regression outputs to span the target manifold and not induce arbitrarily

Original: [Figure]
Revised: [Figure]

Original: large errors.
Revised: large errors.

Original: A schematic of the generator is illustrated in Fig.
Revised: A schematic of the generator is illustrated in Fig.

Original: The discriminator can remain the same as the cGAN: it accepts the reg pathway’s output along with the corresponding target sample as input.
Revised: The discriminator can remain the same as the cGAN: it accepts the reg pathway’s output along with the corresponding target sample as input.

Original: To simplify the notation below, the superscript ‘AE’ abbreviates modules of the AE pathway and ‘G’ modules of the reg pathway.
Revised: To simplify the notation below, the superscript ‘AE’ abbreviates modules of the AE pathway and ‘G’ modules of the reg pathway.

Original: We denote G ( s ( n ) )
Revised: We denote G ( s ( n ) )

Original: =
Revised: =

Original: d ( G ) ( e ( G ) ( s ( n ) ))
Revised: d ( G ) ( e ( G ) ( s ( n ) ))

Original: the output of the reg pathway and G ( AE ) ( y ( n ) ) =
Revised: the output of the reg pathway and G ( AE ) ( y ( n ) ) =

Original: d ( AE ) ( e ( AE ) ( y ( n ) ))
Revised: d ( AE ) ( e ( AE ) ( y ( n ) ))

Original: the output of the AE pathway.
Revised: the output of the AE pathway.

Original: The unsupervised module (autoencoder in the target domain) contributes the following loss term:
Revised: The unsupervised module (autoencoder in the target domain) contributes the following loss term:

Original: [Equation]
Revised: [Equation]

Original: where f d denotes a divergence metric (in this work an (cid:96) 1 loss).
Revised: where f d denotes a divergence metric (in this work an (cid:96) 1 loss).

Original: Despite sharing the weights of the decoders, we cannot ensure that the latent representations of the two pathways span the same space.
Revised: Despite sharing the weights of the decoders, we cannot ensure that the latent representations of the two pathways span the same space.

Original: To further reduce the distance of the two representations in the latent space, we introduce the latent loss term L lat .
Revised: To further reduce the distance of the two representations in the latent space, we introduce the latent loss term L lat .

Original: This term minimizes the distance between the encoders’ outputs, i.e. the two representations are spatially close (in the subspace spanned by the encoders).
Revised: This term minimizes the distance between the encoders’ outputs, i.e. the two representations are spatially close (in the subspace spanned by the encoders).

Original: The latent loss term is:
Revised: The latent loss term is:

Original: [Equation]
Revised: [Equation]

Original: The ﬁnal loss function of RoCGAN combines the loss terms of the original cGAN (eq. 3) with the additional two terms for the AE pathway:
Revised: The ﬁnal loss function of RoCGAN combines the loss terms of the original cGAN (eq. 3) with the additional two terms for the AE pathway:

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: As a future step we intend to replace the latent loss term L lat with a kernel-based method (Gretton et al., 2007) or a learnable metric for matching the distributions (Ma et al., 2018).
Revised: As a future step we intend to replace the latent loss term L lat with a kernel-based method (Gretton et al., 2007) or a learnable metric for matching the distributions (Ma et al., 2018).

Original: 3. R O CGAN WITH SKIP CONNECTIONS
Revised: 3. R O CGAN WITH SKIP CONNECTIONS

Original: The RoCGAN model of sec.
Revised: The RoCGAN model of sec.

Original: 3.2 describes a family of networks and not a predeﬁned set of layers.
Revised: 3.2 describes a family of networks and not a predeﬁned set of layers.

Original: A special case of RoCGAN emerges when skip connections are included in the generator.
Revised: A special case of RoCGAN emerges when skip connections are included in the generator.

Original: In the next few paragraphs, we study the modiﬁcation required, i.e. an additional loss term.
Revised: In the next few paragraphs, we study the modiﬁcation required, i.e. an additional loss term.

Original: Skip connections are frequently used as they enable deeper layers to capture more abstract representations without the need of memorizing all the information.
Revised: Skip connections are frequently used as they enable deeper layers to capture more abstract representations without the need of memorizing all the information.

Original: Nevertheless, the effects of the skip connections in the representation space have not been thoroughly studied.
Revised: Nevertheless, the effects of the skip connections in the representation space have not been thoroughly studied.

Original: The lower-level representations are propagated directly to the decoder through the shortcut, which makes it harder to train the longer path (Rasmus et al., 2015), i.e. the network excluding the skip connections.
Revised: The lower-level representations are propagated directly to the decoder through the shortcut, which makes it harder to train the longer path (Rasmus et al., 2015), i.e. the network excluding the skip connections.

Original: This challenge can be implicitly tackled by maximizing the variance captured by the longer path representations.
Revised: This challenge can be implicitly tackled by maximizing the variance captured by the longer path representations.

Original: To that end, we add a loss term that penalizes the correlations in the representations (of a layer) and thus implicitly encourage the representations to capture diverse and useful information.
Revised: To that end, we add a loss term that penalizes the correlations in the representations (of a layer) and thus implicitly encourage the representations to capture diverse and useful information.

Original: We implement the decov loss introduced by Cogswell et al.
Revised: We implement the decov loss introduced by Cogswell et al.

Original: [Equation]
Revised: [Equation]

Original: (7) where diag () computes the diagonal elements of a matrix and C is the covariance matrix of the layer’s representations.
Revised: (7) where diag () computes the diagonal elements of a matrix and C is the covariance matrix of the layer’s representations.

Original: The loss is minimized when the covariance matrix is diagonal, i.e. it imposes a cost to minimize the covariance of hidden units without restricting the diagonal elements that include the variance of the hidden representations.
Revised: The loss is minimized when the covariance matrix is diagonal, i.e. it imposes a cost to minimize the covariance of hidden units without restricting the diagonal elements that include the variance of the hidden representations.

Original: A similar loss is explored by Valpola (2015), where the decorrelation loss is applied in every layer.
Revised: A similar loss is explored by Valpola (2015), where the decorrelation loss is applied in every layer.

Original: Their loss term has stronger constraints: i) it favors an identity covariance matrix but also ii) penalizes the smaller eigenvalues of the covariance more.
Revised: Their loss term has stronger constraints: i) it favors an identity covariance matrix but also ii) penalizes the smaller eigenvalues of the covariance more.

Original: We have not explored this alternative loss term, as the decov loss worked in our case without the additional assumptions of the Valpola (2015).
Revised: We have not explored this alternative loss term, as the decov loss worked in our case without the additional assumptions of the Valpola (2015).

Original: 3. E XPERIMENT ON SYNTHETIC DATA
Revised: 3. E XPERIMENT ON SYNTHETIC DATA

Original: We design an experiment on synthetic data to explore the differences between the original generator and our novel two pathway generator.
Revised: We design an experiment on synthetic data to explore the differences between the original generator and our novel two pathway generator.

Original: Speciﬁcally, we design a network where each encoder/decoder consists of two fully connected layers; each layer followed by a RELU.
Revised: Speciﬁcally, we design a network where each encoder/decoder consists of two fully connected layers; each layer followed by a RELU.

Original: We optimize the generators only, to avoid adding extra learned parameters.
Revised: We optimize the generators only, to avoid adding extra learned parameters.

Original: The inputs/outputs of this network span a low-dimensional space, which depends on two independent variables x, y ∈ [ − 1 , 1] .
Revised: The inputs/outputs of this network span a low-dimensional space, which depends on two independent variables x, y ∈ [ − 1 , 1] .

Original: We’ve experimented with several arbitrary functions in the input and output vectors and they perform in a similar way.
Revised: We’ve experimented with several arbitrary functions in the input and output vectors and they perform in a similar way.

Original: We showcase here the case with input vector [ x, y, e 2 x ] and output vector [ x + 2 y + 4 , e x + 1 , x + y + 3 , x + 2] .
Revised: We showcase here the case with input vector [ x, y, e 2 x ] and output vector [ x + 2 y + 4 , e x + 1 , x + y + 3 , x + 2] .

Original: The reg pathway accepts the three inputs, projects it into a two-dimensional space and the decoder maps it to the target four-dimensional space.
Revised: The reg pathway accepts the three inputs, projects it into a two-dimensional space and the decoder maps it to the target four-dimensional space.

Original: We train the baseline and the autoencoder modules separately and use their pre-trained weights to initialize the two pathway network.
Revised: We train the baseline and the autoencoder modules separately and use their pre-trained weights to initialize the two pathway network.

Original: The loss function of the two pathway network consists of the L lat (eq. 5) and (cid:96) 2 content losses in the two pathways.
Revised: The loss function of the two pathway network consists of the L lat (eq. 5) and (cid:96) 2 content losses in the two pathways.

Original: The networks are trained either till convergence or till 100 , 000 iterations (batch size 128 ) are completed.
Revised: The networks are trained either till convergence or till 100 , 000 iterations (batch size 128 ) are completed.

Original: During testing, 6 , 400 new points are sampled and the overlaid results are depicted in Fig.
Revised: During testing, 6 , 400 new points are sampled and the overlaid results are depicted in Fig.

Original: 3; the individual ﬁgures for each output can be found in the appendix.
Revised: 3; the individual ﬁgures for each output can be found in the appendix.

Original: The (cid:96) 1 errors for the two cases are: 9 , 843 for the baseline and 1 , 520 for the two pathway generator.
Revised: The (cid:96) 1 errors for the two cases are: 9 , 843 for the baseline and 1 , 520 for the two pathway generator.

Original: We notice that the two pathway generator approximates the target manifold better with the same number of parameters during inference.
Revised: We notice that the two pathway generator approximates the target manifold better with the same number of parameters during inference.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: Implementation details : To provide a fair comparison to previous cGAN works, our implementation is largely based on the conventions of Isola et al.
Revised: Implementation details : To provide a fair comparison to previous cGAN works, our implementation is largely based on the conventions of Isola et al.

Original: ; Salimans et al.
Revised: ; Salimans et al.

Original: ; Zhu et al.
Revised: ; Zhu et al.

Original: A ‘layer’ refers to a block of three units: a convolutional unit with a 4 × 4 kernel size, followed by Leaky
Revised: A ‘layer’ refers to a block of three units: a convolutional unit with a 4 × 4 kernel size, followed by Leaky

Original: RELU and batch normalization (Ioffe & Szegedy, 2015).
Revised: RELU and batch normalization (Ioffe & Szegedy, 2015).

Original: To obtain RoCGAN, we augment a vanilla cGAN model as follows: i) we duplicate the encoder/decoder; ii) we share the decoder’s weights in x y z x y z
Revised: To obtain RoCGAN, we augment a vanilla cGAN model as follows: i) we duplicate the encoder/decoder; ii) we share the decoder’s weights in x y z x y z

Original: the two pathways; iii) we add the additional loss terms.
Revised: the two pathways; iii) we add the additional loss terms.

Original: The values of the additional hyper-parameters are λ l = 25 , λ ae = 100 and λ decov = 1 ; the common hyper-parameters with the vanilla cGAN, e.g. λ c , λ π , remain the same.
Revised: The values of the additional hyper-parameters are λ l = 25 , λ ae = 100 and λ decov = 1 ; the common hyper-parameters with the vanilla cGAN, e.g. λ c , λ π , remain the same.

Original: The decov loss is applied in the output of the encoder, which in our experimentation did minimize the correlations in the longer path.
Revised: The decov loss is applied in the output of the encoder, which in our experimentation did minimize the correlations in the longer path.

Original: The rest hyper-parameters remain the same as in the baseline.
Revised: The rest hyper-parameters remain the same as in the baseline.

Original: We conduct a number of auxiliary experiments in the appendix.
Revised: We conduct a number of auxiliary experiments in the appendix.

Original: Speciﬁcally, an ablation study on the signiﬁcance and the sensitivity of the hyper-parameters is conducted; additional architectures are implemented, while we evaluate our model under more intense noise.
Revised: Speciﬁcally, an ablation study on the signiﬁcance and the sensitivity of the hyper-parameters is conducted; additional architectures are implemented, while we evaluate our model under more intense noise.

Original: In addition, we extend the concept of adversarial examples in regression and verify that our model is more resilient to them than the baseline.
Revised: In addition, we extend the concept of adversarial examples in regression and verify that our model is more resilient to them than the baseline.

Original: The results demonstrate that our model accepts a range of hyper-parameter values, while it is robust to additional sources of noise.
Revised: The results demonstrate that our model accepts a range of hyper-parameter values, while it is robust to additional sources of noise.

Original: We experiment with two categories of images with signiﬁcant applications: images from i) natural scenes and ii) faces.
Revised: We experiment with two categories of images with signiﬁcant applications: images from i) natural scenes and ii) faces.

Original: In the natural scenes case, we constrain the number of training images to few thousand since frequently that is the scale of the labelled examples available.
Revised: In the natural scenes case, we constrain the number of training images to few thousand since frequently that is the scale of the labelled examples available.

Original: The network used in the experiments below, dumped ‘4layer’, consists of four layers in the decoder, while the decoder followed by four layers in the decoder.
Revised: The network used in the experiments below, dumped ‘4layer’, consists of four layers in the decoder, while the decoder followed by four layers in the decoder.

Original: Two inverse tasks, i.e. denoising and sparse inpainting, are selected for our quantitative evaluation.
Revised: Two inverse tasks, i.e. denoising and sparse inpainting, are selected for our quantitative evaluation.

Original: During training, the images are corrupted, for the two tasks, in the following way: for denoising 25% of the pixels in each channel are uniformly dropped; for sparse inpainting 50% of the pixels are converted to black.
Revised: During training, the images are corrupted, for the two tasks, in the following way: for denoising 25% of the pixels in each channel are uniformly dropped; for sparse inpainting 50% of the pixels are converted to black.

Original: During testing, we evaluate the methods in two settings: i) similar corruption as they were trained, ii) more intense corruption, i.e. we drop 35% of the pixels in the denoising case and 75% of the pixels in the sparse inpainting case.
Revised: During testing, we evaluate the methods in two settings: i) similar corruption as they were trained, ii) more intense corruption, i.e. we drop 35% of the pixels in the denoising case and 75% of the pixels in the sparse inpainting case.

Original: The widely used image quality loss (SSIM) (Wang et al., 2004) is used as a quantitative metric.
Revised: The widely used image quality loss (SSIM) (Wang et al., 2004) is used as a quantitative metric.

Original: We train and test our method against the i) baseline cGAN, ii) the recent strong-performing OneNet (Rick Chang et al., 2017).
Revised: We train and test our method against the i) baseline cGAN, ii) the recent strong-performing OneNet (Rick Chang et al., 2017).

Original: OneNet uses an ADMM learned prior, i.e. it projects the corrupted prior images into the subspace of natural images to guide the ADMM solver.
Revised: OneNet uses an ADMM learned prior, i.e. it projects the corrupted prior images into the subspace of natural images to guide the ADMM solver.

Original: In addition, we train an Adversarial Autoencoder (AAE) (Makhzani et al., 2015) as an established method capable of learning compressed representations.
Revised: In addition, we train an Adversarial Autoencoder (AAE) (Makhzani et al., 2015) as an established method capable of learning compressed representations.

Original: Each module of the AAE shares the same architecture as its cGAN counterpart, while the AAE is trained with images in the target space.
Revised: Each module of the AAE shares the same architecture as its cGAN counterpart, while the AAE is trained with images in the target space.

Original: During testing, we provide the ground-truth images as input and use the reconstruction for the evaluation.
Revised: During testing, we provide the ground-truth images as input and use the reconstruction for the evaluation.

Original: In our experimental setting, AAE can be thought of as an upper performance limit of RoCGAN/cGAN for a given capacity (number of parameters).
Revised: In our experimental setting, AAE can be thought of as an upper performance limit of RoCGAN/cGAN for a given capacity (number of parameters).

Original: 4. N ATURAL SCENES
Revised: 4. N ATURAL SCENES

Original: We train the ‘4layer’ baseline/RoCGAN with images from natural scenes, both indoors and outdoors.
Revised: We train the ‘4layer’ baseline/RoCGAN with images from natural scenes, both indoors and outdoors.

Original: The 4 , 900 samples of the VOC 2007 Challenge (Everingham et al., 2010) form the training set, while the 10 , 000 samples of tiny ImageNet (Deng et al., 2009) consist the testing set.
Revised: The 4 , 900 samples of the VOC 2007 Challenge (Everingham et al., 2010) form the training set, while the 10 , 000 samples of tiny ImageNet (Deng et al., 2009) consist the testing set.

Original: The quantitative evaluation with SSIM is presented in Tab. 1.
Revised: The quantitative evaluation with SSIM is presented in Tab. 1.

Original: OneNet (Rick Chang et al., 2017) does not perform as well as the baseline or our model.
Revised: OneNet (Rick Chang et al., 2017) does not perform as well as the baseline or our model.

Original: From our experimentation this can be attributed to the projection to the manifold of natural images that is not trivial, however it is more resilient to additional noise than the baseline.
Revised: From our experimentation this can be attributed to the projection to the manifold of natural images that is not trivial, however it is more resilient to additional noise than the baseline.

Original: In both inverse tasks RoCGAN improve the baseline cGAN results by a margin of 0 .
Revised: In both inverse tasks RoCGAN improve the baseline cGAN results by a margin of 0 .

Original: ( 10 − 13% relative improvement).
Revised: ( 10 − 13% relative improvement).

Original: When we apply additional corruption in the testing images, RoCGAN are more robust with a considerable improvement over the baseline.
Revised: When we apply additional corruption in the testing images, RoCGAN are more robust with a considerable improvement over the baseline.

Original: This
Revised: This

Original: [Figure]
Revised: [Figure]

Original: can be attributed to the implicit constraints of the AE pathway, i.e. the decoder is more resilient to approximating the target manifold samples.
Revised: can be attributed to the implicit constraints of the AE pathway, i.e. the decoder is more resilient to approximating the target manifold samples.

Original: (cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)
Revised: (cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)

Original: [Table]
Revised: [Table]

Original: Faces
Revised: Faces

Original: Natural Scenes
Revised: Natural Scenes

Original: [Table]
Revised: [Table]

Original: Sparse Inpaint.
Revised: Sparse Inpaint.

Original: Denoising
Revised: Denoising

Original: Sparse Inpaint.
Revised: Sparse Inpaint.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: AAE 0.
Revised: AAE 0.

Original: 4. F ACES
Revised: 4. F ACES

Original: In this experiment we utilize the MS-Celeb (Guo et al., 2016) as the training set ( 3 , 4 million samples), and the whole Celeb-A (Liu et al., 2015) as the testing set ( 202 , 500 samples).
Revised: In this experiment we utilize the MS-Celeb (Guo et al., 2016) as the training set ( 3 , 4 million samples), and the whole Celeb-A (Liu et al., 2015) as the testing set ( 202 , 500 samples).

Original: The large datasets enable us to validate our model extensively in a wide range of faces.
Revised: The large datasets enable us to validate our model extensively in a wide range of faces.

Original: We use the whole training set to train the two compared methods (Baseline-4layer and Ours-4layer) and the 4-layer AAE.
Revised: We use the whole training set to train the two compared methods (Baseline-4layer and Ours-4layer) and the 4-layer AAE.

Original: The results of the quantitative evaluation exist in table 1.
Revised: The results of the quantitative evaluation exist in table 1.

Original: Our method outperforms both the baseline and OneNet by a signiﬁcant margin; the difference increases when evaluated with more intense corruptions.
Revised: Our method outperforms both the baseline and OneNet by a signiﬁcant margin; the difference increases when evaluated with more intense corruptions.

Original: The reason that the sparse inpainting task appears to have a smaller improvement remains elusive; in the different architectures in the appendix our model has similar performance in the two tasks.
Revised: The reason that the sparse inpainting task appears to have a smaller improvement remains elusive; in the different architectures in the appendix our model has similar performance in the two tasks.

Original: We include the AAE as an upper limit of the representation capacity of the architecture.
Revised: We include the AAE as an upper limit of the representation capacity of the architecture.

Original: The AAE result speciﬁes that with the given architecture the performance can be up to 0 .
Revised: The AAE result speciﬁes that with the given architecture the performance can be up to 0 .

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We introduce the Robust Conditional GAN (RoCGAN) model, a new conditional GAN capable of leveraging unsupervised data to learn better latent representations, even in the face of large amount of noise.
Revised: We introduce the Robust Conditional GAN (RoCGAN) model, a new conditional GAN capable of leveraging unsupervised data to learn better latent representations, even in the face of large amount of noise.

Original: RoCGAN’s generator is composed of two pathways.
Revised: RoCGAN’s generator is composed of two pathways.

Original: The ﬁrst pathway ( reg pathway ), performs the regression from the source to the target domain.
Revised: The ﬁrst pathway ( reg pathway ), performs the regression from the source to the target domain.

Original: The new, added pathway ( AE pathway ) is an autoencoder in the target domain.
Revised: The new, added pathway ( AE pathway ) is an autoencoder in the target domain.

Original: By adding weight sharing between the two decoders, we implicitly constrain the reg pathway to output images that span the target manifold.
Revised: By adding weight sharing between the two decoders, we implicitly constrain the reg pathway to output images that span the target manifold.

Original: The linear analogy along with the synthetic experiment demonstrate how RoCGAN can create more robust results, while we prove that our model shares similar convergence properties with generative adversarial networks.
Revised: The linear analogy along with the synthetic experiment demonstrate how RoCGAN can create more robust results, while we prove that our model shares similar convergence properties with generative adversarial networks.

Original: The ablation study dictates that our model is more resilient to intense noise and more robust to adversarial examples than the baseline.
Revised: The ablation study dictates that our model is more resilient to intense noise and more robust to adversarial examples than the baseline.

Original: The experimental results with images (natural scenes and faces) showcase that RoCGAN outperform existing, state-of-the-art conditional GAN models.
Revised: The experimental results with images (natural scenes and faces) showcase that RoCGAN outperform existing, state-of-the-art conditional GAN models.

Original:  A CKNOWLEDGEMENTS
Revised:  A CKNOWLEDGEMENTS

Original: We would like to thank Markos Georgopoulos for our fruitful conversations during the preparation of this work.
Revised: We would like to thank Markos Georgopoulos for our fruitful conversations during the preparation of this work.

Original: The work of Grigorios Chrysos was partially funded by an Imperial College DTA.
Revised: The work of Grigorios Chrysos was partially funded by an Imperial College DTA.

Original: The work of Stefanos Zafeiriou was partially funded by the EPSRC Fellowship DEFORM: Large Scale
Revised: The work of Stefanos Zafeiriou was partially funded by the EPSRC Fellowship DEFORM: Large Scale

Original: Shape Analysis of Deformable Models of Humans (EP/S010203/1) and a Google Faculty Award
Revised: Shape Analysis of Deformable Models of Humans (EP/S010203/1) and a Google Faculty Award

Original: A I NTRODUCTION
Revised: A I NTRODUCTION

Original: In this following sections (of the appendix) we include additional insights, a theoretical analysis along with additional experiments.
Revised: In this following sections (of the appendix) we include additional insights, a theoretical analysis along with additional experiments.

Original: The sections are organized as following:
Revised: The sections are organized as following:

Original: • In sec.
Revised: • In sec.

Original: B we validate our intuition for the RoCGAN constraints through the linear equivalent.
Revised: B we validate our intuition for the RoCGAN constraints through the linear equivalent.

Original: • A theoretical analysis is provided in sec.
Revised: • A theoretical analysis is provided in sec.

Original: C. • We implement different networks in sec.
Revised: C. • We implement different networks in sec.

Original: D to assess whether the performance gain can be attributed to a single architecture.
Revised: D to assess whether the performance gain can be attributed to a single architecture.

Original: • An ablation study is conducted in sec.
Revised: • An ablation study is conducted in sec.

Original: E comparing the hyper-parameter sensitivity and the robustness in the face of extreme noise.
Revised: E comparing the hyper-parameter sensitivity and the robustness in the face of extreme noise.

Original: The Fig. 5, 6, 7, 8 include all the outputs of the synthetic experiment of the main paper.
Revised: The Fig. 5, 6, 7, 8 include all the outputs of the synthetic experiment of the main paper.

Original: As a reminder, the output vector is [ x + 2 y + 4 , e x + 1 , x + y + 3 , x + 2] with x, y ∈ [ − 1 , 1] .
Revised: As a reminder, the output vector is [ x + 2 y + 4 , e x + 1 , x + y + 3 , x + 2] with x, y ∈ [ − 1 , 1] .

Original: See Fig.
Revised: See Fig.

Original: 5 for
Revised: 5 for

Original: B L INEAR GENERATOR ANALOGY
Revised: B L INEAR GENERATOR ANALOGY

Original: The exact nature and convergence properties of deep networks remain elusive (Vidal et al., 2017), however we can study the linear equivalent of deep methods to build on our intuition.
Revised: The exact nature and convergence properties of deep networks remain elusive (Vidal et al., 2017), however we can study the linear equivalent of deep methods to build on our intuition.

Original: To that end, we explore the linear equivalent of our method.
Revised: To that end, we explore the linear equivalent of our method.

Original: Since the discriminator in RoCGAN can remain the same as in the baseline cGAN, we focus in the generators.
Revised: Since the discriminator in RoCGAN can remain the same as in the baseline cGAN, we focus in the generators.

Original: To perform the analysis on the linear equivalent, we simply drop the piecewise non-linear units in the generators.
Revised: To perform the analysis on the linear equivalent, we simply drop the piecewise non-linear units in the generators.

Original: See Fig.
Revised: See Fig.

Original: 5 for details.
Revised: 5 for details.

Original: We assume a network with two encoding and two decoding layers in this section; all layers include only linear units.
Revised: We assume a network with two encoding and two decoding layers in this section; all layers include only linear units.

Original: The symbols W ( G ) l with l ∈ [1 , 4] are the l th layer’s parameters (reg pathway).
Revised: The symbols W ( G ) l with l ∈ [1 , 4] are the l th layer’s parameters (reg pathway).

Original: The linear autoencoder (AE) has a similar structure; W ( AE ) l denote the respective parameters for the AE.
Revised: The linear autoencoder (AE) has a similar structure; W ( AE ) l denote the respective parameters for the AE.

Original: We denote with X the input signal, with Y the target signal and ˆ Y the AE output, ˜ Y the regression output.
Revised: We denote with X the input signal, with Y the target signal and ˆ Y the AE output, ˜ Y the regression output.

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: is the reconstruction of the autoencoder and
Revised: is the reconstruction of the autoencoder and

Original: [Equation]
Revised: [Equation]

Original: is the regression of the generator (reg pathway).
Revised: is the regression of the generator (reg pathway).

Original: We deﬁne the auxiliary U TD, ( G )
Revised: We deﬁne the auxiliary U TD, ( G )

Original: =
Revised: =

Original: W ( G ) W ( G ),
Revised: W ( G ) W ( G ),

Original: U E, ( G ) =
Revised: U E, ( G ) =

Original: W ( G )
Revised: W ( G )

Original: [Equation]
Revised: [Equation]

Original: and U E = W ( AE )
Revised: and U E = W ( AE )

Original: W ( AE ).
Revised: W ( AE ).

Original: Then Eq. 8 and 9 can be written as:
Revised: Then Eq. 8 and 9 can be written as:

Original: [Equation]
Revised: [Equation]

Original: The AE approximates under mild condition robustly the target manifold of the data Bengio et al.
Revised: The AE approximates under mild condition robustly the target manifold of the data Bengio et al.

Original: If we now deﬁne U D, ( G ) =
Revised: If we now deﬁne U D, ( G ) =

Original: U D , then the output of the generator ˜ Y spans the subspace of
Revised: U D , then the output of the generator ˜ Y spans the subspace of

Original: U D .
Revised: U D .

Original: Given that U D, ( G ) =
Revised: Given that U D, ( G ) =

Original: U D , we constrain the output of the generator to lie in the subspaces learned with the AE.
Revised: U D , we constrain the output of the generator to lie in the subspaces learned with the AE.

Original: To illustrate how a projection to a target subspace can contribute to constraining the image, the following visual example is designed.
Revised: To illustrate how a projection to a target subspace can contribute to constraining the image, the following visual example is designed.

Original: We learn a PCA model using one hundred thousand images from MS-Celeb; we do not apply any pose normalization or alignment (out of the paper’s scope).
Revised: We learn a PCA model using one hundred thousand images from MS-Celeb; we do not apply any pose normalization or alignment (out of the paper’s scope).

Original: We maintain 90% of the variance.
Revised: We maintain 90% of the variance.

Original: In Fig.
Revised: In Fig.

Original: 9 we sample a random image from Celeb-A and downscale it 5 ; we use bi-linear interpolation to upscale it to the original dimensions.
Revised: 9 we sample a random image from Celeb-A and downscale it 5 ; we use bi-linear interpolation to upscale it to the original dimensions.

Original: We project and reconstruct both the original and the upscaled versions; note that the output images are similar.
Revised: We project and reconstruct both the original and the upscaled versions; note that the output images are similar.

Original: This similarity illustrates how the linear projection forces both images to span the same subspace.
Revised: This similarity illustrates how the linear projection forces both images to span the same subspace.

Original: C T HEORETICAL ANALYSIS
Revised: C T HEORETICAL ANALYSIS

Original: In the next few paragraphs, we prove that RoCGAN share the properties of the original GAN (Goodfellow et al., 2014).
Revised: In the next few paragraphs, we prove that RoCGAN share the properties of the original GAN (Goodfellow et al., 2014).

Original: We derive the optimal discriminator and then compute the optimal value of
Revised: We derive the optimal discriminator and then compute the optimal value of

Original: L adv ( G , D ) .
Revised: L adv ( G , D ) .

Original: [Figure]
Revised: [Figure]

Original: Proposition 1.
Revised: Proposition 1.

Original: If we ﬁx the generator G (reg pathway), the optimal discriminator is:
Revised: If we ﬁx the generator G (reg pathway), the optimal discriminator is:

Original: [Equation]
Revised: [Equation]

Original: where p g is the model (generator) distribution.
Revised: where p g is the model (generator) distribution.

Original: Proof.
Revised: Proof.

Original: Since the generator is ﬁxed, the goal of the discriminator is to maximize the L adv where:
Revised: Since the generator is ﬁxed, the goal of the discriminator is to maximize the L adv where:

Original: [Equation]
Revised: [Equation]

Original: To maximize the L adv , we need to optimize the integrand above.
Revised: To maximize the L adv , we need to optimize the integrand above.

Original: We note that with respect to D the integrand has the form f ( y )
Revised: We note that with respect to D the integrand has the form f ( y )

Original: = a · log ( y ) + b · log (1 − y ) .
Revised: = a · log ( y ) + b · log (1 − y ) .

Original: The function f for a, b ∈ (0 , 1) as in our case, obtains a global maximum in aa + b , so:
Revised: The function f for a, b ∈ (0 , 1) as in our case, obtains a global maximum in aa + b , so:

Original: [Equation]
Revised: [Equation]

Original: with
Revised: with

Original: [Equation]
Revised: [Equation]

Original: thus L adv obtains the maximum with D ∗ .
Revised: thus L adv obtains the maximum with D ∗ .

Original: Proposition 2.
Revised: Proposition 2.

Original: Given the optimal discriminator D ∗ the global minimum of L adv is reached if and only if p g = p d , i.e. when the model (generator) distribution matches the data distribution.
Revised: Given the optimal discriminator D ∗ the global minimum of L adv is reached if and only if p g = p d , i.e. when the model (generator) distribution matches the data distribution.

Original: Proof.
Revised: Proof.

Original: From proposition 1, we have found the optimal discriminator as D ∗ , i.e. the arg max D L adv .
Revised: From proposition 1, we have found the optimal discriminator as D ∗ , i.e. the arg max D L adv .

Original: If we replace the optimal value we obtain:
Revised: If we replace the optimal value we obtain:

Original: [Equation]
Revised: [Equation]

Original: We add and subtract log(2) from both terms
Revised: We add and subtract log(2) from both terms

Original: [Equation]
Revised: [Equation]

Original: which after few math operations provides:
Revised: which after few math operations provides:

Original: [Equation]
Revised: [Equation]

Original: where in the last row KL symbolizes the Kullback–Leibler divergence.
Revised: where in the last row KL symbolizes the Kullback–Leibler divergence.

Original: The latter one can be rewritten more conveniently with the help of the Jensen–Shannon (JSD) divergence as
Revised: The latter one can be rewritten more conveniently with the help of the Jensen–Shannon (JSD) divergence as

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: The Jensen–Shannon divergence is non-negative and obtains the zero value only if p d = p g .
Revised: The Jensen–Shannon divergence is non-negative and obtains the zero value only if p d = p g .

Original: Equivalently, max D L adv ( G , D ) ≥ − log(4) and has a global minimum (under the constraint that the discriminator is optimal) when p d = p g .
Revised: Equivalently, max D L adv ( G , D ) ≥ − log(4) and has a global minimum (under the constraint that the discriminator is optimal) when p d = p g .

Original: D A DDITIONAL EXPERIMENTS
Revised: D A DDITIONAL EXPERIMENTS

Original: In this section, we describe additional experimental results and details.
Revised: In this section, we describe additional experimental results and details.

Original: In addition to the SSIM metric, we use the (cid:96) 1 loss to measure the loss in the experiments of the main paper.
Revised: In addition to the SSIM metric, we use the (cid:96) 1 loss to measure the loss in the experiments of the main paper.

Original: The results in table 2 conﬁrm that RoCGAN outperform both compared methods.
Revised: The results in table 2 conﬁrm that RoCGAN outperform both compared methods.

Original: The larger difference in the cases of more intense noise demonstrates that our model is indeed robust to additional cases not trained on.
Revised: The larger difference in the cases of more intense noise demonstrates that our model is indeed robust to additional cases not trained on.

Original: Additional visualizations are provided in Fig.
Revised: Additional visualizations are provided in Fig.

Original: (cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)
Revised: (cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)(cid:96)

Original: [Table]
Revised: [Table]

Original: Obj
Revised: Obj

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Faces
Revised: Faces

Original: Natural Scenes
Revised: Natural Scenes

Original: Denoising
Revised: Denoising

Original: Sparse Inpaint.
Revised: Sparse Inpaint.

Original: Denoising
Revised: Denoising

Original: Sparse Inpaint.
Revised: Sparse Inpaint.

Original: Rick Chang et al.
Revised: Rick Chang et al.

Original: [Table]
Revised: [Table]

Original: Ours
Revised: Ours

Original: [Table]
Revised: [Table]

Original: Unless otherwise mentioned, the experiments in the following paragraphs are conducted in the face case, while the evaluation metrics remain the same as in the main paper, i.e. the noise during training/testing and the SSIM evaluation metric.
Revised: Unless otherwise mentioned, the experiments in the following paragraphs are conducted in the face case, while the evaluation metrics remain the same as in the main paper, i.e. the noise during training/testing and the SSIM evaluation metric.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: D. L ARGE SCALE NATURAL IMAGES
Revised: D. L ARGE SCALE NATURAL IMAGES

Original: To delineate further the performance of our model in different settings
Revised: To delineate further the performance of our model in different settings

Original: [Table]
Revised: [Table]

Original: we conduct an experiment with Imagenet (Russakovsky et al., 2015), a large dataset for natural images.
Revised: we conduct an experiment with Imagenet (Russakovsky et al., 2015), a large dataset for natural images.

Original: We utilize the training set of Imagenet which consists of 1 , 2 million images and its testset that includes 98 thousand images.
Revised: We utilize the training set of Imagenet which consists of 1 , 2 million images and its testset that includes 98 thousand images.

Original: The experimental results are depicted in table 3.
Revised: The experimental results are depicted in table 3.

Original: The outcomes corroborate the experimental results of the main paper, as RoCGAN outperforms the cGAN in both tasks.
Revised: The outcomes corroborate the experimental results of the main paper, as RoCGAN outperforms the cGAN in both tasks.

Original: We note that the AAE works as the upper limit of the methods and denotes the representation power that the given encoder-decoder can reach.
Revised: We note that the AAE works as the upper limit of the methods and denotes the representation power that the given encoder-decoder can reach.

Original: D. D IFFERENT ARCHITECTURES
Revised: D. D IFFERENT ARCHITECTURES

Original: To assess whether RoCGAN’s improvement is network-speciﬁc, we implement different architectures including more layers.
Revised: To assess whether RoCGAN’s improvement is network-speciﬁc, we implement different architectures including more layers.

Original: The goal of this work is not to ﬁnd the best performing architecture, thus we do not employ an exhaustive search in all proposed cGAN models.
Revised: The goal of this work is not to ﬁnd the best performing architecture, thus we do not employ an exhaustive search in all proposed cGAN models.

Original: Our goal is to propose an alternative model to the baseline cGAN and evaluate how this works in different networks.
Revised: Our goal is to propose an alternative model to the baseline cGAN and evaluate how this works in different networks.

Original: We implement three additional networks which we coin ‘5layer’, ‘6layer’ and ‘4layer-skip’.
Revised: We implement three additional networks which we coin ‘5layer’, ‘6layer’ and ‘4layer-skip’.

Original: Those include ﬁve, six layers in the encoder/decoder respectively, while the ‘4layer-skip’ includes a lateral connection from the output of the third encoding layer to the input of the second decoding layer.
Revised: Those include ﬁve, six layers in the encoder/decoder respectively, while the ‘4layer-skip’ includes a lateral connection from the output of the third encoding layer to the input of the second decoding layer.

Original: The ﬁrst two increase the capacity of the network, while the ‘4layer-skip’ implements the modiﬁcation for the skip case in the ‘4layer’ network 6 .
Revised: The ﬁrst two increase the capacity of the network, while the ‘4layer-skip’ implements the modiﬁcation for the skip case in the ‘4layer’ network 6 .

Original: We evaluate these three networks as in the ‘4layer’ network (main paper); the results are added in table 4.
Revised: We evaluate these three networks as in the ‘4layer’ network (main paper); the results are added in table 4.

Original: Notice that both ‘5layer’ and ‘6layer’ networks improve their counterpart in the ‘4layer’ case, however the ‘6layer’ networks do not improve their ‘5layer’ counterpart.
Revised: Notice that both ‘5layer’ and ‘6layer’ networks improve their counterpart in the ‘4layer’ case, however the ‘6layer’ networks do not improve their ‘5layer’ counterpart.

Original: This can be partly attributed to the increased difﬁculty of training deeper networks without additional regularization techniques (He et al., 2016).
Revised: This can be partly attributed to the increased difﬁculty of training deeper networks without additional regularization techniques (He et al., 2016).

Original: In addition, we emphasize that the denoising and the sparse inpainting results cannot be directly compared, since they correspond to different a) types and b) amount of corruption in all evaluations.
Revised: In addition, we emphasize that the denoising and the sparse inpainting results cannot be directly compared, since they correspond to different a) types and b) amount of corruption in all evaluations.

Original: Nevertheless, the improvement in the sparse inpainting with additional noise is impressive, given that the hyper-parameters are optimized for the denoising case (see sec E).
Revised: Nevertheless, the improvement in the sparse inpainting with additional noise is impressive, given that the hyper-parameters are optimized for the denoising case (see sec E).

Original: The most critical observation is that in all cases our model consistently outperforms the baseline.
Revised: The most critical observation is that in all cases our model consistently outperforms the baseline.

Original: The
Revised: The

Original: [Figure]
Revised: [Figure]

Original: difference is increasing under additional noise during inference time with up to 15 % performance improvement observed in the sparse inpainting case.
Revised: difference is increasing under additional noise during inference time with up to 15 % performance improvement observed in the sparse inpainting case.

Original: D. S EMI - SUPERVISED TRAINING
Revised: D. S EMI - SUPERVISED TRAINING

Original: A side-beneﬁt of our new model is the ability to utilize unsupervised data to learn the AE pathway.
Revised: A side-beneﬁt of our new model is the ability to utilize unsupervised data to learn the AE pathway.

Original: Collecting unlabelled data in the target domain is frequently easier than ﬁnding pairs of corresponding samples in the two domains.
Revised: Collecting unlabelled data in the target domain is frequently easier than ﬁnding pairs of corresponding samples in the two domains.

Original: To that end, we test whether RoCGAN support such semi-supervised learning.
Revised: To that end, we test whether RoCGAN support such semi-supervised learning.

Original: We randomly pick 50 , 000 labelled images while we use the rest three million as unlabelled.
Revised: We randomly pick 50 , 000 labelled images while we use the rest three million as unlabelled.

Original: The ‘label’ in our case is the corrupted images.
Revised: The ‘label’ in our case is the corrupted images.

Original: The baseline model is trained with the labelled 50 ,samples.
Revised: The baseline model is trained with the labelled 50 ,samples.

Original: RoCGAN model is trained with 50 , 000 images in the reg pathway, while the AE pathway with all the available (unlabelled) samples.
Revised: RoCGAN model is trained with 50 , 000 images in the reg pathway, while the AE pathway with all the available (unlabelled) samples.

Original: Table.
Revised: Table.

Original: 5 includes the quantitative results of the semi-supervised case.
Revised: 5 includes the quantitative results of the semi-supervised case.

Original: As expected the performance in most experiments drops from the full training case, however we observe that the performance in
Revised: As expected the performance in most experiments drops from the full training case, however we observe that the performance in

Original: RoCGAN decreases signiﬁcantly less than cGAN
Revised: RoCGAN decreases signiﬁcantly less than cGAN

Original: (‘baseline-4layer-50k’).
Revised: (‘baseline-4layer-50k’).

Original: In other words, RoCGAN can beneﬁt greatly from additional examples in the target domain.
Revised: In other words, RoCGAN can beneﬁt greatly from additional examples in the target domain.

Original: We hypothesize that this enables the AE pathway to learn a more accurate representation, which is reﬂected to the ﬁnal RoCGAN outcome.
Revised: We hypothesize that this enables the AE pathway to learn a more accurate representation, which is reﬂected to the ﬁnal RoCGAN outcome.

Original: D. A LTERNATIVE ERROR METRIC
Revised: D. A LTERNATIVE ERROR METRIC

Original: In our experimental setting every input image should be mapped (close) to its target image.
Revised: In our experimental setting every input image should be mapped (close) to its target image.

Original: To assess the domain-speciﬁc performance for faces, we utilize the cosine distance distribution plot (CDDP).
Revised: To assess the domain-speciﬁc performance for faces, we utilize the cosine distance distribution plot (CDDP).

Original: One of the core features in images of faces is the identity of the person.
Revised: One of the core features in images of faces is the identity of the person.

Original: We utilize the well-studied recognition embeddings (Schroff et al., 2015) to evaluate the similarity of the target image with the outputs of compared methods.
Revised: We utilize the well-studied recognition embeddings (Schroff et al., 2015) to evaluate the similarity of the target image with the outputs of compared methods.

Original: The ground-truth identities in our case are not available in the embeddings’ space; we consider instead the target image’s embedding as the ground-truth for each
Revised: The ground-truth identities in our case are not available in the embeddings’ space; we consider instead the target image’s embedding as the ground-truth for each

Original: [Table] [Table]
Revised: [Table] [Table]

Original: 4 ×
Revised: 4 ×

Original: [Equation]
Revised: [Equation]

Original: comparison.
Revised: comparison.

Original: The plot is constructed as follows: For each pair of output and corresponding target image, we compute the cosine distance of their embeddings; the cumulative distribution of those distances is plotted.
Revised: The plot is constructed as follows: For each pair of output and corresponding target image, we compute the cosine distance of their embeddings; the cumulative distribution of those distances is plotted.

Original: Mathematically the distance of the n th pair is formulated as:
Revised: Mathematically the distance of the n th pair is formulated as:

Original: [Equation]
Revised: [Equation]

Original: where o ( n ) denotes the output of each method, y ( n ) the respective target image and Φ is the function computing the embedding.
Revised: where o ( n ) denotes the output of each method, y ( n ) the respective target image and Φ is the function computing the embedding.

Original: A perfect reconstruction per comparison, e.g. F ( y ( n ) , y ( n ) ) , would yield a plot of a Dirac delta around one; a narrow distribution centered at one denotes proximity to the target images’ embeddings.
Revised: A perfect reconstruction per comparison, e.g. F ( y ( n ) , y ( n ) ) , would yield a plot of a Dirac delta around one; a narrow distribution centered at one denotes proximity to the target images’ embeddings.

Original: The plot with the CDDP is visualized in Fig.
Revised: The plot with the CDDP is visualized in Fig.

Original: 11 for the ‘4layer’ case as detailed in the main paper.
Revised: 11 for the ‘4layer’ case as detailed in the main paper.

Original: The results illustrate that AAE has embeddings that are closer to the target embeddings as expected; from the compared methods the RoCGAN outperform the cGAN in the proximity to the target embeddings.
Revised: The results illustrate that AAE has embeddings that are closer to the target embeddings as expected; from the compared methods the RoCGAN outperform the cGAN in the proximity to the target embeddings.

Original: D. A DDITIONAL EXPERIMENTAL DETAILS
Revised: D. A DDITIONAL EXPERIMENTAL DETAILS

Original: All the images utilized in this work are resized to 64 × 64 × 3 .
Revised: All the images utilized in this work are resized to 64 × 64 × 3 .

Original: In the case of natural scenes, instead of rescaling the images during the training stage, we crop random patches in every iteration from the image.
Revised: In the case of natural scenes, instead of rescaling the images during the training stage, we crop random patches in every iteration from the image.

Original: We utilize the ADAM optimizer with a learning rate of 2 · 10 − 5 for all our experiments.
Revised: We utilize the ADAM optimizer with a learning rate of 2 · 10 − 5 for all our experiments.

Original: The batch size is 128 for images of faces and 64 for the natural scenes.
Revised: The batch size is 128 for images of faces and 64 for the natural scenes.

Original: In table 6 the details about the layer structure for the ‘4layer’ generator are provided; the other networks include similar architecture as depicted in tables 7, 8.
Revised: In table 6 the details about the layer structure for the ‘4layer’ generator are provided; the other networks include similar architecture as depicted in tables 7, 8.

Original: The discriminator retains the same structure in all the experiments in this work (see table 9).
Revised: The discriminator retains the same structure in all the experiments in this work (see table 9).

Original: E A BLATION STUDY
Revised: E A BLATION STUDY

Original: In the following paragraphs we conduct an ablation study to assess RoCGAN in different cases, i.e. effect of hyper-parameters, loss terms, additional noise.
Revised: In the following paragraphs we conduct an ablation study to assess RoCGAN in different cases, i.e. effect of hyper-parameters, loss terms, additional noise.

Original: Unless mentioned otherwise, the architecture used is the ‘4layer’ network.
Revised: Unless mentioned otherwise, the architecture used is the ‘4layer’ network.

Original: The experiments are in face denoising with the similarity metric (SSIM) and the setup similar to the main paper comparisons.
Revised: The experiments are in face denoising with the similarity metric (SSIM) and the setup similar to the main paper comparisons.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original: ’inp’ sparse inpainting.
Revised: ’inp’ sparse inpainting.

Original: E. H YPER - PARAMETER RANGE
Revised: E. H YPER - PARAMETER RANGE

Original: Our model introduces three new loss terms, i.e. L lat , L AE and L decov (in the case with skip) with respect to the baseline cGAN.
Revised: Our model introduces three new loss terms, i.e. L lat , L AE and L decov (in the case with skip) with respect to the baseline cGAN.

Original: Understandably, those introduce three new hyper-parameters, which need to be validated.
Revised: Understandably, those introduce three new hyper-parameters, which need to be validated.

Original: The validation and selection of the hyper-parameters was done in a withheld set of images.
Revised: The validation and selection of the hyper-parameters was done in a withheld set of images.

Original: In the following paragraphs, we design an experiment where we scrutinize one hyperparameter every time, while we keep the rest in their selected value.
Revised: In the following paragraphs, we design an experiment where we scrutinize one hyperparameter every time, while we keep the rest in their selected value.

Original: During our experimentation, we observed that the optimal values of these three hyper-parameters might differ per case/network, however in this manuscript the hyper-parameters remain the same throughout our experimentation .
Revised: During our experimentation, we observed that the optimal values of these three hyper-parameters might differ per case/network, however in this manuscript the hyper-parameters remain the same throughout our experimentation .

Original: [Figure]
Revised: [Figure]

Original: The search space for each term is decided from its theoretical properties and our intuition.
Revised: The search space for each term is decided from its theoretical properties and our intuition.

Original: For instance, the λ ae would have a value similar to the λ c 7 .
Revised: For instance, the λ ae would have a value similar to the λ c 7 .

Original: In a similar manner, the latent loss encourages the two streams’ latent representations to be similar, however the ﬁnal evaluation is performed in the pixel space, hence we assume that a value smaller than λ c is appropriate.
Revised: In a similar manner, the latent loss encourages the two streams’ latent representations to be similar, however the ﬁnal evaluation is performed in the pixel space, hence we assume that a value smaller than λ c is appropriate.

Original: In table
Revised: In table

Original: [Table]
Revised: [Table]

Original: we assess different values for the λ l .
Revised: we assess different values for the λ l .

Original: The results demonstrate that values larger thanthe results are similar, which dictates that our model resilient to the precise selection of the latent loss hyper-parameter.
Revised: The results demonstrate that values larger thanthe results are similar, which dictates that our model resilient to the precise selection of the latent loss hyper-parameter.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: network remains robust for a wide range of values of the hyper-parameter λ ae ; for λ ae > = 20 the test results demonstrate a similar performance.
Revised: network remains robust for a wide range of values of the hyper-parameter λ ae ; for λ ae > = 20 the test results demonstrate a similar performance.

Original: Different values of λ ae are considered in table 11.
Revised: Different values of λ ae are considered in table 11.

Original: RocGAN are robust to a wide range of values and both the visual and the quantitative results remain similar.
Revised: RocGAN are robust to a wide range of values and both the visual and the quantitative results remain similar.

Original: Even though the best results are obtained with λ ae = 250 , we select λ ae = 100 for our experiments.
Revised: Even though the best results are obtained with λ ae = 250 , we select λ ae = 100 for our experiments.

Original: The difference for the two choices is marginal, thus we choose the value 100 since resonates with our intuition ( λ ae = λ c ).
Revised: The difference for the two choices is marginal, thus we choose the value 100 since resonates with our intuition ( λ ae = λ c ).

Original: The third term of λ decov is scrutinized in the table 12.
Revised: The third term of λ decov is scrutinized in the table 12.

Original: In our experimentation, the λ decov has a different effect per experiment; based on the results of our validation we choose λ decov = 1 for our experiments.
Revised: In our experimentation, the λ decov has a different effect per experiment; based on the results of our validation we choose λ decov = 1 for our experiments.

Original: In conclusion, the additional hyper-parameters introduced by our model can accept a range of values without affecting signiﬁcantly the results.
Revised: In conclusion, the additional hyper-parameters introduced by our model can accept a range of values without affecting signiﬁcantly the results.

Original: [Table]
Revised: [Table]

Original: S IGNIFICANCE OF LOSS TERMS
Revised: S IGNIFICANCE OF LOSS TERMS

Original: To study further the signiﬁcance of the four loss terms, we experiment with setting λ ∗ = 0 alternatingly.
Revised: To study further the signiﬁcance of the four loss terms, we experiment with setting λ ∗ = 0 alternatingly.

Original: Apart from the ‘4layer’ network, we implement the ‘4layer-skip’ to assess the λ decov = 0 case.
Revised: Apart from the ‘4layer’ network, we implement the ‘4layer-skip’ to assess the λ decov = 0 case.

Original: The ‘4layer-skip’ includes the same layers as the ‘4layer’, however it includes a lateral connection from the encoder to the decoder.
Revised: The ‘4layer-skip’ includes the same layers as the ‘4layer’, however it includes a lateral connection from the encoder to the decoder.

Original: The experimental results in table 13 conﬁrm our prior intuition that the latent loss ( L lat ) is the most crucial for our model in the no-skip case, but not as signiﬁcant in the skip case.
Revised: The experimental results in table 13 conﬁrm our prior intuition that the latent loss ( L lat ) is the most crucial for our model in the no-skip case, but not as signiﬁcant in the skip case.

Original: In the skip case, the reconstruction losses in both pathways are signiﬁcant.
Revised: In the skip case, the reconstruction losses in both pathways are signiﬁcant.

Original: [Table]
Revised: [Table]

Original: E. A DDITIONAL NOISE
Revised: E. A DDITIONAL NOISE

Original: To evaluate whether RoCGAN/cGAN are resilient to noise, we experiment with additional noise.
Revised: To evaluate whether RoCGAN/cGAN are resilient to noise, we experiment with additional noise.

Original: We include a baseline cGAN to the comparison to study whether their performance changes similarly.
Revised: We include a baseline cGAN to the comparison to study whether their performance changes similarly.

Original: Both networks are trained with the 25% noise (denosing task).
Revised: Both networks are trained with the 25% noise (denosing task).

Original: We evaluate the performance in two cases: i) additional noise of the same type, ii) additional noise of different type.
Revised: We evaluate the performance in two cases: i) additional noise of the same type, ii) additional noise of different type.

Original: For this experiment, we abbreviate noise as x/y where x depicts the amount of noise in denoising task (i.e. x % of the pixels in each channel are dropped with a uniform probability) and
Revised: For this experiment, we abbreviate noise as x/y where x depicts the amount of noise in denoising task (i.e. x % of the pixels in each channel are dropped with a uniform probability) and

Original: y the sparse inpainting task (i.e. y % black pixels).
Revised: y the sparse inpainting task (i.e. y % black pixels).

Original: In both cases, we evaluate the performance by incrementally increasing the amount of noise.
Revised: In both cases, we evaluate the performance by incrementally increasing the amount of noise.

Original: Speciﬁcally, both networks are tested in 25 / 0 , 35 / 0 , 50 / 0 for noise of the same type and 25 / 10 , 25 / 20 and 25 / 25 for different type of noise.
Revised: Speciﬁcally, both networks are tested in 25 / 0 , 35 / 0 , 50 / 0 for noise of the same type and 25 / 10 , 25 / 20 and 25 / 25 for different type of noise.

Original: We note the networks have not been trained on any of the testing noises other than the 25 / 0 case.
Revised: We note the networks have not been trained on any of the testing noises other than the 25 / 0 case.

Original: [Table]
Revised: [Table]

Original: The quantitative results are exported in table 14.
Revised: The quantitative results are exported in table 14.

Original: The results dictate that while cGAN are volatile to additional amount of noise, RoCGAN’s performance is more stable.
Revised: The results dictate that while cGAN are volatile to additional amount of noise, RoCGAN’s performance is more stable.

Original: This becomes even more distinct in the case of different type of noise, where the relative improvement can be up to 17 .
Revised: This becomes even more distinct in the case of different type of noise, where the relative improvement can be up to 17 .

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: To illustrate the difference of performance between the two models, we accumulate the SSIM values of each case and divide them in 20 bins 8 .
Revised: To illustrate the difference of performance between the two models, we accumulate the SSIM values of each case and divide them in 20 bins 8 .

Original: In Fig.
Revised: In Fig.

Original: 13, the histograms of each case are plotted.
Revised: 13, the histograms of each case are plotted.

Original: We note that RoCGAN is much more resilient to increased or even unseen noise.
Revised: We note that RoCGAN is much more resilient to increased or even unseen noise.

Original: Qualitative results of the
Revised: Qualitative results of the

Original: [Table]
Revised: [Table]

Original: difference are offered in Fig.
Revised: difference are offered in Fig.

Original: We consider that this improvement in the robustness in the face of additional noise is in its own a considerable improvement to the original cGAN.
Revised: We consider that this improvement in the robustness in the face of additional noise is in its own a considerable improvement to the original cGAN.

Original: E. A DVERSARIAL EXAMPLES
Revised: E. A DVERSARIAL EXAMPLES

Original: Apart from testing in the face of additional noise, we explore the adversarial attacks.
Revised: Apart from testing in the face of additional noise, we explore the adversarial attacks.

Original: Recent works (Szegedy et al., 2014; Yuan et al., 2017; Samangouei et al., 2018) explore the robustness of (deep) classiﬁers.
Revised: Recent works (Szegedy et al., 2014; Yuan et al., 2017; Samangouei et al., 2018) explore the robustness of (deep) classiﬁers.

Original: Adversarial attacks modify the input image (of the network) so that the network misclassiﬁes the image.
Revised: Adversarial attacks modify the input image (of the network) so that the network misclassiﬁes the image.

Original: To the authors’ knowledge, there has not been much investigation of adversarial attacks in the context of image-to-image translation or any other regression task.
Revised: To the authors’ knowledge, there has not been much investigation of adversarial attacks in the context of image-to-image translation or any other regression task.

Original: However, if we consider adversarial examples as a perturbation of the original input, we explore whether this has any effect in the methods.
Revised: However, if we consider adversarial examples as a perturbation of the original input, we explore whether this has any effect in the methods.

Original: Neither cGAN nor RoCGAN are designed to be robust in adversarial examples, however in this section we explore how adversarial examples can affect them.
Revised: Neither cGAN nor RoCGAN are designed to be robust in adversarial examples, however in this section we explore how adversarial examples can affect them.

Original: We consider the FGSM method of
Revised: We consider the FGSM method of

Original: Goodfellow et al. (2015) as one of the ﬁrst and simplest methods for generating adversarial examples.
Revised: Goodfellow et al. (2015) as one of the ﬁrst and simplest methods for generating adversarial examples.

Original: In our case, we modify each source signal s as:
Revised: In our case, we modify each source signal s as:

Original: [Equation]
Revised: [Equation]

Original: where η is the perturbation.
Revised: where η is the perturbation.

Original: That is deﬁned as:
Revised: That is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: with (cid:15) a hyper-parameter, y the target signal and L the loss.
Revised: with (cid:15) a hyper-parameter, y the target signal and L the loss.

Original: In our case, we select the (cid:96) 1 loss as the loss between the target and the generated images.
Revised: In our case, we select the (cid:96) 1 loss as the loss between the target and the generated images.

Original: We set (cid:15) = 0 .
Revised: We set (cid:15) = 0 .

Original: 01 following the original paper.
Revised: 01 following the original paper.

Original: The evaluation is added in table 15.
Revised: The evaluation is added in table 15.

Original: The results demonstrate that the baseline is affected more from the added noise, while our model is more robust to it.
Revised: The results demonstrate that the baseline is affected more from the added noise, while our model is more robust to it.

Original: [Table]
Revised: [Table]