Original: Evaluating Robust Perceptual Losses for Image Reconstruction
Revised: Evaluating Robust Perceptual Losses for Image Reconstruction

Original: Abstract
Revised: Abstract

Original: Nowadays, many deep neural networks (DNNs) for image reconstructing tasks are trained using a combination of pixel-wise loss functions and perceptual image losses like learned perceptual image patch similarity (LPIPS).
Revised: Nowadays, many deep neural networks (DNNs) for image reconstructing tasks are trained using a combination of pixel-wise loss functions and perceptual image losses like learned perceptual image patch similarity (LPIPS).

Original: As these perceptual image losses compare the features of a pre-trained DNN, it is unsurprising that they are vulnerable to adversarial examples.
Revised: As these perceptual image losses compare the features of a pre-trained DNN, it is unsurprising that they are vulnerable to adversarial examples.

Original: It is known that: (i) DNNs can be robustified against adversarial examples using adversarial training, and (ii) adversarial examples are imperceptible by the human eye.
Revised: It is known that: (i) DNNs can be robustified against adversarial examples using adversarial training, and (ii) adversarial examples are imperceptible by the human eye.

Original: Thus, we hypothesize that perceptual metrics, based on a robustly trained DNN, are more aligned with human perception than those based on non-robust models.
Revised: Thus, we hypothesize that perceptual metrics, based on a robustly trained DNN, are more aligned with human perception than those based on non-robust models.

Original: Our extensive experiments on an image super resolution task show, however, that this is not the case.
Revised: Our extensive experiments on an image super resolution task show, however, that this is not the case.

Original: We observe that models trained with a robust perceptual loss tend to produce more artifacts in the reconstructed image.
Revised: We observe that models trained with a robust perceptual loss tend to produce more artifacts in the reconstructed image.

Original: Furthermore, we were unable to find reliable image similarity metrics or evaluation methods to quantify these observations (which are known open problems).
Revised: Furthermore, we were unable to find reliable image similarity metrics or evaluation methods to quantify these observations (which are known open problems).

Original:  Introduction
Revised:  Introduction

Original: For image reconstruction methods that involve deep neural networks (DNNs), the choice of the loss function is crucial for good reconstruction results.
Revised: For image reconstruction methods that involve deep neural networks (DNNs), the choice of the loss function is crucial for good reconstruction results.

Original: Pixel-wise loss functions, like the mean squared error (MSE) loss, are often a reasonable choice, but often do not completely align with human perception.
Revised: Pixel-wise loss functions, like the mean squared error (MSE) loss, are often a reasonable choice, but often do not completely align with human perception.

Original: For example, it is well known that using the pixel-wise MSE loss leads to blurry reconstructions over reconstructions having misplaced edges.
Revised: For example, it is well known that using the pixel-wise MSE loss leads to blurry reconstructions over reconstructions having misplaced edges.

Original: Nowadays, modern image reconstruction networks are usually trained to minimize a combination of different loss functions.
Revised: Nowadays, modern image reconstruction networks are usually trained to minimize a combination of different loss functions.

Original: While pixel-wise losses are still a crucial ingredient (because they create a strong error signal) the reconstruction quality can be greatly improved by simultaneously minimizing multiple losses, e.g., an adversarial loss or a perceptual loss.
Revised: While pixel-wise losses are still a crucial ingredient (because they create a strong error signal) the reconstruction quality can be greatly improved by simultaneously minimizing multiple losses, e.g., an adversarial loss or a perceptual loss.

Original: In this work, we focus on the latter.
Revised: In this work, we focus on the latter.

Original: 1.1 Perceptual Losses
Revised: 1.1 Perceptual Losses

Original: Instead of considering the error in pixel space, a perceptual loss calculates the error between the feature maps of an already pre-trained and fixed classification DNN.
Revised: Instead of considering the error in pixel space, a perceptual loss calculates the error between the feature maps of an already pre-trained and fixed classification DNN.

Original: This was first done by Johnson et al.
Revised: This was first done by Johnson et al.

Original: [1] who define a perceptual loss between to images x ∈ R c × h × w and y ∈ R c × h × w as
Revised: [1] who define a perceptual loss between to images x ∈ R c × h × w and y ∈ R c × h × w as

Original: [Equation]
Revised: [Equation]

Original: where Φ ( p ) : R c × [Equation] × w → R c p × ( h p w p ) denotes the flattened feature mapping of the VGG-19 net [2] after the p -th layer and ∥·∥ F denotes the Frobenius norm.
Revised: where Φ ( p ) : R c × h × w → R c p × ( h p w p ) denotes the flattened feature mapping of the VGG-19 net [2] after the p -th layer and ∥·∥ F denotes the Frobenius norm.

Original: This is motivated by the fact that the feature maps learned by a classifier specialize on different concepts.
Revised: This is motivated by the fact that the feature maps learned by a classifier specialize on different concepts.

Original: Usually, the first layers of a DNN detect low-level features like edges and corners, while the deeper layers detect high-level features that represent more abstract concepts.
Revised: Usually, the first layers of a DNN detect low-level features like edges and corners, while the deeper layers detect high-level features that represent more abstract concepts.

Original: Thus, it is common practice to combine the perceptual loss of different layers into a single perceptual loss, e.g., the perceptual loss as a sum of perceptual losses
Revised: Thus, it is common practice to combine the perceptual loss of different layers into a single perceptual loss, e.g., the perceptual loss as a sum of perceptual losses

Original: [Equation]
Revised: [Equation]

Original: perceptual
Revised: x

Original: [Equation]
Revised: [Equation]

Original: x, y ) , where P ⊂ N is a set of layer indices and positive weights λ p > 0 for p ∈ P .
Revised: y ) , where P ⊂ N is a set of layer indices and positive weights λ p > 0 for p ∈ P .

Original: A different way of defining a perceptual loss that considers local texture information is the texture matching loss which compares the Gram matrices of the feature maps [3].
Revised: A different way of defining a perceptual loss that considers local texture information is the texture matching loss which compares the Gram matrices of the feature maps [3].

Original: Sajjadi et al.
Revised: Sajjadi et al.

Original: [4] define this loss for k patches x (1) , . . .
Revised: [4] define this loss for k patches x (1) , . . .

Original: , x ( k ) and y (1) , . . . ,
Revised: , x ( k ) and y (1) , . . . ,

Original: y ( k ) of the input images x and y , respectively, as
Revised: y ( k ) of the input images x and y , respectively, as

Original: [Equation]
Revised: [Equation]

Original: (2) where G ( p ) calculates the Gram matrix of the feature map Φ ( p ) .
Revised: (2) where G ( p ) calculates the Gram matrix of the feature map Φ ( p ) .

Original: Again, different layers can be added to form the total loss L Φ texture-matching ( x, y ) =
Revised: Again, different layers can be added to form the total loss L Φ texture-matching ( x, y ) =

Original: (cid:80) p ∈P λ p L Φ ,p texture-matching ( x, y ) for some P ⊂ N and positive weights λ p > 0 for p ∈ P .
Revised: (cid:80) p ∈P λ p L Φ ,p texture-matching ( x, y ) for some P ⊂ N and positive weights λ p > 0 for p ∈ P .

Original: 1.2 Adversarial Examples and Adversarial Training
Revised: 1.2 Adversarial Examples and Adversarial Training

Original: It is common knowledge that DNNs can easily be fooled by adversarial examples, i.e., manipulated images that change the network’s prediction [5].
Revised: It is common knowledge that DNNs can easily be fooled by adversarial examples, i.e., manipulated images that change the network’s prediction [5].

Original: These adversarial examples are usually unrecognizable by the human eye.
Revised: These adversarial examples are usually unrecognizable by the human eye.

Original: Recent work has investigated how DNNs can be robustified against these adversarial examples.
Revised: Recent work has investigated how DNNs can be robustified against these adversarial examples.

Original: A widely-used approach is adversarial training which robustifies DNNs by including adversarial images (with the correct label) during training [6].
Revised: A widely-used approach is adversarial training which robustifies DNNs by including adversarial images (with the correct label) during training [6].

Original: Usually, adversarial training relies on the PGD- K  attack  [6; 7] which crafts an adversarial image (that the model wrongly assigns the class label y target ) by iterating
Revised: Usually, adversarial training applies K steps of the projected gradient descent attack (PGD- K ) [6; 7] which crafts an adversarial image (that the model wrongly assigns the class label y target ) by iterating

Original: [Equation]
Revised: [Equation]

Original:  where x 0 = x is the original image x , α > 0 is the step-size, and P ϵ is a function that projects the iterate back onto an norm-ball B ϵ ( x ) around x .
Revised: (3) where x 0 = x is the original image x , α > 0 is the step-size, and P ϵ is a function that projects the iterate back onto an norm-ball B ϵ ( x ) around x .

Original: After K iterations we obtain the adversarial example ¯ x = x K .
Revised: After K iterations we obtain the adversarial example ¯ x = x K .

Original: The change made to the original image x − ¯ x is usually referred to as a perturbation.
Revised: The change made to the original image x − ¯ x is usually referred to as a perturbation.

Original: These adversarial images combined with the true label are then included in the training dataset.
Revised: These adversarial images combined with the true label are then included in the training dataset.

Original: The reasoning is that the DNN classifier should learn to also classify theses adversarial images correctly.
Revised: The reasoning is that the DNN classifier should learn to also classify theses adversarial images correctly.

Original: In general, the resulting robust classifiers tend to have a slightly worse accuracy, but they are, at the same time, less susceptible to adversarial examples.
Revised: In general, the resulting robust classifiers tend to have a slightly worse accuracy, but they are, at the same time, less susceptible to adversarial examples.

Original: To further strengthen a model’s robustness adversarial logit pairing (ALP) [8] can be used.
Revised: To further strengthen a model’s robustness adversarial logit pairing (ALP) [8] can be used.

Original: The key idea of ALP is that the logits of the classifier
Revised: The key idea of ALP is that the logits of the classifier

Original: F θ ( x ) should be invariant under adversarial perturbations.
Revised: F θ ( x ) should be invariant under adversarial perturbations.

Original: This is done by including an additional regularization term to the loss function used to train the model that encourages logits of adversarial examples F θ (¯ x ) to be similar to those of clean examples F θ ( x ) .
Revised: This is done by including an additional regularization term to the loss function used to train the model that encourages logits of adversarial examples F θ (¯ x ) to be similar to those of clean examples F θ ( x ) .

Original: This regularization term is given as
Revised: This regularization term is given as

Original: [Equation]
Revised: [Equation]

Original: where d is the number of classes.
Revised: where d is the number of classes.

Original: In this paper, we use these two ideas to train the robust classifier used for the perceptual losses.
Revised: In this paper, we use these two ideas to train the robust classifier used for the perceptual losses.

Original: Since DNNs can be affected by adversarial examples, it is unsurprising that perceptual losses are also susceptible to them, as shown by Kettunen et al.
Revised: Since DNNs can be affected by adversarial examples, it is unsurprising that perceptual losses are also susceptible to them, as shown by Kettunen et al.

Original: For many attacks, the human eye is not able to differentiate between the adversarial examples and the original images.
Revised: For many attacks, the human eye is not able to differentiate between the adversarial examples and the original images.

Original: This motivates our hypothesis which we analyze in this paper:
Revised: This motivates our hypothesis which we analyze in this paper:

Original: Hypothesis: Feature maps produced by robust DNNs are more aligned with human perception and should therefore be more suitable to be used in perceptual losses in image reconstruction problems.[Equation]
Revised: Hypothesis: Feature maps produced by robust DNNs are more aligned with human perception and should therefore be more suitable to be used in perceptual losses in image reconstruction problems.

Original: [Table]
Revised: [Table]

Original: Our hypothesis is loosely inspired by Rojas-Gomez et al.
Revised: Our hypothesis is loosely inspired by Rojas-Gomez et al.

Original: [10] who have shown that feature-inversion for robust networks is easier than for non-robust networks.
Revised: [10] who have shown that feature-inversion for robust networks is easier than for non-robust networks.

Original: Moreover, Ross and Doshi-Velez [11] argue that robust DNNs are more human interpretable.
Revised: Moreover, Ross and Doshi-Velez [11] argue that robust DNNs are more human interpretable.

Original:  Experimental Setup
Revised:  Experimental Setup

Original: In the following, we want to evaluate the effectiveness of robust perceptual losses for 4 × -superresolution of images, where we use an EnhanceNet [4] as a base model that learns to upscale the images.
Revised: In the following, we want to evaluate the effectiveness of robust perceptual losses for 4 × -superresolution of images, where we use an EnhanceNet [4] as a base model that learns to upscale the images.

Original: Training: Similar to the original work, we train the EnhanceNet [4] on the MS COCO dataset [12] using various loss functions: a pixel-wise MSE loss (E), a perceptual loss (P), or a texture matching loss (T).
Revised: Training: Similar to the original work, we train the EnhanceNet [4] on the MS COCO dataset [12] using various loss functions: a pixel-wise MSE loss (E), a perceptual loss (P), or a texture matching loss (T).

Original: These losses can also be combined, e.g., EP indicates an EnhanceNet trained with both the pixel-wise MSE (E) and the perceptual (P) losses.
Revised: These losses can also be combined, e.g., EP indicates an EnhanceNet trained with both the pixel-wise MSE (E) and the perceptual (P) losses.

Original: Both the perceptual and the texture matching loss (P and T) rely on a VGG-19 net, which is trained on the ImageNet dataset [13].
Revised: Both the perceptual and the texture matching loss (P and T) rely on a VGG-19 net, which is trained on the ImageNet dataset [13].

Original: For the P loss we use feature maps of the second ( λ 2 = 0 . 2 ) and the fifth ( λ 5 = 0 .
Revised: For the P loss we use feature maps of the second ( λ 2 = 0 . 2 ) and the fifth ( λ 5 = 0 .

Original: 02 ) convolutional layer (after applying the ReLU activation function) and for the T loss we use the feature maps of the first ( λ 1 = 3 × 10 − 7 ), the third ( λ 3 = 1 × 10 − 6 ), and the fifth ( λ 5 = 1 × 10 − 6 ) convolutional layer.
Revised: 02 ) convolutional layer (after applying the ReLU activation function) and for the T loss we use the feature maps of the first ( λ 1 = 3 × 10 − 7 ), the third ( λ 3 = 1 × 10 − 6 ), and the fifth ( λ 5 = 1 × 10 − 6 ) convolutional layer.

Original: The texture matching loss is calculated using 16 -by- 16 -patches.
Revised: The texture matching loss is calculated using 16 -by- 16 -patches.

Revised: These hyperparameters were taken from the original works.

Original: We aim to test our hypothesis by comparing the performance of an EnhanceNet trained with a perceptual loss based on a robust VGG-19 net instead of the commonly used non-robust VGGnet.
Revised: We aim to test our hypothesis by comparing the performance of an EnhanceNet trained with a perceptual loss based on a robust VGG-19 net instead of the commonly used non-robust VGGnet.

Original: We do the same for the texture matching loss.
Revised: We do the same for the texture matching loss.

Original: When we train the EnhanceNet with the P and the T loss we use the same VGG-19 net (either robust or non-robust) for both losses.
Revised: When we train the EnhanceNet with the P and the T loss we use the same VGG-19 net (either robust or non-robust) for both losses.

Original: To train a robust VGG-19 net, we use adversarial training [6] based on the PGD-5 attack with a Euclidean norm constraint of 5 . 82 .
Revised: To train a robust VGG-19 net, we use adversarial training [6] based on the PGD-5 attack with a Euclidean norm constraint of 5 . 82 .

Original: We combined this with ALP [8] with an ALP weight of 0 .
Revised: We combined this with ALP [8] with an ALP weight of 0 .

Original: 5 (see, Section 1.2).
Revised: 5 (see, Section 1.2).

Original: We carefully scaled the weights and biases of both VGG-19 nets such that the feature maps are normalized.
Revised: We carefully scaled the weights and biases of both VGG-19 nets such that the feature maps are normalized.

Original: Note, this does not change the output because of the used ReLU activation functions.
Revised: Note, this does not change the output because of the used ReLU activation functions.

Revised: In the first two rows of Table 1, we compare the accuracies on the ImageNet dataset (with and without adversarial perturbations) of the robust and non-robust VGG-19 networks used for the perceptual and texture-matching loss.

Revised: As expected, the robust models have a slightly lower accuracy than the non-robust ones, but they perform drastically better on the adversarial images.

Original: Evaluation: The EnhanceNets trained using different robust and non-robust perceptual loss functions are evaluated on common super-resolution benchmark datasets: BSD100 [14], Manga109 [15],
Revised: Evaluation: The EnhanceNets trained using different robust and non-robust perceptual loss functions are evaluated on common super-resolution benchmark datasets: BSD100 [14], Manga109 [15],

Original: Set14 [16], Set5 [17], and Urban100 [18].
Revised: Set14 [16], Set5 [17], and Urban100 [18].

Original: We extensively evaluate the performance using the following metrics: PSNR, SSIM [19], gradient magnitude similarity deviation (GMSD)
Revised: We extensively evaluate the performance using the following metrics: PSNR, SSIM [19], gradient magnitude similarity deviation (GMSD)

Original: [20], information fidelity criterion (IFC) [21], and universal image quality index (UQI) [22].
Revised: [20], information fidelity criterion (IFC) [21], and universal image quality index (UQI) [22].

Original: Furthermore, we use the learned perceptual image patch similarity (LPIPS) [23] that is based on a
Revised: Furthermore, we use the learned perceptual image patch similarity (LPIPS) [23] that is based on a

Original: VGG-16 network [2] with additional linear layers to output a single value (the LPIPS score) for a pair of input images.
Revised: VGG-16 network [2] with additional linear layers to output a single value (the LPIPS score) for a pair of input images.

Original: The dataset used during training is collected using a two-alternative forced choice method and consists of images each having two transformed variants.
Revised: The dataset used during training is collected using a two-alternative forced choice method and consists of images each having two transformed variants.

Original: Each of these triplets (original image and two variants) is accompanied by a human-made decision about which of the two variants is perceived to be more similar to the original image.
Revised: Each of these triplets (original image and two variants) is accompanied by a human-made decision about which of the two variants is perceived to be more similar to the original image.

Original: The network is trained to mimic this human-made decision based on the difference of the LPIPS scores.
Revised: The network is trained to mimic this human-made decision based on the difference of the LPIPS scores.

Original: As an additional evaluation metric, we propose a robust LPIPS (R-LPIPS) metric which is trained in a similar manner but where the original image and both of its variants are perturbed using a PGDattack constrained to a Euclidean norm of 1 . 33 .

Original: [Table]
Revised: [Table]

Original: EP
Revised: EP

Original: EPT
Revised: EPT

Original: [Figure]
Revised: [Figure]

Revised: As an additional evaluation metric, we propose a robust LPIPS (R-LPIPS) metric which is trained in a similar manner but where the original image and both of its variants are perturbed using a PGDattack constrained to a Euclidean norm of 1 . 33 .

Original: Besides these image similarity metrics we also compare the classification performance of a
Revised: Besides these image similarity metrics we also compare the classification performance of a

Original: ResNet-50 [24] on sub-sampled images from the ImageNet dataset, upscaled using the differently trained EnhanceNets.
Revised: ResNet-50 [24] on sub-sampled images from the ImageNet dataset, upscaled using the differently trained EnhanceNets.

Original: On the one hand, we consider the classification performance of a robust
Revised: On the one hand, we consider the classification performance of a robust

Original: ResNet-50, trained in a similar manner as the VGG-19 network.
Revised: ResNet-50, trained in a similar manner as the VGG-19 network.

Original: On the other hand, we use a non-robust ResNet-50.
Revised: On the other hand, we use a non-robust ResNet-50.

Original: In Table 1, we compare the accuracies on the ImageNet dataset(with and without adversarial perturbations) of the robust andnon-robust VGG-19 networks used for the perceptual and texturematching loss, as well as, the  ResNet-50 used for evaluation. As expected, the robust models have a slightly lower accuracy than the non-robust ones, but they perform drastically better on the adversarial images.
Revised: The bottom two rows of Table 1 compare the accuracies on the ImageNet dataset of the robust and the non-robust ResNet-50 that we use for evaluation. We observe a similar performance as the VGGnet.

Original:  Results
Revised:  Results

Original: In Table 2, we quantitatively evaluate the differently trained EnhanceNets using the previously introduced metrics.
Revised: In Table 2, we quantitatively evaluate the differently trained EnhanceNets using the previously introduced metrics.

Original: For each set of losses that involved a perceptual loss, we consider the robust and the non-robust variant (indicated in the second column).
Revised: For each set of losses that involved a perceptual loss, we consider the robust and the non-robust variant (indicated in the second column).

Original: We observe that the EnhanceNet trained only with the E-loss performs the best in terms of the PSNR and SSIM metrics.
Revised: We observe that the EnhanceNet trained only with the E-loss performs the best in terms of the PSNR and SSIM metrics.

Original: This is not surprising as the pixel-wise MSE loss (E) and the PSNR metric are strongly related.
Revised: This is not surprising as the pixel-wise MSE loss (E) and the PSNR metric are strongly related.

Original: Comparing the EnhanceNetstrained with robust and non-robust perceptual losses, we are unable to conclude whether the robust perceptual loss improves the performance, e.g., for the SSIM metric, the robust perceptual losses are,
Revised: Comparing the EnhanceNets

Original: [Figure]
Revised: [Figure]

Original:  in many cases, better than their non-robust counterpart, but for the GMSD metric, they are always worse.
Revised: trained with robust and non-robust perceptual losses, we are unable to conclude whether the robust perceptual loss improves the performance, e.g., for the SSIM metric, the robust perceptual losses are, in many cases, better than their non-robust counterpart, but for the GMSD metric, they are always worse.

Original: Moreover, we observe that the EnhanceNet trained with the robust EP loss performs the best on the R-LPIPS metric.
Revised: Moreover, we observe that the EnhanceNet trained with the robust EP loss performs the best on the R-LPIPS metric.

Original: As previously observed, it is challenging to quantitatively conclude which EnhanceNet produces the best images.
Revised: As previously observed, it is challenging to quantitatively conclude which EnhanceNet produces the best images.

Original: Thus, following the reasoning of Sajjadi et al.
Revised: Thus, following the reasoning of Sajjadi et al.

Original: [4], we argue that the classification error of a DNN classifier correlates with the image quality perceived by humans.
Revised: [4], we argue that the classification error of a DNN classifier correlates with the image quality perceived by humans.

Original: To this end, we plot the classification errors of the robust and non-robust ResNet-50 classifiers in Figure 1.
Revised: To this end, we plot the classification errors of the robust and non-robust ResNet-50 classifiers in Figure 1.

Original: The classification error of the restored images using the EnhanceNet-E is depicted by the red dashed line.
Revised: The classification error of the restored images using the EnhanceNet-E is depicted by the red dashed line.

Original: We see in
Revised: We see in

Original: Figure 1a that all EnchanceNets trained with a perceptual loss have a lower error than only using the pixel-wise MSE loss.
Revised: Figure 1a that all EnchanceNets trained with a perceptual loss have a lower error than only using the pixel-wise MSE loss.

Original: Moreover, the EnhanceNets trained with robust perceptual losses almost always have a smaller error when compared with the non-robust perceptual losses.
Revised: Moreover, the EnhanceNets trained with robust perceptual losses almost always have a smaller error when compared with the non-robust perceptual losses.

Original: This trend is inverted when we use a non-robust ResNet-50, as depicted in Figure 1b.
Revised: This trend is inverted when we use a non-robust ResNet-50, as depicted in Figure 1b.

Original: In this case, the EnhanceNets trained with a robust perceptual loss always perform worse than the non-robust counterpart, and even worse than the EnhanceNet-E. Therefore, we conclude that the choice of the classifier highly influences the outcome when evaluating the quality of images based on the classification error of a pre-trained
Revised: In this case, the EnhanceNets trained with a robust perceptual loss always perform worse than the non-robust counterpart, and even worse than the EnhanceNet-E. Therefore, we conclude that the choice of the classifier highly influences the outcome when evaluating the quality of images based on the classification error of a pre-trained

Original: DNN.
Revised: DNN.

Original: In the top left corner of Figure 2, we show a crop of the original HD image from the Urban dataset [18].
Revised: In the top left corner of Figure 2, we show a crop of the original HD image from the Urban dataset [18].

Original: The other images show the reconstructions produced by the EnhanceNet trained with different loss functions.
Revised: The other images show the reconstructions produced by the EnhanceNet trained with different loss functions.

Original: In the top right corner, the reconstruction produced by the EnhanceNet-E (which is trained with the pixel-wise MSE loss) is quite blurry.
Revised: In the top right corner, the reconstruction produced by the EnhanceNet-E (which is trained with the pixel-wise MSE loss) is quite blurry.

Original: This is a known problem of the
Revised: This is a known problem of the

Original: MSE loss [25].
Revised: MSE loss [25].

Original: In contrast to that, the reconstruction produced by the EnhanceNet-EPT (bottom left) shows sharper edges and finer details, e.g., the quality of the texture on the ground is superior.
Revised: In contrast to that, the reconstruction produced by the EnhanceNet-EPT (bottom left) shows sharper edges and finer details, e.g., the quality of the texture on the ground is superior.

Original: In the bottom right, we show the reconstruction produced by the EnhanceNet-EPT trained with the robust perceptual and texture matching losses.
Revised: In the bottom right, we show the reconstruction produced by the EnhanceNet-EPT trained with the robust perceptual and texture matching losses.

Original: Comparing this reconstruction to that of the EnhanceNet-EPT, we see there are many more artifacts and “echoes” in the reconstructed image.
Revised: Comparing this reconstruction to that of the EnhanceNet-EPT, we see there are many more artifacts and “echoes” in the reconstructed image.

Original: The reconstructions of other EnhanceNet trained with robust perceptual and/or texture losses also introduce these artifacts.
Revised: The reconstructions of other EnhanceNet trained with robust perceptual and/or texture losses also introduce these artifacts.

Original: More example reconstructions using all considered combinations of loss functions (as seen in Table 2) can be found in the appendix.
Revised: More example reconstructions using all considered combinations of loss functions (as seen in Table 2) can be found in the appendix.

Original:  Discussion
Revised:  Discussion

Original: We hypothesized that DNNs robust to adversarial examples could provide perceptual losses that are more aligned with human perception.
Revised: We hypothesized that DNNs robust to adversarial examples could provide perceptual losses that are more aligned with human perception.

Original: We expected that these robust perceptual losses, when used to train image reconstruction models, would improve the quality of the reconstructed images.
Revised: We expected that these robust perceptual losses, when used to train image reconstruction models, would improve the quality of the reconstructed images.

Original: In this work, we evaluated this hypothesis on the task of image super-resolution.
Revised: In this work, we evaluated this hypothesis on the task of image super-resolution.

Original: We used an
Revised: We used an

Original: EnhanceNet [4] trained with combinations of different loss functions and compare the effectiveness of robust and non-robust perceptual losses on multiple datasets.
Revised: EnhanceNet [4] trained with combinations of different loss functions and compare the effectiveness of robust and non-robust perceptual losses on multiple datasets.

Original: Using a wide range of evaluation metrics (including the novel R-LPIPS metric) we observe that robust perceptual losses do not perform better.
Revised: Using a wide range of evaluation metrics (including the novel R-LPIPS metric) we observe that robust perceptual losses do not perform better.

Original: Furthermore, we observe that these robust perceptual losses lead to unnatural artifacts in the reconstructed images.
Revised: Furthermore, we observe that these robust perceptual losses lead to unnatural artifacts in the reconstructed images.

Original: Thus, our initial hypothesis was falsified by our extensive experiments.
Revised: Thus, our initial hypothesis was falsified by our extensive experiments.

Original: To explain this phenomenon, we now hold the opinion that these robust perceptual losses are not only invariant to changes in the image invisible to humans (such as adversarial perturbations), but are also invariant to more substantial changes recognizable to the human eye.
Revised: To explain this phenomenon, we now hold the opinion that these robust perceptual losses are not only invariant to changes in the image invisible to humans (such as adversarial perturbations), but are also invariant to more substantial changes recognizable to the human eye.

Original: This leads to visible artifacts when training image reconstruction models with these robust perceptual loss functions.
Revised: This leads to visible artifacts when training image reconstruction models with these robust perceptual loss functions.

Original: Appendix: Additional Reconstruction Examples
Revised: Appendix: Additional Reconstruction Examples

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]