QC-StyleGAN - Quality Controllable Image Generation and Manipulation
Authors: Dat Viet Thanh Nguyen
Phong Tran The
Tan M. Dinh
Cuong Pham
Anh Tuan Tran
Venue: NeurIPS 2022 Accept (2022)
Keywords: StyleGAN
GAN Inversion
Image restoration
Image Generation
Image Manipulation
Image Editing
Degradation Synthesis

Abstract: The introduction of high-quality image generation models, particularly the StyleGAN family, provides a powerful tool to synthesize and manipulate images. However, existing models are built upon high-quality (HQ) data as desired outputs, making them unfit for in-the-wild low-quality (LQ) images, which are common inputs for manipulation. In this work, we bridge this gap by proposing a novel GAN structure that allows for generating images with controllable quality. The network can synthesize various image degradation and restore the sharp image via a quality control code. Our proposed QC-StyleGAN can directly edit LQ images without altering their quality by applying GAN inversion and manipulation techniques. It also provides for free an image restoration solution that can handle various degradations, including noise, blur, compression artifacts, and their mixtures. Finally, we demonstrate numerous other applications such as image degradation synthesis, transfer, and interpolation.

QC-StyleGAN Quality Controllable Image
Generation and Manipulation
Dat Viet Thanh Nguyen !:
Phong Tran 1.2,
Tan M. Dinh
Anh Tuan Tran
Cuong Pham 1.3
VinAl Research
MBZUAI
'Posts & Telecommunications Institute of Technology
datattat,, v. vandu3, v.anhtt152, v. cuwnggee1)evinai.io
the traauua ac..
Abstract
The introduction of high- high-quality image generation models, particularly the Style- GAN family, provides a powerful tool to synthesize and manipulate images. How- ever, existing models are built upon high-quality (HQ) data as desired outputs, making them unfit for in-the-wild low-quality (LQ) images, which are common inputs for manipulation. In this work, bridge this gap by proposing novel GAN structure that allows for generating images with controllable quality. The network can synthesize various image degradation and restore the sharp image via quality control code. Our proposed QC-StyleGAN can directly edit LQ images without altering their quality by applying GAN inversion and manipulation techniques. It also provides for free an image restoration solution that can handle various degra- dations. including noise, blur, compression artifacts, and their mixtures. Finally, we demonstrate numerous other applications such as image degradation synthesis, transfer, and interpolation.
Introduction
Image generation has achieved a marvelous development recent years thanks to the introduction of Generative Adversarial Networks (GAN). StyleGAN models [1
3 4 manage to generate realistic-looking images with the resolution up to 1024x 1024. Their synthetic images of non-existing people/objects can fool human eyes [5 6. The development of such high-quality synthesis models has also introduced a new direction to effectively solve the image manipulation tasks, which usually first fit the input image to the model's latent space via a GAN "inversion" technique [7] 8 9 10 12
2
then apply a learned editing 14 15, 16 on the fitted latent code for the desired change on the generated image. Their impressive manipulation results promise various practical applications in entertainment, design, art, and more.
While the recent GAN models, notably the StyleGAN series, show promising image editing perfor- mance, we argue that image quality is an Achilles' heel, making it challenging for them work on real-world images. StyleGAN models are often trained from sharp and high-resolution images as the desired output quality. In contrast, in-the-wild images might have various qualities depending on the capturing and storing conditions. Many degradations could affect these images, including noises, blur, downsampling, or compression artifacts. They make GAN inversion hard, sometimes impossible, to fit the low-quality inputs to the high high-quality image domain modelled by StyleGAN generators. Incorrect inversions might lead to unsatisfactory editing results with obvious content mismatches. For example, a popular BoyleGAN-baseed image super-resolution method [17] caused controversy by approximating a high-resolution picture of Barack Obama as an image a white man.
"authors contributed equally
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

QC-StyleGAN
StyleGAN
control quality
Inversion
control quality
Input
edit
edit
Ediling
Restoration
control quality
Figure 1: Our QC-StyleGAN allows for synthesizing sharp images, similar to the standard StyleGAN, and degraded images. It provides a quality-control input for easy conversion between degraded images and their sharp versions (dashed arrows). The same quality codes produce the same degradation (yellow arrows), and QC-StyleGAN covers a wide range of degradations (yellow vs. magenta). We can easily edit degraded images using the editing directions learned for sharp images in StyleGAN space (blue dotted arrows). Given low-quality input, QC-StyleGAN allows more accurate GAN inversion, direct image editing with quality preserved, and an efficient image restoration (red arrows).
One possible solution to narrow the quality gap is to train the StyleGAN generator on low- cow-quality images. Although this might improve GAN inversion accuracy, it can possibly fail to model the high-quality image distribution, which is the desired target of standard image generation and many image manipulation tasks. Training the generator on mixed quality data also does not help since the connection to translate between low-quality and high-quality images is missing.
This paper resolves the aforementioned problems by introducing a novel Quality Controllable StyleGAN structure, or QC-StyleGAN, which is simple yet effective architecture that can learn and represent a wide range of image degradations and encode them in controllable vector. We demonstrate our solution by modifying StyleGAN2-As any StyleGAN version. Based on the standard structure, we revise its fine-level layers to input quality code q defining the degradations on the model output. It can generate clean and sharp images similar to the standard StyleGAN counterpart when q and synthesize their degraded, low-quality versions by varying the value q. Our QC-StyleGAN covers popular degradations, including noise, blur, low-resolution and downsampling, JPEG compression artifacts, and their mixtures. has many desired properties. First, QC-StyleGAN can generate sharp and clean images with almost the same quality as the standard StyleGAN, thus preserving all of its utilities and applications. Second, can also model the degraded photos. bridging the gap to real-world images and providing more accurate GAN inversion. The image editing operators, therefore, can be applied on low-quality images in the same way for the high-quality ones without altering image quality. This functionality is particularly important when only a part of the data is manipulated, and the manipulated result must have consistent quality as the rest. One scenario is edit few frames in video. Another scenario is to edit an image crop of a big picture. Third, it allows easy conversion between low- and high-quality outputs, bringing in many applications below.
networks but it should be applicable to
One of our QC-StyleGAN's applications is GAN-based image restoration. Given a low-quality input image, we can fit it into QC-StyleGAN, then recover to the high-quality, sharp version by resetting the quality code to O. While there were several works that used the StyleGAN prior for image super-resolution [17 [8] or deblurring [19), QC-StyleGAN is the first method to solve the general image restoration task. Moreover, while the previous methods tried to fit low-quality data to the high-quality image space, leading to obvious content mismatches, our model maintains the content consistency by bridging the two data realms. This image restoration, while being a simple by-product of our QC-StyleGAN design, shows great potential in handling images with complex degradations.
Besides, by modeling a wide range of image degradations and encoding them in a controllable vector, QC-StyleGAN can be used to synthesize novel image degradations or interpolate between existing ones. It can easily capture the degradation from an input image, allowing degradation transfer. These techniques have various practical applications and will be demonstrated in Section 4

---

Fig. summarizes our proposed SCyStyleGAN, It covers not only the sharp, high-quality image domain of common StyleGAN models but also the degraded, low-quality image one. It bridges these image domains via quality control input. QC-StyleGAN allows better GAN inversion and image editing on low-quality inputs and introduces a potential image restoration method.
2 Related work
2.1 StyleGAN series
Since the seminal paper (20]. Generative Adversarial Networks (GANs) have achieved tremendous progress. Among them, a typical line of work called Style-based GANs (StyleGANs) has attracted much attention from the research community. These works allow us generate images at very high resolution while producing semantically disentangled and smooth latent space. In the initial version [. StyleGAN controls the style of synthesized images by proposing an intermediate disentangled latent space, named W space, mapped from the latent code via an MLP network. Then, they feed this code to the generator at each layer by employing the adaptive instance normalization module. In the next generation, StyleGAN2 [2] proposed a few changes in the network design and training components to further enhance the image quality. StyleGAN (3] allows model training with limited data by introducing the adaptive discriminator augmentation technique. StyleGAN3 4 tackles the aliasing artifact phenomenon in the previous versions and therefore helps to generate images entirely equivariant for rotation and translation. Recently, StyleGAN-XL [21] expanded the ability of the StyleGAN model to synthesize images on the ImageNet [22] dataset. It is worth noting that all of the current StyleGAN models have used the training dataset with high-quality images. Therefore, their outputs also are sharp images. In our work, we explore a new StyleGAN structure that allows to synthesize both high- and low-quality images with explicitly quality control.
2.2 Latent space traversal and GAN inversion
Aside from the ability to synthesize high-quality images, the latent space learned by GANs also encodes diverse set of interpretable semantics, making it an excellent tool for image manipulation. As a result, exploring and controlling the latent space of GANs has been the focus of numerous research works. Many studies 223 24 25 have tried to extract the editing directions from the latent space in a supervised manner by leveraging either a pre-trained attribute classifier or a set of attribute-annotaedd images. Mcanwhile, other works such as [5 26 27 28
have developed
the unsupervised methods for mining the latent space, which reveal many new interesting editing directions. To convey such benefits for editing real images, we first need obtain the latent code in the latent space so that we can accurately reconstruct the input image when we feed this code into the mre-trained generator. This line of work is called GAN inversion, which was first proposed by (2]] (2) 12 40 approaches.
Existing GAN inversion techniques can be grouped into (1) optimization bassd 30 31
ancoder-based [29 33 g TO and (3) two-stage 35
36 37 38
39
13
We recommend visiting the comprehensive survey 41 for a more in-depth review.
2.3 Image enhancement and restoration
Image enhancement and restoration are the task of increasing the quality of given degraded image. Formally, the degradation process can be generalized as:
y = H(x) +
(1)
where H is the degradation operator, v) is noise, x and y/ are the original and the degraded images, respectively. The goal is to find æ given y and probably some assumptions on H and 17. It can be divided into various sub-tasks based on the degraded operator and the noise such as image denoising (H I), image deblurring (H is blur operator), or image super-resolution (H a downsample operator). Existing methods usually focus on one of these sub-tasks instead of solving the general one. Image enhancement and restoration is well-studied yet still challenging field. the past, common methods make handcrafted priors on H. and 1) and use complicated optimization algorithms to solve methods have been proposed and achieved impressive results. These methods mainly differ by the network design data-driven approaches were observed be highly overfitted the training set and hence cannot be applied for real-world degraded images.
r 42 44 45). Recently, many deep-learning-basse
[46. 47 48 However,
3

---

To better restore in-the-wild images, recent works consider the task on specific domain, such as face [7 18 49 by leveraging existing generative models, such as StyleGAN. Unlike previous deep-based models, these methods always produce high-quality results even on real-world degraded images. However, they often produce clear mismatched image content when trying to fit the degraded input into the sharp image space.
3 Proposed method
In this section, we present our proposed QC-StyleGAN that supports quality-controllabe image generation and manipulation. We first define the QC-StyleGAN concept (Section then discuss its structure (Section 3.2) and training scheme (Section 3.3). Next, we discuss the technique to acquire precise inversion results (Section 3.4). Finally, we present its various applications (Section 3.5
3.1 Problem definition
Traditional image generators input random noise vector R.D: NO 7), with D, the number of input dimensions, and output the corresponding sharp image. Let us denote the baseline StyleGAN generator as Fo- The image generation process is / Fo(z). Furthermore, Fo consists of two components: mapping network Mo and a synthesis network Go:
Fo = Go o Mo.
T= Fo(2) = Go(Mo(2)) = 9o(w),
(2)
with w Mo(z) forming commonly used embedding space W.
Our proposed network, denoted as F, requires an extra quality code input, denoted as q E IR (00. I), with Dg as the number of quality-code dimensions. Its image generation process is I= F(2. q). We borrow the mapping function MM and design a new synthesis network for F::
F Moo
I F = G(Mo(=).g) = g(w,
The desired network should satisfy two requirements:
1. When is zero, F synthesizes sharp images similar to the standard StyleGAN:
F(z, 0) Fo(2).
(4)
2. When is nonzero, F synthesizes degraded version of the corresponding sharp image:
F(z, A,(o(z)),,
(5)
where Ag is some image degradation function with the parameter q. A,, is composition of multiple primitive degradation functions such as noise, blur, image compression, and more. It is defined by q and independent to the sharp image content Fo(z).
3.2 Network structure
The structure of QC-StyleGAN illustrated in Fig. 2a As mentioned, it consists of two sub-networks: the mapping network Mo borrowed from the standard StyleGAN and a synthesis one G. We will skip Mo and focus on the design of G. We build G from the standard synthesis network Go with minimal modifications to keep similar high-quality outputs when q = 0. It consists of N synthesis blocks, corresponding to different resolutions from coarse to fine. Since image degradations affect high-level details, we revise only the last M layers of G to input the quality code & and synthesize the corresponding degradation. We empirically found = 2 enough to cover common degradations.
For each revising synthesis block, we pick the output of the last convolution layer, often named conel, to revise by adding feature residuals conditioned on the quality input q. To do so, we introduce pnd novel network block, called DegradBlock, and plug it in as illustrated in Fig. 2a Let us denote the
input of convl as a feature map f E xxx with C as the number of channels and (H, W) as the spatial resolution. DegradBlock, denoted as function DB(,), inputs and E DR and outputs residual E C # with C' the number of output channels of conv1.
When 9 0, we expect to behave similar to Go. Hence, in that case, we enforce the network features unchanged, or there is no residual:
DB(f,0) =0 Vf.
(6)

---

2-100.1)
Quality code
Dg
Masppiin g notwork
DegradBlock
Residual
DegrasBBlok
Input feature
Block
F
Projection
Dog DegrodBlook
MI0.1)
Linear combinator
Convolution
RaLU
(a) Network structure
(b) DegradBlock
Figure 2: QC-StyleGAN structure
Also, when magnifying we expect the strength of image degradation, implied by the residual r. to increase. From those desired properties, we propose the structure of DegradBlock in Fig. 2b comprises the following components:
A convolution layer e with stride to change the number of channels to multiple of Dq- denoted as L Dg:
ri c(f) E RRD,,
2. A linear combinator & that splits the previous output into D, L-channel tensors {ri'), then computes their linear combination using the weights defined by q:
D.
= o(r)) = Σ TT RRLLHXW
(8)
1=1
3. A projection module π to refine r22 and change its number of channels to C²:
r= =(2) E CC XxX
stack of P convolution layers with stride To increase non-linearity, we put a ReLU activation after each convolution layer, except the last one. Note that when (Equation ( and also have 72 0 (according to Equation [8) It leads to (0) = 0. We ensure it by simply setting the convolution layers to have no bias.
We design
q O, expect
This design is inspired by PCA, unlike the common-used AdaIN blocks. We first e predict D, principal components r| then compute their linear combination with q as the component weights. This structure simple satisfies the mentioned properties. When g = 0, the residual is guaranteed to be O. When magnifying the residual increases accordingly (see the Appendix):
DB(F k q) k* DB(F.)
vk E
(10)
3.3 Network training
Next, discuss how train QC-StyleGAN. As mentioned, it differs from the standard StyleGAN only on the last two blocks of the synthesis network. Hence, we initiate our network from the pretrained StyleGAN weights and finetune only those two synthesis blocks.
Our QC-StyleGAN is trained in two modes corresponding to sharp (q = 0) and degraded (g * 0) image generation. We have sharp-image discriminator D, and degraded-image discriminator Da used in each mode. In the degraded image generation mode, we augment the sharp images by a random combination of primitive degradation functions (noise, blur, image compression) to get "real" low-quality images for training the discriminator.
For each mode, we train the networks with similar losses as in the standard StyleGAN training. However, the sharp image generation mode, we employ the standard StyleGAN Fo as the teacher model and apply knowledge distillation to further ensure similar sharp image outputs. We transfer
5

---

knowledge in the feature spaces instead of the output space for more efficient distillation. Let SKD denotes the of conv1 layers with added DegradBlocks, X and X denote features of layer 1 in SKD from the teacher and student networks. We define the extra distillation loss Edist as follows:
Ldist
Σ
|x - 12
(11)
LESKD
This distillation loss is added to the final loss with a
AKD.
3.4
Inversion process
After getting the QC-StyleGAN model, we next discuss how to fit any input image to its space. This process, called GAN Inversion, a critical step in many applications such as image editing. While the general objective is to reproduce the input image, different methods optimize different components of the image generation process. We follow the state-of-the-art technique named PTI (39 to optimize the w embedding and the generator G to acquire both precise reconstruction and high editability. Furthermore, we also need to optimize the newly proposed quality-control input q. Let us revise the denotation of the synthesis network G as Go with 0 as its weights. Our inversion task Ig estimates both the inputs (w, q) and lightly tunes 0 so that the reconstructed image close to the input:
IT 0o) = (w", , 0²) = argmin d(Go(w, (w, q), w.q.6
given that ||0 Poll < 6
(12)
where I is the input image, d(.) is a distance function, 8g is the network weights acquired from Section 3.3 and e is some threshold restricting the network weight change.
PTI 39 proposes a two-step inversion process. It first optimizes the embedding w using the initial model weights O (stage-1), 1), then keeps the optimized embedding and finetunes 0 (stage-2). We can adapt that process to QC-StyleGAN, with a small change to include q in optimization alongside w.
However, one extra requirement for this inversion, specific to QC-StyleGAN, is to have the sharp version of the reconstructed image, i.e., Go(w", 0). to be high-quality. We empirically found that the naive optimization processes in PTI fail to achieve that goal. One degraded image may correspond to different sharp images, e.g., in case of motion or low-resolution blur. PTI, while manages nicely fit the degraded input, often picks non-optimal embeddings that produce distorted corresponding sharp images. Hence, we replace its stage-1 with training- based approach, following pSp (9, with extra supervision on the sharp image domain. In this revised stage-1, we train two encoders to regress the embedding w and the quality-code q separately. Also, we load both the degraded and the corresponding sharp images for training and apply reconstruction losses on both image versions.
3.5 Applications
Image editing. The most intriguing application of StyleGAN models is to manipulate real-world images with realistic attribute changes. They can do that by applying learned editing directions in some embedding space, e.g., the W space. However, these standard models can only do the editing in sharp image domains. Our QC-StyleGAN inherits the editing ability of StyleGAN by using the same network weights except for the last M synthesis blocks, which mainly affect the fine output details. However, QC-StyleGAN covers both low- and high-quality images, broadening the application domains. It can directly apply the learned editing directions of StyleGAN images and keep their degradations unchanged. Specifically, given an input and a target editing direction Aw learned from the standard StyleGAN in the W space, can first invert the image (w,q, q. 0) = [T, (T,0g), then generate the manipulated result I' = Go + Aw, q)-
low- low-quality
Image restoration. This is a new functionality, which is a by-product of QC-StyleGAN design. Given a low-quality image I, can fit it to QC-StyleGAN (w, q, 0) [I 0o), then acquire its sharp, high-qualin version by clearing the quality code: I' = Go (w, 0).
Degradation synthesis. QC-StyleGAN defines the degradation on the output image by quality code q. It allows to revise the image degradation in various ways, such as (1) sampling a novel degradation, (2) transferring from another image, and (3) interpolating new degradation from two reference ones.

---

Table FID scores of our QC-StyleGAN models, in comparison with the baseline StyleGAN2-Add (dG2-Ada) [3], on sharp and degraded image generation modes. means a new, separate SG2-Ada
model trained on degraded images.
FFHQ (256 x 256)
AFHQ Cat (512x512)
LSUN Church (256 (256x256)
FID
SG2-Ada
Ours
SG2-Ada
Ours
SG2-Ada
Ours
Sharp
3.48
3.65
3.55
3.56
3.86
3.61
Degraded
4.38*
3.23
4.70*
3.91
5.16*
4.58
Figure 3: Sample images generated by models FFHQ (left), AFHQ-Cat (middle), and I-SNN- Church (right). For each sample, we provide a pair of sharp (top) and degraded (bottom) images.
Experiments
4.1 Experimental setup
Datasets. We conduct experiments on the common datasets used by StyleGAN, including FFHQ, AFHQ-Cat, and LSUNN-Curch. FFHQ [ is large dataset of 70k high-quality facial images collected from Flickr, introduced since the first StyleGAN paper. We will use the FFHQ images with the resolution 256 256. AFHQ (50] is a HQ dataset for animal faces with image resolution 512 x 512. We demonstrate our method using its Cat subset with about 5000 images. Finally, LSUN-Church is subset of the LSUN [5T] collection. has about 126k images of complex natural scenes of church buildings at the resolution 256 256. We will use LSUN- only for image generation since its inversion results even on sharp image domain are not satisfactory
Synthesis network. We use StyleGAN2-Ada as reference to implement our QC-StyleGAN. The quality code has size Dg 16. In DegradBlock, we use L 32 and P 3. The weight for the distillation loss AKD = Our networks were trained using the same settings as the original work until converged. Details of this training process will be provided in the Appendix.
4.2 Image generation
We compare the quality of our QC-StyleGAN models with their StyleGAN2-Add counterparts in Table using the FID metric. As be seen, models have equivalent results the baselines when generating sharp images. However, while the common StyleGAN2 degraded images, can generate such images directly with good FID scores. Even when training new StyleGAN2-Ada models dedicated on degraded images, their FID scores are worse than ours.
models cannot produce
Fig. 3 provides some samples synthesized by our networks. For each data sample, we synthesize degraded image and the corresponding sharp version. As can be seen, the sharp images look realistic, matching the standard StyleGAN's quality. The degraded images match the sharp ones in content, and the degradations are diverse, covering noise, blur, compression artifacts, and their mixtures.
4.3 GAN inversion and Image editing
We turn to evaluate the effectiveness of our proposed GAN inversion technique (Section 3.4 and image editing (Section on low low-quality image inputs.
test set for evaluation. With the models trained on AFHQ-Cat, we employ its corresponding test set for testing. For each test set, apply different image degradations to the original images to obtain the low-quality
With the model trained on the FFHQ dataset, we use the CelebA-HQ 52 53
7

---

Input
Sharp
Our inversion
Direct editing
SG2-Ada
FFHQ
Cat
Figure 4: GAN Inversion and Image editing, From degraded input image (1 col.), we can fit it into our model get similar reconstructed degraded image (3 col.), which is more accurate than from StyleGAN2-Ada (last col.). The corresponding sharp image (4th col.) is close to the real one (2nd col.). We can apply image editing directly on the degraded image (5th col.) and its sharp version (6th col.) matches the change. From top bottom we apply gender, age, and color change.
images. Our QC-StyleGAN model can fit well to such degraded inputs with the average PSNR of reconstructed images as 29.47dB and 28.91dB for CelebA-HQ and AFHQ Cat, respectively. We also try to apply the image editing directions learned for StyleGAN2-Ada by InterfaceGAN [14) (face) and SeFa (28) (cat) to manipulate the degraded images with QC-StyleGAN. The qualitative results in also do GAN inversion with the StyleGAN2-Ade model on these inputs, its inversions cannot be converted to sharp. Also, direct manipulation StyleGAN2-Ad''s inversions may introduce unrealistic artifacts
Fig. cconfimm the effectiveness of such direct image editing scheme. Note that while
(see the Appendix), making StyleGAN inferior to QC-StyleGAN in handling low-quality inputs.
4.4 Image restoration
As mentioned in Sec. 3.5 a nice by-product of our QC-StyleGAN is a simple but effective image restoration technique. We examine it on the degraded CelebA-HQ images five common restoration tracks, including deblurring, resoutrcsol denoising, JPEG removal, and multiple-degraded restoration. We also compare it with the state-of-the-es image restoration methods. For image restora- tion networks such as NAFNet (54) and MPRNet [55], we re-train the models on our degraded images using their published code with default configuration. For GAN-based methods like HiFaceGAN [56 and PULSE
17].
we use their provided pre-trained models. The results are reported in Table 2
Among the common metrics for this task, find LPIPS more reliable and close human perception. Although our method is not tailored to handle this restoration task specifically, it performs reasonably well and outperforms many baseline methods in each task. Particularly, QC-StyleGAN provides the best LPIPS score when having multiple degradations the input. Also, we find that one degraded image may correspond multiple possible sharp images. Our restoration results sometimes look reasonable but do not match the ground-trus -tuth, severely hurting our LPIPS scores.
To avoid the mismatching ground-truth issue, we also use NIQE (57). which is no-reference image quality metric. QC-StyleGAN provides the best NIQE score in nearly all tracks. It confirms that image restoration can produce the highest output quality in terms of naturalness [57] while still maintaining comparable perceptual similarity [58] compared to the competitors.
Fig. 5 provides restoration results from our method and the image restoration baselines on two extremely degraded images. Our algorithm manages to return sharp and detailed images, while the others fail to handle such images and show clear artifacts their recovered images.

---

Track
Blur
Super-res.
Noise
JPEG comp.
Multiple-deg.
HiFaceGAN [66
5.95 / 0.216
5.32 0.125
6.01 / /0.126
4.95
0.053
5.916 /.336
ESRGAN 55
6.35 0.148
DrCNN 47
6.93 / 0.080
MPRNet 55
6.12/01194
6.73 0.230
7.33 0.143
7.64 0..28
8.97 0.299
PULSE 17
6.29 0.296
mGANPrior 60
6.02 0.265
GLEAN 18
7.29 0.072
Ours
5.83 0.195
5,45 0.177 5.41 0.183 4.51 0.118 5.64 3.64/0.20
Table 2: NIQE [57 and LPIPS [88] scores of image restoration methods five restoration tracks on the CelebA-Ht --HQ dataset. For both metrics, lower value means better. The best and runner-up values are marked in bold and underline, respectively. The mark '-' means the method is not applicable.
Input
Sharp
HiFace
MPRNet
NAFNet
Ours
Figure 5: Comparison between our method and image restoration baselines on CelebA-HQ dataset.
4.5 Degradation synthesis
We provide an example of our proposed image degradation synthesis (Section 3.5 in Fig. From source image with JPEG compression artifacts, we change its image degradation to novel random one (blur, 2nd col.) or copy the degradation from reference image (noise, 6th col.). We can also smoothly interpolate in-between degradations, using an interpolation factor E [0, 1] (3 - 5th col.).
4.6
Quality-code-basse Degradation Classification
To verify the quality of QC-StyleGAN in degradation estimation, conduct experiments of training linear classifiers to predict whether an image is blurry, or at which blur level, based on its quality code For each experiment, we use 1000 facial images for training and 200 images for testing. When we train the classifier to detect if an image is blurry, the accuracy is 97.9%. When we divide the degree of blurring into 5 levels for the classifier predict, the accuracy is 85%. When use 10 blur levels, the accuracy still high (77%), confirming QC-StyleGAN as a quality degradation estimator.
4.7 Stability of Inversion and Editing
We conduct experiments to verify the stability our inversion and editing on degraded CelebA-HO inputs images. We tried the editing magnitudes +3 for 3 editing tasks gender, age, and smiling. For each input image and each editing operator, we executed the operator times to get 3 manipulated outputs and compare them pairwise using the PSNR and LPIPS metrics. The manipulated degraded images are pretty similar, with the PSNR score 44.42 + 2.71 and the LPIPS score 0.004 = 0.003. When recovering the sharp version of these images, the PSNR score is still high (41.84 + 2.07) and the LPIPS score is still very good (0.006 = 0.005).
We generated some quantitative videos and provided them in this link For each video, we show the same manipulation runs. can be seen, our manipulated results are quite stable, with minor flickers mainly appearing the background the hair region. If we mask out the background

---

0.25
0.5
0.75
Source
Random
Interpolated
Transferred
Reference
Figure 6: Degradation synthesis
and keep only the face region (the hair region is still kept), the scores in the previous experiments get improved: For degraded images, the PSNR score is 47.18 + 2.27, and the LPIPS score is 0.0013 + 0.0006. For sharp-recovered images, the PSNR score 43.67 2.28, and the I.PIPS score 0.0028 + 0.0013. It confirms that inversion and editing pretty stable the target object.
4.8 Ablation studies
In this section, we investigate the effectiveness of our model design on the FFHQ dataset. First, we try other ways to inject into the network, e.g., concatenating with the latent input 2, the embedding w, or via DualStyleGAN structure [6]], but find clear mismatches in image content generated sharp and degraded images of the same code w. Second, we try our network training without the distillation loss, and the FID-sharp very high at 11.32. Third, we try a AdalN-style design for the DlgradBlocks, and its FID-sharp is 4.41. If apply DegradBlock at the last synthesis block (M = 1), the FID-sharp is 6.38. If we reduce D, to 8, that FID-sharp increases (4.54) even if we expand L to 64 for similar computation cost. Extra discussions will be included in the Appendix.
4.9 Inference time
In this section, report the running time of the proposed models in Table For the restoration tasks, we report the running time of the two components of the method, including pSp (stage-1) and PTI optimization (stage-2).
Table 3: Average running times of our methods on two tasks, including image generation and image inversion (pSp and PTI optimization). Note that the performance of pSp the both resolutions because it resizes the input to 256 x 256 before running.
Time (s)
Model
256 256
512 x 512
QC-StyleGAN
0.08 + 0.01
0.20 + 0.04
pSp PTI-opt
0.13 0.01
0.11 = 0,01
83.40 - 2.68
222.29 + 23.24
5 Conclusions and future work
This paper presents QC-StyleGAN, a novel image generation structure with quality-controlled output. It inherits the capabilities of the standard StyleGAN but extends cover both high- and low-quality image domains. QC-StyleGAN allows direct manipulation on in-the-wild, low-quality inputs without quality changes. It offers novel functionalities, including image restoration and degradation synthesis.
Limitations. Although we employed many image degradations in training QC-StyleGAN, they might not cover all in-the-wild degradations. Also, we only implemented QC-StyleGAN from StyleGAN2-Ada We plan to improve on these aspects in the future.
Potential negative societal impacts. Our work may have negative societal impacts by allowing better image manipulation on in-the-wild data. However, image forensies an active research domain to neutralize that threat. We believe our work's 's benefits dominate its potential risks.
10

---

References
[1] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. CVPR, 2019.
[2] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of StyleGAN. In CVPR, 2020.
[3] Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In NeurIPS, 2020.
[4] Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. In NeurlPS, 2021.
[5] This X Does Not Exist. https://thhaxxdeennttrr..tt
[Online].
[6] Which Face Real? https://ww..hhchhffeeee
com
[Online].
[7] Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2stylegan: How to embed images into the stylegan latent space? In ICCV, 2019.
[8] Jiapeng Zhu, Yujun Shen, Deli Zhao, and Bolci Zhou, In-domain gan inversion for real image editing. In ECCV, 2020.
[9] Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Os Encoding in style: stylegan encoder for image-to- to-mmae translation. In CVPR, 2021.
[10] Omer Tov, Yuval Alaluf, Yotam Nitzan, Or Patashnik, and Daniel Cohen-Or. Designing encoder for stylegan image manipulation. TOG, 2021.
[11] Yuval Alaluf, Or Patashnik, and Daniel Cohen-Or. Restyle: residual- bastd stylegan encoder via iterative refinement. In ICCV, 2021.
[12] Yuval Alaluf, Omer Tov, Ron Mokady, Rinon Gal, and Amit H. Bermano. Hyperstyle: Stylegan inversion with hyperetwwrrs for real image editing, 2021.
[13] Tan M. Dinh, Anh Tuan Tran, Rang Nguyen, and Binh-Son Hua. Hyperinverter: Improving stylegan inversion via hypemnetorrk. In CVPR, 2022.
[14] Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. Interfacegan: Interpreting the disentangled facee representation learned by gans. IEEE transactions on pattern analysis and machine intelligence, 2020.
[15] Erik Härkönen, Aaron Hertzmann, Jaakko Lehtinen, and Sylvain Paris. Ganspace: Discovering interpretable gan controls. Advances in Neural Information Processing Systems, 33:9841-9888 2020.
[16] Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. Styleclip: Text-driven manipulation of stylegan imagery. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2085-2094, 2021."
[17] Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin. Pulse: Self-supervised photo upsampling via latent space exploration of generative models. In Proceedings of the ieeelcyf conference on computer vision and pattern recognition, pages 2437 2245,24455220 2020.
[18] Kelvin CK Chan, Xintao Wang, Xiangyu Xu, Jinwei Gu, and Chen Change Loy. Glean: Generative latent bank for large-factor image super-resolution. In Proceedings of the IEEF/CVP Conference on Computer Vision and Pattern Recognition, pages 14245-14254, 2021.
[19] Phong Tran, Anh Tran, Quynh Phung, and Minh Hoai. Explore image deblurring via encoded blur kernel space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[20] Ian Goodfellow. Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NewrlPS, 2014.
[21] Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. volume abs/2201.000 000233, 2022.
[22] Alex Krizhevsky, 1lya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. NewrIPS, 2012.

---

[23] Lore Goetschalckx, Alex Andonian, Aude Oliva, and Phillip Isola. Ganalyze: Toward visual definitions of cognitive image properties. In ICCV, 2019.
[24] Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou, Interpreting the latent space of gans for semantic face editing. In CVPR, 2020.
[25] Huiting Yang, Liangyu Chai, Qiang Wen, Shuang Zhao, Zixun Sun, and Shengfeng Hc. Discovering interpretable latent space directions of gans beyond binary attributes. In CVPR, 2021.
[26] Andrey Voynov and Artem Babenko. Unsupervised discovery of interpretable directions in the gan latent space. In ICML, 2020.
[27] Binxu Wang and Carlos R Ponce. A geometric amalysis of deep generative image models and its applications. In ICLR, 2020.
[28] Yujun Shen and Bolci Zhou. Closed-form factorization of latent semantics in gans. In CVPR, 2021.
[29] Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A Efros. Generative visual manipulation on the natural image manifold. In ECCV, 2016.
[30] Antonia Creswell and Anil Anthony Bharath. Inverting the generator of generative adversarial network. Transactions on Neural Networks and Learning Systems, 2018.
[31] Fangchang Ma, Ulas Ayaz, and Sertac Karaman, Invertibility of convolutional generative networks from partial measurements. 2019.
[32] Kyoungkook Kang, Seongtae Kim, and Sunghyun Cho. Gan inversion for out-of-range images with geometric transformations. In ICCV, 2021.
[33] Stanislav Pidhorskyi, Donald Adjeroh, and Gianfranco Doretto, Adversarial latent autoencoders. In CVPR, 2020.
[34] Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Weiming Zhang, Lu Yuan, Gang Hua, and Nenghai Yu. A simple baseline for stylegan inversion. arXiv preprint arXiv:2/04.0766 2021.
[35] David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles. Hendrik Strobelt, Bolei Zhou, and Antonio Torralba. Seeing what a gan cannot generate. In ICCV, 2019.
[36] Shanyan Guan, Ying Tai, Bingbing Ni, Feida Zhu, Feiyue Huang, and Xiaokang Yang. Collaborative learning for faster stylegan embedding. arXiv preprint 2rX1v:2007.01758, 2020.
[37] David Bau, Hendrik Strobelt, William Peebles, Jonas Wulff, Bolci Zhou, Jun- Yan Zhu, and Antonio Torralba. Semantic photo manipulation with generative image prior. TOG, 2019.
[38] Ayush Tewari, Mohamed Elgharib, Florian Bernard, Hans- Peter Seidel, Patrick Pérez, Michael Zollhöfer. and Christian Theobalt. Pie: Portrait image embedding for semantic control. TOG, 2020.
[39] Daniel Roich, Ron Mokady, Amit I Bermano, and Daniel Cohen-Or. Pivotal tuning for latent-based editing of real images. arXiv preprint arXiv:2106.05744, 2021.
[40] Tengfei Wang, Yong Zhang, Yanbo Fan, Jue Wang, and Qifeng Chen. High-fidelity gan inversion for image attrihute editing. In CVPR, 2022.
[41] Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Har Xue, Bolei Zhou, and Ming Hsuan Yang. Gan inversion: A survey, arXiv preprint arXiv: 2101.05278, 2021.
[42] Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior. IEEE transactions on pattern analysis and machine intelligence, 33(12):2341-23 2010.
[43] Jinshan Pan, Deqing Sun, Hanspeter Pfister, and Ming-Hsuan Yang. Blind image deblurring using dark channel prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1628-1636, 2016.
[44] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Transactions on image processing, 1668):2000-2095 2007.
[45] RY Tsai. Multiple frame image restoration and registration. Advances in Computer Vision and Image Processing, 1:1715 -1719-1989, 1989.
12

---

[46] Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence, 38(2):295-307. 2015.
[47] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei 7hang. Beyond gaussian denoiser: Residual learning of deep enn for image denoising. IEEE maasactions on image processing. 26(7):3142- 3155, 2017.
[48] Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, and Jiaya Jia. Scale-recurren network for deep image deblurring. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8174-8182, 2018.
[49] Phong Tran, Anh Tuan Tran, Quynh Phung, and Minh Hoai. Explore image deblurring via encoded blur kernel space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11956 11956-11966 11961,1912220 2021.
[50] Yunjey Choi, Youngjung Uh. Jaejun Yoo, and Jung-Woo Ha. Stargan v2: multiple domains. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition. 2020.
Diverse image synthesis for
[51] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Con- struction a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: 15..615666.336,,. 2015.
[52] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015.
[53] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. In ICLR, 2018.
[54] Liangyu Chen, Xiaojic Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. arXiv preprint NDV:2204.04676 2022.
[55] Syed Waqas 7amir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang. and Ling Shao. Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 14821-14831, 2021.
[56] Lingho Yang, Shanshe Wang. Siwei Ma, Wen Gao, Chang Liu, Pan Wang, and Peiran Ren. Hifacegan: Face renovation via collahorative suppression and replenishment. In Proceedings of the 28th ACM International Conference on Multimedia, pages 1551-, 2020.
[57] Anish Mittal, Rajiv Soundararajan, and Alan C Bovik. Making "completely blind" image quality analyzer. IEEE Signal processing letters, 20(3):209 212, 2012.
[58] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable elfectiveness of deep features as perceptual metric. In CVPR, 2018
[59] Xintao Wang, Lianghin Xie, Chao Dong, and Ying Shan. Real-esrgan: Training real-world blind super- resolution with pure synthetic data. In International Conference on Computer Vision Workshops (ICCVW).
[60] Jinjin Gu, Yujun Shen, and Bolei Zhou, Image processing using multi-code gan prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3012 22222, 2020.
[61] Shuai Yang, Liming Jiang, Ziwei Liu, and Chen Change Loy. Pastiche master: Exemplar-base based high- resolution portrait style transfer. In CVPR, 2022.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See our last section.
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See our last section.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
13

---

2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Our paper is more experimental side. There is only theoretical result, which Equation To and we provide its full set of assumptions.
(b) Did you include complete proofs of all theoretical results? [Yes] Our paper is more on experimental side. There only one theoretical result, which is Equation 10 and we provide full proof in the Appendix.
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We specify any change from the standard training.
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See our Appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes] All of them are public datasets and code for research purposes.
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you' 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "yR-X3hLd1vP", "reviewer": "Reviewer_H13T", "summary": "This paper targeted a very interesting problem. When we use a GAN-inversion method to edit images, the GAN can only generate sharp images, and what if we got a degraded image and we don't want its degradation to be distorted? This paper gives a method to solve this problem. In addition to an original StyleGAN network, this paper proposes a DegradBlock to add degradation that is controlled by an additional degradation quality code. When a new image comes, the method not only inverts the sharp image but also inverts its degradation. The results show good performance and several applications are studied.\n", "questions": "I hope the author responds to the weaknesses I raised above and the following additional questions:\n\n1. Estimating the degradation of an image is a difficult task. From the examples shown, most images are badly degraded. Will the proposed method perform well on small degradations? such as small noise, blurring with small sigma, and jpeg with a quality score of 70?\n2. How good is it as a degradation estimator? Can we estimate degradation using the proposed method?\n3. Are there big differences between the two inversion results?\n4. How good is the restoration performance? i.e. compare with GLEAN [1] or mGANprior [2] (BTW these works should be cited)\n\n\n=================update:\nAfter the reviewer-author discussion, I am willing to increase my rating from 4 to 5.\n", "limitations": "This work is related to face images but I guess there is no obvious potential negative societal impact.\nThe limitation is not discussed. ", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The question of \"what if we got a degraded image\" is interesting.\n2. It is the first GAN method that can be controlled to generate degraded images\n3. Nice performance.\n4. A Generator that knows different degradation is a good idea for blind image restoration.\n\nWeaknesses:\n1. Although interesting, the practice value of the studied problem is not that obvious. Given the mentioned setting, why would people want a degraded image? Maybe the restoration task is more valuable.\n2. The design of the DegradBlock lacks novelty. It is like an AddIn controlled by the quality code and with some conv layers. The direct product operation makes it give 0 when the quality code is 0. This method doesn't give us much insight technically. Although I comment on the novelty, it is the last reason that affects my rating. \n3. The proposed method can be used as a blind Face Restoration method, but the provided experiments are not enough to show the actual potential.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "IRYOEH6_YGq", "reviewer": "Reviewer_mDFp", "summary": "The introduction of high-quality image generation models, particularly the Style-GAN family, provides a powerful tool to synthesize and manipulate images. However, existing models are built upon high-quality (HQ) data as desired outputs, making them unfit for in-the-wild low-quality (LQ) images, which are common inputs for manipulation. In this work, the authors bridge this gap by proposing a novel GAN structure that allows for generating images with controllable quality. The network can synthesize various image degradation and restore the sharp image via a quality control code. The proposed QC-StyleGAN can directly edit LQ images without altering their quality by applying GAN inversion and manipulation techniques", "questions": "The reviewer is not very familiar with the scope of this paper. There are no more questions.", "limitations": "The reviewer is not very familiar with the scope of this paper. The reviewer has no idea of the limitations.", "rating": 7, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The proposed QC-StyleGAN is novel and can generate both clean and degraded images with a quality-control input. \nThe QC-StyleGAN allows more accurate GAN inversion on low-quality images, making image manipulation applicable on these inputs. It provides an efficient GAN-based image restoration solution. QC-StyleGAN also supports novel image degradation synthesis tasks.\n\nThe reviewer think that the proposed method is novel and recommend to accept the paper.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "5bxLIeNsfq", "reviewer": "Reviewer_EPGp", "summary": "StyleGAN is able to generate high-resolution images and produce semantically disentangled and smooth latent space at the same time. This paper proposed an extension of StyleGAN that is able to control the quality of the generated images by giving a controllable vector. This extension not only allows to generate degraded images but also brings some interesting applications such as image editing and image restoration by applying GAN inversion and manipulation techniques. Although there are not many quantitative comparisons, qualitative results show impressive effectiveness.", "questions": "1. I didn't find any main text mentioned the Figure 1.\n2. I noticed that in Table 1, the FID score of LSUN Church of the degraded image generation model is relatively higher than others. Do you have any explanation for it?", "limitations": "As mentioned in my second question, the proposed method may not perform well in some cases. It is important to investigate the reason and it should be discussed in the paper. It will be beneficial to the readers and may help readers to conduct further research in the future.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe idea is easy to understand and I think maybe the concept can be extended to different research. The qualitative results show impressive effectiveness.\n\nWeaknesses:\nThere is no baseline method of the degraded image generation models for comparison. Although the qualitative results are good, it would be more convincing if there are more quantitative comparisons.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "46aOl6V1-Yl", "reviewer": "Reviewer_NzZ7", "summary": "This paper aims to solve the problem that existing GAN methods are unfit for low-quality input images. The authors present a quality controllable image generation and manipulation method. The proposed method can be applied to many applications, like image restoration and image manipulation. ", "questions": "1. Why this problem is important. Can existing inversion methods [7-13] solve this problem? Can the proposed method achieve better results than these inversion methods?\n\n2. The authors use the term “a quality-control input” in Section 1, but explain it in Section 3. \n\n3. There are so many arrows in Fig. 1, resulting in a confusing figure. Moreover, Fig. 1 does not appear in the text part of the paper. \n\n4. My biggest concern for this paper is the validation part. In the experiments section, the authors show the results for image generation, image inversion, image editing, image restoration, image degradation, etc. However, for any application, only some visual results are shown, and no statistical performance values are reported. The details that use other methods to compute the results given in the paper are also missing. Are these methods fine-tuned using the corresponding dataset?\n\n5. As shown in Table 1. the proposed method achieves a lower FID score for the degraded images than the sharp images on the FFHQ dataset, which is inconsistent with the results on the other datasets. Can the authors explain why?\n", "limitations": "The limitations are discussed.", "rating": 4, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe proposed method is applied to many applications, like image restoration and image editing.\n\nWeaknesses:\nThe writing of this paper needs to be improved. The paper is hard to follow. For example, there is no overview of the proposed method. Although there is an overview subsection, it is more like problem formulation. An overview in plain English and an illustration or framework are necessary before giving any formula.\nThe motivation, novelty, and contributions are not well described.\nThe method is not well-validated, and no extensive comparison results are given.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept