# Internet Video Delivery Improved by Super-Resolution with GAN
## 2 Background and Related Work
### Super-Resolution Using Convolutional Networks  
Image and video super-resolution has gained momentum in different research communities. In recent years, the number of papers published that address the super-resolution problem and appear in qualified conferences and journals has shown an upward trend [26; 27]. Combining CNNs, perceptual loss functions, and adversarial training to solve typical SR challenges has fostered this momentum. Video SR refers to scaling low-resolution frames \(f^{LR}\in V^{LR}\) into high-resolution frames \(f^{SR}\in V^{SR}\) of video \(V_{f}\), where \(f=1,\cdots,N\), and each frame \(f\) has dimensions \(W:H:C\), defined by width \(W\), height \(H\), and RGB channel \(C\). \(f^{LR}\) are low-resolution frames taken from high-resolution frames \(f^{HR}\in V^{HR}\) using a downscaling process defined by a factor \(r\). In the following, we present super-resolution models built to upscale images in the context defined by video. In [28; 29; 30], the authors approached the SISR problem using a CNN. The proposed model, called SRCNN, upscales an image using a two-step procedure. The first step scales the low-resolution frame \(f^{LR}\) by a bicubic interpolation function to produce \(f^{Y}\), which has the target resolution. The second step uses \(f^{Y}\) to evolve the super-resolution frame \(f^{SR}\) to reach the quality measured in the original frame \(f^{HR}\). A three-layer CNN performs this quality-evolving process. The main drawback of the SRCNN model is the computational cost associated with the convolution process using an already-high-resolution frame \(f^{Y}\). Despite the smoothing artifacts, which are easily detected by the human vision system, the SRCNN model showed high scores in evaluations using pixel-wise metrics, such as PSNR and SSIM [29]. The model CISRDCNN [31] is a deep convolutional neural-network-based super-resolution algorithm for compressed images. The model is composed of three main blocks. The first part receives low-resolution and compressed images as input, and this part operates to reduce compression artifacts. The second part performs the upscaling operation, and the last part is responsible for quality enhancement and works on the HR image with a considerable computational cost. In addition, the output images show smoothing artifacts. In [32], the authors introduced the ESPCN model to face constraints due to high-resolution input frames \(f^{HR}\). The ESPCN model is a three-layer CNN that performs image up-scaling after the CNN's third layer in an additional subpixel convolutional layer. Image up-scaling deconvolutes the \(n\) layers of features extracted from the low-resolution frame \(f^{LR}\). Compared to the SRCNN model, the ESPCN has lower computational demands and similar image quality, although smoothing remains an unresolved issue. [PERSON] et al. [33] trained a CNN using a perceptual loss function defined using high-level image features for smoothing the image restoration. The authors use a pre-trained convolutional network to extract those features. [PERSON] et al. [21] proposed the SRResNet and SRGAN models, which are CNN-based models with 16 RBs. However, the SRGAN model has a perceptual loss function and uses adversarial training. The evaluations of the SRGAN model showed that the perceptual quality of up-scaled images was better than those models based on mean-error loss functions. Its primary drawback is the training instability, which occasionally leads to image artifacts due to BN, a deep-learning training technique that is broadly applied to the image classification task. [PERSON] et al. [22] proposed the ESRGAN model. It has residual scaling, 23 RRDBs, and a RaD. The latter measures the probability that the generated data are more realistic than the actual data and vice versa. This model helps build fast and stable training sessions and improves the images' perceptual quality. The ESRGAN model is the state-of-the-art SISR technique.