# BFRVSR: A Bidirectional Frame Recurrent Method for Video Super-Resolution
## 2 Related Work  
With the rise of deep learning, computer vision, including image super-resolution and video security [7; 8; 9], have received significant attention from the research community over the past few years.  
Image super-resolution (ISR) is a classic ill-posed problem. To be specific, in most cases, there are several possible output images corresponding to one given input image, thus, the problem can be seen as a task of selecting the most appropriate one from all the possible outputs. The methods are divided into interpolation methods such as nearest, bilinear, bicubic, and dictionary learning [10; 11]; example-based methods [12; 13; 14; 15; 16]; and self-similarity approaches [17; 18; 19; 20]. We refer the reader to three review documents [21; 22; 23] for extensive overviews of prior work up to recent years.  
The recent progress in deep learning, especially in convolutional neural networks, has shaken up the field of ISR. Single image super-resolution (SISR) and video super-resolution are two categories based on ISR.  
SISR uses a single low-resolution image to estimate a high-resolution image. [PERSON] et al. [24] introduced deep learning into the field of super-resolution. They imitated the classic super-resolution solution method and proposed three steps, i.e., feature extraction, feature fusion, and feature reconstruction, to complete the SISR. Then, [PERSON] et al. [25] reached state-of-the-art results with deep CNN networks. A large number of excellent results have emerged [26; 27; 28; 29; 30]. In addition, the loss function also determines the result of image super-resolution, thus, some parallel efforts have studied the loss function [31; 32; 33].  
Video super-resolution combines information from multiple low-resolution (LR) frames to reconstruct a single high-resolution frame. The sliding window method and recurrent method are two of the latest state-of-the-art methods.  
The sliding window method divides the video super-resolution task into multiple independent subtasks, and each subtask generates a single high-resolution output frame from multiple low-resolution input frames [4; 34; 35; 36]. The input is adjacent \(2N+1\) frames of low-resolution images like \(\left\{l_{t-N}^{LR},l_{t-N+1}^{LR}\dots,l_{t}^{LR},\dots l_{t+N-1}^{LR},l_{t+N} ^{LR}\right\}\). Then, an alignment module is used to align \(\left\{l_{t-N^{\prime}}^{LR},l_{t-N+1}^{LR}\dots l_{t+N-1}^{LR},l_{t+N}^{LR}\right\}\) with the \(l_{t}^{LR}\). Finally, \(l_{t}^{HR}\) is estimated through the aligned \(2N+1\) low-resolution frames. [PERSON] and [PERSON] et al. [29] used the optical flow method to align \(l_{t-1}^{LR}\) and \(l_{t+1}^{LR}\) with \(l_{t}^{LR}\) and used them to estimate \(l_{t}^{HR}\).  
The recurrent method generates a high-resolution image from the previous high-resolution image, the previous low-resolution image, and the low-resolution image. [PERSON] et al. [37] used a bidirectional recurrent architecture but did not use any explicit motion compensation in their model. Recurrent structures are also used for other tasks, such as blurring [38] and stylization [39; 40] of videos. [PERSON] et al. [38] and [PERSON] et al. [39] passed the feature representation to the next step, and [PERSON] et al. [40] passed the previous output frame to the next step, generating time-consistent stylizations in parallel work video. [PERSON] et al. [6] proposed a recursive algorithm for video super-resolution. The FRVSR [6] network estimates the optical flow \(F_{t-t-1}^{LR}\) of \(l_{t-1}^{LR}\) and \(l_{t}^{LR}\), and uses \(l_{t-1}^{HR}\). And \(F_{t-t-1}^{LR}\) to generate \(\overline{l}_{t}^{HR}\), and finally, sends \(\overline{l}_{t}^{HR}\) and \(l_{t}^{LR}\) to the network for reconstruction to obtain \(l_{t}^{HR}\). However, insufficient use of information caused by FRVSR leads to the correlation between image quality and time.