# BFRVSR: A Bidirectional Frame Recurrent Method for Video Super-Resolution  
[PERSON]  
1 School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China; [EMAIL_ADDRESS] (X.X.); [EMAIL_ADDRESS] (W.T.) Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China; [EMAIL_ADDRESS] (Z.H.); [EMAIL_ADDRESS] (M.L.)  
2  
[PERSON]  
2  
[PERSON]  
1 School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China; [EMAIL_ADDRESS] (X.X.); [EMAIL_ADDRESS] (W.T.) Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China; [EMAIL_ADDRESS] (Z.H.); [EMAIL_ADDRESS] (M.L.)  
2  
[PERSON]  
2  
[PERSON]  
2  
Footnote 2: email: 2020-0001-6551-  
high-resolution image, the previous low-resolution image, and the current low-resolution image [5; 6]. Each input frame is processed once. The RVSR method is able to process video sequences of any length and enables the details of the video to be implicitly transmitted in longer video sequences. Insufficient use of information caused by the RVSR method leads to a correlation between image quality and time (as shown in Figure 1).  
In short, video super-resolution methods still have the following problems: (a) The super-resolution network that uses the sliding window method has a high computational cost. Each frame of the image needs to be calculated \(2N+1\) times (window size \(2N+1\)). (b) Direct access to the output of the previous frame helps the network generate a temporally consistent estimate for the next frame or previous frame. In the recurrent network, insufficient use of information leads to the correlation between image quality and time.  
In our work, we propose an end-to-end trainable bidirectional frame recurrent video super-resolution (BFRVSR) framework to address the above issues. We adopt forward training and reverse training to solve the problem of insufficient utilization of information and preserve temporal consistency, as shown in Figure 2. The BFRVSR has several benefits, which achieves a balance between RVSR and SWVSR. Each input frame needs to be processed no more than twice, while each output frame makes full use of the information contained by optical flow or any other calculated features. In addition, passing the previous high-resolution estimate directly to the other step helps the model to recreate fine details and produce temporally consistent videos. The work of the BFRVSR method is available at [[https://github.com/IlikethisID/BFRVSR](https://github.com/IlikethisID/BFRVSR)]([https://github.com/IlikethisID/BFRVSR](https://github.com/IlikethisID/BFRVSR)).  
Figure 1: The recurrent video super-resolution (RVSR) method has a problem which is the correlation between time (frame number) and Peak Signal to Noise Ratio (PSNR) which is used to evaluate image quality.  
Figure 2: Overview of the proposed bidirectional frame recurrent video super-resolution (BFRVSR) framework. \(N\) frames are input as a group. In the group, each input is two frames of low-resolution and one frame of high-resolution, and the output is one frame of high-resolution. Forward estimation generates \(N\) frames of high resolution. Reverse estimation generates \(N-1\) frames of high resolution.  
Our contributions are mainly reflected in the following: (a) Propose a bidirectional frame recurrent video super-resolution method, in which no pretraining step is required. (b) Address the correlation between image quality and time and preserve temporal consistency.