# BFRVSR: A Bidirectional Frame Recurrent Method for Video Super-Resolution
## 4 Experiment
### Analysis  
We train baselines and BFRVSR to convergence under the same parameter conditions. We compare and test the pretrained model on the Vimeo-Test. Table 1 shows the comparison image PSNR results of baselines and BFRVSR. As compared with baselines, our proposed framework has the best effect in continuous 7-frame video sequences, and it is 0.39 dB higher than the RVSR method. PSNR of BICUBIC and SISR is only related to current low-resolution images, and no correlation between high-resolution images. PSNR of VSR and RVSR has correlation between image quality and time. Because of motion compensation by optical flow network, the RVSR performance is better than the VSR.  
BFRVSR performs a forward estimation and a reverse estimation. The BRVSR is equivalent to an RVSR network in forward estimation. It transmits global detail information by using \({{}_{t-1}^{HR}}\) and performs timing alignment operations. However, there are some problems, that is, the details of \({{}_{j}^{LR}}\) cannot be obtained for \({{}_{i}^{LR}}\) to optimize the image (\(i>j\)). Reverse estimation solves this problem. Reverse estimation makes each frame implicitly use all the information to estimate the high-resolution image of the frame. Use the \(\left\{{{}_{t}^{HR}},\ {{}_{t-1}^{LR}},\ {{}_{t}^{LR}}\right\}\) to generate \({{}_{t-1}^{HR}}\).  
RVSR can be trained on video clips of any length. However, if the video clip is too long, RVSR has a problem which is the correlation between image quality and time. In fact, RVSR also has the problem on shorter video clips. BFRVSR solves this problem, as shown in Figure 7. BFRVSR has two processes, i.e., forward estimation and reverse estimation. \({{}_{t-1}^{HR}}\) is used to transmit global information and perform timing alignment operations. In the forward estimation, BFRVSR is equivalent to RVSR. The problem in forward estimation is obvious. When the generated video sequence is \(\left\{{{}_{1}^{LR}},\ldots,{{}_{i}^{HR}},\ldots,\ {{}_{i}^{LR}},\ldots,\ {{}_{N}^{HR}}\right\}\), the reference information generated by the forward estimation of \({{}_{t}^{HR}}\) is \(\left\{{{}_{1}^{LR}},\ldots,\ {{}_{t-1}^{HR}}\right\}\), not global information. Reverse estimation solves this problem. Reverse estimation makes each frame implicitly use all global information to estimate the high-resolution image of the frame by using \(\left\{{{}_{t}^{HR}},\ {{}_{t-1}^{LR}},\ {{}_{t}^{LR}}\right\}\) to generate \({{}_{t-1}^{HR}}\). As shown in Figure 8, the result of reverse estimation is better than result of forward estimation.  
\begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & **Frame1** & **Frame2** & **Frame3** & **Frame4** & **Frame5** & **Frame6** & **Frame7** & **Average** \\ \hline BICUBIC & 29.3057 & 27.3187 & 29.3173 & 29.3120 & 27.3087 & 27.3051 & 27.2900 & 27.3082 \\ SISR & 28.5332 & 28.5633 & 28.5240 & 28.5468 & 28.5523 & 28.5447 & 28.5593 & 28.5462 \\ VSR & 28.7632 & 29.4320 & 29.8012 & 29.8122 & 29.8310 & 29.9001 & 29.9212 & 29.6373 \\ RVSR & 29.0803 & 29.8807 & 30.1547 & 30.2898 & 30.3553 & 30.3980 & 30.3991 & 30.0797 \\ BFRVSR (ours) & 30.4772 & 30.4836 & 30.4833 & 30.4739 & 30.4670 & 30.4547 & 30.4145 & 30.4649 \\ \hline \hline \end{tabular}
\end{table}
Table 1: The PSNR index of the image generated by the five methods of BFRVSR, RVSR, video super-resolution (VSR), single image super-resolution (SISR), and BICUBIC are compared. As can be seen in the table, BFRVSR is an upgrade of RVSR, which has the best effect, and also overcomes the shortcomings of RVSR’s unidirectional gain.  
The video super-resolution, based on the sliding window method processes each frame \(2N+1\) times, the video super-resolution based on the recurrent method processes each frame once, and the BFRVSR processes each frame, at most, two times.  
On the RTX-2080 Ti, the time for a single image Full HD frame for 4\(\times\) super-resolution is 291 ms.