# Video Super-Resolution Network with Gated High-Low Resolution Frames
###### Abstract  
In scenes with large inter-frame motion variations, distant targets, and blurred targets, the lack of inter-frame alignment can greatly affect the effectiveness of subsequent video super-resolution reconstruction. How to perform inter-frame alignment in such scenes is the key to super-resolution reconstruction. In this paper, a new motion compensation method is proposed to design an alignment network based on gated high-low resolution frames. The core idea is to introduce a gating mechanism while using the information of high-low resolution neighboring frames to perform motion compensation adaptively. Meanwhile, within this alignment framework, we further introduce a pre-initial hidden state network and a local scale hierarchical salient feature fusion network. The pre-initial hidden state network is mainly used to reduce the impact of unbalanced quality effects between frames that occur in one-way cyclical networks; the local scale hierarchical salient feature fusion network is used to fuse the features of aligned video frames to extract contextual information and locally salient features to improve the reconstruction quality of the video. Compared with existing video super-resolution methods, this method achieves good performance and clearer edge and texture details. video super-resolution; gating mechanisms; high-low resolution frame information; adaptive motion compensation; hidden states; locally significant features +
Footnote †: journal: Medical  
As shown in Figure 1a,b, in the past there were two major types of motion compensation approaches for deep learning-based video super-resolution algorithms: the first approach was to perform motion estimation on low-resolution adjacent frame sequences [1, 8, 9, 10, 11, 12, 13], and the second approach was to perform motion estimation on adjacent low-resolution frame sequences and obtained stream estimates for up-sampling to generate high-resolution stream estimates [3, 4, 5, 14]. The first approach does not make good use of the high-frequency information of the high-resolution features of the adjacent frames. The second approach does not make good use of the similarity between low-resolution frames and high-resolution frames, and the motion compensation performed takes a secondary operation that greatly affects the details lost between up- and down-sampling. Combining the above two alignment ideas, a new motion compensation is proposed, and a gating mechanism is introduced to adaptively select useful information, as shown in Figure 1c. The gated high-low resolution frames network (GHLRF) is designed as a new video super-resolution algorithm based on gated high-low resolution frames, aiming to solve the problems of existing methods that do not make full use of the information of adjacent high-low resolution frames and the large error in motion estimation. The method first obtains motion information between neighboring frames through a motion estimation method and then uses the neighboring frame information for motion compensation. Next, a gating mechanism is used to adaptively select useful information in high-low resolution neighboring frames to avoid the effect of motion estimation errors, thus acquiring aligned video frames. In addition, a pre-initial hidden state network is adopted to reduce the impact of video frame quality imbalance that occurs in a one-way cyclical framework, and a local scaled hierarchical salient feature fusion network is used to fuse the features of the aligned video frames, allowing for better extraction of contextual information, locally salient features, and the extraction of high-frequency detail information of the edges and textures of the video frames to improve the reconstruction quality of the video.