# Efficient Video Frame Interpolation Using Generative Adversarial Networks
## 3 The Proposed Method  
### Network Architecture  
This research is targeted at an effectual GAN-based frame interpolation framework without creating auxiliary data. As is illustrated in Figure 1, the proposed frame interpolation network \(G\) takes two input frames \(x_{n-1}\) (previous frame) and \(x_{n+1}\) (future frame) to derive \(y_{n}\) as a good estimate of the current frame \(x_{n}\). Each frame is a two-dimensional image with three color channels (red, green, blue). The proposed discriminator \(D\) reports a classification value in the range of 0.0 and 1.0 for \(y_{n}\), which indicates the confidence of \(y_{n}\) for being a real frame. Motivated by the DCGAN [4], our networks are fully convolutional, composed of 2D convolutional layers, 2D transposed convolutional layers, batch normalization layers, and Leaky Rectification Linear Unit (LeakyReLU) layers.  
The complete frame interpolation process is separated into two phases, analysis and synthesis, as shown in Figure 2. The analysis phase extracts a feature map from the given input frames, while the synthesis phase produces an output frame from the feature map. No pooling function is involved in our approach. Instead, we use strides in convolutional layers to control the size of the feature map. The discriminator uses down-sampling processes to generate the classification result. The essential parameters of the proposed generator network and discriminator network are in Tables 2 and 3, respectively. Specific values of the coefficients involved in the network are obtained through the training process.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Author** & **Method** & **Application** & **Dataset** & **Result** \\ \hline [PERSON] et al. [7] & Process data in the z-domain with a CNN based network and a GAN. The framework & Image/video compression & For Mode (video compression framework): the KTH actions dataset. & The videos reach PSNR/SSIM of 22.85 \\  & & & & \\  & & & & \\  & & & & \\  & & & & \\  & & & & \\ \hline [PERSON] et al. [8] & Concatenate two GANs for generation with the additional input data obtained by the linear interpolation operation. & Generate N frame(s) between given ones (N \(\geq\) 1). & The KTH actions dataset; the UCF101 dataset; he & Generate seven frames between two given ones (N \(\geq\) 1). & between two given ones \\ \hline [PERSON] et al. [29] & Utilize CNN and GAN for generation and refinement, respectively. & Generate a frame in-between two given ones. & Xiph.org Video Test Media: ‘Bus’, ‘Football’, ‘News’, and ‘Stefan’. & Correct in structure and most of the details, but not close to the ground truth, particularly motion. \\ \hline [PERSON] et al. [30] & Propose a multi-scale GAN that includes a multi-scale residual estimation module and a CNN based refinement network. & Frame interpolation. & Collection of 60 fps videos from Youtube-8m. & Achieve visual quality comparable to SepConv [1] at x4f faster runtime. \\ \hline [PERSON] et al. [31] & Use a phase-based deep learning method that consists of a neural network decoder. & Frame interpolation for challenging scenarios, coping with large motion (larger than existing methods). & DAVIS video dataset. & Produce preferable output in challenging cases containing motion blurs and brightness changes. May produce more disturbing artifacts. \\ \hline \hline \end{tabular}
\end{table}
Table 1: Deep learning-based approaches for frame interpolation.  
To further improve the interpolated frame, particularly the motion and color, the proposed framework applies a refinement network proposed in [6]. This network is the image-to-image translation network that includes a generator network \(G\_RN\) and a discriminator network \(D\_RN\). The generator \(G\_RN\) translates an input frame similar to the ground truth, particularly in motion area.