# Reducing System Load of Effective Video Using a Network Model
## 2 Related Work
### DAIN (Depth Aware Video Frame Interpolation)  
The DAIN network is a method that can have many different applications, such as slow-motion creation, new view compositing, and frame recovery [7; 8]. The high frame rate is a very attractive form of video streaming for viewers and users. However, problems such as network delay or motion blur which can be caused by high frame rates can affect system load [9]. Even if you try to use a network, such as a CNN, to solve the problem, the effect is not significant. Therefore, high-quality frame problems for video are still difficult to solve with basic networks. There are several approaches to handling large motions, such as fine-tuning strategies or adjusting architecture by means of flow estimation.  
Among these approaches, PWC-Net can be used to estimate accurate flow [10]. Another approach is to estimate the mask through the pixel method. There is also a learning method that adapts the space changes and neighboring pixels that have been recently discussed. This method produces effective results by extracting and synthesizing specific frames through a pre-trained classification network. However, these methods may not be efficient as they are highly dependent on the model data that is required for training. Using the DAIN network, an estimated image is synthesized by calculating the intermediate frame between two existing images. This network is a highly effective means of achieving optical flow estimation, depth estimation, content extraction, amongst other things. In particular, it is excellent for estimating flow and depth in both directions for two input frames. It contributes to the intermediate flow by adopting a projection layer through the input frame. Since multiple flow vectors can meet at the same location, the contribution of each flow vector is calculated based on depth value. The proposed approach creates a flow with clearer motion boundaries due to the influence of depth, not just flow. Figure 4 shows a model of DAIN.  
The details of the depth perception proposed are shown in the following figure.  
Figure 5 is a representation of flow projection. Flow projection produces an average flow vector (green arrow). For a depth-aware flow projection layer, the projected flow will point to pixels with a smaller depth value (red arrow). On the other hand, the flow vector does not pass through these pixels, and so it can be that there are voids in the intermediate flow. The empty space created here can be compensated by calculating the average flow between neighbors. With the following formula, we can see the content of the depth.  
Figure 3: Example of 360-degree video warping. (**a**) 360-degree video. (**b**) Warped video.  
The projected flow \(\mathrm{F_{t\to 0}}\) is defined by:  
\[\mathrm{F_{t\to 0}}(\mathrm{x})=-\mathrm{t*}\ \frac{\sum_{y\in S}\omega_{0}(y)* \mathrm{F_{0\to 1}}(y)}{2} \tag{1}\]  
The flow \(\mathrm{F_{0\to 1}}\) (y) passes through x at time t. One can approximate \(\mathrm{F_{t\to 0}}(\mathrm{x})\) by \(-\mathrm{t}\)\(\mathrm{F_{0\to 1}}(\mathrm{y})\). Similarly, we approximate \(\mathrm{F_{t\to 1}}(\mathrm{x})\) by \(-(\mathrm{1}-\mathrm{t})\)\(\mathrm{F_{1\to 0}}(\mathrm{y})\).  
Where the weight \(\mathrm{w_{0}}\) is the reciprocal of depth:  
\[\omega_{0}(y)=\frac{1}{\mathrm{D_{0}}(y)} \tag{2}\]  
\(\mathrm{D_{0}}\) is the depth map of the input frame. The flow in the empty space position is computed by averaging the available flows from its neighbors:  
\[\mathrm{F_{t\to 0}}(\mathrm{x})=\frac{1}{|\mathrm{N}(\mathrm{x})|}\sum_{x^{ \prime}\in\mathrm{N}(\mathrm{x})}\mathrm{F_{t\to 0}}(x^{\prime}) \tag{3}\]  
where N (x) = {x^: 1S(x^) 1 > 0} is the 4-neighbors of x. From (1) and (3), we obtain the dense intermediate flow fields \(\mathrm{F_{t\to 0}}\) and \(\mathrm{F_{t\to 1}}\) for warping the input frames. If the above method is used, we can get many complements by learning the hierarchical features that extract information from most networks and contents [11].