# Real-Time Optical Flow Estimation Method Based on Cross-Stage Network
## 4 Experimental Result  
To evaluate the performance of the model, we used the Flying Chairs [8] dataset and the Flying Things 3D [9] dataset (called dataset T). Learning and evaluation were performed using Flying Chairs and Flying Things 3D datasets, and additionally, the results of fine-tuned models are evaluated through MPI Sintel [22] (referred to as dataset S) and KITTI-2015 [23] (referred to as dataset K). Figure 11 shows examples of the Flying Chair and Things dataset, and Figure 12 shows the MPI Sintel dataset. Also, the equipment used for learning was the Intel i9 CPU, RTX3090 ti GPU, and 128 GB of RAM. Figure 11: Flying Chair and Flying Things dataset. The MPI Sintel dataset is a cinematic 3D animated dataset created in 2012 by the Max Planck Institute for Computer Science and TU Eindhoven. This dataset contains 1064 rendered stereo images and 23 different scenes. Each image has a resolution of 1024 \(\times\) 436 pixels and an 8-bit channel. The MPI Sintel dataset is used to evaluate various visual computational tasks such as optical flow, depth estimation, object recognition, and 3D reconstruction. This dataset mainly evaluates optical flows and provides a variety of metrics that can be used to evaluate the performance of each model. The main indicator is end point error (EPE), which refers to the average of the distance between optical flow prediction and actual optical flow. The KITTI dataset consists of 200 learning and test images, respectively, of stereo, flow, and scene flow. These KITTI datasets have the indicators F1-epe and F1-all. As mentioned above, EPE represents how far it is from the actual value of the prediction flow in pixels. This value averages all pixels in all images. First, EPE is calculated independently in each image and then averaged over all images, and in the KITTI dataset, the indicator is defined as F1-epe. F1-all is a matrix of KITTI datasets that is defined by the author of the data set and is used in KITTI reader boards. This value corresponds to the average of pixels with an EPE of less than 3 pixels, or less than 5% of the flow. In this paper, the proposed optical flow model, CAFT, was implemented using Pytorch (version 1.8) [24]. All layers are initialized with random weights. During learning, the AdamW [25] optimizer is used. We also evaluated the performance of our proposed model on the Sintel and KITTI datasets. Fine-tuning is performed for each data set. To compare with previous studies, we show the results of a fine-tuned model for Sintel and a fine-tuned model for KITTI only. Table 1 shows the number of parameters and inference speed of the networks of the proposed model and traditional methods. Table 2 shows the results of the Sintel and KITTI datasets, and Figure 13 plots comparing parameter counts, inference time, training iterations, and accuracy. After training on FlyingChairs (C dataset) and FlyingThings (T dataset), generalization performance was tested on Sintel (train), and we show that the proposed CAFT method outperforms all existing models on both Clean and Final datasets. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**Model** & **LiteFlowNetX** & **LiteFlowNet** & **IRR-PWC** & **PWCNet** & **FlowNet2** & **VCN** & **RAFT** & **Proposed** \\ \hline Param & 0.9 M & 5.4 M & 6.4 M & 8.8 M & 162.0 M & 6.2 M & 5.8 M & 4.2 M \\ Time(s) & - & 0.971 & 0.211 & 0.026 & 0.124 & 0.262 & 0.102 & 0.059 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The number of parameters and inferences from the networks of the proposed model and traditional methods. Param represents the number of learning parameters in each method, and Time(s) represents the time it takes to infer an optical flow. Figure 12: MPI Sintel dataset. Table 2 shows the performance comparison based on the average value of errors for a specified optical flow point, which is expressed as a mean end-point error (MEPE).