# Real-Time Optical Flow Estimation Method Based on Cross-Stage Network
###### Abstract  
In this paper, a real-time optical flow estimation method based on a cross-stage network is proposed. The proposed model is designed with a network structure with encoders and decoders. The proposed method combines cross-stage network technology with the network structure of FlowNet2 and RAFT to achieve improved parameter number and estimation performance. For real-time optical flow estimation, it is important to maintain performance while reducing the number of parameters in the network. In the proposed method, structural convergence is performed to increase performance while reducing the number of parameters by applying the cross-stage network structure. The proposed model is designed to solve the bottlenecks in model accuracy and complexity by separating feature extraction and flow estimation processes. Flying Chairs, Flying Things 3D, and KITTI datasets were used to evaluate the performance of the proposed model, and the experimental results show superior performance compared to previous traditional methods. optical flow; cross-stage; FlowNet2; RAFT; CSPNet; real-time +
Footnote †: journal: Applied  
0000-0001-5000-0000-0000-0000-0000-0000-0000-0000-0000-00000-0000-00000  
CNN model with an architecture similar to FlowNet2. SpyNet estimates flow fields using spatial pyramid structures. A series of tasks such as PWC-Net [11; 12], LiteFlowNet [13; 14], and VCN [15] have used coarse to sophisticated estimation methodologies. Among them, LiteFlowNet estimates the flow field using cascading flow estimation. At each pyramid level, LiteFlowNet uses flow estimation from the previous level to warp the second image in the direction of the first image. The flow field is then estimated by comparing the warped image with the feature map of the first image. LiteFlowNet uses an f-lconv layer at each pyramid level to normalize the flow fields. The f-lconv layer uses both direction and brightness information in the flow field to normalize the flow field. These models had the problem of missing small and fast-moving objects at rough stages. To address this issue, [PERSON] and [PERSON] [16] proposed RAFT (Recurrent All-Pairs Field Transforms), which uses a multi-step search window and recursive approach while performing optical flow estimation at a rough stage. RAFT is a state-of-the-art optical flow estimation algorithm that is designed to calculate dense optical flow fields. Optical flow refers to the pattern of apparent motion of objects, surfaces, and edges in a visual scene, caused by the relative motion between an observer (e.g., a camera) and the scene. RAFT stands out for its accuracy and the ability to estimate optical flow under various challenging conditions. The success of RAFT largely lies in the large number of iterative refinements it can perform [17]. Therefore, RAFT has better performance than PWC-NET, but it has the disadvantage of being late in prediction speed, and many studies are being conducted to improve these problems [18; 19]. In this paper, we design an optical flow network that is the basis for camera pose tracking and propose a network structure with fewer parameters and improved performance than the conventional models. The proposed technique calculates optical flows through successive input images and conducts experiments comparing them with conventional models. This study aims to explore an alternative CNN architecture for accurate and efficient optical flow estimation. Although this work was inspired by the success of FlowNet2 and RAFT, we take a closer look at the key elements that allow us to fully unleash the potential of deep convolutional networks in combination with classical principles. There are two general principles for improving the design of FlowNet2 and SPyNet. The first principle is pyramid feature extraction. The proposed network, named CAFT (Cross-stage All-Pairs Field Transforms), consists of encoders and decoders. The encoder maps each given pair of images into a pyramid of multi-scale, high-dimensional features.