# Real-Time Optical Flow Estimation Method Based on Cross-Stage Network
## 3 Proposed CAFT Method
### Update Process  
The update operator estimates the sequence of the flow estimate \(\{f_{1},\cdots,f_{N}\}\) at the initial time point \(f_{0}=0\). Each iteration generates \(\Delta f\), an update direction applied to the current estimate. That is, \(f_{k+1}=\Delta f+f_{k}\). The update operator receives the flow, correlation, and potential hidden state as input, and outputs the update \(\Delta f\) and the updated hidden state. The architecture of the update operator is designed to mimic the optimization algorithm. For this, we use weights connected through depth, and we use boundary activation functions to ensure convergence. The update operator is learned by performing an update such that the sequence converges to a fixed point \(f_{k}\to f^{*}\).  
In initialization, the flow field is selected as zero at all locations by default. However, iterative approaches provide the flexibility to experiment with alternatives. When applied to successive images, warm-start initialization is used to project the optical flow of the previous frame pair forward to the next frame pair and to fill and initialize the projection area using neighbor interpolation.  
The input retrieves the correlation feature from the correlation pyramid if the current flow estimate \(f^{k}\) is given. The correlation features are calculated through two convolution layers. In addition, two convolutional layers are applied to the flow estimate itself to generate a flow feature. Finally, the input of the context network is directly injected. The input feature map consists of a connection of correlation, flow, and context features.  
A key component of the update operator is a gate activation unit based on a GRU unit. It replaces the fully connected layer with convolution and is shown in Equation (7).  
\[\begin{array}{c}z_{t}=\sigma\big{(}Conv_{3\times 3}\big{(}[h_{t-1},x_{t}],W_{ 2}\big{)}\big{)}\\ r_{t}=\sigma\big{(}Conv_{3\times 3}\big{(}[h_{t-1},x_{t}],W_{t}\big{)}\big{)}\\ \overset{\sim}{h_{t}}=tanh\big{(}Conv_{3\times 3}\big{(}[r_{t}\odot h_{t-1},x_{t}],W_{ h}\big{)}\big{)}\\ h_{t}=(1-z_{t})\odot h_{t-1}+z_{t}\odot\overset{\sim}{h_{t}}\end{array} \tag{7}\]  
where, \(x_{t}\) is a link between previously defined flows, correlations, and context features. We also evaluate the model with detachable ConvGRU units by replacing \(3\times 3\) convolution with \(1\times 5\) convolution and \(5\times 1\) convolution to increase the acceptance area without significantly increasing the size of the model.  
In the optical flow prediction, the hidden state output by the GRU passes through two convolution layers to predict the flow update \(\Delta f\). The output flow is 1/8 the resolution of the input image. During training and evaluation, the predicted flow field is upsampled with the same resolution as the true value. The network outputs an optical flow at 1/8 the resolution. The optical flow is upsampled to maximum resolution by making the overall resolution flow of each pixel a convex combination of \(3\times 3\) grids with near-low resolution.  
Two convolutional layers are used to predict the \(H/8\times W/8\times(8\times 8\times 9)\) mask, and SoftMax is performed on nine neighbor weights. The final high-resolution flow field is converted into a flow field in the \(H\times W\times 2\) dimension by using this mask to take a weighted combination of neighbors and then changing permutation and shape.