# Estimating Optical Flow with Streaming Perception and Changing Trend Aiming to Complex Scenarios
## 5 Discussion  
We have experimentally demonstrated that the operation of adding the optical flow trend information to correct the estimations when designing optical flow estimation algorithms is very effective during the processing of the image sequence. The source of inspiration given to the idea of this paper is the practices in the multi-object tracking field, where they usually match the detected objects at as many as possible time nodes, or even globally, and then correct and update the key variables such as motion state and trend of the paired objects in Kalman Filter or a similar functional module. Transferring this concept to the field of optical flow estimation, our method considers each pixel as a target, accepts three consecutive image frames as input, then matches and tracks pixels to correct and update them according to their optical flow values and trends for several iterations. However, the optical flow trend proposed in this paper needs to extract the optical flow features from three consecutive image frames as inputs, which obviously may result in a loss or abrupt change of trend when encountering a fast-moving target beyond the image boundary, affecting the accuracy of the optical flow estimation within a certain range of the image boundary. Secondly, as the experiment has shown in the previous section, the performance of our model tested on the Sintel Final dataset does not reach the optimal among the models listed in the table as it does on the Sintel Clean dataset. Comparing the Clean and Final datasets (shown in Figure 4), we can find that the Final dataset has more artistic rendering and environmental effects, and also adds other processing such as motion blur, which make the visual effect smoother when perceived by human eyes, but which \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Training Data**} & \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Sintel (Train)**} & \multicolumn{2}{c}{**KITTI-15**} & \multicolumn{2}{c}{**Sintel (Train)**} \\  & & \multicolumn{2}{c}{**(Train)**} & \multicolumn{2}{c}{**(Train)**} & \multicolumn{2}{c}{**(Test)**} \\ \cline{3-8}  & & **Clean** & **Final** & **AEPE** & **FI-All** & **Clean** & **Final** \\ \hline \multirow{5}{*}{C + T} & RAFT & 1.43 & 2.71 & 5.04 & 17.40 & - & - \\  & RAFT-A & 1.95 & **2.57** & **4.23** & - & - & - \\  & GMA & 1.30 & 2.74 & 4.69 & 17.10 & - & - \\  & GMA + p & 1.33 & 2.87 & 4.83 & **16.60** & - & - \\  & CRAFT & 1.27 & 2.79 & 4.88 & 17.50 & - & - \\  & FM-RAFT & 1.29 & 2.95 & 6.80 & 19.30 & - & - \\ \cline{2-8}  & Ours & **1.24** & 2.58 & 4.90 & 16.81 & - & - \\ \hline \multirow{5}{*}{C + T + S/K (+H)} & RAFT & 0.76 & 1.22 & 0.63 & 1.50 & 1.61 & 2.86 \\  & RAFT-A & - & - & - & - & 2.01 & 3.14 \\ \cline{1-1}  & GMA & 0.62 & **1.06** & **0.57** & **1.20** & 1.39 & 2.47 \\ \cline{1-1}  & GMA + p & 0.65 & 1.11 & 0.58 & 1.30 & 1.54 & 2.63 \\ \cline{1-1}  & CRAFT & 0.60 & **1.06** & 0.58 & 1.34 & 1.45 & **2.42** \\ \cline{1-1}  & FM-RAFT & 0.79 & 1.70 & 0.75 & 2.10 & 1.72 & 3.60 \\ \cline{1-1} \cline{2-8}  & Ours & **0.52** & 1.08 & 0.61 & 1.38 & **1.27** & 2.49 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results on Sintel and KITTI 2015 datasets. “C + T” refers to all the models that are pre-trained on the FlyingChairs [9] and FlyingThings [33] datasets. “S/K” refers to methods that are fine-tuned on the Sintel and KITTI datasets. “(+H)” refers to some methods also fine-tuned on the HD1K dataset. “+p” denotes the joint position and content-wise attention model defined in the GMA paper [14]. “RAFT-A” denotes the method proposed by the paper [36]. The evaluation strategy includes the AEPE and FI-all metrics. provide no help in accurately representing the fast-moving target in each frame. These differences diminish the advantage of our method which incorporates optical flow trend information in the estimating process. Finally, the good news is that the performance of our SPCT model would be more robust in scenes with a moving perspective (such as the camera set on a vehicle or a UAV, etc.)