# Estimating Optical Flow with Streaming Perception and Changing Trend Aiming to Complex Scenarios
## 4 Experiments
### Ablations  
As mentioned in Sections 3.3 and 3.4, in this section, we use three ablation experiments to determine the best way of fusing continuous and momentary optical flow features, searching for the optimal combination of the hyperparameters \(\delta\) and \(\mu\), and verifying the effectiveness of both the C-MFF module and the total loss that we designed. **Best fusion scheme**. During the process of exploring the best fusion strategies for two types of optical flow features, we compared two conventional algorithms (element-wise adding and concatenation) and two advanced algorithms (spatial transformer network [25] and non-local method [26,27]). In Table 1, we can clearly see that only the estimation result after the element-wise adding operation is worse than the baseline; the other three algorithms are more or less improved. This is probably because direct element-wise adding for the two extracted optical flow features would damage the inter-frame information as well as the changing trend, which is absolutely contrary to the whole idea of our SPCT. In addition, among the three methods that can enhance performance, it is clear that we could obtain the most optimal fused flow features just through the simplest concatenation operation. **Best hyperparameter scheme**. As presented in Equation (6), Section 3.4, we decided to use two hyperparameters, \(\delta\) and \(\mu\), as the regulator to adjust the sensitivity of trend cost. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Training** \\ **Data** \\ \end{tabular} } & \multirow{2}{*}{**Method**} & \multicolumn{2}{c}{**Sintel (Train)**} & \multicolumn{2}{c}{**KITTI-15 (Train)**} \\ \cline{3-6}  & & **Clean** & **Final** & **AEPE** & **Fl-All (\%)** \\ \hline \multirow{4}{*}{C + T} & Baseline (RAFT) & 1.43 & 2.71 & 5.04 & 17.40 \\  & Element-wise Adding & 1.47 & 2.81 & 5.18 & 17.74 \\ \cline{1-1}  & NL & 1.38 & 2.69 & 4.96 & 17.14 \\ \cline{1-1}  & STN & 1.33 & 2.65 & 4.92 & 16.98 \\ \cline{1-1}  & Concatenation & 1.24 & 2.58 & 4.90 & 16.81 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Ablation experiments for choosing the best fusion scheme between continuous flow feature and momentary flow feature. We employ a basic RAFT optical flow estimator as the baseline for all experiments. This table shows the results on Sintel and KITTI datasets for all fusion strategies after training on FlyingChairs (C) and FlyingThings (T) datasets. Figure 3: The original image and flow maps of ground truth, original RAFT, and its variant GMA(-p) models compared with the flow map of our SPCT model on the Sintel Clean dataset. The green circles indicate partial errors eliminated by our model where the other listed models did not. What can be seen is that our SPCT model has a clear advantage in estimating the optical flow trend at the contour area, especially for images that have fast-moving targets, such as this flying scene. We attribute this advantage to introducing the information from the optical flow trends into our SPCT model. This sensitivity here is meant to recognize those pixels that change their state of motion abruptly. In detail, \(\mu\) acts as a threshold to monitor the proportion of those emerging pixels in the whole image, while \(\delta\) controls the degree of attention that the network is able to have when considering those emerging pixels. We set the default for \(\delta\) at over 1.0 in order to make the attention of our model fairly loose, to pay to the unmatchable and mutable pixel pairs, and to make it focus on the useful motion trend during pixel movement. In reference to finding the optimized hyperparameter pairs, we handled this with a grid search, as presented in Table 2, and found out that the performance of our model reached the peak when we set \(\delta\) and \(\mu\) at 0.35 and 1.6, respectively. **Effect of the proposed module and loss**. To validate the effect of the C-MFF module and proposed loss function, we conduct ablation experiments on the baseline flow estimator.