# Estimating Optical Flow with Streaming Perception and Changing Trend Aiming to Complex Scenarios
## 5 Discussion
and under perturbation (which are mainly from a poorly fixed camera, such as overhead surveillance cameras, which are susceptible to significant shaking by wind). For this kind of situation, networks with our module will need fewer additional artificial compensation algorithms for parameters such as camera shift and exposure rate than those without. To make our work more engineering and application-oriented, we used a data-collecting vehicle (deployed for collecting autonomous driving scene data) and deployed an overhead surveillance camera in a parking lot to verify the performance of our method for scenes with fast-moving targets and under low brightness environment. For the first case, we captured sequences of images from a front camera on our experimental vehicle when driving into real road intersections during a driving scenario dataset-collecting project in Shanghai. As shown in Figure 5, we compared the optical flow estimation results predicted by RAFT, one of its variants, GMA, and our SPCT model. We pay more attention to those fast-moving targets when processing the captured images. In this frame, we can find out that only our SPCT model predicts the correct optical flow estimation result for the pedestrian (marked with a red dashed box) who just suddenly broke into the lane of the ego car from the obscured blue vehicle. Figure 4: Images from Sintel Clean (left column), Sintel Final (right column), and their visualized optical flow map estimated by our model. The images from the Sintel Final dataset have more background rendering and motion blur effects compared to the Clean one, resulting in our model learning relatively less or even incorrect information about the optical flow trend. The scene from Figure 6 is captured from an overhead monitor set up on a gantry in the parking lot. Camera shaking is caused by windy weather or insufficient fixation of the gantry itself, which has some disturbance to the continuous frame images captured. Comparing the estimating results of RAFT, its variant, GMA and our SPT, we found out that RAFT is nearly beaten by the shaking environment. The GMA does a better job, but also gets confused about the lane lines and the parking lines. Due to the stream perception and the additional changing trend information, our SPT still shows good behavior on this task. Figure 5: Applying our SPT model in the real road scenario. The image sequences are captured from a front camera of our collecting vehicle when driving on an urban road in Shanghai. Comparing the three results, which are separately estimated from RAFT, GMA, and our SPT models, only SPT could predict the optical flow value of this pedestrian. Figure 6: Applying optical flow estimation model for parking lot scenario in the daytime. This scene is collected from an overhead surveillance camera set up on a gantry on our campus. We also use some red dashed boxes to mark the regions of apparently incorrect optical flow estimation. The results presented here demonstrate that our SPT still has a subtle advantage in this scenario. Actually, in a well-light environment, the enhancement of our model is not particularly outstanding, so we performed the same experiment again, only this time the brightness was lacking because it was at night. From this latest experiment, we obtained a set of results with a much larger gap of differences, as shown in Figure 7. Thanks to the input of three consecutive frames, our SPCT structure takes into account the optical flow at the earlier moments and its continuous trend compared to other conventional structures such as RAFT. This improvement makes SPCT more effective in the low-brightness environment and has even more significant performance improvements compared to daytime.