Extending state-of-the-art object detectors from image to video is challenging. The accuracy of detection suffers from degenerated object appearances in videos, e.g., motion blur, video defocus, rare poses, etc. Existing work attempts to exploit temporal information on box level, but such methods are not trained end-to-end. We present flow-guided feature aggregation, an accurate and end-to-end learning framework for video object detection. It leverages temporal coherence on feature level instead. It improves the per-frame features by aggregation of nearby features along the motion paths, and thus improves the video recognition accuracy. Our method significantly improves upon strong singleframe baselines in ImageNet VID [33] , especially for more challenging fast moving objects. Our framework is principled, and on par with the best engineered systems winning the ImageNet VID challenges 2016, without additional bells-and-whistles. The code would be released. Recent years have witnessed significant progress in object detection [11] . State-of-the-art methods share a similar two-stage structure. Deep Convolutional Neural Networks (CNNs) [22, 36, 40, 14] are firstly applied to generate a set of feature maps over the whole input image. A shallow detection-specific network [13, 10, 30, 26, 5] then generates the detection results from the feature maps. These methods achieve excellent results in still images. However, directly applying them for video object detection is challenging. The recognition accuracy suffers from deteriorated object appearances in videos that are seldom observed in still images, such as motion blur, video defocus, rare poses, etc (See an example in Figure 1 and more in Figure 2) . As quantified in experiments, a state-of-the-art stillimage object detector (R-FCN [5] + ResNet-101 [14]) deteriorates remarkably for fast moving objects (Table 1 (a) ).
Based on the aforementioned work, Zhu et al. REF proposed to aggregate nearby features along the motion path, improving the feature quality.
Flow-Guided Feature Aggregation for Video Object Detection
2017 IEEE International Conference on Computer Vision (ICCV)
2017 IEEE International Conference on Computer Vision (ICCV)
Computer Science