We propose an effective approach for spatio-temporal action localization in realistic videos. The approach first detects proposals at the frame-level and scores them with a combination of static and motion CNN features. It then tracks high-scoring proposals throughout the video using a tracking-by-detection approach. Our tracker relies simultaneously on instance-level and class-level detectors. The tracks are scored using a spatio-temporal motion histogram, a descriptor at the track level, in combination with the CNN features. Finally, we perform temporal localization of the action using a sliding-window approach at the track level. We present experimental results for spatiotemporal localization on the UCF-Sports, J-HMDB and UCF-101 action localization datasets, where our approach outperforms the state of the art with a margin of 15%, 7% and 12% respectively in mAP.
Philippe et al. REF adopted the same feature extraction procedure, then utilized a tracking-by-detection approach to link frame-level detections, in combination with a class-specific detector.
Learning to Track for Spatio-Temporal Action Localization
2015 IEEE International Conference on Computer Vision (ICCV)
2015 IEEE International Conference on Computer Vision (ICCV)
Computer Science