Appearance and motion are two key components to depict and characterize the video content. Currently, the two-stream models have achieved stateof-the-art performances on video classification. However, extracting motion information, specifically in the form of optical flow features, is extremely computationally expensive, especially for large-scale video classification. In this paper, we propose a motion hallucination network, namely MoNet, to imagine the optical flow features from the appearance features, with no reliance on the optical flow computation. Specifically, MoNet models the temporal relationships of the appearance features and exploits the contextual relationships of the optical flow features with concurrent connections. Extensive experimental results demonstrate that the proposed MoNet can effectively and efficiently hallucinate the optical flow features, which together with the appearance features consistently improve the video classification performances. Moreover, MoNet can help cutting down almost a half of computational and datastorage burdens for the two-stream video classification. Our code is available at: https://github.com/ YongyiTang92/MoNet-Features.
Monet models the temporal relationship of appearance features and concurrently calculates the contextual relationship of optical flow features using LSTM REF .
Hallucinating Optical Flow Features for Video Classification
ArXiv
ArXiv
Computer Science