Learnable Higher-order Representation for Action Recognition
Capturing spatiotemporal dynamics is an essential topic in video recognition.In this paper, we present learnable higher-order operation as a generic family of building blocks for capturing higher-order correlations from high dimensional input video space.We prove that several successful architectures for visual classification tasks are in the family of higher-order neural networks, theoretical and experimental analysis demonstrates their underlying mechanism is higher-order. On the task of video recognition, even using RGB only without fine-tuning with other video datasets, our higher-order models can achieve results on par with or better than the existing state-of-the-art methods on both Something-Something and Charades datasets.
Proposed higher order operation for context learning', "Proposes a new 3D convolutional block which convolves video input with its context, based on the assumpton that relevant context is present around the image's object.