EXMOVES: Classifier-based Features for Scalable Action Recognition
Authors: Du Tran, Lorenzo Torresani
ICLR 2014

Abstract: This paper introduces EXMOVES, learned exemplar-based features for efficient recognition of actions in videos. The entries in our descriptor are produced by evaluating a set of movement classifiers over spatial-temporal volumes of the input sequence. Each movement classifier is a simple exemplar-SVM trained on low-level features, i.e., an SVM learned using a single annotated positive space-time volume and a large number of unannotated videos. Our representation offers two main advantages. First, since our mid-level features are learned from individual video exemplars, they require minimal amount of supervision. Second, we show that simple linear classification models trained on our global video descriptor yield action recognition accuracy approaching the state-of-the-art but at orders of magnitude lower cost, since at test-time no sliding window is necessary and linear models are efficient to train and test. This enables scalable action recognition, i.e., efficient classification of a large number of different actions even in large video databases. We show the generality of our approach by building our mid-level descriptors from two different low-level feature representations. The accuracy and efficiency of the approach are demonstrated on several large-scale action recognition benchmarks.


Peer review:
[Review]:
This paper explores a computationally efficient way to learn an intermediate representation for action recognition. The technique is somewhat inspired by the approach of ‘action bank’ but focuses on a much more computationally efficient way to achieve a similar effect.  The technique takes advantages of ‘integral videos’ and has the flavor of a modern day Viola & Jones type of technique for recognizing activities in a way similar to the classic technique for face detection. I really like the fact that the authors have taken the issue of computational complexity seriously here. While some might argue that once the community has found techniques that work very well, one could focus on optimizing them for faster run-time performance. However, some choices imply complexity differences that would be very difficult to address in practical working systems and this paper starts by using a technique as a building block that is pretty close to practical right now.

The work here is also capable of learning intermediate representations with a particularly small amount of data, i.e. single exeamples of classes of interest and lots of negative examples, due to the use of Malisiewicz et al,’s exemplar-SVM technique.  This could have advantages in a number of practical situations.

I think this paper is both fairly well presented and executed in terms of the experiment work.  Importantly, it addresses some key practical issues that deserve more attention. The results are not absolutely at the state of the art, but the value added by this work is quite good, especially for those working in the area of action recognition where data processing considerations are particularly challenging.