Learning to encode motion using spatio-temporal synchrony
Authors: Kishore Reddy Konda, Roland Memisevic, Vincent Michalski
ICLR 2014

Abstract: We consider the task of learning to extract motion from videos. To this end, we show that the detection of spatial transformations can be viewed as the detection of synchrony between the image sequence and a sequence of features undergoing the motion we wish to detect. We show that learning about synchrony is possible using very fast, local learning rules, by introducing multiplicative 'gating' interactions between hidden units across frames. This makes it possible to achieve competitive performance in a wide variety of motion estimation tasks, using a small fraction of the time required to learn features, and to outperform hand-crafted spatio-temporal features by a large margin. We also show how learning about synchrony can be viewed as performing greedy parameter estimation in the well-known motion energy model.


Peer review:
[Review]:
One might think that everything has been said about motion estimation and motion energies, though refreshing ideas are always welcome even in subjects with thousands (ten of thousands?) papers. 

The paper overall presentation and discussion are very clear and friendly.

When discussing multiplicative (2.3), isn't this saying we should be working with additive but in the log space? log space is very frequently used in image analysis.

The locality property is very interesting and as the authors claim, very powerful for computation. I believe is worth investigating for other applications and models.

Not all the mentioned results are the state-of-the-art for the used datasets, but this is not critical.

Does it make sense to report results for optical flow standard sets?

While I don't believe the paper is revolutionary, it is nice to read and has some interesting insights.