Learning Human Pose Estimation Features with Convolutional Networks
Authors: Ajrun Jain, Jonathan Tompson, Mykhaylo Andriluka, Graham Taylor, Christoph Bregler
ICLR 2014

Abstract: This paper introduces a new architecture for human pose estimation using a multi- layer convolutional network architecture and a modified learning technique that learns low-level features and higher-level weak spatial models. Unconstrained human pose estimation is one of the hardest problems in computer vision, and our new architecture and learning schema shows significant improvement over the current state-of-the-art results. The main contribution of this paper is showing, for the first time, that a specific variation of deep learning is able to outperform all existing traditional architectures on this task. The paper also discusses several lessons learned while researching alternatives, most notably, that it is possible to learn strong low-level feature detectors on features that might even just cover a few pixels in the image. Higher-level spatial models improve somewhat the overall result, but to a much lesser extent then expected. Many researchers previously argued that the kinematic structure and top-down information is crucial for this domain, but with our purely bottom up, and weak spatial model, we could improve other more complicated architectures that currently produce the best results. This mirrors what many other researchers, like those in the speech recognition, object recognition, and other domains have experienced.


Peer review:
[Review]:
The paper proposes an architecture that takes as input an image and outputs the locations of human body parts (face, shoulder, elbow, wrist). The architecture consists of two parts, of which the second part, however, does not contribute much to classification accuracy (on the data on which the model was tested). 

The first part is a sliding window detector using a binary-output convolutional network. The networks uses smaller pooling regions than what is conventional, in order to retain a high degree of spatial precision, and is otherwise not different from commonly used networks. I find it surprising, that this makes much of a difference, because I would have thought that the peak response of the sliding window conv net be pretty much at the same location, with or without a lot of spatial pooling (especially since you use non-maximum suppression). 

The second part of the architecture is a graphical model (Markov chain) that represents a prior over relative spatial location of the different body parts. It is used to clean up the conv net detections. Unfortunately (but this is also an interesting finding) it does not help much. 

I find the first sentence in section 3.2 strange. Why do you care about false-positives? Or put another way, why don't you increase the detection threshold? It seems like you should really only care about the complete ROC curve. But then, as you show later, the prior you propose here doesn't help much to fix it. 

Using sliding windows with a conv net sounds like it will be slow. Could you say something about the efficiency as compared to the other models? Sapp et al., for example, seem to show that MODEC is not only fairly accurate but also fast (well, as compared to DPM). Apologies, in case this is discussed somewhere and I overlooked it. 

In Section 4.1 the references to the Figures are wrong. 

This is yet another paper showing that conv nets work well in tasks previously dominated by more complicated vision architectures. The paper has some minor issues as pointed out, but overall I enjoyed reading it.