Learning Human Pose Estimation Features with Convolutional Networks
Authors: Ajrun Jain, Jonathan Tompson, Mykhaylo Andriluka, Graham Taylor, Christoph Bregler
ICLR 2014

Abstract: This paper introduces a new architecture for human pose estimation using a multi- layer convolutional network architecture and a modified learning technique that learns low-level features and higher-level weak spatial models. Unconstrained human pose estimation is one of the hardest problems in computer vision, and our new architecture and learning schema shows significant improvement over the current state-of-the-art results. The main contribution of this paper is showing, for the first time, that a specific variation of deep learning is able to outperform all existing traditional architectures on this task. The paper also discusses several lessons learned while researching alternatives, most notably, that it is possible to learn strong low-level feature detectors on features that might even just cover a few pixels in the image. Higher-level spatial models improve somewhat the overall result, but to a much lesser extent then expected. Many researchers previously argued that the kinematic structure and top-down information is crucial for this domain, but with our purely bottom up, and weak spatial model, we could improve other more complicated architectures that currently produce the best results. This mirrors what many other researchers, like those in the speech recognition, object recognition, and other domains have experienced.


Peer review:
[Review]:
This paper examines a way to use convolutional neural networks to estimate human pose features. As many people in the community know, convolutional neural networks have had a major impact in the ImageNet object recognition evaluations. This paper looks at using them for a restricted setting of the challenging problem of human pose estimation. This is clearly an interesting direction to explore, and the details of how one does so are important - as noted by the authors. 

At a high level it seems the main idea and contribution here involves the use of a simple chain structured model to capture spatial relationships between some key parts, namely: faces, shoulders, elbows and wrists. I think the presentation of the spatial model in Figure 3 could be a little cleaner and clearer. Basically it seems like the paper is coming up against the classic problem of how to combine local activity maps with some form of spatial prior or spatial model for how parts fit together. This type of issue has come up a lot in vision in many contexts and a number of approaches have been proposed on how to address the issue within a common theoretical framework, ex CRFs. Here it seems the approach has been to treat the output of the binary predictions for part locations from the CNN as a form of likelihood term that interacts with a prior that has been encoded through the discrete distributions of what seems to be essentially a linear chain Bayesian Network structure. The part that seems like it doesn’t quite match up is the fact that the prior is encoded within the conditionals of the discrete distributions of the Bayesian network while the likelihood is actually the result of the CNNs prediction for a set of binary decisions arising from the sliding window setup.

In general the paper presents some promising results, I think the theoretical framework could be cleaned up a little, but the ideas and results are going in a good direction.