  Deep neural networks have achieved great progress in single-image 3D human
reconstruction. However, existing methods still fall short in predicting rare
poses. The reason is that most of the current models perform regression based
on a single human prototype, which is similar to common poses while far from
the rare poses. In this work, we 1) identify and analyze this learning obstacle
and 2) propose a prototype memory-augmented network, PM-Net, that effectively
improves performances of predicting rare poses. The core of our framework is a
memory module that learns and stores a set of 3D human prototypes capturing
local distributions for either common poses or rare poses. With this
formulation, the regression starts from a better initialization, which is
relatively easier to converge. Extensive experiments on several widely employed
datasets demonstrate the proposed framework's effectiveness compared to other
state-of-the-art methods. Notably, our approach significantly improves the
models' performances on rare poses while generating comparable results on other
samples.
