  Real-time rendering and animation of humans is a core function in games,
movies, and telepresence applications. Existing methods have a number of
drawbacks we aim to address with our work. Triangle meshes have difficulty
modeling thin structures like hair, volumetric representations like Neural
Volumes are too low-resolution given a reasonable memory budget, and
high-resolution implicit representations like Neural Radiance Fields are too
slow for use in real-time applications. We present Mixture of Volumetric
Primitives (MVP), a representation for rendering dynamic 3D content that
combines the completeness of volumetric representations with the efficiency of
primitive-based rendering, e.g., point-based or mesh-based methods. Our
approach achieves this by leveraging spatially shared computation with a
deconvolutional architecture and by minimizing computation in empty regions of
space with volumetric primitives that can move to cover only occupied regions.
Our parameterization supports the integration of correspondence and tracking
constraints, while being robust to areas where classical tracking fails, such
as around thin or translucent structures and areas with large topological
variability. MVP is a hybrid that generalizes both volumetric and
primitive-based representations. Through a series of extensive experiments we
demonstrate that it inherits the strengths of each, while avoiding many of
their limitations. We also compare our approach to several state-of-the-art
methods and demonstrate that MVP produces superior results in terms of quality
and runtime performance.
