consistency
(NC)
wrt.
the
pseudo-ground-truthgeometry.
Duringtheevaluation,consistency(NC)wrt.
thepseudo-ground-truthgeometry.
Duringtheevaluation,
ARAH:
Animatable
Volume
wewe
only
keep
the
largestconnected
componentof ofthereconstructedmeshes.
Note
only
keep
the
largestconnected
component
thereconstructedmeshes.
Note Rendering of Articulated Human SDFs
1
2,3
2,31
ShaofeiWang, Katja
Schwarz , Andreas Geiger , Siyu Tangthatis isin infavor
of of
the
baselinesasas
theyareare
more
prone
toto
producingfloating
blob
that
favor
thebaselines
theymoreprone
producingfloating
blob
1ETH Zürich, 2Max Planck Institute for Intelligent Systems, Tübingen, 3University of Tübingen
artifacts.
artifacts.We
Wealsoalsoremove
removeany
anyground-truth
ground-truthororpredictedpredictedmesh
meshpoints
pointsthat
thatareare
https://neuralbodies.github.io/arah/
below
anestimatedgroundplane
toexclude
outliers
fromthegroundbelow an estimated ground plane to exclude outliers from the groundplane
planerom
fromthetheevaluation.
evaluation.For
Forcompleteness,completeness,wewealsoalsoevaluate
evaluatenovel-view
novel-viewsynthesis
synthesis
Surface Points
withPSNR,
SSIM,andLPIPS
using
theposes
fromthetraining
split.
14
Wang et al.
with PSNR, SSIM, and LPIPS using the poses from the training split.
Near Surface Points

ARAH: Animatable Volume Rendering of Articulated Human SDFs

Far Surface Points

Joint root-finding

Color

SDF

Table
1: 1:Generalization
Table
GeneralizationtotoUnseen
Unseen Table
Table2: 2:Geometry
GeometryReconstrucReconstrucPoses.
We
report
LPIPS
[91]
onon
synWe
report
L2L2
ChamferDistancePoses.
We
reportLPIPS[91]
syn- tion.
tion.
We
reportChamfer
Distancethesizedunder(CD)Consistency(NC)thesizedimages
imagesunderunseen
unseenposes
poses(CD)andandNormal
NormalConsistency(NC)
Inputs:
Output:Our Results onSparse
Multi-viewVideosAnimatable
Avatarrom
thetestsetoftheZJU-MoCapon
poses
thefrom the testset of the ZJU-MoCap onthe
thetraining
training
posesof oftheZJUZJUOut-of-distribution
Poses(Observation Space)(Canonical Space)dataset
[60]
(i.e.
views
except0, 0,6, 6, MoCap
dataset[60](i.e.allall
views
exceptMoCapdataset
dataset[60].
[60].Note
Notethat
thatAniN
AniN
Advantages
GoalANproduce
12,12,and18).
Our
approachconsistently
ANoccasionallyoccasionallyproducelarge
largeand18).
Our
approachconsistently andØand
Photorealistic
Rendering and Detailed
Ø
Inputs:SparseMulti-viewVideosGeometryØ Outputs:theAnimatable
Clothedby
Avatars
withoutperforms
blobs
outperforms
thebaselines
baselines
bya alarge
large backgroundbackgroundblobsthat
thatareareconnectedconnected
Ø Generalizes to Out-of-distribution Poses
Pose-Dependent Effects
margin.
margin.We
WereportreportPSNR
PSNRand
andSSIM
SSIM totothe
thebody
bodyresulting
resultingin inlarge
largedeviadeviaAppendix
G.Synthesis
tions
from
thegroundtruth.
Appendix
G. on Unseen Poses and Views
tions
from
thegroundtruth.
Image
Geometry
Reconstruction
MetricMetric NBNBAniN
AniNANAN Ours
Ours

MetricMetricNBNBAniN
AniNANAN Ours
Ours

LPIPS
# #0.143
LPIPS
0.1430.159
0.1590.199
0.1990.107
0.107

CDCD
# #1.560
1.5602.251
2.2513.351
3.3510.917
0.917

33

Existing Works(NeuralBody, Ani-NeRF)(a) Root-finding
and point sampling

Predict Pixel(b) Canonicalization of sampled points(c) SDF-based volume rendering

Technical Contribution
Ø We leverage SDF-based volume rendering [1] to enable detailed geometry reconstruction
Ø Importance sampling would be too expensive if we run root-finding [2] for every point on a ray
Ø We propose a joint root-finding algorithm to enable efficient point sampling around iso-surface
Out-of-distribution generalization of our approach

SSIM" "0.897
0.863
0.911
NCal.
" "0.652
SSIM
0.8970.892
0.892
0.863
0.911Human
0.6520.518
0.5180.647
0.6470.772
0.77212 SDFs Wang
ARAH:
Animatable
VolumeRenderingof Articulated
11 etNC

4.1
4.1 Generalization
GeneralizationtotoUnseen
UnseenPoses
Poses

We
Wefirstfirstanalyze
analyzethe
thegeneralization
generalizationability
abilityof ofour
ourapproach
approachin incomparison
comparisontoto
thethebaselines.
baselines.Given
Givena atrained
trainedmodel
modelandanda apose
posefrom
fromthethetesttestset,set,wewerenderrenderAN [3]A-NeRF

AniN[4]
Ani-NeRF

NB Body[5]
NeuralOurs
Ours

GTANFig. 3: Generalization to Unseen Poses on the testing poses of ZJU-MoCap.
A-NeRF struggles with unseen poses due to the limited training poses and thelack of a SMPL surface prior. Ani-NeRF produces noisy images as it uses aninaccurate backward mapping function. Neural Body loses details, e.g. wrinkles,
because its conditional NeRF is learned in observation space. Our approach
generalizes well to unseen poses and can model fine details like wrinkles.

4.1

Generalization to Unseen Poses

AniNNBOurs

H.3:theAdditional Generalization Results on Out-of-distribution
Fig. 5: Qualitative Results on Out-of-distribution Poses Fig.
fromPoses. From top to bottom: Neural Body, Ani-NeRF, ours, and our geometry.
Experiments
AMASS[43] and AIST++ [34] datasets. From top to bottom row: NeuralBody,Ani-NeRF, our rendering, and our geometry. Note that Ani-NeRF requires reØ We achieve clean and detailed geometry reconstruction, thanks to the underlying SDF representation
training their backward LBS network on novel pose sequence. We did not showØ The
learnedforwardskinning
enables
photorealistic
volume rendering for unseen out-of-distribution poses
A-NeRF
results
as it alreadyproduces
severeoverfittinge↵ects on ZJU-MoCap
test poses. For more qualitative comparisons, please refer to the Supp. Mat.

Ablation – Learned LBS

Ablation – Joint Root-findingØ Surface rendering cannot learn geometry
with abrupt depth changes.
Ø Uniform sampling results in discretized
A-NeRF
Ani-NeRF
Neural Body
Ours
artifacts

Ø Learning backward LBS is does not
generalize
Ø SMPL skinning weights gives suboptimalresults.

GTWe first analyze the generalization ability of our approach in comparison tothe baselines. Given a trained model and a pose from the test
set, 4:
we render
Fig.
Geometry Reconstruction. Our approach reconstructs more
fineBackward
images of the human subject in the given pose. We show qualitative results
LBS suchgrainedgeometry
thanthebaselines
while
preserving
high-frequency
detailsin Fig. 3 and quantitative results in Table 1. We significantly outperform thePSNR 29.5
Surface
Uniform
Ouras wrinkles.
that we remove an estimated ground plane from all meshes.
baselines
bothqualitativelyandquantitatively.
The
training posesof GT
the ZJU- Note
[1] Yariv et al. Volume Rendering of Neural Implicit Surfaces. NeurIPS, 2021
Rendering
Sampling (64)Sampling (33)
MoCap
dataset are extremely
limited, usuallyPSNR
comprising
just 60-300 frames of[2] Chen et al. SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapes. ICCV, 2021
PSNR 28.5
PSNR 29.6
31.6[3] Su et al. A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose. NeurIPS, 2021
repetitive motion. This limited training data results in severe overfitting for thebaselines. In contrast, our method generalizes well to unseen poses, even when

SMPL
Weights
PSNR 30.5Ours
PSNR 31.6

GT[4] Peng et al. Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies. ICCV, 2021
[5] Peng et al. Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans. CVPR, 2021