MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics
Authors: Changmin Lee
Jihyun Lee
Tae-Kyun Kim
Venue: NeurIPS 2025 poster ()
Keywords: 3D Avatar Creation
Digital Humans
Gaussian Splatting
Motion Modeling
Primary area: applications

Abstract: While there has been significant progress in the field of 3D avatar creation from visual observations, modeling physically plausible dynamics of humans with loose garments remains a challenging problem. Although a few existing works address this problem by leveraging physical simulation, they suffer from limited accuracy or robustness to novel animation inputs. In this work, we present MPMAvatar, a framework for creating 3D human avatars from multi-view videos that supports highly realistic, robust animation, as well as photorealistic rendering from free viewpoints. For accurate and robust dynamics modeling, our key idea is to use a Material Point Method-based simulator, which we carefully tailor to model garments with complex deformations and contact with the underlying body by incorporating an anisotropic constitutive model and a novel collision handling algorithm. We combine this dynamics modeling scheme with our canonical avatar that can be rendered using 3D Gaussian Splatting with quasi-shadowing, enabling high-fidelity rendering for physically realistic animations. In our experiments, we demonstrate that MPMAvatar significantly outperforms the existing state-of-the-art physics-based avatar in terms of (1) dynamics modeling accuracy, (2) rendering accuracy, and (3) robustness and efficiency. Additionally, we present a novel application in which our avatar generalizes to unseen interactions in a zero-shot manner—which was not achievable with previous learning-based methods due to their limited simulation generalizability. Our code will be publicly available.

MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics
Changmin Lee
Jihyun Lee
Tae-Kyun Kim
KAIST
(1cm914, jyun.lee, klntaekkuun@kkiistacc..
Multi-view video input
Novel posc driving
Zero-shot seene interactions
Figure 1. 3D Gaussian avatars with accurate and robust physics-based animation. We present MPMAvatar. a framework for creating 3D Gaussian avatars from multi-view videos, that support physicully accurute and robust unimations, especially for loose garments. Ours is also zero-shot generalizable to novel scene interactions.
Abstract
While there has been significant progress in the field of 3D avatar creation from visual observations, modeling physically plausible dynamics of humans with loose garments remains challenging problem. Although few existing works address this problem by leveraging physical simulation, they suffer from limited accuracy robustness to novel animation inputs. In this work, we present MvMAvatar, a framework for creating 3D human avatars from multi-view videos that supports highly realistic, robust animation, as well photorealistic rendering from free viewpoints. For accurate and robust dynamics modeling, our key idea is to use Material Point Method-based simulator, which we carefully tailor to model garments with complex deformations and contact with the underlying body by incorporating anisotropic constitutive model and novel collision handling algorithm. We combine this dynamics modeling scheme with our canonical avatar that can be rendered using 3D Gaussian Splatting with quasi-shadowing, enabling high-fidelity rendering for physically realistic animations. our experiments, demonstrate that MPMtva significantly outperforms the existing state-of- the-art physics- based avatar in terms of (1) dynamics modeling accuracy, (2) rendering accuracy, and (3) robustness and efficiency. Additionally, present a novel application in which our avatar generalizes to unseen interactions in a zero- shot manner-which was not achievable with previous learning- based methods due to their limited simulation generalizability. Our project page is at: https; //KAISTChangmin. gi thub. 10/MPMAvatar
Introduction
Creating 3D human avatars has been an important research problem due to their broad range of applications, including virtual and augmented reality, computer games, and content creation. The key goals of this field include modeling and learning avatars that support (1) photorealistic rendering from free viewpoints and (2) realistic animation driven by sparse motion inputs (e.g., SMPL-X 511
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

parameters), On the rendering side, recent approaches based on 3D Gaussian Splatting [22] have shown remarkable progress in enabling tigh-fidelity avatar rendering from free viewpoints. However, realistically animating avatars under novel motion inputs remains a challenging problem, especially for loose garments. Existing methods typically model these garment motions using piecewise linear 61 but they are known to be limited in accurately capturing complex deformations and tend to overfit to motions observed during training, as discussed in [78]
transformations [ G 48
72] 47 73 or pose-dependent geometric correctives (5] 38 39 65].
To address these limitations, incorporating thysics-based prior via physically simulating the avatar dynamics can be an effective approach to enhance the realism and generalizability of animation. There are only a few works towards this direction; Xiang et al. [69] proposes an avatar simulated using X-PBD [T], but their method relies on time-consuue consuminn manual parameter search to approximate reasonable cloth behavior. PhysAvatar 78 is the most recent physies-based avatar that adopts C-IPC [3 for garment simulation. However, the simulator fails when the animation inputs, SMPL. X [5 meshes, have a small degree of self- penetraaen,, which can occur as they are practically obtained via estimation (c.g.. fitting the parametric model to input observations). This causes C-IPC to fail to resolve the collision in the Continuous Collision Detection (CCD) stage [31] thus the driving body meshes are manually adjusted to avoid simulation failures in this method. In terms of appearance, PhysAvaaar relies on mesh-basee rendering, which limits its ability to capture fine-grained appearance details.
In this paper, we propose MPMAvatar, framework for creating 3D human avatars from multi-view videos that enables (1) physically accurate and robust animation especially for loose garments, as well as (2) high-fidelity rendering based on 3D Gaussian Splatting (22). For garment dynamics modeling, our key idea is to use a Material Point Method [17] (MPM)-basee simulator, which is capable of simulating objects under complex contacts without failure cases via feedforward velocity projection. However, directly adopting the existing MPM simulator [17, mainly used for general object dynamics modeling, introduces two challenges for effective garment dynamics modeling for our avatar.
First, garments typically exhibit a codimensional manifold structure, and their physical properties vary significantly depending on the direction (e.g.. in-manifold vs. normal directions). Second, the existing collision handling algorithm 117 of MPM is mainly designed for colliders that are analytically represented using level sets (e.g., simple geometries such as spheres), which is not applicable in our scenario where the collider SMPL-- [51) body mesh underlying the garments.
To address these issues, we tailor our MPM simulator by (1) adopting an anisotropic constitutive model [16) to better model the manifold-dependent dynamics of garments, and (2) by introducing a novel collision handling algorithm that can handle more general colliders represented as meshes. We combine this dynamics modeling scheme with our canonical avatar that can be rendered using 3D Gaussian Splatting with quasi shadwwig enabling high-fidelity rendering for physically realistic animations.
In our experiments, we demonstrate that MVMAvatar outperforms the current state-of-the-art physics- based avatar 78 in terms of (1) dynamics modeling accuracy, (2) rendering accuracy, and (3) simulation robustness and efficiency. Additionally, we present a novel application in which our avatar generalizes to novel scene interactions zero-shot manner - demonstrating the generalizability of our physics simulation-based dynamics modeling. Our code will be also publicly available to allow full reproducibility.
Overall, our contributions can be summarized as follows:
We present MPMAvatar, a novel framework for creating 3D clothed human avatars from multi-view videos. Our avatar supports physically realistic and robust animations, especially for loose garments, as well as high-quality rendering.
We present an MPM [ basedd simulation method carefully tailored for effective garment dynamics modeling, based on anisotropic constitutive model []6] and mesh-based collider handling.
We empirically demonstrate that MPMAvatar achieves superior performance than the ex- isting SOTA physics-based avatar (PhysAvatar (78) in terms of (1) dynamics modeling accuracy, (2) rendering accuracy, and also (3) simulation robustness and efficiency.
We show that our physics-based simulation method is zero-shol generalizable to interactions with an unseen external object. To the best of our knowledge, MPMAvatar is the first to empirically demonstrate this ability.
2

---

2 Related Work
Physics-based simulation methods. Physics-based simulation methods 46 41 21
30 31
17
simulate object dynamics by solving the governing differential equations (e.g,, conservation of momentum). Position-Based Dynamics (PBD) [46 41 directly manipulates the positions of particles to satisfy physical constraints. While is fast and stable, is limited in accurately modeling complex material behaviors due to the lack of explicit physical modeling (c.g., plasticity). Variational integrators [2T]. such as IPC [30] and its cloth-specific extension C-IPC [31], formulates simulation as the iterative minimization of a total energy potential, which is possibly augmented with barrier terms to robustly handle contact and friction. While C-IPC is effective for simulating cloth and was thus adopted in the current SOTA physics-based avatar [8] it fails to resolve collisions for noisy colliders during Continuous Collision Detection (CCD), as discussed in Sec. Some recent works (37 33 explore differentiable cloth simulation for inverse problems. However, they primarily operate in simplified settings without complex colliders, making them less applicable to collision- heavy scenarios like clothed human avatars. Material Point Method (MPM) (17] simulates continuum materials by hybridly representing them with Lagrangian particles and an Eulerian grid, and is known for robust handling of large deformations and self-collisions. Due these advantages crucial for simulation stability and accuracy, our work adopts MPM, while further tailoring its simulation scheme for more effective garment dynamics modeling for our avatar. In a later section (Sec. ( we demonstrate that our simulation method outperforms C-IPC used in the SOTA physics-based avatar in both dynamics modeling accuracy and robustness.
Learning-based simulation methods. Some existing methods g use neural network-based simulators trained large datasets to implicitly encode physics. However, they have limited generalizability beyond the training dynamics and cannot guaraniee physically plausible deformations, as object dynamies are predicted by the network. As discussed in (8). this leads to failure cases when objects move faster or more erratically than in the training data. Thus, directly following the motivation of the recent physics-based avatar (78), we adopt physics-based simulation. In Sec. [ we additionally show that our MPM-basee simulation method is zero-shot generalizable to novel scene interactions which is not achievable through learning- sarning-based simulation.
Dynamic avatar reconstruction from visual inputs. Learning 3D human avatars from visual inputs has been an active area of research in computer vision and graphics. Earlier methods rely on (1) mesh representation [51 [66 71
10 24, 22 23 62], which is computationally efficient but struggles
to capture details for rendering, or (2) implicif representation [3 52
68
77
(33
[8] 27 28 which
captures fregufrry geometry and appearance details but complicates dynamics modeling due to the absence of explicit geometry. Most recent avatar reconstruction methods (00
36
45
49 79
use 3D Gaussian Splats (3DGS) (22). which address these limitations by enabling high-fidelity rendering while using explicit representation, facilitating more effective dynamics modeling. For modeling avatar dynamics, existing methods often represent garment motions using piecewise linear transfornations | 48 However, these approaches are limited in their ability to capture complex deformations and tend to overfit to the training motion data, as discussed in 78]. To enable more realistic and physically accurate animations, a few works (58 69 78 have integrated simulators such as XPBD [4] 58)
61
[2
47 73 or pose-dependent geometric correctives 35 39 65)
C-IPC 31 70) However, they require manual parameter search, or manual adjustment for body mesh colliders to avoid simulation failures, discussed in Sec. A related line of work, such as DiffAvatar [34), focuses on generating physically plausible garments from static 3D scans using differentiable simulation. However, their formulation omits appearance modeling and is tailored for scan-based asset preparation, rather than dynamic avatar reconstruction from visual observations. Note that some existing works on garment-only modeling (not the clothed avatars as in our work) adopt a neural network- based simulator [8] (55). but they do not guarantee physically based behavior and are known to have limited generalizability compared to physics- based simulators (78).
3 Preliminaries: Material Point Method (MPM)
Material Point Method (MPM) 17 models an object as a continuum, enabling the simulation of diverse materials including solids, liquids, and gases. MPM advances the simulation by representing the continuum using both Lagrangian particles and an Fulerian grid and solving two governing cquations: (a) conservation of mass and (b) conservation of momentum:
3

---

Dp (a) DE + v
Dv P Dt. v
(6)
(I)
where p and v are density and velocity, respectively. D denotes the material derivative, o is the cauchy stress tensor, and g is the gravitational acceleration. During simulation, physical quantities such as mass and momentum are two-way transferred between the particles and the grid. Here, mass conservation is straightforwardly achieved due to the invariant mass carried by Lagrangian particles, while momentum conservation is performed on the Eulerian grid to efficiently approximate the spatial derivatives. When solving Eq. [Tp for updating momentum on the grid, computing the cauchy stress tensor is the key in capturing the material behavior. More specifically, it is defined as der(FF du F" where F is the deformation gradient that linearly approximates local deformations, and the strain-energy density function v., which depends F, quantifies the energy stored through the deformation. For defining v various constitutive models [59] 25 60 have been developed to define v, such that it can effectively model various material behaviors (c.g. for jelly, snow, sand, and fluids).
Dr
4 MPMAvatar: Photorealistic Avatars with Physics-Based Dynamics
In this section, we present MPMAvatar, a framework that learns 3D human avatars from multi-view videos that support (1) physically accurate and robust animation and (2) high-quality rendering. In the following sections, we first describe our avatar representation (Sec. 4.1p. We then present physics-basee based approach to modeling avatar dynamics, which is particularly effective for realistically animating loose garments (Sec. 4.2). Finally, we explain how the proposed physically-baser dynamic avatar can be learned from multi-view video inputs (Sec. 4.3)
4.1 Avatar Representation
To enable both physically realistic animation and high-fidelity rendering, we use a hybrid representa- tion that combines (1) mesh with physical parameters to enable physically based animation, and (2) 3D Gaussian Splats [22] for high-quality rendering. Formally, we represent the canonical geometry of an avatar with a 3D triangular mesh M₁ = (V1, F), where V1 and F are mesh vertices and faces, respectively. To model the physics-based dynamics of the geometry, the avatar is also represented with physical parameters P = (E, V, KS. p.).). It consists of material parameters used for traditional Material Point Method (MPM) 17 simulation (Young's s modulus E and Possion's 's ratio v), additional material parameters for anistropic dynamics modeling (shear stiffness and normal stiffness which will be introduced in Sec. ). density and the rest geometry parameter a.
To enable high-fidelity rendering, we represent the avatar appearance with 3D Gaussian Splats 22 9 = ) 1= Each Gaussian Splat gi is parameterized by translation vector t, E R quaternion q: RR, scale vector s; R opacity 0 R, and color c; represented by spherical harmonics [22]. Note that our Gaussian Splats are attached to the canonical avatar mesh MM, where each g, is associated with face of Mi. More specifically, following [54], we define all spatial parameters g; in the local coordinate system with respect to its associated mesh face, allowing our Gaussian Splats to naturally deform according to the underlying mesh deformations.
4.2 Physics-Bard -Bsse Dynamics Modeling
We now explain how we model the dynamics of our avatar discussed in Sec. 4.1 to achieve highly
realistic and physically grounded animations. Following the existing physics-based avatar [78].
we animate the body (non-gan) moon-gamment regions) of the avatar using Linear Blend Skinning [15] while
animating its garments driven by the underlying body motions (represented with SMPL-X [5]
meshes) via physical simulation. For the simulation, we adopt Material Point Method (MPM) 17
due to its effectiveness in modeling large deformations and robustly handling collisions (Sec.
While MPM is actively adopted in recent 3D scene simulation methods [4] 766 (70). it is mainly used for modeling the dynamics of general objects (c.g.. flower pots, elastic torus).
While the deformation gradient is also denoted by F. we allow a slight abuse of notation to remain consistent with notation conventions used in related work.

---

Diokls
Mach
Mixnnal Bictmon
FF
Valund
Vaindy
Mani
Cuiliir
Valeett
Relatic Viady
Prasect
Find Ninda Pool----P
Avatar Representation (Sec. 4.1)
Garments Dynamics Modeling (Sec. 4.2.1)
Collision Handling (Sec. 4.2.2)
Figure 2. Overview of dynamic avatar modeling. We hybridly represent canonical avatar with (1)
mesh with physical parameters for geometry and dynamics modeling, and (2) 3D Gaussian Splats (22 for
appearance modeling [Sec.441] This avatar can he animated via linear hlend skinning for non-garment regions and physical simulation for garment regions (Sec, (Sec.4.2211) with our novel collision handling algorithm (Sec. 1.2.2
Visualization key. Blue arrows indicate body grid velocities, green arrows denote garment grid velocities, and red arrows show colliding grid regions where velocity projection is applied.
In this work, we carefully tailor the simulator to achieve more effective modeling of garment dynamics in our avatar. In particular, we (1) adopt the anisotropic constitutive model [16] to better model the manifold-depe -eeeedddd dynamics of garments, and (2) introduce collision handling algorithm designed to effectively resolve garment-body collisions. In what follows, we further elaborate on these two modifications.
4.2.1 Anisotropic Dynamics Modeling for Garments
Garments typically exhibit a codimensional manifold structure, and their physical properties vary depending on For example, garments can easily stretch along in-manifold directions, but along the normal directions. To accurately model this behavior, we adopt the anisotropic constitutive model proposed by Jiang et al. [16] underlying our MPM simulator. In particular, they propose to model the strain energy for anisotropic material depending on each Lagrangian particle's 's material directions, which are approximated using Lagrangian mesh. More specifically,
deformation gradient FF at each particle is compuated as dD where D [D, D2, Da] E R3333 is the original material direction and
(di d2, ds] E 3 is the deformed material direction (see Fig. 2). Since the strain-energy density function 7 must be invariant under rotations, they apply QR-decomposition
F QR, and vb is reparameterized (F) = (R), such that:
F,R,E, = normaal (Ra3/k) + vishea RR13, R2a /?) + Vin-plane RR11 R12, R222 F,, v),
where Vnormal, dheerr and thin-plaee are functions for penalizing normal deformation, shearing, and in- plane deformation, respectively (refer to [16] and Appendix C for details). Note that this constitutive model [16] requires Lagrangian mesh representing the codimensional object to track the material directions, which had motivated our avatar representation (Sec. 4.1) based Gaussian Splats. As the official implementation the MPM solver for this constitutive model not publicly available, re-implemented it using Py Torch [50) and Warp (40), and plan release code to facilitate future research.
hybrid mesh and
4.2.2 Collision Handling
We additionally introduce collision handling algorithm designed to effectively resolve garment- body collisions.
The existing collision handling algorithm of MPM is designed be effective for an external object (i.e., a collider) represented as a dynamic level set. nutshell, MPM resolves collisions by projecting the grid velocity of the colliding region of the object onto the
Collision handling of MPM 17
5

---

collider's tangent space, preventing penetration while modeling tangential motion. Technically, this requires evaluating the velocity and normal of the collider at the Eulerian grids nearby the colliding regions, which are not strictly on the collider surface. MPM originally enables these evaluations by considering colliders whose geometry and velocity fields are analytically defined over all points x E IR' in the ambient 3D space, e.g., a sphere geometry Φf x) ||x cll where and denote center and radius, respectively. However, our collider is a SMPL-X (51) mesh representing the body underlying the garment, thus it is not trivial to directly adopt this algorithm.
Our algorithm. To address the above limitation, we introduce a simple yet effective collision handling algorithm for MPM to support colliders represented meshes. Note that, in this case, normal or velocity is defined only on the collider surface, not at all Euclidean grid nodes. To address this, our idea is to transfer these quantities to nearby grid nodes using B-spline weights, directly analogical to particle-to- grid transfer in MPM simulation. In Fig. 2 we outline our overall collision handling procedure consisting of two stages: (1) mesh- grid transfer (upper row) and (2) relative velocity projection (lower row). In the mesh-to-gun grid transfer stage, we transfer each collider face's s velocity v and normal ny to nearby grid nodes using B-Spline weights, producing the extended collider velocity vf and normal of at each grid node i. In the relative velocity projection stage, we first transform the grid velocity into the collider mesh's reference frame by subtracting the collider velocity v If the relative velocity points inward, we project out its normal component. Finally, we transform the corrected velocities back into the world frame. It is worth noting that the complexity of the existing collision handling algorithm of MPM is O(N gria), where Ngrid is the grid resolution, as it requires evaluating the level set function at all grid nodes for collision check. In contrast, the complexity of our collision handling algorithm is ((N), where N₂ is the number of the collider mesh faces. As it is usually N₂ < N (note that N₂ R 20K and N gnd & 8M in our case), our approach based on a mesh-based collider is extremely more efficient, as well as reflecting a more practical scenario.
Summary. Using our tailored MPM-based simulator, we can effectively model the anisotropic dynamics of garments under complex collisions with the underlying body meshes. Later in the experiments (Sec.5), we show that our simulation scheme leads to SOTA dynamics accuracy.
4.3 Learning from Multi-View Videos
We now explain how our avatar outlined in the previous sections can be learned from multi-view video inputs. As a preprocessing step, we first perform 3D mesh tracking on the input frames to capture dense temporal geomety correspondences, which are used to supervise the subsequent learning stages. In particular, we use the mesh tracking algorithm of the existing physics- which assumes that mesh the first frame M₁ FV1.F) is given (c.g., from off-the-shed static scene reconstruction method), and optimizes the deformed meshes at the subsequent frames (M.)-=2--T,, where M; (V., F), based on rendering loss. For more details on the tracking algorithm, we refer the reader to (7). In the following, we focus on explaining how the physical dynamics (Sec 4.3. p and appearance (Sec 4.3.2) of our avatar can be learned.
avatar work [78),
4.3.1 Physical Parameters Learning
We now explain how we learn the physical parameters used to model our garment dynamics. Given the canonical avatar mesh Mi obtained from the prior mesh tracking stage, the set of physical parameters associated with M₁ is defined as P = (E, v, 7, Ks a),, as previously discussed in Sec. 4.1 Note that E, 7, and are material parameters used for our Material Point Method (MPM) [1 simulation. Following the recent inverse physics work based on MPM (76), which found that Young's 's modulus E is the key parameter dominating the dynamic behavior and thus fixed the rest of the parameters, we also fix V, 7, K. to their default values and focus on learning the other parameters-a F,, P. and o to mitigate over wee-parameteeriztion In the following, we specifically focus on elaborating on our newly introduced parameter a, used for rest geometry modeling.
internally computes forces based on the deformation gradient relative to the object's 's rest geometry (i.e., the canonical geometry in the unstressed state, without external forces such as gravity). Note that existing approaches 76 based on MPM typically assume ideal conditions, i.c., having the initial frame correspond to undeformed rest state, limiting
Rest geometry modeling. MPM 17

---

their applicability in real-world scenarios. In our case, the canonical geometry M1 is obtained from real-world observations and therefore already deformed by gravity. To correct this, we additionally introduce a simple parameter a E [0, 1] to compensate for gravity deformation and learn the unseen rest geometry of the avatar. Formally, for each edge vector e Mı, we decompose into two components: eg. the projection of onto the gravity direction g. and e the component orthogonal to g, such that e = e, + e_. Using a, we simply model each edge erest in the rest geometry as: Crest Ty + Cg. where determines the extent to which the stretch in the gravity direction is compensated. a is optimized end-to-end along with other physical parameters via inverse physics.
Learning physical parameters. We aim to learn E, f. and a such that the simulated mesh dynamics closely model the real-world garment motions. Given the canonical mesh at the first frame Mi, we simulate its future states using our MPM-based simulator (Sec. (42) and the physical parameters P, resulting in simulated meshes (M4))=2...., I over frames [2, 1j. We then optimize the parameters E. P. and a by minimizing the veriex-to-vertex L2 loss between the simulated meshes and the tracked meshes (M,))=2,...." capturing geometric dynamics from the input video. Following (78), we perform this optimization using a finite-difference approach. For more implementation details, we kindly refer the reader to Appendix B.I
4.3.2 Appearance Learning
For learning the appearance of our avatar, we optimize the parameters of 3D Gaussian Splats G defined on the canonical geometry M₁ at = 1. As discussed in Sec. 4.1 the spatial parameters of g: are defined in the local coordinate system with respect to its parent mesh face, allowing it to naturally deform according to the underlying mesh deformations. Using this, we first deform G to the other frames at E f2...., T} based on the tracked mesh deformations ( (M.):=I....?" and render them across all input views and timesteps. We then compute the loss by measuring the photometric discrepancy between the rendered images and the ground-truth images across all training frames and views. We finally optimize the parameters of G via gradient descent to minimize this loss.
Note that the preprocessing mesh tracking stage, which we adopt from 778). also employs 3D Gaussian Splats as surrogate representation to incorporate rendering loss for optimizing meshes, However, it learns the Gaussian colors independently for each frame" In contrast, we learn from all input frames and views to better capture regions that are occluded in some views but visible in others, while still leveraging the previously learned surrogate Gaussian Splats for parameter initialization. Please refer to Appendix B.2 for the details.
Quasi-shadowing. When rendering our avatar using G via 3D Gaussian Splatting, we additionally apply quasi-shadowing to enhance rendering fidelity. Following the prior work (2] we model self- shadowing by leveraging neural network trained on ambient occlusion features extracted from the mesh in our hybrid avatar representation. Specifically, we modulate the color of each Gaussian Splat gi using shading scalar up E [o, 1]. predicted by the network, to obtain the final color.
5
Experiments
5.1 Experimental Setup
For experimental comparisons, we mainly follow the setup used in the state-of-the-art physics-based avatar work (PhysAvatar (8] to perform fair comparisons.
Dataset. We perform our main evaluations on (1) ActorsHQ [14]. In particular, we select four subjects used in (78): two characters in loose dresses and two characters in two-piece outfits. For training each subject, we use 24 frames with large cloth dynamics for physical parameter learning and 200 frames for appearance learning. For testing, we use 200 unseen frames per subject. Whereas the existing work 770 only uses ActorsHQ for evaluation, we additionally include four sequences from (2) 4D-DRESS [66] dataset, to perform more extensive comparisons. We use two subjects in tops and skirts and two in tops and tight jeans. For training. we use 11 frames for physical parameter learning and 100 frames for appearance learning, while testing was carried out on 100 unseen frames.
Note that the existing work (78 with this mesh tracking method discards the surrogate Gaussian Splats and resorts to mesh based rendering for its avatar.
7

---

PhysAvatar [78]
Ours
Ground Truth
PhysAvatar [78]
Ground Truth
(a) Results on Actors [14]
(b) Results on 4D-DRESS [66]
Figure Qualitative results on frames the ActorsHQ [14] and 4D-DRESS (66 datasets. Our method
outperforms PhysAvatar (78) in appearance by rendering sharper. less blurred textures with finer detail and in
geometry by recovering folds and wrinkles that more closely match the ground truth.
Table Quantitative comparisons on ActorsHQ [14] and 4D-DRESS [66] datasets. Bold indicates the best scores, and underline indicates the second best scores. Our proposed method achieves the best results across all geometry and appearance metrics both benchmarks.
Geometry
Appearance PSNR 1
Method
CD (x10%)
F-Score 1
LPIPS $
SSIM
(a) Results ActorsHQ [4] dataset.
65
0.055
TAVA
12
0.66
92.1
0.051
29.6
0.962
GS-Avatar
PhysAvatar
73
0.55
92.9
0.035
30.2
0.957
MI'Mvvatar (Ours)
0.42
95.7
0.033
32.0
0.963
F
Anisotropy
6.24
90.1
0.039
28.7
0.957
Physics
H
Shadow
0.033
31.8
0.963
Results 4D-DRESS (66)
PhysAvatar 78
0.37
MPMAvatar (Ours)
0.33
97.2
0.018
34.1
0.977
avatar reconstruction methods: ARAH (65] TAVA (2). GS-Avatar [2], and PhysAvatar [78]. Here, PhysAvatar is the most related baseline to ours, as it the state-of-the-art work on physics-based avatar. As we already discussed in Sec. PhysAvatar's simulator fails when driving body mesh colliders have self-penetrations. While the original work manually adjusted the body meshes to avoid simulation failures, these meshes are not publicly available after requests. Therefore, we minimally excluded the faces of the body mesh collider during collision check to prevent their simulation failure.
Baselines. We use the same baselines as which are four
Evaluation Metries. To assess our dynamics modeling accuracy, compute Chamfer Distance (CD) 44 and F-Score (64] between the simulated and the ground truth meshes. For F-Score, set the threshold 7 to 0.001. For evaluating our rendering accuracy, we measure Learned Perceptual Ratio (PSNR), and Structural Similarity Index Measure (SSIM) between the rendered and the ground truth images.
Image Patch Similarity (LPIISS) (5), Peak Signal-to-
5.2 Experimental Comparisons
Simulation and rendering accuracy. Tab. aa) (Rows A-E) and Tab. [bb show our main com- parison results on the ActorsHQ [4] and 4D-DRESS (66 datasets, respectively. Ours achieves the best results across all geometry metries on both benchmarks, validating that the animated geometry using our MPM [17 based simulation models the most accurate avatar dynamics. Ours also achieves

---

Anisotropy
Physies
Shadow
Ours
Ground Truth
Anisotropy
Ours
Ground Truth
(a) Full ablation study results
(b) Failure case -Anisotropy
Figure 4. Qualitative ablation study results.
the best results across all appearance metrics, which shows that 3D Gaussian Splatting [22 with quadi-sh with the ambient occlusion prior extracted from mesh can be an effective ren- dering scheme. In Fig. 3 we also show the qualitative comparisons against our most competitive
baseline: AhysAv 78 Our method captures complex cloth deformations (e.g., subtle wrinkles)
more closely to the ground truth, and our rendering based on 3D Gaussian Splatting [22] can more effectively capture high-free nigh-frequency appearance details (e.g., complex cloth patterns) than PhysAvatar based on mesh-bases rendering.
Robustness and efficiency. In Tab 2 we show the simulation success rate and per-frame simulation time the ActorsHQ [14] benchmark compared to PhysAvatar (78.. The success rate measures the average ratio successfully simulated frames to the total number of evaluation frames, where we mark frame a failure if the simulation does not terminate within 20 hours for the single frame. For this evaluation, we disabled the manual relaxation of the collision check for Phys. which we originally applied to prevent its simulation failures (Sec. 5.1). The success rate of Phystvatar is 37.6% while ours is 100%, validating that ours exhibit significantly higher robustness.
For simulation time, report the average per- per-frame simulation time the test sequence. Notably, we re-applied the manual relaxation of the collision check in PhysAvatar, as its simulation fails to terminate without this adjustment. As shown in the table, our method achieves simulation time of 1.1 seconds per frame, compared to 170.0 seconds for Phys/vatar, demonstrating significantly better efficiency. Note that PhysAvatar (78 's iterative solver (C-IPC (1) takes a long time converge for resolving complex collisions, whereas our feed-fore forward MPM simulator runs much faster.
Table Simulation robustness and efficiency comparisons with PhysAvatar scores. All scores are evaluated on the Actors-IIQ dataset 14]
78
Bold indicates the best
Method
Success Rate (%)
Simulation Time (s)
PhysAvatar
78
37.6
170.0
B
MPMAvalar (Ours)
100.0
L.1
5.3
Ablation Study
Anisotropic constitutive model. Anisotropy denotes our method variant which does not an anisotropic constitutive model [16]. As shown in Tab. [ (Row F), this variant results in large degradation in dynamics modeling accuracy, as it does effectively model the manifold -eeendeen
behaviors of cloths. Few cases even have severe tearing artifacts, as shown in Fig. 4p.
Physics denotes our method variant where all the physical parameters are fixed to their default values without learning. As shown in Tab. [ (Row G), this results in suboptimal dynamics modeling accuracy, highlighting the importance of our inverse physics. In Fig. we visually show that this variant leads to less accurate estimation of cloth deformations.
Physical parameters learning.
Quasi-shadowing. Shadow denotes our method variant where quasi-shadowing is not used for rendering. As shown Tab. (Row H), this results degradation in PSNR. In Fig. 4 we qualitatively show that its rendering result exhibits significantly less realism than ours with shadowing.

---

5,4 Application: Zero-shot Scene Interaction
As an additional application, we showcase that our physics- based simulator is zero zero-shot generalizable to interactions with external objects unseen during training. In the right subfigures of Fig. [ our avatar garments are naturally deformed as interacting with a chair or sand. This generalizability can be achieved as our physics-based simulator explicitly leverages the physics prior, unlike in learning-base based simulators [8 [7] known to be less effective in modeling unscen dynamics, We also note that, owing to the versatility of MPM [17] in handling diverse materials, our framework supports interactions with deformable particles (c.g., sand), while simulators like C-IPC [31] are limited to mesh-based simulations. Please see Appendix A for more interaction examples of ours.
Conclusion
We presented MPMAvatar, framework for creating 3D human avatars from multi-view videos that supports (1) physically accurate and robust animation, as well as (2) high-fidelity rendering. Our Gaussian Splat-based avatar is animated based carefully tailored MPM- -asdd simulator designed for effective garment dynamics modeling. enabling physically grounded animations.
Limitations. Although our avatar outperformed the existing state-of-the-art physics-based avatar method (78) in both appearance and geometry, it does not support relighting as in (78). Also, for animation, we directly followed [78] and modeled the dynamics of non-garment regions via linear blend skinning, but this can be further improved, e.g., by using strand-based simulation for hair. We refer Appendix D for more detailed discussion of limitations and future works.
7
Acknowledgement
This work was supported by NST grant (CRC21011, MSIT), ITTP grant (RS-2019-11190075, RS- 2023-00228996, RS- RSS-002400049944, RS-2025-25443318, RS-2025-25441313, MSIT) and KOCCA grant (RS-2024-00442308, MCST)
References
[1] Rameen Ahdal, Wang Yifan, Zifan Shi, Yinghao Xu, Ryan Po, Zhengfci Kuang, Qifeng Chen, Dit-Yan Yeung, and Gordon Wetzstein. Gaussian shell maps for efficient 3d human generation. In CVPR, 2024.
[2] Timur Bagautdinov, Chenglei Wu, Tomas Simon, Fabian Prada, Takaaki Shiratori. Shih-En Wei, Weipeng Xu, Yaser Sheikh, and Jason Saragih. mriving-signal aware full- bolyybody avatars, ACM TOG, 2021.
[3] Alexander Bergman, Petr Kellnhofer. Wang Yifan, Eric Chan, David Lindell, and Gordon Wetzstein. Generative neural articulated radiance fields. NewrIPS, 2022.
[4] Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. Gaussian-informed continuum for physical property identification and simulation. NeurIPS, 2024.
[5] Yue Chen, Xuan Wang. Xingyu Chen, Qi Zhang. Xiaoyu Li. Yu Guo, Jue Wang, and Fei Wang. Uv volumes for real-time rendering of editable free-view human performance. CVPR, 2023.
[6] Zijian Dong. Xu Chen, Jinlong Yang, Michael Black, Otmar Hilliges, and Andreas Geiger. Ag3d: Learning to generate 3d avatars from 2d image collections. In ICCV, 2023.
[7] Artur Grigorev, Giorgio Becherini, Michael Black, Otmar Hilliges, and Bernhard Thomaszewski. Contour- craft: Learning to resolve intersections in neural multi-garment simulations. 2024.
[8] Artur Grigorev, Michael Black, and Otmar Hilliges. Hood: Hierarchical graphs for generalized modelling of clothing dynamics. CVPR, 2023.
[9] Chen Guo, Chen, Jie Song, and Otmar Hilliges. Human performance capture from monocular video the wild. In 3DV, 2021.
[10] Mare Habermann, Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Mop Moll, and Christian Theobalt. real-time deep dynamic characters. ACM TOG, 2021.
10

---

[11] Mare Habermann, Weipeng Xu, Michael Zollhofer, Gerard Pons-Moll, and Christian Theobalt. Deepcap:
Monocular human performance capture using weak supervision. In CVPR, 2020.
[12] Liangxiao Hu, Hongwen Zhang, Yuxiang Zhang, Boyao Zhou, Boning Liu, Shengping Zhang, and Liqiang Nie. Gaussianavatar: Towards realistic human avatar modeling from a single video via animatable 3d gaussians. In CVPR, 2024.
[13] Tao Hu, Tao Yu, Zerong Zheng, He Zhang, Yebin Liu, and Matthias Zwicker. Hytr: Hybrid volumetric- textural rendering for human avatars. In 3DV, 2022.
[14] Mustafa Işik, Martin Rünz, Markos Georgopoulos, Taras Khakhulin, Jonathan Starck, Lourdes Agapito, and Matthias Nießner. Humanrf: High-fideling neural radiance fields for humans in motion. ACM TOG. 2023.
[15] Alec Jacobson, Zhigang Deng, Ladislav Kavan, and John P Lewis. Skinning: Real-time shape deformation (full text not available). In ACM SIGGRAPH 2014 Courses. 2014.
[16] Chenfanfu Jiang, Theodore Gast, and Joseph Teran. Anisotropic elastoplasticity for cloth, knit and hair frictional contact. ACM TOG, 2017.
[17] Chenfanfu Jiang. Craig Schroeder, Joseph Teran, Alexey Stomakhin, and Andrew Selle. The material point method for simulating continuum materials. In ACM SIGGRAPH 2016 Courses. 2016.
[18] Tianjian Jiang, Xu Chen, Jie Song, and Otmar Hilliges. Instantavatar: Learning avatars from monocular video in 60 seconds. In CVPR, 2023.
[19] Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, and Yuexin Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces. In CVPR, 2024.
[20] HyunJun Jung, Nikolas Brasch, Jifei Song, Eduardo Perez-Pellitero, Yiren Zhou, Zhihao Li, Nassir Navab, and Benjamin Busam. Deformable 3d gaussian splatting for animatable human avatars. arXiv preprint arXiv:2312.15059, 2023.
[21] Couro Kane, Jerrold Marsden, Michael Ortiz, and Matthew West. Variational integrators and the newmark algorithm for conservative and dissipative mechanical systems. I/MME, 2000.
[22] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023.
[23] Minje Kim and Tae-Kyun Kim. Bitt: Bi-directional texture reconstruction of interacting two hands from a single image. In CVPR, 2024.
[24] Minje Kim and Tae- Kyun Kim. Sthand: Super-resolving hand images and 3d shapes via view/pose-aware nueral image representations and explicit 3d meshes. In NewrIPS, 2025.
[25] Gergely Klár, Theodore Gast, Andre Pradhana, Chuyuan Fu, Craig Schroeder, Chenfanfu Jiang, and Joseph Teran. Drucker Drucker-prager prager clastoplasticity for sand animation. ACM TOG, 2016.
[26] Youngjoong Kwon, Baole Fang, Yixing Lu, Haoye Dong, Cheng Zhang, Francisco Vicente Carrasco, Albert Mosella-Montoro, Jianjin Xu, Shingo Takagi, Dacil Kim, Aayush Prakash, and Fernando De Torre. Generalizable human gaussians for sparse view synthesis. In ECCV, 2024.
[27] Jihyun Lee, Junbong Jang, Donghwan Kim, Minhyuk Sung, and Tae-Kyun Kim. Fourierhandflow: Neural 4d hand representation using fourier query flow. In NewrlPS, 2023.
[28] Jihyun Lee, Minhyuk Sung, Honggyu Choi, and Tae-Kyun Kyun Kim. Im2hands: Learning attentive implicit representation of interacting two-hand shapes. In CVPR, 2023.
[29] Jihyun Lee, Weipeng Xu, Alexander Richard, Shih-En Wei, Shunsuke Saito, Shaojic Bai, Te-Li Wang. Minhyuk Sung, Tae-Kyun Kim, and Jason Saragih. Rewind: Real-time egocentric whole-body motion diffusion with exemplar- based identity conditioning. In CVPR, 2025.
[30] Minchen Li, Zachary Ferguson, Teseo Schneider, Timothy R Langlois. Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M Kaufman. Incremental potential contact: intersection-and inversion-free, lagge-eeformation dynamics. ACM TOG, 2020.
[31] Minchen Li, Danny M Kaufman, and Chentanfu Jiang. Codimensional incremental potential contact. ACM TOG, 2020.

---

[32] Ruilong Li. Julian Tanke, Minh Vo, Michael Zollhöfer, Jürgen Gall, Angjoo Kanazawa, and Christoph Lassner. Tava: Template-free animatable volumetric actors. In ECCV, 2022.
[33] Yifei Li, Tao Du, Kui Wu, Jie Xu, and Wojeiech Matusik. Diffcloth: Differentiable cloth simulation with dry frictional contact. ACM TOG, 2022.
[34] Yifci Li, Hsiao yu Chen, Egor Larionov, Nikolaos Sarañinns,, Wojeiech Matusik, and Tuur Stuyck. Diffavatar: Simulation-ready garment optimization with diffcrentiable simulation. In CVPR, 2024.
[35] Zhe Li, Zerong Zheng, Yuxiao Liu, Boyao Zhou, and Yebin Liu. Posevocab: Learning joint-structured pose embeddings for human avatar modeling. In SIGGRAPH, 2023.
[36] Zhe Li, Zerong Zheng. Lizhen Wang. and Yebin Liu. Animatable gaussians: Learning pose-dependent gaussian maps for high-fidelity human avatar modeling, In CVPR, 2024.
[37] Junbang Liang, Ming C. Lin, and Vladlen Koltun. Differentiable cloth simulation for inverse problems. In NeurIPS, 2019.
[38] Lingjic Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Thcobalt. Neural actor: Neural free-view synthesis of human actors with pose control. ACM TOG, 2021.
[39] Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Hyeongwoo Kim, Florian Bernard, Mare Habermann, Wenping Wang. and Christian Theobalt. Neural rendering and reenactment of human actor videos. ACM TOG, 2019.
[40] Miles Macklin. Warp: A high- performance python framework for gpu simulation and graphics. https
7 //github www.///wiia///eooooooeeeeieeeeee March 2022. NVIDIA GPU Technology Conference (GTC).
[41] Miles Macklin, Matthias Mäller, and Nuttapong Chentanez. Xpbd: position-hased simulation of compliant constrained dynamics. In MIG, 2016.
[42] Naureen Mahmood, Nima Ghorbani, Nikolaus F Troje, Gerard Pons-Moll, and Michael J Black. Amass: Archive of motion capture surface shapes. In ICCV, 2019.
[43] Fanqing Meng, Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang. Yu Cheng. Dianqi Li, Qiao, and Ping Luo. Towards world simulator: Crafting physical commonsense-basee benchmark for video generation. In ICMI., 2025.
[44] Lars Mescheder, Michael Occhsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In CVPR, 2019.
[45] Arthur Moreau, Jifei Song, Helisa Dhamo, Richard Shaw, Yiren Zhou, and Eduardo Pérez-Pellitero. Human gaussian splatting: Real-time rendering of animatable avatars. In CVPR, 2024.
[46] Matthias Müller, Bruno Heidelberger, Marcus Hennix, and John Ratcliff. Position based dynamics. J. Vis. Comun. Image Represent., 2007.
[47] Atsuhiro Noguchi, Xiao Sun, Stephen Lin, and Tatsuya Harada. Neural articulated radiance field. In ICCV, 2021.
[48] Atsuhiro Noguchi, Xiao Sun, Stephen Lin, and Tatsuya Harada. Unsupervised learning of efficient geometry-aware neural articulated representations. In ECCV, 2022.
[49] Haokai Pang, Heming Zhu. Adam Kortylewski, Christian Theobalt. and Mare Habermann. Ash: Animatable gaussian splats for efficient and photoreal human rendering. In CVPR, 2024.
[50] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang. Zach Vitoto, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: an imperative style, performmree deep learning library. In NeurIPS, 2019.
[51] Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed A. A. Osman. Dimitrios Tzionas, and Michael J. Black. Expressive body capture: 3d hands, face, and body from single image. In CVPR, 2019.
[52] Sida Peng. Junting Dong Qianqian Wang. Shangzhan Zhang, Qing Shuai, Xiaowei Zhou, and Hujun Bao. Animatable neural radiance fields for modeling dynamic human bodies. In ICCV, 2021.
12

---

[53] Sida Peng. Yuanqing Zhang, Yinghao Xu, Qianqian Wang. Qing Shuai, Hujun Bao, and Xiaowei Zhou. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans. CVPR, 2021.
[54] Shenhan Qian, Tobias Kirschstein, I. Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nießner. Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians. In CVPR, 2024.
[55] Boxiang Rong, Artur Grigorev, Wenbo Wang. Michael Black, Bernhard Thomaszewski, Christina Tsalicoglou, and Otmar Hilliges. Gaussian garments: Reconstructing simulation-ready clothing with photorealistic appearance from multi-view video. In 3DV, 2024.
[56] Yu Rong. Takaaki Shiratori, and Hanbyul Joo. Frankmocap: A monocular 3d whole-body pose estimation system via regression and integration. In ICCV, 2021.
[57] Shunsuke Saito. Gabriel Schwartz, Tomas Simon, Junxuan Li, and Giljoo Nam. Relightable gaussian codec avatars. In CVPR, 2024.
[58] Shota Sasaki, Jane Wu, and Ko Nishino. Pbdyg: Position based dynamic gaussians for motion- aware clothed human avatars, arXiv preprint arXiv:2412.0444 2024.
[59] Alexey Stomakhin, Russell Howes, Craig A Schroeder, and Joseph M Teran. Energetically consistent invertible elasticity. In SCA, 2012.
[60] Alexey Stomakhin, Craig Schroeder, Lawrence Chaepplpph Teran, and Andrew Selle. A material point method for snow simulation. ACM TOG, 2013.
[61] Shih- Yang Su, Frank Yu, Michael Zollhöfer, and Helge Rhodin. A-nerf: Articulated neural radiance fields for learning human shape, appearance, and pose. NeurIPS, 2021.
[62] Jiaze Sun, Zhixiang Chen, and Tae-Kyun Kim. Mapconnet: Self-supervised 3d pose transfer with mesh and point contrastive learning. In ICCV, 2023.
[63] Jiapeng Tang, Davide Davoli, Tobias Kirschstein, Liam Schoneveld, and Mathhas Niessner. Gaf: Gaussian avatar reconstruction from monocular videos via multi-view diffusion. In CVPR, 2025.
[64] Maxim Tatarchenko, Stephan R Richter, René Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox. What do single-view 3d reconstruction networks learn? In CVPR, 2019.
[65] Shaofei Wang. Katja Schwarz, Andreas Geiger, and Siyu Tang. Arah: Animatable volume rendering of articulated human sdfs. In ECCV, 2022.
[66] Wenbo Wang, Hsuan-I Ho, Chen Guo, Boxiang Rong, Artur Grigorev, Jie Song, Juan Jose Zarate, and Otmar Hilliges. 4d-dress: A 4d dataset of real-world human clothing with semantic annotations. In CVPR, 2024.
[67] Zhou Wang. Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. TIP. 2004.
[68] Chung- Yi Weng. Brian Curless, Pratul P Srinivasan, Jonathan T Barron, and Ira Kemelmacher Shlizerman. Humannerf: Free-viewpoint rendering of moving people from monocular video. In CVPR, 2022.
[69] Donglai Xiang, Timur Bagautdinov. Tuur Stuyck, Fabian Prada, Javier Romero, Weipeng Xu. Shunsuke Saito, Jingfan Guo, Breannan Smith, Takaaki Shiratori, et al. Dressing avatars: Deep photorealistic appearance for physically simulated clothing, ACM TOG, 2022.
[70] Tianyi Xie, Zeshun Zong, Yuxing Qin, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics-integrated 3d gaussians for generative dynamics. In CVPR, 2024.
[71] Weipeng Xu, Avishek Chatterjee, Michael Zollhöfer, Helge Rhodin, Dushyant Mehta, Hans-Peter Seidel, and Christian Theobalt. Monoperfcap: Human performance capture from monocular video. ACM TOG, 2018.
[72] Yinghao Xu, Wang Yifan, Alexander W Bergman, Menglei Chai, Bolei Zhou, and Gordon Wetzstein. Efficient 3d articulated human generation with layered surface volumes. In 3DV, 2024.
[73] Yuelang Xu, Benwang Chen, Zhe Li. Hongwen Zhang, Lizhen Wang. Zerong Zheng, and Yebin Liu. Gaussian head avatar: Ultra high-fidelin head avatar via dynamic gaussians. In CVPR, 2024.
[74] Youyi Zhan, Tianjia Shao, Yin Yang, and Kun Zhou. Real-time high-fidelity gaussian human avatars with position-base based interpolation of spatially distributed mlps. In CVPR, 2025.
13

---

[75] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable elfectiveness of deep features perceptual metric. In CVPR. 2018.
[76] Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng, Changxi Zheng. Noah Snavely, Jiajun Wu, and William T Freeman. Physdreamer: Physics-based interaction with 3d objects via video generation. In ECCV, 2024.
[77] Fuqiang Zhao, Wei Yang. Jiakai Zhang, Pei Lin, Yingliang Zhang, Jingyi Yu, and Lan Xu. Humannerf: Efficiently generated human radiance field from sparse inputs. In CVPR, 2022.
[78] Yang Zheng, Qingqing Zhao, Guandao Yang. Wang Yifan, Donglai Xiang, Florian Dubost, Dmitry Lagun, Thabo Beeler, Federico Tombari, Leonidas Guibas, et al. Physavatar: Learning the physics of dressed 3d avatars from visual observations. In ECCV, 2024.
[79] Wojeiech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollhöfer, Justus Thies, and Javier Romero. Drivable 3d gaussian avatars. In 3DV, 2025.
14

---

A Additional Results
In the supplementary video (available on the project page), we show additional qualitative results of our experiments.
Qualitative Comparisons (Sec, A.1). We present our video results in comparison with PhysA- vatar 778., state-of- state-of-the-art physics-based avatar method. Our approach consistently achieves more accurate garment dynamics and higher rendering quality.
Additional Qualitative Results (Sec. A.2). We also include additional qualitative results of our method on (1) novel pose driving and (2) zero-shot scene interactions. For novel pose driving, our method accurately models garment deformation driven by motions from the AMASS [42] dataset, which contains relatively more dynamic motions than our training motions in 04 66]. For novel scene interactions, our method models plausible interactions between garments and a variety of materials (e.g., cushion, sand), well as mesh-based colliders (e.g., chair, rotor) that were unseen during training. This generalization is attributed to (1) the versatility of MPM 17 and (2) our effective mesh-based collision handling method.
Ablation Results (Sec. A.3) In the supplementary video, we additionally validate each of the key components of our method: (1) the constitutive model for anisotropic elastoplasticity [16], (2) physical parameter learning, (3) quasi-shadowing, and (4) rest-geometry modeling. In the fourth ablation, we directly use the canonical geometry as the rest geometry. Specifically, we optimize only Young's modulus E and density p. while keeping the rest geometry parameter fixed at 1. Each component is shown to be critical for accurate dynamics and appearance modeling.
Quantitative Evaluation of Physical Plausibility (Sec. A.4). While we focuses on evaluating geometric and appearance fidelity which are the standard metrics in recent physics- based avatar methods 78 69]. the physical plausibility and contact accuracy are also important aspects to assess. However, quantitatively measuring these properties is challenging due to the lack of ground-truth physical annotations in real-world RGB datasets [4 66
Nevertheless, to provide additional insights, we report two metrics that we find to reflect physical plausibility and contact quality in practice. First, we measure the average cloth-body Penetration Depth, defined as the mean of max(0, -d) where d is the signed distance from each cloth vertex to the SMPL-X [5T] body. As shown in Tab. 3 our method significantly reduces penetration depth over 6x lower than Phys.wwttar (78), indicating better contact handling.
For the second metric, we adopt the Key Physical Phenomena Detection metric proposed in Phy- GenBench [43. which computes a plausibility score using a VLM based on how well a given video aligns with physical plausibility prompts. In Tab. 3 we show that our method achieves plausibility score closer to the ground-trui -truh upper bound than PhysAvatar (78), indicating better alignment with the physical plausibility prompts.
Table 3. Quantitative evaluation of physical plausibility and contact quality. To complement standard geometry and appearance metrics, we additionally report metrics that we find to reflect physical plausibility and contact behavior in practice. Specifically, we measure the average penetration depth (mm) and the Key Physical Phenomena Detection score (43 on the ActorsHQ [14 dataset. Our method achieves significantly lower penetration and a higher plausibility score compared to Phys/vatar (78). indicating improved contact handling and physical realism.
Method
Penetration Depch (mm)
Key Physical Phenomena Detection
A
PhysAvatar 78
0.294
1.78
MPMAvatar (Ours)
0.047
188
Ground Truth
1.86
Additional Ablation Results on Hyperparameters (Sec. A.5) To demonstrate that our pipeline remains robust across different simulation setups, we conducted an additional ablation study on the
ActorsHQ [4 benchmark, where we evaluated the framework's 's performance while varying key hyperparameters. In Tab. 4 we observe that variations in (1) time substeps (Rows C-D), (2) physical parameter initialization (Rows E-I), and (3) mesh triangle numbers (Rows J), do not significantly affect the final performance; notably, all variants outperform PhysAvatar 78 by clear margin across all metrics. Note that for grid resolution, the grid and particle resolutions should be roughly aligned to enable stable momentum transfer between the two representations during MPM simulation, which is also a common convention adopted in many existing MPM-based methods 4 70). Therefore, we omitted further ablation this aspect.
15

---

Table 4. Ablation study simulation hyperparameters. To examine the robustness of our pipeline,
ablate key simulation hyperparameters on the ActorsHQ [14] dataset. Specifically, vary (1) the numher of time substeps (Rows C-D), testing half and double our default value (LV 400). (2) initialization physical parameters and E (Rows E-1). including 2x and 0.5x our defaults (p = 1.0, E = 100) as well as random initialization within plausible ranges. and (3) the number of mesh triangles (Row J). reducing it to one quarter of the original resolution. Across all variants, method consistently outperforms the prior state of the (78), showing strong robustness to hyperparameter configurations. Bold indicates the best scores, and underline indicates the second best scores.
Method
CD (x (100)
Geometry
F-Score
I.PIPS 4
Appearanze PSNR
SSIM
A
PhysAvatar 18
0.55
92.9
0.035
30.2
0.957
MPMAvatar (Ours)
0.42
95.7
32.0
0.963
Ours (N 800)
0.42
95.6
0.033
32.0
0.963
Ours (N
0.963
Ours (p = 0.5, = 100)
0.42
95.7
0.034
32.0
0.963
Ours 2.0,
100)
0.42
95.6
0.034
32.0
0.963
Ours (p = 1.0, E 50)
0.42
95.6
0.034
32.0
0.961
1.0, 200)
95.7
0.034
0.963
Ours (randomly initialized)
0.43
95.5
0.033
32.0
0.963
Ours (0.25x triangles)
0.44
95.4
0.033
32.1
0.964
Table 5. Quantitative comparison against concurrent baselines on the ActorsHQ (14) dataset. Bold and underlined values indicate the best and sccond-ber best scores, respectively. Our method consistently outperforms recent baselines across geometry and appearance metries, highlighting the advantage of physics-based simulation.
Geometry
Appearance PSNR
Method
CD (x (x10x0))
F-Score
I.PIS 4
SSIM
A
Gaussian Garments
55
2.39
86.5
0.042
29.5
0.959
MMLPHoman
94.9
0.039
29.3
0.954
MP'Avattrr (Ours)
0.42
95.7
0.033
32,0
0.963
Gaussian Garments [55]
MML.PHuman [74]
Ours
Ground Truth
Figure 5. Qualitative comparison against concurrent baselines on the ActorsHQ [14] dataset. We compare
our method with two recent concurrent methods: Gaussian Garments (55] and MMLPHuman [74], Gaussian
Garments (55) struggles to produce physically accurate deformations, while MML.HHuan 74 exhibits unnatural surface artifacts discontinuities under challenging poses. In contrast, method yields realistic plausible garment dynamics geometry.
Comparison with Additional Concurrent SOTA Baselines (Sec. A.6) further validate our approach, we additionally compare our method against two recent concurrent state-of-the-art avatar reconstruction methods: Gaussian Garments (55) and MMLPHuman [74]. Gaussian Garments replaces explicit simulation with learned dynamics module based graph neural networks. while MMLPHuman models geometry and appearance solely as a function of pose using multiple MLPs.
For both Gaussian Garments 55 and MMLPHuman 74 experiments, we ensure fair comparison by aligning key experimental components with our method. For Gaussian Garments (55), we replace our MPM- breM-baeeddt simulation with the learning- baaning-based garment simulator from ContourCraft [7]) which serves as the simulation backend of Gaussian Garments (55), while keeping tracked meshes and rendering pipeline identical to ours. Since Gaussian Garments (55) models the human body using only SMPI.-X [5T] without explicit body appearance modeling, we also use our body modeling setup ensure consistent evaluation environment. For MMLPHuman (74), follow the official
16

---

implementation and evaluation protocol, and additionally use the same template mesh as in our method.
As shown in Fig. 5 and Tab. 5 our method outperforms both baselines across all geometry and appearance metrics. The learned simulator in Gaussian Garments [SS] struggles to capture physical laws under our setting, where physical parameters must be estimated from only one second of motion, leading to high geometric error. Meanwhile, MMLPHuman [74] lacks explicit surface modeling and physical understanding, producing unrealistic surface artifacts or broken geometry when encountering unseen poses. These results demonstrate the advantage of our physies- based modeling pipeline for reconstructing accurate and plausible dynamic avatars.
B Implementation Details
B.1 Physics-Based Dynamics Modeling
Collision handling. In Alg. we outline our collision handling algorithm. After parameter initialization (lines 1-3), we iterate over every face the collider mesh and transfer velocity v and normal ny to nearby grid nodes based on B-Spline weights (lines 5-11). This yields the extended velocity vf and normal n of the collider at grid node i. Then, given the velocity of the simulating object at grid node i, if the vector vi v points inward, project out the normal component keeping only the tangential part - to model collision (lines 13-24). Note that this entire procedure runs in ((Nr) time, collision checks become simple B-Spline weight lookups rather than costly level-set queries at all grid nodes.
Algorithm Collision Handling
for each i in set of grid node indices do
2: vf, v. 0,0:
Initialize the zero-value grids for collider
3:
end for
4:
5: for each f in a set of collider mesh faces do
for in set neighhoring grid nodes of x, do
W + B (xpline(X X:);
Compute interpolation weights using B-spline kernel
v vf + vy.
nf nf w wyy
10: end for
11:
end for
12:
13:
each set grid node indices
wf + Σ, wfz
15:
if > then
D Detect whether a collision has occurred
16:
V + vf /
17:
nf nf/[[nf]:
18:
v! + - v.:
Transform velocities into the collider's reference frame Check if the relative velocity points inward toward the collider Project relative velocity onto the collider's tangent space
19:
if vreel of < 0 then
20:
ved ynd (v/e-nnf)
21:
end if V. vi +
22:
> Transform velocities back into world frame
23:
end if
24:
end for
Physical parameters learning. As discussed in Sec. 4.3.1 in the paper, we optimize Young's modulus E, density p, and rest geometry parameter by simulating the first-frame canonical mesh M₁ = (V1, F),) F) and minimizing the vertex-wise Lg error with respect to the tracked meshes (M.) where M, F).
In particular, let the initial mesh vertices be V V1. Then, for each subsequent frame, obtain the simulated mesh vertices via
VH = MPM(V, V1,
(2)
where == (E,Ky,RK,pol are physical parameters.
17

---

Here, the canonical vertices V1 and faces F provide the fixed mesh topology used to compute edge vectors and material directions, which turn define the deformation gradients for the anisotropic constitutive model inside the MPM simulation.
We then perform gradient- based optimization for p. E and a, such that they minimize the loss
T Ppyys (P) = Σ -
(3)
Here, each parameter's 's gradient is approximated using finite differences - following PhysAvatar For example, the gradient with respect to density p computed as
78
pppys Op
(,, Ap, a) -
(4)
where Ap is the perturbation size.
Training details.
Our MPM simulation uses a time step of A = 0.04 with N 400 substeps and grid resolution of 200. We optimize the physical parameters over 200 iterations using the Adam optimizer. For finite-difference gradient estimation, the perturbation sizes are set to Ap = 0.05, AE = 5, and Aa 0.005. The corresponding learning rates are 0.01 for p. 0.3 for E, and 0.01 for a. All parameters are initialized as 1.0, E = 100, and a = 1.0 for physical parameter learning, while V, 7 and K are fixed their default values of 0.3, 500, and 500, respectively,
Following PhysAvatar [78), we adopt a stage-wise training scheme: the physical parameters are optimized first and the tracked meshes remain fixed throughout. Appearance learning is then performed independently based on the same tracked meshes (see Sec. (43).
Simulation time and computing resource. As noted in the main paper (Sec. 5.2), simulation rums at approximately 1.1 seconds per frame on a single NVIDIA GeForce RTX 4090.
B.2 Appearance Learning
For the appearance learning (Sec. 4.3.2 in the paper), we leverage the dense temporal correspondences from the tracked meshes (M,,)i=1..1z to transform the canonical Gaussians G (defined in the first frame) into each subsequent frame. Following [54), we compute the transformations that carries every Gaussian from its parent triangle in the canonical mesh to the corresponding triangle in the target frame. This allows to render all training frames [1,..., T] using single shared appearance model G, such that it can be learned jointly from all input views and frames by minimizing:
Lupp = Ligh + Apfequsitin + Ag Lowealing
Here, Ligh measures the photometric discrepancy between the rendered and the ground truth images over all frames and views, and is defined as weighted sum of L1 loss L1, SSIM (67 loss LSSIM- and LPIIS 755 loss LIPIPS:
Cgb A1 + ASSIM + ALPIPS
(6)
In Eq. 5 Cposition || max(u, 6p) [|2 and Cualing || max e,) ll regularize the location offset
and the scale of each Gaussian not to exceed the thresholds = 1.0 and 6, = 0.6. This is to encourage the Gaussians remain closely aligned with their parent triangle structures.
When optimizing the parameters of G, we follow the standard 3DGS optimization procedure 22
and employ adaptive density control to increase the number of Gaussians in regions with high reconstruction error. For the loss weighting hyperparameters, we set Å 0.8, ASSIM = 0.2,
ALpIpS 0.2, Ap 1.0, and X; 1.0.
18

---

C Anisotropic Constitutive Model
In this section, we provide additional details on the anisotropic constitutive model used in our MPM simulation framework, to complement the explanation provided in Sec. B and Sec. 4.2.1 of the main paper. This elaboration aims help readers better understand how our method captures direction-dependent garment dynamics.
As explained in Sec. [ employ the Material Point Method (MPM) 17 to simulate the evolution of deformable objects by solving two governing equations: conservation of mass and conservation of momentum (Eq.. D. Among these, conservation of momentum governs the time evolution of velocity, and its simulation hinges on the computation of the Cauchy stress tensor or This stress tensor depends on the deformation gradient F and the strain-energy density function & via
F
FFt OF
where vb is defined by a material-specift constitutive model.
As introduced in Sec. 4.2. S.c.4.2.1 we adopt the anisotropic constitutive model proposed by Jiang et al. [1]] which is particularly well-suited for modeling thin, codimensional structures like garments. This model captures how cloth exhibits strong resistance to compression and shearing along the surface normal while remaining flexible along in-plane directions.
To compute the deformation gradient F at each particle, the model uses local material directions derived from Lagrangian mesh. Specifically,
F F=dD-1
DD, D2, Dz] denotes the canonical (undeformed) material directions and [dj, dg, ds} denotes the corresponding deformed directions. Since the strain energy function e must be invariant under rotations, the model applies QR decomposition F QR and reparameter- izes the energy function of the upper-triangular matrix R:
where D
(R) q n'normal Vshear Vl
where each term independently penalizes specific type of deformation. The normal component penalizes compression along the surface normal:
nnnmal Ra3 x) = 5 Ra3) if R33 1,
otherwise,
reflecting the assumption that cloth is typically surrounded by air and thus can freely expand but should resist compression.
The shear component penalizes dfi-oiagonal shear deformation between in-plane and normal direc- tions:
Satearr RR13 R22/)) 2 (R,; R23)).
which stabilizes the material frame by discouraging bending or tilting out of plane.
The in-plane component models isotropic stretching within the tangent plane using fixed-corotated formulation:
E Plne R1i R12, R22 E) 2(1 + -
Ev
(0102 - 1)²,
2(1 + U) 2v)
where d1, 02 the singular values of the in- in-plane matrix
RuI
R12 R22
R 2x2
This overall anisotropic formulation provides the strain-energy density i needed to compute the stress tensor in the momentum equation [b). thereby enabling our simulator to capture realistic garment behavior with directionally varying stiffness. This detailed model plays a central role in achieving the accurate and physically plausible dynamics demonstrated in our results.
19

---

D Limitations and Future Work
While our method achieves state-of-the-ar performance in both appearance and physical dynamics modeling, we acknowledge several limitations and outline potential directions for future work.
Scalability of Finite-Differenc Optimization. Our physical parameter optimization adopts finite-differene scheme, which scales linearly with the number of parameters. While this remains practical for our current setting, where per-garment material parameters suffice due limited intra- garment heterogeneity. extending to fine- one-grained parameterizations (e.g., per-vertex) would increase computational cost. As mitigation strategy, incorporating differentiable simulators 33 may improve scalability in future applications.
Relighting. Our current framework does not support relightable rendering. However, recent methods 19 57 have proposed relighting-awan extensions for Gaussian avatars, and our hybrid representation compatible with such techniques. We consider this a promising direction further enhance rendering realism.
Occlusion-Aware Generalization. Our pipeline directly optimizes appearance only regions visible in the multi-view training frames. Consequently, when previously occluded or unseen parts (e.g., the back side of the avatar) become visible under novel poses or viewpoints, rendering quality may degrade. Recent works have explored generative priors inpaint unobserved regions [26), diffusion- based view synthesis to generate pseudo multi-view supervision from monocular videos [63] Incorporating such approaches into pipeline could improve generalization to occluded unseen regions.
E Societal Impact
Our method enables physically accurate dynamic human avatar reconstruction from multi-view videos, supporting a wide range of applications in virtual reality, digital fashion, and entertainment. However, the capability generate lifelike avatars also introduces potential risks, such the misuse of the technology for creating deepfakes or other forms of deceptive content. When publishing our code, we will consider embedding traceable digital watermarks or developing authentication mechanisms to ensure the responsible use of generated avatars.
20

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes, we summarized the paper's contributions and scope in the abstract and the introduction, and our claims match the experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We explicitly acknowledge in the conclusion (Sec. 6) that our method does not support relighting and that our dynamics modeling for non-garment regions can be further improved.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "I "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
21

---

Answer: [NA]
Justification: Our work is application-focused and does not present any new theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Our primary contributions are the novel simulation- bassed avatar framework. All components of this framework are described in the Method section (Sec. 4 and the implementation details are provided in the supplementary document.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
22

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: While the code not submitted with the paper due the need for fur- ther cleanup, we plan to release it before publication to allow full reproducibility of our experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-sourge benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
wtwpsscccppblpcc/// gewiis/gguee//eeeeeeeieiiieiseee CodeeSummeee onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question:
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Our training and test protocols are outlined the Experimental Setup section (Sec. 5.1), with further details provided in the supplementary material.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We were unable to perform comprehensive significance testing or include error estimates due computational resource limitations. However, we will include them in the revision.
Guidelines:
The answer NA means that the paper does not include experiments.
23

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We report our per-frame simulation time Sec. and include the full details of our resources (including memory specifications) in the supplementary material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our study uses publicly available datasets under proper access agreements and does not involve sensitive personal data or human subjects. All experiments were conducted with standard computational resources and practices violate privacy, faireess, or environmental guidelines outlined the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?

---

Answer: [Yes]
Justification: As our work enables realistic human avatar creation, we acknowledge that it can be potentially misused for deepfake generation. We discuss these concerns in the supplementary material.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not involve releasing any high-risk pre-trained models or sensitive datasets.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We have cited all the assets used in this work, including the datasets.
Guidelines:
The answer NA means that the paper does not existing assets.
25

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets..paerssttt paperswi thcode. com/datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our work does not involve releasing new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
26

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: Our core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does involve LLMs as any important, original, or non standardd components.
Please refer to our LLM policy for what should or should not be described.
27


Reviews:
[{"review_id": "czdxa8O6zW", "reviewer": "Reviewer_1m12", "summary": "The paper presents MPM-Avatar, a method for learning 3D avatars with physically grounded deformation behavior using   Material Point Method (MPM) simulations. The approach leverages sparse 3D Gaussian representations to model both geometry and appearance, while also incorporating MPM-based losses to ensure plausible physical responses to external forces. The system jointly optimizes visual fidelity and dynamic realism from monocular RGB videos and physical annotations. Key contributions include the integration of physics priors into Gaussian-based avatar learning, a differentiable MPM framework for supervision, and experiments demonstrating improved physical plausibility over prior purely visual methods.", "questions": "1. Cite relevant prior work and clarify distinctions: Please cite key missing works in differentiable cloth and avatar simulation, including Differentiable Cloth Simulation for Inverse Problems (Liang et al., NeurIPS 2019), DiffCloth (Li et al., SIGGRAPH 2022), and DiffAvatar (Li et al., CVPR 2024). These works are directly relevant to your goals and share conceptual ground in learning physically realistic avatars. In your revision, please clearly articulate how your approach differs from and improves upon these methods, especially in terms of simulation modeling (e.g., MPM vs. mass-spring/cloth) and supervision (e.g., RGB + physics vs. mesh-based optimization).\n\n2. Evaluation of contact handling: The paper introduces contact-aware physics supervision, but there is no targeted quantitative evaluation of this component. Could the authors include an ablation or metric-based analysis (e.g., contact force accuracy, penetration depth, collision fidelity) to isolate and verify the effectiveness of the contact modeling? This would help clarify how much contact dynamics contribute to the observed improvements.\n\n3. Quantitative evaluation of physical plausibility: While qualitative examples are strong, can you provide more quantitative metrics to assess physical realism? This would help substantiate the claim of improved physical behavior.\n\n4. Scalability and computational cost: What is the training and inference cost of incorporating MPM simulation-based gradients, especially using finite differences? Could you provide a breakdown of runtimes on the over learning process and each iteration (forward pass, MPM loss computation, Gaussian updates)? This is important for judging the method’s practicality and applicability in larger-scale setups.", "limitations": "The authors do discuss limitations, particularly regarding physical supervision and force annotations. However, one important limitation not mentioned is the scalability of the framework with respect to the number of physical parameters. The current optimization procedure resembles that of PhysAvatar, which used a non-differentiable simulator and required two simulations per parameter, resulting in linear runtime scaling. If the same approach is used here, the method may become computationally infeasible for identifying a large number of physical parameters. Clarifying this limitation and discussing possible mitigation strategies (e.g., using differentiable simulators) would strengthen the discussion.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The paper proposes a framework that integrates Material Point Method (MPM) simulations with 3D Gaussian splatting for learning avatars that exhibit physically grounded deformations from monocular video. Experiments show quantitative and  qualitative improvements compared to PhysAvatar.\n\nRegarding related works, there is a lack of discussion and citation of several foundational works in differentiable cloth simulation that are directly relevant to the avatar reconstruction problem. In particular, Differentiable Cloth Simulation for Inverse Problems (Liang et al., NeurIPS 2019), DiffCloth (Li et al., SIGGRAPH 2022), and DiffAvatar (Li et al., CVPR 2024) present physically grounded, differentiable simulation frameworks that have been applied to cloth and digital avatars and addressing key aspects such as collisions.  These works should be cited and contrasted to clarify the novelty and scope of the current method. \n\n In addition, while the paper introduces a contact-aware physics loss via MPM simulation, it lacks a dedicated quantitative evaluation of the contact handling. There is no ablation or metric provided to demonstrate how well the method models contact dynamics compared to baselines or ground truth, which limits the ability to assess this component’s contribution.", "was_revised": true, "final_justification": "The authors' response has clarified my questions so I'm raising my score to borderline accept.", "rebuttal": "# General Response\n\nWe thank the reviewers for their constructive feedback and recognizing the **novelty of our collision handling method** (gjLz, 1QGS), **strong quantitative and qualitative results in dynamics, rendering quality, and efficiency** (xXua, gjLz, 1QGS, 1m12), as well as our method’s **zero-shot generalizability to unseen physical interactions** (xXua, 1QGS).\n\nIn response to the reviewers’ feedback, we have made several additions to strengthen the paper. Specifically, we conducted three new experiments:\n* **penetration depth analysis** for evaluating contact handling,\n* a **VLM-based assessment** of physical plausibility using PhyGenBench [R1], and\n* **additional ablations** on key simulation hyperparameters.\n\nWe also updated Tab. 1 to include comparisons with two recent state-of-the-art baselines [43, R2]. We hope these revisions address the reviewers’ concerns and welcome any further suggestions.\n\n**Table R1:** Quantitative comparison results including new baselines\n| Method | CD ($\\times 10^{3}$) (↓) | F-Score (↑) | LPIPS (↓) | PSNR (↑) | SSIM (↑) |\n|----------|----------|----------|----------|----------|----------|\n| MMLPHuman [R2] | 0.47 | 94.9 | 0.039 | 29.3 | 0.954 |\n| Gaussian Garments [43] | 2.39 | 86.5 | 0.042 | 29.5 | 0.959 |\n| PhysAvatar [61] | 0.55 | 92.9 | 0.035 | 30.2 | 0.957 |\n| Ours | **0.42** | **95.7** | **0.033** | **32.0** | **0.963** |\n\n[R1] Meng *et al.* Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation, In ICML, 2025.\n\n[R2] Zhan *et al.* Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs. In CVPR, 2025.\n\n---\n\n# Response to Reviewer 1m12\n\n> **Weakness 1.** \"There is a lack of discussion and citation of several foundational works in differentiable cloth simulation that are directly relevant to the avatar reconstruction problem.\"\n\n> **Questions 1.** \"Cite relevant prior work and clarify distinctions.\"\n\n**Reply:** Thank you for suggesting these additional related works. While our main paper focused on discussing and experimentally comparing against all baselines presented in the most recent SotA work directly addressing the same problem as ours (PhysAvatar [61]), we agree that the suggested works are also valuable in the broader context. We will make sure to include a discussion of them in the revision. In a nutshell:\n\n**DiffAvatar** [R5] is a framework designed to reconstruct a physics-based garment **from a static 3D scan, without modeling appearance**. We think its problem scope differs from ours, which targets photorealistic dynamic avatar reconstruction **from multi-view RGB videos**, with some of our key technical contributions include: (1) **photorealistic appearance modeling** based on Gaussian Splatting with quasi-shadowing (achieving SotA appearance results in Tab. 1), and (2) **learning the dynamic behavior of garments** from real observations (achieving SotA dynamics modeling accuracy in Tab. 1).\n\n**Liang** ***et al.*** [R3] and **DiffCloth** [R4] are related to ours in that they propose differentiable cloth simulators. However, as PhysAvatar [61] already noted, [R4] is difficult to “be directly applied due to limitations in modeling complex body colliders.” Indeed, we find that both [R3, R4] primarily consider simulation settings without any colliders, or only with simple geometric colliders (e.g., spheres). In contrast, our target problem of clothed avatar reconstruction requires handling collisions between garments and complex external colliders (e.g., body meshes), which was also a key aspect of our technical contribution. Accordingly, we prioritized discussing methods that address the same clothed avatar reconstruction task using simulators specialized for handling complex collisions (PhysAvatar [61] based on C-IPC [24]).\n\nNevertheless, we believe that discussing these related works adds valuable context, and we will revise our draft accordingly. Thank you again for your suggestions.\n\n[R3] Liang *et al.* Differentiable Cloth Simulation for Inverse Problems. In NeurIPS, 2019.\n\n[R4] Li *et al.* DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact. In SIGGRAPH, 2022.\n\n[R5] Li *et al.* DiffAvatar: Simulation-Ready Garment Optimization with Differentiable Simulation. In CVPR, 2024.\n\n> **Weakness 2.** \"It lacks a dedicated quantitative evaluation of the contact handling.\"\n\n> **Questions 2.** \"Evaluation of contact handling\"\n\n**Reply:** Thank you for your suggestion. However, we would like to clarify that, in our targeted problem – learning physics-based avatars from **real multi-view RGB video observations** – the existing datasets do not provide the ground truth contact force and collision labels, making it non-trivial to measure the suggested contact force accuracy and collision fidelity metrics.\n\nWhile existing published works addressing the same problem as ours (PhysAvatar [61], Dressing Avatars [53]) and even those proposing novel contact solvers (IPC [23,24]) similarly do not report these two metrics for this reason, they evaluate performance by measuring the discrepancy between the ground-truth geometry (which is obtainable from existing datasets [14, 51]) and the simulated geometry, which is a byproduct of the proposed contact modeling, thus somewhat **indirectly** reflecting the contact modeling accuracy. Regarding these geometry-based metrics, we reported that our method outperforms the existing SotA [61] in Tab. 1 in the paper.\n\nNevertheless, the suggested penetration depth metric is still measurable without access to ground-truth contact assets, and we present additional experimental results in Tab. R2 below. We find that our method yields lower penetration depth than the existing SotA method [61] on the ActorsHQ dataset, achieving over a 6× reduction in the average value. Following your suggestion, we will include these experimental results in the revision.\n\n**Table R2**\n| Method | Penetration Depth (mm) (↓) |\n|----------|----------|\n| **Ours** | **0.047** |\n| **PhysAvatar** [61] | **0.294** |\n\n> **Questions 3.** \"Quantitative evaluation of physical plausibility\"\n\n**Reply:** We appreciate the reviewer’s suggestion. We agree that physical plausibility is an important property to evaluate; however, we would like to note that there is no standard metric for quantitatively measuring physical plausibility in our related works [53, 61]. If the reviewer is referring to a specific metric in this context, we would appreciate it if they could kindly clarify, and we would be happy to discuss it further.\n\nNevertheless, we additionally conducted experiments using two metrics that we believe reflect physical plausibility. For the first metric, we considered penetration depth (also mentioned by the reviewer in another comment), as shown above in Tab. R2, where our method outperforms the baseline (PhysAvatar [61]).\n\nFor the second metric, we adopt the Key Physical Phenomena Detection metric proposed in PhyGenBench [R1], which computes a plausibility score using a VLM based on how well a given video aligns with physical plausibility prompts. In Tab. R3, we show that our method achieves a plausibility score closer to the ground-truth upper bound than PhysAvatar [61], indicating better alignment with the physical plausibility prompts.\n\n**Table R3**\n| Method | Score (↑) |\n|----------|----------|\n| **GT** | **1.86** |\n| **Ours** | **1.83** |\n| **PhysAvatar [61]** | **1.78** |\n\n> **Questions 4.** \"Scalability and computational cost\"\n\n> **Limitations 1.** \"scalability of the framework with respect to the number of physical parameters.\"\n\n**Reply:** Thank you for your insightful feedback. As mentioned, our method uses a finite-difference approach, which requires $n+1$ forward simulations to optimize $n$ physical parameters, resulting in linear runtime scaling with respect to the number of parameters. While this can indeed be “computationally infeasible for identifying a large number of physical parameters,” as pointed out in the comment, we would like to note that in our specific target application—physics-based dynamic avatar reconstruction—the garment behavior is mostly homogeneous. Thus, a single-material assumption **per garment** (which is also adopted in prior work [61]) was sufficient to achieve new SotA dynamics accuracy, without requiring the identification of a larger number of physical parameters.\n\nNevertheless, we agree that scalability is an important consideration for broader future applications. We will discuss this limitation and possible mitigation strategies (e.g., differentiable simulators) in the revision.\n\nRegarding the training and inference costs mentioned in the comment:\n* Our training takes approximately **5–8 hours per garment**, which is substantially faster than the existing SotA method targeting the same problem (PhysAvatar [61]), which reports **1–2 days** of training time.\n* Our inference time is **1.1s per frame**, as reported in Tab. 2 of the paper, representing a **~155× speed-up** compared to [61].\n* Below is a breakdown of the runtimes:\n  * **Simulation forward pass**: 1.1s per frame on average (Tab. 2).\n  * **MPM loss computation**: Requires $n+1=4$ forward passes per step to optimize $n=3$ physical parameters, resulting in approximately $4 \\times 24 \\times 1.1$s $\\approx \\mathbf{100}$**s** per iteration. This linear scaling behavior is also shared by PhysAvatar [61], which adopts the same finite-difference scheme.\n  * **Gaussian update**: Takes **~0.04s per iteration** via back propagation and adaptive density control; total time for optimizing Gaussian Splat parameters is **~1.5 hours per subject** (for 30K steps). Note that this stage is **independent of the number of physical parameters.**"}, {"review_id": "BSqOR2Bte1", "reviewer": "Reviewer_1QGS", "summary": "The paper introduces MPMAvatar, a novel framework that constructs 3D clothed human avatars from multi-view videos with robust animation and high-fidelity rendering. The method integrates a tailored Material Point Method (MPM) simulator that includes an anisotropic constitutive model for garment dynamics and a mesh-based collision handling algorithm for efficient and robust simulation. The method demonstrates strong performance over baselines and enables zero-shot generalization to novel interactions.", "questions": "1.\tIt is unclear how the method distinguishes between the garment and the underlying body regions in the avatar mesh. Could the authors clarify the process used to segment or identify these regions?\n2.\tIn my understanding, the rightmost image of Fig. 2 depicts the body grid with blue arrows, the garment grid with green arrows, and the collision regions between them with red arrows. However, I am not entirely certain this interpretation is correct. It would be helpful if the authors could provide a more detailed explanation to clarify the figure and improve overall readability.\n3.\tWhy is the formula for computing the final velocity in the rightmost image of Fig. 2 defined in that specific way? Is there a theoretical foundation behind this formulation?\n4.\tAre the physical parameter learning and appearance learning processes conducted separately or jointly?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1.\tTailoring of MPM for effective garment dynamics modeling based on anisotropic constitutive model and mesh-based collider handling is novel.\n2.\tBoth quantitative and qualitative results show clear improvements in dynamics accuracy, rendering fidelity, and simulation robustness and efficiency.\n3.\tThe method successfully handles unseen physical interactions, demonstrating its significant zero-shot generalizable ability.\n\nWeaknesses:\n1.\tUnlike PhysAvatar, MPMAvatar does not support relighting, which could be critical for some applications.\n2.\tSome technical descriptions could benefit from clearer exposition or additional illustrations, as certain implementation details may be difficult to follow for readers unfamiliar with MPM or anisotropic constitutive models.\n3.\tAs shown in the supplemental video at 16s, minor holes are visible in the clothing.\n4.\tThe method relies on multi-view video input, which may limit its practicality in real-world applications.\n5.\tThe dynamics of non-garment regions are modeled using linear blend skinning, which may not sufficiently capture realistic motion for complex structures such as hair.", "was_revised": true, "final_justification": "I maintain my positive rating.", "rebuttal": "# General Response\n\nWe thank the reviewers for their constructive feedback and recognizing the **novelty of our collision handling method** (gjLz, 1QGS), **strong quantitative and qualitative results in dynamics, rendering quality, and efficiency** (xXua, gjLz, 1QGS, 1m12), as well as our method’s **zero-shot generalizability to unseen physical interactions** (xXua, 1QGS).\n\nIn response to the reviewers’ feedback, we have made several additions to strengthen the paper. Specifically, we conducted three new experiments:\n* **penetration depth analysis** for evaluating contact handling,\n* a **VLM-based assessment** of physical plausibility using PhyGenBench [R1], and\n* **additional ablations** on key simulation hyperparameters.\n\nWe also updated Tab. 1 to include comparisons with two recent state-of-the-art baselines [43, R2]. We hope these revisions address the reviewers’ concerns and welcome any further suggestions.\n\n**Table R1:** Quantitative comparison results including new baselines\n| Method | CD ($\\times 10^{3}$) (↓) | F-Score (↑) | LPIPS (↓) | PSNR (↑) | SSIM (↑) |\n|----------|----------|----------|----------|----------|----------|\n| MMLPHuman [R2] | 0.47 | 94.9 | 0.039 | 29.3 | 0.954 |\n| Gaussian Garments [43] | 2.39 | 86.5 | 0.042 | 29.5 | 0.959 |\n| PhysAvatar [61] | 0.55 | 92.9 | 0.035 | 30.2 | 0.957 |\n| Ours | **0.42** | **95.7** | **0.033** | **32.0** | **0.963** |\n\n[R1] Meng *et al.* Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation, In ICML, 2025.\n\n[R2] Zhan *et al.* Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs. In CVPR, 2025.\n\n---\n\n# Response to Reviewer 1QGS\n\n> **Weakness 1.** \"Unlike PhysAvatar, MPMAvatar does not support relighting, which could be critical for some applications.\"\n\n**Reply:** While we achieve rendering metrics superior to the existing SotA physics-based avatar [61] even without relighting (Tab. 1), we agree that relighting is important for further enhancing the visual realism of our results — as also discussed in L372–373. Following recent relightable Gaussian avatar frameworks [R3, R4], we believe our framework can be extended to support relighting in the future. We will highlight this discussion in the revision.\n\n[R3] Jiang *et al.*, GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces. In CVPR, 2024.\n\n[R4] Saito *et al.*, Relightable Gaussian Codec Avatars. In CVPR, 2024.\n\n> **Weakness 2.** \"Some technical descriptions could benefit from clearer exposition or additional illustrations, as certain implementation details may be difficult to follow for readers unfamiliar with MPM or anisotropic constitutive models.\"\n\n**Reply:** We thank the reviewer for the helpful comment. We acknowledge that some readers may find it difficult to follow the technical details related to MPM [17] and the anisotropic constitutive model [16], especially without prior familiarity. In the main paper, we prioritized clarity in presenting our novel contributions while referring to established literature for existing components due to space limitations. As we agree that additional illustrations and explanations could further improve readability for a broader range of readers, we will include extended technical details and visualizations in the supplementary, following your suggestion.\n\n> **Weakness 3.** \"As shown in the supplemental video at 16s, minor holes are visible in the clothing.\"\n\n**Reply:** Although these minor holes are rare and our method still achieves superior visual results compared to the existing SotA work (PhysAvatar [61]), we believe they are caused by Gaussian splats that were occluded in the training videos but become exposed in the test sequence. We will discuss this in the revision and consider applying generative priors to inpaint these training-time occluded regions (similar to [R5]) as future work.\n\n[R5] Kwon *et al.*, Generalizable Human Gaussians for Sparse View Synthesis. In ECCV, 2024.\n\n> **Weakness 4.** \"The method relies on multi-view video input, which may limit its practicality in real-world applications.\"\n\n**Reply:** Thank you for your comment. Although our multi-view setup mainly follows the existing SotA physics-based avatar work [61], we agree that this input modality limits further practicability in real-world applications. We will include this discussion and also consider supporting a single-view setup as future work — e.g., by using diffusion-based view synthesis to obtain pseudo multi-view supervision from monocular video inputs, similar to a very recent work ([R6]).\n\n[R6] Tang *et al.*, GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion. In CVPR, 2025.\n\n> **Weakness 5.** \"The dynamics of non-garment regions are modeled using linear blend skinning, which may not sufficiently capture realistic motion for complex structures such as hair.\"\n\n**Reply:** As also discussed in L373–375, we agree that linear blend skinning for non-garment regions is limited in modeling complex dynamics, especially for hair motion. Since our underlying MPM framework with the anisotropic constitutive model [16] is versatile and supports the modeling of strand-like materials as well, we will consider extending our framework to strand-based hair simulation.\n\n> **Questions 1.** \"It is unclear how the method distinguishes between the garment and the underlying body regions in the avatar mesh. Could the authors clarify the process used to segment or identify these regions?\"\n\n**Reply:** For garment and human part segmentation, we mainly follow the pre-processing stage used in the existing SotA physics-based avatar work (PhysAvatar [61]). To ensure a fair comparison with [61], we directly use the garment-body segmentation provided by [61], which is manually annotated into garment and body regions. For the 4D-DRESS [51] dataset, we similarly define garment and body regions based on a consistent mesh topology through a manual process. While this segmentation step is currently manual, we believe it can be automated using existing segmentation approaches, as demonstrated in recent avatar methods such as D3GA [62], where per-image masks are projected onto the 3D mesh.\n\n> **Questions 2.** \"In my understanding, the rightmost image of Fig. 2 depicts the body grid with blue arrows, the garment grid with green arrows, and the collision regions between them with red arrows. However, I am not entirely certain this interpretation is correct. It would be helpful if the authors could provide a more detailed explanation to clarify the figure and improve overall readability.\"\n\n**Reply:** Your understanding of Fig. 2 is correct — blue arrows indicate body grid velocities, green arrows denote garment grid velocities, and red arrows show colliding grid regions where velocity projection is applied. Following your comment, we will clarify these arrow types in the figure caption in the revision.\n\n> **Questions 3.** \"Why is the formula for computing the final velocity in the rightmost image of Fig. 2 defined in that specific way? Is there a theoretical foundation behind this formulation?\"\n\n**Reply:** The overall procedure for computing the final velocity in Fig. 2 is mainly adopted from standard practices used in MPM [17]. In particular, projecting out the inward normal component of the relative velocity between the garment and the collider is a conventional strategy for contact handling in continuum mechanics. Our specific contribution lies in making this velocity projection feasible for complex colliders represented as meshes, where collider velocity and normals are strictly defined on the discretized surface, not on the grid nodes — unlike simple, analytic colliders typically previously considered in MPM [17]. Following your comment, we will explain this point more clearly.\n\n> **Questions 4.** \"Are the physical parameter learning and appearance learning processes conducted separately or jointly?\"\n\n**Reply:** We clarify that physical parameter learning and appearance learning are conducted separately, similar to the existing SotA physics-based work (PhysAvatar [61]). Following your comment, we will describe this more clearly in the revision."}, {"review_id": "RaAvEknrMp", "reviewer": "Reviewer_gjLz", "summary": "This paper presents MPMAvatar, a hybrid framework for learning physically plausible 3D human avatars from multi-view videos. It integrates a Material Point Method (MPM)-based cloth simulator with 3D Gaussian Splatting for high-fidelity rendering. Compared to prior works (e.g., PhysAvatar), the proposed system achieves more accurate garment dynamics, improved visual quality, and greater robustness under challenging animation inputs, including zero-shot scene interactions.\n\nKey contributions include:\n\nIntroducing a tailored MPM pipeline with an anisotropic constitutive model to model cloth dynamics.\n\nProposing a mesh-based collision handling algorithm for robust simulation against SMPL-X driven body inputs.\n\nOverall, I like the paper even I still have several concerns and the score will change according to the discussion and other reviews.", "questions": "Please refer to the weakness part.", "limitations": "Limited discussion of limitations: While the authors briefly mention in the conclusion that they do not support relighting or strand-based simulation, a more thorough analysis of the method's limitations is missing.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\nTechnically sound and novel: Tailoring MPM with anisotropic elasticity and mesh-aware collisions enables robust garment simulation that surpasses C-IPC-based avatars in both quality and efficiency.\n\nEfficient simulation: The proposed MPM implementation reduces simulation time alot compared to PhysAvatar due to the use of MPM.\n\n**Weaknesses:**\n\nSingle-material assumption: The method assumes the entire garment shares a single set of material parameters (e.g., a global Young’s modulus E). This limits its applicability to heterogeneous clothing composed of materials with different physical properties (e.g., jeans + silk top). No support is provided for segment-wise or per-region physical parameter estimation.\n\nDiscussion of simulation parameters: MPM’s accuracy and stability are highly sensitive to parameter initializaton, particle numbers, grid resolution and time step, yet the paper provides few discussion or empirical justification for their selection. These are critical hyperparameters in practical deployment.\n\nLack of failure case analysis: The paper primarily focuses on showcasing positive results but does not include any failure cases or challenging scenarios where the method might break down. For instance: (1) Are there cases where the simulation fails due to numerical instabilities or incorrect body-garment collisions? (2) Are certain garment types (e.g., multilayered skirts, long coats) poorly handled?", "was_revised": true, "final_justification": "After the rebuttal, most of my concerns are resolved, this I will keep my positive score.", "rebuttal": "# General Response\n\nWe thank the reviewers for their constructive feedback and recognizing the **novelty of our collision handling method** (gjLz, 1QGS), **strong quantitative and qualitative results in dynamics, rendering quality, and efficiency** (xXua, gjLz, 1QGS, 1m12), as well as our method’s **zero-shot generalizability to unseen physical interactions** (xXua, 1QGS).\n\nIn response to the reviewers’ feedback, we have made several additions to strengthen the paper. Specifically, we conducted three new experiments:\n* **penetration depth analysis** for evaluating contact handling,\n* a **VLM-based assessment** of physical plausibility using PhyGenBench [R1], and\n* **additional ablations** on key simulation hyperparameters.\n\nWe also updated Tab. 1 to include comparisons with two recent state-of-the-art baselines [43, R2]. We hope these revisions address the reviewers’ concerns and welcome any further suggestions.\n\n**Table R1:** Quantitative comparison results including new baselines\n| Method | CD ($\\times 10^{3}$) (↓) | F-Score (↑) | LPIPS (↓) | PSNR (↑) | SSIM (↑) |\n|----------|----------|----------|----------|----------|----------|\n| MMLPHuman [R2] | 0.47 | 94.9 | 0.039 | 29.3 | 0.954 |\n| Gaussian Garments [43] | 2.39 | 86.5 | 0.042 | 29.5 | 0.959 |\n| PhysAvatar [61] | 0.55 | 92.9 | 0.035 | 30.2 | 0.957 |\n| Ours | **0.42** | **95.7** | **0.033** | **32.0** | **0.963** |\n\n[R1] Meng *et al.* Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation, In ICML, 2025.\n\n[R2] Zhan *et al.* Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs. In CVPR, 2025.\n\n---\n\n# Response to Reviewer gjLz\n\n> **Weakness 1.** \"Single-material assumption: The method assumes the entire garment shares a single set of material parameters (e.g., a global Young’s modulus E). This limits its applicability to heterogeneous clothing composed of materials with different physical properties (e.g., jeans + silk top). No support is provided for segment-wise or per-region physical parameter estimation.\"\n\n**Reply:** We clarify that we use a single-material assumption **per garment**; thus, top and bottom garments (e.g., jeans and a silk top) have different material parameters. Indeed, the datasets [14, 51] used in our experiments contain several subjects wearing two types of garments (e.g., Fig. 3(b)), and we performed per-garment optimization in those cases—directly following the practice used in the current state-of-the-art method (PhysAvatar [61]). We will further clarify these points in the revision.\n\nAlthough we did not initially consider more fine-grained (e.g., per-vertex) material parameterization—as the datasets used in our experiments do not exhibit significant material heterogeneity within the same garment—we agree that this is an important direction for future work to enable more expressive garment modeling. We will include this discussion in the revision as well.\n\n> **Weakness 2.** \"Discussion of simulation parameters: MPM’s accuracy and stability are highly sensitive to parameter initializaton, particle numbers, grid resolution and time step, yet the paper provides few discussion or empirical justification for their selection. These are critical hyperparameters in practical deployment.\"\n\n**Reply:** We thank the reviewer for the insightful comment. To assess the sensitivity of our framework to these hyperparameters, we conducted an additional ablation study on the ActorsHQ [14] benchmark (Actor 01), where we evaluated the framework’s performance while varying key hyperparameters. In Tab. R2, we observe that variations in (1) physical parameter initialization, (2) particle numbers, and (3) time substeps do not significantly affect the final performance; notably, all variants outperform PhysAvatar [61] by a clear margin across all metrics. Note that for grid resolution, the grid and particle resolutions should be roughly aligned to enable stable momentum transfer between the two representations during MPM simulation, which is also a common convention adopted in many existing MPM-based methods [4, 54]. Therefore, we omitted further ablation on this aspect.\n\nWhile we primarily presented our ablation results on Actor 01 (as mentioned above) in the rebuttal due to time constraints, we will also run experiments on all sequences and include the complete results in the revision. We again thank the reviewer for the constructive feedback.\n\n**Table R2:** $N$ denotes the number of time substeps between two consecutive frames. $\\rho$ and $E$ represent the initial values of density and Young’s modulus used for physical parameter learning, respectively.\n| Method | CD ($\\times 10^{3}$) (↓) | F-Score (↑) | LPIPS (↓) | PSNR (↑) | SSIM (↑) |\n|----------|----------|----------|----------|----------|----------|\n| Ours (Full) | **0.38** | 95.7 | **0.025** | **34.1** | 0.964 |\n| Ours ($N=800$) | 0.39 | 95.6 | **0.025** | 34.0 | 0.964 |\n| Ours ($N=200$) | 0.39 | 95.6 | **0.025** | 33.9 | 0.964 |\n| Ours ($\\rho=0.5, E=100$) | **0.38** | 95.7 | 0.028 | 33.9 | 0.964 |\n| Ours ($\\rho=2.0, E=100$) | 0.40 | 95.4 | 0.028 | 33.7 | 0.964 |\n| Ours ($\\rho=1.0, E=50$) | 0.39 | 95.5 | 0.028 | 33.7 | 0.964 |\n| Ours ($\\rho=1.0, E=200$) | **0.38** | 95.7 | 0.028 | 34.0 | **0.965** |\n| Ours ($\\rho=1.7, E=1445$) | **0.38** | **95.8** | **0.025** | 33.9 | 0.964 |\n| Ours ($\\times 0.25$ triangles) | 0.39 | 95.5 | **0.025** | 34.0 | **0.965** |\n| PhysAvatar [61] | 0.45 | 94.8 | 0.034 | 32.0 | 0.962 |\n\n> **Weakness 3.** \"Lack of failure case analysis: The paper primarily focuses on showcasing positive results but does not include any failure cases or challenging scenarios where the method might break down. For instance: (1) Are there cases where the simulation fails due to numerical instabilities or incorrect body-garment collisions? (2) Are certain garment types (e.g., multilayered skirts, long coats) poorly handled?\"\n\n> **Limitations 1.** \"Limited discussion of limitations: While the authors briefly mention in the conclusion that they do not support relighting or strand-based simulation, a more thorough analysis of the method's limitations is missing.\"\n\n**Reply:** We thank the reviewer for pointing out the importance of discussing additional failure cases and limitations. Below, we respond to the questions raised:\n\n**(1) “Are there cases where the simulation fails due to numerical instabilities or incorrect body-garment collisions?”**\n\nThroughout the experiments, we did not observe numerical instabilities, as reflected in Tab. 2, where we report a 100% simulation success rate. Indeed, mitigating the numerical instabilities common in existing physics-based avatars with iterative solvers [24, 31] was one of our main goals, which motivated our design of “an MPM-based simulator capable of simulating objects under complex contacts without failure cases via feedforward velocity projection” (L49–51). We also did not noticeably observe incorrect body-garment collisions in our main experiments; however, we discuss other potential limitations in the following paragraph.\n\n**(2) Are certain garment types (e.g., multilayered skirts, long coats) poorly handled?**\n\nWe believe that our current **appearance** modeling may be limited in multilayered garment settings, especially for inner garments. Similar to existing optimization-based avatar creation methods [19, 27, 33, 37, 62], we directly optimize the appearance parameters from multi-view RGB videos. As a result, artifacts may occur in regions that were occluded during training (e.g., inner garments occluded by outer garments) but become visible in novel poses — stemming from incomplete supervision for occluded areas. More recent avatar creation methods utilize generative priors to inpaint these occluded regions ([R3]), which can be an important future direction for our work. We will discuss this limitation and potential future work in the revision.\n\n\n[R3] Kwon *et al.*, Generalizable Human Gaussians for Sparse View Synthesis. In ECCV, 2024."}, {"review_id": "CyuVWMKBrk", "reviewer": "Reviewer_xXua", "summary": "This paper introduces MPMAvatar, a method for learning physically accurate and robust 3D garment dynamics from multi-view video using the Material Point Method (MPM). The approach simulates only the garments using an anisotropic elastoplastic constitutive model, while treating the human body and external scene objects as colliders. A small set of physical parameters—including Young’s modulus, density, and rest shape scaling—are optimized via finite-difference methods to match tracked garment meshes. For appearance, the method leverages 3D Gaussian Splatting (3DGS) with a shared canonical representation and quasi-shadowing to produce photorealistic rendering across views. Compared to PhysAvatar, MPMAvatar demonstrates superior accuracy, simulation robustness, generalization to novel interactions, and rendering quality. It also runs orders of magnitude faster than an IPC-based baseline.", "questions": "1. The method first constructs a tracked mesh from multi-view videos. Then, it performs MPM material optimization, followed by appearance optimization. Is this a stage-wise optimization, i.e., MPM and appearance are optimized separately? Also, will either of these 2 optimizations modify the underlying tracked-mesh?\n\n2. Regarding the Geometry metric reported in Table 1, are the inputs to PhysAvatar and MPMAvatar exactly the same? Does rendering have any effect on Geometry metrics in this experiment? I ask this because to confirm the effectiveness of MPM, we should rule out any other factors (e.g., 3DGS rendering) that could benefit Geometry metrics.\n\n3. In the bottom row of the right subfigure of Fig. 2, it is unclear what \"Grid Velocity\", \"Projected Velocity\", and \"Final Velocity\" are.\n\n5. The term \"edge vector\" (line 264) is introduced abruptly without a formal definition, which can hinder clarity. Further, it is unclear why this edge vector decomposition is crucial for learning the scale parameter, as the paper does not introduce anywhere how edge vectors are involved in an anisotropic constitutive model. Simply referring to Jiang et al. is not enough; there should at least be a section that builds connections between garment mesh topologies and the actual MPM function. Currently, only Eq. (1) in the Supp. Mat. treats MPM as a function of current vertices and physical parameters, but provides no more details. In this context, it is impossible to judge why the \"edge vector decomposition\" technique is helpful. Also, if MPM takes only the vertices of the garment mesh as input, how do \"edge vectors\" affect MPM at all?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. By using MPM with a novel mesh-based collision handler, the method avoids simulation crashes common in PhysAvatar and achieves a 100% success rate across sequences.\n\n2. The method supports physically plausible cloth behavior (e.g., folds, collisions) with \\~1.1s/frame simulation time—substantially faster than PhysAvatar (\\~170s/frame).\n\n3. MPMAvatar handles unseen pose sequences and novel scene-object interactions (e.g., garments contacting cushions or sand) without retraining, attributed to the physically grounded formulation.\n\n4. The use of 3D Gaussian Splatting, combined with quasi-shadowing, results in visually superior renderings compared to mesh-based avatars.\n\n5.  The supplementary video and ablation studies isolate the contributions of anisotropic modeling, rest geometry correction, and quasi-shadowing.\n\nWeaknesses (please see the Questions section for details):\n1. Although there are some high-level descriptions in the Supp. Mat., the learning process is unclear,\n\n2. The experimental setup also has some clarity issue, which makes the effectiveness of using MPM questionable.\n\n3. Fig.2, especially the collision handling part, is confusing. While the B-spline part is explained sufficiently in Sec. 4.2.2 (main paper) and Alg. 1 (Supp. Mat.), the bottom row needs more clarification.\n\n4. Some quantities, such as edge vectors, are brought up without context and need more clarification.\n\n5. The Quasi-shadow modeling gives only marginal improvement.", "was_revised": true, "final_justification": "The authors addressed my concerns, and I would like to keep my positive scores.", "rebuttal": "# General Response\n\nWe thank the reviewers for their constructive feedback and recognizing the **novelty of our collision handling method** (gjLz, 1QGS), **strong quantitative and qualitative results in dynamics, rendering quality, and efficiency** (xXua, gjLz, 1QGS, 1m12), as well as our method’s **zero-shot generalizability to unseen physical interactions** (xXua, 1QGS).\n\nIn response to the reviewers’ feedback, we have made several additions to strengthen the paper. Specifically, we conducted three new experiments:\n* **penetration depth analysis** for evaluating contact handling,\n* a **VLM-based assessment** of physical plausibility using PhyGenBench [R1], and\n* **additional ablations** on key simulation hyperparameters.\n\nWe also updated Tab. 1 to include comparisons with two recent state-of-the-art baselines [43, R2]. We hope these revisions address the reviewers’ concerns and welcome any further suggestions.\n\n**Table R1:** Quantitative comparison results including new baselines\n| Method | CD ($\\times 10^{3}$) (↓) | F-Score (↑) | LPIPS (↓) | PSNR (↑) | SSIM (↑) |\n|----------|----------|----------|----------|----------|----------|\n| MMLPHuman [R2] | 0.47 | 94.9 | 0.039 | 29.3 | 0.954 |\n| Gaussian Garments [43] | 2.39 | 86.5 | 0.042 | 29.5 | 0.959 |\n| PhysAvatar [61] | 0.55 | 92.9 | 0.035 | 30.2 | 0.957 |\n| Ours | **0.42** | **95.7** | **0.033** | **32.0** | **0.963** |\n\n[R1] Meng *et al.* Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation, In ICML, 2025.\n\n[R2] Zhan *et al.* Real-time High-fidelity Gaussian Human Avatars with Position-based Interpolation of Spatially Distributed MLPs. In CVPR, 2025.\n\n---\n\n# Response to Reviewer xXua\n\n> **Weakness 1.** \"Although there are some high-level descriptions in the Supp. Mat., the learning process is unclear\"\n\n> **Questions 1.** \"The method first constructs a tracked mesh from multi-view videos. Then, it performs MPM material optimization, followed by appearance optimization. Is this a stage-wise optimization, i.e., MPM and appearance are optimized separately? Also, will either of these 2 optimizations modify the underlying tracked-mesh?\"\n\n**Reply:** We clarify that we perform stage-wise optimization, where the physical parameters and appearance are optimized separately. Throughout these stages, we do not modify the tracked meshes obtained during data pre-processing, as they serve as pseudo ground truth to “supervise the subsequent learning stages” (L239–240 in the paper). Specifically, the tracked meshes are used to compute (1) vertex-level L2 losses during physical parameter optimization (L33–36 in the supplementary) and (2) photometric losses during appearance optimization (L53–55 in the supplementary). We also note that our stage-wise optimization and use of tracked meshes as pseudo ground truths were primarily motivated by the common practice in prior physics-based avatar work [61]. We thank you for the suggestion and will further clarify these points in the revision.\n\n> **Weakness 2.** \"The experimental setup also has some clarity issue, which makes the effectiveness of using MPM questionable.\"\n\n> **Questions 2.** \"Regarding the Geometry metric reported in Table 1, are the inputs to PhysAvatar and MPMAvatar exactly the same? Does rendering have any effect on Geometry metrics in this experiment? I ask this because to confirm the effectiveness of MPM, we should rule out any other factors (e.g., 3DGS rendering) that could benefit Geometry metrics.\"\n\n**Reply:** We clarify that, when measuring the Geometry metrics (Tab. 1 in the paper), (1) the inputs to PhysAvatar [61] and our method are the same, and (2) rendering quality does not affect these metrics. As described in L316–317 in the main paper, these Geometry metrics (Chamfer Distance and F-Score) are computed between the simulated and ground-truth meshes, independent of the rendering strategy. While our avatar uses a hybrid representation (mesh + 3D Gaussians), only the mesh component is used for geometry evaluation, ensuring a fair comparison.\n\n> **Weakness 3.** \"Fig.2, especially the collision handling part, is confusing. While the B-spline part is explained sufficiently in Sec. 4.2.2 (main paper) and Alg. 1 (Supp. Mat.), the bottom row needs more clarification.\"\n\n> **Questions 3.** \"In the bottom row of the right subfigure of Fig. 2, it is unclear what \"Grid Velocity\", \"Projected Velocity\", and \"Final Velocity\" are.\"\n\n**Reply:** The bottom row of Fig. 2 follows the standard collision handling procedure in MPM [17], where:\n\n* **Grid Velocity** denotes the original velocity of the garment at each colliding grid node (L224-226);\n\n* **Relative Velocity** denotes the garment’s velocity relative to the corresponding collider grid (L224-226);\n\n* **Projected Velocity** denotes the relative velocity after removing the inward normal component based on the body’s surface normal, to prevent penetration (L226); and\n\n* **Final Velocity** is the projected relative velocity converted back into the world-frame garment velocity after projection (L226-227).\n\nWhile we provided a high-level description of this process in Sec. 4.2.2 of the main paper, as well as in Alg. 1 and L23–30 of the supplementary material, we will make sure to explicitly clarify the figure notations and strengthen the connection between the figure and the text in the revision.\n\n> **Weakness 4.** \"Some quantities, such as edge vectors, are brought up without context and need more clarification.\"\n\n> **Questions 4.** \"The term \"edge vector\" (line 264) is introduced abruptly without a formal definition, which can hinder clarity. Further, it is unclear why this edge vector decomposition is crucial for learning the scale parameter, as the paper does not introduce anywhere how edge vectors are involved in an anisotropic constitutive model. Simply referring to Jiang et al. is not enough; there should at least be a section that builds connections between garment mesh topologies and the actual MPM function. Currently, only Eq. (1) in the Supp. Mat. treats MPM as a function of current vertices and physical parameters, but provides no more details. In this context, it is impossible to judge why the \"edge vector decomposition\" technique is helpful. Also, if MPM takes only the vertices of the garment mesh as input, how do \"edge vectors\" affect MPM at all?\"\n\n**Reply:** Thank you for your comment — we will make sure to include the formal definition of the edge vectors in the revision. We clarify that the edge vectors (L264) refer to the side vectors of each triangle in the canonical mesh. The orthogonal decomposition of these edge vectors is necessary to apply the scale parameters $\\alpha$ along the axis aligned with the gravity direction (and not along other directions; L267-268), as our goal was “to compensate for gravity-induced deformation” (L263–264) when approximating the rest geometry.\n\nRegarding the connections between the mesh topology and the MPM simulation, we clarify that the mesh topology is used to compute the material directions $\\mathbf{D}$ (L191), which are then used to compute the deformation gradients $\\mathbf{F}$ in the anisotropic constitutive model (L190–197). Based on these deformation gradients, MPM [17] “internally computes forces relative to the object’s rest geometry” (L257–258). Initially, we only provided high-level descriptions, as these connections are mostly established in the existing MPM [17] and the anisotropic constitutive model [16] that we adopted. However, we agree that this hinders clarity for readers unfamiliar with these works. Following your comment, we will add a separate section in the supplementary to explain in detail how the edge vectors are used to compute deformations in these existing methods, and we will also modify the simplified Eq. (1) in the supplementary to explicitly take the mesh topology.\n\n> **Weakness 5.** \"The Quasi-shadow modeling gives only marginal improvement.\"\n\n**Reply:** While we agree that quasi-shadow modeling yields only slight improvements in quantitative metrics, we believe its qualitative impact is non-negligible, particularly in garment regions with large deformations or self-occlusions. As shown in our ablation study results (Fig. 4(a) in the paper and the supplementary video), removing quasi-shadowing leads to noticeably less realistic visual results. In photometric avatar rendering applications, where perceptual plausibility plays a central role, we believe such qualitative improvements remain meaningful, even when commonly used pixel- or feature-level metrics show modest gains."}]

Author rebuttal:

Meta-review: Concerns raised during the review process were addressed in the rebuttal with additional experiments and explanations. All reviewers maintained or increased their positive scores. For the camera-ready version, under the pape limit, the authors should: (1) add missing citations and comparison with relevant differentiable simulation work, (2) expand quantitative evaluation of contact handling and physical plausibility, (3) improve clarity of figures and technical exposition (e.g., velocity formulations, edge vectors), and (4) provide a clearer discussion of limitations and assumptions.
Decision: Accept (poster)