Learning 3D Garment Animation from Trajectories of A Piece of Cloth
Authors: Yidi Shao
Chen Change Loy
Bo Dai
Venue: NeurIPS 2024 poster (2024)
Keywords: Garment
Cloth
Simulation
Constitutive Model
3D
Primary area: machine_learning_for_physical_sciences
TL;DR: To learn to animate the garments, we adopt the disentangled scheme where we model the constitutive laws of a piece of cloth from the observations and animate garments using energy optimization constrained by our model.

Abstract: Garment animation is ubiquitous in various applications, such as virtual reality, gaming, and film producing. Recently, learning-based approaches obtain compelling performance in animating diverse garments under versatile scenarios. Nevertheless, to mimic the deformations of the observed garments, data-driven methods require large scale of garment data, which are both resource-wise expensive and time-consuming. In addition, forcing models to match the dynamics of observed garment animation may hinder the potentials to generalize to unseen cases. In this paper, instead of using garment-wise supervised-learning we adopt a disentangled scheme to learn how to animate observed garments: 1). learning constitutive behaviors from the observed cloth; 2). dynamically animate various garments constrained by the learned constitutive laws. Specifically, we propose Energy Unit network (EUNet) to model the constitutive relations in the format of energy. Without the priors from analytical physics models and differentiable simulation engines, EUNet is able to directly capture the constitutive behaviors from the observed piece of cloth and uniformly describes the change of energy caused by deformations, such as stretching and bending. We further apply the pre-trained EUNet to animate various garments based on energy optimizations. The disentangled scheme alleviates the need of garment data and enables us to utilize the dynamics of a piece of cloth for animating garments. Experiments show that while EUNet effectively delivers the energy gradients due to the deformations, models constrained by EUNet achieve more stable and physically plausible performance comparing with those trained in garment-wise supervised manner.

Learning 3D Garment Animation from Trajectories of A Piece of Cloth
Yidi Shao' Chen Change Loy IX Bo Dai
S-Laa, Nanyang Technological University The University of Hong Kong, *Shanghai Artificial Intelligence Laboratory yidi0010e. atu.edu.gg, ccloyEntu. edu.ag, ag. bdai@hku. hk
olis
Given the observed piece of cloth as shown the left, we aim to animate various garments inheriting the attributes from the observations as shown in the middle and right. We disentangle the garment-wise learning into two sub-tasks: learning constitutive relations by proposed EUNet; 2) animating diverse garments through energy optimization constrained by EUNet.
Figure 1:
Abstract
Garment animation is ubiquitous in various applications, such as virtual real- ity, gaming, and film production. Learning-based approaches obtain compelling performance in animating diverse garments under versatile scenarios. Neverthe- less, to mimic the deformations of observed garments, data-driven methods often require large-scale garment data, which are both resource-expensive and time-consuming. addition, forcing models to match dynamics of observed garment animation may hinder the potential to generalize to unseen cases. In this paper, instead of using gurment-wise supervised learning we adopt a disentangled scheme learn how to animate observed garments: 1) learning constitutive behay- iors from the observed cloth; 2) dynamically animate various garments constrained by the learned constitutive laws. Specifically, (EUNet) to model the constitutive relations in the form of energy. Without the priors from analytical physics models and differentiable simulation engines, EU- Net is able to directly capture the constitutive behaviors from the observed piece of cloth and uniformly describes the change of energy caused by deformations. such as stretching and bending. We further apply the pre-trained EUNet to ani- mate various garments based on energy optimizations. The disentangled scheme alleviates the need for garment data and enables us to utilize the dynamics of piece of cloth for animating garments. Experiments show that EUNet effec- tively delivers the energy gradients due to the deformations. Models constrained by EUNet achieve more stable and physically plausible performance compared
propose Energy Unit network
X Corresponding Author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

with those trained in garment-wise supervised manner. Code is available at https://githie http://gh.....o///bbb////////e// Bttbbaaae/EEiee_eeeerPP522222220 gitt.
1 Introduction
To create a realistic virtual world, it is crucial to ensure the clothes worn by digital humans have natural and faithful deformations similar to those in real life. Given its significance, garment animation has been extensively explored and applied in various fields, including filmmaking, virtual try-on, and gaming.
Recent deep-Iearnigg based methods [35, 2, 27, 22, 28] exhibit great potential in mimicking the dynamic patterns of the observed garments with compelling efficiency, generality and robustness. A common strategy of deep-learning based methods is to model garments as functions of SMPL. based human parametric models [35, 23, 2, 31, 4], which is inevitably limited to clothes of similar topologies of human bodies, such as T-shirt, failing to capture the dynamics of loose garments like dresses. To overcome such drawbacks of SMPL-based methods, there are attempts that adapt garment-specific designs [21, 20]. and particle-based designs [28]. Nevertheless, to obtain similar deformation patterns the observed garments, most existing deep-learning based methods require large-scale garment-wise datasets for training, which usually contains hundreds of samples, covering humans of various shapes and actions with garments of diverse topologies and materials. Collecting such a sophisticated dataset is both resource- essoure-exppessvee and laborious, especially when it is designed to ensure the generality and robustness of learned models, which may take months or even years to build. On the other hand, overly relying on large-scale dataset may not be an efficient and effective way to obtain a good garment animation model that can generalize well to unseen garment topologies, human shapes and actions, as well as environmental factors.
To avoid the overhead and drawbacks of directly learning from large-scale garment-level datasets, in this paper we adopt a disentangled framework that learns to animate garments from just a single piece of cloth. A key observation is that while garments may vary in topologies, and their dynamics seem to be different given external forces. for garments and cloth made of the same material the constitutive relations, such as the stress-strain behaviors of clastic materials, remain consistent and are agnostic to garment topologies. Moreover, the dynamics of garments can also be defined as a time evolution problem constrained by both external forces and constitutive relations, which again is agnostic to garment topologies. Therefore, the task of mimicking garments' realistic dynamics disentangled into two sub-tasks: 1) capturing the constitutive behaviors of some materials from piece of cloth; 2) applying the learned constitutiee laws to optimization-ban based scheme [9, 10] to dynamically animate garments made of the same material.
Learning constitutive laws is a non-trivial task. Domain experts design various analytical physics models to describe physical properties of certain materials, such as Piola Kirchhoff stress, bending model [6] by Bridson, and elastic models by St.
and neo-Hookean, Some methods [5, 34, 25, 11] take efforts to estimate the corresponding physics parameters, such as the Lame constant in St. Venant-Kirchhoff" elastic model, to fit the analytical models to observed materials, while others [33, 37, 30, 32, 13, 18, 15] tend to rely on these analytical physics models as priors and embed them in the formulations. However, there is no general standard to choose the most suitable physics model automatically.an existing models cannot cover all materials. As a result, methods relying on these physics models are limited in generality, especially when handling garments whose physical properties are unknown and difficult to measure.
In this paper, we propose Energy Unit Network (EUNet), a method that learns to describe the constitutive laws in the form of energy directly from the observed trajectories. Our EUNet is able to uniformly generate the energy gradients caused by different deformations from a cloth system with dissipation, such as the deformations of stretching and bending and the damping effects due the air. Specifically, to achieve the topology-in appoloyy-independen modeling of constitutive relations, EUNet predicts edge-wist energy units, of which the summations represent the garment- wise energy. Each energy unit consists of potential and dissipation energy units, which are predicted by two sub-networks in EUNet respectively. While the observed sequences are unable to provide explicit supervision of constitutive behaviors, we propose to estimate the systems" total energy, including the dissipation energy, between adjacent frames to extract the variations of cloth's energy. With the system-wise scalars supervising signals, we train EUNet directly from the observed trajectories without the

---

need of analytical clothing models or differentiable simulators as prior. Since different deformations could lead to similar energy in total, the system-wist scalars may confuse EUNet to obtain plausible energy gradients. To further reduce the ambiguity and improve the accuracy of our EUNeet we build vertex- -wiee contrastive loss, which takes inspiration from the energy optimization [9] that the evolution of the dynamics follows the path minimizing the system's 's total energy. Thus, we constrain the EUNet that the sampled vertex-wise disturbance, such as those caused by random noise, will lead to larger energy in total. Once we obtain the pre-trained EUNet, we embed the predicted energy as a part of optimization-base physics loss [26, 10] to rollout the temporal evolution of garments.
The disentangled scheme and EUNet benefit garment animations in several aspects. First, the disentanglement enables us use only a piece of cloth as training data to animate various garments, which is data- efficient compared with the large scale of garments dataset [2. 28]. Second, without auxiliaries of existing physics models as prior, such as St. Senant-Kirchhoff model that need extra estimations of the physics parameters to fit the target materials, EUNet is able to extract the constitutive laws directly from observed cloth and uniformly capture the potential energy derived from different deformations, such stretching and bending. Third, since EUNet is agnostic to the topologies clothes, EUNet can describe the deformation patterns of various garments, such as T-shirts and dresses, and naturally support the animation of different garments based on the energy optimization [26, 10]. Consequently, we bridge the gap between observing cloth and animating various garments inheriting the attributes of observed materials.
To train our EUNet, we collect the dynamics of a piece of square cloth made of common materials, i.e. silk, leather, cotton, and denim. The square cloth, whose corners on top are pinned, is initialized with random positions and deforms under the influence of gravity. To verify the quality of the animated garments constrained by our EUNet, we apply an optimization- based scheme [9, 26, 10] to solve the dynamic deformations. We compare with models trained in garment-wise supervised manner on clothes of the same materials, and evaluate the Euclidean errors between the predictions and ground truth data. As shown in the experiment, models constrained by our EUNet delivers lower errors and physically plausible rollouts comparing with models trained directly in garment-wise manner, even for long-term predictions.
Our contributions can be summarized as follows: 1) we propose to learn the dynamics of observed garments through the disentangled scheme: learning constitutive relations and dynamically animating by energy optimizations; 2) we propose EUNet to directly capture the constitutive laws from observed trajectories using just a single piece of cloth; 3) we combine EUNet with optimization- scheme to animate diverse types of garments inheriting the dynamic patterns from the observed cloth.
2 Related Work
Learning-basse based Garment Animation. Most existing methods model garments conditioned on the parameters of SMPL-based human bodies, including the human pose and shape [35, 2, 31, 4]. The over-reliance on the SMPL parameters inevitably leads to static models, which predict unique deformations for identical parameters of humans, garment-specific design [21, 23, 20], and limited generalization abilities to interact with obstacles beyond SMPL-based human models. While plausible deformations are obtained mostly on skinny garments, which have similar topologies as the underlying human bodies, previous work struggles in animating loose clothes. Extra designs are required to extend the human-dependent garment model to handle motions [27, 36] and loose garments [22]. Recently, by modeling the interactions among garments' vertices, simulation -ased methods [10, 28] achieve topology-indee indeppnneent design and are highly generalizable to unseen garments and obstacles. However, most existing approaches adopt supervised learning and demand large-scale data of high quality. While some methods [3, 26, 10] avoid the need for data by using physics priors from analytical clothing models, the dynamic behaviors of predicted garments are restricted within the space defined by the clothing models and unable to mimic observed cloth with attributes beyond the known physics prior.
In contrast, we apply the disentangled scheme of the supervised task and utilize only a piece of cloth as ground truth data to learn the constitutive relations from observations. Constrained by the learned FUNet, we can directly animate garments of diverse topologies inheriting the attributes of observed cloth.
3

---

Garment- Wise Leaming
Slep
Step 2
Mappings
O Commiiaaiin
FLNEII
Tisaa
Overview of the disentangled learning scheme and EUNet for garment animation. Unlike traditional garment-wise learning which relies large scale of garment data, we first aim capture the constitutive relations from the observed piece cloth using our EUNett. Without the prior analytical clothing models or differentiable simulator, EUNet is able to extract the potential energies of the cloth under different deformations, such as stretching and bending. directly from the observed trajectories in system with dissipation. Secondly, given the external force sequences and the garment templates. we dynamically animate various garments based on the energy optimizations. where EUNet serves as material priors. result, we can animate garments that inherit the attributes, such as the stiffness, from the observed cloth, and achieve robust and physically plausible animations.
Figure 2:
Constitutive Laws. Constitutive laws describe the materials' reactions towards external stimuli, such as the potential energy or forces caused by deformations, and have been ubiquitous in the physics study: elasticity [1], plasticity [8], and fluid [7]. A common practice [5, 34, 25, 11] to mimic objects' constitutive behaviors is estimating physics parameters
fit the analytical physies models to the observed materials, such as predicting the lame constant in St. Venant-Kirchhoff elastic model. Other beariingbassed approaches [29, 33, 37, 30, 32, 13, 18, 15, 17] adopt analytical models as known physics priors, which are used to generate labeled data or embedded as part of the formulations. In addition, some methods [19] depend on differentiable particle- particle-based physics engines enable the training. In this paper, we focus the elasticity behaviors of clothes represented by meshes. Unlike previous learning-basse based methods determined by analytical models or differentiable simulation engines, we formulate our constitutive relations constrained by general physies laws, such as the Lagrangian mechanism, enabling the training directly from observed data in a generic manner.
Incremental Potential. Discretizing in time, the evolution of the objects' states, such as the positions and velocities, can be solved by minimizing the incremental potential of the system [12, 14, 16]. Recent methods [26, 10] follow the scheme and formulate a set of physics-based loss terms. Since minimizing the loss terms is equivalent to resolving the optimization problem, neural networks can serve as robust and differentiable tools to solve equations constrained by the incremental potential. In this paper, the constitutive laws learned by BUNet can be seamlessly embedded as part of the incremental potential, enabling to animate garments with attributes captured by our BUNet.
3 Methodology
The objective of mimicking the observed garment dynamics is to minimize the discrepancies between the predicted clothes and ground truth data under external forces, such as the motions of human bodies. In other words, the task seeks to capture the intrinsic deformation patterns of the observed garments, such as the stress-strain behaviors, which are independent of external influences. To achieve this, as shown in Figure 2, we disentangle the task into 1). learning the constitutive behaviors of several materials and 2). dynamically animating garments constrained by the learned laws through energy optimization. In Section 3.1, we introduce the required data for training, which consists of the trajectories of just pieces of cloth instead the whole garment. Next, Section 3.2, we direct the attention to our EUNet which captures the constitutive behaviors from the observed cloths. Most importantly, we propose a generic scheme to model the cloth 's energy, without the need for an analytical clothing model or differentiable simulation engine. Finally, in Section 3.3, describe the dynamic animations of garments constrained by our BUNct.
3.1 Data of A Piece of Cloth
Since constitutive laws are agnostic to the topologies, the dynamics observed in piece of cloth are consistent with those exhibited by entire garments. Therefore, select square piece of cloth the

---

subject of our observations, which is easier to design and more efficient to collect compared with garments. Specifically, we select clothes made of several materials typical in real life: silk, leather, denim, and cotton. We pin the cloth's s two corners on top shown in Figure 1. With randomly initialized positions and velocities, the cloth moves and deforms under the influence of gravity. The cloth dynamics are synthesized using Blender". More details about the data can be found in the Appendix.
3.2 Capturing Constitutive Laws
Problem Formulations. We denote the mesh-based cloth with triangulated faces at time by
M (x', E). where X = { :=l are the positions for N vertices, and E the un-directed edges of the mesh. The rest state of the cloth, which is the template, is denoted by M = {X, E}. The cloth's 'ss attributes, such the stiffinss, are represented by a. Given observed trajectories of cloth {M* we aim to predict the cloth's 's energy at time by
(I) (,()) is the potential energy describing the constitutive behaviors of the cloth, whereas Pa(.) is the dissipation energy caused by the damping effects in the dynamic process. For simplification, we represent the potential energy at time t by D and the corresponding dissipation energy o in the following sections.
((X', x E, = ( X, E,a) + 0a(X' - E,a)
Supervised Learning of Constitutive Behaviors. Instead of modeling the constitutive relations conditioned the entire cloth, decompose the mesh-level energy o(-) into edge-wise energy units ((-), which are modeled by our EUNet as follows:
Σ Gi EE
Op
a))
(2)
p
xf +
x
+ x
Σ E
2h.
2h
where Al, crj = Ees Jey, are the length differences between deformed edge le,, at time t and template edge le., Aet la 1 [ā : Be where [a'., _ and [āe,, Be,,] represent the relative angles between i-th and j-th vertex normals in deformed states and rest states respectively. In practice, we represent all angles 0 by [cos 0, sin 0] for convenience. 7/y is the summation of the areas of the faces with common edge eij. h is the time interval between adjacent frames. Please refer to the Appendix for more details.
The decomposition into energy units holds several advantages. Firstly, the edge-wise representations are agnostic to topologies and highly generalizable, enabling us to utilize simple cloth mesh for learning and directly apply it to various types of garments. Secondly, by conditioning (-(-) on both the lengths and angles, EUNet can detect possible deformations and corresponding potential energy caused by different types of forces in unified manner. For example, the changes in the edge length allow EUNet to approximate the stretching forces, while the angle variations enable EUNet to perceive the surface bending. Here, Φa(-) captures the air damping effects based on the velocities and scales according to the area of the adjacent faces, preventing the collapse of EUNet as demonstrated in the experiments. In addition, EUNet is translation- and rotation-equivalent as the inputs are invariant under different transformations.
To train our EUNet, we start with the approximation of the system's 's total energy between frames at ++ and & as follows:
1 T
2 mm(
Tof. Vg, -m.gg" an
(4)
E +V = E + vtt") + vL+1
1++ + 41+1
(5)
where m, and of are the mass and velocity of the vertex i, g is the gravity, Vp and V2, are the potential energy and dissipation energy, respectively. In discrete systems, estimate the velocities
"https://www blender.oom .rg/
5

---

by 25 - cloth time is converted into the dissipation energy shown in Equation 5. Thus, we can extract the change of the cloth's 's energy as follows:
Due to the presence of damping effects, part of the energy possessed by the
v
VL+1
V
-- E g.6
+ Σ (T) Vaa))
(6)
During training, we sample pairs of adjacent frames, namely t + 1 and t, and apply square error to train our EUNet as follows:
AVo
$ t4+1 + $1+1 $ p AV v++1
VL+
V pt
(7)
p (AVe AV)?
Suu
(8)
Vertex-Wiis Contrastive Loss. Through the approximation, we obtain system-wise scalars as supervision signals, which are the changes potential energy and the dissipation energy. However, different combinations of deformations may result in similar energy potentials in total, leading to EUNet. To improve the robustness and generalization ability EUNet, we construct the vertex-wise contrastive loss through the analysis of the equilibrium of the system. According to backward-Eull Euler commonly used simulating clothes [9, 26, 10], the forward evolution of dynamics follows:
the loss of accuracy
s² loo
mi
mg
(9)
h2
dx;
Equivalently [9], the updates of the dynamics result from minimizing the constructed energy
K( .;)
2h mi(a; - = - af), Vg(:;) -m:g
(10)
E(x) Σ Ka,, 2.) ).)+ + Σ (2;)))
(11)
where a! = x + haf. In other words, by assuming that the ground truth trajectory minimizes the system's 's energy, any disturbance in the system leads to an increase in total energy. Therefore, for the given trajectory X X and random disturbance A the system's 's energy becomes a function of our EUNet & and satisfies,
E(D:X' +XX) E( x²) 0.
(12)
To further deduce the vertex-wist wise relations, we expand the last term in Equation 9. Since the vertex normals are part of the inputs, the change of æ; affects the vertex normals of the nearest neighhorhood vertices j E N, leading to energy variations in edges involving vertices from N₁:
SØe
Σ
Σ
(13)
dx
dx,
JEN, REN,
More details about the energy variations can be found in the Appendix. Similarly, construct the vertex-wiss energy term so that minimizing the per-vertex energy is equivalent to solving Equation
E(x:) = K(x,, 2:) + Vg (2x)) + Σ Σ
desk
(14)
JEN, KEN;
where the disturbed set of vertices i E * are selected so that the minimum distances between vertices are no less than four hops and the rest of the vertices remain unchanged. The per-vertex constraint given ground truth trajectory and disturbance becomes as follows:
E(P x ōσ) x O, E
(15)
During training, we apply the square error to constrain EUNet with Equation 15 by Lcon and construct our final loss term L as follows:
Loon Σ
min min(E(0; + 6x) E(2;
(16)
C
Coup + XLom
(17)
where À is hyperparameter to adjust the weight of vertex-wise contrastive loss.

---

Table 1:
We report the mean and standard deviations of the square errors as indicated in Equation 8. We exhibit the impacts of our dissipation unit 2a and contrastive loss Loon Both of the components enable EUNet ontain lower errors and smaller standard deviations, leading higher accuracy generalization ability.
Configuration
Overall
Silk
Leather
Denim
Cotton
BUNet
85374_143.06
0.9411..4
136.32.8839 839..99
670.55
2.67:8.49
EUNet w/o Dx BUNet w/o Lcon
211.7611052.00
3.51
219.80
001440-1819.00
14.06.136.50
101.11=671.72
0.96
169.45=10200 230.34+852.87
2.38
FUNet
w/o
Com
EUNet
w/o
Angle[Degrae
0.75
0.50
0.5
0.25
Length
Length
Length
0.00
1
1191 Length
Length (mm)
0.0
(a) Potential energy mappings for silk
(b) Potential energy mappings for leather
Figure 3: Visualization of the potential energies predicted by our EUNet either without the dissipation energy branch &s or the contrastive loss term Lcon- We sample the materials of silk and leather for demonstration, and change both the edge length and angles between vertex normals to verify the energy gradients caused by stretching and hending. Since silk is easier to bend, the energy gradients caused by different angles are smaller than those of leather. Both the dissipation energy branch and the contrastive loss enable EUNet to obtain reasonable energy gradients due to the deformations.
3.3 Dynamic Animation
Once our EUNet is trained, we obtain an energy function describing the constitutive relations together with the dissipation from the observed cloth in a unified manner. Following existing work [26, 10], we solve the dynamics of garments in neural-netwole manner, where apply our learned EUNet as constrain through the loss term:
Ldynamic = Linertia Cgrsvity + Lcollision + Lexternal + Le.
(18)
Specifically, Lexternal E: æs is the energy introduced by constant external force fi applied to each vertex i, Lg = is the energy predicted by our EUNet.
Since minimizing the total loss Equation 18 is equivalent to solving the motion equations constrained by backward Euler and our EUNet, the dynamic model can learn the garments' dynamics without any further data and simultaneously inherits the constitutive relations from the observed clothes. More details about the physics loss and the dynamic model can be found in the Appendix.
Experiments
4.1 Learning Constitutive Laws
As mentioned in Section 3.1, we generate 800 sequences of different clothes for training and 200 sequences for test, with a length of 30 frames for each sequence. The cloth is made of 4 types of materials typical in real life: silk, leather, denim, and cotton. The squared cloth is represented by mesh with 484 vertices and 1. 1.4k edges. To build our EUNet, we apply four blocks of multi-layer perceptron (MLP) with dimensions 128. Each block of MI.P consists of 2 fully connected layers. As for the contrastive loss in Equation 16, we apply randomly sampled noises from 10 normal distributions, whose standard deviation ranging from a 10 to a.. Since the noises small and only 10% of nodes are sampled for disturbance, the contrastive loss is relatively small. Thus, we set À = 10 to ensure sufficient regularization. We adopt the Adam optimizer with an initial learning rate of 0.0002, with a decreasing factor of 0.5 every four epochs. The batch size is set to 4. We train our model for six epochs on V100.
We investigate the effectiveness of our EUNet and the impact of the dissipation energy unit & as well as the contrastive loss in Equation 16. Specifically, we evaluate the change of energy as indicated in Equation The final quantitative results are mean of emror from all adjacent frames as shown in Table I. In addition, further visualize the potential energy mappings predicted by
7

---

Table 2: Euclidean error (mm) on sampled Cloth3D [2] with sequence length of 80 frames. The garment-to- human collision rates are displayed under Collision. We use the garment-wist learning pipeline train MGN [24] and LayersNet [28]. We further train a simulator based on MGN constrained by Saint Venant Kirchhoff' (StVK) clastic model and the bending model as mentioned in [26], which is denoted by MGN-S+PHYS. In addition, based on the disentangled scheme, we combine our EUNet with energy optimization scheme and train models denoted by MGN-S+EUNet and +GGNee, where MGN-H is the hierarchical graph neural network adopted in HOOD[10]. Simulators constrained by our EUNet achieve superior performance without
access to the ground truth garments.
Methods
Tshirt
Jumpsuit
Dress
Overall
Collision( %)
MGN [244]
183.40.1 149.59
150.52.111..0
214.071:31.000
770.34=170.92
17.72.17.23
LayersNet [28]
114.74=97.54 54
80.96+27.66
108 088.06-69.74 7.
92.72=60.93
14.61
MGN-S+PHYS
68. 68.03=211.6
127.07=+45.56
103.72=+62.30
106.63=55.48
2.82=1.61
HOOD [10]
73.75=14.82
90.94+200:3
92.57=43.22
84.85=29.93
0.46=0.99
MGN-S+EUNet (Ours)
56.87:1.9.87
69.63:115.82
156.16:187.03
89.71
0.12
MGN-H+EUNe (Ours)
66.70::27.66
56.12:t:22.8
92.83++52.29
66.39_39.36
0.44=0044
EUNet in Figure 3, indicating the constitutive relations resulting from stretching and bending. For the visualized potential energy unit, we vary the angles between vertex normals from -90 to 90 degree and increase the values of Ale from -5mm to 5mm.
As shown in Table 1, our EUNet obtains low errors in predicting the cloth's 's energy with 1.4k energy units, suggesting the effectiveness of capturing the constitutive behaviors. Both the dissipation energy & and the contrastive loss Coon assist EUNet in reducing the quantitative errors.
Moreover, the visualized potential energy mappings in Figure further demonstrate the accuracy of the predictions and the impacts of the two components. A reasonable potential energy mapping typically has the zero point in the rest state and shows an increase in energy with greater deformations. We sample two typical materials, silk and leather, for demonstration. Generally, since silk bends more easily than leather, the energy gradient of silk is smaller than that of leather. The dissipation energy unit crucial in helping the model obtain reasonable gradients based on deformations, preventing the entanglement of damping effects with potential energy. For example, the energy increases as the bending angles increase. While the loss term Loon also leads reasonable gradients, it is more effective in preventing model collapse. Both components contribute to generating reasonable constitutive relations, demonstrating the effectiveness and robustness of our FUNet.
4.2 Animating Garments
To further verify our EUNet and the disentangled scheme learn garments, we evaluate the anima- tions constrained by EUNet on ground truth garments from Cloth3D [2]. Since garments in Cloth3I are also made of silk, leather, denim, and cotton, the constitutive behaviors of the garments and our observed clothes are the same, leading to consistent dynamic deformations given external forces. Therefore, animations constrained by our EUNet should have similar dynamic sequences as the ground truth garments. We measure the Euclidean errors between the ground truth garments and the predictions for each frames, and report the mean and standard deviations of the errors. We also report the collision rates to demonstrate the plausibility of the predictions. The collision rates are obtained by calculating the number of penetrated vertices over the whole garment vertices. The errors for one sequence are averaged over frames. The final results are the mean of errors from all sequences.
As mentioned in Section 3.3, we obtain the garment animations constrained by EUNet through neural network-based simulators, which are MeshGraph (MGN-S) [24] and hierarchical graph neural network (MGN-H) adopted in HOOD [10]. We denote the simulator constrained by our EUNet as MGN-S+EUNet and MGN-H+EUNed in the following. We first compare models with garment-wise learning pipelines, which utilize the mean square errors and collision loss as mentioned in previous work [28] and train two baselines: MGN and LayersNet [28]. Notice that the MGN and MGN-S mentioned above have the same structures. Both MGN and LayersNet are trained using 50K frames of data from Cloth3D, with garments consist of 4k to 1 11k vertices. In addition, to compare the effectiveness of our EUNet with the analytical clothing model, we further train another simulator,
which we denote by MGN-S+PHYS, constrained by the Saint Venant Kirchhoff (StVK) elastic model and the bending model as mentioned [26]. Besides, we follow the work [10] to train the HOOD for comparisons. To ensure a fair comparison, we build mini network to estimate the material
8

---

(Curs)
MENNHEEUNNE (Oum)
Tx20 Tw42 T-B1
THIST
TxQ
T-14 1422
Figure 4: Qualitative results by our disentangled training scheme. We train MGN-S and MGN-H constrained by our EUNet through energy optimization scheme. Since the observed cloth to train EUNet made of the same materials as the ground truth garments, the constitutive relations captured by EUNet are consistent with the ground truth data, As result, MGN-S S and MGN-H constrained by EUNet deliver similar deformation patterns as ground truth garments without accessing any garment data. Even in long-term predictions, we can obtain plausible wrinkles, which are difficult models trained in garment-wise learning pipeline, and robust interactions with the human body.
LayenAtee
MQU-SPPHY
MON
HOOD
Thio
Figure 5: Qualitative results of garment animations by baselines. While the garment- wise learning scheme enables the baselines to obtain reasonable predictions within short period of time, the errors increase for long-term predictions. Though estimate the physics parameters required by the analytical clothing models, MGN+PHYS and HOOD generate overly soft garments and struggle mimic the deformations of the ground truth data.
parameters required for the analytical clothing model to fit garments in Cloth3D, such as the Lame constants for the StVK elastic model. All models are tested using 10K frames of data from Cloth3D. We report the quantitative results in Table 5, and qualitative results in both Figure and Figure 5. More details about the models be found in the Appendix.
As shown Table 5, our EUNet enables the MGN-S and MGN-H to accomplish superior performance compared to models trained through garment- garment-wise learning in most cases, resulting in lower Euclidean errors and collision rates. Furthermore, MGN- -EUNNet and HGGN-++UNNe obtains higher accuracy than MGN-S+PHY' and HOOD constrained by the analytical clothing model, suggesting that our EUNet more effective in capturing the observed constitutive laws than fine-tuning the physics parameters of analytical clothing model.
As shown in Figure 5, models trained garment-wise obtain faithful predictions short sequences tend to generate more artifacts as the sequence length increases due to the accumulated emors. Though the baselines are trained with the given ground truth data, they struggle to determine the relations between constitutive behaviors and dynamic laws. In contrast, shown in Figure 4, MGN-S and

---

MGN-H constrained by EUNet achieve faithful deformations similar to the ground truth garments, suggesting the efficacy of our EUNet in capturing the constitutive relations. Moreover, since we adopt the disentangled training strategy, MGN-S+EUNet and MGN-H+EUNet are able to inherit the robustness from the energy optimization scheme and maintains stable and reasonable animations even for long-term predictions.
In addition, MGN-S+PHY'S and HOOD, which are constrained by the analytical models, obtain overly soft behaviors of garments as shown in Figure 5. Though we estimate the physics parameters needed by the analytical models from the observed clothes, they struggles to deliver the constitutive behaviors of the ground truth garments, suggesting the gap between the analytical clothing model and the observed materials. On the contrary, MGN-S and MGN-H constrained by our EUNet achieve faithful animations with similar deformation patterns of the ground truth, indicating that our EUNet effectively captures the constitutive laws from the observations.
5 Conclusion
In this paper, we apply the disentangled scheme for learning garment animations into capturing constitutive relations from observations and dynamic animations based on energy optimizations. We deduce the variations of cloth energy from the trajectories to train our EUNet, which consist of potential energy unit and dissipation energy unit in topology independent design. To improve the accuracy and robustness of EUNet, we introduce disturbance to the sampled vertices in equilibrium state and constrain EUNet by vertex-wise contrastive loss. With the pre-trained EUNet, we can animate garments of diverse topologies based on the energy optimization, where EUNet serves as the material priors in the form of energy term. Consequently, we achieve more stable and physically plausible garment animations comparing with the baselines. The disentangled scheme alleviates the requirements of large scale of data, and combines the strong abilities of mimicking by EUNet with robust capabilities of predicting dynamic evolution through energy optimization.
Acknowledgement. This study is supported under the RIE2020 Industry Alignment Fund Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contributions from the industry psrtner(s). It is also supported by Singapore MOE AcRF Tier and partially funded by the Shanghai Artificial Intelligence Laboratory.
References
[1] Ellen M Arruda and Mary C Boyce. A three- dimeesiinai constitutive model for the large stretch behavior of rubber elastic materials. JMPS, 1993.
[2] Hugo Bertiche, Meysam Madadi, and Sergio Escalera. CLOTHHD: Clothed 3D humans. In ECCV, 2020.
2, 3, 8, 21
[3] Hugo Bertiche, Meysam Madadi, and Sergio Escalera. PBNS: physically based neural simulation for unsupervised garment pose space deformation. TOG, 2
[4] Hugo Bertiche, Meysam Madadi, Emilio Tylson, and Sergio Escalera. Deepsd: Automatic deep skinning and pose space deformation for 3d garment animation. In /CCV, 2021. 2,
[5] Kiran S. Bhat, Christopher D. Twigg, Jessica K. Hodgins, Pradeep K. Khosla, Zoran Popovic, and Steven M.
Seitz. Estimating cloth simulation parameters from video. In SIGGRAPH, 2003. 2, 4
[6] Robert Bridson, Sebastian Marino, and Ron Fedkiw. Simulation of clothing with folds and wrinkles. In SIGGRAPH, 2005.
Raj Chhabra. Bubbles, drops, and particles in non-Newtonian fluids. CRC press, 2006. 4
[8] Daniel Charles Drucker and William Prager. Soil mechanics and plastic analysis or limit design. Quarterly of applied mathematics, 1952. 4
[9] Theodore F. Gast, Craig A. Schroeder, Alexey Stomakhin, Chenfanfu Jiang, and Joseph M. Teran. Opti-
mization integrator for large time steps. TVCG, 2015. 2, 6
[10] Artur Grigorev, Michael J. Black, and Otmar Hilliges. HOOD; hierarchical graphs for generalized modelling of clothing dynamics. In CVPR, 2023. 2, 3, 6. 7, 8, 13, 14, 15
[11] Funjung Ju, Kwang- Yun Kim, Sungjin Yoon, Eungjune Shim, Gyoo-Chul Kang, Phil Sik Chang, and Myung Geol Choi. Estimating cloth simulation parameters from tag information and cusick drape test. CGF, 2024. 2,
[12] Couro Kane, Jerrold E Marsden, Michael Ortiz, and Matthew West. Variational integrators and the newmark algorithm for conservative and dissipative mechanical systems. International Journal for momerical methods in engineering, 2000. 4
10

---

[13] Dominik K. Klein, Mauricio Fernández, Robert J. Martin, Patrizio Neff. and Oliver Weeger. Polyconvex
anisotropic hyperelasticity with neural networks. CoRR, 2021. 2.4
[14] Minchen Li. Zachary Ferguson, Teseo Schneider, Timothy R. Langlois. Denis Zorin, Daniele Panozzo, Chenfanfu Jiang, and Danny M. Kaufman. Incremental potential contact: intersection-and inversion-free.
large-deformation dynamics. TOG, 2020. 4
[15] Xuan Li, Yadi Cao, Minchen Li, Yin Yang. Craig A. Schroeder. and Chenfanfu Jiang, Plasticitynet: Learning to simulate metal, sand, and snow for optimization time integration. In NewrIPS, 2022. 2, 4
[16] Xuan Li, Minchen Li, and Chenfanfu Jiang. Energetically consistent inelasticity for optimization time integration. TOG, 2022. 4
[17] Yue Li, Stelian Coros, and Bernhard Thomaszewski. Neural metamaterial networks for nonlinear material design. TOG, 2023.
[18] Burigede Liu, Nikola Kovachki, Zongyi Li, Kamyar Azizzadeneshelii Anima Anandkumar, Andrew M Stuart, and Kaushik Bhattacharya. A learning- multiscale method and its application inelastic impact problems. JMPS, 2022. 2,
[19] Pingchuan Ma, Peter Yichen Chen, Bolei Deng, Joshua B. Tenenbaum, Tao Du Chuang Gan, and Wojeiech Matusik. Learning neural constitutive laws from motion observations for generalizable PDE dynamics. In ICML, 2023.
[20] Qianli Ma, Shunsuke Saito, Jinlong Yang, Siyu Tang, and Michael J. Black. SCALE: Modeling clothed humans with surface codec of articulated local elements. In CVPR, 2021. 2,
[21] Qianli Ma. Jinlong Yang. Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, and Michael J.
Black. Learning to dress 3D people in generative clothing. In CVPR, 2020. 2,
[22] Xiaoyu Pan, Jiaming Mai, Xinwei Jiang, Dongxue Tang. Jingxiang Li. Tianjaa Shao, Kun Zhou, Xiaogang Jin. and Dinesh Manocha. Predicting loose-fitting garment deformations using bone-driven motion networks. In SIGGRAPH. 2022. 2.
[23] Chaitanya Patel. Zhouyingcheng Liao, and Gerard Pons-Moll. TailorNet: Predicting clothing in 3D as a function of human pose, shape and garment style. In CVPR, 2020. 2, [24] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzale Gonzalez, and Peter W. Battaglia. Learning mesh-based simulation with graph networks. ICLR, 2021. 8, 13, 15
[25] Carlos Rodríguez-Parte Pardo, Melania Prieto-Martin, Dan Casas, and Elena Garces. How will it drape like?
capturing fabric mechanies from depth images. CGF, 2023. 2, 4
[26] Igor Santesteban, Miguel A. Otaduy, and Dan Casas. SNUG: superviee neural dynamic garments. In CVPR. 2022. 3, 4, 6. 7, 8,8, 13
[27] Igor Santesteban, Nils Thuerey, Miguel A. Otaduy, and Dan Casas. Self sepf-suue collision handling via
generative 3d garment models for virtual try-on. In CVPR, 2021. 2.
[28] Yidi Shao, Chen Change Loy, and Bo Dai. Towards multi-layered 3d garments animation. CoRR, 2023. 2,
3. 8, 14,
[29] Yuelin Shen, K Chandrashekhara, WF Breig, and LR Oliver. Finite element analysis of v-ribbed belts using neural network based hyperelastic material model. International Journal of Non-Linear Mechanics. 2005. 4
[30] Xiao Sun, Bahador Bahmani, Nikolaos N. Vlassis, WaiChing Sun, and Yanxun Xu. Data-driven discovery of interpretable causal relations for deep learning material laws with uncertainty propagation. CoRR, 2021.
2.
[31] Raquel Vidaurre, Igor Santesteban, Elena Garces, and Dan Casas. Fully convolutional graph neural networks for parametric virtual try-on. CGF, 2020. 3.
[32] Nikolaos N Vlassis and WaiChing Sun. Sobolev training of thermodynamic-inttnn inffommm neural networks for interpretable elasto plastiiplat models with level set hardening. Computer Methods in Applied Mechanics and Engineering, 2021. 2, 4
[33] Bin Wang, Yuanmin Deng. Paul G. Kry, Uri M. Ascher, Hui Huang, and Baoquan Chen. Learning elastic constitutive material and damping models. CGF, 2020. 2,4
[34] Huamin Wang, James F. Brien, and Ravi Ramamoorthi. Data-driven elastic models for cloth: modeling and measurement. TOG, 2011. 2,
[35] Tuanfeng Y. Wang, Tianjia Shao, Kai Fu, and Niloy J. Mitra. Learning intrinsic garment space for interactive authoring of garment animation. TOG, 2019, 2,3
[36] Meng Zhang, Duygu Ceylan, and Niloy J. Mitra. Motion guided deep dynamic 3D garments. TOG, 2022.
[37] Qi Zhang, Yilin Chen, Ziyi Yang, and Eric Darve. Multi-constitutive neural network for large deformation poromechanics problem. CoRR, 2020. 2, 4
11

---

T=1
T=7
T=13
T=19
T=25
Leather
Sik
Figure 6: The observed trajectories of cloth for training BUNet. The cloth is made of different materials and pinned by two corners top. We randomly initialize the positions with different velocities for the cloth and let the cloth deform under the influence forces.
n
Bay
dey
Eij
nj
We represent the angles between vertex normals nı and nzj by [a GEgg) where d'y is the rotation angle along edge vector ey from n to the plane defined by nf and eej. Peiy is the angle between nff and the rotated vertex normal within the plane.
Figure 7:
A
Appendix
A.1
Methodology
A.1.1
Data for Training EUNet
We select cloth made of typical materials in real-life: silk, leather, denim, and cotton. The square cloth is pinned at the two corners on top as shown in Figure 6. The data are simulated by Blender in the format of mesh, which are represented by 484 vertices and 1.4K edges. The topology for the cloth is the same for all trajectories. We generate 800 sequences for training and 200 sequences for test. Each sequence include 30 frames of cloth.
A.1.2 Edge- Wise Potential Energy (.))))
As shown in Equation 2, the function takes the length differences Al' E R, the relative angles AOt Biy E TR2 and the attribute vectors E R inputs. Specifically, shown in Figure 7 to reduce the ambiguity, we represent the angles l esy between two vectors 3D space by two separate angles lo 36 For edge vector eij and vertex normals nę and n/. o is the rotation angle along eij from ni; to the plane defined by nf and eij. St is the angle between nē and the rotated vertex normal within the plane.
'htpp:: ://p::///wwww blender.oog .rg/
12

---

(b)
Figure 8: Demonstration of the impact caused by the disturbed vertex. Suppose figure (a) is the ground truth position for each vertex time 1. When adding noisc to the vertex the center, the vertex normals of the surrounding orange vertices j Ni are disturbed. As result, energy units edges, which connect both vertex and vertices j E Nr and are marked by orange, change accordingly.
A.1.3 Vertex- Wise Contrastive Loss
Here explain how the change of node affects the corresponding edges. The change of the position z; affects the vertex normals of the nearest neighborhood vertices j E N,s as denoted by the orange nodes in Figure 8. Consequently, the energy for edges, marked by orange in Figure 8, is also changed.
A.1.4 Loss Terms for the Energy Optimization
As mentioned in the main text, the total loss to train a model constrained by our EUNet is as follows:
Ldynamic Linertia Leravity feallission + Lexternal + L.v
(19)
where Le + is the predicted energy by our EUNet. Specifically, Linortia and Cgravity the inertia and gravity energies:
a = x
+ hu!
(20)
Linertia (x,#')
Σ
2h mi(a; T Tx;; z!)
1
(21)
Cgravity (x)
Σ
m.g xi.
(22)
where h is the time interval between adjacent frames. The collision term aims to reduce the penetra- tions between garments and human bodies and has a similar format as existing work [26, 10]:
d(x;, Fo)
(x; xo) Tho max(c d(æ,, xo)) 0)3
(23)
Leollision
(24)
where v. is the position of the o-th face center from the human mesh at time + 1 and nı is the face normal. e a safe boundary and is set as 0.002 in practice. The index of o indicates the closest face center to the vertex x within range of 0.03mm at time Additionally, external forces if besides the gravity to each vertex and penalize the corresponding energies:
randomly apply constant
Lexternal (I:)
ESTar. xz.
(25)
A.2 Experiments
A.2.1 Implementation Details for Models
MeshGraphNet (MGN) [24]. The MGNs used for the disentangled scheme and garment- training pipelines are the same. Specifically, the MGN is formatted as follows:
++1
F(M, Mr, M(+1 a. g.
(26)
m,
where 2+1 are the predicted accelerations for each vertex at time + 1, M = (T, E) represents the garment template mesh with corresponding vertices T and E edges. M³ (V.. E.) and
13

---

Table 3: Model configurations. We mark the MGN and LayersNet trained in supervised manner by "Supervised". We denote the HOOD's hierarchical graph neural network by MGN-H. The "Friction" loss term from HOOD introduces the frictions between human body and garments.
Self-Supervised Loss Components
Methods
Neural Network Structure
PHYS Friction
EUNet (ours)
MGN (Supervised)
MGN
LayersNet (Supervised)
NayersNet
NGN-S+PHYS
MGN
MGN-S+EUNG
MGN
MGN-S+PHYS+F
MGN
MGN-H-PHYS+F (HOOD)
MGN-H
MGN-H+EUNet
MGN-H
Mg+ (V++1, E,} are the meshes for the underlying human bodies time / and 1+ respectively, The predicted positions for garment vertices at time + are calculated as follows:
6++1 r² + hát+1
(27)
*++
x² + ho++1
(28)
The input for the MGN has similar format as [10]. In addition, we concatenate the accelerations caused by constant forces as part of the vertex feature.
f
external =
(29)
As for the MGN-S constrained by analytical clothing model, we build a mini-network to approximate the physics parameters from the observed garments in Cloth3d to fit the clothing model.
m, μ4, A = F(m. a).
(30)
where m is the mass for the analytical clothing model. and À are the lame constants for the StVK elastic model. is the constant for the bending model. m is the mass from the Cloth3D. is the attribute vector mentioned in Cloth3D, such as the tension stiffness, bending stiffness, and the tightness. All the input and output attributes are normalized according to the minimum and maximum values. The mini-network consist of 3 layers of multi-layer perceptrons with hidden dimension 64. We train the mini-nette neti-network with initial learning rate 0.0002 and batch size 4 for 4 epochs. We sample 10k frames from Cloth3D for training. We adopt Adam optimizer with decreasing factor of 0.5 for each epoch.
LayersNet [28]. We followed the official work [28] to train the model on Cloth3D dataset. Specifi- cally, we adopt the mean square errors as loss term to penalize the differences between the predicted positions and ground truth data. We also apply the collision loss mentioned in the work [28] to solve the collision between human body. Since we do not consider multiple layers of garments, we only penalize the collisions between the single-layered garments and human body mesh.
All models are trained on V100.
A.2.2 Experiment Results of Garment Animation
In this section, we report extra quantitative and qualitative results in Table 4 and Figure 9, We summarize the models architectures and the loss terms used in training process in Table 3.
As shown in Table 4, simulators constrained by our EUNet deliver superior performance than baselines, suggesting the effectiveness of our EUNet.
In addition, we demonstrate the effectiveness of bending forces captured by our EUNetas shown in Figure 9 Specifically, we force EUNet not to generate energy gradients when bending, and train MGN as simulator. Without the bending forces, the animations tend to generate more wrinkles but different deformations comparing with the ground truth data.
14

---

Table 4: Euclidean error (mm) on sampled Cloth3D with sequence length of 80 frames. Comparing "MGN-S+PHYS" with "MGN-S+PHYS++* overall performance. Comparing "HOOD w/o EST", which does not use the mini-network to estimate the material parameters as mentioned in Section 4.2, with "HOOD", the mini-network is necessary to improve the accuracy for simulators constrained by analytical clothing model. Comparing "HOOD" with "MGN-H+ H+EUNet (Ours)" and "MGN-S+PHYS with "MGN-S+EUNet (Ours)", simulators constrained by our EUNet achieves lower errors, suggesting the effectiveness of EUNet.
the friction loss term does not significantly affect the
Methods
Tshirt
Jumpsuit
Dress
Overall
Collision( %
MGN [24]
183.40+ 149.59
150.52+111.0 .04
214.07++31.00 .00
170.34-170.2
17.72+7.23
LayersNet [28]
114.74+97.54
80.96+:27.69
108.06 74
92.72=60.93
14.61+6.71
MGN-S+PHYS
68.03:1:21.66
127.07:145.56
103.72:162.30
106.631555.48
2.82111.61
MGN-S+PHYY$#F
76.58.1.9.38
130.42:145.76
101.47:141.27
108.83
1.92.1
HOOD w/o EST
111.27:122.84
182.89.166.92
163.90-164.87
153.78163.75
0.36.10.27
HOOD [10]
73.75=11.882
90.94+20:43
92.57-+43.26
84,85=29.93
0.46=00.99
MGN-S+EUNet (Ours)
16.87
65.63
156.16:187.05
89.711147.83
0.12.10.12
MGN-H+EUNNe (Ours)
66.70:1:7..6
56.12.1:22.8
92.83:152.29
66.39
0.44.10.48
Time efficiency comparisons between analytical models and our EUNet. We denote the StVK elastic model and the bending model by "PHYS", which is used in the main text. As formulated in Equation 1, our EUNet is composed of two separate branches: @p for potential energy and a for dissipation. Both branches have the same structures. We report the time separately for each branch and the full EUNet as follows. The forward time averaged on 80 frames of predictions, which include garments composed of 7924 vertices, 23636 edges and 15712 faces. All experiments are run on NVIDIA A100-SXM4-80GB.
Table 5:
PHYS
EUNet Φp EUNet & EUNet 4p+Da
Time (ms) 1.743+0.212 1.055+0.202 1.21640.223
2.271+0.422
A.2.3 Time Efficiency Comparisons
We report the time efficiency in Table 5. We compare the analytical models, which are the StVK elastic model and bending model adopted by the baselines, and our EUNet. As formulated in Equation 1, our EUNet is composed of two separate branches: Φ for potential energy and & for dissipation. Both branches have the same structures. We report the time separately for each branch and the full EUNet as follows. The forward time is averaged on 80 frames of predictions, which include garments composed of 7924 vertices, 23636 edges and 15712 faces. All experiments are run on NVIDIA A A100-SXM4-80GI.
As shown in Table 5, EUNet is comparable to the traditional clothing models in terms of speed.
A.3 Limitations
possible limitation is that our EUNet is based on the edge-wise discretization of the continuous mesh. While polygonal discretization would potentially improve the performance, construct the polygon- wise energy unit by the summation of the edge-wise units. Another limitation is that our EUNet does not consider the self-collisions during training. Since do not require the trajectories needed to train EUNet to be very complex, we can adopt and generate simple cloth motions, such the pinned cloth falling under the influence of gravity, avoid the self-collisions in the training data. As demonstrated in our work, our EUNet effectively captures the constitutive behaviors with the simple motions of the cloth.
also
15

---

NONNG-EUNNNOON 10.m)
1000
MON
We display the qualitative results for all models. Simulators constrained by EUNet deliver dynamic patterns closer to the ground truth. We further demonstrate effectiveness of the bending forces captured by our EUNet, which is shown on the second row. Without the bending forces, the MGN-S+EUNG. tends to generate more wrinkles but different deformation patterns as the ground truth garments.
Figure 9:
16

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: As shown in abstraction Section, abstract summarize paper and include the main contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Please refer Section A.3.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Please refer to Section 3 and Section A.1.3. We also cite the corresponding paper after the assumptions and theories.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please refer to Section and Section A.2.1. The code and model will be publicly available.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
18

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The data used for training our model not one of the main contributions. But we will consider releasing the data if necessary. Code is publicly available at https: / //gi githhubccom/ BaaiiaabieEENNeeeeuuurrre
git.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ce cc/
cublic/guides wttgg/iie/oodeuuiiieee Sotps//ooeSSSSssssSSeeeee wwt.iiisssioooiiyooo.oooii for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ maips.ccppublic. wwwbbbiccguiieeee httpa//gguieeee/eeuuiiiieeeeeee
/CodeSubmiieeeee www.simmissiooooooooooooo onpiiicy) cy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Please refer to Section 4 and Section A.2.1. Code will be publicly available.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Please refer Section 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please refer to Section A.2.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't It make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https : http///eurrpp..eee
Answer: [Yes]
Justification: Our work conforms the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Our work focuses on the cloth simulation and animation, which has little potential societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
20

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [No]
Justification: Since we focus on cloth simulation and animation, our model has low risk for misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: In our work, we only adopt Cloth3D [2], which is publicly available. We cite the original paper when first mentioning this dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not have any new assets to release.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification:
The work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [No]
Justification: The work does involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
23


Reviews:
[{"review_id": "Qu8ZgCzB3Q", "reviewer": "Reviewer_Hyzo", "summary": "This paper proposes to learn garment dynamics using a disentangled learning framework and the Energy Unit Network (EUNet). Instead of relying on extensive garment datasets, the approach learns constitutive behaviors from a single cloth piece and dynamically animates garments through energy optimization.", "questions": "1. Literature Positioning: Can you clarify your awareness and positioning of your method in relation to the existing body of work on learning/recovering cloth dynamics from structured sample tests? As discussed above, many significant studies in this area were not referenced.\n2. Physics Simulation/Graphics Community References: The literature review section did not include important works from the physics simulation and graphics community. How does your approach relate to or differ from the significant contributions in this field? Including a discussion of this could provide a better contextual grounding for your work.\n3. Experimental Validation: Can you provide more details on how your experiments validate the proposed method against these existing works? Specific comparisons and metrics would help clarify the effectiveness and novelty of your approach. Can you validate your approach against different datasets, including synthetic datasets generated from different simulation engines as well as real-world datasets used by current works?", "limitations": "The authors provided a discussion on the limitations of edge-wise discretization and a lack of self-collision handling.", "rating": 4, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "The writing is clear and technical details are described clearly.  The visual aids and diagrams are well-integrated, enhancing understanding.", "weaknesses": "My main problem with the paper is that the problem of learning/recovering cloth dynamics from structured sample tests has been studied intensively for a long time, and the authors seem not to be aware of this whole field.  This is a well-studied problem, and the authors do not position their method against significant prior works. Many existing works have also attempted learning from real-world fabric sample tests or indirect representations (video), which is a much harder problem. To mention a few:\n\n1. \"Predicting the Drape of Woven Cloth Using Interacting Particles\" Breen et al., 1994\n2. \"Estimating Cloth Simulation Parameters from Video\" Bhat et al., 2003\n3. \"Data-driven elastic models for cloth: Modeling and measurement\" Wang et al., 2011\n4. \"How Will It Drape Like? Capturing Fabric Mechanics from Depth Images\" Rodriguez-Pardo et al., 2023\n5. \"Estimating Cloth Simulation Parameters From Tag Information and Cusick Drape Test\" Ju et al., 2024\n\nThe authors should thoroughly review the literature and reposition their contribution and provide experimental comparisons against existing works.  Additionally, the literature review section does not include important works from the physics simulation community. Including these references and discussing how the proposed method builds upon or differs from them would strengthen the paper significantly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed feedback.\n\n# W1: Insufficient Literature Review\nWe argue that our focus is quite different from **Physics Parameter Estimation**.\nThe misunderstanding may come from the similarity in data format,\nwhere a piece of hanged cloth deforms given external forces.\nUnlike physics parameter estimation,\nour EUNet aims at learning the unknown **ENERGY FUNCTION** that describes the constitutive laws. While in physics parameter estimation, the constitutive models are **known a priori and carefully chosen**.\n\nSpecifically,\nwith known constitutive models, physics parameter estimation aims at learning physics parameters, such as the lame constant in St. Venant-Kirchhoff elastic model for the target material.\nComplex devices and procedures, such as KES-F [1], the FAST system [2], and the Cusick drape test [3] are commonly required to estimate the parameters that best fit the analytical cloth models or simulators.\nMoreover,\nfor a specific material, such as cotton,\ndifferent clothing models or simulators require separate procedures or pipelines to estimate different sets of physics parameters.\n\nIn contrast,\nour EUNet directly captures constitutive behaviors of different materials, which takes deformation attributes (i.e., delta of length and bending theta) and material attributes (i.e., stiffness and damping coefficients) as input, \nand outputs the corresponding potential energy.\nThe material attributes remain constant in our study.\nAs discussed at L84-88,\nour EUNet does not need existing analytical physical models and physics parameter estimation. \n\nIn terms of literature review,\nas mentioned above we focus on learning to animate garments where modeling constitutive behaviors is the key challenge,\nwe thus mainly discuss the garment animation and constitutive laws in the literature review section at L108-126 and L127-136 respectively.\nEnergy-based physics simulation serves as a tool to generate dynamics constrained by our EUNet,\nand we briefly introduce the corresponding techniques at L137-144.\n\nWe will clarify the relationship between our focus and physics parameter estimation in the revision.\n\n[1]. Kawabatak, et al. Fabric performance in clothing and clothing manufacture. Journal of the Textile Institute. 1989\n\n[2]. Minazio, et al. FAST–fabric assurance by simple testing. IJCST1995\n\n[3]. Cusick, et al. The dependence of fabric drape on bending and shear stiffness. Journal of the Textile Institute Transactions 1965.\n\n# Q1: Literature Positioning\nPlease refer to W1.\n\n# Q2: Physics Simulation/Graphics Community References\nAs discussed in W1,\nour focus is on how to learn the constitutive laws,\nwhich is discussed at L127-136 and involves several references from graphics community.\nWe verify our EUNet by applying existing physics simulation techniques,\nwhich we briefly discussed at L137-144.\n\n# Q3: Experimental Validation\nAs discussed in W1,\nour work is orthogonal to physics parameter estimation,\nthus comparing with methods mentioned by the reviewer is unnecessary.\n\nIn addition,\nas discussed in W1 and Q2,\nour goal is to faithfully animate garments with observed materials where modeling constitutive relations is the key challenge.\nWe thus verify the effectiveness of our EUNet by integrating with simulation techniques and evaluate the garments animations on Cloth3D [1], which is a large scale dataset and commonly used in garment animations, such as DeePSD [2] and MotionGuided [3].\n\n[1] Hugo Bertiche, et al. CLOTH3D: Clothed 3D Humans. ECCV2020\n\n[2] Hugo Bertiche, et al. DeePSD: Automatic deep skinning and pose space deformation for 3D garment animation. ICCV2021\n\n[3] Meng Zhang, et al.  Motion guided deep dynamic 3D garments. TOG2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LWM8LI6ZjA", "reviewer": "Reviewer_TX4H", "summary": "The paper proposes a novel method for animating garments by learning from a single piece of cloth. This approach circumvents the need for large-scale garment datasets, which are resource-intensive and time-consuming to create. The core idea is to use a disentangled scheme where constitutive behaviors are learned from observed cloth and then applied to animate various garments. The proposed Energy Unit Network (EUNet) captures constitutive relations in the form of energy, bypassing the need for traditional physics models.", "questions": "How does the performance of EUNet compare with traditional physics-based models in terms of computational efficiency and accuracy?\n\nWhat are the limitations of using a single piece of cloth for training, and how can these limitations be mitigated in future work?", "limitations": "Comparisons with a broader range of existing methods would be appreciated, for example, one of the SOTA method named 'HOOD: hierarchical graphs for generalized modelling of clothing dynamics' which has been cited in the paper is worth comparing with.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper introduces a novel disentangled approach that separates the learning of constitutive behaviors from the animation process.\n\n The EUNet models constitutive behaviors using energy units, allowing for direct learning from observed cloth trajectories without traditional physics models.\n\nThe approach significantly reduces the data requirement, relying on a single piece of cloth for training, making it more practical and less resource-intensive.\n\nThe method produces animations that are both robust and generalizable, capable of handling various garment types and materials.", "weaknesses": "The energy optimization process, although effective, can be computationally intensive and may require fine-tuning to achieve optimal results.\n\nThe paper would benefit from more extensive experimental validation, including comparisons with a broader range of existing methods and more diverse garment types.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive and valuable feedback.\n\n# W1: Computationally Intensive Energy Optimization Process\nIn this paper,\nwe primarily focus on the challenge of modeling constitutive relations from observations.\nThe energy-based simulation is used solely as a tool to solve the dynamics constrained by our EUNet.\nConsequently,\naccelerating the energy optimization process is beyond the scope of our work.\n\n# W2: Extensive Experimental Validation\nAs requested by the reviewer,\nwe implement HOOD [1] for further comparisons.\nMoreover,\nthe baselines we have chosen are representative:\nMGN is the pioneering approach for animating mesh-based cloth,\nLayersNet achieves state-of-the-art performance for garment animation trained in a supervised manner,\nand MGN-S+PHYS employs a recent advanced unsupervised learning scheme as shown in SNUG [2] and HOOD.\nThe main difference between MGN-S+PHYS and HOOD is that HOOD adopts a hierarchical graph neural network.\nPlease refer to the additional PDF in the global rebuttal for more details.\nSimulators constrained by our EUNet achieve superior performance over other baselines.\n\nIn addition,\nall garments in Cloth3D [3] differ from each other in terms of length, size, and topology.\nEach category in Table 2 of the main manuscript includes several subclasses.\nFor example,\nthe \"T-shirt\" category also includes \"Jackets\" as shown in Figure 1 of the main manuscript,\nand the \"Dress\" category includes both short and long dresses.\n\"Jumpsuits\" and \"Dresses\" are combinations of upper and lower garments.\n\n[1] Artur Grigorev, et al. HOOD: hierarchical graphs for generalized modelling of clothing dynamics. CVPR2023\n\n[2] Igor Santesteban, et al. SNUG: Self-Supervised Neural Dynamic Garments. CVPR2022\n\n[3] Hugo Bertiche, et al. CLOTH3D: Clothed 3D Humans. ECCV2020\n\n# Q1: Efficiency and Accuracy Comparing with Physics-based Models\nAs shown in global rebuttal,\nour EUNet is comparable to the traditional clothing model in terms of speed.\nWhile our EUNet contains two branch: $\\Phi(\\cdot)$ for potential energy and $\\Phi(\\cdot)$ for dissipation,\neach branch is slightly faster than the traditional clothing model.\n\nIn terms of accuracy,\nas shown in Table 2 of both the main manuscriptand the global PDF, simulators constrained by our EUNet achieves higher accuracy than those constrained by physics-based clothing models.\nFurthermore,\nour method reduces the need to carefully select different types of physics-based clothing models and estimate the corresponding parameters.\nInstead,\nour EUNet directly captures the observed constitutive laws,\nenabling learning-based simulators to achieve superior performance in garment animation.\n\n\n# Q2: Limitations of Using A Piece of Cloth\nOne limitation is that a single piece of cloth may not encompass all possible deformations and corresponding dynamics.\nFor example,\nshear deformation may be less obvious,\nand interactions among different layers of clothing are unavailable.\nA simple solution to enrich the deformations and dynamics is to apply known forces,\nsuch as those controlled by robots,\nand to use multi-layered clothing during data generation to explore these interactions\n\nOn the other hand,\nwe intentionally design the training data to be as simple as possible\nto ensure its applicability in real scenarios.\nAn example is the data from [1],\nwhich can be directly adopted by our model but currently unavailable.\n\n[1] David Blanco Mulero, et al. Benchmarking the Sim-to-Real Gap in Cloth Manipulation. IEEE Robotics Autom2024\n\n# Limitations: Comparison with HOOD\nPlease refer to W2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GeqUseexGi", "reviewer": "Reviewer_QaVe", "summary": "This work proposes a method to learn the constitutive model of cloth materials from observed cloth trajectory using a neural network. It adopts an MLP that operates on individual edges and predicts per-edge distortion based on the deviation of edge geometry from rest shape and trains the network using a combination of supervision on potential energy change with ground truth and optimality of incremental potential. The learned potential energy can be used as a constraint to train neural simulators for garment animation.", "questions": "See the weaknesses section.", "limitations": "The discussion seems adequate.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- I appreciate the novelty in the idea of learning the constitutive model of cloth materials in a data-driven manner. Potentially this formulation could allow the neural networks to understand the intrinsic physical property instead of mimicking the behavior of specific examples, thus of scientific significance if implemented correctly.\n- The paper is well-written and mostly clear.", "weaknesses": "- On the methodology side, the major question is probably the design of dissipative energy. On the one hand, why it is and is merely a function of \\(X^t - X^{t-1}\\) is questionable. In fact, whether it should be modeled as an absolute quantity is a question because the total amount of dissipative energy seems not that meaningful. The only observable quantity is the relative change of dissipative energy in a physical process.\n- On the other hand, with the presented framework, it is very hard to learn the major sources of energy dissipation: collision, and friction, since they are neither present in the training data, nor fully modeled (e.g. self-collision) in the formulation. While the dissipative energy is not the focus, the problem is that without correctly modeling dissipative forces, I doubt the possibility of learning an accurate elastic potential energy function.\n- On the evaluation side, the problem is that the method is only evaluated in a simplified setting, without comparing against methods or in settings that are practically useful (see below). In my opinion, there are two ways to demonstrate that the learned constitutive model is useful: either 1. demonstrate that it is more accurate than an analytical model on real data, or 2. show that it leads to more realistic animation than existing methods (including traditional numerical models).\n- The evaluation section only shows that the MGN trained with the learned constitutive model is better than those trained with ground truth garments or analytical constitutive model. On the one hand, it does not compare with other state-of-the-arts like HOOD, SNUG, and PBNS that are also formulated in a self-supervised manner. On the other hand, the claim that it is better than the analytical constitutive model is not convincing because the discrepancy may be caused by the limited accuracy in the neural simulator (or even by the mini-network mentioned in Sec 3.3). To truly demonstrate that it is better than an analytical one, it must be compared using a numerical integrator that is guaranteed to converge to the energy minimum.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback.\n\n# W1: Design of Dissipative Energy\nAs commonly adopted in physics simulation,\nsuch as the Rayleigh dissipation,\nthe dissipation can be approximated by a function of objects' velocities,\nwhich can be calculated from $X^t-X^{t-1}$ in our work.\n\nIn addition,\nwe agree that the absolute value of the dissipative energy is not meaningful.\nHowever,\nduring the energy optimization process of solving dynamics,\nonly the derivative of the dissipation energy $\\lim_{\\Delta x \\to 0}\\frac{\\Phi_d(x+\\Delta x, \\cdot)-\\Phi_d(x, \\cdot)}{\\Delta x}$ will take effect,\nand eliminate the potential impacts from the absolute values.\n\nLastly,\nif there exists a constant $C>0$ in the output of our $\\Phi_d$,\nit indicates that the system from Blender has a constant damping effect.\nWhen the velocity is 0, the output of $\\Phi_d$ is $C=3\\times 10^{-6}$, which is small enough to be negligible.\n\n# W2: Dealing with Collision And Friction\nWe include the frictions between cloth and air, which is common in daily life,\nin both the generated data and modeling process.\nAs shown in Table 1 and Figure 3 in the main text,\nwith the modeling of dissipation by $\\Phi_d$,\nour EUNet achieves lower errors\nand delivers more reasonable energy mappings as the cloth deforms,\nsuggesting the effectiveness of $\\Phi_d$.\n\nIn addition,\nsince collision forces are independent of the constitutive behaviors,\nwe do not model collisions as part of our EUNet.\nWe avoid the self-collisions in training data by setting an upper bound on the initial cloth's velocities and the range of their directions.\n\nQuantitative results in Table 1 of the main manuscript\nand the performance on garment animations demonstrate the effectiveness of our EUNet.\n\n# W3: More Evaluations to Verify Learned Constitutive Model\nSince the final goal is learning to animate the garments\nwhere EUNet is key module of this framework,\nwe conduct comprehensive experiments on garment animations in Table 2 and Figure 4 in the main text.\n\nIn addition,\nwe further implement HOOD [1] as an extra baseline.\nPlease refer to the PDF in global rebuttal for more details about the performances and the differences between baselines.\nAs shown in Table 2 of both main manuscript and the global PDF,\neither MGN-S or MGN-H (used in HOOD) being the neural simulator architecture,\nsimulators constrained by our EUNet always outperforms baselines (i.e., HOOD and others in the Table 2 of the global PDF),\nindicating the effectiveness of our EUNet.\n\n[1] Artur Grigorev, et al. HOOD: hierarchical graphs for generalized modelling of clothing dynamics. CVPR2023\n\n# W4: Comparisons with HOOD and Analytical Clothing Model\nFirstly,\nas discussed in W3,\nwe further compare with HOOD, where simulators constrained by our EUNet outperforms it.\n\nAs for the claim 'better than the analytical constitutive model', \nit is worth noting that we didn't claim it in abstract and introduction. We only mentioned this in our experimental section as an empirical analysis at L298-299. We will clarify this in the revision.\n\nMoreover,\nto further verify the advance of our EUNet over analytical models is not caused by the limited accuracy of neural simulator,\nwe pair our EUNet and the analytical models with two different neural simulator architectures, namely MGN-S and MGN-H.\nAs a result,\nsimulators constrained by our EUNet always achieve lower errors,\nwhich means adopting our disentangled learning scheme with EUNet is better than using analytical models to train neural simulators. \n\nThirdly,\nwe further report the comparisons that with or without the mini-network in Table 2 from the global PDF.\nThe mini-network enables the unsupervised simulators,\nwhich are constrained by analytical physics model,\nto obtain higher accuracy.\n\nFinally,\nwe admit that implementing numerical simulations or PDE solvers within a short period is challenging. However,\nwithin the scope of learning to animate garments\nand based on the experiments, \nour method can better enhance learning-based simulators\nto achieve lower errors and faithful animations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eK0pObHgUC", "reviewer": "Reviewer_CDgw", "summary": "This submission presents a method that could effectively learn the dynamic patterns of different garments from a single piece of cloth. The key insight is that the motion of different cloths is governed by both external forces and the constitutive relations rather than specific garment topologies. Thus, an Energy Unit Network (EUNet) is proposed to learn the topology independent constitutive laws. Then the animation is performed by energy optimizations. Experimental results shows improved results comparing to previous methods and baseline methods.", "questions": "See weakness.", "limitations": "See weakness.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is well written and easy to read.\n1)\tThe paper is well structured.\n2)\tMany terms are well defined and explained.\nThe proposed method is both novel and interesting.\n1)\tThe disentangled learning scheme of using a network to learn the constitutive law, which generalizes to different garment types, is physically intuitive and natural. More importantly, this design helps alleviate the needs for large amount of training data of various cloth shapes in dynamic for learning based animation.  This disentanglement between topology and energy is achieved by using mesh edge as a unit instead of the whole cloth mesh.\n2)\tThe proposed disturbance training strategy helps stabilize the training and improves the generalization of EUNet. As a constraint, it accompanies the direct supervision on the energy form by taking into account the physical meaning of equilibrium state. This helps the network to learn a more reasonable manifold of the energy distribution.\nExperimental results:\n1)\tImproved results over previous methods are shown both qualitatively and quantitatively.\n2)\tAccording to the ablation study, the design of both the contrastive loss and dissipation unit loss are validated as effective.", "weaknesses": "Some details regarding the design of the EUNet is missing. Although some descripsions on the EUNet design is provided at the experimental section, it is relatively hard for the reader to follow and develop a more coherent understanding of the presented work.\n\nSome limitations and open questions that the work might not cover:\nWhat about anisotropic materials? How to adapt the current model design to also fit to cloths where its material is anisotropic? How well does the method handles cloths with more complex topology that goes beyond a single layer of cloth? As also pointed by the authors, the method does not handle self-collision. It would be interesting to see how it can be adapted in that axis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the positive feedback\nand recognizing the value of our work.\nWe believe the reviewer has sufficient understandings of our framework and pipeline.\n\n# W1: Details Regarding The Design of EUNet\nWe introduce the formulations and training procedures of our EUNet in Section 3.2,\nand include the implementation details in Section 4.1 at L239-249.\n\nFurthermore,\nwe report the time efficiency of our EUNet in the global rebuttal.\n\nPlease let us know if the reviewer still has any confusion about our EUNet.\nWe will clarify it further in our revised version to ensure it is easier for readers to follow.\n\n# W2.1: Anisotropic Materials\nA naive extension is to add the directional information as extra inputs for our EUNet.\nAs a simplified example,\nwe assume a rectangular wooden board that is thin enough for its thickness to be negligible.\nThe board is stiffer along the x-axis than the y-axis. \nWe then discretize the thin board by edges along both the x-axis and y-axis.\nTo model such material,\nwe can extend our EUNet by taking the edges' directions,\nwhich are either x-axis or y-axis in this example,\nas extra inputs by $\\Phi(\\cdots, \\mathbf{d})$,\nwhere $\\mathbf{d}$ indicates the directional information.\n\n# W2.2: More Complex Topology Than A Piece of Cloth\nFor cloth or garments with complex topologies used as training data,\nour EUNet can directly learn their constitutive behaviors thanks to the topology-independent design.\nTo achieve accurate results,\nit is essential to ensure that the training sequences involve minimal collisions,\nso that the dynamics and deformations are primarily caused by internal forces closely related to potential energy.\n\nThe reason we chose to use a simple piece of cloth as training data is that this setup is easier to achieve with high quality in real-life scenarios.\nFor example,\n[1] provides the temporally consistent cloth data represented by vertices from real life,\nwhich can be directly applied by our EUNet but not publicly available yet.\n\n[1] David Blanco Mulero, et al. Benchmarking the Sim-to-Real Gap in Cloth Manipulation. IEEE Robotics Autom2024\n\n# W2.3: Handling of Self-collisions\nSince collisions are independent of the internal forces caused by deformation,\nthe key to accurately modeling potential energy is to minimize the effects of collisions.\nHowever,\nto extend our method to more complex scenarios involving collisions,\none could design a new module that takes the distances between edges or faces of the cloth as inputs\nto model the dissipation energy caused by collisions.\n\nIn this work,\nwe minimize the effects of collisions during the data generation process.\nWe will include such scenarios in future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x8iEhXVfVx", "reviewer": "Reviewer_cPXz", "summary": "The authors propose a method to transfer the deformations of the observed garments to any other garment. Previous methods either rely on a large-scale dataset for training or analytical physics model with limited expressive ability. On the contrast, the proposed method first learns the constitutive relations from the observation by a neural network (EUNet), then use it as an energy prior to regularize the training of garment deformation model. This design addresses the limitation of previous works and show better results.", "questions": "1. In Fig. 4, the results of MGN-S+EUNet on dress are not similar to the ground truth data. \n2. What is the unit of the errors in Table 1? The errors on the leather and denim seem too big compared with the others.", "limitations": "See the weakness.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The strength of the paper is the proposed method does not need to collect huge amount of data with varied body pose and shape and garment types for training. Through theoretical analysis, they prove that they can learn a more physically accurate energy model to describe the deformation of garment. In this way, they do not need an explicit physical model, which tends to have limited expressive power. The derivation is theoretical sound.", "weaknesses": "To learn this energy model by EUNet, the authors rely on the synthetic data simulated with blender. A cloth with known geometry (vertices and faces) is assigned with a specific material type. However, this setting is too ideal. In real scenarios, we are more interested in transferring the material of a real cloth to another garment. But having the geometry of a real cloth usually is not feasible. Even though we can have the mesh of the cloth through some registration process, how to get the shape of mesh when it is hanging and dangling is still a problem. In this paper, I do not see the possibility of using the proposed method in real applications. This is the critical weakness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback.\n\n# W1: Possibility of Using The Method in Real Applications\nFirstly,\nsynthetic data is commonly used and facilitate the research of garment animations,\nsuch as TailorsNet [1], Cloth3D [2], MotionGuided [3], Clo3D [4].\n\nSecondly,\nas shown in the dataset [5],\nit is feasible to obtain temporally consistent cloth data represented by vertices from real life, which can be directly applied by our method to learn the constitutive laws in real world, though it is not yet publicly available.\n\nThirdly,\nthe garment geometry can be extracted either through learning based methods [6, 7] or from raw scans [8],\nmaking it possible to animate realistic garments.\nAs shown in Figure 2 in additional PDF from global rebuttal,\nwe animate the garments from real scans [8] for demonstration,\ntransferring the virtual material from Blender to real-life garments from CAPE [8].\n\nIn conclusion,\nour work can be directly applied to real-life applications related to garment animations.\n\n\n[1] Chaitanya Patel, et al. TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. CVPR2020\n\n[2] Hugo Bertiche, et al. CLOTH3D: Clothed 3D Humans. ECCV2020\n\n[3] Meng Zhang, et al.  Motion guided deep dynamic 3D garments. TOG2022\n\n[4] Xingxing Zou, et al. Cloth4d: A dataset for clothed human reconstruction. CVPR2023\n\n[5] David Blanco Mulero, et al. Benchmarking the Sim-to-Real Gap in Cloth Manipulation. IEEE Robotics Autom2024\n\n[6] Heming Zhu, et al. Registering Explicit to Implicit: Towards High-Fidelity Garment mesh Reconstruction from Single. CVPR2022.\n\n[7] Lingteng Qiu, et al. REC-MV: REconstructing 3D Dynamic Cloth from Monocular Videos. CVPR2023\n\n[8] Qianli Ma, et al. Learning to Dress 3D People in Generative Clothing. CVPR2020\n\n# Q1: Qualitative Results of Dress\nWe acknowledge that predicting long-term dynamics in an auto-regressive manner inevitably leads to error accumulation.\nThe baseline models either struggle with predicting long-term animations or fail to replicate similar constitutive behaviors observed in the ground truth.\n\nIn contrast,\nMGN-S+EUNet (ours) delivers material behaviors that more closely match the ground truth,\nsuch as a more rigid dress in Figure 4 of the main manuscript,\nand achieves lower prediction errors as shown in Table 2 of the main manuscript,\nsuggesting the effectiveness of our learned EUNet.\nPlease refer to the video in supplementary for better comparisons\nand the additional PDF file for more comparisons.\n\n# Q2: Explanations of Errors in Table 1 of The Main Manuscript\nThe unit is the energy $kg\\cdot m^2/s^2$.\nThe errors in Table 1 of the main manuscript are the square errors as indicated in Equation 8,\nwhere we sum up 1.4K energy units of the cloth with 484 vertices.\nThe per-energy-unit errors are significantly smaller in terms of the absolute value.\n\nRegarding the differences between the materials,\nleather and denim are either stiffer or heavier.\nThese characteristics lead to larger energy changes between frames due to significant variations in velocity or increased mass,\nmaking it more challenging for the model to learn.\nWe would include the discussion in the limitation section in our revised version.\n\nIn conclusion,\nas shown in Table 2 of the main manuscript,\nMGN-S+EUNet (ours) still achieves superior performance,\nsuggesting the effectiveness of our EUNet.\nAnd as shown in Figure 1 of the main manuscript and the video in supplementary,\nMGN-S+EUNet delivers reasonable material behaviors for garments made of silk and leather,\nwhich are softer or stiffer respectively.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for their insightful feedback.
We emphasize our contributions and clarify the main points as follows.


To mimic the dynamic patterns from observed clothes, some methods [1, 2] focus on estimating the **PHYSICS PARAMETERS** that best fit the known analytical models or simulators.
In contrast,
we aim to explore the intrinsic **ENERGY FUNCTIONS** that describe the constitutive relations governed by general physics laws, such as the Lagrangian mechanics,
and animate garments constrained by our learned EUNet.

Our work is significant in the following ways
1. our disentangled learning scheme reduces the great need of large scale of garment data to mimic garment dynamics, and relies on only a piece of cloth as training data;
2. our EUNet is able to directly capture the constitutive behaviors from observed trajectories that are relatively easy to obtain in real life,
and is highly generalizable thanks to the topology-independent designs;
3. garment animations constrained by our EUNet deliver lower errors and superior performance comparing with baselines.

In the additional PDF file,
we
1. add HOOD [3] for comparisons and demonstrate the effectiveness of the mini-network mentioned at L290-292 for HOOD;
2. apply our method in real application and animate the realistic garments from CAPE [4].

Below we list the key components of the tables.

# Time Efficiency Comparisons
|        | PHYS  | EUNet $\Phi_p$ | EUNet $\Phi_d$ | EUNet $\Phi_p+\Phi_d$ |
|  ----  | ----  | ----  | ----  | ----  |
|  Time (ms)  | 1.743 $\pm$ 0.212  | 1.055 $\pm$ 0.202 | 1.216 $\pm$ 0.223 | 2.271 $\pm$ 0.422 |

We denote the StVK elastic model and the bending model by "PHYS", which is used in the main text.
As formulated in Equation (1),
our EUNet is composed of two separate branches: $\Phi_p$ for potential energy and $\Phi_d$ for dissipation.
Both branches have the same structures.
We report the time separately for each branch and the full EUNet as above.
The forward time is averaged on 80 frames of predictions,
which include garments composed of 7924 vertices, 23636 edges and 15712 faces.
All experiments are run on NVIDIA A100-SXM4-80GB.

Our EUNet is comparable to the traditional clothing model in terms of speed.

# Comparisons with HOOD
|     Methods   | Overall Euclidean Error (mm)  | Collision (%) |
|  ----  | ----  | ----  |
|  MGN-H+PHYS+F (HOOD) | 84.85 $\pm$ 29.93  | 0.46 $\pm$ 0.99 |
|  MGN-H+EUNet (Ours) | **66.39 $\pm$ 39.36**  | **0.44 $\pm$ 0.48** |

We denote the hierarchical graph neural network used in HOOD as MGN-H.
HOOD adopts the analytical clothing model (PHYS) and friction (F) between human body and garments as loss terms.
As shown in the table,
MGN-H constrained by our EUNet achieves lower errors comparing with HOOD,
suggesting the effectiveness of our EUNet.

[1] Carlos Rodríguez-Pardo, et al. How Will It Drape Like? Capturing Fabric Mechanics from Depth Images. CGF2023.

[2] Eunjung Ju, et al. Estimating Cloth Simulation Parameters From Tag Information and Cusick Drape Test. EG2024.

[3] Artur Grigorev, et al. HOOD: hierarchical graphs for generalized modelling of clothing dynamics. CVPR2023

[4] Qianli Ma, et al. Learning to Dress 3D People in Generative Clothing. CVPR2020

Meta-review: The paper received mixed scores (WA, BA, BA, BR). During the rebuttal phase, the authors and reviewers discussed the paper. Unfortunately, the final scores remained mixed and lukewarm at best, without any reviewer championing the work. The AC reviewed the paper, reviews, and discussion and decided to champion the work. 

+ significantly reduces the requirement for generating/collecting data over all possible garment/body configurations
+ a nice formulation involving energy formulation (this is different from parameter estimation) giving a fresh perspective to the problem
+ a novel direction and can lead to followups (this is related to gradient-domain processing in imaging and animation)

- lack of demonstration on real data (future work!)
- not demonstrated on very complex samples or real captures. AC finds this to be ok, given the novelty of the approach.
Decision: Accept (poster)