Fine-grained Differentiable Physics: A Yarn-level Model for Fabrics
Authors: Deshan Gong
Zhanxing Zhu
Andrew J. Bulpitt
He Wang
Venue: ICLR 2022 Poster (2022)

Abstract: Differentiable physics modeling combines physics models with gradient-based learning to provide model explicability and data efficiency. It has been used to learn dynamics, solve inverse problems and facilitate design, and is at its inception of impact. Current successes have concentrated on general physics models such as rigid bodies, deformable sheets, etc, assuming relatively simple structures and forces. Their granularity is intrinsically coarse and therefore incapable of modelling complex physical phenomena. Fine-grained models are still to be developed to incorporate sophisticated material structures and force interactions with gradient-based learning. Following this motivation, we propose a new differentiable fabrics model for composite materials such as cloths, where we dive into the granularity of yarns and model individual yarn physics and yarn-to-yarn interactions. To this end, we propose several differentiable forces, whose counterparts in empirical physics are indifferentiable, to facilitate gradient-based learning. These forces, albeit applied to cloths, are ubiquitous in various physical systems. Through comprehensive evaluation and comparison, we demonstrate our model's $\textit{explicability}$ in learning meaningful physical parameters, $\textit{versatility}$ in incorporating complex physical structures and heterogeneous materials, $\textit{data-efficiency}$ in learning, and $\textit{high-fidelity}$ in capturing subtle dynamics.

Published as a conference paper at ICLR 2022
FINE-GRAINED DIFFERENTIABLE PHYSICS: A YARN- LEVEL MODEL FOR FABRICS
Deshan Gong', Zhanxing Zhu " Andrew J. Bulpitt' and He Wang
'School of Computing, University of Leeds
School of Informatics, University of Edinburgh Peking University
{sodg, A.J. A.l.Bul w.e.)eageleeeds wac.a.elleds.acckk uc.uk, zhanxing.
zhu@pku. .dd.cn
ABSTRACT
Differentiable physics modeling combines physics models with gradient- gradient-based learning to provide model explicability and data efficiency. has been used learn dynamics, solve inverse problems and facilitate design, and is its incep- tion impact. Current successes have concentrated on general physics models such rigid bodies, deformable sheets, etc, assuming relatively simple struc- tures and forces. Their granularity is intrinsically coarse and therefore incapable of modelling complex physical phenomena. Fine-grained models are still be developed to incorporate sophisticated material structures and force interactions with gradient-based learning. Following this motivation, propose ferentiable fabrics model for composite materials such as cloths, where we dive into the granularity of yarns and model individual yarn physics and yarn-to-yamn interactions. To this end, we propose several differentiable forces, whose counter- parts in empirical physics are indifferentiable, to facilitate gradient- baadienttbased learning. These forces, albeit applied to cloths, are ubiquitous in various physical systems. Through comprehensive evaluation and comparison, we demonstrate our model's explicability in learning meaningful physical parameters, versatility in incorpo- rating complex physical structures and heterogencous materials, lata-efficiency in learning, and high-fidelity in capturing subtle dynamics. Code is available in: iable-P
dif-
https: com real http://www..cmeeeeeaaeeeeee/e/// www.iiinnnngnnnnnneeeeeeeeaneeae grained-a--.n Sysica-A-Yarn-ie -odeee-eeeeeeeeeeeeee---- for fabricic.--aa......
INTRODUCTION
Differentiable physics models (DPMs) have recently spiked interests, e.g. rigid bodies (Heiden et al. 2020), cloth (Liang al.) 2019), and soft bodies (Hu et al. 2019). The essence of DPMs is making physics models differentiable, so that gradient- based learning can be used to make systems adhere strictly to physical dynamics. This is realized via back propagation through series of observed actions, where the system can quickly learn the underlying dynamics. While enjoying neural net- works' capability of modeling arbitrary non-linearity. DPMs also improve the model explicability as the learnable model parameters bear physical meanings. As a result, such models provide a new avenue for many applications such inverse problems, e.g. estimating the mass of a moving rigid body (de Avila Belbute-Peres et al. 2018b), and control, e.g. learning to shake a bottle to shape the fluid in (Li et 2019).
Early research attempted to model simple and general physical systems such as rigid bodies (de Avila Belbute-Peres al. (0118). followed by range systems including deformable objects (Li al., 2019). cloth cloth (Liang et al., 2019), contacts (Zhong et al.] 2021). etc. However, existing models are only gerralllllyyy which do not consider complex structures/topologiee and force interactions. Taking cloth (i.e. fabrics)
example, existing models (Liang et al.) 2019 Li et al., 2019) can learn general cloth dynamics, but only when the cloth is relative simple and homogeneous. Recent research (Wang al. 2020) has started explore articulated systems but the model capacity is in- sufficient to capture the full dynamics of complex systems such as fabrics. Since weal-world physical
"corresponding author

---

Published as a conference paper at ICLR 2022
systems (e.g. materials in engineering) often have sophisticated structures and consist of heteroge- neous materials, we argue that it is crucial to design fine-gras rree-graned DPMs, for differentiable physics to be truly applicable and meaningful to real-world applications.
This paper focuses on fabrics which are composite materials consisting of basic slim units arranged in different patterns. A common example in fabrics is woven cloth which is made from yarns of different materials (silk, cotton, nylon, etc.) interlaced in various patterns (e.g. plain, satin, twill). Fabrics present new challenges in differentiable modeling. First, the dynamics heterogeneity caused by material and structural diversity needs to be incorporated into modeling, which is especially cru- cial for solving inverse problems where the physical properties are learned from data. General DPMs without sufficient gramularity can only upproximate the dynamics and are unable to learn meaningful parameters. Second, certain forces that are essential for fabrics dynamics are indifferentiable. One such example is friction. The standard Coulomb model for rigid bodies has been made differentiable recently (de Avila Belbute-Peres et al. 20186 Zhong et al., 202T). However, it overly simplified for fabrics because the yarn-to-yarn friction shows richer dynamics (Zhou et al. (017) that is beyond the capacity of existing methods. Further, the contact modeling together with friction requires new treatments that previous methods did not have to consider.
To overcome these challenges, we propose a new DPM for fabrics at a more fine-graines level and apply it to cloth modeling. Unlike general DPMs, we start with fine-grained yam--levl model. By modeling each yarn individually, we provide the capacity of modeling fabrics with mixed yarns and different woven patterns, which could not be handled previously. To facilitate gradient-based learn- ing, we propose new differentiable forces on/between yarns, including contact, friction and shear. Finally, we incorporate implicit Euler and implicit differentiation to compute gradients induced by an optimization problem embedded in the simulation.
To our best knowledge, our model is the first differentiable physics model which provides sufficient granularity for heterogeneous materials such as fabrics. We comprchensively evaluate its learning capability, data efficiency and fidelity. Since there no similar model, we compare our model with the most similar work (Liang et al., 2019) and traditional Bayesian optimization on inverse problems. We also compare our work on control learning with popular Reinforcement Learning methods. We show that our model is more explicable, has higher data efficiency, generates more accurate predictions in inverse problem and control learning respectively.
RELATED WORK
Differentiable physics simulator. A differentiable simulator integrates differentiable physics en- gine into the forward and backward propagation of learning. As a strong inductive bias, these simulation engines increase data efficiency and learning accuracy over gradient-free models. Due to these advantages, differentiable simulation demonstrates superiority in a number of problem do- mains such as inverse problem, robot control and motion planning. The early works focused initially on simple rigid bodics (de Avila Belbute- Peruuseeeeeeeseeeee et al.) [0018]] tion of high degrees of freedom systems, such fluids (Schenck & Fox 2018), elastic bodies (Hul jet al. 2019: Huang et al. 202T), and cloth (Liang et al.] 2019). More recently, Jatavallabhula et al. (2021) introduced an end-to-end differentiable simulator that can learn from images by combin- ing differentiable rendering and differentiable simulation. Comparatively, we explore fine-grained DPMs for composite materials, which leads to new challenges in differentiable modeling.
B0 Degrave et al. (2019) and later simula-
Cloth simulation. Cloth simulation initially appeared in textile engineering and was then introduced to computer graphics (Long al., T). Cloth has been modeled as particle systems (Breen al.,
1992). mass-spring systems (Provot et al., 1995). and continuum (Narain et al.) 2012). Kaldor
et al. 2008 proposed yarn-level knit cloth simulator and found that cloth microstructures have a considerable influence on cloth dynamics. Since then the cloth simulation community has shifted the focus to yarn-level cloth simulation. Based on the objectives, the recent research can be classified to increasing efficiency (Kaldor et al. 2011 Cirio et al. 2016)), combining continuum models and yarn- level models (Casafranca et al. 2020; Sperl et al. (020)) introducing woven cloth simulation (Cirio et al. 2014), and optimizations (Pizana et al. 2020; Sänchezz- Banchez-Banderas Banderas al. 2020). Our work is orthogonal to these papers in that we introduce a new methodology to incorporate differentiable physics into yarn-level models.
2

---

Published as a conference paper at ICLR 2022
Machine learning and cloth simulation. Machine learning was initially introduced to cloth sim- ulation to make data-driven simulators, which have inherent advantages simulation efficiency over thysical-based methods (James & &Fatahalian, 2003 Kim & Vendrovsky 2008), and can help improve fidelity (Lahner et al,, 2018). j. In parallel, machine learning has been applied to discover the physical properties from visual information. [Bouman al. (2013) proposed linear regression model for evaluating cloth density and stiffness from the dynamics of wind-blown cloth. Yang et al. (0177)) introduced neural network for classifying cloths based on how their dynamics are affected by stretching and bending stiffness. Rasheed et al. (2020) proposed a model for estimating the fric- tion coefficient between cloth and other objects. By combining physically-bass -based cloth simulators and neural networks, Runia et al. (2020) estimated cloth parameters by training neural networks to adjust a simulator's parameters so that the simulated cloth mimics the observed one videos. Dif- ferent from these gradient-free models, Liang et al. (2019) and Li et al. (202T) proposed sheet-level differentiable cloth models that can be used to estimate cloth parameters. In this work, we dive into fine-gra fine-grained physics and propose new yarn-level diffcrentiable fabrics model which can be embedded into deep neural networks as a layer.
METHODOLOGY
Since cloth is employed as an application in this paper, we use the terms 'cloth' and 'fabric' inter- changeably. We first explain the cloth representation (Sec. [S1c and the (physics) system equation for simulation (Sec. 3.2g Then we present our new force models (Sec. 3.3), and how we solve the system equation to enable back-propagation (Sec. 3.4..
3.1 CLOTH REPRESENTATION
Similar to Cirio al. (2014). our cloth consists of two perpendicular groups of parallel yarns named warps and wefts. Every pair of warp and weft are in contact with each other at one crossing node (Figure I. with a persistent contact. We employ an BitlerannonnLLgrangee
discretization (Sueda et al. 2011), and denote the Degrees of Freedom (DoFs) of every crossing node as qi (X,, M4 02)- X, E I' is the Lagrangian coordinates indicating spatial locations and (ut 2/) is the Eulerian co- ordinates indicating sliding movements between yarns. The end points of yarns do not contact with other yarns and hence they are treated as special crossing nodes that have no Fulerian terms, i.e. c(columns) cloth, there are (r 2) x (c (c-2) 2) crossing nodes with five DoFs and 2r + 2c - 4 crossing nodes with three DoFs. Every two neighboring crossing nodes on the same warp/weft delimit warp/weft segment. A warp segment with end points qo and qi is denoted as [4o q1} and its position is (Xo. X1, Up U1) (Figure M. This way, a woven cloth discretized into crossing nodes and segments which are the primitive units of the cloth. Every segment is assumed to be straight so that linear interpolation can be employed on the segment, i.e. the spatial location of a point in the segment [o, qı is x(u) u-Me
q = x; Therefore, on r(rows)
2u Xo + = X1, where u the point's position in Fulerian coordinates and Au Mi is the crossing nodes distance in Fulerian coordinates. We use L to denote the rest length of the yarn segment and R to denote the yarn radius.
3.2 SYSTEM EQUATION FOR SIMULATION
A cloth's s state at time t, S(g) (Q 2 ). includes all the cross- ing node DoFs Q {q: i .,,,...,N] NN and their velocities é {q:li 2, ...,..... .J. where N is the number of crossing nodes. Knowing the state, we can calculate the internal and exter- nal forces:
WArp
Welt
4
%
q.
0T aV
F Mq
Mq
(I)
dq dq
where q, 4, and q are the general position, velocity, and accelera- tion respectively, with a dimension 3xrxc+2x(r-2)x (c-2). M E | is the general mass matrix. The model assumes mass is distributed homogeneously. T and V are the kinetic and potential energy. As force is related to the partial derivative of energy with respect to position, the right hand terms in Eq. are inertia, conser- vative forces, and part of the time derivative of Mq. Non-conservative forces are added the right
Figure 1: Blue and red rods denote warps and wefts re- spectively. qs are the crossing nodes.
3

---

Published as a conference paper at ICLR 2022
side of the equation. Section 3.3
gives the details of all the forces considered in model. Using
implicit Euler (Baraft & Witkin 1998). we can derive the system equation for simulation:
((()) (1) Oq
OF
OF (1)
M
ht2
((++1) = F
(1))
Ma(()
dq
Oq
where the subscript in brackets t indicates the associate variable at time t. Detailed deduction is in Appendix.
3.3 FORCE MODELS
To simulate cloth, we need to compute the inertia, internal and external forces in Equation 2 Inertia the derivative of kinetic energy with respect to node positions. As woven cloths interlaced yarns, the internal forces can be further classified into 1) forces caused by yarn deformation and 2) forces resulting from yarn-toyyan interactions. We treat each yarn as an elastic rod that can generate elastic energy, including stretching and bending (Jawed et al. 2018 ignoring the twisting due to its triviality cloth dynamics (Cirio et al. 2014). The elastic energy is V V V where V and 4d are the stretching and bending energy respectively. The yarn-to-yarn interaction forces include friction, shear, and parallel yarn collisions.
refer the reader to Appendix for the inertia, stretching, and bending force as it is straightforward to show that they are differentiable.
Yarn-to-yarn contact. While the aforementioned forces are dif- ferentiable, the yarn-to-yarn forces are not. Existing differen- tiable contact models mainly correct after-contact positions and velocities (de Avila Belbute-Pene Peres et al., 2018b Liang al. 2019 Zhong et al., (021) via (multiple) optimization solves, which too simplistic for fabrics. Yarn-to-yarn contact has its unique features. It is relatively sticky and often has small relative velocities. We need contact model that reflects this and leads differentiable contact force which affects the friction/shear, The contact force a combination of the stretching s and bending forces F, at every crossing node along the contact normal n. We assume no-slip contact and compute the contact force by:
q,
4,
Figure 2: Compression force on qu along normal qo-
F, = ReLU( T(F F; F FF))
(3)
2
where u and represent the forces from warp and weft segments. The rectified linear unit (ReLU) ensures the non-negativity of the contact force. The normal n, from warp weft yarn (Fig. 2 is approximated by the normal of the best-fit plane of qo-q4-
Friction. The friction between warps and wefts prohibits relative movements, which is crucial the overall dynamics of the fabric. In differentiable physics, contacts under simple settings have been modeled, such as the standard Coulomb model for kinetic friction (Zhong al., 2021). But this insufficient for our purpose for two reasons. First, the static friction plays key role in stick-slip behaviours of yarns (Zhou et al. (017)) and needs to be modeled. Second, the standard Coulomb friction model is piece-wise function, which is intrinsically indifferentiable the static-to-kinetic transition point. Therefore, we need a new differentiable friction model.
The low relative speed between yarns is special situation where the staticto-kkintte transi- tion could actually be continuous (as opposed to the Coulomb model), experimentally shown by Stribeck (Stribeck] [902). This indicates that continuous and differentiable model has the poten- tial to be more accurate for yarns than the widely used Coulomb model. Further, the breakaway force causing the static-to-kinetic transition depends the rate of the external force (Johannes) et al. 1973), and the nonlinear stick-slip behavior related to self-excited vibrations before transi- tion Awrejcewicz 1988). Inspired by the above research, we propose new differentiable yarn-to- yarn friction model (Figure 3
Fstide (( jy (( K(NF, F.) kyōu + K(Bu))
-d
(4)
2
2
where du = ug ün and K(xx) = tanh(pa). Tn is the anchor position when there no relative movement between the warp and the weft segment, y the friction coefficient, and F, is the external
4

---

Published as a conference paper at ICLR 2022
Friction (N)
Friction (N)
Friction (N)
Fi
Fi
Fk
Fi
Fi
Ft
(m/s)
v (m/s)
(m/s)
-Fi -Fk
-F.
-Fo
-Fk
-Fi
Figure 3: F6 and Ft are the static and kinetic friction. Coulomb model (left) an indifferentiable multi-value function. Stribeck effect (middle) is empirically observed (Stribeck 1902). Our model (right) incorporates the Stribeck effect and also simulates self-excited vibrations around v = 0.
force. We introduce a hyperparameter p control the conversion speed between static and kinetic friction. To understand Equation 4 there are three situations: F, = 0 (no external force), 0 S. Fg S IF,, (static friction), and F, FF, (kinetic friction). When Fu O. 8u O, K(uF, F.) = and the speed iñ O, so Fstide = O; when 0 < F. S μFn, we allow a small displacement du to mimic the self-excited vibration static friction, governed by a Hooke's spring dy with stiffness ky. When F, is small, i.e. K(uF, F.)) is close to 1, Fstide
-k d where dj is a damping coefficient. Fstude is mainly the static friction minus a small damping term (as úgg is small). Due to the time discretization in simulation, the spring force has delayed response which causes -range vibrations. When K (HFF F.) starts to decrease to and the breakaway force drdng and ky K(Bu)uFF is the average of
achieved F. = FFR, Fsudee ky K(u)) the spring force and the maximum static friction. Finally when Fu > pF K(FFn n - F.) quickly becomes -1 and (ōu) becomes du increases. Then Fstide -uF, drún, which is the kinetic friction minus damping. Fig. shows our friction can closely approximate the Stribeck effect while maintaining differentiabilitty. as opposed to the indifferentiable Coulomb model. Also, it incorporates self-excited vibrations within Fstide E [ Fk, Pk] which is when S Fu S pFn.
Shear. A shear force is generated when there is relative rotation between a warp and weft crossing node (Parsons al. 2010), which increases non-linearly when the shear angle in-
4.0 400
335
9,
2.5
2.0
creases (Mohammed et al. 2000; Peng et al.
1.0
2004 Cao et al., (208). Previous differen-
0.5
tial models do not consider this type of forces. Therefore, we propose new differentiable shear model. There are different stages when the shear angle increases (King et al.] (2005). The shear force first grows almost linearly ini- tially, then 'shear lock' is triggered (Wang)
0.0
0
10
20
90
40
50
60
80 00
Shear angle (degree)
9,
Figure Shear force strength vs shear angle Ø Φb, (left) and graphical illustration (right).
et al. 1999) when the angle passes a threshold and the shear force starts to increase exponentially as the angle increases, producing highly non-linear behaviors. We therefore define the shear energy as a function of the shear angle o & (Figure 4) L o)², where ő I is the rest shear angle. k. - STR2 ( F,) is the shear stiffness and S is the shear modulus. We embed the 'shear lock' by boosting k, exponentially with 7 (V21² sin L) R long it stays smaller than the lock threshold Φ 2 aresin R: k2 equals to STR F,) ó > Φt: and STR² + F2) otherwise, where e controls the increase rate of k. with respect to ø when 'shear lock' occurs. Although k2 has discontinuities within [0, $
our new shear stiffness can be defined as:
1 )Fx + 1)STRR + tanh ( h, =
(0(6- - Φi) 0))² + 0000 =
06 61)
(5) where a governs the transition smoothness between lock and no-lock. The smaller the or is, the smoother the transition is. The shear force (Figure derivation is in Appendix.
Yarn-to-yarn collision. The last internal force is the yarn-to-ye collisions between parallel yarns. Although it is theoretically possible existing approach (de Avila Belbute-Peres et al. 2018h
5

---

Published as a conference paper at ICLR 2022
Liang et al. 2019), it would require forming optimization for all segments and therefore become prohibitively slow. Therefore, we introduce a new penalty energy. defined as a function of the nodes' distance in Eulerian coordinates for warp segment [qo. qı] (similar for weft segment): Au))² where d = 4R or 2R which is elaborated Appendix.
Vo,1 = I.
External forces and collisions. Without loss of generality, consider external forces: gravity and wind force. Their impacts can be modeled by defining proper potential energies. Finally, after calculating all forces, the resultant force every crossing node is the combined force of all segments that connect to that node. The cloth is simulated by solving Equation 2 We use bounding volume hierarchy (Tang et al. 2010) with continuous collision detection and non-rigid impact zones (Har- mon et al., 2008) to compute collision. Similar to Liang et al. (2019); Wang et al. (2020), we form an optimization problem for continuous collision. Details can be found in Appendix.
3.4 DERIVATIVES OF THE SIMULATOR
Now have fully differentiable simulator with parameters w. The is the cloth physical pa- rameters (stretching, bending, shearing, etc) when solving inverse problems, or the to be learned external forces in control experiments. Given loss function L, its gradient with respect to the 8c can help learn the right physics parameters via back-propagation. Implicit differ- entiation can be used to derive #C detailed in Appendix. Finally, we define the loss function:
parameters dw
Dw
C(q. à) = NT Σ m==1 Ei= ||q - &r,r 112. where qu and dn,t are the ground-truth and predicted general position. N and T are the total number of nodes and simulation steps respectively. We use Stochastic Gradient Descent and run 70 epochs for training.
Underconstrainednee Mitigation. Learning physical parameters via solving inverse problem is intrinsically under-constrained, leading to multiple solutions or implausible parameter values when fitting data, e.g. unconstrained learning leads to negative density. We mitigate this issue by incorpo- rating prior knowledge. Instead of directly learning parameters w, we set w X sigmoid( + b where a and b are tunable scalars, and we learn y instead, and b essentially limit the range of w. We can induce prior knowledge of parameter ranges such as yarn density ranges, because although the exact value is be learned and known a priori, their ranges are available in practice. Our experiments demonstrate that this strategy effectively mitigates the multi-solution issue.
4 EXPERIMENTS
We employ traditional indifferentiable yarn-level simulator (Cirio al. (2014) to generate the ground-truth data, and build dataset of fabrics with three types of yarns and three types of woven patterns. The yarns vary in density, elas- tic modulus, and bending modulus (Ta- [.. The three woven patterns include plain, twill and satin. We use hybrid fab- rics made from two types of yarns, and exhaustively combine three yarns with three woven patterns (Figure 5) to generate 9 types of fabrics. We denote them as XXX-X, X) where the prefix is the woven pattern and the numbers in the brackets are the yarns, e.g. Plain-(1, 2) means a plain pattern woven with Yarn and Yam22. In our ground-truth data, we use a square piece of cloth hanging its two corners and blown by wind with constant magnitude. The simulation conducted for 500 steps with / 0.001s. Training details are in Appendix.
Figure 5: Woven patterns. Left-to-right: plain, satin, and twill. Teal and coral indicate different yarns.
4.1 LEARNING PHYSICAL PARAMETERS
We first demonstrate our model's effectiveness in learning meaningful physical parameters, under various model sizes and different amounts of training data.
Learning capacity. We first test whether meaningful physical parameters can learned from cloths of different sizes. Small-size cloths tend to show low-frequency features, e.g. the general shape, as

---

Published as a conference paper at ICLR 2022
Table 1: Ground-truth parameters of three yarns.
Parameter/Yam
Yarnl
Yarn2
Yarn3
Density(kg/mm)
0.0020
0.0025
0.0024
Stretch modulus( /m) 500000
170000
120000
Bending modulus(N/m)
0.00014
0.00011
0.00009
opposed to high-frequency features, such as wrinkles and buckling. We test model simulation data with sizes: 5x5, 10x 10, 17 and 25x25, trained on the first 25 frames. Table shows that our model can effectively estimate yarn parameters with underlying physics models of different sizes. This has several implications. First, although cloth size does affect the overall dynamics of the motion in the ground-truth data (e.g. larger cloths have more wrinkles), it does not affect our model's learning capability. Second, since our model can reliably learn the yarn parameters on a small fraction of cloth, it saves the computation of learning from large cloths, which improves the learning scalability. We can learn from small cloths and then scale to simulate large cloths. Further, inter-yarn parameters including shear S and friction coefficient I are highly correlated so that the model can easily end up learning only plausible parameter values rather than the true values. This is where we expect our model suffer from the under-constraineeness problem as Liang al. (2019). Surprisingly, our model can learn the right parameters under different sizes. By introducing prior knowledge as aforementioned, the learned parameters are restricted within valid ranges.
Table 2: Inter/intra parameters learned on Plain-(1, 2), with ground-truti S = 1000Pa, μ 0.5.
Size
Shear S
Friction μ
Yarn
Density
Stretch
Bend
2.028 x 10-3
479523
1.387 x 10
x 5
949
0.437
2.450 x 10-3
172928
1.112 x 10-4
1.991 10-3
484719
1.325 x 10-4
10 x 10
932
0.455
2
2.448 x 10-3
173843
1.026 X 10-4
1.969 x 10-3
505421
1.323 X 10-4
17 17
947
0.402
2.440
10-3
171304
1.034 X 10-4
2.069 x 10-3
510215
1.488 X 10
25 x 25
913
0.380
2.443 x 10-3
173920
1.201 X 10
Data efficiency. Data efficiency is crucial as obtaining the ground-truth data can be expensive. Precise 3D geometry capture of real cloths is difficult and time-consuming, while simulation of high- res cloths is prohibitively slow. We further investigate the data efficiency by varying the amount of training data. We gradually increase the training data from the first 5 frames to the first 25 frames. Table 3 shows that our model has high data efficiency. It can learn reasonably well from as few as the first 5 frames. The benefits are two-fold. First, our model needs just a few frames to train, making it highly applicable. The second benefit is bigger but less obvious. The first 5 frames (from static pose) normally contains little dynamics as the cloth just starts to move. This indicates that our model only requires few frames of low-dynamics motions. This cases real-worke measurements on cloth because no large motions are needed. This also saves time if simulation data is used, as small time step size is usually demanded in high-dynamic motion simulations.
Table 3: Plain Plain-(1,2) learnt parameters on different training data. Left: Yarn,, Right:Yan2.. Yamn2.
Frames
Density
Stretch
Bend
Density
Stretch
Bend
5
2.030 x 10-3
494301
1.357
10 10-4
2.450 x 10 10-3
169597
1.130 x 10
10
2.037
10 10-3
491717
1.379
10-4
2.443
10-3
169543
1.130
10-4
25
2.038 x 10
491873
1.367
10-4
2.447 x 10-3
167217
1.096 x 10-4
All simulations can be found in the supplementary video. We also include simulations with col- lisions and simulations on large cloths using parameters learnt on small cloths. More results and details are in Appendix.
7

---

Published as a conference paper at ICLR 2022
Table 4: Testing errors (x10-") of our model (left) and (Liang et al. 2019) (middle) and BO (right)
trained on 5, 10 and 25 frames generated by yarn-level simulator (c Cirio et 2014).
Fabrics/Frames
5
25
5
25
5
25
Plain-(1,2)
1.152
10-4
3.962
10-5
1.461
0.4124
0.512
0.109
Plain-(1,3)
1.516
10-4
3.555 X 10
1.608
0.4567
1.280
0.738
Plain-(2,3)
5.233 x 10-4
2.117 x 10-5
1.952
0.2294
28.19
18.16
Error Logarithm
-2
Our Yarn-Level Simulator Sheet-Level Simulator
Diff Simulator
Error Logarithm
original
original size
4x size Diff
Bayesian Optimization
Squared
B0 Non-diff Simulator
-2
2x sice Diff
4x size BO
-6
-8
-10
400 800
1200 1600 2000
100
300
500
Number Training Frames
Number of Simulation Steps
Number of Simulation Steps
Figure 6: Simulation errors: data efficiency (left), long (middle) and big cloths simulation (right).
4.2
COMPARISONS
4.2.1
PREDICTION & DATA EFFICIENCY
To our best knowledge, there is no similar fine-grained DPM in the literature. The closest method general sheet model (Liang al. 2019), we compare model with theirs. We employ their settings, and use a 17 model and 50 frames simulation data, with 5, 10 and 25 frames for training and the whole 50 for testing. Since the two methods model cloths different levels of granularity, their physical parameters are not directly comparable. We therefore compare their Mcan Squared Error (MSE). We also include traditional parameter estimation method based on Bayesian Optimization (BO) (Snoek et al., 2012) combined with a yarn-level simulator (Cirio et al.) 2014) another baseline. In BO, randomly select 5 initial points and use the expected improvement (Jones et al. 1998) as the acquisition function. As the learning process of differentiable simulation consists of forward simulation and backward simulation, training 70 epochs be considered as running 140 simulations. Therefore, we run 140 iterations when using BO. Moreover, we impose the same parameter ranges in the BO as we did in our model.
From 4 our model uses data more efficiently than (Liang al., 2019) and BO. From training on 5 frames 25 frames, our model reduces the error by much as 96% on Plain-(2, 3), while the largest improvements by the sheet-level model and BO optimization are 88% on Plain-(2, 3) and 78% on Plain- (1,2) respectively. Moreover, as shown in Figure 6 left, our error 5 frames is already several magnitudes smaller than the baselines. Further reducing it requires the model to be able learn subtle dynamics very well. [iing et al. (2019) essen- tially treats fabrics as a sheet. Since the simulation is from yarn-level simulator (Cirio al. 2014) which contains rich dynamics, the sheet model cannot precisely capture the sub- le dynamics caused by individual yarns and their interactions. Further, the model granularity difference has more profound impact than just prediction. Being able learn yarm parame- ters has immediate benefits for manufacturing and design, in terms of providing guidance on the choices yarns and wo- patterns. In addition, although BO can sometimes perform slightly better than the sheet model benefiting from a yarn-level simulator, its optimization process is not as efficient as ours. We only show results Plain here and refer the readers to Appendix for Satin, Twill, and video comparisons.
GT
Ours
BO
Sheet
Figure 7: Simulation snapshots of the same step. The parameters timated by our model visually closest the ground truth.
Error significance. The MSE errors in Table 4 seem to small, this is because the cloth small and only simulated for short period of time. But the results suggest emrors in parameter estimation, which are amplified when the cloth larger and simulated for a longer time. To demonstrate this, first, forward simulations for 2000 steps with parameters learned by model and BO.
8

---

Published as a conference paper at ICLR 2022
Second, show the compound influence using the parameters estimated by model, Liang al. (2019), and BO, and simulate a 17 x 17 cloth for 500 steps in the original size, 2 times size, and 4 times size. Fig. 6 middle-rightt show both results. which demonstrates the importance of accurate parameter estimation. The errors of BO and Liang et al. (2019) quickly become several times higher than our model when we scale the size and simulation time. We also show visual comparison in Figure 7 and refer the readers to Appendix for more results.
4.2.2 CONTROL LEARNING
We also show that our model can facilitate control learning. We design a task with a cloth placed on table and aims to learn forces applied onto the four corners of the cloth to throw into a box next to the table. The forces are only applied the first 5 frames. We use model to learn a sequence of forces which throw the cloth into the box and com- pare with a reinforcement learning baseline model: PPO (Schulman et al., 2017). In ad- dition, also present variant of model by appending two fully-connected layers after our model output (Ours + FC). We use the center of the box's bottom as the target location. When training our model, we use the l2 distance between the cloth center of mass and the target position as the loss. When training PPO, we use the same l2 distance for the reward.
The result shows both our model and Fu can quickly learn the forces to throw the cloth into the box. By contrast, PPO is model-free and much slower because it needs to sample in huge
Mean Squared Error
1e-3
1.5
PPO Ours
action space. By contrast, the full differentiability of our model enables quicker search for effective control forces. More details can be found in Appendix.
1.0
Fu
0.5
In a broader context, there are also model-free methods (Yang al. 201 7a Pfaff et al. (020) which can also learn physics. The differences between
20
40
60
Episodes
model and theirs are: model explica- bility. The parameters that our model learns are interpretable and have physical meanings, so that it can guide manufacturing and design. 2. data efficiency. The data efficiency is much higher in method. Our model can while model-free methods typically require hundreds to thousands.
Figure 8: The MSE errors against epochs. Our approach learns faster than PPO.
few as 5 frames for learning
5 DISCUSSION AND CONCLUSION
Our method is model-based, which requires domain knowledge and cannot simply 'plug and play' on data model-free methods Pfaff et al.) (2020). However, strong inductive biases from domain knowledge are necessary for differentiable physics to be applied in applications, because the model behaviour needs to be explainable in such applications, and cannot be merely black-box regres- sion. Representative application domains include fabric manufacture/design and computer graphics, where both simulation and inverse problems need to be solved. Albeit focused cloth, our model can be readily extended to general composite materials with mesh structures, e.g. from metal/plastic nets to buildings. In addition, our model can be embedded as a layer into neural network, which helps learning control policies for cloth manipulation. Further, our model potentially enables a synergy between empirical physics modeling and deep learning, where our model can serve deterministic physics layer and other layers can incorporate non-linearity such as high- high-frequency dynamics in the system (Shen et al.) 202T). Finally, our modeling of general forces such as fric- tion and shear contributes to differentiable physical modeling in wider range, given the universal presence of such forces the real world.
To our best knowledge, we proposed the first yarn-level differentiable fabric simulator, in the pur- suit of fine-grained DPMs capable of incorporating domain knowledge. Through comprehensive evaluation, our model can effectively solve inverse problems, provide high data efficiency and facil- itate control. We investigated differentiable modeling of common forces such as friction and shear, which provides foundation for future attempts fine- -raiggr differentiable physics modeling. In future, we will pursue other composite materials such as metal meshes. Also, we will explore more complex dynamics such buckling and permanent damages.

---

Published as a conference paper at ICLR 2022
REFERENCES
J. Awrejcewicz. Chaotic motion in a nonlinear oscillator with friction. KSME Journal, 202):104 109, September 1988. ISSN 1738-494X. doi: 10.1007/BF02953669 URL https::tp///loi g/10 1
David Baraff and Andrew Witkin, Large steps in cloth simulation. In Proceedings the 25th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH '98, pp. 43-54, New York, NY, USA, 1998. Association for Computing Machinery. ISBN 0897919998. doi: 10.1145/ 2808 28.881.
1111//1088881 280821
Katherine L. Bouman, Bei Xiao, Peter Battaglia, and William T Freeman. Estimating the material properties of fabric from video. In 2013 IEEE International Conference Computer Vision, pp. 1984-1991. IEEE, 2013. ISBN 1479928402.
David E Breen, Donald H House, and Phillip H Getto. A physically-based particle model of woven cloth. The Visual Computer, 8(5):264-27) 277, 1992.
J. Cao, R. Akkerman, P. Boisse, J. Chen, H.S. Cheng, E.F. de Graaf, J.L. Gorezyca, P. Harrison, G. Hivet, Launay, W. Lee, Liu, S.V. Lomov, A. Long, E. de Luycker, F. Morestin, Pad- voiskís, X.Q. Peng. J. Sherwood, Tz. Stoilova, X.M. Tao, I. Verpoest, A. Willems, J. Wiggers, T.X. Yu, and B. Zhu. Characterization of mechanical behavior of woven fabrics: Experimental methods and benchmark results. Composites Part A: Applied Science and Manufacturing, 39(6); 1037-1053, 2008 ISSN 1359-835X. doi: httpa//ar...oooooooooooooaaaaaaa
URL /ttp://wwwww.o wwwp::/w........ccc rect. www.co//saaem 8000572
Juan Casafranca, Gabriel Cirio, Alejandro Rodríguez, Eder Miguel, and Miguel A Otaduy. Mixing yarns and triangles in cloth simulation. In Computer Graphies Forum, volume 39, pp. 101-110. Wiley Online Library, 2020.
Gabriel Cirio, Jorge Lopez-Moreno, David Miraut, and Miguel A. Otaduy. Yarn-level simulation of woven cloth. ACM Trans. Graph., 33(6), November 2014. ISSN 0730-0301. doi: 10.1145/2661 229.2661279. URL /ttp:::
Gabriel Cirio, Jorge Lopez-Moreno, and Miguel A Otaduy. Yam-level cloth simulation with sliding persistent contacts. IEEE transactions on visualization and computer graphics, 23(2):1152-116 2016.
Filipe de Avila Belbute-Peres Kevin Smith, Kelsey Allen, Josh Tenenbaum, and Zico Kolter. End- to-end differentiable physics for learning and control. Advances in neural information processing systems, 31:7178-7188 2018a.
Filipe de Avila Belbute-Peres, Kevin Smith, Kelsey Allen, Josh Tenenbaum, and J. Zico Kolter. Baddtt-Ennd Differentiable Physics for Learning and Control. Advances in Neural Information Processing Systems, 31, 2018b. WttLthhtt///pccceeeee mew.eeeiddiinnnunnnnn.eeee. neurips. cc/paper/2 Aattreeaeedststtttttttttttte..ee
Jonas Degrave, Michiel Hermans, Joni Dambre, et al. A differentiable physics engine for deep learning in robotics. Frontiers in neurorobotics, 13:6, 2019.
David Harmon, Etienne Vouga, Rasmus Tamstorf, and Eitan Grinspun. Robust treatment of simul- taneous collisions. SIGGRAPH (ACM Transactions on Graphics), 27 2008
Eric Heiden, David Millard, Hejia Zhang, and Gaurav S. Sukhatme. Interactive Differentiable Sim- ulation. PrXiv:1905, 100. [cs, star), May 2020. /RB//ttp:///rxxtt.or///bb/
0706 arXiv: 1905.10706.
Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua Tenenbaum. William T Freeman, Jiajun Wu, Daniela Rus, and Wojciech Matusik. Chainqueen: A real-time differentiable physical simu- lator for soft robotics. In 2019 International conference on robotics and automation (ICRA), pp-
6265-6221. IEEE, 2019.
10

---

Published as a conference paper at ICLR 2022
Zhiao Huang. Yuanming Hu, Tao Du, Siyuan Zhou, Hao Su, Joshua B Tenenbaum, and Chuang Gan. Plasticinelab: A soft-body manipulation benchmark with differentiable physics. arXiv preprint 1033::2004.03311, 2021.
Doug L. James and Kayvon Fatahalian. Precomputing interactive dynamic deformable scenes. ACM Trans. Craph., 22(3):879-88) 887, July 2003, ISSN 0730-0301. doi: 10.1 (0.114//82226288823555 URL
https://doi. . www.oorg//.... 10. 1145/ 882262228222... 882359
Krishna Murthy Jatavallabhula, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Mar- tin Weiss, Breandan Considine, Jerome Parent-Levesquer Kevin Xie, Kenny Erleben, et al. grad- sim: Differentiable simulation for system identification and visuomotor control. arXiv preprint arXiv:2/04.02644 2021.
M Khalid Jawed, Alyssa Novelia, and Oliver M O'Reilly. A primer on the kinematics of discrete elastic rods. Springer, 2018.
V. Johannes, M. A. Green, and C. Brockley. The role of the rate of application of the tangential force in determining the static friction coefficient. Wear, 24(3):381-3855 June 1973. ISSN 0043- 1648. doi: 10.1016600023-116433339000660000
URL //wp://wwwwww//////
com/
Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4):455-49) 1998.
Jonathan M Kaldor, Doug L James, and Steve Marschner. Simulating knitted cloth at the yarn level. In ACM SIGGRAPH 2008 papers, pp. 1-9, 2008
Jonathan M Kaldor, Doug I. James, and Steve Marschner. Efficient carn-based cloth with adaptive contact linearization. In ACM SIGGRAPH 2010 papers, pp. 1-10. 2010.
Tae- Yong Kim and Eugene Vendrovsky. Drivenshape: A data-driven approach for shape deforma- tion. In ACM SIGGRAPH 2008 Talks, SIGGRAPH '08, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 6751603333 doi: 10.1145/1401032.11 1401 121. URL
M.J. King, P. Jearanaisilawong. and S. Socrate. A continuum constitutive model for the mechanical behavior of woven fabrics. International Journal of Solids and Structures, 42(13):3867-3899 2005. ISSN 0020-7683. doi: htt.://c.................a..aa/a
URL https:
Zorah Lahner, Daniel Cremers, and Tony Tung. Deepwrinkles: Accurate and realistic clothing modeling. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 667- 684, 2018.
Yifei Li, Tao Du, Kui Wu, Jie Xu, and Wojeiech Matusik. Diffcloth: Differentiable cloth simulation with dry frictional contact. arXiv preprint 1.Xx::2006.05306 2021.
Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba. Learning Parti- cle Dynamics for Manipulating Rigid Bodies, Deformable Objects, and Fluids. arXiv:1810.01566 [physics, stat], April 2019. URL ht htpp://aaxxia org.iii.......o..oom abs /1100 01566 1810.01566.
arXiv:
Junbang Liang, Ming Lin, and Vladlen Koltun. Differentiable cloth simulation for inverse problems. 2019.
James Long, Katherine Burns, and Jingzhou James Yang. Cloth modeling and simulation: liter- ature survey. In International Conference on Digital Human Modeling, pp. 312-320. Springer, 2011.
U. Mohammed, C. Lekakou, L. Dong, and M.G. Bader. Shear deformation and micromechanics of woven fabrics. Composites Part A: Applied Science and Manufactuuring 31(4))299-308, 2000. ISSN 1359-835X. doi: http:///oaaooooooo..o.oossssssss
URL https:
7/www. ici encedi www.tetttton.oo wttp://wwcc..cccococcocccooaaaan

---

Published as a conference paper at ICLR 2022
Rahul Narain, Armin Samii, and James F O'brienn Adaptive anisotropic remeshing for cloth simu- lation. ACM transactions on graphics (TOG), 31(6):1-10, 2012.
Ethan M. Parsons, Tusit Weerasooriya, Sai Sarva, and Simona Socrate. Impact of woven fabric:
Experiments and mesostructure- based continuum-level simulations. Journal of the Mechanics and Physics of Solids, 58(1 (3111))9995 11)::)9999222111 2010. ISSN 0022-5096. doi: https://doi.or/1101000
j.jmps. jjimps.210.00500000 006. URL https:tpp/... /www. sci encedi rect. com/ ence//riicc/ pi p11/8002250961000 00000967
Peng, J. Cao, Chen, P. Xue, D.S. Lussier, and L. Liu. Experimental and numerical analysis on normalization of picture frame tests for composite materials. Compovites Science and Technology, 64(1):11-2) 2004. ISSN 0266-3538. doi: http://0ooop/oo00006522666665550
www htt.l//ssss.aaaaiiiccccc.coooeaa
3002021
Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. Learning Mesh-
Based Simulation with Graph Networks. September 2020. URL https:://openreeiewt
José M Pizana, Alejandro Rodríguez, Gabriel Cirio, and Miguel A Otaduy, A bending model for nodal discretizations of yarn-level cloth. In Computer Graphics Forum, volume 39, pp. 181-189. Wiley Online Library, 2020.
Xavier Provot et al. Deformation constraints in mass-spring model to describe rigid cloth be- haviour. In Graphics interface, pp. 147-147. Canadian Information Processing Society, 1995.
Abdullah-Haroon Rasheed, Victor Romero, Florence Bertails-Descoubes, Stefanie Wuhrer, Jean- Sébastien Franco, and Arnaud Lazarus. Learning Measure the Statie Friction Coefficient in Cloth Contact. In CVPR 2020 IEEE Conference on Computer Vision and Pattern Recognition, pp. 9909 9919999988.11 Seattle, United States, June 2020. IEEE. doi: N0.1109/CVPR42200.20000000 URL htR.https:// //httss::/haaa irrriiiii........ / halhaaaaa- 0251 646
Tom F H. Runia, Kirill Gavrilyuk, Cees G M Snoek, and Arnold W M Smeulders. Cloth in the wind: A case study of estimating physical measurement through simulation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Rosa M Banderas, Alejandro Rodríguez, Héctor Barreiro, and Miguel Otaduy. Robust eulerian-onn-agrangiaa rods. ACM Transactions Graphics (TOG), 39(4):59-1, 2020.
Connor Schenck and Dieter Fox. Spnets: Differentiable fluid dynamics for deep neural networks. In Conference on Robot Learning, pp. 317-335. PMLR, 2018.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy 2017.
optimization algorithms. arXiv preprint
Siyuan Shen, Yin Yang, Tianjia Shao, He Wang, Chenfanfu Jiang, Lei Lan, and Kun Zhou. High- order differentiable autoencoder for nonlinear model reduction. SIGGRAPH (ACM Transactions on Graphics), 40(4), August 2021.
Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems, 25, 2012.
Georg Sperl, Rahul Narain, and Chris Wojtan. Homogenized yarn-level cloth. ACM Transactions Graphics (TOG), 39(4):2, 2020.
Richard Stribeck. Die wesentlichen eigenschaften der gieit-und rollenlager-the key qualities of sliding and roller bearings. Zeitschrift des Vereines Seutscher Ingenieure, (6(38):1342-13444 1902.
Shinjiro Sueda, Garrett L. Jones, David I. W. Levin, and Dinesh K. Pai. Large-scale dynamic simulation of highly constrained strands. ACM Trans. Graph., 30(4), July 2011. ISSN 0730- 0301. doi: 211144452200322419999922 URL 64934
12

---

Published as a conference paper at ICLR 2022
Min Tang, Dinesh Manocha, and Ruofeng Tong. Fast continuous collision detection using de- forming non-penetration filters. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, I3D 10. pp. 7-13, New York. NY, USA, 2010. Associa- tion for Computing Machinery. ISBN 978 605589398. doi: 10. 10.1145/1730044. 1730806. URL
https: wrrg//rrgggllo. 1 1111/113010044411 7738877778004440 1736806
Wang, Paton, and J.R. Page. The draping of woven fabric preforms and prepregs for production of polymer composite components. Composites Part A: Applied Science and Manufacturing 30 (6):757-765, 1999. ISSN 3359-835X. doi: http:://woooggg//ooooo66//111s/2
URL mtttos://wiit www.wwww.c........... eeceeiieeeiiicc rect. cow.eeee.oeetc...oooooooooo..ee htt.a//cww...........o.oaaatttta 80C1870 1870
Kun Wang, Mridul Aanjaneya, and Kostas Bekris. A First Principles Approach for Data-Efficient System Identification of Spring-Rod Systems via Differentiable Physies Engines. In Learning for Dynamics and Control, pp. 651-665. PMLR, July 2020. DRR/httpp//prooeeedngg mll.
press/v120 wwng html
ISSN: 2640-3498.
Shan Yang, Junbang Liang, and Ming Lin. Learning-basee based cloth material recovery from video. In 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4393 0493. IEEE, 2017a. ISBN 9781538610329.
Shan Yang, Junbang Liang, and Ming C Lin. Learning- cloth material recovery from video. In Proceedings of the IEEE International Conference Computer Vision, pp. 4383 1383-4393, 2017b.
Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. A Differentiable Contact Model to Extend Lagrangian and Hamiltonian Neural Networks for Modeling Hybrid Dynam- ics. arXiv:2102.c 06794 [cs], February 2021. arXiv: 2102.06794.
Yi Zhou, Muhammad Ali, Xiaozhou Gong, and Dan Yang. overview of yarn pull-out behavior of woven fabrics. Texile Research Journal, 19:00405175177411 November 2017. doi: 10.117 7/0040517517441156
13


Reviews:
[{"review_id": "bHylerHlJwu", "reviewer": "Reviewer_8QNo", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n* The paper studies a challenging topic in differentiable physics, namely, the fine-grained differentiable cloth simulation.\n* The paper contains detailed problem formulation and derivations.\n* The paper demonstrates promising results on the synthetic data compared to the baselines [Liang et al., In NeurIPS 2019] and Bayesian optimization on material estimation.\n\nWeaknesses:\n* The paper does not seem to comment on the computational cost and comparisons to baseline methods. It would be good to provide such information in the main paper, especially when compared with non-differentiable simulator [Cirio et al., 2014] and prior work on differentiable cloth simulation [Liang et al., In NeurIPS 2019].\n* The quantitative results can be presented in a clearer manner. For example, it would be good to show the material prediction error in percentage in Table 2-4 (rather than the absolute value).  It would be good to include the algorithm (refer to Section 3.1 in [Liang et al. 2019]) in the main paper. \n* It would be good to know whether the proposed differentiable simulation is deterministic or not? Whether the material estimation is sensitive to initialization? In this case, please consider reporting the standard deviation for multiple runs of the same setting.\n* It is unclear what happens if the parameter prior was not provided to the proposed method in the control learning experiment. It would be good to provide such ablation studies and discuss the results in the rebuttal.", "summary_of_the_paper": "This paper studies the fine-grained cloth simulation using differentiable physics models. The paper is largely based on the prior work on cloth simulation [Liang et al., In NeurIPS 2019] but with the domain-specific yarn-level formulation. Specifically, it proposes to represent cloth as two perpendicular groups of parallel yarns. Built upon this representation, the paper introduces several differentiable strategies to model inertia, internal and external forces. Experiments have been conducted on simulated data for material estimation and model-based control learning.", "summary_of_the_review": "Overall, this is an interesting paper with solid technical contributions. The main weaknesses include the presentation and experimental evaluations.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "VwXE-yVvshL", "reviewer": "Reviewer_e46u", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The proposed model allows to estimate physical parameters efficiently, ie. :\n    - from a small sized fabric only (then simulations can be carried on a larger cloth size) (Table 2)\n    - from a few frames only (table 3)\n- The comparison of (a) gradient descent on the proposed model with (b) bayesian optimisation and (c) a sheet model turns favorable for (a) in terms of error + convergence speed.\n- The force model is extensive (inertia, stretching, bending, contacts between yarns, friction, shear, collision between parallel yarns, gravity, wind…)\n- The submission is well grounded in previous literature for yarn modelling\n- Supplementary videos are a plus\n- Nice figures and good readability \n\nWeaknesses:\n- In 4.1 the “Learning” experiments should rather be called “fitting” a few parameters of the model. Indeed the applicability is very narrow: the prior on parameters is strong (restricted range), the model is only able to fit data for which there is a solution (it is a simulated yarn fabric) and for which we have the exact positions of the intersections and the number of yarns. This setting is unrealistic. What would happen if we were to fit a 3D mesh, or a 3D sparse scan, for which the positions q_i are not available?\n- An ablation study is missing: what would happen if the proposed differentiable forces were removed? Showing such a result would prove the relevance of devising a differentiable approximation for them\n- Fitted parameters $\\omega$ should be re-stated at the beginning of 3.4\n- The scalability claim does not really hold: the largest demonstration (support video) is for a 70*70 yarns which is still very crude.\n\n\nMinor typos:\n- End of introduction “… and fast in control learning”: it looks like a word is missing\n- Related work / differentiable physics simulator “simulaiton” -> simulation\n- Beginning of Methodology (+repeated): “system equation” -> system of equations?\n- In 3.2 the dimension $l=3rx+2(r-2)(c-2)$ is not equal to what is stated in 3.1 : $(r-2)(c-2)$ crossing nodes with 5 DoFs and $2r+2c$ with 3 DoFs gives a dimension of $l’=l+12$\n", "summary_of_the_paper": "This submission proposes a differentiable parametric force model for fabric, at the level of yarns, by modelling interactions forces between yarns. Since some of these forces are not differentiable, the authors propose a differentiable approximation for them.\nAs a results, gradient descent can be used to recover physical parameters, by fitting the parametric model to a dynamic sequence of frames. Differentiability is also used to optimize forces for control: gradient descent can optimize the required forces to apply to a piece of fabric to throw it in the air and aim for a target position.\n", "summary_of_the_review": "This paper proposes a new differentiable force model for yarn fabrics which is novel and able to fit clean data for which the positions of the yarns are known. Yet the experiments fail to prove it is useful to fit general fabrics in a less controlled setting.\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "UIOHRDG0gF-", "reviewer": "Reviewer_u4Fx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pluses:\n* The authors identify a major shortcoming of previous simulation models, namely the use of discontinuous force terms, and engineer alternative models that approximate the terms from domain literature but have the added benefit of being smooth everywhere. I have to say I quite like this general approach of searching for the sweet spot between textbook physics and usability of the model for solving secondary problems.\n\nQuestions:\n* You write that the non-linear stick-slip force term lends itself to oscillatory behavior. Without being familiar with such simulations in the slightest, I feel that this could lead to free energy being trapped in entropy and therefore thermal loss. Did you ever encounter such behavior?\n\n* Why do differences between inter/intra parameters (Table 2) and ground truth as the grid size increases? Should it not be the other way around?\n\nWeaknesses:\n* In the Comparison video, I could not spot any significant difference between the different weaves. At least not more than between Ground Truth and Ours. Did you also experiment with a bit more extreme differences in fabric?\n\n* It seems that no real-world experiments were done. The range of simulated experiments is not very wide, so I can't judge the expressivity / gamut of the model.", "summary_of_the_paper": "This paper introduces a model for yarn-based fabric simulation that is differentiable and hence particularly well suited for solving inverse problems, such as deriving model parameters from reference video input. \n\nThe technical contribution itself (execution of the forward model) does not necessarily fall into the realm of machine learning but it is nicely embedded in an inverse framework and demonstrated on a scenario that is clearly computer vision. Hence, I would consider the paper on-topic for the venue.\n\n", "summary_of_the_review": "Again, I am no expert, but I think this is a paper that fits its time. It appears to be well executed and evaluated. Whether or not the works falls under \"machine learning\" is probably a matter of personal taste. The contribution is a simulation that serves as forward model component in a regression framework, so I think that should qualify it.\n\nAs the authors acknowledge, neither the specific problem nor the general idea of differentiable simulation is new. Whether the specific solutions introduced in the paper generalize to other problems is unclear. Still, I think this is reasonably nice work to warrant acceptance.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FxBN2AzZKY", "reviewer": "Reviewer_Qcbf", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\nStrengths: \n\nThis is the first fine-grained data-driven yarn-level model for describing internal forces for fabric. The differentiable nature of the forces allows this model to be incorporated in a learning-based cloth simulation framework, handling complex physical structures and heterogeneous materials at the yarn-level.   Experiment results show the model achieves significantly lower errors than previous methods and has high data efficiency.  Although only cloth simulation is demonstrated , the model can have wider applications in fabric manufacture and design areas.  The underlying principle may be extended more generally for other related tasks.  \n\n\nWeakness:   \n\n1. The paper incorporating some prior knowledge about the parameters’ range in order to mitigate illposedness.  However, the entire design process  is rather heuristic: neither analysis nor ablation study is presented to justify how such those design choices were made.  \n\n2. In order to illustrate data efficient of the proposed method,  the authors trained the same model on different frame numbers ranging from 5 to 25 for low-dynamics motions.   There is no mention of how the method performs as a function of the magnitude of the forces.  Additional experiments can be done and should be done. \n\n3. In paragraph-2 of section 4.2.1, the paper claims “When the training data increase from 5 frames to 25 frames, the error reduces by around 100 times.  However, in Fig-6 left, the reduction of error is only 10-fold.  Similarly in Table-4,  the error is reduced only by about 10 times rather than 100 times, unless I misread this section. \n\n4.  Experiments have been focused on simulated woven cloth.  Yet, the models are heavy-handedly designed. This casts some doubt regarding the generalizability of the proposed method. How does  the proposed force model on other different materials and weaving structures is unclear. \n", "summary_of_the_paper": "This paper proposes a new differentiable physics-based model for simulating composite materials such as fabrics in cloths.  This is approached by formulating  several common differentiable forces exhibited in yarn-to-yarn interactions in different waving patterns.  Different from previous generally-purposed physics model, the proposed method is the first data-driven fine-grained yarn level modelling that takes into account complex structure/topologies and force interactions.  It models fabrics with composite materials and slim patterns and proposes new differentiable forces on/between yarns. Experiments show the efficacy of the proposed model, confirming the model’s capability for simulating heterogeneous materials such as fabrics.\n\n", "summary_of_the_review": "Despite  that the experiment section of the paper can be improved in many aspects, I think the core contribution of the work is new and valuable, and the paper is acceptable condition on some minor revision.  \n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DvHNTFExQmZ", "reviewer": "Reviewer_Hh1F", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The modeling of this differentiable physics paper is very fine-grained compare to prior papers. Their contact dynamics can even model the Strobeck effect and also simulate self-excited vibrations. I would very much like to see those detailed dynamics in their video. (By the way, the authors mentioned the supplementary video in the paper several times but I could not find the supplementary materials. Did the authors upload it? Or is there something wrong with my reviewer console?)\nThe authors also conduct extensive experiments to support their claims. \n\nI also have some concerns after reading the paper,\n1. The fine-grained model is accurate. I wonder what its time performance is. Could the author report the wall clock time? I also notice the time step is relatively small (0.001s). What is the biggest time step the simulation can deal with?\n2. How this method can scale to some more complex topology? All the experiments are on one square sheet. Can your method simulate something like a T-shirt or dress, which is probably more relevant to design and fabrication?\n3. Could the author release the code and video as supplementary materials?\n", "summary_of_the_paper": "This paper proposed differentiable yarn-level dynamics for fabrics. Compared to the previous differentiable cloth model, this paper is more specific and fine-grained. They carefully design the shear force and Coulomb friction force so as to keep the physics accurate as well as differentiable. The experiments show that their method can learn the control policy and material parameters successfully.", "summary_of_the_review": "This paper presents a more fine-grained model compared to other differentiable physics papers. Claims are well supported by the experiments. It would be better if the author can better demonstrate their application to more complex scenarios (like a T-shirt or dress instead of a single square sheet).", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "51Zk-4Bn9Vp", "reviewer": "Reviewer_U3Yi", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I believe the implementation of this paper is respectful and non-trivial. It looks like that this paper fills the gap in differentiable simulation with a cloth simulator from a yarn-level perspective. Despite its potential applications, I have a few technical questions and suggestions about it:\n\n1. This manuscript provides a novel view of the intersection of differentiable simulation and cloth simulation by thinking at the yarn level. I believe one of the important baselines is [1], which was the latest work working on differentiable cloth simulation in a mesh-based view. I expect some discussion between this work and [1].\n2. The statement \"Previous research (Liang et al., 2019; Du et al., 2021) suffers from this problem, which unfortunately leads to learning implausible parameter values\" looks technically inaccurate to me. Actually, leveraging prior knowledge for physical parameters is now a necessary step for system identification tasks. For example, because of the huge numerical difference between Young's modulus and Poisson's ratio, [2] proposes a logarithm transform to alleviate the unbalancement.\n3. I am curious about the wall-clock time and memory footprint of the experiments. It will be helpful to note them.\n4. As I questioned in the summary: a physical simulator usually comes with a steep learning curve which disables following works by the implementational difficulty. I wonder if the source code will be released for future follow-up.\n5. My main concern is motivation. The yarn-level simulation was proposed for more accurate collision handling and detailed presentation. However, the experiments are too coarse to show these advantages, which makes me question why the yarn-level simulator is used. As a reference, [3] used much more yarns to represent their cloth. I expect a more valid explanation for motivation or a set of better results that are more complex than 25*25.\n\n[1] Li, Yifei, et al. \"DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact.\" arXiv preprint arXiv:2106.05306 (2021).\n\n[2] Hahn, David, et al. \"Real2sim: Visco-elastic parameter estimation from dynamic motion.\" ACM Transactions on Graphics (TOG) 38.6 (2019): 1-13.\n\n[3] Sperl, Georg, Rahul Narain, and Chris Wojtan. \"Homogenized yarn-level cloth.\" ACM Trans. Graph. 39.4 (2020): 48.", "summary_of_the_paper": "This manuscript enhances the yarn-level cloth simulation to a differentiable level by introducing alternative differentiable operators in place of originally discontinuous/indifferentiable counterparts. In addition to differentiable components for energies, collisions, frictions, etc, this manuscript also differentiates through the implicit Euler method, which enables greater time steps with better stability. The authors empirically studied the efficacy of this differentiable simulator on a variety of environments with multiple baselines. I believe one of the major contributions can be a non-trivial implementation in C++ and its interfaces with modern deep learning architectures (unsure if it will be open-sourced through).", "summary_of_the_review": "This manuscript proposes a combination of yarn-level cloth simulation and differentiable simulation and shows the application of system identification and control. The thing I am worried about is that the motivation of using yarn-level, which is originally designed for fine details but applied on over-simplified models. I would like to know more about why the authors chose the yarn-based method, how they would like to use this simulator, and where they see the simulator fits in. Generally, I appreciate the work but was not convinced by the motivation. I am open to changing my score based on the clarifications I get.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a differentiable yarn-level model of fabrics. The model is more detailed and physically realistic than proposed in earlier work, which may allow for applications to manufacturing guidance and textile design.
The paper is generally well-written and contains detailed problem formulation and derivations.
Experiments show it is possible to successfully learn a control policy and material parameters using the differentiable model.
Decision: Accept (Poster)