Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: MULTI-LAYERED 3D GARMENTS ANIMATION

This paper presents a novel deep learning-based 3D garments animation method. The method learns to model interactions between 1) cloth-body, 2) multiple cloth layers 3) wind and clothes. Specifically, the algorithm decomposes the clothes into connected patches and evaluates the deformation based on a learning-based particle physics simulator (Shao et al., 2022). A new simulation dataset has been proposed to study this new setting. 


strength_and_weaknesses: Pros: 
- The new capacity for learning-based clothes simulation to handle both wind-cloth, body-cloth, and inter-cloth interactions. 
- Superior performance compared against DeePSD through a thoughtful exp design. 
- The paper is clearly presented. Equations/tables/figures are all easy to parse. The intro is clear. Related covers most of the relevant works.
- New large-scale dataset with realistic simulation and unique designs (e.g., wind). 

Cons:
- The problem setting of data-driven animation here is not well-motivated. 
- The comparison study is somewhat limited. 
- Lack of qualitative demonstrations and comparisons. 

Mixed:
- Technical contribution. The paper belongs to a novel application / resemble of existing techniques (1. data-driven particle-based simulation from Shao et al. and 2. patch-based garments representation used in Ma et al. and earlier works. e.g., Kavan et al. Siggraph 11, Feng et al. 2010)
- The above two core contributions the author claimed to differentiate the proposed work from prior works are not entirely original. That said, I still appreciate the novel ensemble and applications to data-driven simulation under the wind. 


clarity,_quality,_novelty_and_reproducibility: The data-driven motivation is not particularly clear to me. 
- In this work, the proposed method learns to mimic a physical simulator’s behavior. The author needs to justify its use cases, i.e., “why data-driven method?” Given that we have options for physical-based clothes animation solutions, I expect the author to discuss when and why data-driven is beneficial. 
- I think the author can answer the question at least from one of the three aspects: 1) the data-driven method is faster than the Blender’s simulator; 2) the data-driven simulator is better than a real-time physical-based simulator (e.g., 3) data-driven simulator can offer other capacities (e.g., system identification, where end-to-end differentiability allows inferring physical params through inverse problem solver; or sim2real, where the learned particle-dynamic model can be applied for real-world data). 
- However, I cannot find a runtime report or comparison against a real-time garment simulator or a demonstration of downstream demonstrations. 

Dataset details:
- It’s also unclear how fast Blender’s clothes simulation is and what method they use (it is also particle based?). 
- Training/testing splits -- during testing how many samples are from unseen poses / unseen human shapes/ unseen clothes materials? 

Overall the comparison experiments are not entirely thorough. 
- First, it should be compared against a differentiable particle-based physical solver, if possible, such as DiffCloth, or a real-time physical solver for a speed-performance trade-off. 
- Another data-driven method, SCALE (Ma et al.), should also be compared. SCALE might not be directly modeling wind. But the author can also add wind as an additional attribute, similar to what the author did for DeePSD. 
- Besides, the paper also lacks qualitative comparison against DeePSD. It seems the performance gap is quite large quantitatively, but I would also like to know how it works visually compared to DeePSD. 

summary_of_the_review: Overall I am on the fence. I would like to see how the author addresses my aforementioned concerns and questions. 


----------------------------------------------
Post-rebuttal. 

The authors addressed most concerns by providing additional experiments and revised the submission accordingly. 

Overall, it is a weak-accept to borderline-ish submission. 

The paper presentation is quite clear, and the technical approach is straightforward, as pointed out by both the authors and reviewers -- fusing particle simulation in TIE (Shao et al.) with patch-based representation in SCALE (Ma et al.) for efficiency. The method achieves the SOTA in runtime and speed and can handle external force and unseen clothes types.  

It is a well-executed and well-presented paper after the revision. The proposed combination is somewhat new and, indeed, solves practical challenges of prior arts (efficiency, external force, and multi-layer interaction). It's not a ground-breaking or surprising idea, but it sets a new state for the field and brings new capacity. Hence it doesn't harm to get accepted for its practicality.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Authors address the task of simulating multi-layered 3D garments on a human body. Authors fist curate a dataset of multi-layered garments under various poses and wind effects and use this data to train LayersNet that predicts garment deformations as a function of body shape and previous garment state. 
strength_and_weaknesses: Strengths:
+ The task of modelling multi-layered garments is quite challenging and not a lot of works address this.
+ Typical works on garment modelling on human body do not consider external factors such as wind, which is interesting in this work.
+ Authors chose particle based physics modelling for the garments which is topology independent and allows modelling external factors like wind as wind can be treated as a set of particles.
+ The authors will release the dataset and code.

Weaknesses:
- Technical novelty seems limited, as the garment modelling is handled by TIE, Shao et al. 2020. Can the author clarify the differences between the approaches.

- In eq. 3 the garment is modelled as a function of body and previous state of the garment. Since the paper primarily deals with multi-layered garments, should't the garment also be a function of other garments on the body?
eg: let's say a person is wearing a loose skirt and a long jacket on top, such that the bottom of the jacket draper around the top of the skirt. Isn't the jacket deformation more dependent on the skirt than the body?

- Eq. 14 only handles collisions between the body and the garments, but since authors deal with multi-layered garments, what about the collisions between different layers of garments?

- Related work on multi-layered garment is completely missing. This important as multi-layered garments are the main contribution of the work. Authors should add a paragraph about it as currently it appears that the authors are the first one to address multi-layered garments which is not true.
--> SMPLicit, CVPR'21 (not cited) has shown generation of multi-layered garments.
--> SimulCap, CVPR'19 (not cited) Also use multi-layered garments + physics based simulation for motion capture.
--> Another concurrent work: Layered-Garment Net, ACCV'22 (not cited) also generates multi-layered clothing using simulated data.

Clarifications:
- Eq. 12: Shouldn't this be a double summation. One over multi-layered garments and other over vertices of the garment.
- What is f(.) in the line just above eq. 3?
- There are existing works like TailorNet, Patel et al. 2021 that also use PBS to drape garments on human body. Can the authors also provide details on how simulating multiple layers is more challenging than simulating single layered clothing?

clarity,_quality,_novelty_and_reproducibility: The paper is well written and I could comfortably follow it. The work should also be reproducible since authors promise to release code. Will the authors also release the simulation code? This could be very useful for the community.The originality seems a bit limited to me as authors primarily rely on existing work TIE, Shao 2020 to handle garment  modelling (see weakness section). It will help if authors address this point as it is one of the main contributions of the work.
summary_of_the_review: I like the task and formulation that authors propose in this work. It is not clear to me how is the existing garment modelling different than the particle based modelling in TIE, Shao 2020. Moreover, parts of the formulation seem to be tailored for modelling single layered garments and not multiple layers. eg: intersection is handled only with the body and garment and not garment-garment.

Can the authors also provide video results from their method. This is important as the authors deal with garment simulation over time and we should see the quality of the simulated data (the proposed dataset) as well as the results from LayersNet. With provided results it is difficult to access whether the clothing deforms smoothly or not.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new dataset of garment motions under different human movements as well as wind forces. It also introduces a new network that can predict garment motions of different layers at the same time.
strength_and_weaknesses: Strengths:
1. Multi-layered cloth motion dataset is indeed in great need. This dataset can boost the advances of learning-based garment estimation and animation research.
2. Results show that LayersNet has a better performance than previous work in terms of predicting multi-layer garment motion.

Weaknesses:
1. Missing references:
 - Physics Simulation by Neural Network: Liang, Junbang, Ming Lin, and Vladlen Koltun. "Differentiable cloth simulation for inverse problems." Advances in Neural Information Processing Systems 32 (2019).
 - Data-driven Cloth Model: Shen, Yu, Junbang Liang, and Ming C. Lin. "Gan-based garment generation using sewing pattern images." European Conference on Computer Vision. Springer, Cham, 2020.
2. In Table 3, is there any reason why the LayersNet is not trained and tested with inner garments only as well? Otherwise, I think DeePSD for inner garments only is not needed either.
3. Does the wind velocity affect the accuracy of the model? How well does the model perform with different wind speeds? In the simulation, very minor differences in the initial setting can cause significantly different end results (i.e., known as instability of the simulation). Does that happen as well for the model? If so, is there any particular way to overcome this so that the prediction errors can make more sense? Please add some discussions regarding this topic.
4. In Table 4, the proposed method does not perform as well as previous works in regards to L-Collision and H-Collision. The authors should analyze and explain the phenomena but I didn't see any in the paper.
clarity,_quality,_novelty_and_reproducibility: See above.
summary_of_the_review: Although the dataset is a good contribution, the proposed method has some clarity issues that prevent this paper from meeting the bar. I would be happy to change my score as long as the issues above are addressed.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary: The paper proposes a method to animate 3D garments by modelling interactions of cloth-body, cloth layers, and cloth-wind.  The proposed method combines data-driven particle-bases simulation (Shao et al), with patch-based garment representation (Ma et al), and uses a neural network to learn to decode the simulated patch embeddings associated with each vertex to obtain updated vertex position and velocity for the next time step.  A new large-scale synthetic 3D garment animation dataset (LAYERS) is also provided to study animation of multi-layer garments.  

Strengths:
- The proposed method involves combining existing techniques to solve limitations of prior work in a practical and efficient way
- The proposed approach demonstrates capability not previously possible (modelling of multi-layer garments, effect of wind on cloth) and can stimulate additional work on data-driving animation of garments
- The proposed method shows clear superior performance against prior approaches
- A new dataset with realistic simulation is provided
- The paper is clearly written and well presented

Weaknesses:
- The writing (esp. introduction and beginning of section 4) should be improved to make it clearer that the proposed method is a combination of prior work 
- Limited comparisons and qualitative examples 
- Reviewers found the technical contribution to be limited as it is a combination of existing techniques 
- The problem of multi-layered 3D garment animation is fairly specialized and may not be of interest to the broader ICLR community 
justification_for_why_not_higher_score: Reviewers are not that excited about the work.  Both positive reviewers indicated they are okay with rejection.  In addition, the AC agrees the work has limited appeal to the ICLR audience and it would be better suited for a graphics / animation venue.  Finally, the AC does believe that the paper would benefit from a clearer indication that it is a combination of prior methods in the introduction and beginning of the methodology section. 
justification_for_why_not_lower_score: The work is well-executed and can potentially stimulate additional work in the area of multi-layer garment animation. The combination of the prior techniques is a novel combination and is also shown to be effective, and demonstrate new capability that was not previously possible.
Decision: Reject