Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: This paper proposes a method to learn the cloth deformation of a t-shirt given
the skeletal pose of an upper body. The method skins a thick tetrahedral mesh to
the skeleton and embeds the t-shirts cloth within. At inference time a network
predicts a rest pose displacement before conducting skinning via barycentric
lookup in the tet mesh. The method is trained (as far as I can tell) on some
groundtruth cloth simulation method (this is not revealed).

I recommend rejecting this paper from ICLR 2021 on several grounds: 1) the
results are poor, 2) the description is hard to follow, 3) the methodological
choices are not well motivated, 4) the method as written is not reproducible,
and 5) the claims are too general.

1) In terms of topic and methodology this paper would be an appropriate
submission to SIGGRAPH or SCA. Callibrating for the expected result quality at
either venue, I would recommend acceptance. Since the machine learning component
of this paper is not a contribution besides being an application of
"off-the-shelf" tools, I do not see reason to lower these standards for ICLR.

2) After reading the paper, I eventually feel I understand this method. 
Important details are left out effecting replicability (see below). The paper
does not clearly state what the input and output is. The paper does not describe
how groundtruth data is generated. 

"This is done by first sorting the tetrahedra on the list based on their largest
minimum barycentric weight, i.e. preferring tetrahedra the vertex is deeper
inside" I don't understand this. Barycentric weights are largest when near a
vertex. Meanwhile tetrahedra can be very pancake/sliver-shaped, so that
regardless of the barycentric coordinates, a point is never deep inside. Using
barycentric coordinates to measure depth of penetration is misguided.

I didn't understand "method 2". In that method is the tet mesh entirely ignored?

Is Figure 4 showing training data or withheld testing poses?

3) The paper immediately jumps into the idea of embedding the cloth of a t-shirt
in a bulbous tetrahedral mesh around the upper body. This isn't questioned until
later when all sorts of issues appear due to overlapping and inverted elements.
There was no reason to think that skinning such a thick tet mesh was a good idea
in the first place. So the "INVERSION AND ROBUSTNESS" section is describing ad
hoc heuristics to a problem that could have been avoided by starting with a more
sound premise.

The working premise is that pose space deformations can be used for efficient
cloth simulation. This is reasonable and traces its heritage to "A powell
optimization approach for example-based skinning in a production animation
environment," which should probably be cited. From there, the choice of using a
thick tet mesh comes without solid motivation. Why not, for example, instead
learn the skinning weights and displacements directly? So, that for a point p on
the cloth the final deformation is:

∑ wi(p,θ) Ti(θ) (p + d(θ,p))

?

To generalize across body types etc., rather than the heavy handed proposed
approach of sharing this mysteriously skinned tet mesh, the learned w and d
function could be predicted based on some relative position to the rest pose and
t-shirt size etc.

4) This paper is far from replicable. How is the groundtruth data computed? Some
cloth simulation? Which method? Are collisions handled in that method? 

Why does the surface boundary in Figure 1 (c) look so spiky yet the input level
set is smooth? This does not appear in the results of the red/green
tetrahedralization results. This looks more like a simply clipped regular grid
tetrahedralization.

How are the weights wkj determined? Manually? Automatically? Optimized during
training? This appears to be crucial to the method but left out.

5) Finally, this paper claims to provide a "Skinning a parameterization of
three-dimensional space for neural network cloth". Even if I accepted this paper
as successful in its results (I do not), then this paper could at best claim to
have skinned a parameterization of t-shirt deformations for upper-body motions.
The fragility of the method as discussed above makes this overclaiming
especially dubious.

review_title: Reject

final_score: rating: 6: Marginally above acceptance threshold
review_content: I'm not very familiar with 3d cloth animation, so may not provide an entirely adequate evaluation. However it looks to me that this paper is more suited to the graphics conference like SIGGRAPH.

The paper present a new method for cloth deformation based on tetrahedral KDSM mesh, to make the deformation more natural the neural network is used to predict the offsets  that match the ground truth deformation. Differently from Jin et al. (2020), in this paper volumetric region of air surrounding the underlying body  is used.

The are several questions about the experimental part of the paper.

1. The paper only compares on a single dataset of the Tshirts. So it is not clear how well the model will perform on other types of  clothes: regular shirts, dresses and so on.  The paper however claims that the method could be potentially to more broad categories such as hair and fluids. Is it possible to add more comparisons on the other clothes or object types?

2. How the dataset of Tshirt meshes been obtained, is it synthetically generated? If it is synthetically generated what is the benefit of using data-driven learning method? Could the method be applied on the real world scanned meshes?

Minor issues:
- Figure 1(d) looks not intuitive, does not look like (d) is modification of (c) and (d) does not look as shirt at all. It is better to use some small modification to make it more intuitive.
- Supplementary material should go along with the paper as a set of Appendixes. Only videos and source code should go as zip archive.
- Paper contain weird green artifacts in the end of first and second pages which should be removed.
- It is hard to switch attention from Figure 3 to Figure 4, these figures should be joined.

—-------—---------------

The rebuttal did not change neither my confidence, nor my impression about the paper. So I did not change my rating, however I acknowledged that other reviews may be more proficient to judge this paper properly. 
review_title: Seems like a good paper, but more suited for some graphics conference

final_score: rating: 4: Ok but not good enough - rejection
review_content: The authors derive a volumetric extension of the surface parameterization approach developed by Jin et.al. Towards this, they propose to use tetrahedral parameterization using well known techniques in computer graphics community. The kinematically deforming skinned mesh (KDSM) formulation for tetrahedral parameterization is borrowed from Lee at. al. 

The combination of the above techniques coupled with some heuristics to increase robustness to inversion suggest improvement over Jin et. al. This is a very niche topic and I am not confident that the general audience stands to benefit from this specific formulation for clothes. The ICLR community would benefit by demonstrating the approach on other deformations of solids/liquids and validating the generality of the approach compared to other representations beyond virtual cloth. Only comparing to Jin et. al significantly limits the scope of the paper. 

The computational complexity of the approach is completely ignored. As the gains over Jin et.al. seem to stem from extending the formulation to 3d domain, the compute should be compared. This becomes important for high dimensional solid/liquid simulation. 
review_title: Volumetric parametrization improves over surface parametrization. Insufficient novelty and experiments.  

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes to model 3D cloth by embedding it into kinematically deforming skinned mesh (KDSM)[1], a tetrahedral mesh that parametrizes the volumetric region around the underlying body. A KDSM can be created and deformed using a variety of skinning and simulation techniques introduced in [1]. This paper extends KDSM by enabling plastic deformation in material space (T-pose), and accurately models the cloth deformation as per-vertex offsets. Inspired by [2], this paper trains a neural network to learn the per-vertex offset as a function of body pose. Once trained, the network is able to infer the 3D cloth on a particular body. Experiments show that the proposed 3D cloth parameterization method is better than the 2D UV parameterization method used in [2].

Strengths
- This paper proposes a new approach to modeling 3D cloth deformation. Experiments demonstrate that it outperforms the UV parameterization method on modeling the per-vertex offset as a function of body pose. It has the potential to be applied to other cloth-related tasks.
- This paper successfully adapts KDSM, which is originally used for hair modeling, to clothes. The inversion and robustness issues are addressed. It would be inspiring for other researchers to apply the similar idea on other types of objects.
- This paper is clearly organized and well-written. There are sufficient technical details presented in the paper.

Weaknesses
- Some existing works parameterize cloth deformation based on SMPL, e.g., SMPL+D [3], TailorNet [4], Tex2Shape [5].  This paper lacks the comparison with these closely related methods. Therefore, it is not clear whether the proposed KDSM-based cloth parameterization method is better than existing SMPL-based methods.
- There is only experimentation on synthetic data. It is not clear how it performs on real world data like BUFF [6].
- At the end of Sec. 6, the authors claim that “the hybrid method is able to achieve greater temporal consistency”. It is not clearly which component of the method enforces temporal consistency.

Suggestion
- In Sec. 5, the authors elaborate on how to robustly handle tetrahedra inversion and overlapping when generating training examples. Being depicted in natural language, the entire process is too complicated for readers to follow. For example, the sentence “We prune this list of tetrahedra, keeping only the most robust tetrahedron near each element boundary …” makes readers wonder how the robust tetrahedron and element boundary are defined. It would be better if the authors can provide a piece of pseudocode to explain the entire process in a compact and precise manner. A graphic illustration of the key operations would also be helpful for readers to understand the process.
 
[1] Lee, Minjae, et al. "A skinned tetrahedral mesh for hair animation and hair-water interaction." IEEE transactions on visualization and computer graphics 25.3 (2018): 1449-1459.
[2] Jin, Ning, et al. "A pixel-based framework for data-driven clothing." In Proceedings of the 19th ACM SIGGRAPH / Eurographics Symposium on Computer Animation, volume 39. Association for Computing Machinery, 2020.
[3] Bhatnagar, Bharat Lal, et al. "Multi-garment net: Learning to dress 3d people from images." Proceedings of the IEEE International Conference on Computer Vision. 2019.
[4] Patel, Chaitanya, Zhouyingcheng Liao, and Gerard Pons-Moll. "Tailornet: Predicting clothing in 3d as a function of human pose, shape and garment style." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.
[5] Alldieck, Thiemo, et al. "Tex2shape: Detailed full human body geometry from a single image." Proceedings of the IEEE International Conference on Computer Vision. 2019.
[6] Zhang, Chao, et al. "Detailed, accurate, human shape estimation from clothed 3D scan sequences." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.
review_title: This paper proposes to model 3D cloth by embedding it into a tetrahedral mesh that parametrizes the volumetric region around the underlying body. The idea is mainly based on KDSM[1]. Lack of experiment makes it not very convincing.

Meta-review: Three of the four reviewers recommend rejection; one additional reviewer considers the paper to be marginally above threshold for acceptance but is very uncertain and this is taken into account.  The AC is in consensus with the first three reviewers that this paper is not ready yet for publication.  

There is concern from the reviewers that ICLR is not the right venue for this submission.  The author response in "Submission Update" does not clarify this concern.  Training a neural network to solve the problem does not automatically mean that ICLR or other ML conferences are necessarily the right venue.  Regardless, due to the many other raised concerns e.g. limited experimental results and comparisons as well as clarity,  the AC recommends rejection for this paper and resubmission at a more appropriate venue.  
Decision: Reject