Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: 
Using the same bar as NeurIPS, I continue to recommend rejecting this paper.
Since the paper remained largely the same. My review remains largely the same.
It is also doubtful that these changes have satisfyingly address the other three
reviews.

This paper proposes an iterative PBD solver which uses a neural network to guess
the constraint force and position updates before polishing with a conjugate
gradient solver. The method utilizes a graph network architecture which makes it
agnostic to the particular discretization allowing generalization (in theory)
across scenarios.

Speeding up simulations is an evergreen topic. A strength of this work is the
problem that it is tackling. The choice of methodology makes it mildly of
interest to the ICLR community. It is an application (without adaptation) of
graph networks. The result is mildly positive (though not earth shattering),
indicating success of graph networks to some degree. I have not seen graph
networks applied to rod simulation or elasticity simulation, although they have
been applied to meshes in geometry processing (similar and more challenging
scenario) and neural networks have been applied more generally to speeding up
elasticity simulations (with what would be a trivial extension to rods). A
possibly unique strength of this paper is the combination of rod simulation and
a convergent solver (i.e., just using the network as an initial guess in an
iterative solve). However, this is a really straight forward idea and the gains
are again small. So, while this is a strength it is minor and possibly not
exciting to the broader ML/ICLR community.

My main criticism of this method have to do with three aspects of scaling: 1)
the network is applied at the inner most loop and message passing occurs by
iterating over edges (which I believe are based on the original connections).
This means that global information passing requires M = O(n) iterations. 2) the
method operates only on the fine scale details (the input resolution). This runs
counter to multigrid literature which would suggest operating on all the
residual at all frequency levels. 3) the method shows a small number of small
examples (low resolution rods with one or two rods in a scene). PBD is often
employed in scenarios with millions of rod segments from thousands of rods
(e.g., hair on a human head will have 100,000 rods each with hundreds/thousands
of segments). For a 1.5x to become impressive I would like to see this operating
at scale.

I worry that the network has simply memorized a mapping from global positions to
guesses. Since the network has access to the raw positions, I would be surprised
if it has learned rotation and translation invariance. Lack of this would be a
good indication that it's learning a spatial mapping. This would be a severe
limitation.

The exposition of main results is very confusing. If I understand this part of
Figure 5 (and I doubt I do), then the pink curve below the red and black curves
is indicating that using this method does not simply speed up the CG solve but
some how negatively affects the total runtime (I guess by confusing the outer
loop down an incorrect path). So the gains by taking an aggressive initial guess
are tempered a bit, though still resulting in a (quite modest) overall speed up
<1.5x.

This modest speedup coupled with the small number of simple experimental
scenarios worries me that these results will not generalize to a statistically
significant speed up in general. 

Collisions appear as an afterthought. If the speedups were more significant, I
would happily excuse this as collisions can often be an extra systems effort.
However, part of the "glory" of a PBD solver rather than an FEM+LCP type
simulation is that one can throw all sorts of constraints into the same system.
So, when this paper tacks on collisions outside the learning aspect of this work
it calls the choice of PBD into question as perhaps needlessly inaccurate or an
"purely-out-of-convenience" type choice rather than a scientifically motivated
one.

If I understand correctly, the paper always compares to a baseline of resetting
the position and multiplier updates to zero. Is this the best baseline? What
about using the previous iteration? Or other momentum based strategies? 

The for-loop on line 8 of the pseudocode is misleading/confusing/incorrect. This
implies that updates for rods are conducted independently. But then line 9
appears to accept as input and send as output coordinates and lagrange
multipliers for the entire system (rather than the rod i). What is the role of
the for loop of line 8 if line 9 does not depend on which rod is being
considered? it would be better to write out the linear solve that is being
conducted (e.g., with CG). The figure and text below confuses me further. Is the
correction guess applied: a) once per time step, b) once per constraint
lineariztaion (just before CG), or c) once per rod?

The revised paper (and perhaps rebuttal) should include a clarified pseudocode
to understand what this method is actually proposing.

The effectiveness of the network depends on a parameter M which would appear to
scale poorly with the resolution of the input shapes. Does M need to be adjusted
for higher resolution examples?


What is the video showing? Training data created using the groundtruth
simulation? Results of this method at test time? If so what was the training
data used for each? This video did not really help supplement this submission. I
wish that the video had included experiments that could be used to gain
intuition about what's being learned and about the accuracy of the initial
guess.

For example, the paper mentions many times that it is *not* replacing the time
integration/constraint projection with an end-to-end trained network for
robustness reasons. Let's see it fail then!

In a future revision I would like to understand
  - under what circumstances does end-to-end learning fail, but this method
    succeeds.
  - under what circumstances does simply using the previous frame's constraints
    as an initial guess out perform this method?
  
It'd be great to see a video where we also see an evolving plot per-frame (like
Figure 6) showing the performance gains of applying this method rather than
naive initialization methods. This would be great to get match up whether the
gains happen far from the rest state, in collision heavy scenarios, or the
opposite etc.

On line 180, I did not understand how the set of input edges to the graph
network is defined. If inputs nodes are assigned to each rod segment, then input
edges should connect two segments. Are only neighboring segments connected with
these edges? (e.g., the dual graph of the polyline representing the central axis
of the rod). Or are all possible pairs of segments generated? Where are Young's
and Torsion moduli stored? It's natural to store Young's modulus at segments but
this would correspond to nodes of the graph not edges.  

The graph network description (which gets at the core contribution of this
paper) is poorly described. I read this section many times and via cross
referencing with [13] could finally understand with low confidence what is being
done.

Figure 4 is very confusing. Is this figure showing that none of the hyper
parameters matter except for MLP width? This is surprising to the point of
indicating a bug/overfitting.

I do not understand Figure 5. What is the purple curve? the vanilla CG solver?
Then wouldn't its speed up be 1x? Are these plots showing two different y-axis
or two different examples (straight vs helix)? The caption is very confusing.

The paper does not accurately categorize past works when it writes "Existing
methods enable learning these systems often in an end-to-end manner and with a
focus on replacing the entire integration procedure." Many fluids papers retain
pressure projection to ensure divergence free-ness and within elasticity
simulation, 

Latent-space Dynamics for Reduced Deformable Simulation
Lawson Fulton, Vismay Modi, David Duvenaud, David I.W. Levin, Alec Jacobson

does not replace the time integration procedure.

In the future, I would appreciate a pdf in supplemental material with
highlighted changes. It is potentially rude to continuous reviewers to have to
hunt for small changes (and then see that their reviews were largely ignored).

review_title: Reject

final_score: rating: 6: Marginally above acceptance threshold
review_content: Summary: the authors present a Graph Network (GN) architecture to speed up the running time of rod physical simulations by predicting the initial guesses to reduce the number of iterations of an iterative position-based solver. The proposed approach offers speed-ups varying from 6-50%, depending on the problem analyzed. 

Strengths: 
The paper is well written and motivated, methods are very clear. There are substantial experiments to test for the paper's main claims on inference efficiency and out of distribution generalization, although mostly on small scale systems (up to 200 nodes) and short time horizons (100 steps).

Weaknesses: 

1) The biggest weakness to me is shown on Figure 6, when plotting the CG interaction ratio as a function of time, for the straight bending rod and the elastic helix, up to 1000 timesteps. Especially on the straight bending rod simulation (orange curve), the speed-up is not consistent across time. Further insight on why those oscillations are happening would be appreciated. In the elastic helix simulation (green), one also sees a huge drop right after the training regime range (50-100 steps). End-to-end approaches will generally offer more significant speed-ups (orders of magnitudes rather than the 6-50% provided by COPINGNet), but they might struggle to remain stable for longer time horizons. Therefore what one would like to see from an alternative approach like the proposed here, is a system that scales well with time. Figure 6 shows that's not the case here yet. 

2) It would have been interesting to see how the GN compares to a different neural network predicting the initial guesses (say a simple LSTM or -- more ambitiously -- a transformer). I understand that many of the successful neural approaches for physics simulations are GN based, so the choice is well motivated -- but those systems are generally trained end-to-end, which isn't the case in the work from this paper. Therefore, it's not clear to me that other non GN approaches wouldn't perform well on this task of producing better initial guesses for an external solver.

3) In figure 4, even if small, you can see a decay on the speed-ups as you have a greater number of nodes (towards the right tail of the curve). Since the main interest in an approach that purely offers speed-ups (rather than from example, additional better generalization or accuracy) would be on large scale simulations, I see this as a fundamental weakness.

4) When comparing to end-to-end approaches, authors cite a few times in the text instability for longer time sequence rollouts. Note that in Sanchez et al. 2020, the authors propose adding noise to the inputs and correcting the predictions as a stabilizing technique, they present stable rollouts for systems with thousands of steps (longer than the ones considered proposed here). This could be integrated in the end-to-end approach tested on the paper. 


All in all, despite its weaknesses, I still think the paper takes a meaningful step towards the efficient use of neural networks for physical simulations. So I am willing to adjust my score after some of my concerns have been addressed. 

Small typos:
In the "Generalization" paragraph from the Evaluation section, should it be Figures 9 and 10, rather than 7 and 10?
review_title: Speed-up on small systems with hundreds of time steps. Lacks stronger evidence for systems with more nodes and longer time horizons. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes a graph network(GN) called COPINGNet (“COnstraint Projection INitial Guess Network”), which learns to compute a good initialization for the traditional PBD method. To simulate a physical system, PBD first computes updated locations of vertices then corrects the estimates of the initial position by constraint projection.  The projection step is computationally expensive, and that is where the proposed COPINGNet is applied to generate a good initial guess for the built-in linear system solver (e.g., CG).

Strengths:  
1. The proposed method is superior to end-to-end approaches in terms of long-term stability and out-of-distribution transfer (initial conditions, discretizations, material parameters, etc.).
2. This paper applies the idea of message passing to design the graph network architecture, which enables propagations of node information (location, speed, etc.) and edge information (Young's module, torsion module, etc.) on the graph.

Weaknesses: 
1.  According to Figure 4 and Figure 6,  the total speedup of the entire simulations is approximately 1.4 compared to vanilla CG solver.  This improvement seems marginal.
2. It seems that the current model cannot generalize to different shapes. If a network needs to be retrained for each different shape, it may significantly limit its applicability in practice.

Questions:
1. Can the initialization guess network accelerate other iterative solvers? 
2. According to Figure 6, helix simulation and bending rod simulation show different speedup rates (when extrapolating in time). Can this be explained? 

Generally speaking, this article provides a new idea. By combining graph neural networks with traditional methods, the speed of solving physical systems is improved. Due to some concerns about the speedup ratio and generalization, the practicability of this method needs further investigation.
review_title: Using GNN to generate good initial guess for an iterative linear system solver to speed up simulations. 

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes an algorithm to accelerate simulations of deformable rods based on position-based dynamics. Despite what the title and abstract suggests, the scope of the paper is limited to a single physical system, and there is no evidence that this approach will work on generic physical systems. The title and abstract should be tuned down and made more specific.

The key idea is novel: instead of replacing the entire time integrator with a neural network, which is typical of previous approaches, the authors propose to use the network to accelerate the constraint project step, and in particular to still rely on the standard projection used in PDB, but using the network to generate the initial guess for the nonlinear optimization. A better initial guess will reduce the number of iterations (an ideal one will cause termination after 1 iteration), thus reducing runtime. Errors in the prediction will still likely be corrected by the non-linear optimization making the approach stable. 

While I think the idea is great on paper, I am concerned by the results and choices made in the paper, in particular:
1. The abstract makes claims of generality, while the approach is very specific to rod simulation with a very specific solver.
2. The approach is based on Daul et al. 2018, it is likely that the authors started from their codebase. However the key contribution of that paper is “However, this solver requires many iterations to converge for complex models and if convergence is not reached, the material becomes too soft. In contrast we use Newton iterations in combination with our direct solver to solve the non-linear equations which significantly improves convergence by solving all constraints of an acyclic structure (tree), simultaneously. Our solver only requires a few Newton iterations to achieve high stiffness and inextensibility”. My understanding is that the authors of this submission are purposely comparing against the CG solver instead of the new faster solver proposed in Daul et al. 2018. Is this correct and if so why? The comparison should be done with the state of the art, which is not CG.
3. I am having a hard time understanding the plot in figure 4. Is this the entire runtime or just the inference time for the projection part? It seems that the speedup is ~20%  which is minor and I believe switching to newton as suggested in Daul et al would likely give you way more. Please report complete runtimes of the entire simulation, and also a plot of the errors. A speedup of 20% only on a specific step of the algorithm is in my opinion not worth training a neural network and thus losing generality. Especially since this is applied to an algorithm that is not the current state of the art.
4. How is CG initialized? Do you use the previous solution?
5. Please add a comparison of your network against a simple nearest neighbor search over the training data, as a simple baseline.
6. Report complete timings of how long it takes to train. How many simulations do you need to run before the training time is amortized by the gained acceleration?
7. Only the simpler examples from Daul et al 2018 are shown in the paper, the rope knots and the teaser model have been omitted. Why? These are the more challenging cases where errors and penetrations would be immediately visible.

While I agree with the author's conclusions that “We discovered that applying GNs for replacing the initial guess has fundamental advantages over end-to-end approaches.” this is by itself not a surprising result. Endtoend approaches are, currently, usually worse than traditional time integrators for integrating physical systems. Instead of replacing the entire system, replacing a smaller piece does less damage than replacing the entire integrator, but it is still likely worse if the comparison is properly done with the state of the art. I don’t think these results are publishable until a clear advantage over the state of the art is shown, or an argument that is not superior performance is made against standard PBD simulators.

A minor comment: “is included corresponding to a parameterization by arc length” I do not understand this sentence.

Update:

The paper has considerably improved:

the title is now accurately describing the paper
complex scenes have been added (and the method works there too)
comparison with knn has been added and it shows a clear improvement for the proposed method

I think the paper has improved, but I still find the comparison with direct solvers problematic. For small scenes like the one shown in this paper, a direct solver is fine, there is no need to use an iterative one. If the scenes are large enough to require an iterative solver (i.e. a direct one runs out of memory) then it should be shown that the proposed methods provide benefits in that specific setting. It could be that my bar for comparison is too high, as I usually publish in a different community where quantitative improvement against the closest baseline is always required.

Overall, I am still mildly positive, but not willing to champion this paper given the many issues raised in the reviews.

review_title: Unclear why the comparison is not done against the state of the art.

Meta-review: The paper proposes speeding up iterative simulations of complex dynamics systems based on connected rods. Traditionally, these systems alternate between forward integration of the dynamics and constraint projections. Instead of replacing this entirely with end-to-end trained ML, here ML is only added a single point in the method to speed up the iterative solver itself, more precisely by providing initial estimates for the constraint projection step. This is done with graph networks.

At initial evaluation, the paper had two slightly favorable reviews (6) and two unfavorable reviews (4) and was therefore on the fence leaning towards rejection.

Reviewers appreciated a well motivated method and in an interesting problem.

However, on the downside, issues raised where lukewarm performance; novelty (a direct application of graph networks); lack of generality of the approach; similarity to graph networks applied on mesh based physics simulations, and similarity to NN applied for speeding up elasticity simulations; application on the finest level only; memorization/lack of generalization; simplicity of baselines; simplicity of tasks (including the added more complex tree task).

There seemed to be some confusion on whether one of the reviewers had read the initial NeurIPS submission only (which he also had reviewed) or also the ICLR submission; the authors seemed to be upset up this possibility and made it clear in their responses, but the AC can assure them that the proper version has been read, reviewed and discussed; the author's responses in that respect were not helpful.

The authors provided responses to most of the raised issues, and several reviewers acknowledged that the paper had been improved, in particular by adding comparisons (e.g NN search).

However, in spite of these improvements, the discussion among reviewers and AC revealed that the paper still has serious issues, in particular minor novelty, lukewarm improvements, and some issues re: comparisons to baselines. While the reviewers acknowledged merits in the idea, the weaknesses hindered them to champion the paper for acceptance at this point, and the AC concurs, recommending rejection.
Decision: Reject