Accurately Solving Rod Dynamics with Graph Learning

Abstract: Iterative solvers are widely used to accurately simulate physical systems. These solvers require initial guesses to generate a sequence of improving approximate solutions. In this contribution, we introduce a novel method to accelerate iterative solvers for rod dynamics with graph networks (GNs) by predicting the initial guesses to reduce the number of iterations. Unlike existing methods that aim to learn physical systems in an end-to-end manner, our approach guarantees long-term stability and therefore leads to more accurate solutions. Furthermore, our method improves the run time performance of traditional iterative solvers for rod dynamics. To explore our method we make use of position-based dynamics (PBD) as a common solver for physical systems and evaluate it by simulating the dynamics of elastic rods. Our approach is able to generalize across different initial conditions, discretizations, and realistic material properties. We demonstrate that it also performs well when taking discontinuous effects into account such as collisions between individual rods. Finally, to illustrate the scalability of our approach, we simulate complex 3D tree models composed of over a thousand individual branch segments swaying in wind fields.

Reviews and author responses:

The paper proposes an algorithm to accelerate simulations of deformable rods based on position-based dynamics. Despite what the title and abstract suggests, the scope of the paper is limited to a single physical system, and there is no evidence that this approach will work on generic physical systems. The title and abstract should be tuned down and made more specific.

The key idea is novel: instead of replacing the entire time integrator with a neural network, which is typical of previous approaches, the authors propose to use the network to accelerate the constraint project step, and in particular to still rely on the standard projection used in PDB, but using the network to generate the initial guess for the nonlinear optimization. A better initial guess will reduce the number of iterations (an ideal one will cause termination after 1 iteration), thus reducing runtime. Errors in the prediction will still likely be corrected by the non-linear optimization making the approach stable. 

While I think the idea is great on paper, I am concerned by the results and choices made in the paper, in particular:
1. The abstract makes claims of generality, while the approach is very specific to rod simulation with a very specific solver.
2. The approach is based on Daul et al. 2018, it is likely that the authors started from their codebase. However the key contribution of that paper is “However, this solver requires many iterations to converge for complex models and if convergence is not reached, the material becomes too soft. In contrast we use Newton iterations in combination with our direct solver to solve the non-linear equations which significantly improves convergence by solving all constraints of an acyclic structure (tree), simultaneously. Our solver only requires a few Newton iterations to achieve high stiffness and inextensibility”. My understanding is that the authors of this submission are purposely comparing against the CG solver instead of the new faster solver proposed in Daul et al. 2018. Is this correct and if so why? The comparison should be done with the state of the art, which is not CG.
3. I am having a hard time understanding the plot in figure 4. Is this the entire runtime or just the inference time for the projection part? It seems that the speedup is ~20%  which is minor and I believe switching to newton as suggested in Daul et al would likely give you way more. Please report complete runtimes of the entire simulation, and also a plot of the errors. A speedup of 20% only on a specific step of the algorithm is in my opinion not worth training a neural network and thus losing generality. Especially since this is applied to an algorithm that is not the current state of the art.
4. How is CG initialized? Do you use the previous solution?
5. Please add a comparison of your network against a simple nearest neighbor search over the training data, as a simple baseline.
6. Report complete timings of how long it takes to train. How many simulations do you need to run before the training time is amortized by the gained acceleration?
7. Only the simpler examples from Daul et al 2018 are shown in the paper, the rope knots and the teaser model have been omitted. Why? These are the more challenging cases where errors and penetrations would be immediately visible.

While I agree with the author's conclusions that “We discovered that applying GNs for replacing the initial guess has fundamental advantages over end-to-end approaches.” this is by itself not a surprising result. Endtoend approaches are, currently, usually worse than traditional time integrators for integrating physical systems. Instead of replacing the entire system, replacing a smaller piece does less damage than replacing the entire integrator, but it is still likely worse if the comparison is properly done with the state of the art. I don’t think these results are publishable until a clear advantage over the state of the art is shown, or an argument that is not superior performance is made against standard PBD simulators.

A minor comment: “is included corresponding to a parameterization by arc length” I do not understand this sentence.

Update:

The paper has considerably improved:

the title is now accurately describing the paper
complex scenes have been added (and the method works there too)
comparison with knn has been added and it shows a clear improvement for the proposed method

I think the paper has improved, but I still find the comparison with direct solvers problematic. For small scenes like the one shown in this paper, a direct solver is fine, there is no need to use an iterative one. If the scenes are large enough to require an iterative solver (i.e. a direct one runs out of memory) then it should be shown that the proposed methods provide benefits in that specific setting. It could be that my bar for comparison is too high, as I usually publish in a different community where quantitative improvement against the closest baseline is always required.

Overall, I am still mildly positive, but not willing to champion this paper given the many issues raised in the reviews.


We updated the supplementary material with a 'changes.pdf' (that was automatically generated) to show the differences between the current revision of our paper and the previous NeurIPS submission. Changes are highlighted in orange, additional content in blue, and removed characters and sentences are indicated by a red dot with an 'x'. 

As indicated by the highlights our revised paper changed significantly compared to the previous version. However, please note that some of the highlights are also just due to changes in the different Latex style packages of NeurIPS and ICLR.

any chance of getting this before the rebuttal period is over?

The memory allocation requirements for our method are similar to other published GN-based methods. To provide a more complete evaluation we can also report the memory allocation costs in addition to the evaluation of time complexity in the next revision.

Thanks, to clarify: by the memory footprint I meant the extra memory required by the GN architecture, not allocation time/runtime. 

Thank you for your additional comments. 

Please not that the reported speedup for our method is the total speedup (including time for memory allocation). We can provide additional information on the memory footprint of our method. 

Please find additional comments to your statements below: 

“There's not much time left for the rebuttal, but I'll do my best to take a look.”: 

A: Thank you for planning to take the time now to read our updated paper. 


“In my annotated pdf from the NeurIPS review nearly every highlighted/noted part remained unchanged in the ICLR submission. Meanwhile nearly all of the text and figures (up to numbering) are the same. Hence, I didn't update my review or go hunting around for small changes. If the important changes addressing the NeurIPS rejection were largely in the appendix (e.g., Figs 10 & 11) then I did not read them initially.”

A: We respect the statement of being misled by a very unfortunate comparison of personally annotated sections between the previous manuscript and the current one. We would appreciate it if a careful review of our paper can now take place.


“This model is perhaps geometrically complex but the motion is simple. Since the branches all move in synchrony there are not many interesting collisions to resolve. To test collision complexity consider a fork pulling out of a bowl of spaghetti. In graphics, these are the kind of tests I'd like to see in 2020.The result in this paper looks very similar to what modal bases can do (e.g., Barbic and James 2007). Under what setting should we consider this state of the art? Certainly not for "CG simulation" in general. Is this statement taken within real-time simulation and specifically including only methods with a similar learning based methodology? ”

A: Please note that the tree dynamics are coupled in real-time with the fluid dynamics of the wind field. This coupling is highly complex and an active topic of research in the CG community [Gissler et al. 2019]. It has been recognized that wind flows in tree crowns exhibit turbulent motion (Role of buoyant flame dynamics in wildfire spread, Finney et al. 2015), which is expected to result in irregular branch swaying. Our method for fluid coupling is based on [Pirk et al 2017] and is capable of simulating turbulence. Please find examples of the complexity of swaying motion in the accompanying video (https://vimeo.com/233082391). Therefore, the statement that all branches sway in synchrony is questionable. Furthermore, it seems to us that this phenomenon is at least as compelling in terms of pushing boundaries in science as computing the rod dynamics required to simulate pulling spaghetti out of a bowl. In summary, we disagree with the assumption that the swaying tree is similar to the complexity of what is shown in [Barbic and James 2007] (which deals with deformable bodies not with fluid-solid coupling) and instead think that it is on par with phenomena investigated by the CG community in 2020. Please also note that our novel experiments indicate that  ML-based methods do not seem able to accurately solve physical systems to the quality of what SOTA methods in CG can do. In this context our method is a clear advancement over SOTA.


“I agree with Rev4 that there should be a comparison to Deul et al. 2018. The rebuttal argues that the point is about improving iterative solvers. That's fair, but there's a question of generalization and practical speed. If Deul et al. is faster and more general in practice (compare their much more complex trees for example), then the applicability and impact of the proposed method is diminished.”

A: Please note that the swaying tree examples in [Deul et al 2018] are not computed in real-time. Instead the discussed solver requires ~10 seconds for the pine, ~8 seconds for the willow and ~6 seconds for the birch tree. We designed our method with real-time simulations in mind. Therefore, this comparison not only seems unnecessary (please see our response to R4) but also non-trivial to perform. 


"Old Figure 5 is now Figure 4 (unchanged) My comment remains the same." 
"Old Figure 4 is now Figure 11 (unchanged) My comment remains the same."

A: We have addressed these comments in our previous rebuttal to NeurIPS. Please respond to these statements if you want to continue this discussion instead of relying on copy-pasted text. 


Summary: the authors present a Graph Network (GN) architecture to speed up the running time of rod physical simulations by predicting the initial guesses to reduce the number of iterations of an iterative position-based solver. The proposed approach offers speed-ups varying from 6-50%, depending on the problem analyzed. 

Strengths: 
The paper is well written and motivated, methods are very clear. There are substantial experiments to test for the paper's main claims on inference efficiency and out of distribution generalization, although mostly on small scale systems (up to 200 nodes) and short time horizons (100 steps).

Weaknesses: 

1) The biggest weakness to me is shown on Figure 6, when plotting the CG interaction ratio as a function of time, for the straight bending rod and the elastic helix, up to 1000 timesteps. Especially on the straight bending rod simulation (orange curve), the speed-up is not consistent across time. Further insight on why those oscillations are happening would be appreciated. In the elastic helix simulation (green), one also sees a huge drop right after the training regime range (50-100 steps). End-to-end approaches will generally offer more significant speed-ups (orders of magnitudes rather than the 6-50% provided by COPINGNet), but they might struggle to remain stable for longer time horizons. Therefore what one would like to see from an alternative approach like the proposed here, is a system that scales well with time. Figure 6 shows that's not the case here yet. 

2) It would have been interesting to see how the GN compares to a different neural network predicting the initial guesses (say a simple LSTM or -- more ambitiously -- a transformer). I understand that many of the successful neural approaches for physics simulations are GN based, so the choice is well motivated -- but those systems are generally trained end-to-end, which isn't the case in the work from this paper. Therefore, it's not clear to me that other non GN approaches wouldn't perform well on this task of producing better initial guesses for an external solver.

3) In figure 4, even if small, you can see a decay on the speed-ups as you have a greater number of nodes (towards the right tail of the curve). Since the main interest in an approach that purely offers speed-ups (rather than from example, additional better generalization or accuracy) would be on large scale simulations, I see this as a fundamental weakness.

4) When comparing to end-to-end approaches, authors cite a few times in the text instability for longer time sequence rollouts. Note that in Sanchez et al. 2020, the authors propose adding noise to the inputs and correcting the predictions as a stabilizing technique, they present stable rollouts for systems with thousands of steps (longer than the ones considered proposed here). This could be integrated in the end-to-end approach tested on the paper. 


All in all, despite its weaknesses, I still think the paper takes a meaningful step towards the efficient use of neural networks for physical simulations. So I am willing to adjust my score after some of my concerns have been addressed. 

Small typos:
In the "Generalization" paragraph from the Evaluation section, should it be Figures 9 and 10, rather than 7 and 10?

Thank you for your reply and for addressing some of my initial concerns. 

I acknowledge my initial comment about the number of nodes was not accurate, thanks for the correction. 

The revised Figure 6 mostly addressed my concerns about oscillations (point 1 on the original review). At least now, on average, you can see there is always a speedup, even if its magnitude varies -- while from the previous plot you couldn't conclude this. On the other hand, I still think that the fact that the speeds diminish overtime (and as the number of node increases, as pointed out in 3)) is a weakness of method -- especially at the speedup range the method provides (6-50%). `I note that all reviewers raised concerns about the speedups observed, and while I see why one might say that 'any amount of speedup while maintaining accuracy is an advancement', there are other factors such as memory footprint which haven't been taken into account. 

I appreciate other changes to the revised manuscript such as including a knn baseline as suggested by R4, new examples and toning down generalization claims. Overall the paper has improved after the rebuttal, and I changed my scores from 5 to 6 -- although I am not willing to fight for the paper to be accepted.

Finally, I found that some of the statements in the replies to the reviewers could also be toned down such as 'This is a strong contribution that should be exposed to the community' and stating a that a reviewer didn't read the paper when they say they did. 
 

Is it possible to post a version of this paper with changes highlighted since the NeurIPS submission?

The rebuttal is arguing that significant changes have been made, but upon second inspection I still can't identify them. There's not much time left for the rebuttal, but I'll do my best to take a look.

In my annotated pdf from the NeurIPS review nearly every highlighted/noted part remained unchanged in the ICLR submission. Meanwhile nearly all of the text and figures (up to numbering) are the same. Hence, I didn't update my review or go hunting around for small changes. If the important changes addressing the NeurIPS rejection were largely in the appendix (e.g., Figs 10 & 11) then I did not read them initially. I have looked at them now.

While waiting for highlighted changes,

Figs. 10 & 11
Thank you for including these. 

The tree.
This model is perhaps geometrically complex but the motion is simple. Since the branches all move in synchrony there are not many interesting collisions to resolve. Are the leaves included in this simulation or just added post facto? This would be impressive with a strong wind rustling all the leaves together. To test collision complexity consider a fork pulling out of a bowl of spaghetti. In graphics, these are the kind of tests I'd like to see in 2020. The result in this paper looks very similar to what modal bases can do (e.g., Barbic and James 2007). Under what setting should we consider this state of the art? Certainly not for "CG simulation" in general. Is this statement taken within real-time simulation and specifically including only methods with a similar learning based methodology?

Clarity
What changed? Section 3.1 looks the same. Highlighted version would help here.

Old Figure 5 is now Figure 4 (unchanged)
My comment remains the same.

Old Figure 4 is now Figure 11 (unchanged)
My comment remains the same.

Reading the other reviews and rebuttals, I agree with Rev4 that there should be a comparison to Deul et al. 2018. The rebuttal argues that the point is about improving iterative solvers. That's fair, but there's a question of generalization and practical speed. If Deul et al. is faster and more general in practice (compare their much more complex trees for example), then the applicability and impact of the proposed method is diminished.

Thank you for your insightful comments. Please find our response to your statements and questions below:

To 1): In Figures 4 and 6 we have shown that our method can extrapolate beyond the training data range. Whereas, in Figures 9 and 10 we show that end-to-end approaches fail to generalize beyond the training data range. These results provide evidence that our method performs better than existing end-to-end strategies in general scenarios. We agree that we have not conclusively shown that our method improves all iterative solvers. However, we think that our method has the potential to stimulate further research towards more generalizable GNs for physical systems. While it would strengthen our claims toward generalization to apply our method to other solvers (e.g. FEM) or other physical systems, we think that this is outside the scope of a single paper. Therefore, after carefully considering your feedback toward our claims of generalization in the abstract (and title) we decided to tone-down these claims. 

To 2): We introduce a novel approach for learning physical systems based on an iterative solver and not a direct solver like the one introduced in [Deul et al. 2018]. We focused on iterative solvers because they offer advantages (e.g. parallel computing, better suited to complex scenarios, more general) compared to direct solvers. For an iterative solver, we have shown that our method outperforms previous GN-based methods in terms of accuracy and generalizability in a specific physical scenario. We did not aim at advancing the state-of-the-art of simulating rod dynamics but instead we wanted to prove that iterative solvers can be optimized with GNs without sacrificing accuracy. Therefore, we think comparing to direct solvers is not necessary. 

To 3): The reported speedup in Figure 6 is NOT of a single step of the numerical time integration, but instead a total speedup of the simulation for the respective physical system. For the helix case we report a persistent total speedup of around 50%, whereas for the rod case it is initially over 50% and later stabilizes beyond the training data range at over 20%. 

To 4): Yes, we use the previous solution corresponding to an initialization of the Delta with zero (see “canonical initialization with zeros” in Section 3).

To 5): We added a baseline against a knn now shown in the updated Figures 6 (left, right). This new result further highlights the technical contribution of our GN architecture. 

To 6): We trained our network for 5 hours for the bending rod scenario and 6 hours for the helix case. 

To 7): We have added the rope knot example to the supplementary material as further evidence that our method can handle complex physical systems (with penetrations). Please note that similar to [Deul et al. 2018] we show 3D models of trees swaying in wind fields that also serve as complex examples.

To “this is by itself not a surprising result”): Our method of integrating GNs to iterative solvers has never been explored before. Our quantitative evaluation indicates non-trivial findings w.r.t. performance improvements. Specifically, we observed gains beyond the training data range for rod and helix simulations, as well as for the generalizability of complex tree dynamics and collision scenarios.


Thank you for your insightful comments. Please find answers to your statements and questions below:

To Weakness 1: 

While we agree that the improvement appears marginal compared to end-to-end approaches, our analysis has shown that end-to-end methods lack required accuracy. In contrast, our method provides solutions to physical systems with sufficient accuracy. Therefore, just comparing end-to-end methods and our approach only based on the total speedup is not meaningful. 

To Weakness 2: 

It seems this is a misunderstanding. Generally, it is acknowledged that end-to-end approaches necessitate retraining for individual shapes. In contrast, in our method we have observed a carry-over across different 3D tree shapes (variations of geometry and topology).  We can provide the failure case for the end-to-end approach as additional result for the final version of the paper if requested. While the carry-over for our method could be enhanced, it is still more generalizable compared to previous work. Please see Section 4 “Generalization”, “Complex Scenarios”, and “Collision” (and, e.g., Figure 5) for support of this claim. If anything, we think in terms of generalizability this is an advancement of the current state-of-the-art rather than a weakness.

To Question 1:  

Yes, we are convinced that future work will show similar improvements to a whole variety of numerical solvers, e.g., finite element solvers or linear complementary problems. However, in this paper we focused on the evaluation using position-based dynamics. We toned-down claims of generalizability in title and abstract.

To Question 2: 

The different speedup rates can be explained by the quality of the predictions provided by the GN. This quality of the initial guesses decreases for shapes which are out of the training distribution. Therefore, in cases where the system diverges from the training distribution, more iterations are required to converge. An oscillating helix is a more constrained scenario compared to the dynamics of swinging rods. To further demonstrate general applicability and speedup ratios, Figure 9 (Supplemental Material) shows a new experiment of a tightening knot as an additional example.


Thank you for your insightful comments. We have conducted experiments on complex systems of over 1000 colliding nodes and not -- as stated in the review -- on only 200 nodes. Please see Figure 5 (botanical tree) and Section 4 (complex scenario) for more details.

To 1): Figure 6 was showing a single arbitrary simulation run using our GN method. We have replaced Figure 6 with two new Graphs (Figure 6 left, right). Figure 6 (left) now shows the average of 20 runs of the same simulation for the bending rod, while Figure 6 (right) shows the average of the elastic helix case. As indicated by the aggregation of these simulation runs oscillations are not periodic. Furthermore, we ran additional experiments where we constrained the dynamical systems in various ways (e.g. removing degrees of freedom for the rod motion), which did not eradicate oscillations. Therefore, after further investigating our experiments we think that the oscillations are caused by context switches of the (single) GPU running the PBD simulation and the GN inference simultaneously. This means that the oscillations do not characterize the performance of our method.

Our goal was to propose a novel method for adapting GNs for physical systems without sacrificing accuracy. Therefore our method is an orthogonal approach to end-to-end methods. Our results indicate that our method significantly outperforms vanilla physical solvers persistently beyond the training data range. This is a strong contribution that should be exposed to the community as it is expected that similar results can be obtained for other physical solvers and physical systems.

To 2) and 4): We have included a comparison to knn as a baseline in the rod and helix case in Figure 6 and in the supplementary material (Figures 10 and 11). This new result further demonstrates the technical contribution of our GN architecture. For the final version we can provide additional baselines based on an LSTM and/or another GNs using the regularization techniques mentioned in [Sanchez et al. 2020].

To 3): We disagree with the notion that increasing the number of nodes with decreasing overall speedup is a fundamental flaw of our method. We have shown in Section 4 (Complex Scenarios), that even in the case of over 1000 colliding nodes noticeable speedups can still be obtained. The goal of our method is to provide a speedup to direct solvers (also for complex scenarios) without losing accuracy -- any amount of speedup while maintaining accuracy is an advancement.


It is apparent that this review is not based on the revised version of our paper. 

We significantly changed the exposition of the paper and added additional results, where our goal was to carefully address the previous reviewer’s comments from NeurIPS. This review appears to be a direct copy of the response to the previous version of our paper. Therefore, the large majority of arguments brought up in this review cannot be considered meaningful feedback any longer. Given the character limit, we cannot exhaustively answer all statements brought up in this review. The examples below demonstrate the disparity between the provided statements in the review and the actual sections and figures in the document.

“For example, the paper mentions many times that it is not replacing the time integration/constraint projection with an end-to-end trained network for robustness reasons. Let's see it fail then!”
We added Figure 10 and 11 in the supplementary material to exactly address this point. Figure 10 illustrates the temporal evolution of our method with the end-to-end approach which fails to correctly solve with an increasing number of time steps. Figure 11 provides quantitative assessment of the extent of the error accumulation.

“This modest speedup coupled with the small number of simple experimental scenarios worries me that these results will not generalize to a statistically significant speed up in general.”
To address this statement, we have included a complex dynamical system of a botanical tree model swaying in a wind field. This example is based on over 1000 colliding nodes simulated at interactive rates, which can be considered a state-of-the-art CG simulation. The review again appears to not consider this additional result.

“The graph network description (which gets at the core contribution of this paper) is poorly described. I read this section many times and via cross referencing with [13] could finally understand with low confidence what is being done.”
This is a direct copy of the previous review. We have significantly reworked the text based on the previous comments and also received positive feedback on the clarity of the text (please see other reviews).

“I do not understand Figure 5. What is the purple curve? the vanilla CG solver? Then wouldn't its speed up be 1x?” 
Figure 5 shows our newly added experiment on simulating botanical tree models as complex systems and not as claimed a purple curve. This statement appears erratic.

“Figure 4 is very confusing. Is this figure showing that none of the hyper parameters matter except for MLP width? This is surprising to the point of indicating a bug/overfitting.”
Figure 4 does not mention hyper parameters. Therefore, we cannot meaningfully respond to this statement. 

Finally, we think that it is potentially rude to accept reviewing a paper and not reading it.


We appreciate that the reviewers found that “this article provides a new idea” (R1), that our method is “superior to end-to-end approaches in terms of long-term stability” (R1), that the paper is “well written and motivated” (R2), that “there are substantial experiments” (R2), and that “the key idea is novel” (R4). Unfortunately, while providing a very verbose review it appears that R3 has not read our manuscript.

Despite the overall positive feedback, the reviews also highlight a number of weaknesses w.r.t. the evaluation of our method. While we carefully evaluated our method, reviewers asked questions regarding the features of reported graphs (oscillations), the runtime performance of our method, and the comparison to baselines (direct solvers, LSTM, KNN). We included additional results and improved the exposition to address these concerns in the submitted revision.

Furthermore, we would like to highlight that these shortcomings do not devalue the essential contribution of our method. Currently, research in this field focuses on either developing numerical solvers or end-to-end learning based methods. In contrast, we have introduced a novel approach for integrating a GN into an iterative solver that shows significant performance improvements compared to standard non-learning based solvers (speed) as well as end-to-end learning based methods (accuracy). This is a significant finding that has the potential to stimulate further research in the direction of using neural networks for physical simulations based on iterative solvers -- it should be read by a greater audience.

The revised version of our manuscript shows all changes highlighted in blue and also includes the supplementary material to facilitate the review of additional results. 


This paper proposes a graph network(GN) called COPINGNet (“COnstraint Projection INitial Guess Network”), which learns to compute a good initialization for the traditional PBD method. To simulate a physical system, PBD first computes updated locations of vertices then corrects the estimates of the initial position by constraint projection.  The projection step is computationally expensive, and that is where the proposed COPINGNet is applied to generate a good initial guess for the built-in linear system solver (e.g., CG).

Strengths:  
1. The proposed method is superior to end-to-end approaches in terms of long-term stability and out-of-distribution transfer (initial conditions, discretizations, material parameters, etc.).
2. This paper applies the idea of message passing to design the graph network architecture, which enables propagations of node information (location, speed, etc.) and edge information (Young's module, torsion module, etc.) on the graph.

Weaknesses: 
1.  According to Figure 4 and Figure 6,  the total speedup of the entire simulations is approximately 1.4 compared to vanilla CG solver.  This improvement seems marginal.
2. It seems that the current model cannot generalize to different shapes. If a network needs to be retrained for each different shape, it may significantly limit its applicability in practice.

Questions:
1. Can the initialization guess network accelerate other iterative solvers? 
2. According to Figure 6, helix simulation and bending rod simulation show different speedup rates (when extrapolating in time). Can this be explained? 

Generally speaking, this article provides a new idea. By combining graph neural networks with traditional methods, the speed of solving physical systems is improved. Due to some concerns about the speedup ratio and generalization, the practicability of this method needs further investigation.


Using the same bar as NeurIPS, I continue to recommend rejecting this paper.
Since the paper remained largely the same. My review remains largely the same.
It is also doubtful that these changes have satisfyingly address the other three
reviews.

This paper proposes an iterative PBD solver which uses a neural network to guess
the constraint force and position updates before polishing with a conjugate
gradient solver. The method utilizes a graph network architecture which makes it
agnostic to the particular discretization allowing generalization (in theory)
across scenarios.

Speeding up simulations is an evergreen topic. A strength of this work is the
problem that it is tackling. The choice of methodology makes it mildly of
interest to the ICLR community. It is an application (without adaptation) of
graph networks. The result is mildly positive (though not earth shattering),
indicating success of graph networks to some degree. I have not seen graph
networks applied to rod simulation or elasticity simulation, although they have
been applied to meshes in geometry processing (similar and more challenging
scenario) and neural networks have been applied more generally to speeding up
elasticity simulations (with what would be a trivial extension to rods). A
possibly unique strength of this paper is the combination of rod simulation and
a convergent solver (i.e., just using the network as an initial guess in an
iterative solve). However, this is a really straight forward idea and the gains
are again small. So, while this is a strength it is minor and possibly not
exciting to the broader ML/ICLR community.

My main criticism of this method have to do with three aspects of scaling: 1)
the network is applied at the inner most loop and message passing occurs by
iterating over edges (which I believe are based on the original connections).
This means that global information passing requires M = O(n) iterations. 2) the
method operates only on the fine scale details (the input resolution). This runs
counter to multigrid literature which would suggest operating on all the
residual at all frequency levels. 3) the method shows a small number of small
examples (low resolution rods with one or two rods in a scene). PBD is often
employed in scenarios with millions of rod segments from thousands of rods
(e.g., hair on a human head will have 100,000 rods each with hundreds/thousands
of segments). For a 1.5x to become impressive I would like to see this operating
at scale.

I worry that the network has simply memorized a mapping from global positions to
guesses. Since the network has access to the raw positions, I would be surprised
if it has learned rotation and translation invariance. Lack of this would be a
good indication that it's learning a spatial mapping. This would be a severe
limitation.

The exposition of main results is very confusing. If I understand this part of
Figure 5 (and I doubt I do), then the pink curve below the red and black curves
is indicating that using this method does not simply speed up the CG solve but
some how negatively affects the total runtime (I guess by confusing the outer
loop down an incorrect path). So the gains by taking an aggressive initial guess
are tempered a bit, though still resulting in a (quite modest) overall speed up
<1.5x.

This modest speedup coupled with the small number of simple experimental
scenarios worries me that these results will not generalize to a statistically
significant speed up in general. 

Collisions appear as an afterthought. If the speedups were more significant, I
would happily excuse this as collisions can often be an extra systems effort.
However, part of the "glory" of a PBD solver rather than an FEM+LCP type
simulation is that one can throw all sorts of constraints into the same system.
So, when this paper tacks on collisions outside the learning aspect of this work
it calls the choice of PBD into question as perhaps needlessly inaccurate or an
"purely-out-of-convenience" type choice rather than a scientifically motivated
one.

If I understand correctly, the paper always compares to a baseline of resetting
the position and multiplier updates to zero. Is this the best baseline? What
about using the previous iteration? Or other momentum based strategies? 

The for-loop on line 8 of the pseudocode is misleading/confusing/incorrect. This
implies that updates for rods are conducted independently. But then line 9
appears to accept as input and send as output coordinates and lagrange
multipliers for the entire system (rather than the rod i). What is the role of
the for loop of line 8 if line 9 does not depend on which rod is being
considered? it would be better to write out the linear solve that is being
conducted (e.g., with CG). The figure and text below confuses me further. Is the
correction guess applied: a) once per time step, b) once per constraint
lineariztaion (just before CG), or c) once per rod?

The revised paper (and perhaps rebuttal) should include a clarified pseudocode
to understand what this method is actually proposing.

The effectiveness of the network depends on a parameter M which would appear to
scale poorly with the resolution of the input shapes. Does M need to be adjusted
for higher resolution examples?


What is the video showing? Training data created using the groundtruth
simulation? Results of this method at test time? If so what was the training
data used for each? This video did not really help supplement this submission. I
wish that the video had included experiments that could be used to gain
intuition about what's being learned and about the accuracy of the initial
guess.

For example, the paper mentions many times that it is *not* replacing the time
integration/constraint projection with an end-to-end trained network for
robustness reasons. Let's see it fail then!

In a future revision I would like to understand
  - under what circumstances does end-to-end learning fail, but this method
    succeeds.
  - under what circumstances does simply using the previous frame's constraints
    as an initial guess out perform this method?
  
It'd be great to see a video where we also see an evolving plot per-frame (like
Figure 6) showing the performance gains of applying this method rather than
naive initialization methods. This would be great to get match up whether the
gains happen far from the rest state, in collision heavy scenarios, or the
opposite etc.

On line 180, I did not understand how the set of input edges to the graph
network is defined. If inputs nodes are assigned to each rod segment, then input
edges should connect two segments. Are only neighboring segments connected with
these edges? (e.g., the dual graph of the polyline representing the central axis
of the rod). Or are all possible pairs of segments generated? Where are Young's
and Torsion moduli stored? It's natural to store Young's modulus at segments but
this would correspond to nodes of the graph not edges.  

The graph network description (which gets at the core contribution of this
paper) is poorly described. I read this section many times and via cross
referencing with [13] could finally understand with low confidence what is being
done.

Figure 4 is very confusing. Is this figure showing that none of the hyper
parameters matter except for MLP width? This is surprising to the point of
indicating a bug/overfitting.

I do not understand Figure 5. What is the purple curve? the vanilla CG solver?
Then wouldn't its speed up be 1x? Are these plots showing two different y-axis
or two different examples (straight vs helix)? The caption is very confusing.

The paper does not accurately categorize past works when it writes "Existing
methods enable learning these systems often in an end-to-end manner and with a
focus on replacing the entire integration procedure." Many fluids papers retain
pressure projection to ensure divergence free-ness and within elasticity
simulation, 

Latent-space Dynamics for Reduced Deformable Simulation
Lawson Fulton, Vismay Modi, David Duvenaud, David I.W. Levin, Alec Jacobson

does not replace the time integration procedure.

In the future, I would appreciate a pdf in supplemental material with
highlighted changes. It is potentially rude to continuous reviewers to have to
hunt for small changes (and then see that their reviews were largely ignored).


Meta-review: The paper proposes speeding up iterative simulations of complex dynamics systems based on connected rods. Traditionally, these systems alternate between forward integration of the dynamics and constraint projections. Instead of replacing this entirely with end-to-end trained ML, here ML is only added a single point in the method to speed up the iterative solver itself, more precisely by providing initial estimates for the constraint projection step. This is done with graph networks.

At initial evaluation, the paper had two slightly favorable reviews (6) and two unfavorable reviews (4) and was therefore on the fence leaning towards rejection.

Reviewers appreciated a well motivated method and in an interesting problem.

However, on the downside, issues raised where lukewarm performance; novelty (a direct application of graph networks); lack of generality of the approach; similarity to graph networks applied on mesh based physics simulations, and similarity to NN applied for speeding up elasticity simulations; application on the finest level only; memorization/lack of generalization; simplicity of baselines; simplicity of tasks (including the added more complex tree task).

There seemed to be some confusion on whether one of the reviewers had read the initial NeurIPS submission only (which he also had reviewed) or also the ICLR submission; the authors seemed to be upset up this possibility and made it clear in their responses, but the AC can assure them that the proper version has been read, reviewed and discussed; the author's responses in that respect were not helpful.

The authors provided responses to most of the raised issues, and several reviewers acknowledged that the paper had been improved, in particular by adding comparisons (e.g NN search).

However, in spite of these improvements, the discussion among reviewers and AC revealed that the paper still has serious issues, in particular minor novelty, lukewarm improvements, and some issues re: comparisons to baselines. While the reviewers acknowledged merits in the idea, the weaknesses hindered them to champion the paper for acceptance at this point, and the AC concurs, recommending rejection.

Decision: withdrawn-rejected-submissions