Inverse Design for Fluid-Structure Interactions using Graph Network Simulators
Authors: Kelsey R Allen
Tatiana Lopez-Guavara
Kim Stachenfeld
Alvaro Sanchez-Gonzalez
Peter Battaglia
Jessica B Hamrick
Tobias Pfaff
Venue: NeurIPS 2022 Accept (2022)
Keywords: design
graph neural networks

Abstract: Designing physical artifacts that serve a purpose---such as tools and other functional structures---is central to engineering as well as everyday human behavior. Though automating design using machine learning has tremendous promise, existing methods are often limited by the task-dependent distributions they were exposed to during training. Here we showcase a task-agnostic approach to inverse design, by combining general-purpose graph network simulators with gradient-based design optimization. This constitutes a simple, fast, and reusable approach that solves high-dimensional problems with complex physical dynamics, including designing surfaces and tools to manipulate fluid flows and optimizing the shape of an airfoil to minimize drag. This framework produces high-quality designs by propagating gradients through trajectories of hundreds of steps, even when using models that were pre-trained for single-step predictions on data substantially different from the design tasks. In our fluid manipulation tasks, the resulting designs outperformed those found by sampling-based optimization techniques. In airfoil design, they matched the quality of those obtained with a specialized solver. Our results suggest that despite some remaining challenges, machine learning-based simulators are maturing to the point where they can support general-purpose design optimization across a variety of fluid-structure interaction domains.

Inverse Design for Fluid-Structure Interactions using Graph Network Simulators
Kelsey R. Allen DeepMind, UK
Tatiana Lopez Guevara DeepMind, UK
Kimberly Stachenfeld DeepMind, UK
Alvaro Sanzhez-Gonzalez
Peter Battaglia DeepMind, UK
Jessica Hamrick DeepMind, UK
Tobias Pfaff DeepMind, UK
DeepMind, UK
(krallen, paallan..eplllaaa.ssttheeeeee patpoeggeeeebbattaaaaaaaaaaaee tetpriikkttpaafffeeeeemmm..cc com
Abstract
Designing physical artifacts that serve a purpose-suce as tools and other func- tional structures-is central engineering well everyday human behavior. Though automating design using machine learning has tremendous promise, exist- ing methods are often limited by the task-dependent distributions they were exposed to during training. Here we showcase task task-agnostic approach to inverse design, by combining general-purpose graph network simulators with gradient-base based design optimization. This constitutes simple, fast, and reusable approach that solves high-dimensional problems with complex physical dynamics, including designing surfaces and tools to manipulate fluid flows and optimizing the shape of an airfoil to minimize drag. This framework produces high-quality designs by propagating gradients through trajectories of hundreds of steps, even when using models that were pre-trained for single-step predictions on data substantially different from the design tasks. In our fluid manipulation tasks, the resulting designs outperformed those found by sampling-based optimization techniques. airfoil design, they matched the quality of those obtained with specialized solver. Our results suggest that despite some remaining challenges, machine learning-bast based simulators are maturing to the point where they support general-purpont design optimization across a variety of fluid-structure interaction domains.
Introduction
Automatically designing objects to exhibit desired property-ofter referred to as inverse design has the potential to transform science and engincering, including aerodynamics (27), material design [1], optics (11. and robotics B5 75 Despite its promise, widespread practice inverse design has been limited by the availability of fast, general-purpose forward models. Traditionally, many methods rely on specialized "classical" solvers, which are handcrafted to simulate a particular physical process. However, these models can sometimes be slow, may provide gradients, in certain domains, might even be unknown 23
To mitigate these issues, machine learning is playing an increasingly important role by substituting all or part the inverse design pipeline. "End-to-end" methods can bypass expensive simulation and data collection by learning mapping directly from design parameters (such as the shape of an airplane wing) to task objective (such as the observed drag) [59] Learned generative
These authors contributed equally and are listed in alphabetical order.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

models or policies can propose candidate design parameters for given task [52] 19 24 12 47
These types of methods can be used solve narrowly defined tasks more efficiently than traditional methods, but come with significant limitations: they require access to a dataset of example designs and tasks for training, and often cannot generalize to tasks or designs outside this distribution [71 77).
In this work, we focus on a task-agnostic approach to inverse design, by employing learned simulator models pre-trained on dynamics with no access to the task objective or design space during training, and that, once trained, can be used to solve many new design tasks. This approach, however, requires models with sufficient long-term stability and strong generalization, which commonly applied black- box model architectures such as CNNs often lack 45 69 72 4).
A class of learned physics simulators based on graph neural networks (GNNs) [57 63 could be promising candidate for supporting inverse design, as they have shown success in general-purpose physical prediction, exhibiting high accuracy and generalization for broad range of physics tasks. However, these models have so far mostly been tested for forward prediction alone. To be useful in a design setting, they must also be smooth and robust, as optimizers will exploit unreliable regions in optimization techniques, simulation
the state space [53]. Moreover, to use scalable gradient-basee models must additionally exhibit gradient stability over extremely large state and action spaces, and over hundreds of time-steps. It is unclear if graph network simulators exhibit these properties.
In this paper, we investigate whether learned GNN simulators can support effective inverse design. We pretrain a GNN simulator on a insk-independent dynamics dataset, and at test time perform gradient-based optimization for zero-shot design across a variety of tasks. We perform large set of experiments and find that (1) the learned simulator model is accurate and flexible enough to support a wide variety of design tasks; (2) it generalizes well enough to support design tasks far outside the training regime; (3) gradients from the learned model turn out to be very useful for inverse design, with gradient-based optimization strongly outperforming variety of sampling-base and surprisingly, gradients remain stable even when rolling the model out for hundreds of steps. In experiments on high-dimensional fluid manipulation (2D FLUID TOOLS and 3D WATERCOURSE) we demonstrate that gradient- traseed optimization on learned models can find high-qualing designs over hundreds of time steps. through states with thousands of particles, in tasks with up to 625 design parameters. The same model matches performance of specialized solver on aerodynamics design (AIRFoII.) with a much simpler setup, highlighting the flexibility and precision of this approach.
optimizers: (4)
2 Related Work
Modern design techniques rely on physical simulators so that designs can be evaluated virtually before being tested physically. Solving inverse design problems with simulators has been studied in a variety of disciplines, including nanophotonics [54]. material science (25]. mechanical design
(22). and acrodynamics [2] 60). Many of these rely on black-box optimization, including active and adaptive sampling methods 668] 3 In recent years, differentiable simulators (29) 44 64] have
garnered attention, as they allow for more sample-efficient gradient-basee optimization. However, like classical solvers, they are typically narrow in application scope, as many simulation techniques are hard to express as differentiable program (e.g. constraint dynamics and multiphysics coupling). Our work aims to lessen this burden by learning these simulators from data.
Machine lcarning for inverse design replaces or augments the simulation model and/or the op- timization procedure with learned modules [e.g. 16
28
37
42
59
66 78)
Generative approaches enable efficient optimization by learning to sample more likely designs (thus calling an expensive simulator fewer times). End-to-end predictive approaches learn surrogate model that maps designs to task rewards directly (replacing the simulator entirely). This permits not only sampling-basee optimization, but also cradient-based techniques when the surrogate model is differentiable [59]. However, surrogate models often generalize poorly to designs that lie outside of their training dataset, so many ML approaches for design must keep the optimization procedure close to the training dataset [2 30), or introduce alternative objectives for the surrogate model to increase robustness 71 77 In all cases, these methods require access to datasets with (design, task reward) pairs on the specific task, making zero-shot generalization to new tasks impossible. Instead, we propose to learn the underlying physical dynamics independent of the design task, allowing both the design space and task to be specified at design time.

---

(a) Simulating and evaluating design
6%
G
G
fo
fu
fa
fa Reward function
Design
Simulator
P
Example optimization
Siulaaiin
Simulation
Simulation
Optimizetion steps
Figure 1: Optimizing physical design. Here, the goal is direct stream of water (shown in blue) into two "pools" (shown in purple) by designing a "landscape" (shown in green) parameterized as a 2D height field. (a) The simulation pipeline takes in design ø and initial conditions and uses the design function fD to produce an initial state. The simulation is rolled out with a pre-trained learned simulator fM for K steps, at which point the final state is passed (along with reward parameters OR) to the reward function fR. which computes the quality of the design.
(b) Each step of optimization involves rolling out the simulation and then adjusting the design (ø) accordingly using an optimizer such as gradient descent or CEM. Shown are selected frames from gradient- based optimization in the 2 Pools task of the 3D WATERCOURSE domain.
In this way, our approach is related to methods in model-based reinforcement learning (MBRL), which learns action-conditione transition models to support flexible downstream planning and control. However, learned simulators are rarely accurate, stable, and generalizable enough to support better downstream task performance [36 53 Techniques from MBRL need to mitigate model error either by learning a global, cask-conditioned policy during model learning cannot be adapted to novel tasks on-the-fly) or by re-planning at every timestep to avoid error accumulation [5]]. Because of such sensitivities, these methods have only been demonstrated in relatively small-scale problems, for example with (up to 21 degrees of freedom DOF [10 62 73
451 (which then
or more complex domains with small action spaces (6 DOF (1),, unlike the thousands of state dimensions and hundreds of action dimensions and timesteps we consider here.
We focus on graph neural networks as our simulator model, as they have recently demonstrated high accuracy, stability, domain-generality and generalization performance in the forwards direction for physics domains 57 63). However,
not known whether these simulators can now support gradient- based, high- sigh--imensional inverse design which demands high accuracy, well-behaved gradients, long-tew peggtterm rollout stability, and generalization beyond the training data. Our experimental investigations show that they can, and that particular architectural choices are critical to this success.
3 Problem Formulation
Consider the design task depicted in Figure I| in which the goal direct stream water (shown in blue) into two "pools" (shown in purple) by designing "landscape" (shown in green) parameterized as a height field. Here, an ideal design will create ridges and valleys that direct fluid into the two targets. In the next sections, we formalize what it means to find and evaluate such a design and discuss our choices for simulator and optimizer.
Learned simulators To demonstrate the utility of learned simulators for finding physical designs, we rely on the recently developed MESHGRAPHNETi model [57] which is an extension of the Graph Network Simulator (GNS) model for particle simulation 63 MESHGRAPHNETSS is type of message-passie
graph neural network (GNN) that performs both edge and node updates [2] and which was designed specifically for physics simulation by enforcing locality and translation equivariance. We briefly summarize the learned simulator here, and refer interested readers to original papers for details.

---

We consider simulations over physical states represented as graphs G E G. The state = (V, E) has nodes V connected by edges E. where each node u V is associated with a position u, and additional dynamical quantities q.. These graphs may be either meshes (as in MESHGR APHNETS or particle systems (as in GNS). to vertices and edges in the mesh, respectively. In a particle system (such as 2D FL.ID TOOLs), each node corresponds to a particle and edges are computed dynamically based on proximity. Under this framework, we can also consider hybrid mesh-particle systems (such as 3D WATERCOURSE), The model is constrained to perform local computation, by performing message passing over these edges, and to perform equivariant computations by encoding position the relative displacement and distance between neighboring particles. Intuitively. this allows to generalize well to scenarios where the local interactions are the same but the global arrangement of particles changes. Sce|Appendis B for model implementation details, and Appendix for domain-specific details.
mesh-based system (such as AIRFOII.), V and F. correspond
The simulation dynamics are given by a "ggoound-ruthh simulator fs 9 - 9 which maps the state at time to that at time + At. The simulator fs can be applied iteratively over K time steps to yield "rollout," which we denote (G"o ,.... G**K), Using MESHGRAPHNETTS
trajectory of states, we learn an approximation far of the ground-truth simulator fs. The learned simulator fM can be similarly applied to produce rollouts (G Gh ,.., G*x). where 6oD = G represents initial conditions given as input. We note that a learned simulator allows us to take larger time-steps than Is permitting shorter rollout lengths: in our running example, one model step corresponds to 200 steps of the classical simulator. See Figure 1l for an illustration of simulation using learned model. To improve stability for longer rollouts, random Gaussian noise was added to the inputs during training.
To optimize a physical design, we leverage the pipeline shown in Figure (1) transform design parameters into an initial scene, (2) simulate the scene using far or fs, (3) evaluate how well the simulation achieves the desired behavior, and (4) adjust the design parameters accordingly.
Design parameters To produce the initial state Goo, we introduce a differentiable design function fo & A - G which maps design parameters ø & and other initial conditions a e A to an initial state: G fo a:). In Fiiuree 1 é is the 2D height field of the mesh while is the non-controllable objects in the scene like the initial position of the fluid.
Maximizing reward The reward function fR : 9 OR - R maps the final state of length- trajectory (G or (*K) ) and reward function parameters OR E O a scalar value. In our running example, the reward function is defined as the Gaussian likelihood of each fluid particle under the closest "pool", averaged across particles (Figure Ih). Reward function parameters On are the means and standard deviations of these pools.
We define the objective under the ground-truth simulator fs as Js(Ø) :- fR(SS where k(( indicates K applications of the simulator. We want to find the design parameters that maximize Js, i.e. 6² argmax, Jo)).. We can approximate this optimization using learned fR(FF/)(F()))) (fo(e, a)); 0R). Crucially, multiple reward
(K) (fD(ó,
simulation model by maximizing JAr == fR(fM (K) functions can be optimized with the same model fM.
Optimizers Optimal design parameters d² can be found using any generic optimization technique. Given the differentiability of the learned simulator fM. we are particularly interested in evaluating gradient-based optimization, which requires fewer function evaluations and scales better to large design spaces than campling-based techniques [10. We use the Adam optimizer [46] to find ø* by computing the gradient V Jh (Ø). This involves backpropagating gradients through the reward function fn. length-K K rollout produced by [(K) and design function Jn-
As baseline, consider the cross- method (CEM) [61) gradient-free sampling- based technique that is popular in model-based control 20 73 CEM can be used with any simulator and works by sampling a population of candidates for Φ and evolving them to maximize the reward. However, CEM requires multiple evaluations of fM or fs per optimizer iteration (depending on the population size), whereas Adam requires only single evaluation per iteration. We additionally consider covariance matrix adaptation (CMA-ES) (38 and Bayesian Optimization (BO) [68] in Section D.2 as two additional gradient-free optimizers.
Across our design tasks (Section 4) we compared: gradient descent with Adam on the learned simulator model (GD-M), CEM with the learned simulator model (CEM-M), and CEM with the

---

(a] GD-M design evolution
(b) CEM-M
(c) Reward
Iteration B
Itoration 100
Final cesign
Final design
15
Contain
Reward
30
Contain
r-0.13
/- 5.44
9.90
3.72
Ramo
Reward
Ramp
10
0.05
-- 3.98
<- 9.98
C'm 4.44
GD-M
Reward
8
CEM-
CBN 5
Figure 2: 2D FLUID TOOLS results. The state spaces consist of 102-10s particles and the design spaces of 16- 36 parameters. (a) Evolution of designs found by GD-M during optimization for each 2D FLUID TooLS task. Visualizations correspond to simulations of the designs under fs. The design shown in black, fluid particles in blue, and Gaussian reward yellow. The transparent particles show the location of fluid for / < tx, and the solid particles show the location of fluid at the final frame (K 150). r denotes reward for the current design. (b) Final designs found by CEM-M. (c) Mean reward over 50 reward locations (with bootstrapped 95% confidence intervals) obtained by each optimizer across the 2D FLUID TOOLS tasks. For Contain and Ramp, results are shown for 16 joints; for Maze, results are shown for 6 x 6 grid of 36 rotors. Across these tasks, GD-M outperforms both CEM-M and CEM-S.
ground-truie truth simulator (CEM-S). We compare to further optimizer baselines, such Bayesian Opti- mization (68] and CMA-ES -SS (38) in the Appendix (Section D.2). In all tasks, fs is non non-differentiable. preventing comparison GD-S. However, in the special case of AIRFOII, we compare to DAFoam specialized solver which computes gradients with the adjoint method.
40
Evaluation Unless otherwise noted, we always evaluate the quality of an optimized design using the ground-truth objective Js(Ø"), regardless of whether o² was found using the learned model fas (as in CEM-M and GD-M) or the ground-truth simulator fs (as CEM-S). We also use rollouts from fs to produce visualizations in the figures.
4 Design Tasks
We formulate a set of design tasks across three different physical domains with high-dimensional state spaces and complex dynamics. Each domain uses different ground-truth simulator fs. which is used to evaluate designs and pre-train the learned simulator model fM in task-agnostic way. During optimization, all the design tasks on each domain share the same pre-trained model.
2D FLUID TOOLS Inspired by existing 2D physical reasoning benchmarks 1 5 these tasks
involve creating one or more 2D "tool" shapes to direct fluid into a particular goal region (Figure 2 Section C.I. We consider three tasks with 4-48 design parameters where the goal is to guide the fluid such that each particle comes as close as possible to the center of a randomly saamolll--smpeedd yellow reward region. In Contain, the joint angles @joints of a multi-segment tool must be optimized to catch the fluid by creating cup- or spoon-like shapes. In Ramp, the joint angles Øjoints tool must be optimized to guide the fluid to distant location. In Maze, the rotation Pzot of multiple tools must be optimized to funnel the fluid to the target location. Fluid dynamics are represented using 102-10- particles, and unrolled for up 300 time steps.
multi- multi-segment
The learned model Jar is trained on the 2D WaterRamps dataset released in 63 which uses the solver in 43 generate trajectories. The dataset contains scenes with 1-4 straight line segments;
5

---

(a] Direction (GD-M) 1.19 (.2134
Pools (GD-M) 0.90 (0.897, 0.909]
Pools (GD-M) r- 0.87 [0.844, 0.887]
(d) Pools (CEM-M) 0.01 10.004, 0.019)
Lower Higher
Figure 3: 3D WATERCOURSE results found with GD-M or CEM-M and evaluated with fs. Sim- ulations use up 2000 particles and 625 design parameters. The heightmap of a 2D landscape is optimized to redirect the fluid towards the purple targets; birds eye views of heightmaps are shown in the upper right corner of each subplot. In this high-dimensional task domain, GD finds designs with high reward (a-c), while CEM fails to find meaningful solutions for Pools (d) as well as the other tasks (Figure A.5 Each subplot reports the mean reward (r) and bootstrapped [lower, upper] 95% confidence intervals for the corresponding optimizer and task (averaged over 10 randomized initial designs for each task variation, see Section C.2).
curved lines or large numbers of obstacles are shown. Therefore, designs which solve 2D FLUID TOOLS tasks are necessarily out of distribution for the learned model.
To evaluate higheerdimeensinnll inverse design, we created a "landscaping" task that requires optimizing 3D surface to guide fluids into different areas of environment Figure 3. Section C.2). Specifically, water flows out of a pipe and onto an obstacle parameterized by Pmap, 25 x 25 heightmap (625 design parameters). In Direction, @map is optimized to redirect the fluid stream towards a specified direction. In Pools and 3 Pools, @map is optimized to split the fluid stream such that particles hitting the floor land as close possible to of two three specified pools. Each task has multiple variants, such as different target directions in Direction. The simulation is unrolled for 50 time steps, and contains up 2048 particles. The learned model fas is trained on data generated from the simulator in
3D WATERCOURSE
Shape optimization in aerodynamics is area where gradient- based optimization is routinely applied using traditional simulators [13. . Here, we consider the well-studied task of drag optimization of 2D airfoil profile (Figure 5| Section C.3) In this task, wing is defined using a curve on a 2D mesh, which can be deformed using a set of 10 control points Dent The task is formulated optimizing the wing shape minimize drag under certain constraints such constant lift and bounds on the shape to prevent degenerate (e.g. infinitely thin) configurations. Lift and drag coefficients are computed by running an acrodynamics simulation on 4158 node mesh. The learned model f is trained on data generated with the OpenFOAM solver (55), which pairs wing shapes with targets of RANS pressure and Reynolds stress fields discretized 4158 node mesh. Note that unlike the fluids domains, this simulation is steady-state (ie. rollout length of 1).
AIRFOIL
Model learning and optimization While each domain has different state space structure and uses a different ground-truth simulator fs. the learned simulators far all share the same architecture (with identical perrarartmeerss in 2D FLUID TOOLS and 3D WATERCOURSE, and only minor variations of the hyperparameters for AIRFOIL due to it being a steady state simulation; see Appendix B). The models are trained for stet prediction on task- independent datasets (random perturbations of the design space for AIRFOIL and 3D WATERCOURSE, and an open-source, qualitatively distinct dataset for 2D FLUID TOOLS), and are unrolled for up to 300 time steps during design optimization without further fine-tuning (see [Section 3 for details). Importantly, all the pre-trained models are shared across the multiple design tasks for each domain.
Results
Our results show that learned simulators can be used to effectively optimize various designs despite sig- nificant domain shift and long rollout lengths. The same underlying model architecture is used for each domain, highlighting the generality of learned simulators for design. Examples of designs found with

---

(b)
(e)
14-
neward (Contain)
12.5
Reeeind (Contain)
Reward (Maze)
Reooani (Cantainn
Rewsan (Cantain)
10.0
12-
-0-
20-
Full Model
No Koloe
10-
GD-M
5.0
CBV
CBN-5
50 100 150 200 250 300
37 Number of Jaints
15
26
48
30
250 500 750 1000 Enviranment Calls
50 100 150 zóo 250 ada Rollout Length
Rollout Length
Number of Tools
Figure 4: Ablation experiments on the Contain and Maze tasks. (a) Performance of all optimizers increase with rollout length; GD-M performance starts to deteriorate around step 225. (b) In Contain, CEM performance drops when increasing the number of joints above 24, while GD-M remains stable. (c) We observe similar trend with the number of tools in Maze. (d) CEM often gets stuck in sub-optimal solutions early in optimization, while GD-M performance continues to increase. (e) Model choice matters: good performance GD-M is enabled by key architectural choices of the learned model, such as training noise and translational equivariance.
our approach are available at: https:/ http://ossssss.ooleeeeee ccwm///ee//pppiiiizzineeeeeeeeee Ex- ample code can be found at h/tp:::///ttttut com deepmind httpp//www..///eeoeeeeeeeeaaaaaa design/| Performance is always evaluated using the ground-trute simulator fs (see|Section 3). Here discuss these results, and compare the capabilities of gradient descent with learned simulators over classical simulators and sampling-bar -basid optimization techniques.
Overall results We first asked whether a learned simulator combined with gradient descent (GD-M) could produce good-quality designs at all. This approach might fail in various ways: accumulating model error, vanishing or exploding gradients [9] or domain shift 36 (T]. However, as the following results show, GD-M produced high-quality designs across all three domains.
Figure 2 shows qualitative results for 2D FLUID TOOLS (eccionn 4), where our approach (GD-M) produces intuitive, functional designs to contain (Contain), transport (Ramp), or funnel (Maze) the fluid to target location. On average, GD-M outperforms CEM-M by 16.1-118 9%, indicating a substantial benefit of gradient-based optimization. GD-M also outperforms CEM-S by 3.9-37.5%. despite using a learned simulator rather than the ground-truth. However, these design spaces are still relatively small (between 16 and 36 dimensions). In 3D WATERCOURSE (Section 4), we substantially increase the dimensionality to a 625-dimensional landscape. Here, GD-M produces robust designs, creating ridges to re-route water in particular directions valleys to direct water into pools (Figure 3/-- c). By comparison, CEM-M cannot solve any of these tasks, with performance 30-85x x worse than GD-M. In the Appendix (Section D.2) we demonstrate that these results also hold for other optimizer choices (Bayesian Optimization, CMA-ES).
In AIRFOIL (Figure 5p), GD-M recovers the characteristic S-curve shape for a low-drag airfoil under a small angle of attack, and matches the design obtained with DAFoam, an adjoint aerodynamics solver which computes lose-to-optimal designs for this task. The design obtained with DAFoam yields drag coefficient of 0.01902, while GD-M finds designs with drag between 0.0.898-0.0119 depending on ensemble size (see [Section 5). Importantly. DAFoam's 's solver and optimizer are highly specialized for the particular task of airfoil design, while our approach is more general- purppss,,, requiring only trajectory data for training and generic gradient-based optimizer.
Model stability & gradient quality We investigated accuracy over long timescales by measuring the effect of rollout length on design quality 2D FLUID TooLs (Figure 4b and A. Longer roll- outs can in principle allow for higher reward in this task as they give the fluid time to settle; however, with Icarned models, they can also be unstable due to error accumulation [69 (72. Nevertheless, we find that the learned simulator does not seem to be severely impacted by this problem. Specifically, the quality of designs found by GD-M increases up to 225 steps (Figure 4h,, indicating that the learned simulator's s accuracy and gradients remain stable for surprisingly long time. Across the episode lengths evaluated, we find that GD-M outperforms not only CEM-M (by 18. 1% on average) but also CEM-S (by 4.4% average). This indicates that the benefits of having learned model that supports better optimization techniques can outweigh the small errors incurred by long rollouts.
The strong performance of GD-M on longer rollout lengths is noteworthy. Gradients tend to degrade when passed through chains of many model evaluations, and as result, previous work generally only optimizes gradients in small action spaces over just a few time-steps [5]].
7

---

(a) Initial design
(b) Optim zed designs
(c) Drag
Reward (higherss is better)
(d) Reward
Vally (lower better)
1e-2
10-5
Initiel Design
25
Ensemble-3
2.0-
SD-M
2/Fasm
1.0
0.5
Ensemble-1 1
0.0
Ensemble
Ensemble Size
Figure 5: AIRFOIL results. (a) An initial airfoil design warped by moving 10 control points (orange dots). The physics model simulates the resulting aerodynamics on a 4158 node mesh, based on which lift and drag are computed. (b) For the task of finding minimum-drag configuration under constant lift constraint, gradient-based based learned design is able to find similar designs to specialized solver DAFoam, both using single models and ensembles. (c-d) Larger ensemble sizes of 3-5 achieve a close quantitative match to DAFoam for both drag and overall reward. Shown are means over 10 randomized initial designs, with bootstrapped 95% confidence intervals.
We hypothesize that a key reason for the success of GD-M is the addition of noise in training the model fM. which promotes stability on the forward pass and smoother gradients. We test this by considering a model variant trained without noise (Figure 46, red). We observe much worse design quality compared to the full model, particularly for longer rollouts.
Improving accuracy with ensembles In engineering tasks like AIRFOIL, simulators must be especially accurate, as small differences in the predicted pressure field can cause large errors in lift and drag coefficients. While GD-M (without ensembling) produces designs close to DAFoam's, 's we notice a slightly rounder wing front (Figure 5b, bottom) causing small increase in drag (0.01919 versus 0.01902 in DAFoam).
To further improve performance, we implemented an ensemble of learned simulators trained on separate splits of the training set. Ensembles are a popular choice for training transition models for use in control 200 as they can provide higher quality predictions and are more resistant to delusions particularly problematic issue for accuracy-sensitiv domains such as airfoil design. During optimiza- tion, we make predictions with all models in the ensemble, each trained on different data split, and average the gradients. As shown in Figure 5b-c, larger ensembles yield designs with significantly lower drag (3- 5.33 x 10-5 p 0.0003, where ,3 is linear regression coefficient) and higher over- all reward (3- = 5.6 10 p = 0.0001), and are able to produce designs very close to the solution found by DAFoam, with drag coefficient of 0.01898 (size-5 ensemble). Thus are able to achieve performant designs with a general- pprppse learned simulator, indicating that we can use learned models for design optimization in spaces traditionally reserved for specialized solvers like DAFoam.
Scalability to larger design spaces In larger design spaces, sampling- based optimization proce- dures quickly become intractable, especially with relatively slow simulators. We hypothesized that gradient descent with fast, learned simulators could overcome this issuc, especially as the size of the design space is increased. We therefore compared different optimizers on 2D FLUID TOOLS as a function of the dimensionality of the design space (the number of tool joints in Contain or the number of tools in Maze) and on the higher-dimensiee
3D WATERCOURSE.
For Contain (Figure 4p), the performance of GD-M increases with the number of joints as increasingly fine grained solutions are made possible. In contrast, for both CEM-M and CEM-S, design quality deteriorates with more joints as high quality solutions become harder to find with random sampling. In the highest dimensional Contain task with 48 tool joints, GD-M outperforms CEM-M by 154.9% and CEM-S by 126.5%. When CEM does find solutions (Figure 2p). they lack global coherence and appear more jagged than solutions found with GD-M. Similarly, for Maze (Figure 4c), the performance of GD-M is largely unaffected by the number of tools, while the performance of CEM- M and CEM-S both degrade as the design space grows. For the highest dimensional Maze problem with 36 joints, GD-M outperforms CEM-M by 207.4 44% and CEM-S by 133.7%.
In the 625- 225-eemnnsional 3D WATERCOURSE task. CEM-M performs 30-85x worse than GD-M (Figure 3) despite extensive hyperparameter tuning. This trend held across all tasks (Figure A.3p), and even persisted when using fewer control points in the design space Figure A.7). This is due not only to 3D WATERCOURSE'S larger design space, but also because this problem requires a globally coherent solution: small, independent changes have negligible effect on the global movement of the fluid.
8

---

(a)
In Ditribution Designs
(b)
GNN Simulatoe (GD-M)
(c)
Endito End (GD-E2E)
(d) Actual End End Predicted
22.5*
157.5*
22.5
157.5
0.2
Delign
Devigns
Rectusl Reward
Flooss
Fooes
-5 End to End Predicted Reward
65
Figure 6: Analysis of an End-to-En surrogate reward model (GD-E2E) versus our approach (GD-M). (a) Test designs sampled from the same distribution as the training data. (b,c) Optimized designs and top view of resulting flow trajectories from the (b) Learned Simulator (GD-M) and (c) End (GD-E2E) model for two Direction tasks in 3D WATERCOURSF. (d) Ground Truth (evaluated using f.) vs Predicted reward for different sets of designs. (black dots) held-out in-distribution test designs, (blue) designs optimized using (GD-M), (orange) designs optimized using (GD-E2E).
Model speed and sample efficiency Learned simulators can provide large speedups over traditional simulators in certain domains by learning compensate for coarser sub- stepping and making optimal use of hardware acceleration. In AIRFOIL, although we use a very simple GD setup, our approach is able to find very similar designs as DAFoam's 's specialized optimizer. Moreover, approach requires only 21s (single model) to 62s (size-5 ensemble) on a single A100 GPU, compared to 1021s for DAFoam run an 8-core workstation, despite requiring 10x more optimization steps.
In 2D FLUID TOOLS, the ground-truth simulator runs at a similar speed to the learned model [see 63. but is non-differentiable and therefore depends on more expensive gradient-free optimization techniques which require more function evaluations. We use 20-40 function evaluations per opti- mization step of CEM, compared to single evaluation with GD (which is about 3x more costly, due to the gradient computation). Thus, GD with a differentiable learned model can be much more efficient than using the ground truth simulator with sampling -based method.
Generalization Deep networks often struggle to generalize far from their training data [31] This poses a problem for design: to produce in-distribution training data, we would already need to know what good designs look like, thus defeating the aim of wanting to find new designs. However, find that the GNN-based simulators generalize well in the setting studied here. As noted in [Section 4 the learned simulator for 2D FLUID TOOLS trained on pre-existing, highly simplified dataset where only one to four straight line segments interact with a fluid (Appendix G. However, it still can support the design tasks studied here, which involve highly articulated, curved obstacles (Contain, Ramp) or a larger number of obstacles (Maze), without further finetuning (Figure 2).
In Section D.6 we show similarly compelling generalization results for 3D WATERCOURSE. Without fine-tuning, we optimize the heightfields of two meshes along with their global rotations to redirect water. These design tasks consist of 1250 2 design dimensions, while also being further from the training distribution (only one, un-rotated heightfield was ever observed). Despite these added challenges, the method still produces good designs with reward between 0.52 and 1.16.
This generalization capability emerges due to certain design choices of the simulator model fM- Specifically, the GNN operates on local neighborhoods of particles, and particle coordinates are represented in a relative manner, which encourages learning local rules that can transfer very different setups. We tested a model variant which breaks this property by encoding absolute, instead of relative particles coordinates (Figure 4e, purple). We find that this variant can no longer robustly simulate out-of-domain designs, leading to worse performance at any rollout length.
Using learned simulator trained in relatively simple environment has another unexpected advantage. In rare cases, classical simulators suffer from degenerate behavior around certain edge cases. For example, with the classical simulator for 2D FLUID TOOLS, particles can get stuck in between joint segments, especially when there are large number of parts or joints (Figure A.10). However, since the learned simulator was trained on simpler data where these effects are unobserved, it picks up only on the appropriate collision performance and not the unrealistic edge cases. Thus, the learned simulator produces more plausible rollouts than the classical simulator in these cases, and might therefore be a better candidate for producing designs that would transfer to the real world. There are also cases where we would expect the classical simulator to generalize better than the learned simulator, such as in dealing with fluids with viscosities outside the training data with different local dynamics.

---

Comparing to end-to-end surrogate models To further benchmark the generalization capabilities of our approach, we compare to an end-to-end surrogate model trained to map design parameters (0map) directly to rewards for each of the d = 8 target angles of the Direction task in 3D WATER- COURSE, To train the end-to-end surrogate model, we use the same designs as those used to train the GNN dynamics model (Section C.2), but additionally paired with the rewards predicted by the ground truth simulator, so that Dsur = ((mm (duap.. d))( Ji=0 We use ResNet- 18 [39 as the surrogate model with a 25 x 25 dimensional imput image corresponding to the heightfield mesh and we train separate model for each target direction. We focus on 3D WATERCOURSE for these experiments, as the training data and design problems contain meshes in the same domain (25 x 25 meshes). For 2D FL.IID TOOLS the training data and design spaces are different, making the end-to-end surrogate approach fundamentally under-determined.
The surrogate model could fit the training set and an in-distribution test set well (MSE=0.0070 + 0.0025, Figure 6)).. However, when used for design optimization with GD optimizer, the surrogate moves into out of distribution regions of design space where it makes overly optimistic predictions (predicted=1.12 1. + 0.31, actual=0.40 0.05, MSE=0.615 0.403, Figure 6p.c). The surrogate model also severely underestimates the reward of high-scoring designs found using our method GD-M (predicted=0.33 = 0.06, actual=1.21 0.18, MSE=0.813 + 0.344, Figure 6ft). Overall, this suggests that using task-agnostic, dynamics-based models for design can have appealing regularization properties which avoid issues of model delusion that are commonly seen with end-to-end surrogate approaches.
6 Discussion
We find that state-of- sfate-of-the-art learned, differentiable physics simulators can be used with gradient-bast based optimization to solve challenging inverse design problems. Across three domains and seven tasks, which involved designing landscapes and tools to control water flows or optimizing the shape of an airfoil, we demonstrated that gradient descent with pre-trained simulators can discover high-quality designs that match or exceed the quality of those found using alternative methods. This approach succeeds in variety of interesting and surprising ways: permits gradient backpropagation through complex physical trajectories for hundreds of steps; scales to tasks with large design and state spaces (100s and 1000s of dimensions, respectively); and successfully generates designs which require the learned simulator to generalize far beyond its training data. In the classic aerodynamics problem of airfoil shape optimization, our approach produces a design comparable that of specialized solver using only simple, general-purye purpose strategies like model ensembling.
While our results have exciting implications for inverse design, they also open up possibilities for explaining everyday human behavior like tool invention-a longstanding puzzle in cognitive science With general-purposs learned simulators, we have the potential not just to create highly
1156 667. specialized tools in enginecring domains, but also to model everyday tool creation, such as creating a hook from pipe cleaner, building blanket fort, or folding paper boat.
Our approach has limitations that should be visited in future work. Gradient descent is inappropriate for design spaces with regions of zero gradients, such in 2D FLUID TOOLS tasks where the fluid may not always make contact with the tool (Figure A.9). Many interesting design tasks also have variably-sized combinatorial design spaces that cannot easily be optimized with gradient descent, such as computer-aided design (CAD) approaches to 3D modeling. An exciting future direction will be integrate general-purposs learned simulator ators with hybrid optimization techniques such those used in material science and robotics [17] (70). As learned simulators continue to improve, we could also use them to do even broader cross domas,, multi-physic design. While challenges remain, our results represent a promising step towards faster and more geurroe-purpose inverse design.
Acknowledgements
We thank Yulia Rubanova and Evan Shelhamer for their detailed comments on the manuscript, and Yusuf Aytar, Jonathan Godwin, Luis Piloto, Remi Lam, and Meire Fortunato for helpful discussions.
10

---

References
[1] Kelsey R. Allen, Kevin A. Smith, and Joshua B. Tenenbaum. Rapid trial-and-error learning with simulation supports flexible tool use and physical reasoning. Proceedings the National Academy of Sciences, 2020.
[2] W Kyle Anderson and V Venkatakrishnan. Acrodynamic design optimization unstructured grids with a continuous adjoint formulation. Computers Fluids, 28(4-5):443-480, 1999.
[3] Christof Angermüller, David Belanger, Andreea Gane, Zelda Mariet, David Dohan, Kevin Mur- phy, Lucy Colwell, and Sculley. Population-be baased black optimization for biological se- quence design. CoRR, abs/2006.03240 00227, 2020. URL https://arxiv org/ abs////..0000... 2006 03227
[4] Aharon Azulay and Yair Weiss. Why do deep convolutional networks generalize so poorly to small image transformations? CoRR, 121/1 URL http:httpp//arxi..orr//////
[5] Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. Phyre: A new benchmark for physical reasoning. Advances in Neural Information Processing Systems, 32:5082-5093, 2019.
[6] Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, al. Relational inductive biases, deep learning, and graph networks. arXiv preprint 1rXiv:1806.01200 2018.
[7] Filipe de Avila Belbute-Peres, Thomas D. Economon, and J. Zico Kolter. Combining differen- tiable PDE solvers and graph neural networks for fluid flow prediction. In Proceedings of the 37th International Conference Machine Learning ICML 2020, 2020.
[8] Jan Bender and Dan Koschier. Divergence-free smoothed particle hydrodynamics. In Proceed- ings of the 14th ACM SIGGRAPPI/Eunngruphics symposium computer animation, pages 147-155, 2015.
[9] Yoshua Bengio, Patrice Simard, and Paolo Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks, 5(2):157-166, 1994.
[10] Homanga Bharadhwaj. Kevin Xie, and Florian Shkurti. Model-predictive control via cross- entropy and gradient-based optimization. In Learning for Dynamics and Control, pages 277-286. PMLR, 2020.
[11] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne and Qiao Zhang. JAX: composable transformations of Python+NumPPy programs, 2018.
[12] David H. Brookes, Hahnbeom Park, and Jennifer Listgarten, Conditioning by adaptive sampling for robust design, 2019. URL
org. http://awww.ar/a////////00../..0
[13] Howard Buckley, Beckett Y Zhou, and David W Zingg. Airfoil optimization using practical aerodynamic design requirements. Journal of Aircraft, 4755) 2011,
[14] Keith T Butler, Jarvist M Frost, Jonathan M Skelton, Katrine Svane, and Aron Walsh. Computational materials design of crystalline solids. Chemical Society Reviews, 45(22):6138- 6146, 2016.
[15] Eduardo F Camacho and Carlos Bordons Alba. Model predictive control. Springer science & business media, 2013.
[16] Adithya Challapalli, Dhrumil Patel, and Gouqiang Li. Inverse machine learning framework for optimizing lightweight metamaterials. Materials & Design, 208: 109937, 2021.
[17] Chun-Teh Chen and Grace X. Gu. Generative deep neural networks for inverse materials design using backpropagation and active learning, Advanced Science, 195)6000 2020.

---

[18] Tianqi Chen, Bing Xu, Chiyuan Zhang, and Carlos Guestrin. Training deep nets with sublinear memory cost. arXiv preprint arXiv:1604.06/ 06174, 2016.
[19] Thomas Christensen, Charlotte Loh, Stjepan Picek, Domagoj Jakobović, Jing, Sophie Fisher, Vladimir Ceperie, John D. Joannopoulos, and Marin Soljačić, Predictive and generative machine learning models for photonic crystals. Nanophotonics, 913):4183-4192, 2020.
[20] Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforce- ment learning in handful of trials using probabilistic dynamics models. arXiv preprint /805 121 2018.
[21] Shane Colburn and Arka Majumdar. Inverse design and flexible parameterization of meta-optics using algorithmic differentiation. Communications Physics, 4(1):1-11, 2021. 2021.
[22] Stelian Coros, Bernhard Thomaszewski, Gioacchino Noris, Shinjiro Sueda, Moira Forberg, Robert W. Sumner, Wojciech Matusik, and Bernd Bickel. Computational design of mechanical characters. ACM Trans. Graph., 32(4), jul 2013.
[23] Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation based inference. Proceedings of the National Academy of Sciences, 1 117(48):30055-300 -30022, 2020.
[24] Yabo Dan, Yong Zhao, Xiang Li, Shaobo Li, Ming Hu, and Jianjun Hu. Generative adversarial networks (GAN) based efficient sampling of chemical composition space for inverse design of inorganic materials. Nature Computational Materials, 6(1):84, 2020.
[25] Marjolein Dijkstra and Erik Luijten. From predictive modelling to machine learning and reverse engineering of colloidal self-assembly. Nature Materials, 20(6):62--773, 2021.
[26] Frederik Ebert, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex Lee, and Sergey Levine. Visual foresight: Model-based deep reinforcement learning for vision-based robotic control. arXiv preprint arXiv: 18/200056, 2018
[27] Richard Eppler. Airfoil design and data. Springer Science & Business Media, 2012.
[28] Antonio Elia Forte, Paul Z. Hanakata, Lishuai Jin, Emilia Zari, Ahmad Zareei, Matheus C. Fernandes, Laura Sumner, Jonathan Alvarez, and Katia Bertoldi. Inverse design of inflatable soft membranes through machine learning. Advanced Functional Materials, page 211 1610, 2022.
[29] Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem. Brax-a a differentiable physics engine for large scale rigid body simulation, 2021.
[30] Justin Fu and Sergey Levine. Offline model- optimization via normalized maximum likelihood estimation. CoRR, abs/2102.07970, 2021. URL https://axxxi... wttpaaaiiii.rrg////////////////
07970
[31] Robert Geirhos, Carlos R Medina Temme, Jonas Rauber, Heiko H Schütt, Matthias Bethge, and Felix A Wichmann. Generalisation in humans and deep neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 7549-7561, 2018.
[32] Justin Gilmer, Samuel S Schoenholz, Patrick Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning. pages 1263-1272. PMLR, 2017.
[33] Daniel Golovin, Benjamin Solnik. Subhodeep Moitra, Greg Kochanski, John Karro, and David Sculley, Google vizier: A service for black-box optimization. In Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1487-1495, 2017.
[34] Rafael Gômez-Bombarell Jennifer N. Wei, David Duvenaud, JoséMiguel Hernández-Lobato Benjamín Sánchez-Lengeling Dennis Sheberla, Jorge Aguilera-Iparraguiree Timothy D. Hirzel, Ryan Adams, and Alán Aspuru-Guzik. Automatic chemical design using a data-driven continuous representation of molecules. ACS Central Science, 4(2):268-22 276,, 02 2018.
12

---

[35] Agrim Gupta, Silvio Savarese, Surya Ganguli, and Li Fei-Fei. Embodied intelligence via learning and evolution. Nature Communications, 12(1):5721, Oct 2021.
[36] Jessica B Hamrick, Abram L Friesen, Feryal Behbahani, Arthur Guez, Fabio Viola, Sims Withersppon, Thomas Anthony, Lars Buesing, Petar Veličković, and Théophane Weber. On the role of planning in model-based deep reinforcement learning. arXiv preprint arXiv:2011.0402 2020.
[37] Paul Z. Hanakata, Ekin D. Cubuk, David K. Campbell, and Harold S. Park. Forward and inverse design of kirigami via supervised autoencoder. Phys. Rev. Research, 2:042006, Oct 2020.
[38] Nikolaus Hansen, Pedro Larrañaga, Iñaki Inza, and Endika Bengoetxea. The CMA Evolution Strategy: Comparing Review, pages 102. Springer Berlin Heidelberg, Berlin, Heidelberg. 2006.
[39] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[40] Ping He, Charles A Mader, Joaquim RRA Martins, and Kevin Maki. Dafoam: An open-source adjoint framework for multidisciplinary design optimization with openfoam. AIAA journal, 58 (3):1304-1319 2020.
[41] Matteo Hessel, David Budden, Fabio Viola, Mihacla Rosca, Eren Sezener. and Tom Hennigan. Optax: composable gradient transformation and optimisation, in JAX!, 2020.
[42] Stephan Hoyer, Jascha Sohl-Dickstein, and Sam Greydanus. Neural reparameterizatioon improves structural optimization. CoRR, http///99904444 2019.
[43] Yuanming Hu, Yu Fang, Ziheng Ge, Ziyin Qu, Yixin Zhu, Andre Pradhana, and Chenfanfu Jiang. A moving least squares material point method with displacement discontinuity and two-way rigid body coupling. ACM Transactions on Graphics (TOG), 37(4):1-14, 2018.
[44] Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Bagan-Kelley, and Frédo Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprint 19X00:191,.00935, 2019.
[45] Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model-based policy optimization. In Advances in Neural Information Processing Systems, 2019.
[46] Diederick P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Interna- tional Conference on Learning Representations (ICLR), 2015.
[47] Aviral Kumar, Xue Bin Peng, and Sergey Levine Reward policies. CoRR, 134/55 2019. URL httt::tttaa.ii..ooo org/ www..//gg........a..a... 19tb.//9//..... 13465
[48] Siddhant Kumar, Stephanie Tan, Li Zheng, and Dennis M. Kochmann. Inverse-designed spinodoid metamaterials. Nature Computational Materials, 6(1):73, 2020.
[49] Charles L Ladson. Effects of independent variation of Mach and Reynolds numbers the low-speed aerodynamic characteristics of the NACA 0012 airfoil section, volume 4074. National Aeronautics and Space Administration, Scientific and Technical.
1988.
[50] Jichao Li, Mengqi Zhang, Joaquim Martins, and Chang Shu. Efficient aerodynamic shape optimization with deep-learning-bee based geometric filtering. AIAA Journal, 58:1-17, 07 2020.
[51] Yunzhu Li, Jiajun Wu, Russ Tedrake, Joshua B. Tenenbaum, and Antonio Torralba. Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids. In International Conference on Learning Representations, 2019.
[52] Zhaocheng Liu, Dayu Zhu, Sean P. Rodrigues, Kyu-Tae Lee, and Wenshan Cai. Generative model for the inverse design of metasurfaces. Nano Letters, 18(10)6550-6657 Sep 2018. ISSN 1530-6992.
13

---

[53] Michael Lutter, Leonard Hasenclever, Arunkumar Byravan, Gabriel Dulac-Amold, Piotr Trochim, Nicolas Heess, Josh Merel, and Yuval Tassa. Learning dynamics models for model predictive agents. arXiv preprint arXiv:2 14311,
[54] Sean Molesky, Zin Lin, Alexander Y Piggott, Weiliang Jin, Jelena Vucković, and Alejandro W Rodriguez. Inverse design in nanophotonics. Nature Photonics, 12(1 ((11)))65-670 2018.
[55] OpenCFD Ltd. OpenFOAM CFD solver. www. opwnfoam.ommmmmmm org 2021.
[56] François Osiurak and Arnaud Badets. Tool use and affordance: Manipulation- based versus weasoning-based approaches. Psychological review, 123(5):534, 2016.
[57] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter Battaglia. Learning mesh- based simulation with graph networks. In International Conference Learning Representations, 2021.
[58] Janeita Reid. Free-form deformation. caffam.githute io docs FFt//FFDD/aaaaa/ pdf 2021.
[59] Simiao Ren, Willie Padilla, and Jordan Malof. Benchmarking deep inverse models over time, and the neural-adjoint method. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 38-48 Curran Associates, Inc., 2020. URL https://pprocedding www.ig....euuss.... wwweurrpps.ccppppe//2222. 000100000880000000ff142222266660 Papee pdff
[60] Li Rhie, C Chow. Numerical study of the turbulent flow past an airfoil with trailing edge separation. AIAA Journal, 21 (11)::525511533 1983.
[61] Reuven Y Rubinstein and Dirk P Kroese. The cross-entropy method: A unified approach to monte carlo simulation, randomized optimization and machine learning. Information Science & Statistics, Springer Verlag, NY, 2004.
[62] Alvaro Sunchez-Gonzaler Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Ried- miller, Raia Hadsell, and Peter Battaglia. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning, pages 4479.
PMLR, 2018
[63] Alvaro Peter Battaglia. Learning to simulate complex physics with graph networks, In International Conference on Machine Learning, pages 8459-8468, PMLR, 2020.
Jonathan Godwin, Tobias Pfaff, Rex Ying. Jure Leskovec, and
[64] Connor Schenck and Dieter Fox. Spnets: Differentiable fluid dynamics for deep neural networks. Conference on Robot Learning (CoRL), 2018.
[65] Ney Secco, Gaetan K. W. Kenway, Ping He, Charles A. Mader, and Joaquim R. R. A. Martins. Efficient mesh generation and deformation for aerodynamic shape optimization. AIAA Journal, 2021.
[66] Wuxin Sha, Yan Li, Shun Tang, Jie Tian, Yuming Zhao, Yaqing Guo, Weixin Zhang, Xinfang Zhang, Songfeng Lu, Yuan-Cheng Cao, and Shijie Cheng. Machine Learning in polymer informatics. InfoMat, 3(4):333-361, 2021.
[67] Robert W Shumaker, Kristina R Walkup, and Benjamin B Beck. Animal tool behavior: the use and manufacture of tools by animals. JHU Press, 2011.
[68] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In F. Pereira, C.J. Burges, L. Bottou, and K.Q Wein- berger, editors, Advances in Neural Information Processing Systems. volume 25. Curran Associates, Inc., 2012. URL p/recee///roocedingg neurips wttpp/cee////////iiiiii/// pdf
[69] Erik Talvitie. Model regularization for stable sample rollouts. In UAI, pages 780-789, 2014.
[70] Marc A Toussaint, Kelsey Rebecca Allen, Kevin A Smith, and Joshua B Tenenbaum. Differen- tiable physies and stable modes for tool-use and manipulation planning. In Proceedings of the Robotics: Science and Systems, RSS 2018, 2018
14

---

[71] Brandon Trabucco, Aviral Kumar. Xinyang Geng, and Sergey Levine. Conservative objective models for effective offline model-based optimization. International Conference on Machine Learning, pages 10358-10368. PMIR, 2021.
[72] Arun Venkatraman, Martial Hebert, and Andrew Bagnell. Improving multi-step prediction of learned time series models. In Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
[73] Tingwu Wang, Xuchan Bao, Ignasi Clavera, Jerrick Hoang, Yeming Wen, Eric Langlois, Shunshi Zhang, Guodong Zhang, Pieter Abbeel, and Jimmy Ba. Benchmarking model-based reinforcement learning. arXiv preprint S.KI:.I977000057, 2019.
[74] Christopher K Williams and Carl Edward Rasmussen. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006.
[75] Jie Xu, Tao Chen, Lara Zlokapa, Wojciech Matusik, Shinjiro Sueda, and Pulkit Agrawal. An end-to-end differentiable framework for contact-aware robot design. Robotics: Science and Systems, 2021.
[76] Sunwoong Yang, Sanga Lee, and Kwanjung Yee. Inverse design optimization framework via a two-step deep learning approach: application wind turbine airfoil. Engineering with Computers, 2022. doi: 10. 00007//000666-02--016161
URL https: org/ 12///10.11 1007/
[77] Sihyun Yu, Sungsoo Ahn, Le Song, and Jinwoo Shin. Roma: Robust model adaptation for offline model-based optimization. Advances in Neural Information Processing Systems, 34, 2021.
[78] Li Zheng, Siddhant Kumar, and Dennis M. Kochmann. Data-driven topology optimization of spinodoid metamaterials with seamlessly tunable anisotropy. Computer Methods in Applied Mechanics and Engineering. 383:113894 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See e.g. failure cases in
Section D.3. and Discussion.
(c) Did you discuss any potential negative societal impacts of your work? [No] We didn't identify anything specific to this paper, the same considerations as for most work in
ML apply (c.g. cost/impact of compute).
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed reproduce the main exper- imental results (either in the supplemental material or as a URLY? [Yes] The core model we use (MeshGraphNetts) has an open-source implementation available, as do the solvers we used to for the task domains (cited in the paper). Detailed description of the task setup can be found Appendix C
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendixx A) Appendix B
(c) Did you report error bars (e-g., with respect the random seed after running experi- ments multiple times)? [Yes]
15

---

(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [No] We didn't 't calculate the total amount; but model training was done on typical hardware (up to 4 TPUv3/v 100 GPUs, see Appendix By.
4. If you using existing assets (e.g., code, data, models) curating/releasing
assets... (a) If your work uses existing assets, did you cite the creators? [Yes] We didn't 't use any data, but cite the model, and solvers used for the task domains.
(b) Did you mention the license of the assets? [No] All of them are available under open-source licenses.
(c) Did you include any new assets either the supplemental material or as URL? [No]
(d) Did you discuss whether and how consent was obtained from people whose data you're re using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
16


Reviews:
[{"review_id": "VU2Kjdd0EcO", "reviewer": "Reviewer_p9AQ", "summary": "The authors investigate the performance of learned GNN simulators for inverse design.", "questions": "* GNNs simulators are chosen here without much explanation other than \"they work\".  Do you think the good performance of GNNs here might be due to tasks and dynamics involved, rather than some intrinsic (unexplained) superiority of the method? As the core focus of the  paper is performance in downstream design with learned simulators, an analysis of how other architectures used as surrogate solvers (for example, neural operators (1, 2, 3, 4, 5) ) compare would have been appreciated, particularly for the airfoil example.\n* Why has CEM been chosen as the only baseline? The empirical performance of GNN simulators is the entire focus of this work, and any claim made would be stronger with at least another gradient-free optimization technique. I would appreciate comparisons with at least a single simple baseline from reinforcement learning, since it is mentioned in the introduction.\n\n(1) Li et al., Fourier Neural Operator for Parametric Partial Differential Equations\n\n(2) Guibas et al., Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers\n\n(3) Gupta et al., Multiwavelet-based Operator Learning for Differential Equations\n\n(4) Poli et al., Transform Once: Efficient Operator Learning in Frequency Domain\n\n(5) Tran et al., Factorized Fourier Neural Operators", "limitations": "The authors discuss some limitations of the method. Some suggestions are provided in **questions** above (lack of baseline comparisons, no comparison with other types of learned simulators, and thus limited analysis on which tasks are appropriate for the approach and which require different models).", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n* The approach is appealing in its simplicity: using a fixed, trained surrogate solvers for inverse problems, rather than relying on devising \"in-the-loop\" strategies to fine-tune the solver during solution of downstream tasks.\n* The experiments are varied and interesting. The authors evaluate the model across particle-based fluid simulations as well as field-based steady-state flow around airfoils.\n\n**Weaknesses:**\n\n* From the perspective of advancing state-of-the-art (empirical or theoretical), the paper is not particularly satisfying due to a lack of contextualization of the proposed results. Part of it is certainly due to the fact that the tasks chosen involve a large amount of moving pieces. The authors provide ablations and helpful comments; however, the experimental section of the paper reads more like a post-hoc description of \"we did this specific thing and it worked\". \n* I agree that showcasing GNN surrogate solvers on inverse problems is interesting. I also agree that this is a high-impact and open area of research. I do not think the experiments provided here do a good enough job at motivating the choice of models and baselines. The paper could benefit from adding targeted, designed \"small scale\" experiments aimed at clarifying why a specific choice made in the approach is appropriate. \n* Only a single baseline (CEM) is considered. \n* It would be of great value to the community if the code was open-sourced, particularly for an empirical work such as this one. Even with a detailed appendix, I do not believe the results are completely reproducible as is, especially given the amount hyperparameters involved (in both models and ground-truth simulators such as OpenFOAM). The authors attempt, wherever possible, to provide rules of thumb and empirical takeaways, which is appreciated.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "LQnPJFYVjzx", "reviewer": "Reviewer_J3rt", "summary": "This paper approaches the problem of modeling fluids using GNNs. The paper then shows how using GNNs as differentiable simulators enables the design of components that help solve fluid manipulation tasks. ", "questions": "* Just to confirm, a different pretrained model is used per domain, but the same one is used across tasks?  E.g. all tasks for the 3D Watercourse use the same pretrained model but not the same one as for the airfoil.\n* While the GNN is learnt independent of the reward function, it does seem like knowledge of the final application is included in the trained data set. E.g. for the waterworks model, the data includes a randomized obstacle plane that was sampled from random rotations and sine wave deformations. This seems like the deformations were purposely made to be smooth. What would happen if you added shot noise to the training data? Also how do you know when your model is sufficiently general? This seems vital to the motivation of the paper.\n", "limitations": "A limitation is that the paper focuses on fluid dynamics but references the area domain as a more general physical design problem. This should be highlighted.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths:\n\n* The paper is well-written and follows a clear structure.\n* The experiments section is particularly well-written. The Figures are clear and really show the strength of the approach and also highlight a good choice of experiments. The additional supplementary materials on the webpage were helpful.\n* The GNN does seem to do a good job of modeling the dynamics of the fluids for the three different domains.\n\n## Weaknesses:\n\n* The strength of this paper is as an applications paper. As a result the novelty is limited to the applications and not to the approach. It is not clear how this particular surrogate model is novel when compared to GNS and MeshGraphNets. It is not necessarily the case that just submitting an applications paper is a problem, it is just that the novelty of the approach must be taken into consideration.\n* As an applications paper with valuable examples, it would be important to the community to make these examples available. The code does not seem to be provided with the paper and this would seem like a serious issue when considering whether the community would benefit from this paper.\n\n\n## Minor Weaknesses:\n\n* The title seems a little misleading as it is especially general. The abstract is also very general “a task-agnostic approach to inverse design…”. There is no mention of the fact that all experiments are within the domain of computational fluid dynamics. The reason the approach is so successful seems to be because the GNNs nicely match the simulators and the relationships between the fluid particles. It would probably make sense to make this clearer with a more specific title/abstract.\n* Line 280-283: This seems to be an artifact of the simulator and not necessarily a strength/robustness of the approach. This part of the paper reads as if the fact that the ML model did some “smoothing” means that the ML model manages to avoid predicting the edge cases where the simulator fails and therefore is a more robust model choice. However, the reality is the ML model is actually failing when it does not predict these simulator edge cases as the ML model has only seen data from the “flawed” simulator and has only learnt information about the real-world through the “flawed” simulator. Perhaps this point should be removed.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "F9PlzmCeh16", "reviewer": "Reviewer_oQtT", "summary": "In this work, the authors applied a learned physical simulator MESHGRAPHNETS [55] to physical design. \n\nMESHGRAPHNETS is a graph network that can learn and predict the interaction among local particles (in fluids) or nodes (in meshes). Since the dynamics are described by a neural network, the simulation sequence is naturally differentiable when the time series is rolled out. Therefore, people can define some task-related objective functions on the trajectory and optimize the boundary condition using gradient-based methods.\n\nExperiments demonstrate that this strategy works work even with the learned physical simulator. Although neural networks are prone to overfitting, and the long timestep could easily result in error accumulation, the experiments show that designs optimized on the MESHGRAPHNETS simulator work pretty well on ground-truth simulators.\n\n", "questions": "Generally, this paper writes well and I only have several minor questions.\n\n1. In L279, \n\n\"However, since the learned simulator was trained on simpler data where these effects are unobserved, it picks up only on the appropriate collision performance and not the unrealistic edge cases. Thus, the learned simulator produces more plausible rollouts than the classical simulator in these cases, and might therefore be a better candidate for producing designs that would transfer to the real world.\"\n\nIt is common that learned physics can have low fidelity in uncovered corner cases. The corner cases are a major issue for other domains like autonomous cars. I wonder in the experiments whether the inaccuracy helps more or causes more problems for the designing process. \n\n2. What is the definition of theta_R in L142? Is it defined somewhere before?", "limitations": "As I said in the Strengths And Weaknesses part, I think the technical contribution of this paper is limited. I suggest, if possible, doing some more challenging experiments or proposing some new algorithms.\n\n1. Is it possible to involve some control policy networks within the open loop of design? Some new RL algorithms could be introduced in such a pipeline.\n\n2. Can this method deal with larger-scale or more realistic scenarios? Like 3D Airfoil or more realistic fluids experiments (the particles in the fluids videos looks very toy demo).\n\n3. Can this method be applied to the real world? A 3D printed watercourse or a 2D 'fluid' tool on a table with marbles can be very cool and much more convincing.", "rating": 5, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "I honestly like the message and ideology in this paper very much. However, the biggest concern for me is that the novelty of this paper itself is not that much.\n\nStrengths\n\nFirst, the paper is well-written and easy to follow. The motivation for conducting gradient-based design optimization using the learned physics simulator makes total sense. There have been a lot of previous methods that did this as well.\n\nSecond, although the experiment settings are simple, the results of the optimization results are impressive. It is known that prior learned surrogate physics models (I have also played with some of them) usually generalize poorly to out-of-domain data, let alone in a long simulation sequence where errors accumulation can be significant. But this paper demonstrates that MESHGRAPHNETS [55] performs pretty well and can optimize a design that is reasonably well in the ground-truth simulator.\n\nWeaknesses\n\nNevertheless, in my view, the good results demonstrated in this paper should be credited to MESHGRAPHNETS. This paper simply wraps MESHGRAPHNETS into an optimizer package and then gets good results. As mentioned in L72, utilizing a differentiable learned module to do design optimization is very common. For the tasks in this submission, it is the design of MESHGRAPHNETS that makes this end-to-end optimization stable, generalizable, scalable, and efficient. The authors did a good verification for MESHGRAPHNETS but I hope to see more original and novel techniques.\n\nFor example, it would be good if the authors can show more challenging tasks, for example, training a control policy in the learned simulation which can instead be deployed in the ground-truth simulator or even the real world.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9sfrwY6YPd1", "reviewer": "Reviewer_zFDa", "summary": "This paper proposes to 'invert' graph neural networks (GNNs) in the search for optimal solutions in the context of engineering design. The paper uses stochastic gradient descent for optimization and observes that the gradients can be reliably propagated in a coarse trajectory. The paper evaluates the method in different CFD tasks and shows significant advantages compared to sampling-based techniques used in model-based control. An important advantage of the method is its generalization to unseen inverse-design scenarios. ", "questions": "\n- Line 108: The abbreviation GNS is not defined. \n- Generalization in the 2D Fluid Tools is well explained (Lines 265-269). I could not find the same explanation for the airfoil setup. It is unlikely, but it appears that (Lines 198-205) the GNN is learned in a typical supervised way. I checked the supplementary materials and was left a bit confused. For example, speaking of a rollout of length 1 (supplementary materials Line 766) means the whole learning has been carried out without transitions specific to GNNs? ", "limitations": "- I think the generality argument could have been better supported using a more systematic approach. \n- The suitability of the forward model toward CFD tasks could be highlighted. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "STRENGTHS: \n- The important observation of suitability of using GNNs for inverse design. \n- The solid execution of the idea and an extensive set of applications in different scenarios. \n- The writing. \n\n\nWEAKNESSES: \n- The lack of evaluation against a fully data-driven approach to inverse design. These models, which are learned from pairs of data in a supervised fashion (for example, MO-PaDGAN: Reparameterizing Engineering Designs for Augmented Multi-Objective Optimization), are arguably as important as the sampling methods used for comparison in the paper.  \n- The method is incremental in a straightforward way in the sense it observes the general behavior of GNNs and proposes gradient-based optimization in the context of inverse design. The strengths of the method (long trajectories, generalization) are native to the forward surrogate models (GNNs) introduced in earlier works and not because of special treatment of the inverse problem per se. This makes the paper more an application of the GNN simulations. However, given the broad implication, this should not prevent the paper from a presentation to a broad audience. \n- The tile is unnecessarily broad. \"Differentiable learned simulator\" could pertain to a lot of different ML models (MLP, CNN, GP, etc.). Also, the applications seem well suited to computational fluid dynamics (CFD) tasks as shown in all 3 examples. In a more focused approach, this paper could have been called something along the lines: Inverse design in CFD using graph neural networks. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept