Original: Inverse Design for Fluid-Structure Interactions using Graph Network Simulators
Revised: Inverse Design for Fluid-Structure Interactions using Graph Network Simulators

Original: Abstract
Revised: Abstract

Original: Designing physical artifacts that serve a purpose—such as tools and other functional structures—is central to engineering as well as everyday human behavior.
Revised: Designing physical artifacts that serve a purpose—such as tools and other functional structures—is central to engineering as well as everyday human behavior.

Original: Though automating design using machine learning has tremendous promise, existing methods are often limited by the task-dependent distributions they were exposedto during training.
Revised: Though automating design using machine learning has tremendous promise, existing methods are often limited by the task-dependent distributions they were exposed to during training.

Original: Here we showcase a task-agnostic approach to inverse design, bycombining general-purpose graph network simulators with gradient-based designoptimization.
Revised: Here we showcase a task-agnostic approach to inverse design, by combining general-purpose graph network simulators with gradient-based design optimization.

Original: This constitutes a simple, fast, and reusable approach that solveshigh-dimensional problems with complex physical dynamics, including designingsurfaces and tools to manipulate fluid flows and optimizing the shape of an airfoilto minimize drag.
Revised: This constitutes a simple, fast, and reusable approach that solves high-dimensional problems with complex physical dynamics, including designing surfaces and tools to manipulate fluid flows and optimizing the shape of an airfoil to minimize drag.

Original: This framework produces high-quality designs by propagatinggradients through trajectories of hundreds of steps, even when using models thatwere pre-trained for single-step predictions on data substantially different from thedesign tasks.
Revised: This framework produces high-quality designs by propagating gradients through trajectories of hundreds of steps, even when using models that were pre-trained for single-step predictions on data substantially different from the design tasks.

Original: In our fluid manipulation tasks, the resulting designs outperformedthose found by sampling-based optimization techniques.
Revised: In our fluid manipulation tasks, the resulting designs outperformed those found by sampling-based optimization techniques.

Original: In airfoil design, theymatched the quality of those obtained with a specialized solver.
Revised: In airfoil design, they matched the quality of those obtained with a specialized solver.

Original: Our results suggestthat despite some remaining challenges, machine learning-based simulators arematuring to the point where they can support general-purpose design optimizationacross a variety of fluid-structure interaction domains.
Revised: Our results suggest that despite some remaining challenges, machine learning-based simulators are maturing to the point where they can support general-purpose design optimization across a variety of fluid-structure interaction domains.

Original:  Introduction
Revised:  Introduction

Original: Automatically designing objects to exhibit a desired property—often referred to as inverse design —has the potential to transform science and engineering, including aerodynamics [27], material design[14], optics [21] and robotics [35, 75].
Revised: Automatically designing objects to exhibit a desired property—often referred to as inverse design — has the potential to transform science and engineering, including aerodynamics [27], material design [14], optics [21] and robotics [35, 75].

Original: Despite its promise, widespread practice of inverse design hasbeen limited by the availability of fast, general-purpose forward models.
Revised: Despite its promise, widespread practice of inverse design has been limited by the availability of fast, general-purpose forward models.

Original: Traditionally, many methodsrely on specialized “classical” solvers, which are handcrafted to simulate a particular physical process.
Revised: Traditionally, many methods rely on specialized “classical” solvers, which are handcrafted to simulate a particular physical process.

Original: However, these models can sometimes be slow, may not provide gradients, or in certain domains,might even be unknown [23].
Revised: However, these models can sometimes be slow, may not provide gradients, or in certain domains, might even be unknown [23].

Original: To mitigate these issues, machine learning is playing an increasingly important role by substitutingall or part of the inverse design pipeline.
Revised: To mitigate these issues, machine learning is playing an increasingly important role by substituting all or part of the inverse design pipeline.

Original: “End-to-end” methods can bypass expensive simulationand data collection by learning a mapping directly from design parameters (such as the shape ofan airplane wing) to a task objective (such as the observed drag) [59, 76, 19].
Revised: “End-to-end” methods can bypass expensive simulation and data collection by learning a mapping directly from design parameters (such as the shape of an airplane wing) to a task objective (such as the observed drag) [59, 76, 19].

Original: Learned generativemodels or policies can propose candidate design parameters for a given task [52, 17, 19, 24, 12, 47].
Revised: Learned generative models or policies can propose candidate design parameters for a given task [52, 17, 19, 24, 12, 47].

Original: These types of methods can be used to solve narrowly defined tasks more efficiently than traditionalmethods, but come with significant limitations: they require access to a dataset of example designsand tasks for training, and often cannot generalize to tasks or designs outside this distribution [71, 77].
Revised: These types of methods can be used to solve narrowly defined tasks more efficiently than traditional methods, but come with significant limitations: they require access to a dataset of example designs and tasks for training, and often cannot generalize to tasks or designs outside this distribution [71, 77].

Original: In this work, we focus on a task-agnostic approach to inverse design, by employing learned simulatormodels pre-trained on dynamics with no access to the task objective or design space during training, and that, once trained, can be used to solve many new design tasks.
Revised: In this work, we focus on a task-agnostic approach to inverse design, by employing learned simulator models pre-trained on dynamics with no access to the task objective or design space during training, and that, once trained, can be used to solve many new design tasks.

Original: This approach, however, requiresmodels with sufficient long-term stability and strong generalization, which commonly applied blackbox model architectures such as CNNs often lack [45, 69, 72, 4].
Revised: This approach, however, requires models with sufficient long-term stability and strong generalization, which commonly applied blackbox model architectures such as CNNs often lack [45, 69, 72, 4].

Original: A class of learned physics simulators based on graph neural networks (GNNs) [57, 63] could be apromising candidate for supporting inverse design, as they have shown success in general-purposephysical prediction, exhibiting high accuracy and generalization for a broad range of physics tasks.
Revised: A class of learned physics simulators based on graph neural networks (GNNs) [57, 63] could be a promising candidate for supporting inverse design, as they have shown success in general-purpose physical prediction, exhibiting high accuracy and generalization for a broad range of physics tasks.

Original: However, these models have so far mostly been tested for forward prediction alone.
Revised: However, these models have so far mostly been tested for forward prediction alone.

Original: To be useful in adesign setting, they must also be smooth and robust, as optimizers will exploit unreliable regions inthe state space [53].
Revised: To be useful in a design setting, they must also be smooth and robust, as optimizers will exploit unreliable regions in the state space [53].

Original: Moreover, to use scalable gradient-based optimization techniques, simulationmodels must additionally exhibit gradient stability over extremely large state and action spaces, andover hundreds of time-steps.
Revised: Moreover, to use scalable gradient-based optimization techniques, simulation models must additionally exhibit gradient stability over extremely large state and action spaces, and over hundreds of time-steps.

Original: It is unclear if graph network simulators exhibit these properties.
Revised: It is unclear if graph network simulators exhibit these properties.

Original: In this paper, we investigate whether learned GNN simulators can support effective inverse design.
Revised: In this paper, we investigate whether learned GNN simulators can support effective inverse design.

Original: We pretrain a GNN simulator on a task-independent dynamics dataset, and at test time performgradient-based optimization for zero-shot design across a variety of tasks.
Revised: We pretrain a GNN simulator on a task-independent dynamics dataset, and at test time perform gradient-based optimization for zero-shot design across a variety of tasks.

Original: We perform a large set ofexperiments and find that (1) the learned simulator model is accurate and flexible enough to support awide variety of design tasks; (2) it generalizes well enough to support design tasks far outside thetraining regime; (3) gradients from the learned model turn out to be very useful for inverse design,with gradient-based optimization strongly outperforming a variety of sampling-based optimizers; (4)and surprisingly, gradients remain stable even when rolling the model out for hundreds of steps.
Revised: We perform a large set of experiments and find that (1) the learned simulator model is accurate and flexible enough to support a wide variety of design tasks; (2) it generalizes well enough to support design tasks far outside the training regime; (3) gradients from the learned model turn out to be very useful for inverse design, with gradient-based optimization strongly outperforming a variety of sampling-based optimizers; (4) and surprisingly, gradients remain stable even when rolling the model out for hundreds of steps.

Original: Inexperiments on high-dimensional fluid manipulation (2D F LUID T OOLS and 3D W ATERCOURSE )
Revised: In experiments on high-dimensional fluid manipulation (2D F LUID T OOLS and 3D W ATERCOURSE )

Original: we demonstrate that gradient-based optimization on learned models can find high-quality designsover hundreds of time steps, through states with thousands of particles, in tasks with up to 625 designparameters.
Revised: we demonstrate that gradient-based optimization on learned models can find high-quality designs over hundreds of time steps, through states with thousands of particles, in tasks with up to 625 design parameters.

Original: The same model could match performance of a specialized solver on aerodynamics design(A IRFOIL ) with a much simpler setup, highlighting the flexibility and precision of this approach.
Revised: The same model matches performance of a specialized solver on aerodynamics design (A IRFOIL ) with a much simpler setup, highlighting the flexibility and precision of this approach.

Original:  Related Work
Revised:  Related Work

Original: Modern design techniques rely on physical simulators so that designs can be evaluated virtuallybefore being tested physically.
Revised: Modern design techniques rely on physical simulators so that designs can be evaluated virtually before being tested physically.

Original: Solving inverse design problems with simulators has been studiedin a variety of disciplines, including nanophotonics [54], material science [25], mechanical design[22], and aerodynamics [2, 60].
Revised: Solving inverse design problems with simulators has been studied in a variety of disciplines, including nanophotonics [54], material science [25], mechanical design [22], and aerodynamics [2, 60].

Original: Many of these rely on black-box optimization, including activeand adaptive sampling methods [68, 3].
Revised: Many of these rely on black-box optimization, including active and adaptive sampling methods [68, 3].

Original: In recent years, differentiable simulators [29, 44, 64] havegarnered attention, as they allow for more sample-efficient gradient-based optimization.
Revised: In recent years, differentiable simulators [29, 44, 64] have garnered attention, as they allow for more sample-efficient gradient-based optimization.

Original: However,like classical solvers, they are typically narrow in application scope, as many simulation techniquesare hard to express as a differentiable program (e.g. constraint dynamics and multiphysics coupling).
Revised: However, like classical solvers, they are typically narrow in application scope, as many simulation techniques are hard to express as a differentiable program (e.g. constraint dynamics and multiphysics coupling).

Original: Our work aims to lessen this burden by learning these simulators from data.
Revised: Our work aims to lessen this burden by learning these simulators from data.

Original: Machine learning for inverse design replaces or augments the simulation model and/or the optimization procedure with learned modules [e.g. 16, 19, 34, 28, 37, 42, 48, 50, 52, 59, 66, 78].
Revised: Machine learning for inverse design replaces or augments the simulation model and/or the optimization procedure with learned modules [e.g. 16, 19, 34, 28, 37, 42, 48, 50, 52, 59, 66, 78].

Original: Generative approaches enable efficient optimization by learning to sample more likely designs (thuscalling an expensive simulator fewer times).
Revised: Generative approaches enable efficient optimization by learning to sample more likely designs (thus calling an expensive simulator fewer times).

Original: End-to-end predictive approaches learn a surrogatemodel that maps designs to task rewards directly (replacing the simulator entirely).
Revised: End-to-end predictive approaches learn a surrogate model that maps designs to task rewards directly (replacing the simulator entirely).

Original: This permits notonly sampling-based optimization, but also gradient-based techniques when the surrogate model isdifferentiable [59].
Revised: This permits not only sampling-based optimization, but also gradient-based techniques when the surrogate model is differentiable [59].

Original: However, surrogate models often generalize poorly to designs that lie outsideof their training dataset, so many ML approaches for design must keep the optimization procedureclose to the training dataset [12, 30], or introduce alternative objectives for the surrogate model toincrease robustness [71, 77].
Revised: However, surrogate models often generalize poorly to designs that lie outside of their training dataset, so many ML approaches for design must keep the optimization procedure close to the training dataset [12, 30], or introduce alternative objectives for the surrogate model to increase robustness [71, 77].

Original: In all cases, these methods require access to datasets with (design, taskreward) pairs on the specific task, making zero-shot generalization to new tasks impossible.
Revised: In all cases, these methods require access to datasets with (design, task reward) pairs on the specific task, making zero-shot generalization to new tasks impossible.

Original: Instead,we propose to learn the underlying physical dynamics independent of the design task , allowing boththe design space and task to be specified at design time.
Revised: Instead, we propose to learn the underlying physical dynamics independent of the design task , allowing both the design space and task to be specified at design time.

Original: In this way, our approach is related to methods in model-based reinforcement learning (MBRL),which learns action-conditioned transition models to support flexible downstream planning andcontrol. However, learned simulators are rarely accurate, stable, and generalizable enough to supportbetter downstream task performance [36, 53]. Techniques from MBRL need to mitigate modelerror either by learning a global, task-conditioned policy during model learning [45] (which thencannot be adapted to novel tasks on-the-fly), or by re-planning at every timestep to avoid erroraccumulation [15]. Because of such sensitivities, these methods have only been demonstrated in
Revised: [Figure]

Original: [Figure] [Table] [Figure]
Revised: [Table] [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Figure]
Revised: [Figure]

Original:  relatively small-scale problems, for example with (up to 21 degrees of freedom DOF [10, 62, 73])or more complex domains with small action spaces (6 DOF [51]), unlike the thousands of statedimensions and hundreds of action dimensions and timesteps we consider here.
Revised: In this way, our approach is related to methods in model-based reinforcement learning (MBRL), which learns action-conditioned transition models to support flexible downstream planning and control. However, learned simulators are rarely accurate, stable, and generalizable enough to support better downstream task performance [36, 53]. Techniques from MBRL need to mitigate model error either by learning a global, task-conditioned policy during model learning [45] (which then cannot be adapted to novel tasks on-the-fly), or by re-planning at every timestep to avoid error accumulation [15]. Because of such sensitivities, these methods have only been demonstrated in relatively small-scale problems, for example with (up to 21 degrees of freedom DOF [10, 62, 73]) or more complex domains with small action spaces (6 DOF [51]), unlike the thousands of state dimensions and hundreds of action dimensions and timesteps we consider here.

Original: We focus on graph neural networks as our simulator model, as they have recently demonstratedhigh accuracy, stability, domain-generality, and generalization performance in the forwards directionfor physics domains [7, 57, 63].
Revised: We focus on graph neural networks as our simulator model, as they have recently demonstrated high accuracy, stability, domain-generality, and generalization performance in the forwards direction for physics domains [7, 57, 63].

Original: However, it is not known whether these simulators can nowsupport gradient-based, high-dimensional inverse design which demands high accuracy, well-behavedgradients, long-term rollout stability, and generalization beyond the training data.
Revised: However, it is not known whether these simulators can now support gradient-based, high-dimensional inverse design which demands high accuracy, well-behaved gradients, long-term rollout stability, and generalization beyond the training data.

Original: Our experimentalinvestigations show that they can, and that particular architectural choices are critical to this success.
Revised: Our experimental investigations show that they can, and that particular architectural choices are critical to this success.

Original:  Problem Formulation
Revised:  Problem Formulation

Original: Consider the design task depicted in Figure 1, in which the goal is to direct a stream of water (shown inblue) into two “pools” (shown in purple) by designing a “landscape” (shown in green) parameterizedas a 2D height field.
Revised: Consider the design task depicted in Figure 1, in which the goal is to direct a stream of water (shown in blue) into two “pools” (shown in purple) by designing a “landscape” (shown in green) parameterized as a 2D height field.

Original: Here, an ideal design will create ridges and valleys that direct fluid into thetwo targets.
Revised: Here, an ideal design will create ridges and valleys that direct fluid into the two targets.

Original: In the next sections, we formalize what it means to find and evaluate such a design anddiscuss our choices for simulator and optimizer.
Revised: In the next sections, we formalize what it means to find and evaluate such a design and discuss our choices for simulator and optimizer.

Original: Learned simulators To demonstrate the utility of learned simulators for finding physical designs,we rely on the recently developed M ESH G RAPH N ETS model [57], which is an extension of the
Revised: Learned simulators To demonstrate the utility of learned simulators for finding physical designs, we rely on the recently developed M ESH G RAPH N ETS model [57], which is an extension of the

Original: Graph Network Simulator (GNS) model for particle simulation [63].
Revised: Graph Network Simulator (GNS) model for particle simulation [63].

Original: M ESH G RAPH N ETS is a typeof message-passing graph neural network (GNN) that performs both edge and node updates [6, 32],and which was designed specifically for physics simulation by enforcing locality and translationequivariance.
Revised: M ESH G RAPH N ETS is a type of message-passing graph neural network (GNN) that performs both edge and node updates [6, 32], and which was designed specifically for physics simulation by enforcing locality and translation equivariance.

Original: We briefly summarize the learned simulator here, and refer interested readers to theoriginal papers for details.
Revised: We briefly summarize the learned simulator here, and refer interested readers to the original papers for details.

Original: We consider simulations over physical states represented as graphs G ∈ G .
Revised: We consider simulations over physical states represented as graphs G ∈ G .

Original: The state G =
Revised: The state G =

Original: ( V, E )has nodes V connected by edges E , where each node v ∈ V is associated with a position u v andadditional dynamical quantities q v .
Revised: ( V, E ) has nodes V connected by edges E , where each node v ∈ V is associated with a position u v and additional dynamical quantities q v .

Original: These graphs may be either meshes (as in M ESH G RAPH N ETS )or particle systems (as in GNS).
Revised: These graphs may be either meshes (as in M ESH G RAPH N ETS ) or particle systems (as in GNS).

Original: In a mesh-based system (such as A IRFOIL ), V and E correspondto vertices and edges in the mesh, respectively.
Revised: In a mesh-based system (such as A IRFOIL ), V and E correspond to vertices and edges in the mesh, respectively.

Original: In a particle system (such as 2D F LUID T OOLS ),each node corresponds to a particle and edges are computed dynamically based on proximity.
Revised: In a particle system (such as 2D F LUID T OOLS ), each node corresponds to a particle and edges are computed dynamically based on proximity.

Original: Under this framework, we can also consider hybrid mesh-particle systems (such as 3D W ATERCOURSE ).
Revised: Under this framework, we can also consider hybrid mesh-particle systems (such as 3D W ATERCOURSE ).

Original: The model is constrained to perform local computation, by performing message passing over theseedges, and to perform equivariant computations by encoding position as the relative displacement anddistance between neighboring particles.
Revised: The model is constrained to perform local computation, by performing message passing over these edges, and to perform equivariant computations by encoding position as the relative displacement and distance between neighboring particles.

Original: Intuitively, this allows it to generalize well to scenarios wherethe local interactions are the same but the global arrangement of particles changes.
Revised: Intuitively, this allows it to generalize well to scenarios where the local interactions are the same but the global arrangement of particles changes.

Original: See Appendix Bfor model implementation details, and Appendix C for domain-specific details.
Revised: See Appendix B for model implementation details, and Appendix C for domain-specific details.

Original: The simulation dynamics are given by a “ground-truth” simulator f S : G → G which maps the stateat time t to that at time t + ∆ t .
Revised: The simulation dynamics are given by a “ground-truth” simulator f S : G → G which maps the state at time t to that at time t + ∆ t .

Original: The simulator f S can be applied iteratively over K time steps to yielda trajectory of states, or a “rollout,” which we denote ( G t 0 , ..., G t K ) .
Revised: The simulator f S can be applied iteratively over K time steps to yield a trajectory of states, or a “rollout,” which we denote ( G t 0 , ..., G t K ) .

Original: Using M ESH G RAPH N ETS ,we learn an approximation f M of the ground-truth simulator f S .
Revised: Using M ESH G RAPH N ETS , we learn an approximation f M of the ground-truth simulator f S .

Original: The learned simulator f M canbe similarly applied to produce rollouts ( ˜ G t 0 , ˜ G t 1 , ..., ˜ G t K ) , where ˜ G t 

Revised: The learned simulator f M can be similarly applied to produce rollouts ( ˜ G t 0 , ˜ G t 1 , ..., ˜ G t K ) , where ˜ G t

Original: = G t 0 represents initialconditions given as input.
Revised: = G t 0 represents initial conditions given as input.

Original: We note that a learned simulator allows us to takemuch larger time-stepsthan f S, permitting shorter rollout lengths: in our running example, one model step corresponds to200 internal steps of the classical simulator.
Revised: We note that a learned simulator allows us to take larger time-steps than f S permitting shorter rollout lengths: in our running example, one model step corresponds to 200 steps of the classical simulator.

Original: See Figure 1a for an illustration of simulation using alearned model.
Revised: See Figure 1a for an illustration of simulation using a learned model.

Original: To improve stability for longer rollouts, random Gaussian noise was added to theinputs during training.
Revised: To improve stability for longer rollouts, random Gaussian noise was added to the inputs during training.

Original: To optimize a physical design, we leverage the pipeline shown in Figure 1: (1) transform designparameters into an initial scene, (2) simulate the scene using f M or f S , (3) evaluate how well thesimulation achieves the desired behavior, and (4) adjust the design parameters accordingly.
Revised: To optimize a physical design, we leverage the pipeline shown in Figure 1: (1) transform design parameters into an initial scene, (2) simulate the scene using f M or f S , (3) evaluate how well the simulation achieves the desired behavior, and (4) adjust the design parameters accordingly.

Original: Design parameters To produce the initial state G t 0 , we introduce a differentiable design functionf D : Φ ×
Revised: Design parameters To produce the initial state G t 0 , we introduce a differentiable design function f D : Φ ×

Original: A → G which maps design parameters ϕ ∈ Φ and other initial conditions
Revised: A → G which maps design parameters ϕ ∈ Φ and other initial conditions

Original: α ∈
Revised: α ∈

Original: A toan initial state: G t 0{+
+} = f D ( ϕ, α ) .
Revised: A to an initial state: G t {+
+} = f D ( ϕ, α ) .

Original: In Figure 1, ϕ is the 2D height field of the mesh, while α is thenon-controllable objects in the scene like the initial position of the fluid.
Revised: In Figure 1, ϕ is the 2D height field of the mesh, while α is the non-controllable objects in the scene like the initial position of the fluid.

Original:  The reward function f
Revised: Maximizing reward The reward function f

Original: R : G × Θ R → R maps the final state of a length- Ktrajectory ( G t K or ˜ G t K ) and reward function parameters θ R ∈ Θ R to a scalar value.
Revised: R : G × Θ R → R maps the final state of a length- K trajectory ( G t K or ˜ G t K ) and reward function parameters θ R ∈ Θ R to a scalar value.

Original: In our runningexample, the reward function is defined as the Gaussian likelihood of each fluid particle under theclosest “pool”, averaged across particles (Figure 1a).
Revised: In our running example, the reward function is defined as the Gaussian likelihood of each fluid particle under the closest “pool”, averaged across particles (Figure 1a).

Original: Reward function parameters θ R are the meansand standard deviations of these pools.
Revised: Reward function parameters θ R are the means and standard deviations of these pools.

Original: We define the objective under the ground-truth simulator f S as J S ( ϕ ) :
Revised: We define the objective under the ground-truth simulator f S as J S ( ϕ ) :

Original: = f R ( f ( K ) S ( f D ( ϕ, α )); θ R ) ,where f ( K ) S indicates K applications of the simulator.
Revised: = f R ( f ( K ) S ( f D ( ϕ, α )); θ R ) , where f ( K ) S indicates K applications of the simulator.

Original: We want to find the design parameters thatmaximize J S , i.e. ϕ ∗ =
Revised: We want to find the design parameters that maximize J S , i.e. ϕ ∗ =

Original: argmax ϕ
Revised: argmax ϕ

Original: J S ( ϕ ) .
Revised: J S ( ϕ ) .

Original: We can approximate this optimization using a learnedsimulation model by maximizing J M ( ϕ ) :
Revised: We can approximate this optimization using a learned simulation model by maximizing J M ( ϕ ) :

Original: = f R ( f ( K ) M
Revised: = f R ( f ( K ) M

Original: ( f D ( ϕ, α )); θ R ) .
Revised: ( f D ( ϕ, α )); θ R ) .

Original: Crucially, multiple rewardfunctions can be optimized with the same model f M .
Revised: Crucially, multiple reward functions can be optimized with the same model f M .

Original: Optimizers Optimal design parameters ϕ ∗ can be found using any generic optimization technique.
Revised: Optimizers Optimal design parameters ϕ ∗ can be found using any generic optimization technique.

Original: Given the differentiability of the learned simulator f M , we are particularly interested in evaluatinggradient-based optimization, which requires fewer function evaluations and scales better to largedesign spaces than sampling-based techniques [10].
Revised: Given the differentiability of the learned simulator f M , we are particularly interested in evaluating gradient-based optimization, which requires fewer function evaluations and scales better to large design spaces than sampling-based techniques [10].

Original: We use the Adam optimizer [46] to find ϕ ∗by computing the gradient ∇ ϕ J M ( ϕ ) .
Revised: We use the Adam optimizer [46] to find ϕ ∗ by computing the gradient ∇ ϕ J M ( ϕ ) .

Original: This involves backpropagating gradients through the rewardfunction f R ,
Revised: This involves backpropagating gradients through the reward function f R ,

Original: length- K rollout produced by f ( K ) M , and design function f D .
Revised: length- K rollout produced by f ( K ) M , and design function f D .

Original: As a baseline, we consider the cross-entropy method (CEM) [61], a gradient-free sampling-basedtechnique that is popular in model-based control [20, 73].
Revised: As a baseline, we consider the cross-entropy method (CEM) [61], a gradient-free sampling-based technique that is popular in model-based control [20, 73].

Original: CEM can be used with any simulatorand works by sampling a population of candidates for ϕ and evolving them to maximize the reward.
Revised: CEM can be used with any simulator and works by sampling a population of candidates for ϕ and evolving them to maximize the reward.

Original: However, CEM requires multiple evaluations of f M or f S per optimizer iteration (depending on thepopulation size), whereas Adam requires only a single evaluation per iteration.
Revised: However, CEM requires multiple evaluations of f M or f S per optimizer iteration (depending on the population size), whereas Adam requires only a single evaluation per iteration.

Revised: We additionally consider covariance matrix adaptation (CMA-ES) [38] and Bayesian Optimization (BO) [68] in

Revised: Section D.2 as two additional gradient-free optimizers.

Original: Across our design tasks (Section 4) we compared: gradient descent with Adam on the learnedsimulator model ( GD-M ), CEM with the learned simulator model ( CEM-M ), and CEM with theground-truth simulator ( CEM-S ).
Revised: Across our design tasks (Section 4) we compared: gradient descent with Adam on the learned simulator model ( GD-M ), CEM with the learned simulator model ( CEM-M ), and CEM with the [Figure] ground-truth simulator ( CEM-S ).

Original: We compare to further optimizer baselines, such as Bayesian Optimization [68] and CMA-ES [38] in the Appendix (Section D.2).
Revised: We compare to further optimizer baselines, such as Bayesian Optimization [68] and CMA-ES [38] in the Appendix (Section D.2).

Original: In all tasks, f S is non-differentiable,preventing a comparison to GD-S.
Revised: In all tasks, f S is non-differentiable, preventing a comparison to GD-S.

Original: However, in the special case of A IRFOIL , we compare to DAFoam[40], a specialized solver which computes gradients with the adjoint method.
Revised: However, in the special case of A IRFOIL , we compare to DAFoam [40], a specialized solver which computes gradients with the adjoint method.

Original: Evaluation Unless otherwise noted, we always evaluate the quality of an optimized design ϕ ∗using the ground-truth objective J S ( ϕ ∗ ) , regardless of whether ϕ ∗ was found using the learned model[Figure] f M (as in CEM-M and GD-M) or the ground-truth simulator f S (as in CEM-S).
Revised: Evaluation Unless otherwise noted, we always evaluate the quality of an optimized design ϕ ∗ using the ground-truth objective J S ( ϕ ∗ ) , regardless of whether ϕ ∗ was found using the learned model f M (as in CEM-M and GD-M) or the ground-truth simulator f S (as in CEM-S).

Original: We also use rolloutsfrom f S to produce visualizations in the figures.
Revised: We also use rollouts from f S to produce visualizations in the figures.

Original:  Design Tasks
Revised:  Design Tasks

Original: We formulate a set of design tasks across three different physical domains with high-dimensionalstate spaces and complex dynamics.
Revised: We formulate a set of design tasks across three different physical domains with high-dimensional state spaces and complex dynamics.

Original: Each domain uses a different ground-truth simulator f S , whichis used to evaluate designs and pre-train the learned simulator model f M in a task-agnostic way.
Revised: Each domain uses a different ground-truth simulator f S , which is used to evaluate designs and pre-train the learned simulator model f M in a task-agnostic way.

Original: During optimization, all the design tasks on each domain share the same pre-trained model.
Revised: During optimization, all the design tasks on each domain share the same pre-trained model.

Original: 2D F LUID T OOLS
Revised: 2D F LUID T OOLS

Original: Inspired by existing 2D physical reasoning benchmarks [1, 5], these tasksinvolve creating one or more 2D “tool” shapes to direct fluid into a particular goal region (Figure 2,
Revised: Inspired by existing 2D physical reasoning benchmarks [1, 5], these tasks involve creating one or more 2D “tool” shapes to direct fluid into a particular goal region (Figure 2,

Original: Section C.1).
Revised: Section C.1).

Original: We consider three tasks with 4–48 design parameters where the goal is to guide thefluid such that each particle comes as close as possible to the center of a randomly-sampled yellowreward region.
Revised: We consider three tasks with 4–48 design parameters where the goal is to guide the fluid such that each particle comes as close as possible to the center of a randomly-sampled yellow reward region.

Original: In Contain , the joint angles ϕ joints of a multi-segment tool must be optimized to catchthe fluid by creating cup- or spoon-like shapes.
Revised: In Contain , the joint angles ϕ joints of a multi-segment tool must be optimized to catch the fluid by creating cup- or spoon-like shapes.

Original: In Ramp , the joint angles ϕ joints of a multi-segmenttool must be optimized to guide the fluid to a distant location.
Revised: In Ramp , the joint angles ϕ joints of a multi-segment tool must be optimized to guide the fluid to a distant location.

Original: In Maze , the rotation ϕ rot of multipletools must be optimized to funnel the fluid to the target location.
Revised: In Maze , the rotation ϕ rot of multiple tools must be optimized to funnel the fluid to the target location.

Original: Fluid dynamics are representedusing 10 2 – 10 3 particles, and unrolled for up to 300 time steps.
Revised: Fluid dynamics are represented using 10 2 – 10 3 particles, and unrolled for up to 300 time steps.

Original: The learned model f M is trained on the 2D WaterRamps dataset released in [63] which uses thesolver in [43] to generate trajectories.
Revised: The learned model f M is trained on the 2D WaterRamps dataset released in [63] which uses the solver in [43] to generate trajectories.

Original: The dataset contains scenes with 1–4 straight line segments; nocurved lines or large numbers of obstacles are shown.
Revised: The dataset contains scenes with 1–4 straight line segments; no [Figure] curved lines or large numbers of obstacles are shown.

Original: Therefore, designs which solve 2D F LUID
Revised: Therefore, designs which solve 2D F LUID

Original: T OOLS tasks are necessarily out of distribution for the learned model.
Revised: T OOLS tasks are necessarily out of distribution for the learned model.

Original: 3D W ATERCOURSE
Revised: 3D W ATERCOURSE

Original: To evaluate higher-dimensional inverse design, we created a “landscaping”task that requires optimizing a 3D surface to guide fluids into different areas of an environment(Figure 3, Section C.2).
Revised: To evaluate higher-dimensional inverse design we created a “landscaping” task that requires optimizing a 3D surface to guide fluids into different areas of an environment (Figure 3, Section C.2).

Original: Specifically, water flows out of a pipe and onto an obstacle parameterized byϕ map , a 25 × 25 heightmap (625 design parameters).
Revised: Specifically, water flows out of a pipe and onto an obstacle parameterized by ϕ map , a 25 × 25 heightmap (625 design parameters).

Original: In Direction , ϕ map is optimized to redirect the[Figure] fluid stream towards a specified direction.
Revised: In Direction , ϕ map is optimized to redirect the fluid stream towards a specified direction.

Original: In 2 Pools and 3 Pools , ϕ map is optimized to split the fluidstream such that particles hitting the floor land as close as possible to one of two or three specifiedpools.
Revised: In 2 Pools and 3 Pools , ϕ map is optimized to split the fluid stream such that particles hitting the floor land as close as possible to one of two or three specified pools.

Original: Each task has multiple variants, such as different target directions in Direction .
Revised: Each task has multiple variants, such as different target directions in Direction .

Original: The simulationis unrolled for 50 time steps, and contains up to 2048 particles.
Revised: The simulation is unrolled for 50 time steps, and contains up to 2048 particles.

Original: The learned model f M is trained ondata generated from the simulator in [8].
Revised: The learned model f M is trained on data generated from the simulator in [8].

Original: A IRFOIL
Revised: A IRFOIL

Original: Shape optimization in aerodynamics is one area where gradient-based optimization isroutinely applied using traditional simulators [13].
Revised: Shape optimization in aerodynamics is one area where gradient-based optimization is routinely applied using traditional simulators [13].

Original: Here, we consider the well-studied task of dragoptimization of a 2D airfoil profile (Figure 5, Section C.3).
Revised: Here, we consider the well-studied task of drag optimization of a 2D airfoil profile (Figure 5, Section C.3).

Original: In this task, a wing is defined usinga curve on a 2D mesh, which can be deformed using a set of 10 control points ϕ ctrl .
Revised: In this task, a wing is defined using a curve on a 2D mesh, which can be deformed using a set of 10 control points ϕ ctrl .

Original: The task isformulated as optimizing the wing shape to minimize drag under certain constraints such as constantlift and bounds on the shape to prevent degenerate (e.g. infinitely thin) configurations.
Revised: The task is formulated as optimizing the wing shape to minimize drag under certain constraints such as constant lift and bounds on the shape to prevent degenerate (e.g. infinitely thin) configurations.

Original: Lift and dragcoefficients are computed by running an aerodynamics simulation on a 4158 node mesh.
Revised: Lift and drag coefficients are computed by running an aerodynamics simulation on a 4158 node mesh.

Original: The learnedmodel f M is trained on data generated with the OpenFOAM solver [55].
Revised: The learned model f M is trained on data generated with the OpenFOAM solver [55], which pairs wing shapes with targets of RANS pressure and Reynolds stress fields discretized on a 4158 node mesh. Note that unlike the fluids domains, this simulation is steady-state (ie. a rollout length of 1).

Original: Model learning and optimization While each domain has a different state space structure and usesa different ground-truth simulator f S , the learned simulators f M all share the same architecture (withidentical hyperparameters in 2D F LUID T OOLS and 3D W ATERCOURSE , and only minor variationsof the hyperparameters for A IRFOIL due to it being a steady state simulation; see Appendix B).
Revised: Model learning and optimization While each domain has a different state space structure and uses a different ground-truth simulator f S , the learned simulators f M all share the same architecture (with identical hyperparameters in 2D F LUID T OOLS and 3D W ATERCOURSE , and only minor variations of the hyperparameters for A IRFOIL due to it being a steady state simulation; see Appendix B).

Original: Themodels are trained for next-step prediction on task-independent datasets (random perturbations ofthe design space for A IRFOIL and 3D W ATERCOURSE , and an open-source, qualitatively distinctdataset for 2D F LUID T OOLS ), and are unrolled for up to 300 time steps during design optimizationwithout further fine-tuning (see Section 3 for details).
Revised: The models are trained for next-step prediction on task-independent datasets (random perturbations of the design space for A IRFOIL and 3D W ATERCOURSE , and an open-source, qualitatively distinct dataset for 2D F LUID T OOLS ), and are unrolled for up to 300 time steps during design optimization without further fine-tuning (see Section 3 for details).

Original: Importantly, all the pre-trained models areshared across the multiple design tasks for each domain.
Revised: Importantly, all the pre-trained models are shared across the multiple design tasks for each domain.

Original:  Results
Revised:  Results

Original: Our results show that learned simulators can be used to effectively optimize various designs despitesignificant domain shift and long rollout lengths.
Revised: Our results show that learned simulators can be used to effectively optimize various designs despite significant domain shift and long rollout lengths.

Original: The same underlying model architecture is used foreach domain, highlighting the generality of learned simulators for design.
Revised: The same underlying model architecture is used for each domain, highlighting the generality of learned simulators our approach are available at: https://sites.google.com/view/optimizing-designs .

Original: Examples of designs foundwith our approach are available at: https://sites.google.com/view/optimizing-designs .
Revised: Example code can be found at https://github.com/deepmind/deepmind-research/inverse_ design/ .

Original: Performance is always evaluated using the ground-truth simulator f S (see Section 3).
Revised: Performance is always evaluated using the ground-truth simulator f S (see Section 3).

Original: Here we discussthese results, and compare the capabilities of gradient descent with learned simulators over classicalsimulators and sampling-based optimization techniques.
Revised: Here we discuss these results, and compare the capabilities of gradient descent with learned simulators over classical simulators and sampling-based optimization techniques.

Original: Overall results We first asked whether a learned simulator combined with gradient descent (GD-M)could produce good-quality designs at all.
Revised: Overall results We first asked whether a learned simulator combined with gradient descent (GD-M) could produce good-quality designs at all.

Original: This approach might fail in various ways: accumulatingmodel error, vanishing or exploding gradients [9], or domain shift [36, 71].
Revised: This approach might fail in various ways: accumulating model error, vanishing or exploding gradients [9], or domain shift [36, 71].

Original: However, as the followingresults show, GD-M produced high-quality designs across all three domains.
Revised: However, as the following results show, GD-M produced high-quality designs across all three domains.

Original: Figure 2a shows qualitative results for 2D F LUID T OOLS (Section 4), where our approach (GD-M)produces intuitive, functional designs to contain ( Contain ), transport ( Ramp ), or funnel ( Maze ) thefluid to a target location.
Revised: Figure 2a shows qualitative results for 2D F LUID T OOLS (Section 4), where our approach (GD-M) produces intuitive, functional designs to contain ( Contain ), transport ( Ramp ), or funnel ( Maze ) the fluid to a target location.

Original: On average, GD-M outperforms CEM-M by 16.1–118.9%, indicating asubstantial benefit of gradient-based optimization.
Revised: On average, GD-M outperforms CEM-M by 16.1–118.9%, indicating a substantial benefit of gradient-based optimization.

Original: GD-M also outperforms CEM-S by 3.9–37.5%,despite using a learned simulator rather than the ground-truth.
Revised: GD-M also outperforms CEM-S by 3.9–37.5%, despite using a learned simulator rather than the ground-truth.

Original: However, these design spaces are stillrelatively small (between 16 and 36 dimensions).
Revised: However, these design spaces are still relatively small (between 16 and 36 dimensions).

Original: In 3D W ATERCOURSE (Section 4), we substantiallyincrease the dimensionality to a 625-dimensional landscape.
Revised: In 3D W ATERCOURSE (Section 4), we substantially increase the dimensionality to a 625-dimensional landscape.

Original: Here, GD-M produces robust designs,creating ridges to re-route water in particular directions or valleys to direct water into pools (Figure 3ac).
Revised: Here, GD-M produces robust designs, creating ridges to re-route water in particular directions or valleys to direct water into pools (Figure 3ac).

Original: By comparison, CEM-M cannot solve any of these tasks, with performance 30–85 × worse than
Revised: By comparison, CEM-M cannot solve any of these tasks, with performance 30–85 × worse than

Original: GD-M.
Revised: GD-M.

Original: In the Appendix (Section D.2)
Revised: In the Appendix (Section D.2)

Original: we demonstrate that these results also hold for other optimizerchoices (Bayesian Optimization, CMA-ES).
Revised: we demonstrate that these results also hold for other optimizer choices (Bayesian Optimization, CMA-ES).

Original: In A IRFOIL (Figure 5b), GD-M recovers the characteristic S-curve shape for a low-drag airfoil undera small angle of attack, and matches the design obtained with DAFoam, an adjoint aerodynamicssolver which computes close-to-optimal designs for this task.
Revised: In A IRFOIL (Figure 5b), GD-M recovers the characteristic S-curve shape for a low-drag airfoil under a small angle of attack, and matches the design obtained with DAFoam, an adjoint aerodynamics solver which computes close-to-optimal designs for this task.

Original: The design obtained with DAFoamyields a drag coefficient of 0.01902, while GD-M finds designs with drag between 0.01898–0.01919depending on ensemble size (see Section 5).
Revised: The design obtained with DAFoam yields a drag coefficient of 0.01902, while GD-M finds designs with drag between 0.01898–0. depending on ensemble size (see Section 5).

Original: Importantly, DAFoam’s solver and optimizer are highlyspecialized for the particular task of airfoil design, while our approach is more general-purpose,requiring only trajectory data for training and a generic gradient-based optimizer.
Revised: Importantly, DAFoam’s solver and optimizer are highly specialized for the particular task of airfoil design, while our approach is more general-purpose, requiring only trajectory data for training and a generic gradient-based optimizer.

Original: Model stability & gradient quality
Revised: Model stability & gradient quality

Original: We investigated accuracy over long timescales by measuringthe effect of rollout length on design quality in 2D F LUID T OOLS (Figure 4a and A.11).
Revised: We investigated accuracy over long timescales by measuring the effect of rollout length on design quality in 2D F LUID T OOLS (Figure 4a and A.11).

Original: Longer rollouts can in principle allow for higher reward in this task as they give the fluid time to settle; however,with learned models, they can also be unstable due to error accumulation [69, 72].
Revised: Longer rollouts can in principle allow for higher reward in this task as they give the fluid time to settle; however, with learned models, they can also be unstable due to error accumulation [69, 72].

Original: Nevertheless, wefind that the learned simulator does not seem to be severely impacted by this problem.
Revised: Nevertheless, we find that the learned simulator does not seem to be severely impacted by this problem.

Original: Specifically,the quality of designs found by GD-M increases up to 225 steps (Figure 4a), indicating that thelearned simulator’s accuracy and gradients remain stable for a surprisingly long time.
Revised: Specifically, the quality of designs found by GD-M increases up to 225 steps (Figure 4a), indicating that the learned simulator’s accuracy and gradients remain stable for a surprisingly long time.

Original: Across theepisode lengths evaluated, we find that GD-M outperforms not only CEM-M (by 18.1% on average)but also CEM-S (by 4.4% on average).
Revised: Across the episode lengths evaluated, we find that GD-M outperforms not only CEM-M (by 18.1% on average) but also CEM-S (by 4.4% on average).

Original: This indicates that the benefits of a having a learned modelthat supports better optimization techniques can outweigh the small errors incurred by long rollouts.
Revised: This indicates that the benefits of a having a learned model that supports better optimization techniques can outweigh the small errors incurred by long rollouts.

Original: The strong performance of GD-M on longer rollout lengths is noteworthy.
Revised: The strong performance of GD-M on longer rollout lengths is noteworthy.

Original: Gradients tend to degradewhen passed through chains of many model evaluations, and as a result, previous work generally onlyoptimizes gradients in small action spaces over just a few time-steps [51].We hypothesize that a key reason for the success of GD-M is the addition of noise in training themodel f M , which promotes stability on the forward pass and smoother gradients.
Revised: Gradients tend to degrade when passed through chains of many model evaluations, and as a result, previous work generally only optimizes gradients in small action spaces over just a few time-steps [51].

Original: We test this byconsidering a model variant trained without noise (Figure 4e, red).

Original: We observe much worse designquality compared to the full model, particularly for longer rollouts.

Original: Generalization Deep networks often struggle to generalize far from their training data [31].

Original: Thisposes a problem for design: to produce in-distribution training data, we would already need to knowwhat good designs look like, thus defeating the aim of wanting to find new designs.

Original: However, wefind that the GNN-based simulators generalize well in the setting studied here.

Original: As noted in Section 4,the learned simulator for 2D F LUID T OOLS was trained on a pre-existing, highly simplified dataset

Original: 5 Ensemble Size 1.
Revised: 5 Ensemble Size 1.

Original: V a l u e ( l o w e
Revised: V a l u e ( l o w e

Original: r
Revised: r

Original: i s b e tt e r ) 1e
Revised: i s b e tt e r ) 1e

Original: InitialDAFoamGD-M (c) Drag    5 Ensemble Size 0.
Revised: InitialDAFoamGD-M (c) Drag    5 Ensemble Size 0.

Original: R e w a r d ( h i g h e
Revised: R e w a r d ( h i g h e

Original: r
Revised: r

Original: i s b e tt e r ) 1e 3 (d) Rewardwhere only one to four straight line segments interact with a fluid (Appendix C).
Revised: i s b e tt e r ) 1e 3 (d) Reward

Original: However, it still cansupport the design tasks studied here, which involve highly articulated, curved obstacles ( Contain , Ramp ) or a larger number of obstacles ( Maze ), without further finetuning (Figure 2). This generalization capability emerges due to certain design choices of  thesimulator model f M . Specifically, the GNN operates on local neighborhoods of particles, and particle coordinates arerepresented in a relative manner, which encourages learning local rules that can transfer to verydifferent setups.
Revised: We hypothesize that a key reason for the success of GD-M is the addition of noise in training the model f M , which promotes stability on the forward pass and smoother gradients.

Original: We tested a model variant which breaks this property by encoding absolute , insteadof relative particles coordinates (Figure 4e, purple).
Revised: We test this by considering a model variant trained without noise (Figure 4e, red).

Original: We find that this variant can no longer robustlysimulate out-of-domain designs, leading to worse performance at any rollout length. Using a learned simulator trained in a relatively simple environment has another unexpected advantage. In rare cases, classical simulators suffer from degenerate behavior around certain edge cases. Forexample, with the classical simulator for 2D F LUID T OOLS , particles can get stuck in between jointsegments, especially when there are a large number of parts or joints (Figure A.10).
Revised: We observe much worse design quality compared to the full model, particularly for longer rollouts.

Original: However, sincethe learned simulator was trained on simpler data where these effects are unobserved, it picks uponly on the appropriate collision performance and not the unrealistic edge cases.

Original: Thus, the learnedsimulator produces more plausible rollouts than the classical simulator in these cases, and mighttherefore be a better candidate for producing designs that would transfer to the real world. Improving accuracy with ensembles In engineering tasks like A IRFOIL , simulators must beespecially accurate, as small differences in the predicted pressure field can cause large errors in liftand drag coefficients.
Revised: Improving accuracy with ensembles In engineering tasks like A IRFOIL , simulators must be especially accurate, as small differences in the predicted pressure field can cause large errors in lift and drag coefficients.

Original: While GD-M (without ensembling) can produce designs close to DAFoam’s,we notice a slightly rounder wing front (Figure 5b, bottom) causing a small increase in drag (0.01919versus 0.01902 in DAFoam).
Revised: While GD-M (without ensembling) produces designs close to DAFoam’s, we notice a slightly rounder wing front (Figure 5b, bottom) causing a small increase in drag (0. versus 0.01902 in DAFoam).

Original: To further improve performance, we implemented an ensemble of learned simulators trained onseparate splits of the training set.
Revised: To further improve performance, we implemented an ensemble of learned simulators trained on separate splits of the training set.

Original: Ensembles are a popular choice for training transition modelsfor use in control [20], as they can provide higher quality predictions and are more resistant todelusions—a particularly problematic issue for accuracy-sensitive domains such as airfoil design.
Revised: Ensembles are a popular choice for training transition models for use in control [20], as they can provide higher quality predictions and are more resistant to delusions—a particularly problematic issue for accuracy-sensitive domains such as airfoil design.

Original: During optimization, we make predictions with all models in the ensemble, each trained on a differentdata split, and average the gradients.
Revised: During optimization, we make predictions with all models in the ensemble, each trained on a different data split, and average the gradients.

Original: As shown in Figure 5b-c, larger ensembles yield designs withsignificantly lower drag ( β = − 5 .
Revised: As shown in Figure 5b-c, larger ensembles yield designs with significantly lower drag ( β = − 5 .

Original: 3 × 10 − 5 , p = 0 .
Revised: 3 × 10 − 5 , p = 0 .

Original: 0003 , where β is a linear regression coefficient)and higher overall reward ( β = 5 .
Revised: 0003 , where β is a linear regression coefficient) and higher overall reward ( β = 5 .

Original: 6 × 10 − 5 , p = 0 . 0001 ), and are able to produce designs veryclose to the solution found by DAFoam, with a drag coefficient of 0.01898 (size-5 ensemble).
Revised: 6 × 10 − 5 , p = 0 . 0001 ), and are able to produce designs very close to the solution found by DAFoam, with a drag coefficient of 0.01898 (size-5 ensemble).

Original: Thus,with ensembles, we are able to achieve performant designs with a general-purpose learned simulator,indicating that we can use learned models for design optimization in spaces traditionally reserved forspecialized solvers like DAFoam.
Revised: Thus we are able to achieve performant designs with a general-purpose learned simulator, indicating that we can use learned models for design optimization in spaces traditionally reserved for specialized solvers like DAFoam.

Original: Scalability to larger design spaces In larger design spaces, sampling-based optimization procedures quickly become intractable, especially with relatively slow simulators.
Revised: Scalability to larger design spaces In larger design spaces, sampling-based optimization procedures quickly become intractable, especially with relatively slow simulators.

Original: We hypothesized thatgradient descent with fast, learned simulators could overcome this issue, especially as the size ofthe design space is increased.
Revised: We hypothesized that gradient descent with fast, learned simulators could overcome this issue, especially as the size of the design space is increased.

Original: We therefore compared different optimizers on 2D F LUID T OOLSas a function of the dimensionality of the design space (the number of tool joints in Contain or thenumber of tools in Maze ) and on the higher-dimensional 3D W ATERCOURSE .
Revised: We therefore compared different optimizers on 2D F LUID T OOLS as a function of the dimensionality of the design space (the number of tool joints in Contain or the number of tools in Maze ) and on the higher-dimensional 3D W ATERCOURSE .

Original: For Contain (Figure 4b), the performance of GD-M increases with the number of joints as increasinglyfine grained solutions are made possible.
Revised: For Contain (Figure 4b), the performance of GD-M increases with the number of joints as increasingly fine grained solutions are made possible.

Original: In contrast, for both CEM-M and CEM-S, design qualitydeteriorates with more joints as high quality solutions become harder to find with random sampling.
Revised: In contrast, for both CEM-M and CEM-S, design quality deteriorates with more joints as high quality solutions become harder to find with random sampling.

Original: In the highest dimensional Contain task with 48 tool joints, GD-M outperforms CEM-M by 154 .
Revised: In the highest dimensional Contain task with 48 tool joints, GD-M outperforms CEM-M by 154 .

Original: 9%and CEM-S by 126 .
Revised: 9% and CEM-S by 126 .

Original: When CEM does find solutions (Figure 2b), they lack global coherenceand appear more jagged than solutions found with GD-M. Similarly, for Maze (Figure 4c), theperformance of GD-M is largely unaffected by the number of tools, while the performance of CEMM and CEM-S both degrade as the design space grows.
Revised: When CEM does find solutions (Figure 2b), they lack global coherence and appear more jagged than solutions found with GD-M. Similarly, for Maze (Figure 4c), the performance of GD-M is largely unaffected by the number of tools, while the performance of CEMM and CEM-S both degrade as the design space grows.

Original: For the highest dimensional Maze problemwith 36 joints, GD-M outperforms CEM-M by 207 .
Revised: For the highest dimensional Maze problem with 36 joints, GD-M outperforms CEM-M by 207 .

Original: 4% and CEM-S by 133 .
Revised: 4% and CEM-S by 133 .

Original: In the 625-dimensional 3D W ATERCOURSE task, CEM-M performs 30–85 × worse than GD-M(Figure 3) despite extensive hyperparameter tuning.
Revised: In the 625-dimensional 3D W ATERCOURSE task, CEM-M performs 30–85 × worse than GD-M (Figure 3) despite extensive hyperparameter tuning.

Original: This trend held across all tasks (Figure A.5a),and even persisted when using fewer control points in the design space (Figure A.7).
Revised: This trend held across all tasks (Figure A.5a), and even persisted when using fewer control points in the design space (Figure A.7).

Original: This is due notonly to 3D W ATERCOURSE ’s larger design space, but also because this problem requires a globallycoherent solution: modifying small areas independently is unlikely to have much effect on the globalmovement of the fluid. 
Revised: This is due not only to 3D W ATERCOURSE ’s larger design space, but also because this problem requires a globally coherent solution: small, independent changes have a negligible effect on the global movement of the fluid. [Figure]

Original: Model speed and sample efficiency Learned simulators can provide large speedups over traditionalsimulators in certain domains by learning to compensate for coarser sub-stepping and making optimaluse of hardware acceleration.
Revised: Model speed and sample efficiency Learned simulators can provide large speedups over traditional simulators in certain domains by learning to compensate for coarser sub-stepping and making optimal use of hardware acceleration.

Original: In A IRFOIL , although we use a very simple GD setup, our approachis able to find very similar designs as DAFoam’s specialized optimizer.
Revised: In A IRFOIL , although we use a very simple GD setup, our approach is able to find very similar designs as DAFoam’s specialized optimizer.

Original: Moreover, our approachrequires only 21s (single model) to 62s (size-5 ensemble) on a single A100 GPU, compared to 1021sfor DAFoam run on an 8-core workstation, despite requiring 10 × more optimization steps.
Revised: Moreover, our approach requires only 21s (single model) to 62s (size-5 ensemble) on a single A100 GPU, compared to 1021s for DAFoam run on an 8-core workstation, despite requiring 10 × more optimization steps.

Original: In 2D F LUID T OOLS , the ground-truth simulator runs at a similar speed to the learned model [see63], but is non-differentiable and therefore depends on more expensive gradient-free optimizationtechniques which require more function evaluations.
Revised: In 2D F LUID T OOLS , the ground-truth simulator runs at a similar speed to the learned model [see 63], but is non-differentiable and therefore depends on more expensive gradient-free optimization techniques which require more function evaluations.

Original: We use 20–40 function evaluations per optimization step of CEM, compared to a single evaluation with GD (which is about 3 × more costly,due to the gradient computation).
Revised: We use 20–40 function evaluations per optimization step of CEM, compared to a single evaluation with GD (which is about 3 × more costly, due to the gradient computation).

Original: Thus, GD with a differentiable learned model can be much moreefficient than using the ground truth simulator with a sampling-based method.
Revised: Thus, GD with a differentiable learned model can be much more efficient than using the ground truth simulator with a sampling-based method.

Revised: Generalization Deep networks often struggle to generalize far from their training data [31].

Revised: This poses a problem for design: to produce in-distribution training data, we would already need to know what good designs look like, thus defeating the aim of wanting to find new designs.

Revised: However, we find that the GNN-based simulators generalize well in the setting studied here.

Original: Discussion
Revised: As noted in Section 4, the learned simulator for 2D F LUID T OOLS was trained on a pre-existing, highly simplified dataset where only one to four straight line segments interact with a fluid (Appendix C). However, it still can support the design tasks studied here, which involve highly articulated, curved obstacles ( Contain , Ramp ) or a larger number of obstacles ( Maze ), without further finetuning (Figure 2). In Section D.6, we show similarly compelling generalization results for 3D W ATERCOURSE .

Revised: Without fine-tuning, we optimize the heightfields of two meshes along with their global rotations to redirect water.

Revised: These design tasks now consist of 1250 + 2 design dimensions, while also being further from the training distribution (only one, un-rotated heightfield was ever observed).

Revised: Despite these added challenges, the method still produces good designs with reward between 0.52 and 1.16.

Revised: This generalization capability emerges due to certain design choices of the simulator model f M .

Revised: Specifically, the GNN operates on local neighborhoods of particles, and particle coordinates are represented in a relative manner, which encourages learning local rules that can transfer to very different setups.

Revised: We tested a model variant which breaks this property by encoding absolute , instead of relative particles coordinates (Figure 4e, purple).

Revised: We find that this variant can no longer robustly simulate out-of-domain designs, leading to worse performance at any rollout length.

Revised: Using a learned simulator trained in a relatively simple environment has another unexpected advantage.

Revised: In rare cases, classical simulators suffer from degenerate behavior around certain edge cases.

Revised: For example, with the classical simulator for 2D F LUID T OOLS , particles can get stuck in between joint segments, especially when there are a large number of parts or joints (Figure A.10).

Revised: However, since the learned simulator was trained on simpler data where these effects are unobserved, it picks up only on the appropriate collision performance and not the unrealistic edge cases.

Revised: Thus, the learned simulator produces more plausible rollouts than the classical simulator in these cases, and might therefore be a better candidate for producing designs that would transfer to the real world.

Revised: There are also cases where we would expect the classical simulator to generalize better than the learned simulator, such as in dealing with fluids with viscosities outside the training data with different local dynamics.

Revised: Comparing to end-to-end surrogate models To further benchmark the generalization capabilities of our approach, we compare to an end-to-end surrogate model trained to map design parameters ( ϕ map ) directly to rewards for each of the d = 8 target angles of the Direction task in 3D W ATERCOURSE .

Revised: To train the end-to-end surrogate model, we use the same designs as those used to train the GNN dynamics model (Section C.2), but additionally paired with the rewards predicted by the ground truth simulator, so that D sur = { ( ϕ map , f R dir ( ϕ map , d ))

Revised: ( i ) } 1000 i =0 .

Revised: We use a ResNet-18 [39] as the surrogate model with a 25 × 25 dimensional input image corresponding to the heightfield mesh, and we train a separate model for each target direction.

Revised: We focus on 3D W ATERCOURSE for these experiments, as the training data and design problems contain meshes in the same domain ( 25 × 25 meshes).

Revised: For 2D F LUID T OOLS the training data and design spaces are different, making the end-to-end surrogate approach fundamentally under-determined.

Revised: The surrogate model could fit the training set and an in-distribution test set well (MSE= 0 .

Revised: 0070 ± 0 .

Revised: 0025 , Figure 6d).

Revised: However, when used for design optimization with a

Revised: GD optimizer, the surrogate moves into out of distribution regions of design space where it makes overly optimistic predictions (predicted= 1 .

Revised: 12 ± 0 .

Revised: 31 , actual= 0 .

Revised: 40 ± 0 .

Revised: MSE= 0 .

Revised: 615 ± 0 .

Revised: Figure 6b,c).

Revised: The surrogate model also severely underestimates the reward of high-scoring designs found using our method GD-M (predicted= 0 .

Revised: 33 ± 0 .

Revised: 06 , actual= 1 .

Revised: 21 ± 0 .

Revised: 18 , MSE= 0 .

Revised: 813 ± 0 .

Revised: Figure 6d).

Revised: Overall, this suggests that using task-agnostic, dynamics-based models for design can have appealing regularization properties which avoid issues of model delusion that are commonly seen with end-to-end surrogate approaches.

Revised:  Discussion

Original: We find that state-of-the-art learned, differentiable physics simulators can be used with gradient-basedoptimization to solve challenging inverse design problems.
Revised: We find that state-of-the-art learned, differentiable physics simulators can be used with gradient-based optimization to solve challenging inverse design problems.

Original: Across three domains and seven tasks,which involved designing landscapes and tools to control water flows or optimizing the shape of anairfoil, we demonstrated that gradient descent with pre-trained simulators can discover high-qualitydesigns that match or exceed the quality of those found using alternative methods.
Revised: Across three domains and seven tasks, which involved designing landscapes and tools to control water flows or optimizing the shape of an airfoil, we demonstrated that gradient descent with pre-trained simulators can discover high-quality designs that match or exceed the quality of those found using alternative methods.

Original: This approachsucceeds in a variety of interesting and surprising ways: it permits gradient backpropagation throughcomplex physical trajectories for hundreds of steps; scales to tasks with large design and state spaces(100s and 1000s of dimensions, respectively); and successfully generates designs which require thelearned simulator to generalize far beyond its training data.
Revised: This approach succeeds in a variety of interesting and surprising ways: it permits gradient backpropagation through complex physical trajectories for hundreds of steps; scales to tasks with large design and state spaces (100s and 1000s of dimensions, respectively); and successfully generates designs which require the learned simulator to generalize far beyond its training data.

Original: In the classic aerodynamics problem ofairfoil shape optimization, our approach produces a design comparable to that of a specialized solverusing only simple, general-purpose strategies like model ensembling.
Revised: In the classic aerodynamics problem of airfoil shape optimization, our approach produces a design comparable to that of a specialized solver using only simple, general-purpose strategies like model ensembling.

Original: While our results have exciting implications for inverse design, they also open up possibilities forexplaining everyday human behavior like tool invention—a longstanding puzzle in cognitive science[1, 56, 67].
Revised: While our results have exciting implications for inverse design, they also open up possibilities for explaining everyday human behavior like tool invention—a longstanding puzzle in cognitive science [1, 56, 67].

Original: With general-purpose learned simulators, we have the potential not just to create highlyspecialized tools in engineering domains, but also to model everyday tool creation, such as creating ahook from a pipe cleaner, building a blanket fort, or folding a paper boat.
Revised: With general-purpose learned simulators, we have the potential not just to create highly specialized tools in engineering domains, but also to model everyday tool creation, such as creating a hook from a pipe cleaner, building a blanket fort, or folding a paper boat.

Original: Our approach has limitations that should be visited in future work.
Revised: Our approach has limitations that should be visited in future work.

Original: Gradient descent is inappropriatefor design spaces with regions of zero gradients, such as in 2D F LUID T OOLS tasks where the fluidmay not always make contact with the tool (Figure A.9).
Revised: Gradient descent is inappropriate for design spaces with regions of zero gradients, such as in 2D F LUID T OOLS tasks where the fluid may not always make contact with the tool (Figure A.9).

Original: Many interesting design tasks also havevariably-sized or combinatorial design spaces that cannot easily be optimized with gradient descent,such as computer-aided design (CAD) approaches to 3D modeling.
Revised: Many interesting design tasks also have variably-sized or combinatorial design spaces that cannot easily be optimized with gradient descent, such as computer-aided design (CAD) approaches to 3D modeling.

Original: An exciting future direction willbe to integrate general-purpose learned simulators with hybrid optimization techniques such as thoseused in material science and robotics [17, 70].
Revised: An exciting future direction will be to integrate general-purpose learned simulators with hybrid optimization techniques such as those used in material science and robotics [17, 70].

Original: As learned simulators continue to improve, we couldalso use them to do even broader cross-domain, multi-physics design.
Revised: As learned simulators continue to improve, we could also use them to do even broader cross-domain, multi-physics design.

Original: While challenges remain, ourresults represent a promising step towards faster and more general-purpose inverse design.  
Revised: While challenges remain, our results represent a promising step towards faster and more general-purpose inverse design.  Acknowledgements

Revised: We thank Yulia Rubanova and Evan Shelhamer for their detailed comments on the manuscript, and

Revised: Yusuf Aytar, Jonathan Godwin, Luis Piloto, Remi Lam, and Meire Fortunato for helpful discussions.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See e.g. failure cases in Section D.3.1 and Discussion.
Revised: [Yes] See e.g. failure cases in Section D.3.1 and Discussion.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] We didn’t identify anything specific to this paper, the same considerations as for most work in ML apply (e.g. cost/impact of compute).
Revised: [No] We didn’t identify anything specific to this paper, the same considerations as for most work in ML apply (e.g. cost/impact of compute).

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to 
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: them? [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] The core model we use (MeshGraphNets) has an open-source implementation available, as do the solvers we used to for the task domains (cited in the paper).Detailed description of
Revised: [Yes] The core model we use (MeshGraphNets) has an open-source implementation available, as do the solvers we used to for the task domains (cited in the paper).

Original:  the task setup can be found in Appendix C. (b) Did you specify all the training details (e g., data splits, hyperparameters, how they 
Revised: Detailed description of the task setup can be found in Appendix C. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: were chosen)?

Original: [Yes] See Appendix A, Appendix B. (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: [Yes] See Appendix A, Appendix B. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: (d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [No] We didn’t calculate the totalamount; but model training was done on typical hardware (up to 4 TPUv3/v100 GPUs,see Appendix B).
Revised: [No] We didn’t calculate the total amount; but model training was done on typical hardware (up to 4 TPUv3/v100 GPUs, see Appendix B).

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We didn’t use anydata, but cite the model, and solvers used for the task domains.
Revised: [Yes] We didn’t use any data, but cite the model, and solvers used for the task domains.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [No] All of them are available under open-source licenses.
Revised: [No] All of them are available under open-source licenses.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] 
Revised: [No] (d) Did you discuss whether

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] 
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A](b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A](c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?[N/A]
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: A Optimizer hyperparameters

Original: Optimization hyperparameters were chosen to reflect good performance for each optimizer in eachdomain.

Original: We therefore performed sweeps for major hyperparameters of each optimizer for eachdomain, with those used for experiments in the paper shown in the table below.

Original: CEM maintains a population of samples and uses these to estimate the mean µ and standard deviationσ of a Gaussian distribution over design parameters.

Original: To optimize µ and σ , it takes the top performingfraction, deemed the “elite portion,” from the current step.

Original: The initial standard deviation of thisdistribution is given by “Initial σ ”, and the initial mean is set to 0.

Original: We found that for CEM, thepopulation sample size had a significant effect on overall optimization quality (Figure A.1).

Original: Due tocomputational considerations, we picked the smallest value for this hyperparameter that performedwithin 1 standard deviation of the optimal sample size.

Original: Both the elite portion and initial σ parameterswere chosen as the best performing values on a set of held-out random tasks for each domain.

Original: For GD, we only performed a hyperparameter sweep over the learning rate, which was the onlyparameter to significantly affect performance.

Original: For the 2D F LUID T OOLS tasks, we introducedgradient clipping to eliminate the effect of rare gradient spikes over the course of optimization.

Original: However, not using gradient clipping still produced qualitatively and quantitatively similar results.

Original: For additional Adam parameters, we used the default values for the exponential decay rates that trackthe first and second moment of past gradients of b 1 = 0 .

Original: 9 and b 2 = 0 .

Original: [Table]

Original: 3D

Original: [Table]

Original: ATERCOURSE

Original: A IRFOIL

Original: GD

Original: Contain

Original: Ramp

Original: Maze

Original: Direction

Original: Pools

Original: Learning rate 0.

Original: Momentum term b0.

Original: Momentum term b0.

Original: Gradient clip   — — —

Original: CEM

Original: Sampling size     —

Original: Elite portion 0.

Original: Initial µ     —

Original: Initial σ 0.

Original: Evolution smoothing 0.

Original: Optimization steps

Original: B Model architecture and training

Original: For each task domain, we train a GNN for next-step prediction of the system state.

Original: For the domainsconsidered in this paper, we unify the approaches of GNS [63] and M ESH G RAPH N ETS [57]: Inthe A IRFOIL domain, we encode/decode mesh nodes and mesh edges as a graph as described in theaerodynamics examples of M ESH G RAPH N ETS , while for particle-based fluids, edges are generatedbased on proximity as in GNS.

Original: In the case of 3D W ATERCOURSE , both particles (fluid) and a mesh(the designed obstacle) are present; hence, edges are generated based on proximity (for fluid-fluid andfluid-obstacle interaction) or from the landscape mesh.

Original: As the landscape does not have any internaldynamics, we did not find it necessary to distinguish between world- and mesh edges, and use asingle edge type.

Original: Once encoded as a graph, the core model and training procedure is largely identical between GNSand M ESH G RAPH N ETS , and we refer to the above papers for full details on architecture and modeltraining.

Original: Briefly, we use an Encode-Process-Decode GNN with 10 processor blocks.

Original: All edge andnode functions are 2-layer MLPs of width 128, with ReLu activation and LayerNorm after each MLPblock.

Original: The model is trained with Adam and a mini-batch size of 2, with training noise, for up to 10Msteps.

Original: We implemented this model in JAX [11].

Original: In addition to the different encoding procedures formesh vs. particle systems, the parameters for training noise and connectivity radius have to be setper-domain, to account for differences in particle size/mesh spacing.

Original: These details are described in

Original: Appendix C.

Original: The models were trained on 4 TPUv3 for 4-10 days.

Original: Gradient computation In our experiments, we pass gradients through long model rollouts of up to300 steps.

Original: As it is prohibitive to store all forward activations for the backwards pass, we use gradientcheckpointing [18] to store activations only at the beginning of each step of the trajectory during theforward pass, and recompute the intermediate activations for each step as needed when the backwardspass walks the trajectory in reverse.

Original: Gradient calculation using this method has roughly 3 times thetime cost of a pure forward simulation: forward dynamics have to be computed twice for each step,in addition to the computation of the backwards pass itself.

Original: C Task domains

Original: C. 2D F LUID T OOLS

Original: Tasks in 2D F LUID T OOLS are procedurally generated from templates specified in Table A.1.

Original: Thesimulation domain is a 2D box, with the lower left corner specified as [0 , 0] , and upper right cornerspecified as [1 , 1] .

Original: Fluid particles are initialized as a box of size Initial fluid box with bounding boxesgiven in format [ x min , y min , x max , y max ] .

Original: Certain task parameters were varied for ablation experimentsin Figure 4 (rollout length, # joints ( Contain ), # tools ( Maze )); Table A.1 contains default values usedunless otherwise specified.

Original: Design space A “tool” in this task domain is a 2D curve composed of several line segmentsconnected by joints.

Original: For a large number of joints, a tool can thus approximate a smooth curve(Figure A.2).

Original: Each task’s design space consists of the relative joint angles controlling the tool’sshape.

Original: We consider tasks with a single, multi-segment tool ( Contain , Ramp ) and a task with multiple,single-segment tools ( Maze ).

Original: For each tool, relative angles are calculated by moving from the anchorpoint on the left, along the tool segments to right, such that angle i =

Original: angle i − 1 + ϕ joints i for the i thjoint from the anchor.

Original: We also experimented with two additional design space parameterizations: (1)jointly optimizing the joint angles and

Original: a global position offset [ x, y ] for each tool, and (2) changingthe parameterization of angles to be absolute (such that angle i =

Original: ϕ joints i directly).

Original: We discuss theeffects of these alternate parameterizations in Section D.3.

Original: Simulation and objective Both fluids and tools are represented as particles with different types,and simulated with the learned model for 150 steps (with the exception of the ablation experimenton rollout length).

Original: Scenes consist of N = 100 . . .

Original: 1000 fluid particles.

Original: For ground-truth evaluationof the designs, we simulate particle dynamics with an MPM solver [43].

Original: Task reward is calculatedusing the Gaussian likelihood of the final particle positions after rollout ( u v from ˜ G t K ).

Original: That is, for atask with reward parameterized with mean µ and spherical covariance σ ( θ R = [ µ, σ ] ), the reward is

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: [Table]

Original: calculated as

Original: [Equation]

Original: Contain For this task, the center of the goal region µ is sampled uniformly from a rectangularreward region in the lower-middle section of the 1 × 1 simulation domain ( [0 . 4 , 0 . 6] ×

Original: Atool protruding to the right is initially placed below the fluid rectangle.

Original: By optimizing a single tool’srelative joint angles, successful solutions must “contain” the fluid in the region by creating a cup orspoon.

Original: Ramp The fluid and tool are initialized as in Contain , and µ is sampled from a region lower andfurther to the right than in Contain ( [0 . 8 , 1] × [0 , 0 . 2] ).

Original: By again optimizing a single tool’s relativejoint angles, successful solutions will create a “ramp” from the initial fluid position to the goallocation in the bottom right.

Original: Maze The goal is sampled from a long region near the bottom of the domain ( [0 . 1 , 0 . 9] × [0 . 1 , 0 . 2] ).

Original: By optimizing the rotation angles of a grid of rigid, linear tools, successful solutions will create adirected path from the top of the screen to the goal location at the bottom.

Original: Model training We trained the learned simulator on the W ATER R AMPS datasets released by

Original: Sanchez-Gonzalez et al.

Original: This dataset consists of 1000 trajectories featuring a single large blockof water falling on one to four randomized straight line segments (see image below for examples).

Original: Model architecture and hyperparameters are described in Appendix B, with a training noise scale of6 .

Original: 7 10 − 4 and connectivity radius of 0 . 015 .

Original: [Figure]

Original: . 3D W ATERCOURSE

Original: [Figure]

Original: Design space This domain has a design space ϕ map of 625 parameters, which determine the ycoordinate offset to nodes of a 25 × 25 square mesh centered at c = (0 . 3 , 0 . 5 , 0 . 5) in the simulationdomain.

Original: While we could directly mapping the parameters to coordinates, we use the design functiony i = γ H tanh( ϕ map i ) ( γ H = 0 . 3 for all tasks) to prevent trivial task solutions (i.e. obstacles whichtouch the floor).

Original: Simulation The simulation consists of an inflow pipe located at (-0.5, 1.0, 0.5) above the landscapewhich continually emits a stream of liquid, represented as particles.

Original: These particles are then redirectedby the designed landscape, and finally removed once they hit the floor at y = 0 .

Original: In our experimentswe observed up to 2084 particles present in the scene at one time.

Original: We unroll the learned simulationmodel for trajectories of 50 time steps, and store the final particle positions u v , as well as the positionsof removed particles that touched the floor at any point u Dv to be passed to the reward function.

Original: Ground truth simulations for evaluation are performed by running the same setup with an SPH solver.

Original: We note that SPH requires very small simulation time steps, and performs ≈ 10 4 internal steps for atrajectory of the same length.

Original: Direction In this task, we want to align the water stream with a given direction vector d .

Original: We canformalize this using the reward function

Original: [Equation]

Original: where d ⊥ is orthogonal to d .

Original: The first term aligns the direction of the particle relative to the domaincenter, and the second term concentrates the stream.

Original: The last term is a smoothness regularizeron the design landscape, which prefers smooth solutions ( γ R = 300 for both tasks).

Original: Absolutereward numbers for this task can be positive or negative, hence we report the normalized rewardf R dir − f initial R dir , i.e. an unchanged initial design corresponds to a zero reward, to make the scoreseasier to interpret.

Original: Rewards can be in 8 different directions, spaced between 0 and 180 deg .

Original: We collapse across directionsfor reporting reward means and confidence intervals for each optimizer.2 Pools and 3 Pools

Original: In these tasks, we define two and three pools, respectively, with center µ p onthe floor.

Original: For each particle which has hit the floor, we assign it to its closest pool ˆ µ p , and define thereward as the Gaussian probability under ˆ µ p , i.e.

Original: [Equation]

Original: with σ = 0 .

Original: 4 and a regularization term as above.

Original: To showcase different ways of splitting the water stream, we consider one positioning of the poolsfor the two pool case, and two for the three pool case.

Original: In the two pool case, pools are placedat [1 . 49 , − 0 .

Original: 35] and [1 . 49 , 1 . 35] .

Original: In the three pool case, pools are placed either at [1 . 6 , − 0 . 45] ,[1 . 85 , 0 . 5] , and [1 . 6 , 1 . 45] , or at [0 . 5 , − 0 .

Original: 5] , [1 . 7 , 0 . 5] , and [0 . 55 , 1 . 5] .

Original: These were selected to ensurethe task was solveable – pools directly beneath the landscape, or too far away from the landscape,would not be reachable even with dramatically warped surfaces.

Original: Model training We trained a model on next-step prediction of particle positions, on a dataset of1000 trajectories of water particles interacting with a randomized obstacle plane (random rotationsand sine-wave deformations of the planar obstacle surface).

Original: The data was generated using the SPHsimulator SPlisHSPlasH [8].

Original: The noise scale is set to 0 .

Original: 003 and a connectivity radius of 0 .

Original: 01 toaccount for the different particle radius of the 3D SPH simulation compared to 2D MPM.

Original: All otherarchitectural and hyperparameters are as described in Appendix B.

Original: C. A IRFOIL

Original: The airfoil optimization task is modeled similarly to the NACA0012 aerodynamic shape optimizationconfiguration for incompressible flow for the DAFoam solver (see details here), to make it easier tocompare design solutions to this solver.

Original: Design space

Original: The design space consists of the y -coordinate of 10 control points (see Figure 5a).

Original: Moving these control points deforms both the airfoil, and the simulation mesh surrounding it.

Original: Theairfoil shape is deformed using B-spline interpolation as described by Reid [58], and the mesh isdeformed using IDWarp [65].

Original: We thus define a design function G t 0 =

Original: f D ( ϕ ctrl , G α ) which takes aninitial, undeformed airfoil mesh (we use the standard NACA0012 airfoil), encoded as a graph G α , aswell as the control point position ϕ ctrl as input.

Original: It returns the graph of the deformed airfoil mesh G t 0to be passed to the simulator.

Original: We note that the coefficients for spline interpolation and mesh warpingcan be precomputed for a given initial mesh, making it easy to define a differentiable function to usefor design optimization.

Original: Simulation Given the initial mesh, as well as simulation parameters, the simulator or learned modelpredict the steady-state incompressible airflow around the wing, sampled on each of the 4158 nodeson the simulation mesh.

Original: The entire simulation domain and an example prediction of the pressure fieldare shown in Figure A.4a,b. For drag minimization we require predictions of the pressure field p , aswell as the effective Reynolds stress ρ eff at each mesh node, i.e. q v = ( p, ρ eff ) .

Original: Unlike the otherdomains in this paper, this is a single-step prediction task, and model rollouts are of length one.

Original: Forthis task, we consider an inflow speed of 0.1 mach, under an 5 . 1 ◦ angle of attack.

Original: Task objective

Original: The task reward is defined as f R :

Original: = − C D − γ L || C L − C L 0 || 2 − γ A a ( ϕ ctrl ) , i.e.we minimize the drag coefficient C D under soft constraints of unchanged lift C L and a wing areaa of 1-3 times the initial area.

Original: We use γ L = 10 , γ A = 1 , and a tanh nonlinearity to enforce thevolume inequality.

Original: Lift and drag can be computed from the simulation output p, ρ eff by integrationaround the airfoil, see e.g. [49].

Original: We report the normalized reward f R − f initial R such that the initial,undeformed wing design corresponds to a zero reward.

Original: Model training We trained a model to predict p, ρ eff on a dataset of 10000 randomized airfoilmeshes, simulated with OpenFoam [55].

Original: For training ensemble models, this dataset is split into 5non-overlapping blocks, and a separate model is trained on each section.

Original: Since this is a steady-stateprediction task, information needs to propagate further at each model evaluation.

Original: We therefore usetwice-repeated processor blocks with shared parameters, i.e. the model performs 20 message passingsteps, with 10 blocks of learnable parameters.

Original: We found that this increases accuracy in the one-stepsetup by being able to pass messages further across the mesh.

Original: Training noise is often cited for stabilityover long rollouts, but even in this one-step setting, training noise and data variation can be useful.

Original: Toincrease robustness to unseen wing configurations, we varied the grid resolution between 1000-10000nodes for each sample in the training set, and added training noise to the input mesh coordinates.

Original: Weuse a normal noise distribution with the scale of 1% of the average edge lengths surrounding the nodenoise is applied to.

Original: All other aspects of model architecture and training procedure are as described in

Original: Appendix B.

Original: D Further results

Original: [Figure]

Original: D.1 Model accuracy

Original: [Figure]

Original: In order for a learned simulator to be useful for design, it must be sufficiently accurate in the forwarddirection.

Original: We study this question directly for each of the domains (3D W ATERCOURSE , 2D F LUID

Original: T OOLS and A IRFOIL ) by examining the magnitude of the error between the model predictions ofreward for the discovered designs, and the ground truth reward for those designs.

Original: The results for eachdomain are shown in Figure A.5.

Original: Broadly, the learned model very successfully mimics the ground truth simulator in reward predictionacross all three domains.

Original: The accuracy for A IRFOIL is within a single standard deviation across allensemble sizes, while the predictions in 3D W ATERCOURSE match very closely for both the highperforming designs (GD-M) and low performing designs (CEM-M).

Original: However, we do notice some discrepancies in the predicted and ground truth reward for the 2D

Original: F LUID T OOLS domain, particularly the Maze task.

Original: As mentioned in the main text, the ground truthsolver sometimes produces unrealistic rollouts for this domain (see Figure A.10), with fluid particlesbecoming stuck between the different tools.

Original: Despite this issue, we find that the model is sufficientlysimilar to the ground truth to produce designs that still achieve high reward overall.

Original: Maze , where the ground-truth dynamics sometimes struggles to correctly simulate “sticky” bottlenecks (see Figure A.10).

Original: (c) Model predictions of drag (GD-M eval w/ M) are relatively close to the ground-truth simulator evaluation (GD-M), particularly for larger ensemble sizes.

Original: D.2 Other optimizers

Original: In our experiments in Section 5, we chose the cross-entropy method (CEM) as a popular optimizerrepresentative of model predictive control methods [26, 20], but there are several other possiblechoices of optimizer.

Original: In particular, here we provide results from optimizers that represent two alternative classes: bayesian optimization [68] (BO, which uses active sampling to explore the designspace more effectively), and CMA-ES [38] (which is an evolutionary search method and represents amore sophisticated sampling technique than CEM).

Original: We used the Bayesian Optimization implementation from [33] with their automatic hyperparametertuning that uses automatic relevance determination (ADR) [74] with a Matérn kernel and the expectedimprovement (EI) acquisition function.

Original: For CMA-ES we swept across the top k to keep, the smoothingparameter across iterations, the number of samples to use in each step, and the initial standarddeviation of the sampling distribution.

Original: These were fit on the direction template with a single directionacross 2 seeds per hyperparameter.

Original: The best performing hyperparameters for CMA-ES were to keepthe top 10 % of samples, no smoothing across optimization iterations, 40 samples per optimizationstep, and an initial standard deviation of 0.025.

Original: We show results for the 3D W ATERCOURSE Directiontemplate to showcase the scalability of both methods at the 3x3 design space and 25x25 design spaceresolutions.

Original: For a low-dimensional design space (3x3), all optimizers perform comparably.

Original: While CMA-ESand GD slightly outperform CEM and BO, the differences are minor.

Original: For larger design space sizes(25x25), GD-M significantly outperforms all alternative optimizers.

Original: Given enough optimizationtime, CMA-ES is able to achieve about three quarters of the reward obtained by GD, although ittakes roughly 50 × as long to do so, as, like CEM, it needs to sample a significantly larger numberof environment calls for each optimization step.

Original: It outperforms CEM because it represents a fullcovariance matrix across the top samples in the population, rather than a single scalar.

Original: D.3 Effects of design parameterization

Original: In this section, we study how different parameterization choices for the design space affect bothgradient-based and sampling-based optimizers.

Original: First, in the 3D W ATERCOURSE domain, we investigate a parameterization of the design spacethat uses interpolation to minimize the number of control points on the 2D heightfield (Figure A.7).

Original: Control points are placed evenly across the grid, and bi-linearly interpolated onto the 25 × 25 mesh.

Original: We vary the number of control points from 2 × 2 up to 14 × 14 , and find that while CEM-M performssimilarly to GD-M when very few control points are allowed, its performance quickly drops as morecontrol points are added.

Original: Second, in the 2D F LUID T OOLS domain, we investigate what happens when we change the designspace to use absolute joint angles rather than relative ones.

Original: When using relative joint angles, changesto joints near the tool’s pivot (left side) affect the global properties of the tool.

Original: We hypothesized thatthis could be selectively benefiting the sampling-based approaches, as this makes the effective designspace much lower dimensional.

Original: We therefore change the design space to be absolute, with a tool’sjoint angles calculated directly: angle i = ϕ

Original: i .

Original: As hypothesized, this change does dramatically decrease the performance of the sampling-basedtechnique (see Figure A.8).

Original: Perhaps more surprisingly, the gradient-based optimizer is almostcompletely unaffected by this reparameterization.

Original: While the qualitative solutions it finds differ (withtools now containing “kinks” to prevent the motion of the fluid rather than curves, Figure A.8 top),the overall reward achieved is similar.

Original: D.3.1 Failure modes of gradient descent

Original: [Figure]
Revised: [N/A]

Original: Other parameterizations of the design space can badly affect the performance of gradient-basedoptimizers.

Original: In particular, gradient-based optimizers suffer when there are regions of zero gradients.

Original: Inthe A IRFOIL and 3D W ATERCOURSE domains, this is not normally a problem, as the design alwaysinteracts with the physical system on which reward is being measured.

Original: But in the 2D F LUID T OOLSdomain, we can manipulate this.

Original: In particular, for this experiment we changed the design space for 2D F LUID T OOLS to includea global position offset [ x, y ] for the tool.

Original: Making this simple change often has no effect on thediscovered designs, but occasionally the gradient-based optimization procedure can move the toolsuch that it no longer interacts with the fluid (Figure A.9).

Original: Once the tool has been moved out of therange of the fluid, there is no longer any way to affect the reward, and therefore there is no gradientsignal to recover.

Original: To overcome this problem, future work would need to consider more sophisticatedhybrid optimization techniques [70].

Original: D.4 Failure modes of the MPM solver

Original: [Figure]

Original: One of the advantages of using a learned simulator over a classic simulator is learned simulatorscan be trained in regions of the state and action space that are known to exhibit regularized, smoothbehaviors.

Original: For example, as mentioned in Section D.1, the MPM solver [43] we use for evaluation in2D F LUID T OOLS shows surprising irregularities with “sticking” behavior when there are a largenumber of different tools.

Original: In the Maze task, this is particularly prevalent, as fluids often become stuckstochastically in some funnels but not others of similar sizes (Figure A.10).

Original: Since the learned simulator was trained on much simpler scenarios where this effect is not observed,it only learns the smooth behavior of the fluid’s movement, which makes the resulting trajectorieslook more realistic.

Original: This may enable better generalization to real world scenarios.

Original: D.4.1 Further designs found in 2D F LUID T OOLS

Original: In the figures below, we demonstrate the range of found solutions for different solvers in tasks in the2D F LUID T OOLS domain.

Original: D.5 End2End Surrogate Reward Models

Original: D.5. 3D W ATERCOURSE

Original: In this section we trained an end-to-end surrogate to map design parameters ( ϕ map ) directly torewards on the same designs as those contained in the training set for the GNN model (Section C.2),but extended with the rewards predicted by the ground truth simulator in each of the d = 8 targetangles of the Direction task, so that D sur = { ( ϕ map , f R dir ( ϕ map , d ))

Original: ( i ) } 1000 i =0 .

Original: We used a ResNet-18as the reward predictive model with a 25 × 25 dimensional input corresponding to the heightfieldmesh, and we trained a separate model for each target direction.

Original: The model could fit the training set, as well as an in-distribution test set very well (MSE= 0 .

Original: 0070 ±0 . 0025 ).

Original: However, the model performs predictably poorly out of distribution (Figure A.17):

Original: It (1)severely underestimates the reward of high-scoring designs found using GD-M (predicted= 0 .

Original: 33 ± 0 .

Original: 06 , actual= 1 .

Original: 21 ± 0 .

Original: 18 , MSE= 0 .

Original: 813 ± 0 . 344 ), and (2) obtains poor designs when using this model in an

Original: GD optimizer (predicted= 1 .

Original: 12 ± 0 .

Original: 31 , actual= 0 .

Original: 40 ± 0 .

Original: MSE= 0 .

Original: 615 ± 0 . 403 ).a

Original: c

Original: E n d - t o - E n d ( G D )

Original: G NN

Original: S i m u l a t o r (

Original: G D - M )

Original: [Figure]

Original: D.5. 2D F LUID T OOLS

Original: [Figure]

Original: Unlike the dataset used in the previous section, the W ATER R AMPS dataset does not share a designspace with the 2D F LUID T OOLS tasks (i.e. there are no joints, only a small number of line

Original: segments),so we were unable to train a reward model end-to-end as above on this data.

Original: Therefore, we insteadcomputed the reward for 1000 random designs (the same size as the training set for the GNN model)for the 2D F LUID T OOLS - Contain task using 4 up to 48 design parameters (Table A.2) to assess thedifficulty of creating a dataset which includes high rewards using random sampling.

Original: The maximum reward found decreased quickly for > 12 design parameters: i.e., for 48 joints, thehighest reward encountered is 3 × less than the average reward design obtained using the GD-M

Original: Number of joints

Original: Random design reward 12.

Original: (Figure 4-b) for the same dimensionality.

Original: Hence, a reward model trained on such data would notperform very well, as effective designs are highly out-of-distribution.

Original: D.6 Further tasks for 3D W ATERCOURSE

Original: To demonstrate the flexibility and further scaleability of the approach, we designed two further tasksfor the 3D W ATERCOURSE environment that require optimizing two meshes in order to manipulatethe flow of a fluid: D OUBLE M ESH R EROUTE and D OUBLE M ESH R AMP .

Original: Unlike in the original 3D

Original: W ATERCOURSE tasks, in these two tasks the global rotation θ g (in the z plane) of the meshes can beoptimized as well as the heightfields, leading to a design dimensionality of 1252 (each mesh has 626parameters to optimize).

Original: Both tasks are particularly out of distribution for this environment, sincethere was only ever one mesh (with a sinewave heightfield) without any vertical rotation seen duringtraining.

Original: In both cases, the positions of the meshes are fixed to ( − 0 .

Original: 25 , − 0 .

Original: 1 , 0 . 0) and (0 .

Original: 7 , − 0 .

Original: 2 , 0 . 0) respectively, allowing them to be offset in both the height and distance from the water emitter.

Original: D.6. D OUBLE M ESH R EROUTE task

Original: In this task, the reward region is placed at (0 . 5 , 0 . 0 , 0 . 5) , which is just underneath the edge of thefirst mesh.

Original: In order to move the fluid to this location, the fluid needs to be guided by the first meshinto a stream, and then rerouted by the second mesh to bounce back and underneath the first mesh.

Original: We define the reward as the Gaussian probability for each particle u v under ˆ µ with µ = (0 . 5 , 0 . 0 , 0 .

Original: 5)(a region at (0.5, 0.5) on the floor), i.e.

Original: [Equation]

Original: with σ = 0 . 1 and a regularization term γ R = 100 to provide smoothness, as in the original 3D

Original: W ATERCOURSE tasks.

Original: The mesh rotations are initialized to 20 ◦ and − 40 ◦ respectively, and then allowed to be optimizedfurther along with the heightfields (rotations end up at 35 .

Original: 6 ◦ and − 44 .

Original: 2 ◦ respectively after optimization).

Original: After 100 optimization steps, in this task the designs achieve a reward of 1 . 16 when evaluatedwith the ground truth simulator.

Original: To do so the meshes must cooperate to reroute the

Original: D.6. D OUBLE M ESH R AMP task

Original: In this task, the reward region is placed at (2 . 5 , 0 . 0 , 0 . 5) , which is past the edge of the second mesh.

Original: In order to move the fluid to this location, the fluid needs to be guided by the first mesh into a stream,and then rerouted by the second mesh to bounce back and underneath the first mesh.

Original: We define thereward as the Gaussian probability under ˆ µ with µ = (2 . 5 , 0 . 0 , 0 . 5) , i.e.f R ramp :

Original: = mean v ( N ( u v ; ˆ µ, σ )) − γ R mean( ∇ ϕ map ) with σ = 0 .

Original: 1 and a regularization term γ R = 100.

Original: The mesh rotations are initialized to 0 ◦ , and then allowed to be optimized further along withthe heightfields (rotations end up at − 4 .

Original: 4 ◦ and − 7 . 8 ◦ respectively after optimization).

Original: After 100optimization steps, in this task the designs achieve a reward of 0 .

Original: 52 when evaluated with the groundtruth simulator.