NTopo: Mesh-free Topology Optimization using Implicit Neural Representations

1 Introduction . Deep neural networks are starting to show their potential for solving partial differential equations ( PDEs ) in a variety of problem domains , including turbulent flow , heat transfer , elastodynamics , and many more [ 1 , 2 , 3 , 4 , 5 ] . Thanks to their smooth and analytically-differentiable nature , implicit neural representations with periodic activation functions are emerging as a particularly attractive and powerful option in this context [ 4 ] . In this work , we explore the potential of implicit neural representations for structural topology optimization—a challenging inverse elasticity problem with widespread application in many fields of engineering [ 6 ] . Topology optimization ( TO ) methods seek to find designs for physical structures that are as stiff as possible ( i.e . least compliant ) with respect to known boundary conditions and loading forces while adhering to a given material budget . While TO with mesh-based finite element analysis is a well-studied problem [ 7 ] , we argue that mesh-free methods provide unique opportunities for machine learning . We propose the first self-supervised , fully mesh-free method based on implicit neural representations for topology optimization . The core of our approach is formed by two neural networks : a displacement network representing force-equilibrium configurations that solve the forward problem , and a density network that learns optimal material distributions in the domain of interests . To leverage the power of these representations , we cast TO as a stochastic optimization problem using Monte Carlo sampling . Compared to conventional mesh-based TO , this setting 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . introduces new challenges that we must address . To account for the nonlinear nature of implicit neural representations , we introduce a convex density-space objective that guides the neural network towards desirable solutions . We furthermore introduce several concepts from FEM-based topology optimization methods into our learning-based Monte Carlo setting to stabilize the training process and to avoid poor local minima . We evaluate our method on a set of standard TO problems in two and three dimensions . Our results indicate that neural topology optimization with implicit representations is able to match the performance of state-of-the-art mesh-based solvers . To further explore the potential advantages of this approach over conventional methods , we show how our formulation enables self-supervised learning of continuous solution spaces for this challenging class of problems . 2 Related work . Neural Networks for Solving PDEs Deep neural networks have been widely used in different fields to provide solutions for partial differential equations for both forward simulation and inverse design problems [ 1 , 4 , 8 ] . In this context , PDEs can be solved either in their strong form [ 9 , 10 , 11 ] or variational form [ 12 , 13 ] . We refer to DeepXDE [ 5 ] for a detailed review . Explorations into using deep learning alongside conventional solvers for simulation have been conducted with the goal of accelerating computations [ 14 ] or learning the governing physics [ 15 , 16 , 17 , 18 , 19 , 20 ] . With their continuous and analytically-differentiable solution fields , neural implicit representations with periodic activation functions [ 4 ] offer a promising alternative to mesh-based finite element analysis . We leverage this new representation to solve high-dimensional inverse elasticity problems in a fully mesh-free manner . Differentiable Simulation for Machine Learning There is growing interest in differentiable simulation methods that enable physics-based supervision in learning frameworks [ 21 , 22 , 23 , 24 , 25 , 15 ] . Liang et al . [ 22 ] proposed a differentiable cloth simulation method for optimizing material properties , external forces , and control parameters . Hu et al . [ 21 ] targets reinforcement learning problems with applications in soft robotics . Geilinger et al . [ 24 ] proposed an analytically differentiable simulation framework that handles frictional contacts for both rigid and deformable bodies . To reduce numerical errors in a traditional solver Um et al . [ 25 ] leverage a differentiable fluid simulator inside the training loop . Similar to these existing methods , our approach relies on differentiable simulation at its core , but targets mesh-free , stochastic integration for elasticity problems . Neural Representations Using implicit neural representation for complex signals has been an ongoing topic of research in computer vision [ 26 , 27 ] , computer graphics [ 28 , 29 ] , and engineering [ 4 , 1 ] . PIFu [ 26 ] learns implicit representations of human bodies for monocular shape estimation . Sitzmann et al . [ 4 ] use MLPs with sinusoidal activation functions to represent signed distance fields in a fully-differentiable manner . Takikawa et al . [ 30 ] introduced an efficient neural representation that enables real-time rendering of high-quality surface reconstructions . Mildenhall et al . [ 28 ] describe a neural radiance field representation for novel view synthesis . We leverage the advantages of implicit neural representations , demonstrated in these previous works , to learn the high-dimensional solutions of topology optimization problems . Topology Optimization with Deep Learning Topology optimization methods aim to find an optimal material distribution of a design domain given a material budget , boundary conditions , and applied forces . Building on a large body of finite-element based methods [ 31 , 6 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 ] , recent efforts have explored the use of deep learning techniques in this context . One line of work leverages mesh-based simulation data and convolutional neural networks ( CNNs ) to accelerate the optimization process [ 45 , 46 , 47 , 48 , 49 , 50 , 51 ] . Perhaps closest to our work is the method by Hoyer et al . [ 52 ] , who reparameterize design variables with CNNs , but use mesh-based finite element analysis for simulation . While Hoyer et al . [ 52 ] map latent vectors to discrete grid densities , Chandrasekhar et al . [ 53 ] use multilayer perceptrons to learn a continuous mapping from spatial locations to density values . However , as we show in our comparisons , their choice of using ReLU activation functions leads to overly simplified solutions whose structural performance is not on par with results from conventional methods [ 54 , 55 , 56 ] . In addition , both methods use an explicit mesh for forward simulation and sensitivity analysis , whereas our method is entirely based on neural implicit representations . 3 Problem Statement and Overview . Given applied forces , boundary conditions , and a target material volume ratio V̂ , the goal of topology optimization is to find the material distribution that leads to the stiffest structure . This task can be formulated as a constrained bilevel minimization problem , Lcomp ( ρ ) = ∫ Ω e ( ρ , u , ω ) dω s. t. u ( ρ ) = arg min u′ Lsim ( u′ , ρ ) , ρ ( ω ) ∈ { 0 , 1 } , 1 |Ω| ∫ Ω ρ dω = V̂ , ( 1 ) where the loss Lcomp measures how compliant the material is , ρ is the material density field , ω runs over the domain Ω and |Ω| is the volume of Ω . The displacement u is a result of minimizing the simulation loss Lsim , ensuring the configuration is in force equilibrium . e is the pointwise compliance , which is equivalent to the internal energy up to a constant factor and is measuring how much the material is deformed under load ; see Section 4.1 for details . Although manufacturing typically demands binary material distributions , densities are often allowed to take on continuous values while convergence to binary solutions is encouraged . We follow the same strategy and parameterize densities ρ and displacements u using implicit neural representations , Φρ ( ω ; θ ) and Φu ( ω ; γ ) , respectively . By sampling a batch of locations ωb and ρb = ρ ( ωb ) , we compute an estimate of Lcomp using Monte Carlo integration , Lcomp ≈ |Ω|n ∑n i e ( ω b i ) . If u is a displacement in force equilibrium , we can compute the total gradient of the compliance loss with respect to the densities of the batch as s = dLcomp dρ = ∂Lcomp ∂ρ + ∂Lcomp ∂u du dρ = −∂Lcomp ∂ρ . ( 2 ) We will refer to this expression as the density-space gradient ; see the supplemental document for a detailed derivation . The density-space gradient indicates how the compliance loss changes w.r.t . the density values , assuming that the force equilibrium constraints remain satisfied . On this basis , we compute the total gradient of the compliance loss with respect to the neural network parameters as dLcompdθ = dLcomp dρ ∂ρ ∂θ . Using this gradient together with a penalty on the volume constraint would be one potential option for solving Equation ( 1 ) . In practice , however , we observed that this approach does not lead to satisfying behavior . We elaborate on this problem below . TO is a non-convex optimization problem , whose solutions depend on the optimization method [ 52 ] and the path density values take . Much research has been done on developing optimization strategies that converge to good local minima for FEM-based solvers . The density-space gradient is generally considered a good update direction for mesh-based approaches . However , when using a structural point of view ; see Section 5.4 . To analyze this unexpected behavior , we consider the change in densities induced by a step in the negative direction of the density-space gradient ∆ρ = −αdLcompdρ , where α is the learning rate . The corresponding first-order change in network parameters ∆θ is −αdLcompdρ ∂ρ ∂θ . How- ever , using this parameter change in the Taylor series expansion of the network output , we obtain ∆ρ′ = ρ ( θ + ∆θ ) − ρ ( θ ) = −αdLcomp dρ ∂ρ ∂θ ∂ρ ∂θ T +O ( α2 ) . ( 3 ) It is evident that , unless the Jacobian ∂ρ/∂θ of the neural network is a unitary matrix , the direction of density change ∆ρ′ is different from ∆ρ even as α→ 0 . To avoid converging to bad local minima , we seek to update the network parameters such that the network output changes along the density-space gradient . To this end , we define point-wise density targets ρ̂b indicating how the network output should change . We then minimize the convex loss function Ltopo ( θ ) = 1 nbn nb∑ b n∑ j ∥∥ρ ( ωbj ; θ ) − ρ̂ ( ωbj ) ∥∥2 . ( 4 ) While our density-space optimization strategy greatly improves results , we have still observed convergence to minima with undesirable artefacts . Drawing inspiration from mesh-based TO methods [ 58 ] , we solve this problem with a sensitivity filtering approach ( termed FT in Algorithm 1 ) . To further accelerate convergence , we also adapt the optimality criterion method [ 7 ] to our setting ( OC in Algorithm 1 ) . Our resulting neural topology optimization algorithm , which we dubbed NTopo , is summarized in Algorithm 1 and further explained in the following Section . Algorithm 1 NTopo : Neural topology optimization . 1 : Initialize γ ( −1 ) , θ ( 0 ) for Φρ , Φu ; Initialize two optimizers : optθ , optγ 2 : Run initial simulation : γ ( 0 ) ← arg minγ Lsim ( γ , θ ( 0 ) ) 3 : for nopt iterations do 4 : for nsim iterations do γ ( l+1 ) ← optγ .step ( γ ( l ) , ∂Lsim/∂γ ) end for 5 : for batch b = 1 , .. , nb do compute ρb , sb end for 6 : ŝ← FT ( ρ1 , ... , nb , s1 , ... , nb ) 7 : ρ̂← OC ( ρ1 , ... , nb , ŝ1 , ... , nb , V̂ ) 8 : for batch b = 1 , .. , nb do θ ( l+1 ) ← optθ.step ( θ ( l ) , ρ̂b , ∂Ltopo/∂θ ) end for 9 : end for 10 : return Φρ

Summary: The paper proposes a method for topology optimization (TO) in solid mechanics by modeling displacement and material densities as implicit neural networks. TO is a very important field in mechanics and it is exciting to see successful ML applications in the area. The paper shows that their method can solve for topologies under various load and displacement constraints in both 2D and 3D.