NTopo: Mesh-free Topology Optimization using Implicit Neural Representations
Authors: Jonas Zehnder
Yue Li
Stelian Coros
Bernhard Thomaszewski
Venue: NeurIPS 2021 Poster (2021)
Keywords: implicit representations
topology optimization
bilevel optimization
multilayer perceptrons
mesh-free

Abstract: Recent advances in implicit neural representations show great promise when it comes to generating numerical solutions to partial differential equations. Compared to conventional alternatives, such representations employ parameterized neural networks to define, in a mesh-free manner, signals that are highly-detailed, continuous, and fully differentiable. In this work, we present a novel machine learning approach for topology optimization---an important class of inverse problems with high-dimensional parameter spaces and highly nonlinear objective landscapes. To effectively leverage neural representations in the context of mesh-free topology optimization, we use multilayer perceptrons to parameterize both density and displacement fields. Our experiments indicate that our method is highly competitive for minimizing  structural compliance objectives, and it enables self-supervised learning of continuous solution spaces for topology optimization problems.

NTopo: Mesh-free Topology Optimization using Implicit Neural Representations
Jonas Zehnder
Yue Li Department of Computer Science ETH Zurich yue que.li@inf ethz. ch
Department of Computer Science and Operations Research Université de Montréal
j jonas www.der@umontreal.......
Stelian Coros
Bernhard Thomaszewski
Department of Computer Science ETH Zurich cerros@inf.eth....
Department of Computer Science
ETH Zurich bthomasz@ethz.co ch
Abstract
Recent advances in implicit neural representations show great promise when it comes to generating numerical solutions partial differential equations. Compared to conventional alternatives, such representations employ parameterized neural networks define, mesh-free manner, signals that are highly-deaalled, contin- uous, and fully differentiable. In this work, we present novel machine learning approach for topology optimization-an important class inverse problems with high-dimensional parameter spaces and highly nonlinear objective landscapes. To effectively leverage neural representations in the context of mesh-free topology optimization,
multilayer perceptrons to parameterize both density and dis- placement fields. Our experiments indicate that our method is highly competitive for minimizing structural compliance objectives, and it enables self supervuee learning of continuous solution spaces for topology optimization problems.
Introduction
Deep neural networks are starting to show their potential for solving partial differential equations (PDEs) in variety of problem domains, including turbulent flow, heat transfer, elastodynamics, and nature, implicit neural representations with periodic activation functions are emerging particularly attractive and powerful option this context 4 In this work, we explore the potential of implicit neural representations for structural topology optimization challenging inverse elasticity problem with widespread application in many fields of engineering [6].
many more I 1222 3 5]. Thanks to their smooth and
Topology optimization (TO) methods seek to find designs for physical structures that are as stiff as possible (i.e. least compliant) with respect to known boundary conditions and loading forces while adhering to a given material budget. While TO with mesh-based finite element analysis a well-studied problem [7], we argue that mesh free methods provide unique opportunitics for machine learning. We propose the first self-supervised, fully mesh-free method based implicit neural representations for topology optimization. The core of our approach is formed by two neural networks: displacement network representing force- enree-qquilibrim configurations that solve the forward problem, and a density network that learns optimal material distributions in the domain of interests. To leverage the power of these representations, cast TO stochastic optimization problem using Monte Carlo sampling. Compared to conventional mesh-based TO, this setting
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

introduces new challenges that we must address. To account for the nonlinear nature of implicit neural representations. we introduce a convex density-space objective that guides the neural network towards desirable solutions. We furthermore introduce several concepts from FEM- topology optimization methods into our learning-baser Monte Carlo setting to stabilize the training process and to avoid poor local minima.
We evaluate our method on of standard TO problems in two and three dimensions. Our results indicate that neural topology optimization with implicit representations is able to match the performance of state-of-the- mesh-basee solvers. To further explore the potential advantages of this approach over conventional methods, we show how our formulation enables self-supervised learning of continuous solution spaces for this challenging class of problems.
Dansity-Spacc Space Gradicnt
Target Densitis
min Setm (u(y))*
(u(T)a))
dicomm ap(n)
Filtering
Copo (p(#),p) B
(2(0,w)
Train 6x
Target Densily Computation
Train 4,
Figure 1: Neural topology optimization pipeline. We compute optimal material distributions by alternately training two neural networks: the displacement network 4 and the density network 4; mapping spatial coordinates to equilibrium displacements and optimal densities p, respectively. In each iteration, we first update Q, by minimizing the total potential energy the system. We then perform sensitivity analysis to compute density-space gradients which, after applying sensitivity filtering, give rise to target density fields p. Finally, we update & by minimizing the convex objective Liopo based mean squared error between current and target densities.
Related work
Neural Networks for Solving PDEs Deep neural networks have been widely used in different fields provide solutions for partial differential equations for both forward simulation and inverse design problems [1] 4 8 In this context, PDEs can be solved either in their strong form [9] [1] We refer DeepXDE [5] for a detailed review. Explorations into
or variational form using deep learning alongside conventional solvers for simulation have been conducted with the goal 18
of accelerating computations [14] or learning the governing physics [5] 16
17
19
20). With
their continuous and analyticall--difereniia differentiable solution fields, neural implicit representations with periodic activation functions
[ offer promising alternative to mesh-based finite element analysis.
We leverage this new representation to solve high-dimensional inverse elasticity problems in fully mesh-free manner.
Differentiable Simulation for Machine Learning There growing interest in differentiable simulation methods that enable physics-based supervision in learning frameworks [21 23 24
25 [5]. Liang et al. [22) proposed differentiable cloth simulation method for optimizing material properties, external forces, and control parameters. Hu al. (21) targets reinforcement learning problems with applications in soft robotics. Geilinger et al. (24) proposed an analytically differentiable simulation framework that handles frictional contacts for both rigid and deformable bodies. To reduce numerical errors a traditional solver Um et al. (25) leverage differentiable fluid simulator inside the training loop. Similar to these existing methods, our approach relies differentiable simulation at its core, but targets mesh-free, stochastic integration for elasticity problems.
Neural Representations Using implicit neural representation for complex signals has been an on- going topic of research in computer vision 226 27]. computer graphics [28 29]. and engineering [
PIFu 26 leams implicit representations of human bodies for monocular shape estimation. Sitzmann el use MLPs with sinusoidal activation functions to represent signed distance fields fully-liiferentiable manner. Takikawa el al. (30 introduced an efficient neural representation that enables real-time rendering of high-quality surface reconstructions. Mildenhall al. (2 describe neural radiance field representation for novel view synthesis. We leverage the advantages of implicit

---

neural representations, demonstrated in these previous works, to learn the high -ineesiiennional solutions of topology optimization problems.
Topology Optimization with Deep Learning Topology optimization methods aim to find an optimal material distribution of design domain given material budget, boundary conditions, and
applied forces. Building on a large body of finite-element based methods [31] 6 32
33
34
335 (36
37
42 43 (4), recent efforts have explored the use of deep learning techniques in this context. One line of work leverages mesh-based simulation data and convolutional neural networks (CNNs) to accelerate the optimization process
38 39
[45] 46 47 48 49 [50 [T]] Perhaps closest to our
work is the method by Hoyer er al. [52], who reparameterize design variables with CNNs. but use mesh-based finite element analysis for simulation. While Hoyer et [52] map latent vectors to discrete grid densities, Chandrasekhar et al. [53] use multilayer perceptrons to learn a continuous mapping from spatial locations to density values. However, as we show in our comparisons, their choice of using ReLU activation functions leads to overly simplified solutions whose structural performance is not on par with results from conventional methods [54 55 56]. In addition, both methods use explicit mesh for forward simulation and sensitivity analysis, whereas our method is entirely based on neural implicit representations.
Problem Statement and Overview
Given applied forces, boundary conditions, and target material volume ratio V, the goal of topology optimization is to find the material distribution that leads to the stiffest structure. This task can be formulated as a constrained bilevel minimization problem,
Lcomp (p) = J/1 c(p, u. dw
[ pdw = pdw v,
s. t. u()) = arg min (u's p),
p(w) E (0,1),
where the loss Leomp measures how compliant the material is, P is the material density field, runs over the domain 12 and [S2] is the volume of Q.. The displacement u is a result of minimizing the simulation loss Lsim. ensuring the configuration is in force equilibrium. compliance, which is equivalent to the internal energy up to a constant factor and is measuring how much the material is deformed under load; see Section 4.] for details. Although manufacturing typically demands binary material distributions, densities are often allowed to take on continuous values while convergence to binary solutions is encouraged. We follow the same strategy and parameterize densities and displacements using implicit neural representations. ( 0) and &u (w; respectively. By sampling a batch of locations w and p² p(w²), we compute an estimate of Ceomp using Monte Carlo integration, Lcomp [[]] Σ. e(w?)) If is displacement in force equilibrium, we can compute the total gradient of the compliance loss with respect to the densities of the batch
is the pointwise
dLcomp dp
ecoomp 0p
ecoopp du aCcomp du dp
(2)
0p
We will refer to this expression as the density-space gradient; see the supplemental document for detailed derivation. The density-space gradient indicates how the compliance loss changes w.r.t. the density values, assuming that the force equilibrium constraints remain satisfied.
On this basis, compute the total gradient of the compliance loss with respect to the neural network parameters as dCoomp dCcomp Op
dp 80
Using this gradient together with penalty the volume constraint would be one potential option for solving Equation (1) In practice, however, we observed that this approach does not lead to satisfying behavior. We elaborate this problem below.
TO a non-convex optimization problem, whose solutions depend on the optimization method [52] and the path density values take. Much research has been done developing optimization strate- gies that converge to good local minima for FEM-based solvers. The density-space gradient is generally considered good update direction for mesh-based approaches. However, when using
3

---

standard optimizers to update the parameters for our density network, the resulting change in densities is not at all aligned with the density- space gradient; see Figure 2 We tested both ADAM [57] and SGD with little difference in the results results-cventuallyy both approaches con- verge to local minima that are meaningless from structural point of view; see Section 5.4
0.4
04
05
80
0.
0.0
0.5
15 0.0
0.5
1.0
Figure 2: Failure of naive gradient descent. Left: negative density-space gradient on the domain for the beam example in the first iteration. Log-scale coloring is used to emphasize structure. Right: after one step of gradient descent (learnnng rate 10 10-5), "), the neural network output has lost all struc- ture from the density-space gradient.
To analyze this unexpected behavior, we con- sider the change in densities induced by step in the negative direction of the density-space gra- dient Ap
dCcomp where is the learning dp
rate. The corresponding first-order change in network parameters A -a dEcomp Op How- dp 8e ever, using this parameter change the Taylor series expansion of the network output, we obtaín
Ap' - p(0 A0) - p(0) dfcomp Op 0p T
+ +0a²)).
(3)
dp
00 00
is evident that, unless the Jacobian 0p/00 of the neural network a unitary matrix, the direction of density change Ap' is different from Ap even + 0.
To avoid converging to bad local minima, we seek update the network parameters such that the network output changes along the density-space gradient. To this end, we define point-wise density targets pe indicating how the network output should change. We then minimize the convex loss function
Coopo (0)
1
Σ (I((w):)) -(w)))) Σ
T2
(4)
T2
While density-space optimization strategy greatly improves results, have still observed
convergence to minima with undesirable artefacts. Drawing inspiration from mesh- based TO methods
[58] solve this problem with a sensitivity filtering approach (termed FT in Algorithm D To
further accelerate convergence, we also adapt the optimality criterion method 7 to our setting (OC
in Algorithm]] D). Our resulting neural topology optimization algorithm, which we dubbed NTopo, is summarized in Algorithm and further explained the following Section.
Algorithm 1 NTopo: Neural topology optimization.
1: Initialize 7(11), ((0) for p, Q.; Initialize two optimizers: opts, opt,
2: Run initial simulation: (0) arg min, (7,
3: for Nape iterations do
4: for Algim iterations do (1+1) + opt, (t) @Lsim end for
5: for batch = L,.., no do compute p², end for
6: $ +++F +TFT(pp (p'......i s!......s)
7: p+ SC-O0pp...... 88. sl.....r,,
8: for batch = 1, ns do ((+1) t- opt ste ,p,, OLropo (
end for
9: end for
10: return 4,
Neural Topology Optimization
We start the technical description of our algorithm with brief overview of the neural representation that we use. We then explain how to compute equilibrium configurations and how to update the density field. Finally, we introduce an extension of our method from individual solutions to entire solution spaces for a given continuous parameter range.

---

We use SIREN as neural representation, which is dense multilayer perceptron (MLP) with sinusoidal activation functions. SIREN-MLE with layers, / - hidden layers, and h neurons in each hidden layer is defined
(x) 1i o 0 ..... (w)()) + by, ΦI(x) = sin(Waa W, + b.)
(5)
where æ is the imput vector, y is the output vector, W; are weight matrices, b, are biases, and wo is frequency dependent factor. We
standard five-layer MLP with residual links and regularization. The weights of all layers are initialized as suggested by Sitzmann et al. [4]
4.1 Computing Static Equilibrium Solutions
We parametrize the displacement field using neural network 4, (,(wrr) 7) with network weights ?- To find the displacement field u in static equilibrium, we minimize the variational form of linear elasticity which is given by
min Lai (u(7)) min / 1 Ze(w): o(() o(u)) - u²= dw s.t. w/an un
(6)
where 8gp is the part of the boundary of the domain with prescribed displacement internal energy in linear elasticity is given through 1 a, where the tensor contraction ":" is defined as e a = Eij Eij In two dimensions we compute the stress tensor under plane stress assumption (E/ (1- ((1- D)E + trace(s)I) where is Poisson's 's ratio, the Young's 's modulus, I E R222 is the identity matrix and ε = (Vu + Vu²) /2 is the linear Cauchy strain. In 3D we use Hooke's law o A Atrace(e)I+ + 2ue where À Ev/((1 + 2) 2v)) and = E/(2(1 + v)). Following SIMP (37]. we parameterize the Young's 's modulus E using the density field as E(p) E1E Larger values for the exponent p together with the volume constraint encourage binary solutions.
The
Sampling evaluate the integral in Equation ( we resort to a Quasi-Monte Carlo sampling approach. We generate stratified samples on grid with x ny cells in 2D (and my M2 in 3D). We adjust Rz; Ry, ng to match the aspect ratio of the domain.
Enforcing Dirichlet Boundary Conditions By constructing function d that is zero the fixed boundary and an interpolator of the function up, we enforce the displacement field u(w) = P(w) (w: + (I o up) (w)()) to always satisfy the essential boundary conditions, thus turning the constrained problem into an unconstrained one [59]. We use simple boundaries our examples for which analytic functions d are readily available; see the supplemental document for more details.
4.2 Density Field Optimization
We reparameterize the density field using a neural network t which maps spatial locations to their corresponding density values. The bound constraints on the densities are enforced by applying scaled logistic sigmoid function to the network output, specifically p(w) = sigmoid 5 + (w)). The total volume constraint is satisfied by the optimality criterion method described below.
Moving Mean Squared Error (MMSE). Equation (4 can be interpreted as mean squared error
S / [Ip(w:)) |[
We minimize this loss using mini-batch gradient descent strategy, where we use every batch only once. We collect multiple batches of data from á before we update and p. specifically á changes once every outer iteration. For this reason, we refer this updating scheme as moving mean squared error in the following.
Sensitivity filtering FT. In conventional TO algorithms, filtering is an essential component for discovering desirable minima that avoid artefacts such as checkerboarding (33]. While our neural representation does not suffer from the same discretization limitations that give rise to checkerboard patterns, we have nevertheless observed convergence undesirable minima. Indeed, the neural
5

---

representation alone does not remove the inherent reason for such artefacts: TO is an underconstrained optimization problem with a high-dimensionall null-space. Isolating good solutions from this null- space requires additional priors, filters, or other regularizing information.
In order to address this problem, we propose continuous sensitivity filtering approach that, instead of using discrete approximations [58). is based on continuous convolutions. Following this strategy, we obtain filtered sensitivities as
s(w) = I [((W)))Ww + (w) Aw) dAw
(8)
max(e, ((w)) / (Aw) dAw
where is set to 10-3 and H is the kernel (Aw)) samples w' are distributed inside a grid, we can compute an approximation to the continuous filter as
(0, r
||w]])) with radius r. Since the
ΣKENs H(w wi (a p3) ERENs [[w) w
where the neighhorhood N is defined by cell sizes and radius r such that points inside the footprint of the kernel H are in the neighborhood N. Although this approximation is not unbiased estimator of Equation (8) it led to satisfying results in all our experiments.
Multi Batch-based Optimality Criterion Method OC We leverage the optimality criterion method [36] to compute density targets that automatically satisfy volume constraints, thus avoiding penalty functions or other constraint enforcement mechanisms. To this end, we extend the discrete, mesh-based formulation to the continuous Monte Carlo setting. One chooses a Lagrange multiplier such that the the volume constraint is satisfied after the variables have been updated. Since is computationally infeasible to compute the constraint exactly, we choose to satisfy the constraint in terms of its estimator using the collected batches. Additionally, we adopt a heuristic updating scheme very similar to the ones proposed by other authors [36], which leads to the following scheme: First a set of multiplicative updates B are computed, which then are applied to compute the target densities o² clamp(p' (B*)", max(0, p² m), min(1, '² + m))) where m limits the maximum movement of a specific target density and r a damping parameter. We used m 0.2 and rj = 0.5. The /> @v)) where À is found using binary search such that the estimated volume of the updated densities using Monte Carlo integration matches the desired mam Ex Σ Σー =V.
updated 8 are computed using 3
volume ratio across all batches
4.3 Continuous Solution Space
Apart from solving individual TO problems for fixed boundary conditions and material budgets, our method can be readily extended to learn entire spaces of optimal solutions, e.g., continuous range of material volume constraints [V*} or boundary conditions such as force locations. To this end, we seek to find a density function p(w, q) which yields the optimal density at any point w in the domain for any parameter vector q representing, e.g., material volume ratio == V in the target range Q = IV. V). In a supervised setting, a common approach is to first compute solutions corresponding to different parameters {q*} and then fit the neural network using a mean squared error. By contrast, our formulation invites a fully self-superviser approach based on modified moving mean squared error,
Q /
[lp(w, 0) p(w, g)l dwdq
(10)
We minimize this loss by sampling q random and then update the density network using the same method as described for the single target volume case.
5 Results
To analyze our method and evaluate its results, we start by comparing material distributions obtained with our approach on a set of 2D examples. We demonstrate the effectiveness of our method through comparisons to a otate-of-the-art FEM solver (Section 5.T). We then investigate the ability of our formulation to learn continuous spaces of solutions in fully self-supervised manner. Comparisons

---

to data-driven supervised approach indicate better efficiency for our method, suggesting that our approach opens new opportunities for design exploration engineering applications (Section (52)]. We then turn to TO problems with non-trivial boundary conditions and demonstrate generalization to 3D examples (Section 5.3) An ablation study justifies our choices using sensitivity filtering and casting the nonlinear topology objective into an MMSE form (Section 5.4). We further compare the impact of different activation functions and provide detailed descriptions of our learning settings (Section 5.5).
5.1 Comparisons with FEM Solutions
We demonstrate that results competitive those produced by a SIMP, reference FEM approach (36) for mesh-based topology optimization. As can be seen in Figure 3 results are qualitatively similar, but our method often finds more complex supporting structures that lead lower compliance values (see [.. To put these results in perspective, there is no topology optimization method fundamentally better than SIMP, despite decades of research. For fair comparisons, the compliance values of these structures are evaluated using the FEM solver and we consistently use fewer degrees of freedom (DoFs). The DoFs in these two methods are the number of network weights and the number of finite element cells, respectively. As can be seen, our method is more computationally expensive, but we would like to emphasize that the goal of our approach is not to outperform conventional solvers for single-solution cases, but rather find insight for a learning- and fully mesh-free approach to topology optimization that allows self-supervised learning of solution spaces. We refer to the supplemental material for further details of these examples.
Ours TEAN
Short Beam
Long Beam
Distributed
Bridge
Figure 3: 2D comparison on four example problems. Solutions produced by our method are qualitatively equivalent to corresponding FEM solutions. Force and Dirichlet boundary conditions are visualized in the top-row images.
Table I: Statistics of 2D comparisons. Our results achieve quantitatively lower compliance values (Comp.) than the reference solver (SIMP) for all examples.
FEM
OurS
EXP
СОмP:
DoFs
ITER.
TIME
COMP.
DoFs
ITER.
TIME
SHORTBEAM
1.173
30.000
122
34s
1.166 x 10
28,300
200
33 min
2.595 10-4
31,250
47
10
DISTRIBUTED
2.026 10¹
30,000
454
114s
2.012 x 10
28,300
400
66 min
BRINGR
4.441 10"
31,250
233
72
4.385 x 109
28,300
200
33 min
5.2 Learning Continuous Solution Spaces
Optimal designs for different volume constraints Here we demonstrate the capability of our method to learn continuous spaces of optimal density distributions for continuous range of material volume constraints. We minimize the objective defined in Equation (TO) for volume constraint samples drawn randomly from the range (30%, 70%]. To evaluate the accuracy of our learned solutions, we apply our single-volume algorithm for 11 discrete material volume constraints sampled uniformly across the target range. As can be seen in Figure [5] our solution space network does not compromise quality of individual designs. The mean and maximum errors in compliance and volume violation are 0.75%, 3.83%, 0.5% and 2.83%, respectively. We therefore conclude that our model successfully learned the continuous solution space. Furthermore, we argue that the level of accuracy is acceptable for design exploration in many engineering applications.
7

---

Different solution spaces To further analyze the behavior of our solution space approach, we conducted two additional experiments: the beam example with fixed volume but varying location for the applied forces and the bridge example with varying density constraint; see Figure 4 Both examples confirmed our initial observations, showing smoothly evolving topology and compliance values close the single-solution reference. For both examples, we sample 25 volume fractions/force locations during each iteration using stratified sampling. In addition to our single solution setup, we shuffle the density pairs randomly during the MMSE fitting step. The training takes 280 iterations in total, leading to 37.3 hours, Once trained, the inference enables real-time exploration of the solution space 0.011s/71.4FPP for 300 x 100 samples).
Figure 4: Different solution spaces. Here we show additional results varying the volume fraction constraints for the bridge example (first row) well as varying the applied force location for the cantilever beam (second row). See Figure for boundary conditions.
Comparison with supervised setting We further compare self- self-supervised training techniques with a supervised setting under the same computational budget. The cost of performing 1, 000 optimization steps (5.25h) with our larger solution space network is similar (but somewhat lower than) the cost of computing 11 solutions under single volume constraints. We select 11 volume constraint values uniformly and train the network from these solutions in a supervised fashion. As can be seen in Figure the network performs poorly for unseen data leading to infeasible designs and significant volume violations. On the contrary, our self-supervised approach leads physically valid material distributions.
5.3 Irregular BCs and 3D Results
Our formulation extends to more complex boundaries, which we illustrate on set of addi- tional examples. Figure shows multiple examples where densities are constrained circu-
-10-3
2.1
Nickods Siagl Sclatio CT Salatin
Melhod
Superiasl
22
20
1.8
122
1.0
54
Piscertage
Figure 5: Results and compliance comparisons between evaluating our solution space network at discrete volume locations and the reference solution produced by our method the single volume constraint case. As can be seen, the learned solution space does not compromise the quality of individual solutions.
Figure 6: Comparisons with supervised setting. Our self-supervised learning methods outper- forms its supervised counterparts with less com- putational cost. The target volume constraint is shown on the z-axis and the constraint viola- tion (in percentage) of resulting structures is given the y-axis.

---

Figure 7: The solutions for two 3D examples demonstrate the promise of our method in 3D. A 3D cantilever beam (left) and 3D bridge (right). The mesh has been generated using marching-cuber algorithm [60]
lar sub-domains and the Dirichlet boundary M&p is also circular in of the two examples. Our method shows promise in 3D, as demonstrated on the two examples shown in Figure 7
Due the symmetry of the configuration, apply symmetric boundary conditions to reduce the domain of interests half and quarter for the cantilever beam and bridge example respectively save computational cost and memory usage. Our method finds smooth solutions with various supporting features for the two tasks.
Figure 8: Curved boundaries: In the left example 3 holes have been put the design using constraints on the density field. On the right, the density field constrained to have hole in the middle.
5.4 Ablation Studies
Here we provide evidence for the necessity of using sensitivity filter, moving mean squared error loss during the optimization process, and the influence of different neural networks. In the first test, we do not not rely on the optimality criterion method compute the target density field nor do a soft penalty term to satisfy the volume constraint and update the neural network only once per iteration. the second test, adopt the proposed method without filtering. Comparing with our reference solutions at different iterations, the alternatives either converge significantly slower or arrive poor local minima with many artefacts, such disconnected struts rough boundaries. We further compare our network structure with ReLU-MLP as proposed in TOuNN (53). As can be seen from Figure I0] this approach fails to capture much of the geometric features that our method (and SIMP) produce, leading to more compliant (i.e., less optimal) designs (compliance: 0.001196). The Fourier feature network (29) leads to comparable results (compliance: 0.001172) and thus also a valid option for our method.
the mean squared error. Rather, add
150 iter 50 iee
No MMSEOCC
No Filtering
ReLU-MLP
Fourierttanh
SIREN
Figure 9: Ablation studies on the beam example. ss column: without moving mean squared error and optimality criterion-the density-space gradient is directly applied to the network, which is updated once per optimization iteration. 2nd columm: proposed method without filtering. 3rd column: proposed method using ReLU-MLP as the density network, SIREN is adopted as simulation network to obtain accurate equilibrium displacement. 4th column: Proposed method with Fourier feature network. 5th column: Proposed method with SIREN network. We verify that our moving mean squared error, filtering, and choice of activation function are all quintessential.
5.5 Training Details.
We use Adam [57] as our optimizer for both displacement and density networks and the learning rate of both is set to be 3- 10 for all experiments. We use 60 for the first layer and 60 neurons in each hidden layer 2D, and 180 hidden neurons 3D. For the solution space learning setup, we use 256 neurons in each hidden layer in the density network to represent the larger solution space. For all experiments, we initialize the output of the density network close to a uniform density distribution of the target volume constraint by initializing the weights of the last layer close to zero and adjusting

---

SIREN Fourier ReLU
Figure 10: We use different network architectures to parameterize both the density and displacement field. Since the ReLU-MLP fails to capture the high-frequency details, its solution for the forward problem is far from being accurate, resulting in meaningless structures for the inverse problems. Fourier feature and SIREN networks produced similar results, thus, suitable for our method.
the bias accordingly. We used Ej = 1,
50 and [Ma, myy [150, 50] in 2D and [ny, My, nz] [80, 40, 20] in 3D. The training iterations for the 3D examples are 100 with per iteration cost of 131.7s. All timings in the paper are reported on GeForce RTX 2060 SUPER graphics card.
Conclusion
We propose novel, fully mesh-free TO method using implicit neural representations, At the heart of our method lie two neural networks that represent force-equilibium configurations and optimal material distribution, respectively. Experiments demonstrate that our solutions qualitatively on par with the standard FEM method for structural compliance minimization problems, yet further enables self-supervised learning of solution spaces. The proposed method can handle irregular boundary conditions due to its mesh-frree nature and is applicable to 3D problems as well. As such we consider steppingstone towards solving many varieties of inverse design problems using neural networks.
Limitations and Future Works As we adopted the sigmoid function to enforce box constraints, naturally leads small gradients when approaching 0 or Although did not lead convergence issue for us in practice, better ways of enforeing box constraints in density space is an interesting avenue for future work. Like for other Monte Carlo-based methods, advanced sampling strategies, e.g., importance sampling can also be explored to speed up the optimization process.
Acknowledgements
This research was supported by the Discovery Accelerator Awards program of the Natural Sciences and Engineering Research Council of Canada (NSERC), the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (Grant No. 866480) and the Swiss National Science Foundation (Grant No. 200021 2200444..
10

---

References
[1] Oliver Hennigh, Susheela Narasimhan, Mohammad Amin Nabian, Akshay Subramaniam, Kaustubh Tangsali, Max Rietmann, Jose del Aguila Ferrandis, Wonmin Byeon, Zhiwei Fang, and Sanjay Choudhry. Nvidia simnet" {TM}: an ai-accelerated multi-physics simulation framework. arXiv preprint arXiv:2012.07933. 2020.
[2] Maziar Raissi, Paris Perdikaris, and George E. Karniadakis. Physics-informed neural networks:
A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686-7017 7076666777720 2019.
[3] Chengping Rao, Hao Sun, and Yang Liu, Physics informed deep learning for computational elastodynamics without labeled data. arXiv preprint arXiv:2006.0877, 2020.
[4] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Im- plicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems, 33, 2020.
[5] Lu Lu, Xuhui Meng. Zhiping Mao, and George E Karniadakis. Deepxde: A deep learning library for solving differential equations. arXiv preprint arXiv: 197.4:190. 2019.
[6] Martin Philip Bendsoe and Ole Sigmund. Topology optimization: theory, methods, and applica- tions. Springer Science & Business Media, 2013.
[7] Martin P Bendsøe and Ole Sigmund. Optimization of structural topology, shape, and material, volume 414. Springer, 1995.
[8] Maziar Raissi, Alireza Yazdani, and George Em Karniadakis. Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science, 36766481):1026-1000, 2020.
[9] MWMG Dissanayake and Nhan Phan-Thien. Neural-network-based approximations for solving partial differential equations. communications in Numerical Methods in Engineering, 10(3):195- 201, 1994.
[10] Isaac E Lagaris, Aristidis Likas, and Dimitrios Fotiadis. Artificial neural networks for solving ordinary and partial differential equations. IEEE transactions neural networks, 9())9877 1000, 1998.
[11] Justin Sirignano and Konstantinos Spiliopoulos. Dgm: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339-1366 2018.
[12] Juncai He, Lin Li, Jinchao Xu, and Chunyue Zheng. Relu deep neural networks and linear finite elements. arXiv preprint 1rXiv:1807.03973 2018.
[13] Weinan and Bing Yu. The deep ritz method: a deep saaring-basee numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1112, 2018.
[14] Tianju Xue, Alex Beatson, Sigrid Adriaenssens, and Ryan Adams. Amortized finite element analysis for fast pde-constrained optimization. In International Conference on Machine Learn- ing, pages
PMLR, 2020.
[15] Philipp Holl, Vladlen Koltun, and Nils Thuerey. Learning control pdes with differentiable physics. arXiv preprint arXiv:2001, 20000007707757,, 2020.
[16] Peter W Battaglia, Razvan Pascanu, Matthew Lai, Danilo Rezende, and Koray Kavukcuoglu. Interaction networks for learning about objects, relations and physics. arXiv preprint arXiv:1612.00222, 2016.
[17] Alvaro Sanchez-Gonzaler Jonathan Godwin, Tobias Pfaff, Rex Ying, Jure Leskovec, and Peter Battaglia. Learning to simulate complex physics with graph networks. In International Conference on Machine Learning, pages 8459-8468, PMLR, 2020.
[18] Tobias Pfaff, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W Battaglia. Learning mesh-based simulation with graph networks. arXiv preprint 2rXiv:2010.03409, 2020.
11

---

[19] Radek Grzeszezuk, Demetri Terzopoulos, and Geoffrey Hinton. Neuroanimator: Fast neural network emulation and control of physics-based models. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pages 9-20, 1998.
[20] Daniel Holden. Bang Chi Duong, Sayantan Datta, and Derek Nowrouzezahrai. Subspace neural physics: Fast data-driven interactive simulation. In Proceedings of the 18th annual ACM SIGGRRAP//Fungrrahhic.. Symposium on Computer Animation, pages 1-12, 2019.
[21] Yuanming Hu, Jiancheng Liu, Andrew Spielberg, Joshua B Tenenbaum, William T Freeman, Jiajun Wu. Daniela Rus, and Wojciech Matusik. Chainqueen: A real-time differentiable physical simulator for soft robotics. In 2019 International conference on robotics and automation (ICRA), pages 6265-6271. IEEE, 2019.
[22] Junbang Liang, Ming Lin, and Vladlen Koltun. Differentiable cloth simulation for inverse problems. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Bue, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
[23] Yuanming Hu, Luke Anderson, Tzu-Mao Li, Qi Sun, Nathan Carr, Jonathan Ragan- seeley,,lley, and Frédo Durand. Difftaichi: Differentiable programming for physical simulation. arXiv preprint UrXiv:1910.00900 2019.
[24] Moritz Geilinger, David Hahn, Jonas Zehnder, Moritz Bächer, Bernhard Thomaszewski. and Stelian Coros. Add: analytically differentiable dynamics for multi-body systems with frictional contact. ACM Transactions on Graphics (TOG), 39(6):1-15, 2020.
[25] Kiwon Um, Philipp Holl, Robert Brand, Nils Thuerey, et al. polver-in-the-loop Learning from differentiable physics interact with iterative pde-solvers. arXiv preprint arXiv:2007.00016. 2020.
[26] Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel- aligned implicit function for high-resolution 3d human digitization. In Proceedings of the IEEE/CVT Conference on Computer Vision and Pattern Recognition, pages 84-93, 2020.
[27] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 165-174, 2019.
[28] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, pages 405-421. Springer, 2020.
[29] Matthew Tancik, Pratul P Srinivasan, Ben Mildenhall, Sara Fridovich-Ked Keil, Nithin Ragha- van, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. arXiv preprint arXiv:2006. 10739, 2020.
[30] Towaki Takikawa, Joey Litalien, Kangxue Yin, Karsten Kreis, Charles Loop, Derek Nowrouzezahrai, Alec Jacobson, Morgan McGuire, and Sanja Fidler. Neural geometric level of detail: Real-time rendering with implicit 3d shapes. arXiv preprint arXiv:2101, 10994, 2021.
[31] Martin Philip Bendsée and Noboru Kikuchi. Generating optimal topologies in structural design using a homogenization method. Computer Methods in Applied Mechanics and Engineering, 71(2):197-224, 1988.
[32] Michael Yu Wang, Xiaoming Wang, and Dongming Guo. A level set method for structural topology optimization. Computer methods in applied mechanics and engineering, 192(1-2) 246, 2003.
[33] Ole Sigmund. Morphology-hased black and white filters for topology optimization. Structural and Multidisciplinary Optimization, 33(4-5):4014425 2007.
12

---

[34] Martin P Bendsøe and Ole Sigmund. Material interpolation schemes in topology optimization. Archive of applied mechanics, 69(9-10):335-663 654, 1999.
[35] Martin P Bendsøe. Optimal shape design as material distribution problem. Structural optimization, I 14):1193200, 1989.
[36] Erik Andreassen, Anders Clausen, Mattias Schevenels, Boyan S Lazarov, and Ole Sigmund. Ef- ficient topology optimization in matlab using 88 lines code. Structural and Multidisciplinary Optimization, 43(1):1-16, 2011.
[37] Ole Sigmund. A 99 line topology optimization code written in matlab. Structural and multidis- ciplinary optimization, 21(2):120-127, 2001.
[38] Grégoire Allaire, François Jouve, and Anca-Maria Toader. Structural optimization using sensitivity analysis and level-set method. Journal of computational physics, 194(1):363-393, 2004.
[39] Grégoire Allaire, Frédéric De Gournay, François Jouve, and Anca-Maria Toader. Structural optimization using topological and shape sensitivity via a level set method. Control and cybernetics, 34(1):59, 2005.
[40] Xu Guo, Weisheng Zhang, and Wenliang Zhong. Doing topology optimization explicitly and geometrically-a moving morphable components based framework. Journal of Applied Mechanics, (8), 2014.
[41] Weisheng Zhang, Jian Zhang, and Xu Guo. Lagrangian description based topology optimiza- tion-a a revival of shape optimization. Journal of Applied Mechanics, (3),), 2016.
[42] Weisheng Zhang, Junfu Song, Jianhua Zhou, Zongliang Du, Yichao Zhu, Zhi Sun, and Xu Guo. Topology optimization with multiple materials via moving morphable component (mmc) method. International Journal for Numerical Methods in Engineering,
2018.
[43] Zhen Luo, Michael Yu Wang, Shengyin Wang, and Peng Wei. A level set-based parameterization method for structural shape and topology optimization. International Journal for Numerical Methods in Engineering, 76(1):1-26, 2008
[44] Nico P van Dijk, Kurt Maute, Matthijs Langelaar, and Fred Van Keulen. Level-set methods for structural topology optimization: a review. Structural and Multidisciplinary Optimization, 48(3):437-472, 2013.
[45] Yiquan Zhang, Airong Chen, Bo Peng, Xiaoyi Zhou, and Dalei Wang. A deep convolutional neural network for topology optimization with strong generalization ability. arXiv preprint arXiv: 2019.
[46] Yonggyun Yu, Taeil Hur, Jaeho Jung, and In Gwun Jang. Deep learning for determining a near- optimal topological design without any iteration. Structural and Multidisciplinary Optimization, 59(3):787-799, 2019.
[47] Saurabh Banga, Harsh Gehani, Sanket Bhilare, Sagar Patel, and Levent Kara. 3d topology optimization using convolutional neural networks. arXiv preprint arXiv: 1808,,000.00000 07440, 2018
[48] Erva Ulu, Rusheng Zhang, and Levent Burak Kara. A data-driven investigation and estimation of optimal topologies under variable loading configurations. Computer Methods Biomechanics and Biomedical Engineering: Imaging & Visualization, 4(2):61-72, 2016.
[49] Zhenguo Nie, Tong Lin, Haoliang Jiang, and Levent Burak Kara. Topologygan: Topology optimization using generative adversarial networks based on physical fields over the initial domain. arXiv preprint arXiv:2003. 04685, 2020.
[50] Xin Lei, Chang Liu, Zongliang Du, Weisheng Zhang, and Xu Guo. Machine learning-driven real-time topology optimization under moving morphable component- based framework. Journal of Applied Mechanics, 86(1), 2019.
13

---

[51] Qiyin Lin, Jun Hong, Zheng Liu, Baotong Li, and Jihong Wang. Investigation into the topology optimization for conductive heat transfer based on deep learning approach. International Communications in Heal and Mass Transfer, 97:103-109, 2018.
[52] Stephan Hoyer, Jascha Sohl-Dickstein, and Sam Greydanus. Neural reparameterizatioo improves structural optimization. arXiv preprint
2019.
[53] Aaditya Chandrasekhar and Krishnan Suresh. Tounn: Topology optimization using neural networks. Structural and Multidisciplinary Optimization, pages 1-15, 2020.
[54] Haixiang Liu, Yuanming Hu, Bo Zhu, Wojciech Matusik, and Eltychios Sifakis. Narrow-band topology optimization on sparsely populated grid. ACM Transactions on Graphics (TOG), 37(6):1-14, 2018.
[55] Yue Li, Xuan Li, Minchen Li, Yixin Zhu, Zhu, and Chenfanfu Jiang. Lagrangian- eulleian multidensity topology optimization with the material point method. International Journal for Numerical Methods in Engineering, March 2021.
[56] Niels Aage, Erik Andreassen, Boyan Lazarov, and Ole Sigmund. Giga-voxel computational morphogenesis for structural design. Nature, 550(7674):84 86, 2017.
[57] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[58] Blaise Bourdin. Filters in topology optimization. International journal for numerical methods engineering, 50(9):2143 2258, 2001.
[59] Kevin Stanley McFall and James Robert Mahan, Artificial neural network method for solution of boundary value problems with exact satisfaction of arbitrary boundary conditions. IEEE Transactions on Neural Networks, 20(8):1221 2009.
[60] Thomas Lewiner, Hélio Lopes, Antônio Wilson Vieira, and Geovan Tavares. Efficient imple- mentation of marching cubes' cases with topological guarantees. Journal of graphics tools, 8(2):1-15, 2003.
14


Reviews:
[{"review_id": "b3d-53fVJ2", "reviewer": "Reviewer_jpbD", "summary": "The paper proposes a method for topology optimization (TO) in solid mechanics by modeling displacement and material densities as implicit neural networks. TO is a very important field in mechanics and it is exciting to see successful ML applications in the area. The paper shows that their method can solve for topologies under various load and displacement constraints in both 2D and 3D.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n\n- The results appear to be very strong on a variety of cases compared to state-of-the-art methods.\n\n## Weaknesses\n\n- The paper seems to be targeted to topology optimization audience and as such lacks background on a lot of concepts. Specifically I think the multi-batch optimality criterion lacks details and should perhaps be alluded to before (4). A figure on how sensitivity filtering helps might make it more intuitive for a reader. \n\n## Questions\n\n- In Appendix 2, how does the Lagrange multiplier in (4) become the strain in (5)? Also is there a reason why not accounting for u's dependence on $\\rho$ is okay in (9). Some additional clarity here would be useful as this defines the density space gradients which is key to the paper.\n\n- How sensitive are the methods to initialization of the network?  \n\n- The sensitivities are first introduced in Algorithm 1 and then in (8); can you elaborate on what these are? \n\n## Originality\n\nThe key idea seems novel and solid.\n\n## Clarity\n\nA lot of the sections in the paper were not clear to me (see weaknesses). I think the paper will do well if written with a broader audience at NeurIPS in mind. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations were discussed but potential negative societal impact needs to be added. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10-12 hours"}, "extra_text": {}}, {"review_id": "Z1WPRR48xg", "reviewer": "Reviewer_F4BW", "summary": "This work introduces a method for topology optimization based on implicit neural representations. \nThe goal of topology optimization generally is to find optimal material distribution that produces the structure which is as stiff as possible. The key idea presented in this work is to use two neural networks: the density network, which implicitly represents optimal material density field, and the displacement network, which implicitly represents displacement field. \nThe optimization algorithm then proceeds by alternatively training these two networks, while applying additional filtering and ad-hoc supervision to steer the process to better optima. Evaluation is conducted on a set of standard 2D topology optimization tasks. ", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "\n### Paper writing / clarity\n**-** Paper is often hard to follow, especially for people who are not closely familiar with topology optimization. See the list of minor issues below which could help improve readability / clarity. \n\n### Motivation / novelty\n**+** Using neural nets to approximate solutions to complex physical systems is in general a very promising direction, and it seems like implicit neural representations quite naturally fit the topology optimization domain. \n\n**-** However, it appears that the idea of using a neural network to implicitly model optimal density field and use those in an optimization loop has been explored before [53]. It is not clear if simply using SIREN in place of a relu MLP is a sufficiently large technical contribution, even if it does improve performance. It would be great if authors discuss other key differences between their proposed framework and \"TOuNN\".\n\n### Evaluation \n**+** Numerical evaluation indicates that proposed method outperforms a strong baseline - SIMP by a small margin. However, the resulting solutions seem to be _very_ similar qualitatively to the baseline. It would be great to have some context for non-domain experts to understand how large an improvement is reported in Table 1.\n\n**-** Discussion of efficiency of the proposed approach with respect to the baseline (SIMP) is not present. \n\n**-** Evaluation in 3D settings is not very convincing: authors report an example without any numerical comparisons to any baselines. At very least this should be addressed by changing the claims.\n\n**+** I liked the examples of continuously changing volume constraints in the supplementary: if those can be obtained in real-time, it might make sense to elaborate on this in the main paper (and also would be great to know if competing approaches such as [53] and SIMP could get to similar results).\n\n### Minor \n* Might be a good idea to adapt notation a little bit to indicate which quantities are vectors vs scalars (e.g. use bold).\n* L29: domain of interest\n* L78: \"use fully-connected layers\": you mean MLPs?\n* L90: notation \":\" confused me until I reached 4.1.\n* Notations in \"Algorithm 1\" are a bit too dense and confusing. Maybe makes sense to unroll the \"opt\" notation for better readability. \n* L146-147: what is meant by \"standard MLP with residual links\"? if my understanding is correct, SIRENs do not use residual connections. \n* L170: why 5 ? mb makes sense to introduce a notion for this scalar? \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Authors did discuss minor limitations of their approach.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "8NDCtYpsHNT", "reviewer": "Reviewer_YJ3u", "summary": "In this work, neural networks with sinusoidal activation functions are used as continuous function approximators for both the density and displacement fields in structural topology optimization problems. This allows for the development of a mesh-free topology optimization procedure, which is claimed to be on par with existing traditional FEM-based approaches, while also allowing for the learning of continuous solution spaces over certain parameters.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work is clearly presented.\n\nThe authors method seems well motivated, and draws from recent advances in physics informed neural networks and the utilization of periodic activation functions (SIRENs). However, supposedly given the complexity of the optimization tasks being studied, a number of additional tweaks are necessary to ensure good behavior of the optimization algorithm. These are studied in ablation studies in the experimental section.\n\nThe empirical results are the core of the paper. Below I comment on these, and raise some questions which I hope the authors will address.\n\n First, the authors demonstrate that the proposed method is able to achieve results on par (or slightly better) than traditional FEM-based approaches. These are good results, yet it would be important for the authors to also have made available the comparison along other axes that make the tradeoffs between methods more evident. For example, what are the computational costs and runtime of each, both for training and evaluation (for the machine learning-based approach). Similarly for the \"learning continuous solution spaces\", how would the run time of a traditional solver compare here? (Maybe for solving each of the evaluated positions). It would be important to have these values so that the reader can have a clearer picture of what are the pros and cons of each method.\n\nThe 3D evaluation seems qualitatively good, yet quantitative results are not presented, and a comparison against the traditional methods is also absent. (An analysis of computational cost tradeoffs, as mentioned in the previous paragraph would be interesting in this case too.) Given this, the claim in the conclusion that the \"can generalize well to 3D\" (line 303) seems a bit strong, given the brevity (and qualitative nature) of the experimental results in this area.\n\nThe ablation studies seem to support the claims of the authors. However, to make the results more clearly interpretable, it would probably be a good idea to combine the values reported into lines 283 and 284 into Table 1 (or a similar table), where the reader could easily see the comparison across methods with similar units. Additionally, it is not clear to me why results on these baselines were only reported on one experiment. As mentioned previously, it would probably make sense to have a filled in Table 1 (or similar table) with the values for baselines for all experiments, so that a more comprehensive comparison can be performed, not only between the proposed methods and traditional FEM approaches, but also to the baselines.\n\nDespite the issues raised in the previous paragraphs, I believe the methods presented in this paper are interesting and the results look promising. As such, I am willing to improve my evaluation if the authors address these concerns. \n\n-------------\n\nUpdate after authors' response:\n\nThe authors have addressed the concerns I raised above, and as such I am updating my rating of the paper.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Societal impact was not discussed and claimed to be N/A.\n\nThe work is lacking some discussion of possible limitations, these are discussed in the main comments above (with respect to comparison to other methods). The authors should address these.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "I do not review my assigned papers sequentially and thus have not estimated how much time was spent on each paper"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper got three 6 ratings and is therefore a borderline accept.

The authors carefully answered reviewers comments and the reviewers were satisfied by the answers.

Moreover, the meta review of the previous ICML submission has been taken into account.

I would therefore like to accept the paper.

The authors should make sure to improve the clarity of the paper for readers without background in topology optimization.
Decision: Accept (Poster)