NITO: Neural Implicit Fields for Resolution-free and Domain-Adaptable Topology Optimization
Authors: Amin Heyrani Nobari
Lyle Regenwetter
Giorgio Giannone
Faez Ahmed
Venue: Accepted by TMLR ()

Abstract: Structural topology optimization plays a crucial role in engineering by determining the optimal material layout within a design space to maximize performance under given constraints. We introduce Neural Implicit Topology Optimization (NITO), a deep learning regression approach to accelerate topology optimization tasks.
We demonstrate that, compared to state-of-the-art diffusion models, NITO generates structures that are under 15% as structurally sub-optimal and does so ten times faster. Furthermore, we show that NITO is entirely resolution-free and domain-agnostic, offering a more scalable solution than the current fixed-resolution and domain-specific diffusion models. 
To achieve this state-of-the-art performance, NITO combines three key innovations. First, we introduce the Boundary Point Order-Invariant MLP (BPOM), which represents loads and supports in a sparse and domain-agnostic manner, allowing NITO to train on variable conditioning, domain shapes, and mesh resolutions. Second, we adopt a neural implicit field representation, which allows NITO to synthesize topologies of any shape or resolution. Finally, we propose an inference-time refinement step using a few steps of gradient-based optimization to enable NITO to achieve results comparable to direct optimization methods. These three innovations empower NITO with a precision and versatility that is currently unparalleled among competing deep learning approaches for topology optimization. Code & Data: https://github.com/ahnobari/NITO_Public

Published in Transactions on Machine Learning Research (12/2024)
NITO: Neural Implicit Fields for Resolution-free and Domain- Adaptable Topology Optimization
Amin Heyrani Nobari Massachusetts Institute of Technology
ahnobari@mitt.cm edu
Lyle Regenwetter
vegenwetS@miii.cm edu
Massachusetts Institute of Technology
Giorgio Giannone
ggiorgio@mit. edu
Amazon
Massachusetts Institute of Tecknology
Facz Ahmed
faco@mill edu
Massachusetls Institute of Technology
Reviewed on OpenReview:
Abstract
Structural topology optimization plays crucial role in ongineering by determining the optimal material layout within a design space to maximize performance under given constraints. We introduce Neural Implicit Topology Optimization (NTTO), a deep learning regression approach to accelerate topology optimization tasks. We demonstrate that, compared state-of-the-an diffusion models, NITO generates structures that are under 15% structurally sub-optimal and does so ten times faster. Furthermore, we show that NITO is entirely resolut ion-free free and donaann-gnostic, offering more scalable solution than the current fixed-resolution and domain--pecifie diffusion models. To achieve this state-of-the-arl performance, NITO combines three key innovations. First, we introduce the Boundary Point Order-Invariant MLP (BPOM), which represents loads and supports in a sparse and domain-agnostic manner, allowing NITO to train on variable conditioning, domain shapes, and mesh resolutions. Second, adopt neural implicit field representation, which allows NITO to synthesize topologies of any shape resolution. Finally, propose inference-tine time refinement step using few steps of gradient-basee optimization to enable NITO to achieve results comparable to direct optimization methods. These three innovations empower NITO with a precision and versatility that is currently umparalleled among competing deep learning approaches for topology optimization. Code & Data: https:/ h//ggggthbbcooo com aaahaaaaonhn//a// /NITi Public
1 Introduction
Deep learning methods have established near-uncontested dominance in a variely of generative problems, such image and text synthesis (Rombach et al., 2022; Achiam et al., 2023). In engineering-relaee design synthesis tasks, despite a wealth of promising exploratory work (Regenwetter et al., 2022; Song et al., 2023; Hu et al., 2024; Wu et al., 2021), the performance of such models remains highly contested. Design synthesis is a particularly challenging problem for statistical models (Regenwetter et al., 2023). This is in part. due to the nonlinearities between 'feature space' and 'performance space' very similar designs may have extreme discrepancy in performance. Additionally, design constraints (which may be safety-critical) cause difficult-to-lernn gaps in the distributional support of the design space. For these reasons, direct optimization of design parameters has classically been preferred since many optimization algorithms efficiently navigate complex constraints and objectives (Boyd & Vandenberghe, 2004).
Work completed Massachusetts Institute of Technology

---

Ground Truth
Neural Fields
NITO (5)
NITO (10)
Figure 1: NITO framework. Left: Ground truth obtained using SIMP. Second Column: Output of neural
fields. Right Columns: the NITO framework output leveraging a few steps (5 and steps) of optimization. NTTO rapidly generates high-quality, constraint-satiilyingg topologies.
The numerous challenges of successful design synthesis using deep learning and the relative success of direct optimization is exemplified by a prominent and trequently-occurring design problem known as structural topology optimization. Structural topology optimization (TO) aims to allocate a finite amount of material within a designated space maximize structural stiffness, known as compliance minimization. However, depending on the specific engineering application, other performance goals like heat transfer might also be considered. A popular TO method is the Solid Isotropic Material with Penalization method (Bendsøe & Kikuchi, 1988; Rozvany et al., 1992) (SIMP). While methods such as SIMP are powerful engineering tools, they struggle with computational cost when solving large-scale problems due to their iterative nature and expensive per-step computations (Sigmund & Mauto, 2013).
In recent years, many works have investigated the prospect of tackling topology optimization problems using machine learning (Regenwetter et al., 2022; Shin et al., 2023). Many of these are methods for direct topology synthesis, which are often trained on a dataset of optimized topologies while conditioned on the corresponding loads and supports that yielded each optimized topology (Nie et al., 2021b; Mazé Ahmed, 2023; Giannone et al., 2023; Giannone Ahmed, 2023; Hu et al., 2024). The main justification for these approaches is significant time savings compared to TO solvers like SIMP. This could allow designers to run TO in real time during interactive design process, remedying frequent pain-points of current direct-optimization TO users (Saadi & Yang, 2023).
Despite early success, deep learning approaches have struggled with several issues. One of these is generaliz- ability, which is critical for their application to real-world TO problems, which are almost always unique. These real TO problems feature one-of-a-kind load and support locations, allow material placement in a specific domain shape and aspect ratio, and require a particular level of detail (i.c., mesh resolution). Most existing deep learning methods discretize the physical domain into pixels or voxels to leverage a convolutional neural network (CNN) as their basic building block (Mazé Ahmed, 2023; Giannone & Ahmed, 2023; Giannone et al., 2023; Nie et al., 2021b; Hu et al., 2024). This limits their generation capabilitics to specific resolution and shape (such as square domain). Thus, applying the problem new resolution or aspect ratio necessitates entirely new dataset and model.
2

---

In addition to their generalizability issues, pure data-driven approaches typically do not generate optimal topologies compared to classic iterative optimizers like STMP, despite being faster (Woldseth et al., 2022). This is primarily attributable to the fact that data-driven models typically focus on density estimation and remain agnostic to the physics of a given problem. To address this, researchers often try to incorporate the physics into their model. These approaches typically rely on physical fields (or approximations) as a representation of the loads and supports of the underlying problems (Mazé Ahmed, 2023; Giannone & Ahmed, 2023; Gianmone et al., 2023; Nie al., 2021b; Chen et al., 2023; Hu et al., 2024). These physical fields are represented as images to be handled by CNNs. Not only is this approach computationally expensive, particularly for high dimensionality, but it further exacerbates the lack of generalizability across problems and domains. These non-generalizable ML-based TO methods are essentially unable to solve new problems and commonly receive criticism from the computational structural engineering community (Woldseth et. al., 2022).
In this work, we take a stride toward overcoming key generalizability and optimality shortcomings of ML-basod TO. To avoid the generalizability pit,fall of CNN-based generative schemes, we propose approach based on neural implicit fields that can be applied to physical domains of any size, shape, and resolution. We also propose generalizable scheme for representing sparse loading and support conditions in mumerical physics domains, eliminating the need for computationally expensive and domain-limiting physical field calculations, previously thought to be crucial for constraint representation (Chen et al., 2023). Instead, we show that our computationally- and memory-efficiet scheme capable of representing sparse loads and constraints effectively, without loss of performance. To address optimality issues, we propose to couple our deep learning approach with few steps of direct optimization to refine generated topologies during inference. Our approach outperforms the statc-of-the-art in ML-based direct topology synthesis by more than 80% in key performance metrics while running 2.5-50 times faster.
Contributions:
1. We introduce a novel framework for TO called "Neural Implicit Topology Optimizer (NITO)," a model capable of generating near-optimal topologies. NITO achieves domain- and pnpolution-agnostionn through its use of neural implicit fields and our novel "Boundary Point Order-Invariant MLP (BPOM)," " It achieves state-of-the-arl performance through its use of few-step optimization- refinement. during inference.
2. NITO is resolution-free and domain-agnostic during both training and inference. We show in our experiments that NITO can be trained on multiple resolutions and domain shapes simultaneously and can generate topologies across a variety of resolutions, overcoming a major shortcoming of prior works.
3. We empirically show that NITO is an effective framework, producing topologies with up to 80% lower compliance errors than SOTA learning-based models, while requiring an order of magnitude fewer parameters, allowing 2.5-50x faster inference. These performance metrics are consistent across a variety of resolutions, despite maintaining the same architecture and mumber of parameters.
4. We implement a new STMP optimizer for minimum compliance TO. Compared to solvers used to generate existing datasets, our new optimizer produces topologies that are on average 3% lower in compliance, yielding dataset comprised of significantly more optimal topologies. Our optimizer runs up to 6x faster than widely-used Python implementations. We also introduce a high-quality datasct and improved versions of state-of-the-art models.
2
Background
This section will provide background information on topology optimization and neural implicit fields. We introduce previous work using neural fields for optimal topology generation in this section and review existing work using deep learning for TO in the Appendix.
2.1 Structural Topology Optimization
In structural applications, TO often aims to minimize compliance (structural deformation) given a set of force loads, supports, and a material volume limit (Tiu & Tovar, 2014). These loads, supports, and volume
3

---

Topology Optimization Problem
Resulting Topology
Force
Topology Optimization (SIMP, NITO, etc.)
Fixed in Both X and
f= d
Fixed in
Ω
minimize f
K F
Fixed in Y
Σ
(0) S
eEl
Volume Fraction 0.38
Fixed in Both a
Figure 2: Topology Optimization. Given a domain, loads, supports, and volume fraction, TO aims to find the design variables & that maximize performance (such as minimizing compliance f), fulfilling all the prescribed constraints and respecting the underlying physics (Static equilibrium). Note that the structure displayed the left is defined by the design variables which indicate which parts of space have material and what parts of space do not.
limit ('volume fraction') ) are casually referred to as the problem constraints, which are illustrated in Fig. 2. The goal in TO to find an optimal structural layout (topology), given the problem constraints. Optimality is calculated as the compliance (deformation per unit force applied) of the structure when subjected the applied loads and and when the supported locations of the structure are constrained not to move. A prominent technique in TO is the Solid Isotropic Material with Penalization (SIMP) method, introduced by Bendsøe (1989). This method models material propertics through a density field, where the density value signifies the amount of material in a specific region. The optimization iteralively simulates the system to evaluate the objective, and then adjusts the density field based on the gradient of the objective score for the field. Mathematically, we represent the density distribution as p(ő), where á is a set of design variables, each representing a value within the problem domain Ω. Note that the number design variables is dictated by the number of elements Q which depends on the underlying discretization, and mesh, this paper, we use structured meshes in rectangular domains, and refer to resolution instead of element counts. A more in-depth discussion the topic and comprehensive explanation of the optimization problem provided in Appendix A. The optimization task is structured follows:
min f=P
8. t.
K k
(1)
Ecer (o)e SV
Omin S Φmax Vie
The objective is to minimize the compliance d' f is the load vector and d the nodal displacement derived from K(Ø)d= f, where K(Ø) the stiffness matrix contingent on the design variables ø. The constraint EcEn ( (0))) < V ensures the total volume does not exceed a maximum limit V (often expressed as a fraction of the domain volume). The optimization seeks design variables within specified bounds (ømin and 0max) for every element in the domain Ω. By allowing continuous variation of design variables between O and 1, this formulation supports gradient- based optimization. Nonetheless, the expensive finite element simulation (solving K(o)d = F) at cach iteration creates significant computational cost. Notably, solving the linear system of the equations scales cubically (O(n²)) with the number of nodes. This, however, is the cost of solving a fully dense linear system of equations. In reality, linear elasticity results in very sparse symmotric positive definite matrices which are much faster to solve. Specifically, when working with structurod meshes like the ones in this work there are many solvers that perform much better than O(n²) such as the conventional multi-grid approaches proposed by Träff et al. (2023) even more advanced narrow-band
4

---

multi-grid solvers like the one developed by Liu et al. (2018) more specialized solvers like Aage et al. (2017). In theory, a perfect multi-grid solver could have convergence rate that yields an ((n) complexity, however, in practice, this is rarely possible. Furthermore, such solvers require algorithms that work iteratively, which makes them not readily parallelizable, hence making the cost of solving each system of equations very high in compule time even if not in complexity. Unlike this, machine learning methods oflen involve models that are highly parallclizable, and in our case as we will see, the complexity of the proposed methods scales linearly, guaranteeing (n)) complexity while being fully parallelizable. All this does not take into account that a conventional solver has to be employed over potentially hundreds of iterations to solve the optimization problem, while our proposed method can produce structures in one step, and as we will demonstrate can reduce the needed steps for optimization greatly.
2.2 Deep Learning For Topology Optimization
The limitations associated with conventional optimization have motivated a significant body of work around deep learning approaches for topology optimization (TO). These approaches have been discussed in detail in dedicated reviews (Woldseth et al., 2022; Shin et al., 2023). Most relevant to our work are other methods that perform the task in an end-to-end manner, meaning they take loads, supports, and volume fraction as input and produce near-optimal topologies to minimize compliance (Oh ct al., 2019; Sharpe & Scepersad, 2019; Parrott et al., 2022). For example, Yu et al. (2019) proposes an auto-encoding GAN approach that uses an autoencoder for topology generation and GAN for super-resolution which is applied simultaneously. Rawat & Shen (2019) and Li et al. (2019) take similar approach but use GANs for both initial generation and super-rosolution. This is while Sharpe & Seepersad (2019), Nic et al. (2021a), and Behzadi & Ilies (2021) propose directly generating lopologies using conditional GANs and Wang et al. (2021c) introduce U-Net-based architectures for TO. More recently, researchers have looked at using implicit neural fields to generate topologies. Specifically, Hu et al. (2024) use implicit neural representations to produce topologies in their approach called IF-TONIR. In their approach, however, the authors use stress and strain fields for their representations of loads and supports which rely on CNNs and somewhat reduce the generalizability of the implicit neural fields' versatility.
Conditioning on Physics Fields: Most of the recent otate-of-the-art approaches (Mazé & Ahmed, 2023; Nie et al., 2021b; Chen et al., 2023) have adopted a conditioning method in which the loads and supports of TO problems are represented through physical fields such as stress and strain energy. Although these fields are typically simulation-derived kernel-based approximations have also been proposed to reduce cost (Giannone & Ahmed, 2023). Regardless, this field-based conditioning approach is essential for these models to generate high-performing topologies. Problematicallyy field-based conditioning has only successfully been handled by CNN models, which prevent the model from being applied to any new problem with different domain shape or resolution than the training data.
Diffusion-based Topology Optimization: New mothods that utilize diffusion models have recently achieved significant performance improvements over the aforementioned methods (Mazé & Ahmed, 2023; Giannone & Ahmed, 2023; Giannone et al., 2023). Mazé & Ahmed (2023) were the first to do so and demonstrated that diffusion models are significantly better than GANs at optimal topology generation. They also showed that introducing guidance based on regression model for predicting compliance and classiffer trained to identify floating material significantly improves the performance of such models. Like iterative optimization, however, diffusion models are slow, requiring the models to be run as many as 1000 times to generate a sample. This puts their value into question when the time savings are not notable compared to the optimizer itself. Giannone et al. (2023) propose aligning the diffusion models denoising with the optimizer's 's intermediate designs as a way to reduce the number of itcrations needed for the diffusion model and show that they can indeed reduce the number of sampling steps significantly.
Despite the upsides of the best-performing models, which are diffusion- biffusion-based, they lack speed and are all based around CNNs, which means that they treat the problem as images. Furthermore, the mochanism they use for conditioning based on loads and supports is fieldbbsed, meaning these methods compute some physical or energy field to represent the loads and supports, treat these fields as images, and employ CNN-based architectures to handle them. This method introduces two limitations. These models generate images instead of density fields, restricting them to a specific problem domain and resolution (e.g. a square-shaped domain treated as 64x64 images) and making them unsuitable for different domain shapes and resolutions.
5

---

Additionally, the use of image-based fields for loads and supports further narrows their generalizability. This approach fails to capture loads and supports in their raw form, limiting applicability to problems with loads and supports defined for the exact domain they are trained on and excluding problems with irregular loads and supports.
Shape Optimization And Ncural Operators Another approach that has been applicd in physics-basod optimization of designs is shape-optimization. Such models attempt to predict optimal shapes for physics- based objectives such as aerodynamic properties like lift and drag coefficients Viquerat et al. (2021); Durasov et al. (2024); Wei et al.; Durasov et al. (2023); Baque et al. (2018); Heyraní Nobari et al. (2021); Chen & Ahmed (2021); Chen & Fuge (2021). In such approaches, the objective is often to optimize shape typically defined by continuous functions such as Béxier curves splines and in some cases discrete meshes defined as graphs. Given the nature of these problems being defined in function spaces, they lend themselves to neural operators Kovachki et al. (2023) which are machine Icarning models that map between continuous function spaces, which is particularly useful when the shapes being optimized are defined in such spaces. Neural operators are effective in modeling PDEs and shapes. On its surface, such models provide a road map for resolution-free and adaptable models for structural TO problems similar to the problem we target in this paper. Unlike shapes, however, here we intend to optimize the material distribution in continuum. This means that a simple function space like Bézier curves or other shape functions would limit the space of structures with detailed features. This is a well-known issue with such representations in the structural topology optimization ion literature Wang et al. (2021a). As such, in conventional structural TO approaches, function spaces are defined in a different manner. The most commonly explored approach that is analogous to shape optimizations is the level-set methods for topology optimization Wang et al. (2021a). In these approaches, a field function is defined and sampled at specified level. This means that a continuous field is defined by continuous function and the existence of material is based on whether the value of a field is above set value or level, hence the name level-set. Like shape optimization, this allows for sampling at any resolution for any shape. It is often difficult to come up with level-set functions that enable stable and effective inuuin lopology optimization, and it is well known that despite being continuous and resolution-free, the details obtained by these topologies are limited by the resolution of the underlying solver Wang et al. (2021a). However, neural fields used as the neural operator or level set function has been shown to be effective for structural topology optimization Sanu al. (2024); Erzmann & Dittmer (2024); Chandrasekhar & Suresh (2021) using conventional optimization loops. In these approaches, however, a direct neural operator in its conventional form is often not practical, rather a neural operator with level set addition is usually applied Erzmann & Dittmer (2024). As such, we choose to work with neural fields with level set as our primary approach rather than using neural operators, since no clear function space has emerged in the conventional optimization literature. In the scction that follows we describe our approach in more detail.
2.3 Implicit Neural Fields
A neural field is a field that is partially or fully defined by a neural network (Xie et al., 2022). This neural network takes some form of coordinate representation in space x E RD as input and outputs a set of values (i.e. a measure of the desired field) (xx) E R":
((x)=
=
(2)
where fe is the neural network function given parameters O These neural fields have proven effective in representing audio (Du et al., 2021), images (Skorokhodov et al., 2021; Sitzmann et al., 2020), videos (Yu et al., 2022), 3d objects (Park et al., 2019), 3D scenes (Mildenhall et al., 2020), and many more applications (see the review by Xie et al. (2022) for more comprehensive picture). It has also been shown that neural implicit reprosentations can be directly optimized using gradient-based optimizers similar to SIMP to represent the optimal topology for any given problem (Zehnder et al., 2021; Joglekar et al., 2023; Chandrasekhar & Suresh, 2021), which shows that these implicit neural representations are indeed capable of generating density fields that represent individual optimal topologics. Compared to existing work, which represents one individual topology per neural field, we propose a conditional implicit neural field approach that generates different optimal topologies based on conditions (i.e., volume, loads, supports).
6

---

Supports
Boundary Condition
Target Volume Fraction
Boundary Condition Representations
Point Cloud Representation
Layer
XM XYTO
Faoed mn
Racd Xonhh
Supports
Food
Fioed in andi)
PEBE BBB- Latent Constraint Representation
Problom
Field Representation
Point Cloud Represemation
Force Point Cloud
Final Topology
Fel Step Direct Optimiaatin
FC Layer
FC Layer
Mesh Element Element Centroid
(CC)
B(C)
TC
sin Gin(LNii (
+ +
Generated Density Field
Conditioning Through Modulation of Layer Normalization
Cloud
Point Cloud Representation
Min Pooling
FC Layer
FC Layer
BPOM:
1.4
Average Pooling
Max Pooling
Figure 3: The NITO framework for topology optimization: Loads and supports processed as point clouds using 3POM(Illustrattr
the Bottom); neural field conditioned on these representations by modulating layer normalization based on latent constraint representation predict optimal density field; predicted field refined through direct optimization steps. Top-right: Point cloud loads and supports (right) vs. representations like stress fields (middle), given TO problem (left). Point clouds offer generalizable, memory- efficient representation for loads and supports, unlike expensive, domain-limiting iterative FEA method.
3 Methodology
We introduce NTTO:
Neural Implicit Topology Optimization. Unlike the majority of existing deep learning approaches in TO, NITO effortlessly generalizes across different solution domains and problem resolutions. NITO also achieves more effective balance of speed and optimality than existing approaches. To enable these achievements, NITO combines three key innovations. First is the neural field-based topology representation that affords much of NITO's flexibility. Second is the Boundary Point Order-invariant MLP (BPOM) which is the second ingredient in NITO's domain- and resolution-agnoosiiiinl NITO's third and final component is a few-step optimization based refinement of topologies during inference, allowing NITO to rapidly generate solutions that rival the highly-optimal structures generated by direct optimization methods. This section discusses these three key innovations in detail. A detailed schematic of how the NITO framework operates is shown in Figure 3.
3.1 Implicit Topology Representation using Neural Fields
The spatial density distribution used in SIMP is highly compatible with neural fields. We denote p(x), where x is the coordinate of a given point in space and p(x) is the density of material in that part of space. Since we need to learn a conditional distribution based on the loads, supports, and volune fraction for a given problem, we model density as:
p(x|C; fx(xx
where denotes the neural field function which is defined by the neural network architocture and parameters (0) and is a condition vector that includes information about the domain shape, loads, supports, and desired volume ratio. In practice, this density field is desired to have density of exactly 0 1 at any point,

---

representing where material should should not be placed. As such, the density fields that are generated by the SIMP optimizer for given problem have binary values.
In practice it is more practical to formulate the problem as the probability of material being placed at any given coordinate. As such, the formulation the objective can be written as:
(())- Ex,c [p(x|C) log (x, C)
(4)
+(1 (-- log(1 fx(x, C))]
where C(x(C) is the probability of material existing at x, which in this case is the same as the output of the SIMP optimizer.
We implement our neural fields using simple multi-layer perceptrons (MLPs) with SIREN (Sitzmann et al., 2020) layers which apply sine activation functions the output of each layer. Since neural fields tend to ignore higher-frequency features (Tancik et al., 2020), we apply Fourier feature mapping to the input coordinates of the model to mitigate this issuc.
3.2 Latent Constraint Representation using Boundary Point Order-invariant MLP (BPOM)
Having discussed the neural field implementation, we now consider loads and supports in discretized domains, as seen in TO problems. In particular, we discuss how arbitrary discrete loads and supports can condition generative models in sparse and generalizable way. This is particularly important for large domains, where defining constraints over grids is computationally cumbersome and memory-expensive.
In most existing studies, authors only considered simple input-only conditioning, which is prone to be ignored by the model (Perez et al., 2017). We suspect this tendency of models to ignore sparse input conditions explains the singular popularity of the rigid field-based conditioning discussed in Sec. 2.2, despite its lack of gencralizability. In this paper, we instcad propose a simple and effective condition representation model that leverages more advanced conditioning methods to incorporate conditioning information. This allows our model to train simple and generalizable point-cloud-best based conditions, expanding its applicability to various domains and resolutions.
Force
X
Fixed in Both X and
Fixed in
Y
X
X Y
Fixed in Y
X Y
Fixed in Both X and Y
Problem
Field Representation
Point Cloud Representation
Figure 4: Comparison of field-based (middle), and point-cloud-based (right) representations, given a TO problem (left). Unlike the expensive and domain-limiting iterative FEA method, the point clouds offer generalizable and memory-efficien representation of the loads and supports.
Loads and Supports as Point Clouds: In our approach, we represent TO loads and supports as point clouds, which are agnostic to the problem domain and solution resolution. In all problems, conditioning is based on loads, supports, and volume fraction (see Figure 4). Since a problem may have arbitrarily many loads or supports, we need mechanism that can take all loads and supports for a problem and reduce them a single latent representation. To do this, we treat these sparse conditions as set of two point clouds one for loads, and another for supports (see Figures 3 and 4). Then process each point cloud using ResP
8

---

Layers proposed by Ma et al. (2022) in their point cloud model, PointMLP. However, unlike PointMLP, we removed the geometric affine module (as proposed by the authors for complex point cloud geometries) due to the simpler geometric nature of our sparse load and support point clouds.
Order-Invariant Aggregation: Since the point clouds can have any size, we take the output of the point cloud model and perform order-invariant pooling to reduce every point cloud into one single vector representing the loads and supports. We do this by computing minimum, maximum, and average pooling and concalenating the three pooling results. In the end, the three point clouds are each reduced and then concatenated into one vector. We call this representation Boundary Point Order-Invariant MLP (BPOM) and show that this representation works just as well as physical fields. BPOM's inner workings are illustrated in Figure 3. Finally, for the volume fraction condition (which is always single floating point value), we use single fully-connected layer and concatenate its output to the BPOM outputs of the load and support point clouds.
Feature-wise Linear Modulation: Next, we discuss how the condition vector is fed into the neural fields model. Neural fields possess the capability to assimilate prior behaviors and exhibit generalization to novel fields through conditioning on a latent variable C, which encapsulates the characteristics of a field. Specifically, this includes the loads, supports and the volume fraction our problem. Perez et al. (2017) propose Feature- wise Linear Modulation (FiLM) which applies conditioning by modulating the outputs of the different layers of the model. The conditioning mechanism includes two networks (C) and 3(C), which predict multiplicative and additive adjustment to the outputs of each layer. Our implementation is inspired by similar mechanisms built around normalization layers such as adaptive instance normalization (AdaIN) (Huang & Belongie, 2017). Instead of modulating the outputs of each layer, we apply layer normalization and modulate the scale and shift of the layer norm for each feature of the layer output. Our neural field can be described as:
fe(x, o(x f(l-1) o...o f(0) (x, C))
(5)
where f(ر for i E (1,2, 1} indicates the function applied at each layer of the neural field except the first and last layer. Each layer in the model takes a hidden state h(() as input and performs a linear transformation and layer normalization with modulation based on the condition vector:
/'',' = sin(LN ( +b+ x aCC)+ + B(C),,
where LN1, is layer normalization with scale 1 and shift zero and and B are fully-connected (FC) layers that determine the feature-wise scale and shift based on the condition inputs.
BPOM allows NITO to generalize to any domain shape and TO problem as long as that information is contained in the condition latent representation C.
3.3 High-quality Solutions using Optimization- Beeiii Based Refinement
Despite the success of generative models and other deep learning techniques for TO, we observe that the performance of these models always lags behind the optimization baseline. To address this, we follow an optimization- based refinement approach proposed by Giannone et al. (2023) who suggest applying few steps (5-10) of SIMP optimization to topologies synthesized by deep learning models during inference (compared to 200-500 iterations for full optimization). This allows SIMP to rapidly hone in on more accurate structures for the given loads and supports using near-optimal generated topologies as a starting point. To ensure that the optimization itself is performed optimally, we implement the SIMP method based on the latest research on TO (Wang et al., 2021b) and use this solver for our experiments and training. This optimization- refinement. represents the third and final key innovation of our NITO framework.
3.4 Training and Model Details
The training of the conditional neural implicit model is performed in 3 stages. In the first stage, we perform training by sampling points of a 16x16 grid (i.e., we sample points in space that each belong to one of the 16x16 divisions in space for the unit square domain). Furthermore, when sampling, if the matorial exists in a given sampling grid, we randomly choose point with material rather than picking points that are void. The first stage of training is carried out for 20 epochs. Tn the second stage, we sample on 32x32 grid for 20
9

---

epochs. Finally, we train for 10 more epochs, sampling on 32x32 grid but rather than preferring material points, we sample completely randomly. We use AdamW optimizer with starting learning rate of 10-1 which is reduced on a cosine annealing schedule to be reduced at the end of each epoch to reach 5 x 10-6 at the final epoch. The learning rate is stepped at the end of each epoch.
In our implementation, we use 8 layers of size 1024 for the neural fields and use 4 layers of size 256 for the three point cloud models (one for applied load and one for supports in each x and y direction). Although intermediate layers of the neural fields model use the aforementioned FILM activations, the final layer's activation function is simple sigmoid. Further details can be found in our code.
4 Experiments
In this section, we present an assortment of experiments to compare the performance of NITO to existing state-of-the-art models in the literature. With these experiments, we provide compelling evidence that NITO is a scalable, resolution-free and domain-adaptable paradigm that outperforms convolution-bass methodologies with a significantly smaller parameter count. We further demonstrate that NITO can generate mustraint-satisfying
pigh-pmcformance topologies much faster than a state-of-the-art SIMP optimizer, while merging the power of deep learning models as an efficient first stage, and the precision and guarantoes of optimization as a reliable second stage. For more experiments, discussion, and visualization, we refer the reader to the Appendix.
4.1 Experimental Details
Before presenting the results, we will first discuss key experimental details.
Dataset and Solver: We create a dataset comprised of various domain shapes and resolutions using a custom SIMP optimizer leveraging recent advancements in iterative TO (Wang et al., 2021b). Noting that much of the previous data-driven TO work uses datasels generated by ToPy (Hunter el al., 2017), which lacks these recent TO advancements and is much slower than our solver, our dataset is larger and more optimal than those used in previous works. We then used this new and faster solver to generate a new dataset with mixed resolution and shapes. To allow for comparison to prior works, we re-created the 64x64 dataset from Mazé & Ahmed (2023) with our solver, finding significant improvements in optimality. We additionally created high-resolution 256x256 data. Finally, we also generated 64x32, 64x48, and 64x16 data, to (est. NITO's ability to handle mixed domain shapes (something prior works cannot handle). We include loads, supports, volume fractions, and stress and strain energy fields in our dataset. A total of 199,000 samples can be found in our datasct including 48,000 64x64 samples, 61,000 256x256 samples, and 30,000 samples of 16x64, 32x64, and 48x64 each. For each resolution, 1,000 samples are used for testing. To keep results consistent with prior works, we use the exact same 1,000 test samples in the 64x64 subset as prior works have used. For fairness in comparison, we retrained the TopoDiff model (Mazé & Ahmed, 2023) (the best-performing model in literature) on our new ro-optimized dataset (64x64 and 256x256 only), while also aeknowledging the original performance metries reported by the authors. Further details on the dataset and solver are included in the Appendix.
We evaluate the models in terms of performance (i.e., minimum compliance) and constraint satisfaction. First, to measure how well the models perform in compliance minimization, we measure compliance error (CE), which is calculated by subtracting the compliance of a generated sample from the compliance of the SIMP-optimized solution for the corresponding problem, then normalizing by the STMP-optimized compliance value. Since mean compliance error tends to be dominated by just a few samples, we report mean and median compliance error. We also report median volume fraction error (VFE), which quantifies the absolute error between the generated topology's 's actual volume fraction and the target volume fraction for the given problem. Beyond these base performance metrics, we quantily inference time. The speed of the SIMP optimizer is also benchmarked for comparison. We remove outlying samples, which have extremely high compliance errors (above 1000%) due to models failing to place material in locations where the load is applied. We do this for all models including other SOTA models that we compare against, as is common practice in prior works (Mazé & Ahmed, 2023; Giannone et al., 2023; Nie et al., 2021b).
Evaluation Metrics:
Sctup: We train NITO for 50 epochs in four different scenarios: (1) We train on topologies with a resolution of 64x64. (2) We train 256x256 topologies. (3) We train on both 64x64 and 256x256 datasets simultancously
10

---

to demonstrate resolution-free generalizability. (4) We train all 64x64, 256x256, 64x48, 64x32, and 64x16 datasets to demonstrate the generalizability of our approach across both resolution and domain shapes. To compare our approach to the state of the art, we also train TopoDiff on the 64x64 in the manner the original authors did (Mazé & Ahmed, 2023). TopoDiff is currently tho-at in litcrature (Giannone et al., 2023). Training a diffusion model like Tifo on large images of size 256x256 is computationally expensive. Despite this, we train Tifo on the 256x256 dataset for 500,000 steps and report the results, however, we do not train TifoDiff with guidance (which, demonstrated in the original work (Mazé & Ahmed, 2023) and corroborated by our experiments, does not lead to any significant improvement in performances).
Constraint Satisfaction: NITO generates fields as its output. In our experiments, we do not explicitly enforce the volume fraction constraint in the problem, rather simply threshold the field at value of 0.5 and use that to evaluate the topologies. This is because this aligns with the greedy sampling of the probabilities predicted by NITO. Note that one could euforce the volume fraction constraint on NITO by performing binary search on the generated density-field to find the threshold that ensures the volume fraction is met. Here we do not do this since this would not be practical to apply to methods that are very close to binary predictions such as TopoDiff and since the threshold search is not strictly part of NITO's training and archilecture we don't implement this in our experiments. However, a practical setting, the cost of binary search is very small and should be incorporated when deploying models such as NITO.
4.2 Performance Comparison to State of The Art
Table 1: Quantitative evaluation on 64x64 datasets (All models only trained on 64x64 data). All Topo
variants are re-trained on optimized topologies obtained using our improved optimizer. "TopologyGAn and cDDPM results from Mazé & Ahmed (2023) and Giannone et al. (2023). w/ G: using classifier and regression guidance. CE: Compliance Error. VFE: Volume Fraction Error. NITO achieves SOTA performance topology optimization in terms of compliance and volume fraction error.
Model
Refinement Steps CE % Mean CE % Median VFE % Mean
GAN'
48.51
2.06
11.87
cDDPM*
60.79
3.15
1.72
TopoDiff
3.23
0.45
1.14
TopoDiff w/ G
2.59
0.49
1.18
NITO (ours)
8.13
0.47
1.40
NITO (ours)
5
0.30
0.12
0.40
NITO (ours)
10
0.17
0.071
0.25
Table 2: Quantitative Evaluation on 256x256 datasets (All models are only trained on 256x256 data).
The columns are the same as Table 1. NITO effective at generating topologies with high performance irrespective of the considered problem resolution. On the other hand, the performance of TopoDiff has degraded so much that NITO outperforms it even without a refinement step.
Model
Refinement Steps CE % Mean CE % Median VFE % Mean
TopoDiff
16.62
0.59
2.92
NITO (ours)
9.178
0.96
1.52
NITO (ours)
0.25
0.09
0.34
NITO (ours)
10
0.033
0.012
0.128
In Table we present quantitative evaluation of SOTA models for topology optimization on the commonly- tested 64x64 resolution. Mean compliance error results are average much higher than median results due
11

---

to small number of samples having extreme compliance. ToP and cDDPM show higher values in CE % Mean and VFF % Mean compared to other models. TopologyGAN is consistent with prior benchmarks of the model and cDDPM is a naive conditioning of diffusion models, which struggles to perform well in such a complex problem. A vanilla neural fields model without optimization does not perform as well as TopoDiff mean compliance error; but performs similarly in median compliance error and volume fraction errors. Unambiguously though, Table demonstrates that our NITO framework achieves a significant leap performance compared to the SOTA. Neural fields start with average compliance errors of more than double TopoDiff. However, after even 5 steps of direct optimization, NITO outperforms TopoDiff and other methods by a large margin. When taking 10 steps of refinement using direct optimization, NITO finds solutions that are on average only 0.1% more compliant than the solutions yielded by 500 steps of optimization.
Next, we proceed to test NITO and TopoDiff on the much more challenging dataset of 256x256 topologies. Table 2 presents the results of this study for NITO and TopoDiff (the leading SOTA approach). In this case, we train TopoDiff with a larger model size to allow for effective learning on the 256x256 images. Despite this, we see a significant performance degradation of Tofo with increased resolution. This performance decrease is so severe that the NITO outperforms TopoDiff in most metries even without refinement step (highlighted in green in table 2). With refinement, NITO achieves results on par with or slightly better than the 64x64 tests, all while using the same model size, architecture, and training. This illustrates that NITO scales effectively to higher resolutions without performance degradation.
4.3 Topology Refinement Across Baselines
Table 3: In this table, we consider the effects of few-step direct optimization to topologies generated by TopoDiff. While direct optimization significantly improves the topologies generated by the neural field, it only slightly improves topologies generated by TopoDiff.
Model
Refinement Steps
CE % Mean
CE % Median
VFE % Mean
TopoDiff
3.23 (-)
0.45 (-)
1.14 (-)
TopoDiff
5
3.55
(+9.91%)
0.42 (-6.67%)
0.67 (-41.2%) 0.45 (-60.5%)
TopoDiff
10
1.38 (-57.3%)
0.33 (-26.7%)
TifoDiff w/ G
2.59 (-)
0.49 (-)
1.18 (-)
TifoDiff w/ G
5
2.24 (-13.5%)
0.44 (-10.2%)
0.69 (-41.5%)
TopoDiff w/2 G
10
1.05
(-59.5%)
0.32 (-34.7%)
0.45 (-61.%%
NITO (ours)
8.13 (-)
0.47 (-)
1.40 (-)
NITO (ours)
5
0.30
(-96.3%)
0.12 (-74.5%)
0.40 (-71.4%)
NITO (ours)
10
0.17 (-97.9%)
0.071 (-84.9%)
0.25 (-82.1%)
NITO uses a few steps of optimization to refine generated topologies. In this section, we consider the performance of SOTA models, should they be subjected to the same few-step refinement using optimization. Table 3 indicates that optimization of topologies generated by TopoDiff yields a much smaller performance boost compared to NITO and even reduces performance in some cases. After just 5 iterations of refinement, NITO outperforms all variants of TopoDiff tested (including TopoDiff with 10 iterations). NITO's 's lead continues to grow with more refinement steps. Visual examination suggests that while TopoDiff- topologies are detailed, topologies generated by the neural field tend to be slightly more blurry (see Fig. 1). Therefore, we hypothesize that Topodiff finds crisp locally-optimal solutions, while NITO, despite being a much smaller model and not using FEA-based physical fields, finds less locally-optimal solutions that lie closer to stronger 'global' optimum. This suggests that NITO is more robust, generalizale lizalle, and efficient.
4.4 Inference Speed & Efficiency
Speed is one of the key benefits of ML-based TO over direct optimization. In Table 4 we present a comparative analysis showing that NITO is significantly faster than both iterative TO methods and competing generative models in inference time. For the 64x64 and 256x256 data respectively, resolution, NITO 83% and 63% faster than the fastest state-of-the-art model DOM (Giannone et al., 2023), and 97% and 96% faster than our
12

---

Table 4: Average inference time for different problem resolutions. We include 10 SIMP iterations when computing NITO inference time. NITO is resolution-free, i.c. we can leverage the same small model for 64x64, 256x256, and any intermediate resolution. These times are measured using an RTX 4090 GPU and Intel Core 19-13900K CPU. We run the SIMP optimizer for 300 iterations. For 5000x5000 resolution, multi-grid GPU SIMP solver is used, and NITO is sampled at 500x500 parallel patches.
64x64 Resolution
256x256 Resolution
5000x5000 Resolution
Model
Parameters (M) Inference (s)
Paramoters (M) Inference (s)
Paramoters (M) Inference
TopoDiff
121
1,86
553
10.81
TopoDiff w/
239
4.79
1092
22.04 04
DOM
121
0.82
553
7.82
SIMP (Hunter et al., 2017)
18.12
316.02
SIMP (our implementatton)
3.45
69.45
6,676.06
NTTO
(No Ref.) (ours)
22
0.005
22
0.16
22
27.15
NITO (ours)
22
0.14
22
22
245.12
fast SIMP implementation (which itself is up to faster than existing implementations) Thus, in addition to generating superior topologies, NITO's speed and efficiency is orders of magnitude faster than the SOTA.
Table 4 also highlights the number of parameters each model uses in their architecture. NITO can be trained on both image resolutions with the same number of parameters, 22 million, achieving SOTA performance, while CNN-based models have to be made larger and still face significant performance degradation. This is further evidence of NITO's efficiency and scalability. We train NITO for both 64x64 and 256x256 resolution for the same number of steps while sampling the same number of points for each batch during training, which means that the model trains roughly for the same amount of time and with the same memory requirements. In fact, single consumer GPU (we models like TopoDiff or DOM must grow to match larger resolutions and therefore require more memory and time to run and train. For resolutions above 256x256 or 3D TO problems, these frameworks can be impractical to train or run for most practitioners. In contrast, NITO is built to generalize to different domains Nomains/resoluttan rrsoluuiions without issue, allowing for practical training of large problems with consumer-level computational resources. In the following, we specifically showcase the versatility of NITO when handling different domains.
RTX 4090) is enough to train NITO. On the contrary, diffusion
4.5
Resolution-Free Free and Cross- Domain Generalization
Training Data
64x64
256x2566
64x64 & 256x256
All Data
Testing Data
CE
CE
VFE
CE
CE Med.
VFE
CE
CE Med.
VFE
CE
CE Med.
VFE
Mean
Med.
Mean
Mean
Mean
Mean
Mean
Mean
Mean
64x64
0.17
0.071
0.25
0.27
0.11
0.30
0.22
0.072
0.29
0.22
0.074
0.31
256x256
0.058
0.016
0.13
0.033
0.012
0.128
0.048
0.027
0.14
0.066
0.032
0.14
64x48
0.43
0.20
0.35
64x32
0.90
0.20
0.41
64x16
1.16
0.14
0.72
Table Quantitative evaluation of NITO trained on mixod resolution and aspect ratio data using steps of refinement. All metrics reported as percentages. The results show that NITO trained mixed-resolution and mixed-shape data performs well across different resolutions and shapes.
To showcase NITO's ability to generalize across multiple resolutions and physical domains, we present results for different training configurations of NITO and its performance on different domains and resolutions. These results are presented detail in Table 5. We see that the model simultaneously trained on data of all resolutions and aspect ratios performs on par and better than other SOTA models on both the 256x256 data and the 64x64 data while also effectively handling the non-square configurations in the dataset. This illustrates NITO's resolution-free and domain-adaptable nature. It can be trained multiple domains and
13

---

perform well on all of them. Similarly, NITO can train on resolution and be applied to problems at new resolution. This capability is demonstrated by testing NITO models trained on 64x64 data with 256x256 resolution tasks, and vice versa. Remarkably, NITO's performance is consistent across resolutions and shapes: it performs similarly when trained on lower-rosolution data compared to higher or mixed-rosolution data, and likewise when tested on ligher-resolution or lower-resolution problems. This underscores NITO's exceptional adaptability, indicating that its architecture not only supports training across multiple domains but also facilitates the transfer of learning from one domain to another, provided the problems share related distributions. This ability signifies critical advantage of such frameworks- the potential to train on cost-effective low-resolution data and immediately apply or quickly fine-tune the models for higher resolutions.
Table 6: In this table, we consider how NITO performs when sampled at much higher resolution of 5000x5000 compared to running SIMP at this native resolution. We see that as expected, NITO's performance degrades in such scenario, which is expected since the training data is much lower in resolution, however, we see that with few steps of optimization NITO starts to perform much better, showing how even at higher resolutions sampling NITO can enable faster optimization. NOTE: This experiment is conducted on a smaller subset of 10 samples since the cost of running SIMP for such high resolutions is high.
Model
Refinement Steps
CE % Mean CE % Median VFE % Mean
NITO w/o Optimization
7.46 (+)
8.51 (-)
1.06 (-)
NITO
5
2.55 (-65.8%)
0.56 (-93.42%)
0.84 (-20.75%)
NITO
10 0.78 (-89.54%)
0.59(-93.07%
0.28 (-73.58%)
Beyond looking at different resolutions and shapes in the training data, we also performed limited experiment the use of NITO at resolutions beyond the training data. For this study, we use limited test set of 10 randomly selected examples from the test data with resolution of 256x256 and run a multi-grid GPU accelerated solver on the same problem at a resolution of 5000x5000 pixels (25M clements). On average, to run the optimizer for 300 iterations, on a RTX 4090 GPU requires over an hour to run (6, 676.06 seconds), which is a rather long time even with GPU acceleration and advanced multi-grid solvers. We then use the NITO trained on all data to sample at this resolution, and perform a few steps of optimization, using the same solver and summarize the quantitative results in Table 6. The quantitative results show that NITO's performance degrades slightly in comparison lo the native resolution it was trained on, however, we can still see that warm-starting the optimizer with NITO sampled at high-resolution does yield results very close to native resolution SIMP optimizer. However, we must make this point that despite the compliance difference being minimal, the level of detail in few-step optimized NITO samples is not like the detail we see in the native STMP ions In Appendix E we visualize an example of this and show this clearly. However, we can see that NITO can be sampled at much higher resolution and much lower cost in comparison to even one iteration of the solver at this resolution (Table 4) which means that given the scalability of NITO this framework serves a platform for developing models at very high resolutions contingent upon the data being available. Finally, we also present the compute time for each method in Table 4. Note that for 5000x5000 resolution we use a more advanced SIMP solver that runs on GPU instead of CPU and uses a multi-grid solver to accelerate the process. We can see that NITO's inference scales significantly better than the FEA based solver with
5 Conclusion & Limitations
In this paper, we introduce Neural Implicit Topology Optimization (NITO), novel resolution-free and domain- agnostic deep learning framework for topology optimization. Using a neural implicit field representation and MLP (BPOM) to represent loads and supports, NITO can be effortlessly
our Boundary Point trained and deployed on mixed domain shapes and resolutions. Thanks to its few-step optimization based refinement, NITO siguificantly outperforms stale-of-the-art models in topology optimality across multiple resolutions and domains. Notably, NITO is much faster and has significantly fewer parameters than previous state of the art models. As such, NITO offers a solution to high-dimensional problems that were previously insurmountable with CNN- methods. Furthermore, NITO's scalability and generalizability offer a robust foundation for future models in topology optimization and other physics-based problems.
14

---

Limitations: While NITO presents significant advancements in topology optimization, it is not without shortcomings. Firstly, NITO is not generative model, which means that the outputs of NITO are deterministic and not diverse for a given set of constraints. This limits NITO's ability to generate solutions that enable exploration of the design space. This characteristic could potentially hinder NITO's 's efficacy in addressing problems outside training distribution, given the lack of generative mechanism that enhances performance in entirely new tasks (Soe the Appendix for a broader discussion). Future work could be devoted to addressing this matter, potentially by leveraging recent advances in generative frameworks for neural implicit fields (You et al., 2023; Kosiorek et al., 2021). In addition, NITO generates topologies that lack sufficient detail before refinement, making it heavily reliant on its refinement step. Future improvements could aim to refine NTTO's training procedures and architecture to reduce the reliance on direct optimization. Beyond this, we can see that NITO and prior methods for direct optimization using deep learning models for lopology optimization are often trained and work best for lower fidelity structures. Despite our working moving beyond the existing 64x64 limits of prior works, the reality remains that NITO even at 256x256 resolution, is no where near the very high fidelity optimization of structures seen using conventional methods sometimes extending to billions of elements Liu et al. (2018); Aage et al. (2017). Despite this, the framework of NITO can be sampled at any resolution, and most importantly trained in scalable fashion for very high resolutions as discussed before. This means that despite the fact that in this paper we have not explored very high gigapixel/ /voxel resolutions we have introduced a generalizable and scalable framework which can enable this kind of extension in the future, and we intend to explore few-shot training on very high resolution samples and studying the effectiveness and data requirements for such resolutions.
15

---

References
Niels Aage, Erik Andreassen, Boyan S. Lazarov, and Ole Sigmund. Giga-voxel computational morphogenesis for structural design. Nature, 350(7674):84-86, Oct 2017. doi: 10.1038/naii 1000088/aaur22311...
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, llge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Pierre Baque, Edoardo Remelli, Francois Fleuret, and Pascal Fua. Geodesic convolutional shape optimization. In International Conference on Machine Learning, pp. 472 481. PMLR, 2018.
Mohammad Mahdi Behzadi and Horca T. Iics. Gantl: Toward practical and rcal-time topology optimization with conditional generative adversarial nelworks and transfer learning. Journal of Mechanical Design, 144(2), 12 2021. ISSN 1050-0472. doi: 10.1115/1.4052757 URL https://doi org/10 ottg/...11//11......... 021711.
1ww./1111111111 40527557
Martin P Bendsøe. Optimal shape design as a material distribution problem. Structural optimization, 1: 193 202, 1989.
Martin Philip Bendsøe and Noboru Kikuchi. Generating optimal topologies in structural design using homogenization method. Computer Methods in Applied Mechanics and Engineering, 71 -24, 1988. ISSN 0045-7825. doi:
URL http:://ww..sieneeddii..
Stephen Boyd and Lieven Vandenberghe. Convex optimization, Cambridge university press, 2004.
Aaditya Chandrasekhar and Krishnan Suresh. Toun:: Topology optimization using neural networks. Structural and Multidisciplinary Optimization, 63(3):1135-1149, 3 2021. ISSN 1615-147X. doi:
(4.1007//00158-000-0077270 CRL http:://inn..srrinngee. cww/10.cccco.ccote 1ttp:/10..100/0001111100 0000000000018000000774----88
Hongrui Chen, Aditya Joglekar, and Levent Burak Kara. Topology Optimization Using Neural Networks With Conditioning Field Initialization for Improved Efficiency. Journal of Mechanical Design, 146(6))061102, 12 2023. ISSN 1050-0472. doi: 10.1115/1.404131. URL https://doi. org/10 10 1115/1. 4064131.
Wei Chen and Faez Ahmed. Padgan: Learning to generate high-quality novel designs. Journal of Mechanical Design, 143(3))031703, 2021.
Wei Chen and Mark Fuge. Béziergan: Automatic generation of smooth curves from interpretable low- dimensional parameters, 2021. URL attps://arxxvvorr//ae . 08871.
Yilun Du, Katie Collins, Josh Tenenbaum, and Vincent Sitzmann. Learning signal-agnostic mani- folds of neural fields. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. $320-8831, Cur- ran Associates, Inc., 2021. URL httpss:/prroceednngg www.eedddnnnnn.nnnp....e.. newrips....... hwtpspeeepppeeteeeeeeee ftlles/paper2222//11e pdf.
Nikita Durasov, Artem Lukoyanov, Jonathan Donier, and Pascal Fua. Debosh: Deep bayesian shape oplimization, 2023. URL https://arxii: http::::/axxxi.or//bbb....
13337.
Nikita Durasov, Doruk Oner, Jonathan Donier, Hieu Le, and Pascal Fua. Enabling uncertainty estimation iterative neural networks. arXiv preprint arXiv:2403.16732. 2024.
David Erzmann and Sören Dittmer. Equivariant neural operators for gradient-consisten topology optimization. Journal of Computational Design and Engineering, 11(3):91 100, 04 2024. ISSN 2288-5048. doi: 10.1093/ jede/qwac039. URL https://doi. jttp://o.o.rrrooooooo//o//////aa htt.:/01/////////////////
Giorgio Giannone and Faez Ahmed. Diffusing the optimal topology: generative optimization approach. arXiv preprint arXiv:2303.0970) 2023.
16

---

Giorgio Giannone, Akash Srivastava, Ole Winther, and Facz Ahmed. Aligning optimization trajectories with diffusion models for constrained design generation. arXiv preprint arXiv:2305.18411 2023.
Amin Heyrani Nobari, Wei Chen, and Faez Ahmed. Pedgan: A continuous conditional diverse generative adversarial network for inverse design. ACM, Aug 2021. ISBN 9781450383325, doi: 10.1145/3447544834477444 URL http://dx: www.oo/..ooooooooooooooo.oo.. owg/10..///. 1145/1444//44.... 344477444..46444....
Jiangbei Hu, Ying He, Baixin Xu, Shengfa Wang, Na Lei, and Zhongxuan Luo. I[tonir: Iteration-free topology optimization based on implicit neural representations. Computer-Aided Design, 167:103639, 2024. ISSN 0010-4485. doi: https://doi.rrg//00000/// com/science/ http://concc/artice/pppiiie///.e /ttp://r....ee////00044300077770
URL
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization, 2017.
William Hunter et al. Topy-topology optimization with python, 2017.
Aditya Joglekar, Hongrui Chen, and Levent Burak Kara. Dmf-tomm: Direct mesh-free topology op- timization using neural networks. Engincering with Computers, Oct 2023. ISSN 1435-5663. doi: 10.1007/400366--22--000- URL https://doo http:://ddi..oo///////// 100.1000000006000--10000000
Adam R. Kosiorek, Heiko Strathmann, Daniel Zoran, Pol Moreno, Rosalia Schneider, Soña Mokrá, and Danilo Rezende. Nerf-vae: A geometry aware 3d scene generative model, 2021.
Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1 97, 2023.
Baolong Li, Congjia Huang, Xin Li, Shuai Zheng, and Jun Hong. Non-iterative structural topology optimization using deep learning. Computer-Aide Design, 115:172-188 2019. ISSN 0010-4485. doi:
com//ccience/arrtccl... http:///ddol.rr//11011///////3 000.0.00.00........00........... 2019.05.03.. URL https: /twp::://wwwwww.. wtt////wwwwweeeeeeeetttttt/ htt.://wwaedrreee..cccccceeeeeee pi1//90004448593018BXx
Haixiang Liu, Yuanming Hu, Bo Zhu, Wojeiech Matusik, and Eftychios Sifakis. Narrow-badd topology
optimization sparsely populated grid. ACM Trans. Graph., 37(6), December 2018. ISSN 0730-0301. doi: 10. 00.1145//272227.32755555
URL https://doi. http::///oi.oor/10... 1145/3272227 3275011.
Kai Liu and Andrés Tovar. An efficient 3d topology optimization code written in matlab. Structural and Multidisciplinary Oplimization, 50:1175 1196, 2014.
Xu Ma, Can Qin, Haoxuan You, Haoxi Ran, and Yun Fu. Rethinking network design and local geometry in point cloud: A simple residual mlp framework, 2022.
F. Mazé and F. Ahmed. Diffusion models beat gans topology optimization. In Procoodings of the AAAI Conference on Artificial Intelligence (AAAI), Washington, DC, 2023. URL https: 09591.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Zhenguo Nie, Tong Lin, Haoliang Jiang, and Levent Burak Kara. Topologygan: Topology optimization using generative adversarial networks based on physical fields over the initial domain. Journal of Mechanical Design, 143(3), 02 2021a. ISSN 1050-0472. doi: 10.1115/1.40490 URL https://doi.orgg 1115//. 4049533. 031715.
Zhenguo Nic, Tong Lin, Haoliang Jiang, and Levent Burak Kara. Topologygan: Topology optimization using generative adversarial networks based on physical fields over the initial domain. Journal of Mechanical Design, 143(3), 2021b.
17

---

Sangeun Oh, Yongsu Jung. Scongsin Kim, Ikjin Lee, and Namwoo Kang. Deep generative design: Integration of topology optimization and generative models. Journal of Mechanical Design, 141 09 2019. ISSN 1050-0472. doi: 10. 10.1115/1.4044229, URL https://dois org/ 1111/111111422 111405.
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Dcepsdf: Learning continuous signed distance functions for shape representation. In Proceodings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
Corey Parrott, Diab Abueidda, and Kai A James. Multi-head self atf-attennion gans for multiphysics topology optimization. In ATAA AVIATION 2022 Forum, pp. 3726, 2022.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. arXiv preprint arXiv:1709.07871, 2017.
Sharad Rawat and M.-H. Herman Shen. A novel topology optimization approach using conditional deep learning. CoRR, (bs/1991.04859, 2019. URL bttt://arxtie warg/abs/190/ 04859.
Lyle Regenwelter, Amin Heyrani Nobari, and Faez Ahmed. Deep generative models in engineering design: A review. Journal of Mechanical Design, 144(7):071704, 2022.
Lyle Regenwetter, Akash Srivastava, Dan Gutfreund, and Faez Ahmed. Beyond statistical similarity: Rethinking metrics for deep generative models in engineering design. Computer-Aided Design, 165:103609, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
G. 1. N. Rozvany, M. Zhou, and Torben Birker. Generalized shape optimization without homogenization. Structural optimization, 4:250-252, 1992.
Jana I Saadi and Maria C Yang. Generative design: reframing the role of the designer in carly-stage design process. Journal of Mechanical Design, 145(4):041411, 2023.
Suryanarayanan Manoj Sanu, Alejandro M. Aragon, and Miguel A. Bessa. Neural topology optimization: the http.//240...3999
good, the bad, and the ugly, 2024. URL
Conner Sharpe and Carolyn Conner Seepersad. Topology design with conditional generative adversarial networks. Volume 2A: 45th Design Automation Conference, 08 2019. doi: 101115/DETC2201--7733.. URL 97833. V02AT03A062.
https://doi. org/ /011000021211 1
Seungyeon Shin, Dongju Shin, and Namwoo Kang. Topology optimization via machine learning and deep learning: A review. Journal of Computational Design and Engineering, 10(4):1736-1760) 1766, 2023.
Ole Sigmund and Kurt Maute. Topology optimization approaches: A comparative review. Structural and Multidisciplinary Optimization, 1866):1031-1055, 2013. ISSN 1615-147X. doi: 10.1007/500158--11-00777
Vincent Sitzmann, Julien N. P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions, 2020.
Ivan Skorokhodov, Savva Ignatyev, and Mohamed Elhoseiny. Adversarial generation of continuous images, 2021.
Binyang Song, Rui Zhou, and Faez Ahmed. Multi-modal machine learning in engineering design: A review and future directions. arXiv preprint arXiv:2302.10909 2023.
Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains, 2020.
18

---

Erik A. Träff, Anton Rydahl, Sven Karlsson, Ole Sigmund, and Niels Aage. Simple and efficient gpu accelerated topology optimisation: Codes and applications. Computer Methods in Applied Mechanics and Engineering, 410:116043, 2023. ISSN 0045-7825. doi: https://www.
//tp///j...maaam 2aa..002..110 116043. URL
sciencedirect https///www.cceeccdrrettttttttt.
Jonathan Viquerat, Jean Rabault, Alexander Kuhnle, Hassan Ghraieb, Aurélien Larcher, and Elic Hachem. Direct shape oplimization through deep reinforcement learning. Journal of Computational Physics, 428: 110080, 2021. ISSN 0021-9991. doi: https://dbb.org//1.000//// je0.00i......22.000000000 URL attps://www. sciencedirect. wcd.meeaaeee...o/sstttaaaaaataaa
Chao Wang, Zhi Zhao, Ming Zhou, Ole Sigmund, and Xiaojia Shelly Zhang. A comprehensive review of educational articles on structural and multidisciplinary optimization. Structural and Multidisciplinary Optimization, 64(5)::827-22880 Nov 2021a. ISSN 1615-1488. doi: 00.0007/000158-001-0000555 URL
10.
Chao Wang, Zhi Zhao, Ming Zhou, Ole Sigmund, and Xiaojia Shelly Zhang. A comprehensive review of educational articles on structural and multidisciplinary optimization. Structural and Multidisciplinary Optimization, (88):2227-2288, 11 2021b. ISSN 1615-147X. doi: 10.1007/s00155-002.1
Dalei Wang, Cheng Xiang, Yue Pan, Airong Chen, Xiaoyi Zhou, and Yiquan Zhang. A deep convolutional neural network for topology optimization with perceptible generalization ability. Engineering Optimization, 54(6):973 988, 2021c. doi: 2021. 1902999.
URL https: //do:::///..//// org/100.1000/0002225
Zhen Wei, Edouard Dufour, Colin Pelletier, Pascal Fua, and Michaël Bauerheim. DiffAirfoil: An Efficient Novel Airfoil Sampler Based Latent Space Diffusion Model for Acrodynamic Shape Optimization. doi:
http://aa..aaaa..g...//.. /0..4//6.2244777 URL attps://arz aiaa. http:///////////...05555553 2514/6. 2.2..244....222555.... 2024-3755.
Rebekka Woldseth, Niels Aage, J Andreas Bærentzen, and Ole Sigmund. On the use of artificial neural networks in topology optimisation. Structural and Multidisciplinary Optimization, 65(10):294, 2022.
Rundi Wu, Chang Xiao, and Changxi Zheng, Deepcad: A deep generative network for computer-aided design models, 2021.
Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. Neural fields in visual computing and beyond, 2022.
Tackgeun You, Mijeong Kim, Jungtaek Kim, and Bohyung Han. Generative neural fields by mixtures of neural implicit functions, 2023.
Sihyun Yu, Jihoon Tack, Sangwoo Mo, Hyunsu Kim, Junho Kim, Jung- Woo Ha, and Jinwoo Shin. Generating videos with dynamics-aware implicit generative adversarial networks, 2022.
Yonggyun Yu, Tacil Hur, Jaeho Jung, and In Gwun Jang. Deep learning for determining a near-optimal topological design without any iteration. Structural and Multidisciplinary Optimization, 59(3):787 2019. doi: 10.1007/s000 1011007/00115-00--2000000
URL //toii http::://boi.org//10..00 http:///..................-00000
Jonas Zehnder, Yue Li, Stelian Coros, and Bernhard Thomaszewski. Ntopo: Mesh-free topology optimization using implicit neural representations. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and
J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 10368- 10381. Curran Associates, Inc., 2021. URL https: https://proces: www.eeiiinnnnnnnnss..... httpipsscccppppefiiee/pppppepp files/paper/ (t22//illee/ddd9a5//2222 htw.eeaaaaaaaaaaaaaaeeeaeaaa pdd.
19

---

A Minimum Compliance Topology Optimization Background
Here, we will briefly discuss the minimum compliance topology optimization problem and clarify some of the terminology and nomenclature used in the main body of the paper. In minimum compliance problems, the objective of the problem is to minimize the deformation under load, i.e,, the compliance of a structure while using a specified amount of material. To compute compliance we must first look at the physics behind the problem and how we model compliance.
A.1 The Physics Model: Linear Elasticity
In this work and most structural topology optimization problems, the physics model we use to model the behavior of a structure under load is the linear clasticity model. In lincar clasticity, an assumption is made that a given structure is made up of a material that has the same elastic properties regardless of shape and stress applied the material. In simple ID setup, this is similar to having spring whose spring constant remains the same regardless of dcformation and load applicd, and the size of the spring (i.c., Hooke's law). When extending this to structures in a continuum, arrive at the following partial differential equation (PDF), also known as the linear elasticity (a special case of the Navier-Cauche equation with constant linear elastic behavior):
E 2(1+
EV + (1++)(1--2)
E 2(1
Vuu+
V(V u) +f= pü,
(7)
Where u (ū is second derivative with respect to time acceleration) is the displacement field (a vector field) which determines how much the structure displaces from equilbrium/initia position, f is forcing term (also vector field), which determines the load applied to the martial in the domain of the problem, E refers to the young's modulus of the material a physical constant dictating how stiff a material is, is the Poisson ratio another physical constant dictating how the material deform is directions perpendicular to the load, and p is the density of the material. In static or equilibrium analysis, we are interested in the steady-state deformation under load, which allows us to set the right hand side of this equation to zero:
E Vuu+ 2(1+V)
EV (1I+ 2v)
E
+
V(V-u) +f==
2(1 + v)
This equation what we will solve for our problems to determine the displacement under load and compliance.
A.2 Finite Element: Solving The Linear Elasticity PDE
To determine the compliance of a given structure, we must solve the PDE in Equation 8. This is not possible in closed form, as such, usually this equation is solved numerically. Specifically, the finite-clement method is typically applied to solve this equation. In finite element analysis (FEA), we will discretize the domain (often denoted as Q2 of the problem into a mesh, with a finite number of elements and vertices (nodes). FEA then allows us to integrate the PDE in Equation 8 for each element and determine the overall stiffnesss matrix of given structure with right hand side vector dictated by the forcing term f. The details of how this is done in finite elements is outside the scope of this brief discussion, however, the result of FEA is that given material propertics E and one can obtain a lincar system of equations of the form:
Kd=f
where d is the displacement vector (can be assembled into the displacement vector field in the original PDE) which determines the displacement in each node of given mesh and f is the force load applied to the structure. Note that solving the lincar clasticity PDE without any boundary conditions is not possible, as such, when performing FEA we will also set the boundary condition of a given problem. In the case of minimum compliance TO the boundary conditions are simply the locations of the supports (i.e., points in space the structure fixed and not allowed to move). These supports can fix a set of nodes in the mesh and dictate no movement in all or any directions (see Figure 2. Such supports are introduced to the system by adjusting the stiffness matrix in FFA.
20

---

A.3 Optimizing Compliance
So far discussed how we solve the linear elasticity PDE using PEA, but now must find way to optimize the distribution of finite amount of material within mesh/ fossh/dome domain. In the linear elasticity equation, the stiffness of the material is set to be constant E. However, to find an optimal shape within domain, we allow variation in the Young's modulus based on how much material is placed in an clement of the discretized domain (mesh). Specifically, we define the field p which determines the density of material in space and adjust the stiffness of each element by setting E Eg + p(E Eo), where Eg is a very small stiffness for void/empt elements (needed to prevent singular K in FEA). By doing this, we are now able to obtain the compliance of structure by setting the density value for each element in mesh. It is also important to note that in most cases we do not want. continuous values for rather we are interested in binary values determining which elements have material and which ones do not. However, this leads to a combinatorial optimization problem which is not practical to solve in most cases. To overcome this, the Solid Isotropic Material with Penalization (SIMP) method, introduced by Bendsøe (1989) is usually used. In SIMP we model density as a function of design variables ø which dictate the density with a non-lincar exponent penalty p, which is usually set to In this setup P = (dt)P, hence E Eg + (P Eo) for element. in the mesh. Based on this and the fact that we want to minimize compliance, f" u, the optimization problem becomes:
min f= s. k Eeen (o) SV
(10)
Pmin 500 S Pmax E
where V is the target volume/mass of material we desire in the final structure, and Φmin is the lower bound on the design variables, often O, and Φmer is the upper bound, often 1. This formulation is the common formulation of the minimum compliance problem which we also use in this work. The goal in NITO is to predict directly based on the loads, supports, and target volume fraction.
21

---

B Additional Experiments
Here, we will discuss some further experiments and discuss limitations of our approach.
B.1 Out of Distribution Experiments
Table 7: Quantiative Evaluation on out-of-distribution 64x64 datasets. w/ G: using a classifier and/or regression guidance. FS-SIMP: Few-Steps of optimization. Error.
CE: Compliance Error. VFE: Volume Fraction
Model
Train Res. FS-SIMP
CE
Mean
CE % Median VFE % Mcan VFE % Modian
out-of-distro
NeuralField
64
73.24
12.81
8.12
6.87
TifoDiff
64
8.57
1.14
1.14
0.97
TopoDiff w/< G
64
7.79
1.26
1.21
1.00
TopoDiff
64
6.20
1.07
0.93
0.55
TopoDiff w/C G
64
5.44
1.05
0.89
0.55
NITO (ours)
9.33
2.37
2.22
1.32
TopoDiff
64
10
2.91
0.71
0.64
0.33
TopoDiff w/C G
64
10
2.25
0.71
0.63
0.35
NITO (ours)
64
10
6.38
1.43
1.55
0.85
Something that should be looked at when it comes to these models is their performance generalizability to problems that are very different from the distribution data used for training. To do this, we test the performance different models on an out-of-distribution test set for the 64x64 dataset. These results are presented in Table for TopoDiff and NITO. As it can be seen, NITO's performance has significantly deteriorated on out-of-distribution data. This can be attributed to two matters. The first and most impactful is the nature of these models. TopoDiff generalizes better when it comes to these out-of-distribntion tests given the fact that the model is generative in nature. This allows TopoDiff to handle out-of-distribution conditions better. To understand this we can look at generative models as sort of retrieval approach, which allows the models to generate detailed and high-quality samples by finding the most similar topologies from the training data even when faced with very different inputs. On the other hand, NITO a deterministic model that learns to map specific loads and supports to near-optimal density fields. This makes it rather challenging for models like NITO when comes generating topologies for unseen and very different inputs since the mapping is deterministic one. This highlights the importance of future work focusing transforming our framework to be generative, which should be possible since many works have shown implicit neural approaches can be made generative (You et al., 2023). The second potential reason contributing to TopoDiff's better performance on out-of-distributtin samples is the use of physics- based fields for conditioning. This is because mapping stress fields to topologics makes easier to handle very different loads and supports since the stress fields may still be similar to samples the dataset. However, this creates the kind of limitation that we discussed before, as such is better to focus on making more robust generative schemes rather than using physical fields.
22

---

Ground Truth
V
NITO
V
Topo
Ground Truth
TopoDiff
Figure 5: Visual comparison of samples generated for the out-of-distributio test. Each row is labeled. Ground truth samples SIMP-optimized samples.
23

---

B.2 Further Examination Of Results & Outperforming The Optimizer
700
600
500
300
200
100
2
2
Compliance Error (%)
200
150
Count
100
50
Volume Fraction Error (%)
Figure 6: Distributions of volume fraction error (Boltom) and Compliance Error(Top). We see that NITO is capable of out- out-performing SIMP 19% of the time highlighted by yellow in the negative compliance errors (meaning better than SIMP) on the top histogram. This data for the 64x64 test set.
In Figure 6, we show the distribution of compliance error and volume fraction error for NITO on the 64x64 dataset. We see that the majority of volume fraction error is below 1% and small number of high error samples skew the average. Similarly, we see that the majority of the compliance errors for NITO are below 1% as well. Most notably, we observe that on 19.2% of the samples the test set NITO actually outperforms SIMP indicated by the negative compliance errors visible on the histogram. This rather interesting outcome where NITO is capable of doing better than the optimizer that the training data came from. In Figure 24, visualize few instances of this phenomenon. It can be seen that in some problems NITO actually comes up with different solution which, as it happens, out-performs SIMP, while in other instances the topologies are very similar and NITO has adjusted some of the finer details and redistributed the material differently to achieve better performance.
24

---

SIMP
NITO
Figure 7: Example topologies where NITO outperforms SIMP. It can be seen that in some instances NITO finds very different topologics that outperform SIMP and in some instances, NITO has removed some details and redistributed the material in a way that has improved performance.
25

---

C
Visualizations
Here we will provide set of different visualizations for each test case.
C.1 Direct Optimization Visualization
In this section provide some visualizations, demonstrating the effect of direct optimization on neural field-generated density fields. The main objective here is to showcase the importance and effectiveness of few steps of direct optimization seen in Figures 8 and 9, neural fields tend to have some smoothing and averaging in areas of high detail in the topology, which makes the baseline performance of neural fields worse. With only a small number of direct optimization iterations, however, we see that NITO can resolve the complex details effectively, showcasing why direct optimization
the generated samples which completes the NITO framework. As can be
crucial aspect of our framework.
In contrast, when we look at the effect of direct optimization on TopoDiff (Figures 10 and 11) see that in cases where the details predicted by TopoDiff are accurate, the optimizer does not provide much benefit, and when TifoDiff has not generated the correct details the generated topology is far from optimal which means that few steps of optimization do not provide significant benefits.
26

---

Ground Truth
Neural Fields
NITO (5)
NITO (10)
Ground Truth
Neural Fields
NITO (5)
NITO (10)
Figure 8: Visualization of the effect. of direct optimization samples generated by neural fields. The first column of each set of images shows the ground truth, the next column shows the raw predictions from neural fields, and the two columns that follow show the effects of 5 and 10 steps of direct optimization on the samples respectively. These samples are from the 64x64 test set.
27

---

Ground Truth
Neural Fields
NITO (5)
NITO (10)
Ground Truth
Neural Fields
NITO (5)
NITO (10)
Figure 9: Visualization of the effect. of direct optimization samples generated by neural fields. The first column of each set of images shows the ground truth, the next column shows the raw predictions from neural fields, and the two columns that follow show the effects of 5 and 10 steps of direct optimization on the samples respectively. These samples are from the 256x256 lest set.
28

---

Ground Truth
TopoDiff
TopoDiff
TopoDiff+
Ground Truth
TopoDiff
TopoDiff+5
TopoDiff- -10
Figure 10: Visualization of the effect of direct optimization samples generated by TopoDifT. The first. column of each set of images shows the ground truth, the next column shows the samples generated by TopoDiff, and the two columns that follow show the effects of and 10 steps of direet optimization on the samples respectively. These samples are from the 64x64 test set.
29

---

Ground Truth
TopoDiff
TopoDiff
TopoDiff+10
Ground Truth
TopoDiff
TopoDiff+5
TopoDiff 10
ATATA
Visualization of the effect of direct optimization samples generated by TopoDifT. The first. column of each set of images shows the ground truth, the next column shows the samples generated by TopoDiff, and the two columns that follow show the effects of and 10 steps of direct optimization on the samples respectively. These samples are from the 256x256 lest set.
Figure 11:
30

---

C.2 Generated Samples Visualizations
In this section, we provide visualizations of samples predicted by different configurations of our model on different problems.
1N.
Figure 12: Ground truth images from the 64x64 SIMP datasets, Images that follow visualize NITO generated samples for the same problems.
31

---

A
Figure 13: NITO generated topologies using a model trained on 64x64. Tested on the 64x64 data.
32

---

Figure 14: NITO generated topologies using a model trained on 256x256. Tested on the 64x64 data.
33

---

L7
A
Figure 15: NITO generated topologies using a model trained on both 64x64 and 256x256. Tested on the 64x64 data.
34

---

K A
Figure 16: Ground truth images from the 256x256 STMP datasets. Images that follow visualize NITO generated samples for the same problems.
35

---

NLKX
DA
Figure 17: NITO generated topologies using a model trained on 256x256. Tested on the 256x256 data.
36

---

A
Figure 18: NITO generated topologies using a model trained on 64x64. Tested on the 256x256 data.
37

---

NLAX
<
Figure 19: NITO generated topologies using a model trained on both 64x64 and 256x256. Tested on the 256x256 data.
38

---

D Dataset & Optimizer Details
Our experiments use dataset of SIMP-optimized topologies in unit square domain with size of 64x64 and 256x256 and rectangular domains with different aspect ratios including 64x48, 64x32, and 64x16. For each topology in the dataset, information about the loads, supports, and volume fraction are included. Furthermore, we include the stress and strain energy fields. Our dataset is similar to the one proposed by Mazé Ahmed (2023), however, we noted that the dataset used in prior works has been generated using an older version of SIMP, namely ToPy (Hunter al.; 2017).
This solver, although robust implementation of SIMP, does not implement the latest improvements to the SIMP algorithm and uses slower solver, which causes two issues. Firstly, the topologies that are used in the dataset proposed by Mazé & Ahmed (2023) are powerppefforming topologies in comparison to what the latest solvers produce, hence possibly overestimating the performance of these models in some cases. Secondly, the SIMP method itself which prior studies compared their inference time with, did not use the fastest solvers, making those comparisons also somewhat. inaccurate. To ensure that this is not the case in our studies,s we implement the SIMP optimizer from scratch in Python (The code for which will be publicly available), which performs the optimization using the latest and fastest implementation of the SIMP algorithm as far as the authors are aware (Wang et al., 2021b).
As such, we re-create the 64x64 dataset proposed by Mazé & Ahmed (2023) using our solvers and find that the resulting topologics using our mothod are significantly better performing in comparison to the prior datasets used by many other works of research (Giannone el al., 2023; Giannone & Ahmed, 2023; Mazé & Ahmed, 2023; Nie et al., 2021b). Given this, is safe to assume that the performance of the models in prior studies that compare to the inferior dataset may have been overestimated while also training models on power-performance samples. However, to allow for fair comparison, in the literature TopoDiff (Mazé & Ahmed, 2023), on our new dataset and rerun their experiments on this new dataset. However, we report the performance of other models as the original authors measured them overestimated or otherwise.
retrain the best-performing model
The 64x64 dataset includes 48,000 training samples and 1,000 test samples, which we use for testing our models. The 256x256 dataset includes 60,000 training samples and 1,800 samples for testing, and the other three domains have 29,000 samples each with 1,000 lest samples each. The figures that follow visualize some of the training samples for each dataset, In the following figures, include figures showing samples from all datasets.
39

---

Constrained in Constrained in
Applied Load
Figure 20: Random samples from the 64x64 dataset. Green arrows show the locations and directions of
the loads applied in each problem. Blue triangles indicate points that are constrained in x and red triangles indicate points that are constrained in y.
40

---

Constrained in
Constrained in X
Applied Load
Figure 21: Random samples from the 256x256 dataset. Green arrows show the locations and directions of
the loads applied in each problem. Blue triangles indicate points that are constrained in x and red triangles indicate points that are constrained in y.
41

---

Constrained in
Constrained in X Applied Load
Figure 22: Random samples from the 64x48 dataset. Green arrows show the locations and directions of
the loads applied in each problem. Blue triangles indicate points that are constrained in and red triangles indicate points that are constrained in y.
42

---

KZ P
Constrained in Y
Constrained in X Applied Load
Figure 23: Random samples from the 64x32 dataset. Green arrows show the locations and directions of
the loads applied in each problem. Blue triangles indicate points that are constrained in and red triangles indicate points that are constrained y.
1
T
Constrained in Y
Constrained in X Applied Load
Figure 24: Random samples from the 64x16 dataset. Green arrows show the locations and directions of
the loads applied in each problem. Blue triangles indicate points that are constrained in x and red triangles indicate points that are constrained y.
43

---

E
Sampling At 25 Million Pixels
SIMP Optimizer
NITO w/o Optimization
NITO(S)
NITO(10)
Figure 25: Comparing SIMP native resolution of 5000x5000 (25 Million Elements) NITO with and without direct optimization. We see that with optimization, the quality of samples increases, however, given NTTO has not seen any data at this resolution, the solution still lacks very fine detail compared to the native resolution solution of the optimizer. Regardless we saw in Table these topologies perform very well despite the lack of very fine grained features.
As we mentioned, we tested NITO at much higher resolutions and observed lower performance compared to resolutions in the dataset. This because NITO is not trained at these resolutions and simply lacks the fine-grained details needed at these resolutions. However, we saw that NITO is capable of scaling efficiently very high dctail sampling and even with less detail few steps optimization brings performance close the nalive resolution solution from the optimizer. However, here we present Figure 25 which shows that the with few steps of direct optimization. This means that suspected data at very high resolution is needed to truly scale to such highly detailed topologies. In future works, will be exploring how fine-tuning on few-shot examples at higher resolutions impacts this kind of experiment.
issue of lacking detail persists
44


Reviews:
[{"review_id": "QQpUEWBVIB", "reviewer": "Reviewer_uPWv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "S: The method is principled, well-motivated and appropriate for the problem at hand.\n\nS: The results are comprehensive and well-presented, and show the performance to be competetive without post-tuning, and SOTA with. \n\nW: The presentation assumes special domain knowledge, and the setting is not opened up sufficiently for general ML audience.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents an implicit neural representation (INR) network for learning 2D topologies. The method combines SIREN with fourier expansions, and uses FiLM to introduce conditioning by the topology parameters. The topology constraints are handled by a PointMLP type network. The overall network combines multiple architectures that are all well-motivated and appropriate for the problem. Finally, the method is also shown to work well with simulator post-refinement, and achieve better performance than diffusion based method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I am not familiar with this problem domain. The paper uses special terminology, which was not translated into common english. The math presentation is cursory and skips over details and definitions. The paper is likely easy to follow for a domain expert, but for an outsider the topic and material should be made more explicit and concrete.\n\nCan you explain what are we looking at in figs1+2: at first this looks like ink blobs. Are these some physical objects, like buildings? These seem 2D: why is modelling things in 2D relevant?\n\nThe figures need to be properly annotated. For instance, there are triangles and arrows that are not explained. The figure contents should be labelled, and coupled with the math symbols as appropraite.\n\nI’m a bit confused of the circularity of eq 1’s min(Fd) and Kd=F. This then becomes min Kd=d, which looks like an eigenvalue problem, but without the eigenvalues (ie. Kd=lambda*d). Can you give a bit more exposition on how should we interpret the Kd=d? Can you explain what K,F,D mean?\n\nAll math presentation and notation needs to be explicit, well-defined, self-evident and properly introduced. As an example, eq 1 is already confusing:\n- The domains and sizes of the variables are not defined\n- It’s not clear what the “design variables” mean, or what “problem domain” means. Can you give some helpful examples?\n- The Fd reads like a matrix-vector product that results in a vector, but the symbol f is not boldface. Is the result then vector or scalar? If the result is vector, what does “min” mean over a vector?\n- v is undefined\n- I don’t understand the ^e notation: is this an exponent or some indexing notation?\n- I don’t understand the basic terminology here, eg. “load”, “displacement”, “nodal”, “element”, etc.\n- What is the coordinate space here?\n\nThe treatment of rho’s is quite confusing to me. We begin with rho(phi), then later have rho(x), then hat{rho}(x|C,theta), then rho(x|C). Why do you need so many rho’s, and what does each of them represent? What does the \\hat, phi and theta mean as arguments of rho? In eq 3 we have theta on both sides: what does this mean?\n\nThe BPOM description is vague, and it should be similarly mathematical as in eqs 5+6. PointMLP already uses pooling, so how is your pooling strategy different and why?", "broader_impact_concerns": "No issues.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZTYav4NSa7", "reviewer": "Reviewer_idtH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n\n* The paper contributes to the field of learning-based physics methods, particularly topology optimization, which is steadily gaining attention.\n* The authors provide a comprehensive introduction to topology optimization and modern existing approaches.\n* The proposed method, while conceptually simple to implement, demonstrates significant improvements over other baselines.\n\n# Weaknesses\n\n* The paper primarily focuses on a specific type of topology optimization that aims to predict shapes based on a given set of conditions. Meanwhile, several works, such as [1, 2, 3, 4, 5], also operate within the domain of optimal shape generation under given conditions while simultaneously optimizing a physical property (e.g., aerodynamic drag).\n* One of the method’s key contributions is the use of a neural implicit field representation for shape prediction, allowing for resolution-independent predictions. While this is a notable advantage, existing methods like Neural Operators [6] were designed precisely for this reason and are provably capable of generating predictions irrespective of resolution.\n* Additionally, the proposed conditioning procedure does not ensure that the imposed conditions are satisfied in the final predictions. This issue could potentially be addressed with an additional SIMP iteration, but without it, the constraints remain unmet.\n\n[1] Baque, Pierre, et al. \"Geodesic convolutional shape optimization.\" ICML 2018.\n\n[2] Durasov, Nikita, et al. \"Debosh: Deep bayesian shape optimization.\" arxiv 2021.\n\n[3] Durasov, Nikita, et al. \"Enabling Uncertainty Estimation in Iterative Neural Networks.\" ICML 2024\n\n[4] Viquerat, Jonathan, et al. \"Direct shape optimization through deep reinforcement learning.\" Journal of Computational Physics\n\n[5] Wei, Zhen, et al. \"Diffairfoil: An efficient novel airfoil sampler based on latent space diffusion model for aerodynamic shape optimization.\" AIAA Aviation Forum 2024.\n\n[6] Kovachki, Nikola, et al. \"Neural operator: Learning maps between function spaces with applications to pdes.\" JMLR 2023", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors propose a learning-based topology optimization method called NITO. At its core, the method introduces three ideas to improve topology optimization in terms of speed and quality compared to existing approaches: 1) the proposed Boundary Point Order-Invariant MLP (BPOM) approach effectively integrates conditioning for the predicted shapes, such as boundary conditions and loads, 2) the authors use a neural implicit field representation, enabling NITO to synthesize topologies of any shape or resolution, and 3) finally, we introduce an inference-time refinement step using a few steps of gradient-based optimization, allowing NITO to achieve results comparable to direct optimization methods.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* [Strengthen] Section 1: Replace “deep learning issues have struggled with several issues” with “deep learning approaches have struggled with several issues.”\n\n* [Strengthen] Section 2.1: Correct “folume” to “volume.”\n\n* [Strengthen] Section 2.2: Extend the discussion of existing learning-based topology optimization approaches, including relevant works such as [1, 2, 3, 4, 5].\n\n* [Strengthen] Consider discussing the neural operator [6] approach in the context of the considered task, specifically addressing why it was not used instead of neural fields.\n\n* [Strengthen] Clarify how exactly the proposed method ensures the fulfillment of the imposed conditions. It would be beneficial to provide more details in the method and experimental sections.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bsEtTvOG6q", "reviewer": "Reviewer_TmN7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is very well-written and easy to follow. I enjoyed reading it and did not spot major technical issues. Incorporating a resolution-free, neural-implicit surface representation into the standard topology optimization problem is a valuable proposal. Although some recent works have tackled similar settings before, the idea is still timely and can potentially spark some interesting downstream applications.\n\nI feel the major weakness of this work is that the problem size (2D with 256 x 256 grid resolution) is relatively small compared with what classic TO methods tackle, e.g., giga-voxel 3D grids in [1, 2]. A neural implicit surface representation supports “infinite” grid resolution, and a sweet spot of such representation would be their potential to express delicate geometric details that would have needed large numbers of voxels in traditional grid-based representation. However, the proposed network seems to capture only relatively coarse geometric features. I can agree that this work seems superior to existing deep-learning-based TO methods. Still, their benefits over classic TO methods in terms of speed, optimality, and expressiveness on larger-scale problems seem unclear.\n\n[1] Liu et al “Narrow-Band Topology Optimization on a Sparsely Populated Grid.” ACM Transactions on Graphics 2018.\n\n[2] Aage et al “Giga-voxel computational morphogenesis for structural design.” Nature 2017.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a neural network model for topology optimization (TO) problems with minimum compliance objectives. Specifically, the model maps the topology optimization problem setup (e.g., load and boundary conditions) to a neural-implicit representation of material distribution in the domain, which is further fine-tuned by standard SIMP optimization. The paper demonstrates its efficacy by comparing it with existing neural TO methods on 2D TO examples at up to 256 x 256 grid resolution.\n\nThe main contribution of this work includes the network model itself and the experiments that confirm its efficacy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- I suggest the paper adds reviews of and discussions about classic TO methods to position this work among them more properly. [1, 2] and their related works could be a good starting point. In particular, I suggest the paper reconsiders its claims about SIMP’s and this work’s pros and cons on large-scale TO problems.\n- It is not obvious to me that Eq. 4 is “the same as the output of the SIMP optimizer.” It would be great to provide a derivation of it.\n- “Notably, solving the linear system of the equations scales cubically (O(n3 )) with the number of nodes.” This seems a bit misleading. O(n^3) is the time complexity of solving a dense linear system naively, but K is typically a sparse and (semi-)positive symmetric matrix. In practice, TO solvers can benefit from conjugate gradient, multigrid, or [1] that are much faster than O(n^3).", "broader_impact_concerns": "I don’t see any concerns in this category.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In a nutshell, the manuscript has introduced a reasonable method for topology optimization by leveraging Neural Implicit Fields. While the technical novelty has not been highly pronounced,  the engineering pipeline turns out to be useful for a number of tasks related to the increasingly popular field of physics-based machine learning.
Decision: Accept (poster)