Geometry-Informed Neural Networks
Authors: Arturs Berzins
Andreas Radler
Eric Volkmann
Sebastian Sanokowski
Sepp Hochreiter
Johannes Brandstetter
Venue: ICML 2025 poster (2025)
Keywords: geometry
implicit neural representation
neural fields
theory-informed learning
geometric deep learning
physics-informed neural networks
generative design
Primary area: applications->everything_else
TL;DR: We introduce GINN -- a framework for training shape-generative neural fields without data by leveraging design constraints and avoiding mode-collapse using a diversity loss.

Abstract: Geometry is a ubiquitous tool in computer graphics, design, and engineering. However, the lack of large shape datasets limits the application of state-of-the-art supervised learning methods and motivates the exploration of alternative learning strategies. To this end, we introduce geometry-informed neural networks (GINNs) -- a framework for training shape-generative neural fields without data by leveraging user-specified design requirements in the form of objectives and constraints. By adding diversity as an explicit constraint, GINNs avoid mode-collapse and can generate multiple diverse solutions, often required in geometry tasks. Experimentally, we apply GINNs to several problems spanning physics, geometry, and engineering design, showing control over geometrical and topological properties, such as surface smoothness or the number of holes. These results demonstrate the potential of training shape-generative models without data, paving the way for new generative design approaches without large datasets.

Geometry-Informed Neural Networks
Arturs Berzins *
Andreas Radler" 11 Eric Volkmann" *
Sebastian Sanokowski
Sepp Hochreiter
Johannes Brandstetter 13
Abstract
problem descriptions, such objectives and constraints. Previous works for PDEs (Raissi et al., 2019), molecular science (Noé et al., 2019), and combinatorial optimization (Bengio et al., 2021) demonstrate these can suffice to train models even in the absence of any data. The success of these data-free approaches motivates an analogous attempt in geometry, raising the question: it possible to train a shape-generative model on objectives and constraints alone, without relying any data?
Geometry is ubiquitous tool in computer graph- ies, design, and engineering, However, the lack of large shape datasets limits the application of state-of-the-art supervised learning methods and motivates the exploration of alternative learning strategies. To this end, we introduce geometry- informed neural networks (GINNs) - frame- work for training shape-generative neural fields without data by leveraging user-specified design requirements in the form of objectives and con- straints. By adding diversity as an explicit con- straint, GINNs avoid mode-collapse and can gen- erate multiple diverse solutions, often required in geometry tasks. Experimentally, we apply GINNs to several problems spanning physics, geometry, and engineering design, showing control over ge- ometrical and topological properties, such as sur- face smoothness or the number of holes. These results demonstrate the potential of training shape- generative models without data, paving the way for new generative design approaches without large datasets.
We address this question by introducing rcometry-infornee neural networks or GINNs. GINNs are trained to satisfy specified design requirements and to produce feasible shapes without any training samples. A GINN solves a structural optimization problem using neural fields, which offer de- tailed, smooth, and topologically flexible geometry repre- sentations, while being compact to store. This setup is analogous to physics-informe a high number of varied constraints: differential, integral, geometrical, and topological.
neural networks but with
In contrast to both physics-informed neural networks and classical structural optimization, GINNs allow to generate multiple solutions by enforcing solution diversity as ex- plicit constraint. This is of high interest when applied to problems with solution multiplicity, e.g., induced by under- determinedness or mear-optimality common in geometry problems. Connecting back to our research question, the proposed framework allows us to train neural fields that sat- isfy user-specified design constraints, and by adding diver- sity explicit constraint, we can generate multiplicity of solutions. GINNs can thus be used as chape-generativy models trained purely on constraints and without data.
1. Introduction
Recent advances in deep learning have revolutionized fields with abundant data, such as computer vision and natural language processing. However, the scarcity of large datasets in many other domains, including 3D computer graphics, design, engineering, and physics, restricts the use of ad- vanced supervised learning techniques, necessitating the exploration of alternative learning strategies.
We take several steps to demonstrate GINNs experimen- tally. We formulate a tractable learning problem using con- strained optimization and by converting constraints into differentiable losses. We demonstrate the generality of the framework by solving several tasks, including an under- determined PDE, geometric optimization, and engineering design (Figures 4, 5). Figure I illustrates the task of de- signing a jet-engine lifting bracket, or geometrically - con- necting cylindrical interfaces within the given design region. We show different GINNs trained with various smoothness and topology requirements, illustrating the robustness of the
Fortunately, these disciplines are often equipped with formal
'Equal contribution 'LIT AI Lab, Institute for Machine Learn- ing, JKU Linz. Austria *Part of this work was done while at SIN- TEF and Department of Mathematics, University of Oslo, Nor- way 'Emmi AI GmbH, Linz, Austria. Correspondence to: Arturs Berzins <berzins@ @ ml jku.at>.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Ecometry-Informeer Neural Networks
Figure I: We train geometry -inforredd neural networks to produce shapes satisfying geometric design requirements. For example, we generate parts that connect the cylindrical interfaces within the sketched design region depicted on the left. To highlight the user's 's control over the problem and the solutions, we specify different additional requirements on the number of holes and surface smoothness. By complementing the design requirements with a diversity constraint, we can train a Stape-generative model without data as further illustrated in Figures and
constrained optimization approach and the user's control over the framework. Figure 6 shows GINN trained on the same task but with an additional diversity constraint. Sur- prisingly, this constraint induces a structured latent space, with generalization capacity and interpretable directions.
of model (Karpatne al., 2017). Such knowledge can be included in the model via equations, logic rules, hu- man feedback (Dash et al., 2022; Muralidhar et al., 2018; Von Rueden et al., 2021). Geometric deep learning (Bron- stein et al., 2021) introduces principled way to characterize problems based on symmetry and scale separation princi- ples, e.g. group equivariances or physical conservation laws.
We show that training chape-generaile networks using con- straints and objectives without data feasible learning strategy, paving the way for new generative design ap- proaches without large datasets. Our main contributions are summarized as follows':
Notably, most works operate in the typical deep learning regime, i.e., with abundance of data. However, in theory- informed learning, training on data can be replaced by training with objectives and constraints. More formally, one searches for a solution f minimizing the objective o()) s.t. f Kc, where defines the feasible set which the constraints are satisfied. For example, in Boltzmann generators (Noé al., 2019), is probability function parameterized by neural network to approximate an in- tractable target distribution. Another example is combinato- rial optimization where f E {0, 1 is often sampled from probabilistic neural network (Bello et al., 2016; Bengio et al., 2021; Sanokowski al., 2024).
1. We introduce GINN a framework for training shape- generative neural fields without data by leveraging de- sign constraints and avoiding mode-collapse using a diversity constraint.
2. We apply GINNs to several problems spanning physics, geometric optimization, and engineering design, in- vestigating the user's control, the robustness of the method, key framework choices, and emerging latent space structure.
Physics-informer neural networks (PINNs) (Raissi et al., 2019) a prominent example of neural optimization. In PINNs, f is a function that must minimize the violation of a partial differential equation (PDE), the initial and boundary conditions, and, optionally, some measurement data. Since PINNs can incorporate noisy data and are mesh- free, they hold the potential to overcome the limitations of classical mesh-based solvers for high- tigh-dimensional, paramet- ric, and inverse problems. This has motivated the study of the PINN architectures, losses, training, initialization, and sampling schemes (Wang et al., 2023). We further refer to the survey by Karniadakis et al. (2021).
2. Related Work
We begin by reviewing and relating three important facets of GINNs: theory-informed learning, neural fields, and generative modeling.
2.1. Theory-Inforinn Learning
Theory-informed learning has introduced paradigm shift in scientific discovery by using scientific knowledge remove physically inconsistent solutions and reducing the variance
Same as PINNs, GINNs use neural fields to represent the solution. Consequentially, we also observe that some best
'Code is available at https: petpss:/githieg http://ggttttuutte com.uul.cooommmmm///// jjku/ CTNNs-Gearen Matss-Geertrryy--n--nn innry---iffrrmmrr
2

---

SecomttryInforran Neural Networks
measured from a single shape, no ground truth is available and the problem is ill-defined (Atzmon & Lipman, 2020; Berger al., 2016).
Neural lells
Condinocal NFs.
Generative modeling
Regularization methods have been proposed to counter the ill-posedness in geometry problems. These include leverag- ing ground-truth normals (Atzmon & Lipman, 2021) and curvatures (Novello et al., 2022). minimal surface property (Atzmon & Lipman, 2021), and off-surface penalization (Sitzmann al., 2020). A central effort is achieve the distance field property of the scalar field for which many reg- ularization terms have been proposed: eikonal loss (Gropp et al., 2020), divergence loss (Ben-Shabat et al., 2022), di- rectional divergence loss (Yang et al., 2023), level-set align- ment (Ma al., 2023). or closest point energy (Marschner et al., 2023). The distance field property can be expressed as PDE constraint called eikonal equation VV f()) = 1, establishing relation of regularized INSs to PINNs (Gropp et al., 2020). Inductive bias. addition to explicit loss terms, the architecture, initialization, and optimizer can also limit or bias the learned shapes. For example, typical INSs are limited watertight surfaces without boundaries or self- intersections (Chibane et al., 2020; Palmer et al., 2022). ReLU networks are limited to piece-wise linear surfaces with sharp vertices and edges and, together with gradient descent, are biased toward low frequencies (Tancik et al., 2020). Fourier-featur encoding (Tancik et al., 2020), sine activations (Sitzmann et al., 2020), and wavelet activations (Saragadam al., 2023) allow for controlling this frequency bias. Similarly, initialization techniques are important to converge toward desirable optima (Sitzmann et al., 2020; Atzmon & Lipman, 2020; Ben-Shabat et al., 2022; Wang et al., 2023).
GINN
PINN
Belizmann genenstor
Thcory-informed leamning
Figure 2: GINNs build on neural fields, generative modeling, and iheory-informed learning.
practices of training PINNs (Wang et al., 2023) transfer to training GINNs. However, PINNs may suffer from ill nu- merical properties due to minimizing the squared residual of the strong-form different to classical PDE solvers (Rathore al., 2024; Ryck et al., 2024). In contrast, GINNs share the same underlying formulation and numerical properties as classical topology optimization methods. In addition to high number of various constraints (differential, integral, geometrical, and topological), geometric problems often require solution multiplicity, motivating the generative ex- tension.
2.2. Neural Fields
A neural field (NF) (also coordinate-basse based NN, implicit neural representation (INR)) is a INN (typically a multilayer- perceptron) representing a function f x +> y that maps a spatial and/or temporal coordinate x to quantity y. Com- pared to discrete representations, NFs significantly more memory- memory-efficient while providing higher fidelity, continu- ity. and access to automatic differentials. They have seen widespread success in representing and generating a vari- ctt of signals, including shapes (Park et al., 2019; Chen & Zhang, 2019; Mescheder et al., 2019), scenes (Mildenhall al., 2021), images (Karras et al., 2021), audio, video (Sitz- mann et al., 2020), and physical quantities (Raissi et al., 2019). For more comprehensive overview, we refer to the survey by Xie et al. (2022).
2.3. (Data-Free) Generative Modeling
Generative modeling (Kingma & Weeling, 2013; Goodfel- low et al., 2014; Rezende & Mohamed, 2015; Tomezak, 2021) is almost exclusively performed in data-driven (i.c., supervised) setting to capture and sample from the underly- ing data distribution. However, notable exceptions exist.
Boltzmann generators (Noé al., 2019) prominent example of data-free generative models. They are trained to capture the Boltzmann distribution associated with an energy landscape. In the generative setting, GINNs also learn a distribution minimizing an energy as an implicit combination of constraint violations and objectives. How- ever, Boltzmann generators avoid mode-collapse using an mntropy-regularizg term, which presupposes invertibility, making them not directly applicable to function spaces. In- stead, GINNs use more general diversity term to hinder mode-collapse over the function space of shapes.
Implicit neural shapes (INSs) represent geometries through scalar fields, such as occupancy (Mescheder et al., 2019; Chen & Zhang, 2019) or signed-distance (Park al., 2019; Atzmon & Lipman, 2020). In addition the prop- erties of NFs, INSs also enjoy topological flexibility sup- porting shape reconstruction and generation. We point out the difference between these two training regimes. In the generative setting, the training is supervised on the ground truth scalar field of every shape. However, in surface recon- struction, i.e., finding smooth surface from a set of points
3

---

Geometry-Informat Neural Networks
Conditional neural fields allow for generative modeling of functions. By conditioning base network F on mod- ulation (i.e., latent) variable 2, a conditional NF encodes multiple fields simultaneously: fax = F(z;2) z). The dif- ferent choices of the conditioning mechanism lead to a zoo of architectures, including imput concatenation (Park et al., 2019), hypernetworks (Ha et al., 2017), modulation (Mehta et al., 2021), and attention (Rebain et al., 2022). These can be classified into global and local mechanisms, which also establishes a connection between conditional NFs and operator learning (Wang et al., 2024). For more detail, we refer to Xie et al. (2022); Rebain et al. (2022).
lar, neural fields, and formulating differentiable constraints. In Section 3.2, we generalize to representing and finding diverse solutions using conditional neural fields.
3.1. Finding a Solution
Representation of solution. Let f X' H R be con- tinuous scalar function the domain X' C R". The sign of f implicitly definss the shape Ω = fx E f()) - 0} and its boundary o = {ar E (x) 0}. We use a NN f = fo with parameters 0 to represent the implicit function, i.e., an implicit neural shape, due to its memory efficiency, continuity, and differentiabilityy Nonetheless, the GINNs extend other representations, shown in Section 4.2. We additionally require f to approximate the signed- distance function (SDF) of Ω (defined in Eq. 21). This alleviates the ambiguity of many implicit functions repre- senting the same geometry and aids the computation of persistent homology, surface point samples, and diversity. In training. the eikonal constraint is treated analogously to the geometric constraints.
Generative design refers to computational methods that automatically conduct design exploration under constraints set by designers (Jang al., 2022). It holds the potential of streamlining innovative solutions, e.g., in material design, architecture, engineering. GINNs can be seen solving the general task of topology optimization finding the mate- rial distribution that minimizes a specified objective subject to constraints. However, while classical methods optimize a single shape directly. we optimize a model that generates diverse feasible shapes. This supports design space explo- ration and downstream tasks while allowing to incorporate sparse data samples, if available. While generative design datasets are not abundant, deep learning has shown promise in material design and topology optimization. For more de- tail, we refer to surveys on generative models in engineering design (Regenwetter et al., 2022) and topology optimization via machine learning (Shin et al., 2023).
Constraints on solution. To perform gradient-based optimization, we must first ensure each constraint can be written as a differentiable constraint violation q; : F + R. A geometric constraint has the general form c, 02) = 0. By representing the shape and the boundary implicitly via the function f. the constraints on the sets can be translated into constraints on f. This in turn allows to formulate dif- ferentiable constraint violations C, although this choice is not unique. Table 1 shows several examples using the constraints from our experiments. Some losses are straight- forward, and some have been previously demonstrated as regularization terms for INSs (see Section 2.2). Section 4.1 discusses two complex losses in more detail: connectedness and smoothness.
3. Method
Consider the metric space (F, d) of functions, such as those representing a shape or a PDE solution. Let the set of con- straints define the feasible set K- = { E (fc = 0, i = 1..m). Additionally, let the problem be equipped with an ob- jective o: F + R. Selecting the objectives and constraints of geometric nature lays the foundation for a GINN, which is trained to produce an optimal feasible solution by solv- ing min (f). A key feature of geometric problems is that one is often interested in finding different near-optimal solutions, for example, due to incompleteness, uncertainty, or under-determminedeess in the problem specification (e.g., see Figure B.4). This motivates making GINN generate a set of sufficiently diverse near-optimal solutions S C K::
3.2. Generating Diverse Solutions
Representation of the solution set. The generator G :2 + f maps a latent variable = E Z function f. The solution set is hence the image of the latent set under the generator: S = Tme (Z). Furthermore, the generator transforms the input probability distribution Pz over Z to an output probability distribution p over S. In practice, the gen- crator is a modulated base network producing a conditional neural field: f(x) = F(x;2).
min O(S). 8(
(1)
Constraints on the solution set. By adopting proba- bilistic view, extend each constraint violation and the ob- jective to their expected values: Cs(S)= = EE-pp [ca (G(z) and O(S) = Ex-pz [o{G(=)]
OSS) aggregates the objectives o(f) of all solutions f E S and d captures some intuitive notion of diversity. While it can be treated as another constraint, it acts on the entire solution set instead of each solution separately. Section 3.1 first discusses representing shapes as functions, in particu-
Diversity of the solution set. The last missing piece to training a generative GINN is making S a diverse collection of solutions. In the typical supervised generative modeling
4

---

Geometry-Informng Neural Networks
Design region
Interface
Connectedness
Smoothness
Diversity
Figure 3: The user can define geometric problems and solve them using the GINN framework. Here, we illustrate the results of progressively adding more design requirements, overall resulting shape generative model trained without data.
Set constraint (x())
Function constraint
Constraint violation cs(
Design region
N C
f(x) > 0 Va & E
Ser [min[(,, [(x)) dr [f(x)))] dx
Interface
an I
f(x) = 0 Vr EZ
Prescribed normal
n(x) = Vrez
ff(x) n(x) Va EI
[V(((() n(x)] de
Topology
Using persistent homology; see Section 4.1 and Appendix E
Table I: Geometric constraints used several experiments. The shape Q and its boundary dn are represented implicitly by the (sub-)level set of the function f. The shape must contained within the design region C X and attach to the interface I C & with a prescribed unit normal nzx). Other interesting constraints are listed in Table 8.
setting, the diversity of the generator is inherited from the diversity of the training dataset. The violation of this is studied under phenomena like mode-collapse in GANs (Che et al., 2017). Exploration beyond the training data has been attempted by adding an explicit diversity loss, such as en- tropy (Noé et al., 2019), Coulomb repulsion (Unterthiner al., 2018), determinantal point processes (Chen Ahmed, 2020; Heyrani Nobari et al., 2021), pixel difference. and structural dissimilarity (Jang et al., 2022). We observe that simple generative GINN models are prone to mode-collapse, which we mitigate by adding a diversity constraint.
Equation 2 is well-defined only for finite sets, so, in prac- tice, we apply to a batch of k i.i.d. sampled shapes S = [[ tion of diversity on infinite sets, especially with manifold structure, to future research.
sid pz}- We leave the considera-
Distance. A simple choice for measuring the distance be- tween two functions is the L function distance d2 fz)
V (x())))) fx))) dr. However, recall that ulti- mately want to measure the distance between the shapes, not their implicit function representations. For example, consider a disk and remove its central point. While we would not expect their shape distance be significant, the 2 distance of their SDFs is. This is because local changes in the geometry can cause global changes in the SDF For this reason, we modify the distance (derivation in Appendix F) to only consider the integral on the shape boundaries an, 0n, which partially alleviates the globality issue:
Many scientific disciplines require to measure the diversities of sets which has resulted in range of definitions of diver- sity (Parreño et al., 2021; Enflo, 2022; Leinster & Cobbold, 2012). Most start with distance d F2 + [0, oo), which can be transformed into the related dissimilarity. Diversity 6 2F + [0, 20) is then the collective dissimilarity of a set (Enflo, 2022), aggregated in some way. In the follow- ing, we describe these two aspects: the distance d and the aggregation into the diversity 8.
V Jon. f,(x) dr
{ fi(x)2
((f..J)) -(f
(3)
Aggregation. Adopting terminology from Enflo (2022), use the minimal aggregation measure:
Jon,
If f, (analogously f() is an SDF then Soa, fz() dx Jon, min || x' I|2 dæ and i closely related the chamfer discrepancy (Nguyen et al., 2021). We note that d is not metric distance on functions, but recall that we care about the geometries they represent. Using appropriate boundary samples, one may also directly compute a geomet- ric distance, e.g., any point cloud distance (Nguyen et al., 2021).
1/2
8(S)
Σ min ((fe.. jæL
This choice is motivated by the concavity property, which promotes uniform coverage of the available space, as de- picted in Figure 13. Figure 5c illustrates how it coun- teracts mode-collapse geometric problem. However,
5

---

Geometry-Informnc Neural Networks
In summary, training a GINN corresponds solving constrained optimization problem, i.e. improving the pected objective O(S) and feasibility (S) w.r.t. to each geometric constraint i = 1..m and the diversity constraint (Sk) = (SSx) - 6, 0). In practice, we convert this into a sequence of unconstrained optimization prob- lems using the augmented Lagrangian method introduced in Section 4.1.
Topology describes properties of shape that are invariant under deformations, such the number of holes. Certain materials and objects display specific topological properties (Moore, 2010; Caplan et al., 2018; Bendsoe & Sigmund, 201 I), e.g., connectedness, which is a basic requirement for the propagation of forces and, by extension, manufacturabil- ity and structural function.
Despite topological properties being discrete-valuee persis- tent homology (PH) allows formulate differentiable loss. It identifies topological features (e.g., connected compo- nents) and quantifies their persistence w.r.t. some filtration function. For our implicit shapes, this is the implicit func- tion f itself. Consequently, the birth and death of each feature can be matched to pair of critical points of f. Their values can then be adjusted achieve the desired topology. In practice, we follow the standard procedure of first dis- cretizing the continuous function onto a cubical complex. We filter cells outside the design region & to prevent invalid connections. In some experiments, we use an additional constraint that minimizes the number of holes. We detail PH and our approach in Appendix E.
4. Experiments
We demonstrate the proposed GINN framework experimen- tally on a range of problems spanning physics, geometry, and engineering, summarized in a problem-constraint ma- trix (Table 2). To the best of our knowledge, data-free shape-generative modeling is an unexplored field with no established baselines, problems, and metrics. Thus, in addi- tion to the problems defined and solved in Section 4.2, we define metrics for each constraint as detailed in Appendix C.I. We use these to perform quantitative ablation studies in Appendix C.2 and evaluate baselines in Appendix C.4, reserving the primary text for the main findings. We proceed with an overview of key experimental considerations, with more experimental and implementation details available in Appendix B.
Smoothness is another computationally non-trivial design requirement that we consider. Many alternative smoothing energies exist, each leading to different surface qualities (Westgaard & Nowacki, 2001; Song, 2021), but a broad class of smoothing energies can be written as the surface integral Jannz e(K1 (z), k2(z)) dx of some curvature expression e R2 R. The principal curvatures K1 and fig, same as other differential-geomerrii quantities, can be computed from Vzf and I in closed-form (Goldman, 2005). To solve Plateau's 's problem, we use the mean curvature KH := Ky =2 In the bracket experiment, we use the surface-strain and a variant thereof Elug == log(1 + E).
4.1. Experimental Details
Constrained optimization. To solve the aforementioned constrained optimization problem in Equation 1, we employ the augmented Lagrangian method (ALM). It is well studied in the classical and more recently deep learning literature. ALM balances the feasibility and optimality of the solution by controlling the influence of each constraint while avoid- ing the ill-conditioning and convergence issues of simpler methods. Specifically, we use an adaptive ALM (Basir & Senocak, 2023) that uses adaptive penalty parameters μ4 for each constraint to solve Equation as the unconstrained optimization problem max, ming L(8, X, p1) where
2 E == o + n
Surface sampling is required estimate the surface inte- grals for smoothness and diversity. We describe our sam- pling strategy in Appendix B, noting that it gives lower variance of the losses and a better convergence compared to naive strategy.
m+1
Models. Across the experiments, we consider several NF models that primarily differ in their activation function, in- cluding softplus, SIREN (Sitzmann al., 2020), and WIRE (Saragadam et al., 2023). We require all models to have well-defined and non- man-annisiin first and second derivatives Vz and Hz to compute the iso-level normals and curva- tures. As the NF conditioning mechanism, we always use input concatenation (see Section 2.3). denoting the latent space dimension as dim(z). We continue the model choice discussion in Appendix B.
+ Σ (.C((S(())
m-1 2 Σ (S (4)
The multipliers X and the penalty parameters μ4 are updated during training according to Equations (18)- (20). Adaptive ALM allows GINNs to handle different constraints without manual hyperparameter tuning for each loss. However, ALM works best if the losses are already on a similar scale. Appendix D provides a more detailed introduction and motivation for this approach.
4.2. Problems
Generative PINN solving an under-determined PDE to demonstrate the generality of the approach. Although
6

---

Geometry-Informed Informed Neural Networks
admitting infinitely many solutions, the naive application of the generative softplus-MLJ with dim(z) leads to mode-collapse. This is mitigated by employing the addi- tional diversity constraint as illustrated in Figure Sc.
Wheels as a simple design problem. Consider the domain X = |-1, 1Ja containing the ring-shaped design region E {ir E X I oी + x2 S 0.8²) and the interface I OE As before, the shapes must also satisfy the con- nectedness and diversity constraints. Additionally, a 5-fold cyclic symmetry constraint is required. We implement this as a soft constraint by sampling point, rotating it by द four times, and minimizing the variance of the implicit func- tion f at these five points. Alternatively, exact symmetry can be imposed using a periodic encoding of the input. The result of traversing the 2D latent space of discovered shapes is shown in Figure 5d (larger version in Figure 7).
Figure 4: A generative PINN producing Turing patterns that morph during latent space interpolation. This is result of searching for diverse solutions to an under-determined PDE.
we focus mainly on geometric tasks, we show that the idea of a diversity-constrainee data-free neural field generator generalizes to other areas, such physics. While most familiar problems in physics are well-defined, cases exist where, e.g., the initial conditions are irrelevant and general PDE solutions are sought, such as in chaotic systems or animations. Here, we seek stationary solutions to a reaction- diffusion system with no initial condition. Note that this system has infinitely many solutions. Figure illustrates the resulting Turing patterns that continuously morph dur- ing latent space traversal of the trained generative PINN. Appendix B. describes the problem in greater detail.
Jet engine bracket to demonstrate scaling to 3D engineer- ing design problems. The problem specification is based on an engineering design competition hosted by General Electric and GrabCAD (Kiis et al., 2013). The challenge was to design the lightest possible lifting bracket of a jet engine subject to both physical and geometrical constraints. Here, we focus only on the geometric requirements: the shape must fit in the given freeform design region & and attach to five cylindrical interfaces I: a horizontal loading pin and four vertical fixing bolts (see the sketch in Figure 1). Instead of minimizing the volume subject to a linear elasticity PDE constraint, we minimize the surface smooth- ness E subject to a topological connectedness constraint. Conceptually, this formulation is similar but avoids PDE solver in the training loop, which we address in follow-up work. These requirements and several GINN solutions are illustrated in Figures 1 and 3, which also explore additional requirements, including the modified surface energy Elog. number of holes, and diversity. The result of latent space traversal of the GINN model trained with diversity is illus- trated in Figure 6. In all cases, GINNs produce smooth, singly connected shapes that attach to the interfaces while remaining within the given design space. These properties are quantified in Appendix C.
Plateau's problem to demonstrate GINNs on a well- posed problem. Plateau's 's problem is to find the surface M with the minimal area given a prescribed boundary T (a closed curve in X C RR)). A minimal surface is known to have zero mean curvature Kи everywhere. Minimal surfaces have boundaries and may contain intersections and branch points (Douglas, 1931) which cannot be represented implic- itly. For simplicity, we select a suitable problem instance, noting that more appropriate geometric representations exist (Wang& & Chern, 2021; Palmer et al., 2022). Altogether, we represent the surface M- an n X and the two con- straints are r M and F (1)) 0 Var E M. The result in Figure 5a approximates the known solution.
Parabolic mirror to demonstrate a different geometry representation. Although we mainly focus on the im- plicit representation, the GINN framework extends to other representations, such as explicit, parametrie, or discrete shapes. Here, the GINN learns the explicit height function f 1. + R of a mirror with the interface constraint f(0) = 0 and that all the reflected rays should intersect at the focal point 10. The result in Figure 5b approx- imates the known solution: a parabolic mirror. This a very basic example of caustics, an inverse problem in optics, which we hope inspires future work leveraging the recent advancements in neural rendering techniques.
4.3. Discussion
Solution diversity, generalization, and latent structure. With the diversity constraint, GINNs not only produce mul- tiple solutions, but we also observe the emergence of a latent space structure. This is best seen in Figure 6 using 2D latent space from which k = 9 random samples are drawn every training iteration. Traversing the latent space of the trained GINN produces continuously morphing fea- sible shapes, i.e., the model generalizes. Furthermore, the latent space is organized - the solutions vary consistently over large latent distances, and latent directions account
Obstacle to introduce connectedness and diversity con- straints. Consider a 2D rectangular design region& & with a circular obstacle in the middle. The interface I consists of two vertical line segments and has prescribed outward- facing normals n. We seek shapes that connect these two interfaces while avoiding the obstacle. Despite this problem

---

Geometry- Informeed Neural Networks
(a) Plateau's problem
(b) Parabolic mirror
(c) Obstacle
(d) Wheels
Figure 5: GINN applied four different problems. (a) Plateau's 's problem: the unique minimal surface that attaches the prescribed boundary. (b) Parabolic mirror: the unique curve that collects reflected rays into single point. (c) Obstacle: connecting the two interfaces within the allowed design region. A superposition of 16 solutions is shown trained with (bottom) and without (top) diversity loss, which is required to avoid mode-collapse. (d) Wheel: The diverse and structured latent space of GINN trained to discover cyclically symmetric wheel designs. A larger version is available in Figure 7.
for different aspects. We see similar behavior in all four experiments that include the diversity constraint. However, we find that the learned structure depends on the exact setup of the diversity constraint. In particular, we observe more pronounced organization emerge for larger 8, however, over- specifying it impedes convergence.
User control. The variations of the bracket problem il- lustrated in Figure I highlight the user's ability to tune the problem and the resulting solutions. We solve these with the same setup and hyperparameters, illustrating the robustness of GINNs and the adaptive ALM approach to constrained optimization.
Optimization. A representative convergence behavior of the training is shown in Figures and 12. Despite up to seven loss terms, adaptive ALM automatically balances these and minimizes each constraint violation. However, the variance in several losses remains high. This is largely due to the diversity and smoothness terms, which are hard to optimize and increase the necessary number of iterations by roughly a factor of two and five, respectively. Curvature a second-order differential operator and is expected to be ill-condiiioned, motivating the future use of second-order optimizers (Ryck al., 2024; Rathore et al., 2024).
Figure 6: With the diversity constraint, GINNs not only produce multiple solutions but also discover latent space structure. Traversing the 2D latent space morphs solutions, i.., the model generalizes. The latent space is also orga- nized central bulky shape becomes thinner in the radial direction, and the axes can be identified by how the shape connects on the sides. Figure 9 shows a large x 9 version.
Runtime of single bracket shape is roughly 10K itera- tions or 30 minutes. Similarly, the diverse model trains for around 50K iterations or 5 hours on single GPU. Of the total time, the surface strain takes roughly 10% (due to the Hessian) and the PH solver 75% (expensive multi procesied CPU task). The runtime also increases when ablating the eikonal constraint it destroys the geometric regularity of the implicit function, hindering efficient surface point sampling that usually takes 15% of total time. More details are available in Appendix C.3.
obvious the strong impact of the eikonal constraint on the connectedness, smoothness, and diversity metries, again due to the lack of geometric regularity of the implicit function. A progressive ablation of all requirements of the bracket problem is illustrated in Figure 3. Figure 8 also visually compares different NF models, highlighting the superiority of WIRE over softplus-Mi MML and SIREN due to their spectral behavior.
Ablations are performed in Appendix C.2, measuring the expected role of each constraint, as well as ALM. Less
8

---

Geometry-Inforaet Neural Networks
5. Conclusion
Impact Statement
We have introduced geometry-informed neural networks demonstrating shape-generative modeling driven solely by geometric constraints and objectives. After formulating the learning problem and discussing key theoretical and practical aspects, we applied GINNs range of problems.
Our work aims to advance the field of machine learning and may contribute its broader societal impact. In addi- tion, there an ongoing discussion on the rights to data, since data is fundamental to training the majority of current machine learning models. The demonstrated data-free ap- proach to generative modeling brings forth a less explored perspective on this matter. It circumvents the copyright problem and facilitates practitioners who lack exclusive ac- cess to datasets. However, for the foreseeable future, the applicability and hence the impact is limited scientific and engineering applications. The demonstrated results, in particular, might foster path toward improved approaches to engineering design.
Limitations and future work. GINNs combine several known and novel components, each of which warrants an in-depth study of theoretical and practical aspects, including alternative shape distances, their aggregation into diversity, conditioning mechanisms, constraints, and optimization. In this work, focused on building the conceptual frame- work of GINNs and validating it experimentally. This in- cluded a realistic engineering design task. However, we considered a modified version of the original task and did not compare to established aypology-optimizatinn methods as this required the integration of PDE solver a task we address in a follow-up work.
References
Atzmon, M. and Lipman, Y. SAL: Sign agnostic learning of shapes from raw data. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
Even though ALM is significant improvement over the naive approach of manually weighted loss terms, the recent literature on multi-objectio and second-order optimizers suggests further possible improvements.
Atzmon, M. and Lipman, Y. SALD: sign agnostic learn- ing with derivatives. In 9th International Conference on Learning Representations, ICLR 2021.
Finally, we investigated GINNs in the limit of no data. How- ever, GINNs can integrate partial observations of single or multiple shapes. This combination of classical and machine learning methods suggests new approach to generative design in data-sparse settings, which are of high relevance in practical engineering settings.
Basir, S. and Senocak, An adaptive augmented lagrangian method for training physics and equality constrained arti- ficial neural networks. arXiv preprint arXiv:2306. 04904, 2023.
Acknowledgments
Bello, I., Pham, H., Le, Q. V., Norouzi, M., and Bengio, S. Neural combinatorial optimization with reinforcement learning. arXiv preprint arXiv:1611.00 1,1999900 2016.
We sincerely thank Georg Muntingh and Oliver Barrow- clough for their feedback on the paper.
The ELLIS Unit Linz, the LIT AI Lab, the Institute for Machine Learning, are supported by the Federal State Up- per Austria. We thank the projects FWF AIRI FG 9-N (10.55776/FG9), AlAGreenHeatingGriae (FFG- 899943), WTWT:OON.C..-22222LCMMAAA...
Ben-Shabat, Y., Hewa Koneputugodage, C., and Gould, S. DiGS: Divergence guided shape implicit neural represen- tation for unoriented point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19323-19332, 2022.
Stars4Waters FWF Bilateral Artificial Intelligence (10.55776/C0E12) We thank the European High Performance Computing ini- tiative for providing computational resources (EHPC-DEV-
Bendsoe, M. P. and Sigmund, O. Topology optimization. Springer, Berlin, Germany, edition, July 2011.
2023D08-019, 2024D06-055, 2024D08-061 We thank NXAI GmbH, Audi AG, Silicon Austria Labs (SAL), Merck Healthcare KGaA, GLSS (Univ. Waterloo), TÜV Holding GmbH, Software Competence Center Hagenberg GmbH, dSPACE GmbH, TRUMPF SE + Co. KG.
Bengio, Y., Lodi, A., and Prouvost, A. Machine learning for combinatorial optiiization: a methodological tour d'horizon. European Journal of Operational Research, 290(2):405-421, 2021.
Berger, M., Tagliasacchi, A., Seversky, L.., Alliez, P., Guen- nebaud, G., Levine, J., Sharf, A., and Silva, C. A Survey of Surface Reconstruction from Point Clouds, Computer Graphics Forum, pp. 27, 2016.
Arturs Berzins was supported by the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement number 860843.
Bronstein, M. M., Bruna, J., Cohen, T., and Veličković, P. Geometric deep learning: Grids, groups, graphs,
9

---

Geometry-Informed Neural Networks
geodesics, and gauges. arXiv preprini urXiv:210..34777 2021.
Fioretto, F,, Van Hentenryck, P., Mak, T. W., Tran, C., Baldo, F., and Lombardi, M. Lagrangian duality for constrained deep learning. In Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track: European Conference, ECMI PKDD 2020, Ghent, Belgium, September 14-18, 2020, Proceedings, Part V, pp. 118-135. Springer, 2021.
Brübe-GGbriilsssnn R., Sanapathi-Subramaniie V., Skraba, P., and Guibas, L. J. Topology- surface reconstruc- tion for point clouds. Computer Graphics Forum, 39(5): 197-207, 2020. doi: https://doi.org/ http://oo..o//11..11111 1twl11111...... 14079.
Caplan, M. E., Schneider, A. S., and Horowitz, C. J. Elas- ticity of nuclear pasta. Phys. Rev. Lett., 121:132701, Sep 2018. doi: 10. 10.1103/hysReewectt.. 121.11.7...11.1700
Gabrielsson, R. B., Nelson, B. J., Dwaraknath, A., and Skraba, P. topology layer for machine learning. In Chiappa, S. and Calandra, R. (eds.), Proceedings of the Twenty Third International Conference on Artificial In- telligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 1553-1563. PMLR, 26- 28 Aug 2020.
Che, T., Li, Y., Jacob, A. P., Bengio, Y., and Li, W. Mode regularized generative adversarial networks. In 5th Inter- national Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
Giampaolo, F., Rosa, M., Qi, P., Izzo, S., and Cuomo, Physics-informed neural networks approach for 1d and 2d gray-scott systems. Advanced Modeling and Simulation in Engineering Sciences, 9(1):5, May 2022.
Chen, W. and Ahmed, F. PaDGAN: Learning to Generate High- High-Qualin Quality Novel Designs. Journal of Mechanical Design, 143(3)0037003 11 2020. ISSN 1050-0472.
Goldman, R. Curvature formulas for implicit curves and surfaces. Computer Aided Geometric Design, 22(7):632- 658, 2005. ISSN 0167-8396. Geometric Modelling and Differential Geometry.
Chen, Z. and Zhang. H. Learning implicit fields for gener- ative shape modeling. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 5939-5948, 5948, 2019.
Goodfellow, I., Pouget- Abadic, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial nets. Advances in neural informa- tion processing systems, 27, 2014.
Chibane, J., Mir, A., and Pons-Moll, G. Neural unsigned distance fields for implicit function learning, In Advances in Neural Information Processing Systems (NeurlPS), December 2020.
Gropp, A., Yariv, L., Haim, N., Atzmon, M., and Lipman, Y. Implicit geometric regularization for learning shapes. In III, H. D. and Singh, A. (eds.). Proceedings of Machine Learning and Systems 2020, volume 119 of Proceedings of Machine Learning Research, pp. 3569-3579. PMLR, 13-18 Jul 2020.
J.ough, R., Byme, N., Oksuz, I., Zimmer, V. A., Schnabel, J. A., and King, A. P. A topological loss function for deep-learning based image segmentation using persistent homology. IEEE Transactions on Pattern Analysis & Machine Intelligence, 44(12):8766-87 8778, dec 2022. ISSN 1939-3539.
Ha, D., Dai, A. M., and Le, Q. V. HyperNetworks. In Sth International Conference Learning Representations, ICLR 2017, OpenReview.net, 2017.
Dash, T., Chitlangia, S., Ahuja, A.. and Srinivasan, A. A review of some techniques for inclusion of domain- knowledge into deep neural networks. Scientific Reports, 12(1):1040, 2022.
Heyrani Nobari, A., Chen, W., and Ahmed, F. PcDGAN:
A continuous conditional diverse generative adversarial network for inverse design. In Proceedings of the 27th
Douglas, J. Solution of the problem of plateau. Transactions of the Americun Mathematical Society, 33(1):263-321, 1931. ISSN 00029947.
ACM SIGKDD Conference on Knowledge Discovery
Data Mining, KDD '21, pp. 606-616, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383325.
Dugas, C., Bengio, Y., Bélisle, F., Nadeau, C., and Garcia, R. Incorporating second-order functional knowledge for bet- ter option pricing. In Leen, T., Dietterich, T., and Tresp, V. (eds.), Advances in Neural Information Processing Systems, volume 13. MIT Press, 2000.
Hu, X., Li, F., Samaras, D., and Chen, C. Topology- preserving deep image segmentation. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
Enflo, K. Measuring one diedeinniional diversity. Inquiry, O (0)1-34, 2022.
10

---

Geometry-Informet Neural Networks
Jang, S., Yoo, S., and Kang, N. Generative design by re- inforcement learning: Enhancing the diversity of topol- ogy optimization designs. Computer-Aided Design, 146: 103225, 2022. ISSN 0010-4485.
Marschner, Z.,, Sellán, S., Liu, H.-T. D., and Jacobson, A. Constructive solid geometry neural signed distance fields. In SIGGRAPII Asia 2023 Conference Papers, SA '23, New York. NY. USA, 2023. Association for Computing Machinery. ISBN 9798400703157.
Jia, Y., Wang, C.,, and Zhang, X. S. Fenitop: a simple fen- iesx implementation for 2d and 3d topology optimization supporting parallel computing, Structural and Multidisci- plinary Optimization, 67(8))140, 2024.
McClenny, L. and Braga-Neto, U. Self-adaptive physics- informed neural networks using a soft attention mecha- nism. arXiv preprint arXiv:2009.045: 2020.
McGough, J. S. and Riley, K. Pattern formation in the gray-scott model. Nonlinear Analysis: Real World Appli- cations, 5(1):105-121, 2004. ISSN 1468-1218.
Kaji, S., Sudo, T., and Ahara, K. Cubical ripser: Software for computing persistent homology image and volume data. arXiv preprint 2xXv:20005.12692, 2020.
Mehta, I., Gharbi, M., Barnes, C., Shechtman, E., Ra- mamoorthi, R., and Chandraker, M. Modulated periodic activations for generalizable local functional representa- tions. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 14194-14203, Los Alami- tos, CA, USA, oct 2021, IEEE Computer Society.
Karniadakis, G. E.,, Kevrekidis, G., Lu, I.., Perdikaris, P. Wang, S., and Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422-440, June 2021.
Karpatne, A., Atluri, G., Faghmous, J. H., Steinbach, M., Banerjee, A., Ganguly, A., Shekhar, S., Samatova, N., and Kumar, V. Theory-guider data science: A new paradigm for scientific discovery from data. IEEE Transactions on knowledge and data engineering, 29(10)22318-23 2331, 2017.
Mescheder, L.., Oechsle, M., Niemeyer, M., Nowozin, S., and Geiger, A. Occupancy networks: Learning 3d re- construction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4460-4470, 2019.
Mezghanni, M., Boulkenafed. M., Licutier, A., and Ovs- janikov, M. Physically-aware generative network for 3d shape modeling. In 2021 IEEE/CVF Conference on Com- puter Vision and Pattern Recognitiio (CVPR), pp. 9326- 9337, 2021. doi: 10. 00.1199CVVR44:3322221..9999
Karras, T., Aittala, M., Laine, S., Härkönen, E., Hellsten, J., Lehtinen, J., and Aila, T. Alias-free generative adversarial networks. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 852-863. Curran Associates, Inc., 2021.
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., and Ng, R. Nerf: Representing scenes as neural radiance fields for view synthesis. Communica- tions of the ACM, 65(1):99- 2021.
Kiis, K., Wolfe, J., Wilson, G., Abbott, D., and Carter, W. Ge jet engine bracket challenge. https::/grabbest cttpb/aaadccooeeeoooooooeaoeeee wwt.e/ccc//cccccceeeeeeecceae challenges/ ge-jet no-jjt-engle ceeiine-bbbe bra bww.kee--halllegeeeeeeeee----e 2013. Accessed: 2024-05-22.
Moore, J. E. The birth of topological insulators. Nature, 464(7286):194-11 198, March 2010. ISSN 1476-4687 doi: 10. 10.1038/nature08916
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv:1312.61144, 2013.
Muralidhar, N., Islam, M. R., Marwah, M., Karpatne, A., and Ramakrishnan, N. Incorporating prior domain knowl- edge into deep neural networks. In 2018 IEFE interna- tional conference on big data (big data), pp. 36-45. IEEE, 2018.
Kotary, J. and Fioretto, F. Learning constrained optimization with deep augmented lagrangian methods. arXiv preprint arXiv:2403.03455 034554 2024.
Nadimpalli, K. V., Chattopadhyay, A., and Rieck, B. A. Euler characteristic transform based topological loss for reconstructing 3d images from single 2d slices. 2023 IEEF/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 571-579, 2023.
Leinster, T. and Cobbold, C. A. Measuring diversity: the importance of species similarity. Ecology, 93(3):477-489 March 2012.
Ma, B., Zhou, J., Liu, Y., and Han, Z. Towards better gradient consistency for neural signed distance functions via level set alignment. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp- 17724-17734, Los Alamitos, CA, USA, jun 2023, IEEE Computer Society.
Nguyen, T., Pham, Q., Le, T., Pham, T., Ho, N., and Hua, B. Point-set distances for learning representations of 3d point clouds. In 202 IEEE/CVF International Confer- ence on Computer Vision (ICCV), pp. 10458-10467, Los Alamitos, CA, USA, oct 2021, IEEE Computer Society.

---

Ieomelry-Informed Neural Networks
Novello, T., Schardong, G., Schirmer, L.,, da Silva, V., Lopes, H., and Velho, L. Exploring differential geometry in neural implicits. Computers & Graphics, 108:49-60, 2022.
Sangalli, S., Erdil, E., Hötker, A., Donati, O., and Konukoglu, E. Constrained optimization train neu- ral networks critical and under-representeed classes, Advances in neural information processing systems, 34: 25400-25411, 2021.
Noé, F,, Olsson, S., Köhler, J., and Wu, H. Boltzmann gener- ators: Sampling equilibrium states of many-body systems with deep learning. Science, 365(6457) 1 147, 2019.
Sanokowski, S., Berghammer, W., Hochreiter, S., and Lehner, S. Variational annealing on graphs for combinato- rial optimization. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2024. Curran Associates Inc.
Palmer, D., Smirnov, D., Wang. S., Chern, A., and Solomon,
J. DeepCurrents: Learning implicit representations of shapes with boundaries. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Saragadam, V., LeJeune, D., Tan, J., Balakrishman, G., Veer- araghavan, A., and Baraniuk, R. G. Wire: Wavelet im- plicit neural representations. In 2023 IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), 2023.
Park, J. J., Florence, P., Straub, J., Newcombe, R., and Love- grove, DeepSDF: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 165-174, 2019.
Shin, S., Shin, D., and Kang, N. Topology optimization via machine learning and deep learning: review. Journal of Computational Design and Engineering, 10(4))1736- 1766, 07 2023. ISSN 2288-5048.
Parreño, F., Álvarez Valdés, R., and Martí, R. Measuring diversity. a review and an empirical analysis. European Journal of Operational Research, 289(2)5515 2021.
Sitzmann, V., Martel, J. N., Bergman, A. W., Lindell, D. B., and Wetzstein, G. Implicit neural representations with periodic activation functions. In Proc. NeurIPS, 2020.
ISSN 0377-2217.
Pearson, J. E. Complex patterns in a simple system. Science, 261(5118):189-111 192, 1993.
Son, H., Cho, S. W., and Hwang, J. J. Enhanced physics- informed neural networks with augmented lagrangian relaxation method (al-pinns). Neurocomputing, 548: 126424, 2023.
Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics- informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
Song, A. Generation of tubular and membranous shape textures with curvature functionals. Journal of Mathemat- ical Imaging and Vision, 64(1):17-40, aug 2021. ISSN 1573-7683. doi: 10. 10111001-11000110999.99
Rathore, P., Lei, W., Frangella, Z., Lu, L., and Udell, M. Challenges in training pinns: A loss landscape perspec- 2024.
Tancik, M., Srinivasan, P., Mildenhall, B., Fridovich-KKeare S., Raghavan, N., Singhal, U., Ramamoorthi, R., Bar- ron, J., and Ng, R. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33: 7537-7547, 2020.
tive. arXiv preprint 2:Xiv:2402.01880
Rebain, D., Matthews, M. J., Yi, K. M., Sharma, G., Lagun, D., and Tagliasacchi, A. Attention beats concatenation for conditioning neural fields. Trans. Mach. Learn. Res., 2023, 2022.
Tomczak, J. M. Why deep generative modeling? In Deep Generative Modeling, pp. 1-12. Springer, 2021.
Regenwetter, L., Nobari, A. H., and Ahmed, F. Deep Gener- ative Models in Engineering Design: A Review. Journal of Mechanical Design, (44(7):077704, 03 2022. ISSN 1050-0472.
Turing, A. M. The chemical basis of morphogenesis. Philos. Trans. R. Soc. Lond., 237(641):37-72, August 1952.
Rezende, D. and Mohamed, S. Variational inference with normalizing flows. In International conference on ma- chine learning, pp. 1530-1538. PMLR, 2015.
Unterthiner, T., Nessler, B., Seward, C., Klambauer, G., Heusel, M., Ramsauer, H., and Hochreiter, S. Coulomb GANs: provably optimal nash equilibria via potential fields. In 6th International Conference on Learning Rep- resentations, ICLR 2018, Vancouver, BC, Canada, April 30 30 3, 2018, Conference Track Proceedings. Open- Review.net, 2018
Ryck, T. D., Bonnet, F., Mishra, S., and de Bezenac, E. An operator preconditioning perspective on training in informed machine learning. In The Twelfth Inter- national Conference on Learning Representations, 2024.
12

---

Ieometry-Informed Neural Networks
Von Rueden, L.,, Mayer, S., Beckh, K., Georgiev, B., Gies- selbach, S., Heese, R., Kirsch, B., Pfrommer, J., Pick, A., Ramamurthy, R., et al. Informed machine learning-a taxonomy and survey of integrating prior knowledge into learning systems. IEFE Transactions on Knowledge and Data Engineering, 35(1):6146633, 2021.
Wang, F., Liu, H., Samaras, D., and Chen, C. TopoGAN: A topology-awas awarre generative adversarial network. In Proceedings of European Conference on Computer Vision, 2020.
Wang, S. and Chern, A. Computing minimal surfaces with differential forms. ACM Trans. Graph., 40(4): 13:11- 113:14, August 2021.
Wang, S., Teng, Y., and Perdikaris, Understanding and mitigating gradient flow pathologies in inffrme- neural networks. SIAM Journal on Scientific Computing. 43055533055-3381 2021.
Wang, S., Yu, X., and Perdikaris, P. When and why pinns fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 449:1 1 10768, 2022.
Wang, S., Sankaran, S., Wang. H., and Perdikaris, An ex- pert's guide training pnforre-d neural networks, 2023.
Wang. S., Seidman, J. H., Sankaran, S., Wang, H., Pappas, G. J., and Perdikaris, P. Bridging Operator Learning and Conditioned Neural Fields: A Unifying Perspective. arXiv preprint 2rXiv:2405. 2024.
Westgaard, G. and Nowacki, H. Construction of Fair Sur- faces Over Irregular Meshes Journal of Computing and Information Science in Engineering, 144):376-384, 10 2001. ISSN 1530-9827, doi: 10.1115/1.1433444
Whalen, E., Beyene, A., and Mueller, C. Simjeb: Simulated jet engine bracket dataset. Computer Graphics Forum, (5):9-17, 2021. doi: https://doi.or//10.111 1og../////.111111//1.333333.
Xie, Y., Takikawa, T., Saito, S., Litany, O., Yan, S., Khan, N., Tombari, F., Tompkin, J., Sitzmann, V., and Sridhar, S. Neural fields in visual computing and beyond. Computer Graphics Forum, 2022. ISSN 1467-8659.
Yang. H., Sun. Y.. Sundaramoorthi, G., and Yezzi, A. StEik: Stabilizing the optimization of neural signed dis- tance functions and finer shape representation. Thirty- seventh Conference on Neural Information Processing Systems, 2023.
Yifan, W., Wu, S., Oztireli, C., and Sorkine-Hornung, O. Iso- points: Optimizing neural implicit surfaces with hybrid representations. CVPR, 2020.
13

---

Geometry-Informed Neural Networks
A. Constam-Conttr Matrix
Table 2 provides an overview of the variety of considered tasks using a problem-constrane matrix. Not captured by this matrix are also the differences in domain dimension, shape representation, and problem symmetries.
Table 2: Problem-coostrrintt matrix.
Turing patterns
Platcau's problem
Parabolic mirror
Obstacle
Wheels
Jet engine bracket
Reaction-diffusio PDE
Interface
+
Mean-curvature
Reflection
Design region
Prescrihed normal
Conneetedness
Diversity
Eikonal
Rotational symmetry
Holes
Surface strain
Minimum thickness
B. Implementation and Experimental Details
We report additional details on the experiments and their implementation. We run all experiments on a single GPU (one of NVIDIA RTX2080Ti, RTX3090, A40, P40, or A100-SXM). The largest GPU memory requirement is 16GB for the multi-shape training of the jet engine bracket (9 shapes).
Surface sampling is required to estimate the surface integrals for smoothness and diversity. We first sample points in the envelope and project them onto the surface using Newton iterations. We then repel the points the surface achieve more uniform distribution similar to Yifan et al. (2020). Finally, we exclude points sampled within small distance to the interface I, as the surface should not change here. We also begin to sample surface points and compute the surface integrals only after a warm-up phase of 500 iterations. In combination, these aspects lead to a lower variance and better convergence.
B.1. Draction-Diffusion
The high-level idea of GINNs is to find diverse solutions to an under-determined problem. While the main focus of the paper is on geometry, we show that this idea generalizes to other areas. In physics, problems are often well-defined and have a unique solution. However, cases exist where the initial conditions are irrelevant and a non- non-particular PDE solution is sufficient, such as chaotic systems or animations.
We demonstrate an analogous concept of generative PINNs on a reaction-diffusion system. Such systems were introduced by Turing (1952) explain how patterns in nature, such stripes and spots, can form as a result of simple physical process of reaction and diffusion of two substances. A celebrated model of such system is the Gray-Scott model (Pearson, 1993), which produces a variety of patterns by changing just two parameters the feed-rate and the kill-rate B - in the following PDB:
du = D,Au - w + a(1 - u)
dv D.Av + 2 - (a + Byv.
01
at
This PDE describes the concentration u, v of two substances U, V undergoing the chemical reaction U + 2V + 3V. The rate of this reaction is described by u while the rate of adding U and removing V controlled by the parameters a and B. Crucially, both substances undergo diffusion (controlled by the coefficients D., Dz), which produces an instability leading to rich patterns around the bifurcation line a = 4(a + 3)3.
14

---

Lnonetry-Informed Neural Networks
Computationally, these patterns are typically obtained by evolving given initial condition u(æ; I 0) =0( v(a, 0)= (2) on some domain with periodic boundary conditions. A variety of numerical solvers can be applied, but previous PINN attempts fail without data (Giampaolo et al., 2022). To demonstrate a generative PINN on problem that admits multiple solutions, we omit the initial condition and instead consider stationary solutions, which are known to exist for some parameters ax 3 (McGough & Riley, 2004). We use the corresponding stationary PDE (ōu/ōt dv/Ot = 0) to formulate the residual losses:
Lu / (D.Au - u + aa(1 - u))2 Lo -==
/ (D:So+ ub² (a + 3)0) dx
(6)
To avoid trivial (i.e. uniform) solutions, we encourage non-zero gradient with a loss term - max(1, (D((()))) + (Vv(x))² dz). We find that architecture and initialization are critical (described below). Using the diffusion coefficients D, = 1.2 10-5, Du 2D, and the feed and kill-rates 0.028, / 0.057, the generative PINN produces diverse and smoothly changing pattern of worms, illustrated in Figure 4. To the best of our knowledge, this is the first PINN that produces 2D Turing patterns in a data-free setting.
Experimental details. We use two identical SIREN networks for each the fields u and i. They have two hidden layers of widths 256 and 128. We enforce periodic boundary conditions on the unit domain X = [o, 1]2 through the encoding 2; > (sin 2T,;; cos 2xx;;) for = 1, 2. With this encoding, we use wo = 3.0 to initialize STREN. We also find that the same shaped Fourier-feature network (Tancik et al., 2020) with an appropriate initialization of a works equally well. We compute the gradients and the Laplacian using finite differences epoch. Automatic differentiation produces the same results for an appropriate initialization scheme, but finite differences are an order of magnitude faster. The trained fields u, can be sampled at an arbitrarily high resolution without displaying any artifacts. The generative PINNs are trained with Adam for 20000 epochs with a 10-3 learning rate taking a few minutes.
64 x 64 grid, which is randomly translated in each
B.2. Plateau's 's Problem
The model is an MLP with [3, 256, 256, 256, 1] neurons per layer and the tanh activation. We train with Adam (default parameters) for 10,000 epochs with learning rate of 10-3, taking around three minutes. The three losses (interface, mean curvature, and eikonal) are weighted equally, but the mean curvature loss is introduced only after 1000 epochs. To facilitate a higher level of detail, the corner points of the prescribed interface are weighted higher.
B.3. Parabolic Mirror
The model is an MLP with 20,0040 1] neurons per layer and the tanh activation. We train with Adam (default parameters) for 3000 epochs with a learning rate of 10-3, taking around ten seconds.
B.4. Obstacle
The obstacle experiment serves as a proof of concept for including several losses, in particular the connectedness loss.
Problem definition. Consider the domain X' [-1, 1] x 0-0.5, 0.5) and the design region that is a smaller rectangular
domain with circular obstacle in the middle: 8 = (1-0.9, 0.9) x [-0.4, 0.4)) \ {भी + x < 0.13). There is interface
consisting of two vertical line segments I {(+0.9, x2) 0.4 S x2 S 0.4) with the preseribed outward facing normals n(100.9, 00.4 < x2 < 0.4) 0).
Softplus-MLP. The neural network model f should be at least twice differentiable with respect to the inputs x, as necessitated by the computation of surface normals and curvatures. Since the second derivatives of an ReLU MLP are zero everywhere, we use the softplus activation function as a simple baseline. In addition, we add residual connections (Dugas et al., 2000) mitigate the vanishing gradient problem and facilitate learning. We denote this architecture with "softplus-MLP". We train a softplus-MLP with 512 hidden layers with Adam (default settings).
Conditioning the model. For training the conditional models, we approximate the one-dimensional latent set Z = [-1, with N = 16 fixed equally spaced samples. This enables the reuse of some calculations across epochs and results in well-structured latent space, illustrated through latent space interpolation in Figure 5c
Computational cost. The total training wall-clock time is around 10 minutes for a single shape and approximately 60 minutes for 16 shapes.
15

---

Ecometry-Informee Neural Networks
Figure 7: GINNs produce diverse shapes with structured latent space. The shapes morph continuously into another when traversing the 2D latent space. From the top-left to bottom-right, the holes become smaller, while from bottom-left to top-right, the holes move inwards.
B.5. Wheel
The problem setup is as described in the main text. The ML setup is shared with the bracket problem. A larger illustration of the results available in Figure 7, showing diverse shapes and structured latent space.
B.6. Jet Engine Bracket
The jet engine bracket (JEB) is our most complex experiment. We tested different architectures (c.f. Figure 8) and found that WIRE (Saragadam al., 2023) produced the best results, while being casier to train with ALM than softplus-MLP or SIREN (Sitzmann al., 2020). We train the WIRE with 3 128 hidden layers with Adam (default settings) and learning rate scheduler 0.5t/10000 for 10000 iterations for the single shape and / 50000 for multiple shapes. To decrease the training time, we use multi-processing asynchronously create diagnostic plots computing the PH loss for multiple shapes.
WIRE. For the jet engine bracket settings, early experiments indicated that the softplus-MLP cannot satisfy the given constraints. We therefore employ WIRE network (Saragadam et al., 2023), which is biased towards higher frequencies of the input signal. As mentioned by the authors, the spectral properties of a WIRE model are relatively robust. Several values for wa and so, which control the frequency and scale of the Gaussian of the first layer at initialization, were tested. As there was no big difference in the results, we fixed them to win 18 and so For more detailed results, refer Section C.
Conditioning the model. In the generative GINN setting, we condition WIRE using input concatenation which can be interpreted as using different biases at the first layer. As we refer in the main text, we leave more sophisticated conditioning techniques for future work. We use N = 9 different latent codes spaced in the interval Z [0, 0.1] and are resampled every iteration. The results are shown in Figure 9.
Spatial resolution. The curse of dimensionality implies that with higher dimensions, exponentially (in the number of
16

---

Ecometry-Informeee Neural Networks
Figure 8: Comparison of architectures trained for 10k epochs to produce a single shape. From left to right: softplus-ML.. SIREN, WIRE. The softplus-Mil MMLP is unable fit the interfaces due to the low-frequency bias. SIREN converges much slower than WIRE, especially at the interfaces, and does not produce smooth shape.
Figure 9: GINNs produce diverse shapes with structured latent space. The shapes morph continuously into one another
when traversing the 2-dimensional latent space. These shapes produced by the same model Figure 6. A trained GINN allows the user to sample densely in the latent space with shapes all meeting the constraints: Interfaces are modeled correctly, shapes are not disconnected or leave the design space.
17

---

Geometry-Informed Neural Networks
dimensions) more points are needed consequently memory and compute) are needed than in 2D. In our experiments, observe that low spatial resolution around the interfaces prevents the model from learning high-frequency details, likely due stochastic gradient. Increased spatial resolution results in a better learning signal, and the model picks up the details more easily, To facilitate learning, we additionally increase the resolution around the interfaces.
the space equidistantly. Therefore, in 3D, substantially more points (and
C. Additional Evaluation
After defining the metrics used in experiments in Appendix C.I, perform additional study in Appendix C.2 ablating each constraint, well as the ALM, for three tasks. Additionally, in Appendix C.3, measure the impact of each constraint on the runtime. Finally, Appendix C.4 discusses and applies two baseline methods to the jet engine bracket problem.
C.1. Metries
We introduce several metrics for each constraint. We will use the chamfer divergence (Nguyen et al., 2021) to compute the divergence measure between two shapes P and Q. For better interpretability, we take the square root of the common definition of chamfer divergence
CD,(P,Q) -
Σ =EQ
min | VEP
v1l
(7)
V
Q
and, similarly, for the two-sided chamfer divergence.
CD2(P,Q))
Σ min | FEQ
-v11
1 [P
min || Σ PEP VEQ
v11
(8)
VEP
Reusing the notation from the main text, let E be the design region, 5E the boundary of the design region, I the interface consisting of nı connected components, X' the domain, Ω the shape and 8l2 its boundary. Let vol(P) = Sp. dP be the generalized volume (i.e., length, area, or volume) of P.
Design region. We introduce two metrics quantify how well shape fits the design region. Intuitively, for 3D, the first metric quantifies how much volume is outside the design region & compared to the overall available volume. The second metric compares how much surface intersects the boundary of the design region. For both, the optimal values are O, with lower being better.
v The d-volume (i.e. volume for d or area for d 2) outside the design region, divided by the total d-volume outside the design region.
vol(())
vol(Rn&E),
vol((E) The (d 1)-volume (i.e. the surface area for or length of contours for 2) of the shape intersected
with the design region boundary, normalized by the total (d 1)-volume of the design region.
Interface. To measure the goodness of fit to the interface, we use the one-sided chamfer distance of the boundary of the shape to the interface, as we do not care if some parts of the shape boundary are far away from the interface, as long as there are some parts of the shape which are close to the interface. The best is O, with lower being better.
CD, The average minimal distance from sampled points of the interface to the shape boundary.
Connectedness. For the connectedness, we care whether the shape itself and the interfaces are connected. Since the shape could potentially connect through paths outside the design region, we also introduce a connectedness metric that accounts for this undesirable effect. DC((2) denotes all connected components of a shape Ω except the largest. We define the metrics as follows:
18

---

Geometry-Informec Neural Networks
(9t): The zeroth Betti number represents the number of connected components of the shape. The target in our work is always 1.
(1 E):
The zeroth Betti number of the shape restricted to the design region.
vol() The normalized d-volume (i.c. volume for d = 3 and area for 2) of disconnected components. vol(2)
)) The normalized d-volume of disconnected components inside the design region.
vol(E)
The share of connected interfaces. If interface is an e-distance from connected component of shape, consider it connected to the shape. This metric then represents the maximum number of connected interfaces of any connected component, divided by the total number of interface components. By default, we set 0.01 when the domain bounds are comparable to the unit cube.
C((a,2);
ng
Diversity. define the diversity 6mean on a finite set shapes {S4, E [N]] as follows:
$72
mmean (S)
Σ SE[N]
1 N-1
Σ CD2(Dr,)))) SFIE[N]
(9)
Smoothness. There are many choices smoothness measures in multiple dimensions. In this paper, we use a Monte Carlo estimate of the surface strain (Goldman, 2005) (also mentioned Section 4). To make the metric more robust to large outliers (e.g.. tiny disconnected components have very large curvature and surface strain), we clip the surface strain of sampled point 34,4 E [N] with value Kmax 1, 000, 000.
min div? ( fzi) [f(x)]
Estrain (12)
Kmax
(10)
N
iE[N]
C.2. Ablations
Using the metrics defined above, the impact of ablating each constraint, as well as ALM, reported for the obstacle (Table 3), wheel (Table 4), and bracket (Table 5) tasks.
Table Metrics for the ablations of obstacle problem for multiple shape solutions.
Connectedness
Interface
Design region
Diversity
Ablation
(Q)
(o(On))
DCL()
vold B) val(X]
CT(R)))
ID,
vol(fin5)) vil(d))
t &ueu
wol())
(None)
1.11
1.11
0.02
0.00
0.39
0.03
0.01
0.15
Interface
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
Design region
1.00
1.00
0.00
0.53
2.00
0.00
0.71
0.08
Prescribed normal
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
Connectedness
0.56
1.00
0.00
0.00
0.00
0.00
0.00
1.45
Diversity
1.22
1.22
0.02
0.00
0.15
0.03
0.00
0.12
Surface strain
1.11
1.11
0.01
0.00
0.22
0.03
0.00
0.15
Augmented Lagrangian
1.22
1.22
0.00
0.28
0.03
0.01
0.14
19

---

Neural Networks
Table 4: Metrics for the ablations of the wheel problem for multiple shape solutions.
Connectedness
Interface
Design region
Diversity
Ablation
()((n
(C((2) Bo)
vol VOL
CDi
yolkfnnE) vol(d
t 6mee
(Nonc)
1.11
1.00
0.00
0.00
1.00
0.00
0.07
0.31
Interface
1.00
1.00
0.00
0.00
0.44
0.07
0.02
1.10
Design region
6.89
1.00
0.06
0.11
1.00
0.00
0.44
0.68
Prescribed normal
1.00
1.00
0.00
0.00
1.00
0.00
0.03
0.19
Connectedness
1.22
1.22
0.02
0.00
0.89
0.00
0.11
0.34
Diversity
1.00
1.00
0.00
0.00
1.00
0.00
0.08
0.03
Rotational symmetry
1.00
1.00
0.00
0.00
1.00
0.00
0.06
0.54
Minimum thickness
1.11
1.00
0.00
0.00
1.00
0.00
0.10
0.38
Eikonal
1.00
1.00
0.00
0.00
1.00
0.00
0.09
0.03
Curvature & Diversity
1.00
1.00
0.00
0.00
1.00
0.00
0.08
0.03
Augmented Lagrangian
1.00
1.00
0.00
0.00
1.00
0.00
0.01
0.21
Table 5: Metrics for ablations of the jet engine bracket problem for single and multiple shape solutions.
Connectedness
Interface
Design region
Smoothness
Diversity
Ahlation
J. (o2
yolL vol
soltEr vall VET
1
CH 15
1 CD (0,
sullED EEE vollBB]
t Aman
Multiple shapes
(None)
1.00
1.00
0.00
0.00
1.00
0.00
0.00
410
0.14
Interface
1.11
1.11
0.00
0.00
1.00
0.02
0.00
373
0.13
Design region
4.44
1.00
-0.01
0.97
1.00
0.00
0.14
95
0.09
Prescribed normal
1.00
1.00
0.00
0.00
1.00
0.01
0.00
343
0.13
Connectedness
25.11
19.44
0.00
0.00
0.17
0.01
0.00
178740
0.02
Diversity
1.00
1.00
0.00
0.00
1.00
0.00
0.00
341
0.05
Surface strain
1.00
1.00
0.00
0.00
1.00
0.00
0.00
263
0.10
Minimum thickness
1.00
1.00
0.00
0.00
1.00
0.01
0.00
432
0.15
Eikonal
6.67
6.44
0.00
0.00
1.00
0.00
0.00
722
0.07
Curvature & Diversity
1.00
1.00
0.00
0.00
1.00
0.00
0.00
305
0.05
Augmented Lagrangian
3.22
2,00
0.00
0.00
1.00
0.00
0.01
198
0.07
Single shape
(None) Augmented Lograngian
1.00
1.00
0.00
0.00
1.00
0.01
0.00
291
1.00
1.00
0.00
0.01
1.00
0.01
0.06
170
C.3. Runtimes
In addition to quantifying the impact of the constraint ablations on the metrics, we also report the iteration times in Table 6, focusing on the 3D jet engine bracket problem, which has the highest number and complexity of constraints. The other experiments show similar behavior. Most constraints have little effect on the time per iteration. Of the total runtime, surface strain takes 10% (due the Hessian) and the PH solver 75% (expensive multi-processe CPU task). The runtime increases when ablating the eikonal constraint efficient surface point sampling that usually takes 15% of total time. We use A100-SXM GPU, and the peak memory usage for a batch of shapes does not exceed 16 GB. These two losses also have strong impact on the iterations needed. As discussed in the main text, adding the smoothness loss increases the number of iterations roughly two-fold, and adding the diversity increases it roughly five-fold. Overall, the biggest effect is not so much the number of constraints, but rather losses that are ill co-conniii.oed. This conditioning issue also known in the PINN literature, and its mitigation is an open and active research topic.
destroys the geometric regularity of the implicit function, hindering
20

---

Ieonetry-Informed Neural Networks
Table 6: Time per iteration when ablating each constraint. Largest differences from the baseline (None) indicate compu- tationally expensive components of the method, most notably the connectedness constraint (due to the CPU-based PH calculation) and the boundary sampler required for calculating both the diversity and surface-strain.
Ablated constraint
Time [ms] per iteration
(None)
260
Bikonal
291
Interface
264
Design region
262
Prescribed normal
265
Diversity
265
Surface strain
230
Surface strain & Diversity (no boundary point sampler)
187
Connectedness
94
C.4. Baselines
While a complete and fair comparison to a baseline is not available, we compare to two at least partially fitting baselines.
Topology optimization. We consider classical TO, specifically, FeniTop (Jia al., 2024) which implements the standard SIMP method with popular FEM solver. We define TO problem that is similar possible, applying diagonal force to the top cylindrical pin interface and allowing 7% volume fraction in the same design region. The other interfaces are fixed in the same way. The shape compliance is minimized for 400 iterations on a 104 x 172 x 60 FEM grid (taking 190 min on a 32 core CPU to give sense of runtime, although a fair timing comparison requires a more nuanced discussion). The produced shape is visualized in Figure 10.
We then compute the surface strain (the objective we use) for this TO shape and, conversely, the compliance for a GINN shape (Section 4.2: illustrated in the penultimate column of Figure 3). Unsurprisingly, both shapes perform best the objective they optimized for while satisfying the constraints up to the relevant precision. This serves as a sanity check and confirmation of the constraint satisfaction.
Table 7: GINN compared to the topology optimization baseline for single jet engine bracket shape. Both methods perform best on the objectives they are optimized for while satisfying the shared constraints up the relevant precision.
Metric
Topology optimization
GINN
I Connectedness (O-th Betti-number
1
J. Interface (Chamfer distance)
0.00
0.00
J. Design region (Volume outside)
0.00
0.00
I. Curvature
442
144
+ Compliance
0.99
0.344
Human-expert dataset. A unique aspect of GINN the data-free shape generative aspect. Comparison to classical TO trivial since it is inherently limited single solution with null diversity. Instead, we use the simJEB (Whalen al., 2021) dataset give an intuitive estimate of the diversity of the produced results. The dataset is due the design challenge on related problem deseribed in Section 4.2. The shapes in the dataset were produced by human experts, many of whom also used topology optimization. To compute the diversity metric, we sample 196 clean shapes from the simJEB dataset, producing a diversity of 0.099, and 14 x 14 equidistant samples from the 2D latent space generative GINN model, producing a diversity of 0.167. Even though these sets are not directly comparable as they optimize for different objectives, these results indicate that GINNs can produce diversity on the same and larger magnitude as a dataset that required an estimated collective effort of 14 expert human years (Whalen et al., 2021).
21

---

Eeometry-Informeee Neural Networks
Figure 10: A solution to the jet engine bracket problem found by the FeniTop (Jia et al., 2024) topology optimization software. The quantitative metries are reported in Table 7.
D. Optimization
In general, an equality-conssrainee optimization problem can be written as
min (8)
such that
C2(0) =0 Vi E O......,
(11)
where O, O,C1 ... Om are smooth scalar functions R.V R. the objective function and constraint functions C2 represent the collection of equality constraints. A naive approach to solve this optimization problem is simply relax the constraints into the objective function and solve the unconstrained optimization problem
min ( - Mu.
(
(12)
2=0
for sequence } with μo, S yo,, for all l and μo. However, this penalty method can suffer from numerical instabilities for large Mo, hence the sequence is generally capped at maximum value Jmaz. A further problem, which has recently been studied regarding PINNs, is that the different objectives in 12 behave on different scales, leading to instabilities in training as the gradients of the larger objective functions dominate training.
This issue is addressed by weighting each constraint term individually
min ( +
Σ Pie CF 1=0
(13)
Besides manual tuning of the weights μig, several schemes to dynamically balance the different terms throughout training have been proposed, such as loss-balande balancinng via the uub-gradients ((Wang et al., 2021)), via the eigenvalues of the neural tangent kernel (Wang et al., 2022) or using a soft-attention mechanism (McClenny Braga-Neto, 2020).
22

---

Eecmetry-Inforrme Neural Networks
different method for solving 11 is the augmented Lagrangian method (ALM) defined as:
min max L(0, 2, =- == 0(0) + Σ x,,,(
2 Σ C0
1
(14)
A.
1=0
Using the min-mi max inequality or weak duality
max min C(0,), S min max C, A.p
(15)
X.p
we can solve the max- max-min -in problem instead. each epoch k minimization over network parameters 0g is performed using gradient descent, yielding new parameters Os+1- Then, the Lagrange multipliers updated follows:
Aizzi Asz + Hox Hox (ok+1)))
Vi 0, O........ m.
(16)
Note that this so-called dual update of the Lagrange multipliers is simply gradient ascent step with learning rate foo, for each multiplier Xi. Typically, there is also an increase of fo, up to maximum value Pmaz in the penalty method. Constrained optimization with neural networks using the ALM has been shown to perform well in previous works, such as in (Son et al., 2023), (Kotary & Fioretto, 2024), (Sangalli et al., 2021), (Fioretto et al., 2021), and (Basir & Senocak, 2023).
In this classical ALM formulation, there is only single penalty parameter μo. which is monotonically increased during optimization. As outlined above, this often insufficient to handle diverse constraints with different scales. Thus, we opt for the adaptive ALM proposed in (Basir & Senocak, 2023) using adaptive penalty parameters for each constraint, solving 11 the unconstrained optimization problem:
m max min L(8, == o(0) + Σ ,.C,()) + 2 1=0
Σ (0)
(17)
==
In each epoch k, again a minimization step over the parameters 0g via gradient descent performed. Then the penalty parameters Mae which simultaneously the learning rate of the Lagrange multipliers A updated using RMSprop followed by the gradient ascent step for Aia
Dien & aùi, + - (1-
(18)
Piets &
(19)
Vixx te
Aiesi &- di + Pie ((x+1)
(20)
where ū; is the weighted moving average of the squared gradient w.r.t. A; is the discounting factor for old gradients, is global learning rate and is constant added for the numerical stability of the division. This adaptive approach enables us to handle the diverse set of constraints in GINNs without the need for manual hyperparameter tuning.
Algorithm shows the full algorithm used to train for T epochs and specifies the hyperparameters we used. The only difference to (Basir& & Senocak, 2023) is that we set a = 0.90, which the default value RMSprop in PyTorch, instead of 0.99.
23

---

Geometry-Informee Neural Networks
Algorithm 1 Adaptive augmented Lagrangian method
1: Parameters: 7 = 1 10 2. = 0.90, = 1 x 10
2: Input: 80
3: Initialize: Ao,; M0,i f Vo,i t 0 Vi
4: fort 1 to T do
5: 0, 4 argming [(Og- A H4-1) (primal update: gradient descent step over 0)
6: Ut.s - a + (1 - a) (2 (
7: Pr.i - Vur,tee Vi {penalty )pdate)
8: At, 11 (1,,C(((0)) vi {dual update}
9: end for
10: Output: 0z
D.1. Loss plots
In Figures 11 and 12, we show the loss plots for training single and multiple shapes, respectively. As expected. the unweighted losses (middle rows in the Figures) decrease, while the Lagrange terms (bottom rows) increase over training.
E. Topology loss
We provide additional details on our approach to the connectedness loss. We break this down in three parts: First, we define the signed distance function of shape Ω which the neural field we train approximates. Then, we give a short rundown on computing the persistent homology (PH), in particular the PH of a neural field in a non-rectangular region. Lastly, we explain how to obtain a differentiable loss on the field from outputs of the non-differentiable PH computation.
Signed distance function (SDF) boundary point:
I of shape Q gives the (signed) distance from the query point æ to the closest
f(x) = [dxx, 022 [dxx 0Q)
if x E Q (if a is outside the shape), if e Q (if x inside the shape).
(21)
point x E X belongs the medial axis if its closest boundary point is not unique. The gradient of SDF obeys the eikonal equation V fz))]l everywhere except the medial axis, where the gradient is not defined. In INS, the SDF is approximated by NN with parameters 0: fo x f-
Connectedness refers to an object Ω consisting of a single connected component. is a ubiquitous feature enabling the propagation of mechanical forces, signals, energy, and other resources. Consequently, connectedness is an important constraint for enabling GINNs. In the context of machine learning, connectedness constraints have been multiply applied in segmentation (Wang al., 2020; Clough et al.. 2022; Hu al., 2019), surface reconstruction Sebril-Garrellsson et al., 2020), and 3I) shape generation with voxels (Nadimpalli et al., 2023), point clouds (Gabrielsson et al., 2020), and INSs (Mezghanni et al., 2021).
E.1. Persistent Homology
Persistent homology (PH) of the primary tools that has emerged from topological data analysis extract topological features from data. Data modalities such as point clouds, time series, graphs, and n-dimensional images all be transformed into weighted cell complexes from which the homology can be computed. The homology provides global information about the underlying data and is generally robust.
Homology is an invariant originating from algebraic topology. A topological space is encoded as cell complexes (X) consisting of n-dimensional balls 8d (n
0, 1, 1, ...) and boundary maps o, from dimension n which satisfy 0, 0n+1 0 and O The homology Hx,()) is then defined as the quotient space
ker(On)
H, (x)
(22)
im(0
24

---

Chometry-Informee Neural Networks
10
curvoture
connecteddeos
iiterfuue
2000
4000
5000
8000
20000
101
10"
normal
design reqion
ciional
2000
4000
5000
8000
20000
107
10-3
connectodiess
20"*1
region
Figure 11: Loss plots for training a single shape of the bracket problem. The solid lines are exponential-moving averages (factor 0.99) of the noisy values in lighter colors. (a) The losses used for backpropagation. (b) The unweighted losses of each constraint. (c) The À values of each constraint.
25

---

Crometry-Informeee Neural Networks
10
10"3
connectediess
10-4
30
diverity
10000
20000
30000
40000
50000
10
10
20-1
design region
aiconal
diverity
I0000
20000
30000
80000
50000
107
10-1
10
Interfaco
ducriit
Figure 12: Loss plots training multiple shapes of the bracket problem. The solid lines are exponential-moving averages (factor 0.99) of the noisy values in lighter colors. (a) The losses used for backpropagation. (b) The unweighted losses of each constraint. (c) The À values of each constraint.
26

---

Inooeley-Informed Neural Networks
The dimension of (X) counts the number of n-dimensional features and defines the Betti number bn for 0 the number of connected components, for n 1 the number of holes, for
the number of voids.
Filtrations on the space X are defined using filter function f X + R. Using sequence of increasing parameters with Og < an for k < n we can define sequence of nested subspaces of X as sub-level sets X, = f (1-00, We then have
Ø J S ... X
(23)
The homology of each of these nested complexes (X,) can be computed.
Persistent Homology encodes how the homology of an increasing sequence of complexes changes under a given filtration. Topological features appear and vanish as the filter function sweeps over X. The birth time b of feature is defined the value on at which the homology of C₂ (X,) changes to include this feature. The death time d of feature is analogously defined as the value Ot, at which it is removed from C, (X,). The persistence of a feature is defined as the length of its lifetime/ | = d - 6.
For each Betti number b, (for each homology class H₂) the information about the persistent homology of a given filtration encoded in a persistence diagram containing the points (b, d) of the birth and death pairs of all n-dimensional topological features (changes in the dimension of H,)). For a sufficiently fine filtration, the persistence diagrams contain the entire topological information about the underlying space shape.
To compute the persistent homology of neural field, we evaluate the network on cubical complex the domain of the field, i.e., a grid in R. The output is simply gray-scale image (since we are only dealing with scalar fields in this work), and the PH can be computed with existing algorithms. The current state-of-the-an algorithm for PH computation on cubical complexes is CRipser (Kaji et al., 2020).
Given a grayscale image and a filtration value a, the sublevel ser the binary image resulting from thresholding the image for values smaller or equal to a. For every such binary image, which defines weighted cubical complex with coefficients in Z/2Z, the homology can be computed. The persistence homology is then obtained by sweeping the thresholding value a through R.
In general, we are interested in computing the PH within a given design region or envelope, which is not necessarily a rectangular region. We achieve this by sampling the field a rectangular domain containing the envelope and setting the value of points not in the envelope to co. Applying the PH computation to this altered image then correctly returns the evolution of persistence features within the envelope. The only drawback of this method is the additional computational cost of having include the grid points outside the envelope in the PH computation, which is why the bounding domain should be chosen tightly around the envelope.
The PH computation itself does not have to be differentiable (and the CRipser library we use is not) because the cells, i.c., the grid points of the image, at which given persistence feature born or killed, are stored. Hence, we can simply use the network output at this grid coordinate to compute the loss and there are no issues concerning differentiability or having to re-implement the PH computation into PyTorch.
E.2. Differentiable topology loss
To compute differentiable loss, we use the outputs of the PH computation: For each homology class H,, we obtain the points "p; in the persistence diagram with the associated birth and death times "b,, "d, and the coordinates of these births Jrbb, y 2m,, and deaths d'd,, yrd,, d....
Remark: The representatives of a homology class are not uniquely determined. The CRipser library internally chooses representative and then outputs its coordinates. In practice, this caused no issues.
For selected iso-level ag we select all "p: for which "6, < ag < " d; and sort them by lifetime "4 = "d - "b,. Now let the index i run from 1... M sorting the selected "p,. To train the network fo to produce a single connected component at lso-level ag the loss is given by the residuals of the deaths d, to ag for all i = 2... U...M M, effectively pushing down all but the
most persistent component.
27

---

Crometry-Informee Neural Networks
Figure 13: A visual comparison of different diversity losses in a simple 2D example (F IR. and the feasible set the partial annulus). Each point f E F represents candidate solution. The points are optimized maximize the diversity within the feasible set. The top shows the minimal aggregation ômin as defined in Equation 26. The bottom row shows the total aggregation 8um defined in Equation 27. Each column uses different exponent p E {0.5, 1, For 0 S p S 1 the minimal aggregation diversity Ami is concave meaning it favors increasing smaller distances over larger distances. This leads to a uniform coverage of the feasible set. In contrast, the 6mi is convex for p 2 1 as indicated by the formed clusters for p = 2. Meanwhile, Sam pushes the points to the boundary of the feasible set for all p.
Lee
Σ ao
3ad 20, )²
(24)
is immediately clear that this term is diffcrentiable with respect 0.
More generally. to obtain shape with Betti number bn iso-level ans the summation above runs from é m + 1 1... M. The full topology loss for an N-dimensional shape is then given
N
M Σ n=0 i=m+1
Lropo
Σ
(an folxnd, yr s=))))²
(25)
E. Diversity
Concavity. We elaborate on the aforementioned concavity of the diversity aggregation measure with respect to the distances. We demonstrate this in a basic experiment in Figure 13, where we consider the feasible set K. as part of an annulus. For illustration purposes, the solution is point 2D vector space of E c R2. Consequentially, the solution set consists of N such points: (fa E X, 1,..., N). Using the usual Fuclidean distance dz(fi, fz), we optimize the diversity of
28

---

Ecometry-Informee Neural Networks
within the feasible set K. using minimal aggregation measure
1/8 min d2(ftt fi) dalfo jボL
mmin (S)
Σ
(26)
as well as the total aggregation measure
sum (S) = Σ Σ Edstiittt)) ))
1/8
(27)
Using different exponents p E 11 illustrates how 8min covers the domain uniformly for 0 S p S 1, while clusters form for p > 1. The total aggregation measure always pushes the samples to the extremes of the domain.
Distance. We detail the derivation of our geometric distance. We can partition X into four parts (one, both neither of the shape boundaries): 02, \ a,,, 02, \ 022, an, n an,, X' \ (on, U @02). Correspondingly, the integral of the I distance can also be split into four terms. Using (x) 0 Va E an obtain
((.i11) (f((z) - fy()))
[ Janvon, (0 (x)) dx I Jonjvon (fi(x) 0)P dx
I Joa.noa,
(0 (0- 0- do + Ixironuoa) (Ji(z) {5(x))
(28)
/ / f5(x) dx +
Von (x / Sxy(oniuon) (f:(x) - fz(2)) dr
Janionn
I [5(1)P dr on.
k1(1) dx + 1 Sxyroawwo,, (f((x) (x)) dr
on,
G. Geometric constraints
In Table 8, we provide non-exhaustive list of more constraints relevant to GINNs.
Constraint
Comment
Volume
Non-trivial to compute and differentiate for level-set function (casier for density).
Area
Non-trivial to compute, but easy to differentiate.
Minimal feature size
Non-trivial to compute, relevant to topology optimization and additive manufac- turing.
Symmetry
Typical constraint in engineering design, suitable for encoding.
Tangential
Compute from normals, typical constraint engineering design.
Parallel
Compute from normals, typical constraint engineering design.
Planarity
Compute from normals, typical constraint engineering design. Compute from normals, relevant additive manufacturing.
Angles
Curvatures
Types of curvatures, curvature variations, and derived energies.
Euler characteristic
Topological constraint.
Table 8: A non-exhaustive list of geometric and topological constraints relevant to GINNs but not considered in this work.
29


Reviews:
[{"review_id": "kCvzVCyk66", "reviewer": "Reviewer_8unU", "summary": "The paper introduces Geometry-Informed Neural Networks (GINNs), a framework that trains shape-generative neural fields without relying on data. Instead, the method leverages user-specified design constraints (e.g. connectivity, smoothness, and topology) to drive the generation of feasible shapes. A key novelty of the paper is the explicit incorporation of diversity constraint to avoid mode collapse, which is essential when multiple solutions are desired. The framework is demonstrated on a range of problems, including real-world engineering design challenges such as wheel design and jet-engine bracket design.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their considerate review. In the following, we address the three weaknesses.\n\n## Task diversity\nWhile we agree that more experiments are almost always better, we would like to present a matrix of problems and constraints that illustrates the variety of considered settings, available in the [rebuttal pdf](https://github.com/ginn-rebuttal-icml-2025/ginn-rebuttal/blob/main/GINN_rebuttal_icml_2025.pdf), together with an additional ablation study. Not captured by this matrix are also the differences in domain dimension, shape representation, and problem symmetries (e.g., the jet engine bracket has no symmetries, making this a very general problem). Since this is an unexplored research setting, unfortunately, there does not exist a catalogue of standard problems, so defining each new problem in addition to the main research question is a considerable effort. However, we agree this is necessary to develop this research direction further and hope this is addressed in future work.\n\n\n## Baselines \nWhile the reviewer and we seem to agree that a fully satisfactory comparison to a baseline is difficult to produce, we have followed the reviewer’s advice and implemented at least partially comparable baselines.\n\n### Topology optimization\nAs suggested, we consider classical TO, specifically FeniTop [1] which implements the standard SIMP method with a popular FEM solver. We define a TO problem that is as similar as possible, applying a diagonal force to the top cylindrical pin interface and allowing a 7% volume fraction in the same design region. The other interfaces are fixed in the same way. The shape compliance is minimized for 400 iterations on a 104x172x60 FEM grid (taking 190 min on a 32 core CPU to give a sense of runtime, although a fair timing comparison requires a more nuanced discussion). The produced shape is visualized [here](https://postimg.cc/1n4KD0QN).\n\nThen, we compute the surface strain (the objective we use) for this TO shape and, conversely, the compliance for a GINN shape (Section 4.2; illustrated in the penultimate column of Figure 3). Unsurprisingly, both shapes perform best at the objective they were optimized for while also satisfying the constraints up to the relevant precision. At the very least, this serves as a sanity check and confirmation of the constraint satisfaction.\n\n| Metric | Topology optimization | GINN |\n|---|---|---|\n| ↓ Connectedness (0-th Betti-number) | 1 | 1 |\n| ↓ Interface (Chamfer distance) | 0.00 | 0.00 |\n| ↓ Design region (Volume outside) | 0.00 | 0.00 |\n| ↓ Curvature | 442 | **144** |\n| ↓ Compliance | **0.99** | 0.344 |\n\n### Human-expert dataset\nA unique aspect of GINN is the data-free shape generative aspect. Comparison to classical TO is trivial since it is inherently limited to a single solution, and its diversity would be 0. Instead, we use the simJEB [2] dataset to give some intuitive estimate on the diversity of the produced results. The dataset is due to a design challenge on a related problem described in Section 4.2. The shapes in the dataset are produced by human experts, many of whom also used topology optimization. To compute the diversity metric, we sample 196 clean shapes from the simJEB dataset and 14x14 equidistant samples from the 2d latent space generative GINN model.\n\nEven though these sets are not directly comparable as they optimize for different objectives, this comparison indicates that GINNs can produce diversity on the same and larger magnitude as a dataset that required an estimated collective effort of 14 expert human years [2].\n\n| Metric | SimJEB | GINN |\n|---|---|---|\n| ↑ Diversity | 0.099 | **0.167** |\n\n## Computational cost\nWe thank the reviewer for highlighting the runtimes, which made us realize that we have reported an outdated runtime in the submission. The up-to-date runtimes on an A100-SXM GPU are 30 min for 10k iterations of a single-shape model and 4.5h for 50k iterations for the generative model, not 72h, as mistakenly reported in the submission. This large discrepancy was due to previously logging plots at every training iteration.\n\nSince reviewer 9JbR also asks about runtimes, we invite the reviewer to read our other rebuttal, where we report a more comprehensive constraint ablation study. In brief, the two difficult losses are surface strain and diversity, both in terms of runtime and iterations needed. \n\nRespecting the character limit, we are happy to continue with a more detailed discussion upon the reviewer's request.\n\n[1] Jia, Y., Wang, C., and Zhang, X. S. FEniTop: a simple FEniCSx implementation for 2D and 3D topology optimization supporting parallel computing. Structural and Multidisciplinary Optimization, August 2024.\n\n[2] Whalen et al., SimJEB: Simulated Jet Engine Bracket Dataset, Computer Graphics Forum 2021", "claims_and_evidence": "Most claims made in the submission supported by clear and convincing evidence. However I have some concerns on overall applicability to real-world problems.", "methods_and_evaluation_criteria": "The proposed methods make sense for the problem.", "theoretical_claims": "The theoretical claims are valid.", "experimental_designs_or_analyses": "The experimental designs are valid but limited to a small range & scale of applications.", "relation_to_broader_scientific_literature": "The method is likely to be significant and inspire many downstream applications.", "essential_references_not_discussed": "References are adequate.", "other_strengths_and_weaknesses": "Strengths:\n- The paper tackles the important problem of shape generation in domains lacking large datasets. GINN is a new paradigm that does not requiring any training data at all. The pipeline learns neural implicit fields using only analytical objectives and constraints provided by the user. This approach draws inspiration from physics-informed neural nets, and is novel in the field of 3D generation.\n- A major contribution is introducing an explicit diversity term in the optimization. By penalizing similarity among solutions, the framework yields diverse shapes that all meet the design criteria. This addresses mode collapse and is particularly valuable for design tasks where multiple viable solutions are desired,\n- The method allows fine-grained control of shape properties through constraint formulation. The experiments show GINNs can enforce geometrical constrains like connectivity, smooth surfaces, and topological features such as a specified number of holes in the shape. This level of control demonstrates the flexibility of the framework to incorporate domain-specific knowledge and design intent.\n\nWeaknesses\n- *Limited Evaluation of Generality*: While GINN is demonstrated on several tasks, the set of problems is still relatively narrow and somewhat tailor-made. It’s unclear how generally the method would perform on other shape domains or more complex design scenarios beyond these settins. The paper establishes its own metrics for each constraint (which is understandable, since no standard benchmarks exist), but this makes it difficult to judge if GINN’s success extends broadly. A more convincing evaluation would include additional diverse tasks or show that the method scales to higher-complexity shapes.\n- *Lack of Baseline Comparisons*: The study does not compare GINN against existing approaches, largely because “data-free shape-generative modeling is an unexplored field with no established baselines”. However, the absence of any baseline or alternative (even a simplified classical method) leaves the improvements unclear. For instance, in the engineering bracket task, the authors did not compare to classical topology optimization methods or other generative strategies, making assessment more difficult.\n- *Computational Cost*: Training GINNs appears computationally expensive and potentially impractical. The paper reports that even a single shape optimization (for the jet engine bracket) requires 10k iterations (1 hour), and incorporating diversity (multiple shapes) jumps to 50k iterations (72 hours) on a single GPU. Such high time costs raise concerns about scalability for real-world design use-cases. Additionally, managing many constraints (up to seven losses) is complex; while the adaptive augmented Lagrangian helps balance them, some losses (e.g. smoothness, diversity) remain hard to optimize and could significantly slow down convergence. The paper would be stronger if it discussed or mitigated these efficiency issues, or at least compared computational load with alternative methods.", "other_comments_or_suggestions": "Overall the method is clean, straightforward, and could inspire further studies of PINN on the problem of 3D generation. \n\nThe related work section is extremely well-written and informative. However, the authors are encouraged to incorporate a more comprehensive and generalized experiments under a more complex setting, ideally with fair baseline comparisons. The authors could consider trim the related work a bit to fit in such experiments.", "questions_for_authors": "See Other Strengths And Weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ehV4mIVMsm", "reviewer": "Reviewer_9JbR", "summary": "Paper introduces geometry-informed neural networks — a gradient-based way to optimize neural implicit function (SDF) without data based on local and global geometric constraints. Authors propose a set of practically relevant constraints (particular design region, particular interface, connectedness, smoothness, topology in form of Betti numbers) and formulated them in differentiable manner to be used in proposed optimizations.  Authors demonstrate validity of the approach via a set of qualitative and quantitative evaluations.\n\n\n=====POST REBUTTAL=====\n\nAs I said in my initial review, I think this is a very promising work that might lead to a lot of interesting follow-up work as well. Authors have addressed some of my concerns in the rebuttal: time ablation with respect to different constraints, question about sharp features. I maintain my rating of 'weak accept' and I am willing to champion this paper if needed. I am not going to rise my rating because I agree with reviewer dCHc on a notion that \"the proposed system seems to be fragile and hard-to-tune\" but I think that this is something to be studied in follow-up work. \n\nI strongly recommend including additional discussion into revised version's supplementary of the paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed review. In the following, we will address the questions.\n\n## Sharp features\nThe method is not limited to smooth surfaces. The smoothness of the surface is inherited from the NN used to represent the shapes. The smoothness properties of MLPs are governed by the activation function. We use at least $C^2$ activations to have twice continuously differentiable surfaces, for which the Hessian and, hence, curvatures are well-defined. If this is not required, sharp features can be easily represented with, e.g., ReLU MLPs. We will also add that the log-surface-strain objective promotes sharp features that can be approximately represented with $C^2$ NNs, such as WIRE (see top row in Figure 1 or [this image](https://postimg.cc/75Pn1ScB) for a close-up).\n\n## Impact of different constraints\nPer the reviewer’s inquiry, we have done a more comprehensive constraint ablation study collecting the time per iteration and metrics. The timings reported below are for the 3D jet engine bracket problem, which has the highest number and complexity of constraints. The other experiments show similar behaviour, so we omit these for brevity but can include them upon request.\n\nMost constraints have little effect on the time per iteration. Of the total runtime, the surface strain takes ~10% (due to the Hessian) and the PH solver ~75% (expensive multi-processed CPU task). To pre-empt a potential confusion – the runtime *increases* when ablating the eikonal constraint as it destroys the geometric regularity of the implicit function, hindering efficient surface point sampling that usually takes ~15% of total time. We use an A100-SXM GPU, and the peak memory usage for a batch of 9 shapes does not exceed 16 GB.\n\nThese two losses also have a strong impact on the iterations needed. As we describe around line 430, adding the smoothness loss increases the number of iterations roughly two-fold, and adding the diversity increases it roughly five-fold. The [rebuttal pdf](https://github.com/ginn-rebuttal-icml-2025/ginn-rebuttal/blob/main/GINN_rebuttal_icml_2025.pdf) also collects the impact of these ablations on the metrics.\n\nOverall, the biggest effect is not so much the number of constraints, but rather losses that are ill-conditioned as described in “Limitations and Future Work”. This conditioning issue is also known in the PINN literature, and its mitigation is an open and active research topic.\n\n| Ablated constraint | ms/it |\n|---|:---:|\n| (None) | 260 |\n| Eikonal | 291 |\n| Interface | 264 |\n| Design region | 262 |\n| Prescribed normal | 265 |\n| Diversity | 265 |\n| Surface strain | 230 |\n| Surface strain & Diversity (no boundary point sampler) | 187 |\n| Connectedness | 94 |\n\n\n## Additional examples\nWe would first like to clarify a potential misunderstanding that “almost all qualitative examples in the paper use one particular set of constraints (Figures 3 and 6)”. Figures 1, 3, and 6 illustrate the same jet engine bracket problem for presentation continuity. However, we also solve five other problems illustrated in Figures 4 and 5 and discussed in Section 4.2. We have prepared a visual overview of the problem-constraint matrix in the [rebuttal pdf](https://github.com/ginn-rebuttal-icml-2025/ginn-rebuttal/blob/main/GINN_rebuttal_icml_2025.pdf). If this does not answer the reviewer’s question for “additional qualitative examples beyond Figure 3”, we kindly ask for clarification.\n\n## Failure cases\nA few failure cases are discussed in the Appendix of the submission. For example, Figure 10 shows the bracket trained with softplus, SIREN, and WIRE, demonstrating the importance of the model’s inductive bias. Around line 329, we also discuss the impact of the surface sampling strategy, which should ensure that point samples are distributed uniformly and that high-curvature regions are not missed. We observed that naive sampling strategies lack these properties and lead to ridge artifacts that contain all the curvature but are missed during the optimization. We provide a basic illustration [here](https://postimg.cc/s1jLspm2). We discuss another failure mode around line 415: the emergence of latent space structure is sensitive to the diversity constraint.\n\nOne could also view the ablations as failure cases. Appendix B.2 and the additional ablation study now contain more examples. We are glad to produce additional figures illustrating these failure modes if the reviewer suggests this would further strengthen the submission.", "claims_and_evidence": "- Proposed method can be trained with data to satisfy design constraints to produce diverse shapes. This claim is supported by method design and experimental evaluation with different sets of constraints.\n- Proposed method is applicable to a variety of problems: geometric optimization, engineering design, physics. This claim is supported via toy examples in respective domains. \n\nOverall, I find claims and provided examples convincing (see more detailed comments in other sections).", "methods_and_evaluation_criteria": "- Method evaluates qualitatively in the main paper using several toy examples;\n- Additional quantitative evaluation is provided in the supplementary (Tables 2 and 3), where authors introduce a set of metrics to evaluate adherence to the constraints.\n- Design region adherence is evaluated via ratio of volume/surface area outside of boundary region normalized by total volume/area outside of design region; \n- Fit to interface is computed via one-sided Chamfer Distance between GT interface and interface area;\n- Connectedness is evaluated on the resulting shape via connected component analysis and Betti numbers.\n- Diversity is computed as variance across pairwise chamfer distances across the generated set.  \n- Smoothness evaluated via Monte Carlo estimation of surface strain. \n\nOverall I find qualitative measures well designed for proposed tasks and quantitative results look good to me.\n\nHowever, I think that qualitative evaluation is limited — almost all qualitative examples in the paper use one particular set of constraints (Figures 3 and 6). Paper might benefit from additional qualitative examples (and corresponding quantitative examples).", "theoretical_claims": "- Method optimizes neural implicit function SDF via adaptive augmented Langrangian method to satisfy a set of constraints (examples are given in Table 1). These losses are formulated in differentiable manner in supplement. \n\nI have checked losses and differentiable formulations and I haven’t found any issues with them.\n\nHowever, I think paper should discuss couple of important theoretical considerations:\n- To me, it looks like the proposed method can only produce smooth surfaces. This is important limitations because in CAD design surfaces are often non-smooth;\n- Authors mention that wall-clock time for the method is 10 min per single shape. Since qualitative evaluation in the paper is limited, I assume it is the same set of constraints as depicted in Figure 3. I think paper might benefit from additional ablation that investigates the number of constraints versus optimization speed trade-off.", "experimental_designs_or_analyses": "Almost all experiments in the paper are done using one set of constraints (Figure 3). It is not clear how well the method generalizes to a diverse set of constraints. This is the main reason for my rating. Paper can heavily benefit from additional examples derived from diverse sets of constraints.", "relation_to_broader_scientific_literature": "This paper can be viewed as constrained optimization of neural implicit function (SDF) via set of differentiable practically relevant constraints. The main theoretical contribution here is observation that neural implicit function can be optimized via a set of differentiable constraints instead of sampled ground truth implicit function.  The main practical contribution (based on limited set of examples) is that this optimization leads to plausible shapes.", "essential_references_not_discussed": "Mariem Mezghanni, Malika Boulkenafed, Andre Lieutier, Maks Ovsjanikov; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 9330-9341\n\nMezghanni M, Bodrito T, Boulkenafed M, Ovsjanikov M. Physical simulation layer for accurate 3d modeling. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 (pp. 13514-13523).\n\nTerzopoulos D, Witkin A. Physically based models with rigid and deformable components. IEEE Computer Graphics and Applications. 2002 Aug 6;8(6):41-51.\n\nYang L, Kim B, Zoss G, Gözcü B, Gross M, Solenthaler B. Implicit neural representation for physics-driven actuated soft bodies. ACM Transactions on Graphics (TOG). 2022 Jul 22;41(4):1-0.\n\nSchirmer L, Novello T, da Silva V, Schardong G, Perazzo D, Lopes H, Gonçalves N, Velho L. Geometric implicit neural representations for signed distance functions. Computers & Graphics. 2024 Dec 1;125:104085.\n\nZhao M, Wang Y, Yu F, Zou C, Mahdavi-Amiri A. SweepNet: Unsupervised Learning Shape Abstraction via Neural Sweepers. InEuropean Conference on Computer Vision 2024 Sep 29 (pp. 302-320). Cham: Springer Nature Switzerland.\n\nPujol E, Chica A. Rendering piecewise approximations of SDFs through analytic intersections. Computers & Graphics. 2024 Aug 1;122:103981.\n\nXiang H, Jianbing S, Wang J, Ji M, Zhang C. Boundary Adaptive Physics-Informed Neural Network for the Fluid Flow Around Circular Cylinders with Complex Boundary Conditions. Available at SSRN 5110040.", "other_strengths_and_weaknesses": "+ Authors provided code\n- Examples in the paper could be more diverse", "other_comments_or_suggestions": "I do think that paper is somewhat poorly titled: any neural implicit function that was trained on some shapes is geometry informed in some sense. I think that the name should be something like “Constraint-based optimization of neural implicit functions” but this does not hold weight in my decision.", "questions_for_authors": "- Am I correct that the method is only capable of producing smooth surfaces? If yes, this should probably be clearly stated in the paper. \n- How well does the method scale with respect to the number of constraints? Imagine that we add constraints the same way it is done in Figure 3. Does optimization run the same amount of time for all sets of constraints? Or convergence slows down when we increase the set of constraints?\n- Do you have additional qualitative examples beyond Figure 3? Especially ones that have thin wire-like constraints (e.g. shape that contains thin spring-like shape as interface; or swiss cheese interfaces with large number of small and large holes). \n- Are there any failure cases for your method that you have observed?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FDDQjK7YCj", "reviewer": "Reviewer_dCHc", "summary": "This paper studies a novel and interesting problem of learning a generative model of shapes under certain geometric constraints. It tackles this question in a simple yet effective manner: perform constrained optimization with a diversity penalty term. Some applications are included, and some analyses are conducted to show the property of the proposed framework.\n\n## Update after rebuttal\nThanks for the rebuttal! I agree with the authors and other reviewers that the problem and the solution are both interesting. However I keep my opinion that only two use cases are not enough to fully support the soundness of the proposed method. Given that this paper also does not have adequate theoretical contribution, I do expect extensive qualitative evaluations for this paper to meet the bar of ICML. Therefore I would keep my original evaluation.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed and balanced review. In the following, we address the main questions.\n\n## Q1\nThe reviewer writes that while the experiments look promising, their scope is insufficient and, in particular, only two studied problems are relevant to the core research question. We believe the first four problems do support the central theme by isolating two key aspects: shape optimization with NN representations (Plateau and mirror) and training a generative model through a diversity constraint (physics and obstacle). We invite the reviewer to view these tasks as ablations of solution multiplicity and shape representations. The reviewer might also find relevant the rebuttal pdf (linked below) and the discussion with reviewer 8unU, where we provide a matrix overview of all problems and tasks and discuss their variety. However, we do agree that we can strengthen the existing problems with more ablations, which we do under Q3.\n\n## Q2\nWhile the reviewer is right that our method does not come with convergence guarantees, this is the case for most adjacent methods in machine learning, physics-informed learning, and topology optimization. The convergence is a complex interplay between the model, the problem, and the optimizer, we do not foresee that such guarantees are easily obtained for the investigated non-linear differential losses on moving domains.\n\nConcerning the desired distribution, our best mental model is the Boltzmann distribution induced by the objective function over the feasible set. As described in Section 2.3, this distribution is over a function space which prohibits the direct use of Boltzmann generators (BGs) and motivates the use of an explicit diversity term that plays the role of the entropy term in BGs. Since this is an unexplored setting, we also call for further investigation under limitations and future work.\n\n## Q3 \nWe have performed additional studies ablating each constraint for three tasks (obstacle, wheel, bracket) with the metrics collected in the [rebuttal pdf](https://github.com/ginn-rebuttal-icml-2025/ginn-rebuttal/blob/main/GINN_rebuttal_icml_2025.pdf). The majority of these results quantitatively confirm the obvious roles of the constraints (e.g., interface losses improve interface metrics, etc.). The diversity ablation reduces the diversity in the bracket and wheel solutions roughly five and ten-fold, respectively. Notably, diversity and smoothness losses are competing (less diversity allows for lower smoothness). The reviewer might also be interested in how these ablations impact the computational speed discussed in the response to reviewer 9JbR.\n\nWe also include the ablation of the augmented Lagrangian. For the simpler problems, the constraint satisfaction is similar. For the more complex bracket problem with diversity and smoothness terms, we find that ALM finds more diverse shapes with more latent structure. This is best summarized visually ([this 3x3 plot](https://postimg.cc/SJyZfVx7) of ALM ablation can be juxtaposed with the 5x5 plot in Figure 6): while the overall structure of the shape is learnt, we see almost no interesting diversity or latent structure, and there are floaters and spurious surface features. We believe this is because ALM helps provide a stronger training signal throughout the training by adaptively increasing the weight of each loss. For some losses, the Lagrangian weight changes over two orders of magnitude. \n\n## Q4 \nWe thank the reviewer for the additional references.\n\n## Other comments\nThe reviewer writes that the “system is fragile and hard-to-tune”. We must briefly comment that the wheel and bracket experiments share the exact same setup. While we chose simpler models for simpler tasks, all experiments (including the generative PINN task which is very sensitive to initialization) can be repeated with WIRE. However, in under-determined problems (those without objective, specifically, obstacle), the inductive bias of the model impacts the identified solutions, so the reviewer is right that there is a setup-dependence in this sense. Correspondingly, the low-frequency biased softplus MLP provides smooth and visually pleasing solutions, motivating such a choice. The reviewer might appreciate the [following plot](https://postimg.cc/DJhtq1nS) illustrating the role of inductive biases and diversity in connection to Q3. We are open to adding this discussion and experiments to the manuscript if the reviewer suggests this would strengthen our work.\n\nLastly, the reviewer asks how we could incorporate ground-truth data. Conceptually, this is simple by adding a supervision loss. Using auto-decoder style training, the model should learn to organize the ground-truths in the latent space (imagine placing ground-truth shapes in Figure 6). Alternatively, the latent positions of the ground-truths can be enforced with a specific interpolation in mind (e.g., vertices of a hypercube). We believe this is a very exciting future direction.", "claims_and_evidence": "- **Claim 1: Proposes a novel and important problem.** This claim is supported by the survey of related works (Fig 2 and Sec 2). To the best of the reviewer's knowledge, the problem setting itself is novel. The potential application of constraint-driven shape design also makes sense to me.\n\n- **Claim 2: The proposed method is effective.**  This claim is partially supported.\n  - (**Q1**) This claim is **not** sufficiently supported solely from the exisiting experiments. Although the authors propose six questions, only two of them are related to the core task (shape generation, problem 5 and problem 6). Although the experiment result looks convincing, just working on one or two problems does not show the effectiveness of the proposed method on a wide range of problems.\n  - (**Q2**) There is no theoretical guarantee on the convergence of the proposed optimization process. The task itself can also be ill-defined from a theoretical perspective: how should we evaluate whether the generative model indeed recovers the desired distribution, or, what is the desired distribution from the first place?", "methods_and_evaluation_criteria": "The method itself is straightforward and sound. However the evaluation does not make sense to me. As mentioned above, only two problems studied are related to the central problem studied (shape generation). The other four problems, although providing some insights, have no direct relationship with the actual problem (shape generation).", "theoretical_claims": "There are no proofs.", "experimental_designs_or_analyses": "Yes I checked the soundness/validity of the experimental designs. As mentioned above and in Q1 and Q2, the experiment design is not sound. The experiments are so sparse that we can barely make any conclusions from it.\n\n**(Q3)** The ablation studies are also sparse. Only Fig 8. ablates the influence of different network choice for only one task. (Fig. 3 cannot be regarded as ablation study most of the time) It's expected to include more results, such as performing ablation studies on every single task, and performing ablation on other techniques used, such as optimization methods. How is the usage of ALM influencing the effectiveness? How useful is the diversity constraint in wheel and jet engine? I noted Table 2 which serves for part of the ablations I mentioned above, however no ablations are performed for other cases, e.g., wheel.", "relation_to_broader_scientific_literature": "The contribution of this paper is properly discussed in Fig. 2 and Sec. 2. The closest fields, as mentioned, are neural fields, generative modeling, and theory-informed learning.", "essential_references_not_discussed": "- (**Q4**) Enforcing geometric constraints on neural fields and/or learning generative models for neural fields have been well-studied before. There are some missing literatures to be discussed:\n  - Mehta et al., A Level Set Theory for Neural Implicit Evolution under Explicit Flows\n  - Yang et al., Geometry Processing with Neural Fields\n  - Schwarz et al., GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis", "other_strengths_and_weaknesses": "Beyond the points discussed above, there are several other strengths and weaknesses as detailed below:\nOther strengths:\n\n- The paper is well-written. The writing is coherent with consistent logical flow, which presents the novel task and proposed solution effectively.\n- The studied problem is interesting. This problem might be useful in several communities, especially in CAD (computer-aided design).\n- The properties of the learned generative model is interesting. I find Fig. 6 particularly interesting to me, as the latent space seems to be organized and interpretable.\n\nOther weaknesses:\n\n- The proposed system seems to be fragile and hard-to-tune. The authors use different network architectures and different training parameters for different tasks, which indicates the complexity to tune the optimization process.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "If we have a limited number of ground-truth data, how would the learning framework benefit from those expert data?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FQJiZ59s0I", "reviewer": "Reviewer_q1uT", "summary": "The paper proposes geometry-informed neural network (GINN) a general framework that allows for generating a diverse set of implicit shapes all satisfying a set of constraints while minimizing an objective function. The paper formulates this problem as a probabilistic generative problem with a novel diversity loss that prevents the model from mode-collapsing. It also provides a set of constraints including connectedness, smoothness, interface, and design region that can be used to enforce design constraints.\n\n## Post Rebuttal \nI appreciate the authors' rebuttal addressing most of my concerns. Therefore, I'd like to keep my original rating as it is. However, as discussed by other reviewers dCHc and 9JbR, I agree that the current experimental results do not validate the robustness of the system when optimizing for a wide range of setups. Thus, I also encourage the authors to include more settings in their experiment section for their revised version.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their positive feedback. We would merely like to answer the question concerning the extension of the method to constraints that require running a physics solver. Conceptually, this is straightforward by adopting a solver-in-the-loop [1], which we already do with the persistent-homology (PH) constraint. Loosely speaking, such a solver maps an input geometry to a differentiable scalar objective by discretizing the domain and solving an algebraic problem. This high-level procedure is the same for a PDE solver. Our framework can reuse PDE solvers from classical (differentiable) topology optimization, e.g., [2]. \n\nWe are currently developing the use of a PDE solver with the proposed framework in a follow-up work. Despite the conceptual similarity, there are technical differences; for example, PDE solvers are generally more expensive, require more careful discretization, and have more complicated loss landscapes, which is the reason we chose to omit this discussion in this submission.\n\n\n[1] Um, K., Brand, R., Fei, Y., Holl, P., and Thuerey, N. Solver-in-the-Loop: Learning from Differentiable Physics to Interact with Iterative PDE-Solvers. Advances in Neural Information Processing Systems, 2020.\n\n[2] Jia, Y., Wang, C., and Zhang, X. S. FEniTop: a simple FEniCSx implementation for 2D and 3D topology optimization supporting parallel computing. Structural and Multidisciplinary Optimization, August 2024.", "claims_and_evidence": "The paper claims to introduce GINN, a new type of neural field that is able to optimize for a given constraint while satisfying a set of constraints. The optimized network is further able to create diverse outputs all of which satisfy the constraint to some extent. \n\nThis claim is well supported by the experiments that the paper shows in the paper and the supplement.", "methods_and_evaluation_criteria": "The paper proposed a general framework that is able to handle a wide range of modeling tasks. To support this claim the authors evaluated GINN on multiple tasks all of which require using GINN on different sets of data. The paper also provides an extensive ablation study which I found very helpful.", "theoretical_claims": "I did not find discrepancies in the theoretical claims made by the paper. Although I did not go into details into the derivations presented in the supplement.", "experimental_designs_or_analyses": "The experiment design is comprehensive with lots of qualitative and quantitative metrics provided to support the effectiveness of GINN. The ablation study in the supplement is also helpful in analyzing the effect of different losses and components of GINN. Overall I found the experiments to be complete.", "relation_to_broader_scientific_literature": "The paper proposed the first data-free generation method that is able to generate a diverse set of outputs given a set of constraints and an objective function. This can potentially open up a new interesting research direction.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "N/A", "other_comments_or_suggestions": "N/A", "questions_for_authors": "Overall I don't have major concerns regarding the paper. I found its methodology insightful and experiment results complete. \n\nHowever, the experiments do seem like they are fake problems and do not have any significant practical applications. For example, in designing the engine bracket, often the design criteria cannot be straightforwardly expressed as a differentiable constraint as those presented in the paper (e.g., parameters only obtainable after physics simulation). How would the paper tackle such cases?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This is a very promising work. It presents GINN, geometry-informed neural networks that can be trained with constraints instead of usual data. This is surprising and promising- the work should generate many follow-up efforts. 

There were concerns around evaluation, ablation, and effects on other contraints. The authors' rebuttal addressed many of these concerns satisfactorily. The authors are encouraged to include the additional details (about explicit constraints in the various cases) as supplemental. Authors already agreed to provide code. 

The paper received 3WAs and one WR. Given the potential of the work and the quality of results/evaluation, I am happy to recommend acceptance (as is the consensus among the reviewers). The reviews and rebuttals add a lot of value to the material included in the paper (e.g., updated timing, more ablation), and authors are encourgaed to incorporate them in the final version.
Decision: Accept (poster)