Geometry-Informed Neural Networks

Authors: Arturs Berzins, Andreas Radler, Eric Volkmann, Sebastian Sanokowski, Sepp Hochreiter, Johannes Brandstetter
Primary area: learning on graphs and other geometries & topologies
Keywords: geometry, implicit neural representation, neural fields, theory-informed learning, geometric deep learning, physics-informed neural networks, generative design

Abstract:
Geometry is a ubiquitous tool in computer graphics, design, and engineering. However, the lack of large shape datasets limits the application of state-of-the-art supervised learning methods and motivates the exploration of alternative learning strategies.  To this end, we introduce geometry-informed neural networks (GINNs) - a framework for training shape-generative neural fields *without data* by leveraging user-specified design requirements in the form of objectives and constraints. By adding *diversity* as an explicit constraint, GINNs avoid mode-collapse and can generate multiple diverse solutions, often required in geometry tasks. Experimentally, we apply GINNs to several introductory problems and a realistic 3D engineering design problem, showing control over geometrical and topological properties, such as surface smoothness or the number of holes. These results demonstrate the potential of training shape-generative models without data, paving the way for new generative design approaches without large datasets.

Decision: reject
Reviewer scores: 6, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1729593935345

thread_type:
official_review

<thread_object>
creator: reviewer_wuar

<content>
summary:
The paper proposes geometry-informed neural networks. The idea is to train a generative model not from data, but using objective function and constraints. Basically, the generative model is trained by a specification similar to an optimization problem.

strengths:
The paper proposes an interesting idea that is worthy of further exploration. The research proposes the idea of a framework, but it does not demonstrate a working instantiation of the idea, so the research is only in its beginning stages. I think the concept is potentially very promising, but there are also predictable and significant obstacles that must be overcome. Design by optimization is a longstanding topic in many fields, but there are also many problems. Most notably, there aren't many or even any good examples of where optimization alone can give rise to interesting designs. Most of the time, this requires a good initialization that is already a design or the combination of user input (design) and optimization. For example, in architectural design you typically need an initial surface and you can design a pattern on the surface (e.g. a paneling). In furniture design, you also typically need to restrict the design space to a meaningful subset, e.g. by providing procedural rules or templates. At the moment, the method is an extension of an idea that does not have a working instantiation.

weaknesses:
The major weakness of the paper is the lack of meaningful designs. Without interesting design examples, the paper is not meaningful. None of the designs shown in the paper are interesting and they cannot be recognized as mechanical, biological, architectural, ... objects. These are abstract designs like abstract art, and it is not meaningful to create art by optimization and then infer how this would transfer to engineering. I would suggest a three-step approach to tackling this problem:
1) The project should identify an example where a single interesting design can be created by optimization. This example should not be abstract but specific to an engineering problem and be recognizable as an intentional and meaningful design. This example can be generated by optimization alone. It doesn't matter if the design is from architecture, mechanical engineering, biology, geology, ... but it should be meaningful. If you want to go for something more discrete, I would recommend furniture or CAD designs. This may not work well with your framework, so I could imagine that free-form architecture could be a better application area, e.g. "Geodesic patterns", developable surfaces, self-supporting surfaces, or quad meshes.
2) The project should expand from this single design to generate a set of diverse designs by combining optimization with a diversity constraint.
3) The project should example from a set of designs generated by optimization to combining optimization with generative modeling.

The current submission does not have a demonstration of either 1, 2, or 3. Competing work (not cited) has at least somewhat of a demonstration of points 1 and 2 (e.g., "Fit and Diverse: Set Evolution for Inspiring 3D Shape Galleries"), but it would be desirable to have even better examples. 


It is possible to accept the paper for conceptual novelty, but I am not in favor of such a philosophy. A paper should introduce a conceptual novelty and at the same time introduce a working realization, not only the conceptual novelty. The conceptual novelty by itself should not be enough for publication. I conjecture that the reason why there aren't many other papers on this topic is because people could not find meaningful and working instantiations of this concept. There is an obvious approach: first generate a set of objects by optimization (e.g. furniture) and then train a generative model on this set. I am not aware of such a successful approach and a paper in this space should possibly demonstrate that it can beat such a baseline..

questions:
N / A
</content>
</thread_object>

</thread>
<thread>
creation date:
1730360203792

thread_type:
official_review

<thread_object>
creator: reviewer_whb8

<content>
summary:
This paper presents the framework of GINN, which formulates the objectives and constraints of geometry problem as the loss function to train the neural networks. More importantly, it proposes a diversity constraint, which avoids the mode-collapse of other generative models and promotes the model to generate diverse solutions for the geometry problem. In the experiment section, it validates the performance of GINN on four problems and additionally conducts an engineering design case study. It is able to find the accurate solution for the famous geometry problems, and produce diverse solutions for the engineering design problem.

strengths:
- Although there have been a number of existing works using some similar losses to train their network, as mentioned in the related work, this paper presents the GINN framework with a very comprehensive summary and discussion on the common constraints used in the validation experiments. This would help the following works in formulating their own applications.
- Comparing to the physics-informed neural networks or the topology optimization, this paper proposes the diversity constraints for geometry problems. Comparing to other generative models such as boltzmann generators, the proposed diversity constraints help to avoid mode-collapse.

weaknesses:
- My main concern is about the technical novelty and the evaluation. Although this paper has presented many validation experiments, most of the results only show the performance of the GINN and some ablation settings, without comparing to other existing methods. In Section 4 it claims that there’s no established baselines, problems, and metrics for the data-free shape-generative modeling. Does it mean that there is no related work trying to solve the engineering design problem presented in Section 4.3??
- I understand that there are very much information to be presented in the paper. But it is a bit difficult to capture the key information. For example, the “topology” “smoothness” “surface sampling” are the defined metrics for each constraint? It’s better to have “constrained optimization”, “metrics”, “models” in bold font, and list all the metrics under the heading. The same problem exists in Section 4.3, there are too many headings to understand the structure of the writing.

questions:
I’m not very familiar with the specific field (engineering design) of this paper. In my understanding, the idea of using objectives and constraints as loss functions is not completely new in the field of 3D generative models. But this paper has extended this idea with many more kinds of constraints and proposed the diversity constraint specifically for the engineering design problem. My only doubt is whether there are indeed no existing works to be compared with. I'd like to increase my rating if the authors can give more explanation on this.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730462010478

thread_type:
official_review

<thread_object>
creator: reviewer_7pfz

<content>
summary:
This paper presents a framework for achieving controllable generation of industrial part geometry. Specifically, the authors utilize an implicit neural field to represent the surface of industrial parts. Through optimization objectives in different aspects, the implicit neural field is adjusted to closely match the given targets, such as ensuring the zero-level set of the implicit field aligns with the geometric surface and the first-order derivatives of the implicit field align with the surface normals. Additionally, the authors attempt to introduce a regularization term to promote diverse optimization results. Finally, the authors conduct experiments on a challenging example and demonstrate the effectiveness of their approach.

strengths:
This paper presents a well-formulated approach for shape-generative modeling driven by geometric constraints and objectives. It starts by considering the problem from a theoretical perspective and successfully translates it into an executable framework. The research problem addressed in this paper is valuable, and the preliminary experimental results provided by the authors demonstrate the effectiveness of their proposed method. Additionally, the paper is well-written and easy to follow.

weaknesses:
The framework solution proposed in the paper is effective from a high-level perspective, but I anticipate numerous challenges when it comes to practical implementation. My primary concern is whether we truly have the capability to account for all objective functions based on individual intuition, especially considering that these functions must be feasible, differentiable, and ideally, non-conflicting. For instance, how should we approach the generation of a nut that matches a specific type of screw, or a particular joint bearing?

Moreover, while the article emphasizes its data-free approach, it appears to me as an optimization-based strategy, with the optimization target being the implicit field represented by the neural network—a relatively conventional method. I do not find this particularly innovative.

Additionally, although Figure 6 does showcase a variety of workpieces, I doubt the compliance of these pieces with standard usage requirements; many seem to be diverse merely for the sake of diversity. 

Lastly, the experiments are conducted on only one workpiece, which, despite its complexity, does not suffice to demonstrate the universality of the method.

questions:
Although the authors have attempted to provide some formulations in Table 1, I still feel that these modules are far from sufficient to support the design of a workpiece. However, designing new constraints requires extreme caution and a significant amount of skill. Therefore, I am not particularly optimistic about this section. If the authors could propose a more universal paradigm for generating such constraints, many of my concerns would be alleviated.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730675922063

thread_type:
official_review

<thread_object>
creator: reviewer_qgvs

<content>
summary:
This work introduces a novel framework for data-free geometry learning. The framework relies on implicit geometry representation - which is a  modulated conditional neural field - where conditioning is on latent variables to ensure diversity. Given such a representation, authors propose to formulate a constrained optimization problem, in practice written down as a set of differentiable losses. The proposed constraints include: minimizing genus, smoothness, and diversity of generated shapes. Authors conduct a series of toy experiments to validate the model, and test it on a single engineering design problem.

strengths:
+ Paper is well-written and is easy to follow.
+ Overall framework is meaningful and has a large number of potential applications (in particular in generative design and shape optimization).
+ The promise of representation that does not require significant amount of data is very valuable specifically because the amount of data in generative design is often scarce or requires expensive simulations. 
+ Diversity measure that is demonstrated to work in practice for tackling mode collapse in implicit field learning is a potentially a critical contribution.

weaknesses:
- (minor) Most of the proposed constraints have been developed in previous work. 
- (minor) For the method to actually be useful for the real-world applications (e.g. in shape optimization in engineering fields), a mapping to an explicit representation could be a hard requirement to be compatible with existing simulation.  
- Although the representation/optimization framework that does not require data has promise, it would be interesting to see if these methods can be used in combination with available data.  
- The experimental evaluation is very limited. In particular for shape optimization (which is the only truly realistic test iiuc), there is only single example provided, which makes it hard to understand how much of the results is due to parameter tuning.

questions:
- How robust is the method to the choice of hyperparameters?
- Would it be possible to demonstrate results on more than one example for engineering design?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732553400760

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
As several reviewers expressed an interest in another engineering experiment, we would like to share an additional result. We investigate the design of a wheel, similar to a task in a data-driven design paper [1]. 

We want to stress that while we present a 2D experiment due to the limited time for running the experiments and visualizing the results, this experiment generalizes to a 3D wheel design by modifying the interfaces and adding an extrusion constraint.

The design requirements are:
- Domain: $X = [-1,1]^2$
- Design space: the ring between the inner radius $r_i=0.2$ and outer radius $r_o=0.8$
- Interface constraints: inner and outer circles,
- Connectedness constraint,
- Diversity constraint,
- 5-fold cyclic symmetry constraint: implemented as the soft-constraint (sample a point, rotate it four times by $\frac{2}{5}\pi$, evaluate the implicit function at the five points, require the variance of these to be 0.) Alternatively, a hard constraint using a periodic encoding can also be used to achieve exact symmetry.

The cyclic symmetry constraint has not been employed in the previous experiments demonstrating the flexibility of the method. We also reuse the same WIRE architecture with no changes to the ALM hyperparameters.

The results for both 2D ([link](https://imgur.com/a/oUDjoQy)) and 3D ([link](https://imgur.com/a/iut4pZd)) latent space (animated along the third latent dimension) reinforce the previous observations of recovering near-feasible, diverse results organized in the latent space.

[1] Jang et al., Generative design by reinforcement learning: enhancing the diversity of topology optimization designs, Computer-Aided Design 2022
</content>
</thread_object>

</thread>
<thread>
creation date:
1733239992602

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Comparison to SimJEB dataset

comment:
As requested by the reviewers wHB8 and QgvS, we computed the metrics to compare GINN and the SimJEB dataset [1]. The shapes in the dataset are produced by human experts, many of which relied on topology optimization. For single shape metrics, we use [shape 148](https://grabcad.com/library/ge-engine-brkt-2) available in the [SimJEB sample files](https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/XFUWJG) and the GINN generated shape described in Section 4.3. The multi-shape metrics for SimJEB were computed on the first 196 clean shapes. For GINNs, these were computed by equidistantly sampling 14*14 shapes from the 2d latent space model. 

- **Interface, design region, connectedness**: GINN and SimJEB shapes both satisfy these constraints exactly up to discretization errors.
- **Curvature**: GINNs outperform SimJEB, as this was used as the objective for training GINNs.
- **Compliance**: SimJEB outperforms GINNs, as the objective of the design contest was to optimize for mechanical performance. Specifically, we use *compliance* (stored deformation energy) as a metric. While the GINN framework can also optimize for compliance, we avoided it due to the large computational overhead.
- **Diversity**: GINNs produce more diverse shapes than the dataset created by human experts due to the explicit constraint in GINN training. This metric describes the expected distance between two shapes randomly sampled from the respective collections. 

Overall, both methods produce equally feasible shapes and perform best on the objectives that they were created with. Our method also produces a collection of shapes that is more diverse than the dataset that required an estimated collective effort of 14 human years. We hope these metrics and the additional wheel design experiment provide further proof of our contribution and clarify the reviewers’ remaining concerns.


| Metric                       	| SimJEB   | GINN  	|
| -------------------------------- | -------- | --------- |
| **Single shape**             	|      	|       	|
| ↓ Connectedness (0-th Betti-number)               	| 1    	| 1     	|
| ↓ Interface (Chamfer distance) | 0.00 	| 0.00  	|
| ↓ Design region (Volume outside)	| 0.00 	| 0.00  	|
| ↓ Curvature                  	| 838  	| **144**   |
| ↓ Compliance                 	| **2.01** | 7.81  	|
| **Multi shape**              	|      	|       	|
| ↑ Diversity  | 0.099	| **0.167** |



[1]: Whalen et al., SimJEB: Simulated Jet Engine Bracket Dataset, Computer Graphics Forum 2021
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces geometry-informed neural networks (GINNs) for optimizing 3D shapes given user-specified design requirements and constraints. The authors specifically incorporate the diversity constraint to get many different shapes and prevent mode collapse.  The authors show results on the realistic problem of designing a a jet-engine lifting bracket. The reviewers note this work can have  large number of potential applications. Due to scarce amount of data in generative design, being able to design shapes without having training is a desirable property. However, reviewers agree that a single experiment on real-world shapes (a jet-engine lifting bracket) was not sufficient to show generality of the approach. I suggest the paper to be rejected for ICLR.