Real-time design of architectural structures with differentiable mechanics and neural networks

Authors: Rafael Pastrana, Eder Medina, Isabel M. de Oliveira, Sigrid Adriaenssens, Ryan P Adams
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Differentiable physics, mechanical design, physics-in-the-loop neural networks, inverse problems, architectural structures

Abstract:
Designing mechanically efficient geometry for architectural structures like shells, towers, and bridges, is an expensive iterative process.
Existing techniques for solving such inverse problems rely on traditional optimization methods, which are slow and computationally expensive, limiting iteration speed and design exploration.
Neural networks would seem to offer a solution via data-driven amortized optimization, but they often require extensive fine-tuning and cannot ensure that important design criteria, such as mechanical integrity, are met.
In this work, we combine neural networks with a differentiable mechanics simulator to develop a model that accelerates the solution of shape approximation problems for architectural structures represented as bar systems.
This model explicitly guarantees compliance with mechanical constraints while generating designs that closely match target geometries.
We validate our approach in two tasks, the design of masonry shells and cable-net towers.
Our model achieves better accuracy and generalization than fully neural alternatives, and comparable accuracy to direct optimization but in real time, enabling fast and reliable design exploration.
We further demonstrate its advantages by integrating it into 3D modeling software and fabricating a physical prototype.
Our work opens up new opportunities for accelerated mechanical design enhanced by neural networks for the built environment.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 8

Reviews and discussion:
<thread>
creation date:
1730407280213

thread_type:
official_review

<thread_object>
creator: reviewer_h1ff

<content>
summary:
The authors propose an AI-driven design algorithm for architectural structure that can generate mechanically feasible designs for bar systems according to a target geometry. Previous approaches such as pure neural networks cannot guarantee mechanical design requirements. Traditional direct optimization methods can satisfy mechanical criteria, however are inefficient. The authors propose a hybrid scheme, combining neural networks and mechanical simulators to achieve computational efficiency and satisfaction of mechanical constraints. The authors approach consists of a encoder-decoder neural network, where the decoder is replaced by a differentiable mechanical simulator. The encoder structure provides efficiency, while the decoder ensures the design satisfies mechanical constraints. Experimental results show that the authors method achieves satisfactory performance in terms of design and computational efficiency.

strengths:
The paper presentation is very good, the motivation is clear, and the method is justified well. Experimental results are extensive. The authors also provide a real life sample designed from their algorithm, which the readers will find very interesting.

weaknesses:
The authors could do some revision of the figure. In figure 1, it seems that 3 figures are simultaneously provided to the encoder network, although the text description indicates only one shape is provided. Please take care to avoid such confusion from readers. Also describe the dimension of the matrix K. In figure 3, it is hard to see how the x vectors are being updated. Zooming in and adding some texts will be better for understanding.

questions:
See weakness section
</content>
</thread_object>

</thread>
<thread>
creation date:
1730410999837

thread_type:
official_review

<thread_object>
creator: reviewer_jnkr

<content>
summary:
This paper combines neural networks with a differentiable mechanics simulator to develop a model that accelerates the solution of shape approximation problems for architectural structures modeled as bar systems. The proposed approach achieves better accuracy and generalization than the neural network approach and gets comparable accuracy to direct optimization. The two design domains are masonry shells and cable-net towers.

strengths:
* The presentation of the approach is nicely structured, and the figures are clean and easy to understand. 
* As an application paper, the work demonstrates a nice implementation of combining a differentiable simulator (as the decoder) with a neural network (as the encoder). The authors also build a physical prototype (Figure 6) based on the output of the approach. It is highly unusual to take this extra step to build a physical prototype and should therefore be commended.
* The results successfully highlight how using the simulator ensures that the final design results in a pin-jointed structure where the resolved forces at the internal pins are at equilibrium.

weaknesses:
* The motivation for this approach is not entirely clear. The design pipeline appears to start with a $\tilde{\mathbf{X}}$ and then produces a set of stiffness values. These stiffness values, along with the boundary conditions are then fed to the mechanical simulator to result in the final design (presumably, with the neural network’s stiffness values):
    * In practice, how does one come up with a design $\tilde{\mathbf{X}}$? 
    * How does the simulator only require $\mathbf{p}$ and $\mathbf{b}$ for shape prediction? It seems like the simulator is missing information to build a shell with $N$ pins. 
* The cost of amortization (i.e. the number of direct optimization runs to build the data set for training) does not seem to be mentioned as a limitation of the approach. Are there instances in the real world where amortization of the optimization routine would be needed? For example, with a training set, it seems like 100 x direct optimization is required to train the model. This overhead would need at least 100 runs of the proposed approach to make it well-motivated. In a real design pipeline, how often would it be expected to call such a function? It already runs in less than 2 seconds for cable-net towers and around 6 seconds for the masonry shell. Is this this wall-clock time a problem in practice? If so, the paper would benefit from this written explicitly. 

Minor
* $p$ for norm and $\mathbf{p}$ for loads are a bit confusing in terms of notation.
* Suggestion: use test-time inference rather than inference for predictions, since inference can also refer to inference over parameters.

questions:
* Are the authors going to be releasing the code? As an application paper, it would be beneficial to the community if they were.
* Does the chosen bar stiffness affect the loading on the design since a stiffer bar might lead to a heavier design? 
* Could you provide further clarification on how the stress direction is determined a-priori? 
* When training the neural network component, how is this loss accounting for permutation invariance with locations, or is there some implicit consistency in the ordering of the pin coordinates.
* What mechanism is in place to prevent the model providing excessively large stiffness values for all the bars?
* Could you explain why the masonry shell is modeled as a pin-jointed structure but is then physically built as a shell (and how the two structure types are related)?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730609807031

thread_type:
official_review

<thread_object>
creator: reviewer_vhna

<content>
summary:
This manuscript presents an interesting application of AE-type of network architecture to facilitate design and optimization of mechanical structures, mostly in (building) architectural structures. Although the method presented is mostly ad-hoc, the manuscript is well written, the proposed method appears sound, and the potential application is clearly indicated.

strengths:
The paper is nicely written, although there are a few small details which could make the paper more understandable for neurips attendees, who may not have prior knowledge on the mechanical structure presented.

The method presented appears sound. The architecture as shown in Fig 1, leveraging JAX, makes intuitive sense.

weaknesses:
Concerns to the paper:

1. Although it is not clearly explained, it appears that each trained model is closely related to the underlying architectural structure (masonry shells or cable-net tower). The question is whether the trained model is generalizable. Even for the shells, if $M$ (number of bars) and $N$ (the number of nodes) change, the model may have to be retrained. It's understandable that this does not reduce the practical value of the proposed method. 

2. The runtime measurements have such large variances that they are not meaningful. The author(s) should consider changing the evaluation platform (macOS has many background processes and is not a good platform for benchmarking), and deploy better runtime measurement libraries.

3. In line 200 in Eq. (3), the fact that $\tau$ can be used to specify minimal stiffness is very interesting. But there are no more follow-up discussion on this topic. 

4. Structure of the paper: the decoder is not discussed in the main body of the paper, mostly in the appendices. This need to be fixed. 

Minor issues:
The terms *mechanical integrity* and *mechanical efficiency*  should be explained early in the paper to make it easier for readers to appreciate the paper.

questions:
1. Does the term *fabrication requirements* imply that each bar in the shell examples has to follow additional constraints?
2. Provide more discussion on the usage of $\tau$ as pointed out in the previous section
3. Revise the manuscript to discuss the decoder in the main body of the manuscript.

Minor issues:
1. Define (or discuss) *mechanical integrity* and *mechanical efficiency* earlier in the manuscript. What do they exactly mean?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730783683819

thread_type:
official_review

<thread_object>
creator: reviewer_nvmt

<content>
summary:
The paper introduces a physics-in-the-loop scheme for the design of architectural structures. The authors here use a neural network(MLP) to learn the mapping from the desired structural shape to intermediate mechanical properties (bar stiffnesses) and then use a physics model to generate a physically feasible shape approximating the target.  The authors then apply this method to the design of masonry shells and cable towers,  comparing it with two neural network baselines: one trained to produce physically feasible shapes and the other trained to produce feasible shapes while also ensuring mechanical stability. Results on both these case studies show the proposed approach outperforming both these baselines and being on par with numerical optimization.

strengths:
1. The idea of combining physics with ML for architectural design is promising, as it removes the need for generating costly training labels. Instead, the model can learn by integrating the physics model with inexpensive loss functions.
2. The out-of-distribution performance is also interesting, as it potentially reduces the need for extensive variability in the training data.

weaknesses:
1. The case studies provided are relatively simplistic and do not reflect real-world applications. The physics model used is also quite basic, limiting the method's applicability in practical scenarios.
2. Additionally, as the authors acknowledge, even when trained, the proposed parameterization lacks flexibility and requires retraining whenever the design representation changes.

questions:
Overall, the paper is technically sound and proposes an interesting integration of ML and physics. However, there are some concerns as follows:

1. The proposed hybrid approach requires running the physics model during inference, likely increasing training costs. While the authors report inference time, the training time is not provided. Given the limited representational capacity of the network, it must be retrained whenever the design is re-parameterized (a likely scenario during conceptualization). Reporting training times would enable a better assessment of the method’s real-world viability; if training significantly exceeds the duration of several optimizations, direct optimization might be preferable.

2. Similarly, providing training time metrics for the other baselines would help clarify trade-offs and be useful in scenarios where slight accuracy losses are acceptable for performance gains.

3. From Fig.9 , the optimization initialized with the proposed method converges quickly as compared to the other initializations. This leads to an interesting question of how the NN initialized optimizations would perform. This approach would have the benefit of outputting a guaranteed local minima while avoiding the additional physics overhead during training. Including these results would enhance the paper's contribution to the community.

4. The case studies considered here have relatively low DOFs, and the physics relies on a linear FDM. The authors could discuss the viability of this approach for structures where linear FDM is inapplicable or for dynamic scenarios. In such cases—and even when FDM is applicable but the structure has much higher DOFs—would this approach remain feasible?

5. Furthermore, the variation in MLP and PINN inference times is puzzling. Since the input sizes remain constant, one would expect the inference times for the fully trained networks to be similar. Could the authors comment on this discrepancy?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732288012858

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General response

comment:
We thank all reviewers for their time and feedback on our submission.

We are encouraged by the reviewers’ commendation for building a physical prototype based on our model predictions (jNKr, H1FF). We are also motivated because they found our work valuable as it removes the need for costly training labels and has good out-of-distribution performance (NVMt); and because it allows setting a lower bound on the inputs to the decoder (VHnA). The reviewers found the manuscript was well-written (VHnA), well-presented (H1FF, jNKr), and technically sound (VHnA, NVMt).

To address the reviewer’s comments, we have uploaded a revised version of our manuscript. For convenience, we explicitly track modifications in our resubmission by coloring text changes in blue. We list the main changes below and respond to each of the reviewer’s comments individually. 

- Added a new experiment in the towers task, where we use the PINN predictions as initialization for direct optimization. We updated Figure 9 to include the convergence curve of this initialization (NVMt).
- Added a new set of experiments in Appendix B.3 where we quantify trade-offs between our model and direct optimization in terms of training vs inference time in problems with up to 1000 DOFs (NVMt).
- Included a subsection in Section 2.3 of the main text that describes the mechanical decoder at a high level. We also extended Appendix D to better explain the minutiae of how our mechanical decoder works (VHnA).
- Fixed a bug in our timing protocol that increased the variance of the test inference times. The large variances stemmed from mistakenly including the JIT compilation time in our statistics. We have updated Tables 1 and 2 accordingly. The updated inference times make for stronger neural baselines, but the benefits of our model w.r.t. optimization in terms of inference time are preserved (NVMt, VHnA).
- Defined and distinguished between mechanical efficiency and integrity (VHnA).
- Described what mechanical theory allows us to model masonry shells as pin-jointed bar systems (jNKr).
- Updated Figures 1 and 3 to enhance legibility (H1FF).
- Changed the term “bar stress” to “bar force” to be consistent with the information our mechanical simulator outputs.
- Shortened the abstract to meet page limits.
- Fixed typos.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces a surrogate model for designing architectural shape structures modeled as bar systems, a simplified structural representation consisting of nodes connected by bars. This model combines a neural network (NN) with a differentiable mechanical simulator. Given an input target shape, provided, for example, by a designer, the NN computes stiffness parameters that encode the mechanical behavior of the structure, while the simulator outputs the corresponding structure. The model is trained end-to-end. Validation is performed on two problems: masonry shells and cable-net towers. The baselines consist of an encoder-decoder NN, the same architecture augmented with an additional mechanical loss (similar to a physics-informed neural networks (PINNs) approach), and a traditional direct optimization. The proposed hybrid model approaches the performance of direct simulation methods at a much lower computational cost.

This is an application paper, and the idea of combining differentiable physical models with NNs has been explored in various fields. The originality of this work lies in its application domain. The reviewers consider this work, even though it is evaluated on relatively simple mechanical design problems, to be a significant first step toward developing practical applications. The authors’ responses and the additional experiments provided during the rebuttal adequately address the reviewers' concerns. I recommend accepting the paper.