# Topology Optimisation under Uncertainties with Neural Networks
## 3 Neural Network Architectures
### Topology Convolutional Neural Networks (TCNN)
#### 3.1.2 Data Preparation
Regarding the chosen architectures, an interesting extension would be to consider graph neural networks (GNN) since there, the underlying mesh structure is mapped directly to the NN. Consequently, the costly transfer operators from current mesh to reference mesh of the design space could be alleviated, removing perhaps the largest computational burden of our approach. Moreover, transformer architectures have probably superseded LSTMs and it would be worth examining this modern architecture in the context of this work. The loss function used in the training also leaves room for improvements. For example, instead of the simple mean squared error used here, one could approximate the objective functional of Equation (5) directly in the loss function. Regarding the training process, there are modern techniques to improve the efficiency and alleviate over-fitting such as early stopping, gradient clipping, adaptive learning rates and data augmentation as discussed in [25]. Moreover, transfer learning in a limited-data setting could substantially reduce the amount of training data required. This work mainly serves as a proof of concept for treating the considered type of optimisation problems with modern NN architectures. An important step towards practicability is the further generalisation of this model, e.g., to arbitrary problems (with parameterised boundary data and constraints), determined by descriptive parameters drawn from arbitrary distributions according to the problem at hand. Moreover, the models presented here can be used as a basis for theoretical proofs (e.g., regarding the complexity of the representation) and further systematic experiments. Conceptualisation, M.E. and J.N. ; methodology, M.E., M.H. and J.N. ; software and numerical experiments, M.H. ; investigation, M.H. and J.N. ; writing--original draft, M.E. and M.H. ; supervision, M.E. ; project administration, M.E. ; funding acquisition, M.E. All authors have read and agreed to the published version of the manuscript. The first author acknowledges partial funding by the DFG SPP 1886. Not applicable. We thank the WIAS for providing the IT infrastructure to conduct the presented numerical experiments. We are also grateful to the anonymous reviewers and [PERSON] for remarks and suggestions that improved the original manuscript. The authors declare no conflict of interest. The following abbreviations are used in this manuscript: \begin{tabular}{l l} CNN & convolutional neural network \\ CVaR & conditional value at risk \\ DNN & deep neural network \\ FEM & finite element method \\ GNN & graph neural network \\ LSTM & long short-term memory \\ MC & Monte Carlo \\ NN & neural network \\ PDE & partial differential equation \\ TCNN & topology CNN \\ TLSTM & topology LSTM \\ TO & topology optimisation \\ \end{tabular} 

## Appendix A Bridge Benchmark Problem

### Deterministic Bridge Optimisation

For a comparison between Algorithm A1 from [1,2] and its extension to NNs, we make use of a bridge benchmark problem at selected locations. The name comes from the optimal shape resembling a bridge, which exhibits the best stiffness under the given constraints and forces acting on it. To be specific, the parameters are given as follows: Assume design domain \(D=[-1,1]\times[0,1]\) with boundaries \(\Gamma_{D}=[-1,-0.9]\times\{0\}\), \(\Gamma_{g}=[-0.02,0.02]\times\{0\}\) on which the load \(g=(0,-5000)^{T}\) is applied and the slip condition \(\Gamma_{S}=[0.9,1.0]\times\{0\}\) is set. The Lame coefficients are given by \(\mu_{\text{mat}}=\lambda_{\text{mat}}=150\). Furthermore, the volume constraint is \(m=0.4\) and \(\epsilon=\frac{1}{16}\). This is the same setting as in the deterministic experiment in [1,2]. The initial material distribution is given by \(\varphi_{0}(x)=0.5\)\(\forall x\in D\). Several iteration results of \(\varphi_{n}\) from Algorithm A1 are depicted in Figure A1.