# Topology Optimisation under Uncertainties with Neural Networks
## 3 Neural Network Architectures
### Topology Convolutional Neural Networks (TCNN)
#### 3.1.2 Data Preparation
Using the optimiser in Algorithm A1, \(S\in\mathbb{N}\) sample paths of gradient steps \(\varphi(g)\) and solutions of the state equation \(u(g)\) are generated by drawing \(S\) realisations of \(g\). In contrast to Section 4.1.1, this time we do not only store every \(T\in\mathbb{N}\) iteration step of \(\varphi_{n}\) and \(u_{n}\) but instead store all iteration steps of the optimisation of Algorithms A1. Afterwards, these are merged into disjoint subsets, each consisting of a sequence of \(T\) iteration steps. Thus, the feature or the sequence \(\varphi_{n+1},\dots,\varphi_{n+T}\) is the label for the assembled sequence from \(\varphi_{n-T},\dots,\varphi_{n}\) and \(u_{n-T+1},\dots,u_{n+1}\). More precisely, with

\[\left( \begin{bmatrix}\varphi_{n-T}&u_{n-T+1}^{x}&u_{n-T+1}^{y}\\ &\vdots&\\ \varphi_{n}&u_{n+1}^{x}&u_{n+1}^{y}\end{bmatrix} , \begin{bmatrix}\varphi_{n+1} \\ \vdots\\ \varphi_{n+T}\end{bmatrix} \right),\]

for \(0\leq n\leq N-(T-1)\), a total of \(S(\lfloor\frac{N}{k}\rfloor-1)\) tuples are stored in an unsorted dataset \(\mathcal{D}_{\text{LSTM}}\). ### TLSTM Predictions

After training the TLSTM from Equation (29) with the data set \(\mathcal{D}_{\text{LSTM}}\) generated in Section 4.2.1, we wish to investigate its predictive ability of \(\mathcal{N}_{\text{LSTM}}^{T}(\varphi_{n}):=\mathcal{N}_{\text{LSTM}}^{T}( \varphi_{n-T},\dots\), \(\varphi_{n},u_{n-T+1},\dots,u_{n+1};\theta)\) compared to the real sequence \((\varphi_{n+T})_{n}\). For this purpose we have visualised both sub-sequences for \(T=10\) in Figure 9 using the iteration sequence generated for a load \(g=(0,-5000)\). The distorted topology in the first forecasts is striking. This can be attributed to the comparatively low weighting of training data in which the distribution of the material is constant \(\varphi_{n}(x)=0,5\) for all \(x\in D\) or almost constant. This forces us to choose a correspondingly high \(c_{m}\in\mathbb{N}\) in Algorithm 3. Furthermore, it can be seen that especially in early phases of the partial sequence in which the change \(\|\varphi_{n}-\varphi_{n+1}\|\) is very high, \(\mathcal{N}_{\text{LSTM}}^{10}(\varphi_{n})\) provides a better forecast from a visual perspective, i.e., the topology \(\mathcal{N}_{\text{LSTM}}^{10}(\varphi_{n})\) has already converged further than the target image \(\varphi_{n+10}\). Since the topologies on the finer meshes no longer show any major visual changes and therefore the differences in the predictions are no longer recognisable, we have decided not to present them at this point. The architecture of the TLSTM allows the length of the input sequence as well as the output sequence to be chosen independently of the training data. The expected consequence is a decrease in prediction quality. Despite this, Figure 10 depicts the prediction results of Equation (29) with unchanged input sequence (\(T_{en}=5\)) and output sequence of length \(40\)Since a shorter output sequence (\(T_{des}=10\)) is used to train Equation (29), the results of the longer output sequence indicate that \(N_{\text{LSTM}}\) has indeed learned to predict a gradient step for the given setting and that the training data from Section 4.2.1 describe the problem correctly. Analogous to Section 3.1.1, the intention behind the construction of \(\mathcal{N}_{\text{LSTM}}\) is to replace the gradient step in reference Algorithm 1 with Example 3. Algorithm 3 describes the integration of the LSTM prediction. When evaluating \(\mathcal{N}_{\text{LSTM}}^{T}\), only the last iteration step \(\varphi_{n+T}\) of the predicted sequence is projected back to the current mesh \(\mathcal{T}_{m}\) in order to save computational resources. We examine the performance of Algorithm 3 in the following deterministic experiment. As for the TCNN above, the beneficial performance of a single-gradient prediction transfers to the stochastic setting since it consists of a Monte Carlo estimator with \(N\in\mathbb{N}\) samples in each step. It is hence not necessary to examine this in more detail.