# Topology Optimisation under Uncertainties with Neural Networks
## 3 Neural Network Architectures
### Topology Convolutional Neural Networks (TCNN)
#### 3.1.2 Data Preparation
The autoencoder for an input sequence \(\mathcal{I}\in\mathbb{R}^{T\times C_{in}\times H\times W}\) can be described by the following system of equations of the encoder block,

\[ \begin{split}\text{LSTM}^{1}(\mathcal{I}_{t},\mathcal{C}_{t-1}^ {1},\mathcal{H}_{t-1}^{1};\theta^{1})&=[\mathcal{C}_{t}^{1}, \mathcal{H}_{t}^{1}],\\ \text{LSTM}^{2}(\mathcal{H}_{t}^{1},\mathcal{C}_{t-1}^{2}, \mathcal{H}_{t-1}^{2};\theta^{2})&=[\mathcal{C}_{t}^{2}, \mathcal{H}_{t}^{2}],\\ &\vdots\\ \text{LSTM}^{L-1}(\mathcal{H}_{t}^{L-2},\mathcal{C}_{t-1}^{L-1}, \mathcal{H}_{t-1}^{L-1};\theta^{L-1})&=[\mathcal{C}_{t}^{L-1}, \mathcal{H}_{t}^{L-1}],\\ \text{LSTM}^{L}(\mathcal{H}_{t}^{L-1},\mathcal{C}_{t}^{L}, \mathcal{H}_{t-1}^{L-1};\theta^{L})&=[\mathcal{C}_{t}^{L}, \mathcal{H}_{t}^{L}],\end{split}  \tag{26}\]

with \(1\leq t\leq T_{en},\ t\in\mathbb{N}\). This is combined with the following decoder block by setting \(\widetilde{\mathcal{O}_{0}}=\mathcal{H}_{T_{en}}^{L}\) and \(\mathcal{H}_{0}^{L+1}=\mathcal{H}_{T_{en}}^{1},\ldots,\mathcal{H}_{0}^{2L}= \mathcal{H}_{T_{en}}^{L}\) and \(\mathcal{C}_{0}^{L+1}=\mathcal{C}_{T_{en}}^{1},\ldots,\mathcal{C}_{0}^{2L}= \mathcal{C}_{T_{en}}^{L}\),

\[ \begin{split}\text{LSTM}^{L+1}(\widetilde{\mathcal{O}}_{t-1}, \mathcal{C}_{t-1}^{L+1},\mathcal{H}_{t-1}^{L+1};\theta^{L+1})&=[ \mathcal{C}_{t}^{L+1},\mathcal{H}_{t}^{L+1}],\\ \text{LSTM}^{L+2}(\mathcal{H}_{t}^{L+1},\mathcal{C}_{t-1}^{L+2}, \mathcal{H}_{t-1}^{L+2};\theta^{L+2})&=[\mathcal{C}_{t}^{L+2}, \mathcal{H}_{t}^{L+2}],\\ \vdots\\ \text{LSTM}^{2L-1}(\mathcal{H}_{t}^{2L-2},\mathcal{C}_{t-1}^{2L-1},\mathcal{H}_{t-1}^{2L-1};\theta^{2L-1})&=[\mathcal{C}_{t}^{2L-1},\mathcal{H}_{t}^{2L-1}],\\ \text{LSTM}^{2L}(\mathcal{H}_{t}^{2L-1},\mathcal{C}_{t}^{2L}, \mathcal{H}_{t-1}^{2L};\theta^{2L})&=[\mathcal{C}_{t}^{2L}, \widetilde{\mathcal{O}}_{t}],\end{split}  \tag{27}\]

with \(1\leq t\leq T_{dec},\ t\in\mathbb{N}\). It should be mentioned that the input and output sequences do not have to be of the same length (in fact, in general \(T_{en}\
eq T_{dec}\)). Furthermore, \(C_{out}^{l-1}=C_{in}^{l}\) holds for individual LSTM layers \(2\leq l\leq 2L\) defined in Equations (26) and (27). Especially, since the output sequence \(\widetilde{\mathcal{O}}\) is also input of \(\text{LSTM}^{L+1}\), it holds that \(C_{out}^{2L}=C_{in}^{L+1}\). In order to be able to select the dimensions of the output tensor \(\widetilde{\mathcal{O}}\) completely independently of the hidden channels, we additionally apply a convolutional layer \(\text{Conv}(\ \cdot\ ;\theta^{\text{final}}):\mathbb{R}^{C_{out}^{2L}\times H \times W}\rightarrow\mathbb{R}^{C_{final}\times H\times W}\)with activation function \(\sigma_{\text{final}}:\mathbb{R}\rightarrow\mathbb{R}\) to concatenate the hidden channel to an arbitrary number of output channels \(C_{\text{final}}\in\mathbb{N}\) given by

\[\text{Conv}(\widetilde{\mathcal{O}}_{t};\theta^{\text{final}})=\mathcal{O}_{t} \quad,1\leq t\leq T_{\text{dec}}. \tag{28}\]

**Definition 5** (TLSTM architecture). : _Let \(L,H,W,T_{en},T_{dec},C_{final},C^{1}_{in}\in\mathbb{N}\) as well as \(C^{l}_{out}\in\mathbb{N}\), with \(1\leq l\leq 2L\) be given. Hence, the respective LSTM layers from the encoder defined in Equation (26) and decoder in (27) block as well as the output layer in Equation (28) can be described by the parameter vectors of the CNN and LSTM layers (see Definition 3 for \(L=1\) and Definition 4) \(\theta^{\text{final}}\in\mathbb{R}^{d^{f}},\quad\theta^{l}\in\mathbb{R}^{d}\) with \(d^{f}\in\mathbb{N},d^{l}\in\mathbb{N}\) for \(1\leq l\leq 2L\) and an activation function \(\sigma_{\text{final}}:\mathbb{R}\rightarrow\mathbb{R}\) of the output layer.