# Topology Optimisation under Uncertainties with Neural Networks
## 3 Neural Network Architectures
### Topology Convolutional Neural Networks (TCNN)
#### 3.1.2 Data Preparation
Let \(\text{Conv}(\ \cdot\ ;\theta):\mathbb{R}^{\mathbb{C}_{\text{in}}\times H \times W}\to\mathbb{R}^{\mathbb{C}_{\text{out}}\times H\times W}\) be a convolutional layer and \(\mathcal{I}\in\mathbb{R}^{T\times\mathbb{C}_{\text{in}}\times H\times W}\) a sequence of inputs ordered by the discrete time dimension \(T\in\mathbb{N}\). A convolutional LSTM layer to an input sequence \(\mathcal{I}_{t\leq T}\) and \(\mathcal{H}_{0}=0\in\mathbb{R}^{\mathbb{C}_{\text{in}}\times H\times W},\ \mathcal{C}_{0}=0\in \mathbb{R}^{\mathbb{C}_{\text{in}}\times H\times W}\) (since at \(t=1\) we do not yet have any information about earlier steps in the sequence) is given by a system of equations,

\[ \begin{split}\hat{i}_{t}(\mathcal{I}_{t},\mathcal{H}_{t-1}, \mathcal{C}_{t-1})&=\sigma\Big{(}\text{Conv}(\mathcal{I}_{t}; \theta_{xi})+\text{Conv}(\mathcal{H}_{t-1};\theta_{hi})+\mathcal{W}_{ci}\odot \mathcal{C}_{t-1}\Big{)},\\ \hat{f}_{t}(\mathcal{I}_{t};\mathcal{H}_{t-1},\mathcal{C}_{t-1})& =\sigma\Big{(}\text{Conv}(\mathcal{I}_{t};\theta_{xf})+\text{Conv}( \mathcal{H}_{t-1};\theta_{hf})+\mathcal{W}_{cf}\odot\mathcal{C}_{t-1}\Big{)},\\ \mathcal{C}_{t}(\hat{f}_{t},\mathcal{C}_{t-1},\hat{i}_{t}, \mathcal{I}_{t})&=\hat{f}_{t}\odot\mathcal{C}_{t-1}+\hat{i}_{t} \odot\tanh\Big{(}\text{Conv}(\mathcal{I}_{t};\theta_{xc})+\text{Conv}( \mathcal{H}_{t-1};\theta_{hc})\Big{)},\\ \hat{o}_{t}(\mathcal{I}_{t},\mathcal{H}_{t-1},\mathcal{C}_{t})& =\sigma\Big{(}\text{Conv}(\mathcal{I}_{t};\theta_{xo})+\text{Conv}( \mathcal{H}_{t-1};\theta_{ho})+\mathcal{W}_{co}\odot\mathcal{C}_{t}\Big{)}, \\ \mathcal{H}_{t}(\hat{o}_{t},\mathcal{C}_{t})&=\hat{o}_{t} \odot\tanh\left(\mathcal{C}_{t}\right),\end{split}  \tag{25}\]

with \(t\leq T,\ t\in\mathbb{N}\) and \(\mathcal{H}\in\mathbb{R}^{T\times\mathbb{C}_{\text{out}}\times H\times W}\) the output of the convolutional LSTM layer as well as \(\mathcal{W}_{ci}\), \(\mathcal{W}_{cf}\in\mathbb{R}^{\mathbb{C}_{\text{in}}\times H\times W}\), \(\mathcal{W}_{co}\in\mathbb{R}^{\mathbb{C}_{\text{out}}\times H\times W}\) in Equation (24). The subscript \(t\) indicates a cutout of the \(t\)-th element of sequence dimension \(T\) of the respective tensor. **Definition 4** (LSTM layer). : _Let \(L,H,W,T,C_{in},C_{out}\in\mathbb{N}\) as well as \(\mathcal{W}_{ci},\mathcal{W}_{cf}\in\mathbb{R}^{C_{in}\times H\times W}\), \(\mathcal{W}_{co}\in\mathbb{R}^{C_{out}\times H\times W}\) and parameter vectors, specifying the convolutional layer as in Definition 3 for \(L=1\) from Equation (25),_

\[\theta_{xi}\in\mathbb{R}^{d_{xi}},\theta_{hi}\in\mathbb{R}^{d_{hi}},\theta_{xc} \in\mathbb{R}^{d_{xc}},\theta_{hc}\in\mathbb{R}^{d_{hc}},\]

\[\theta_{xf}\in\mathbb{R}^{d_{xf}},\theta_{hf}\in\mathbb{R}^{d_{hf}},\theta_{xo} \in\mathbb{R}^{d_{xc}},\theta_{ho}\in\mathbb{R}^{d_{ho}},\]

_and described by the parameter vector \(\theta\in\mathbb{R}^{d}\), with_

\[d=d_{xi}+d_{hi}+d_{xc}+d_{hc}+d_{xf}+d_{hf}+d_{xo}+d_{ho}+2\cdot C_{in}\cdot H \cdot W+\cdot C_{out}\cdot H\cdot W.\]

_Furthermore, let \(\sigma:\mathbb{R}\rightarrow\mathbb{R}\) with \(\sigma(x)=\frac{1}{1+e^{x}}\) and \(\tanh\) be evaluated element-wise. We call a function,_

\[\text{LSTM}(\ \cdot\ ;\theta):\mathbb{R}^{3\times T\times C_{in}\times H \times W}\rightarrow\mathbb{R}^{2\times T\times C_{out}\times H\times W},\]

_an LSTM layer, if it satisfies the mapping rule given by the system of Equation (25)._

For the forecasting of our gradient sequence, we use an encoder-decoder architecture (i.e., an \"autoencoder\") consisting of \(2L,\ L\in\mathbb{N}\), LSTM layers,

\[\text{LSTM}^{l}(\ \cdot\ ;\theta^{l}):\mathbb{R}^{3\times T\times C_{in}^{l} \times H\times W}\rightarrow\mathbb{R}^{2\times T\times C_{out}^{l}\times H \times W},\]

that satisfies the mapping rule given by the equation system (25) with \(1\leq l\leq 2L\). Therefore, the encoding and decoding blocks of the autoencoder have the same number of layers \(L\in\mathbb{N}\).