# Topology Optimisation under Uncertainties with Neural Networks
## 3 Neural Network Architectures
### Topology Convolutional Neural Networks (TCNN)
#### 3.1.1 TCNN Architecture  
We follow the presentation of the pytorch documentation [18]. The input of a layer of the CNN architecture is a tensor \(\mathcal{I}\in\mathbb{R}^{S\times\mathbb{C}_{m}\times H\times W}\). Here, \(S\in\mathbb{N}\) is the number of input samples, in our case the evaluations of \(\varphi\) and \(u\) as presented in Section 3.1.2. It is therefore possible to calculate the gradient step \(\varphi_{n}\) from Equation (6) for several different loads \(g\) simultaneously. This way, Monte Carlo estimates become very efficient. \(C_{in}\in\mathbb{N}\) corresponds to the number of input channels and each channel represents one dimension of an input (\(\varphi\) or \(u\)). \(H\in\mathbb{N}\) and \(W\in\mathbb{N}\) provide information about the dimension of the discretisation of the space \(D\). The output of one CNN layer is specified by \(\mathcal{O}\in\mathbb{R}^{S\times C_{out}\times H\times W}\). For fixed \(s\leq S\) and \(i\leq C_{\text{out}}\in\mathbb{N}\) with \(C_{\text{out}}\) the number of output channels is given as  
\[\mathcal{O}_{s,i}(\  
_Furthermore, let \(\sigma^{1},\ldots,\sigma^{L}\) be given continuously differentiable activation functions. We call an NN of the form of Equation (17) an \(L\)-layer topology convolutional neural network (TCNN) and characterise it as the mapping_  
\[\mathcal{N}_{\text{CNN}}(\ \cdot\ ;\theta):\mathbb{R}^{\mathbb{C}_{\text{in}} \times H\times W}\rightarrow\mathbb{R}^{\mathbb{C}_{\text{out}}\times H \times W}.\]  
The approximation of \(\mathcal{N}_{\text{CNN}}\) is hence determined by its parameter vector \(\theta\). For general CNNs, the dimension \(H\times W\) does not have to be constant across the different layers. The same holds true for the dimensions \(H_{K}\times W_{K}\) of the kernel matrices. In fact, before implementing the convolution, we embed each channel of our input in a \((H+\lfloor\frac{H_{K}}{2}\rfloor)\times(W+\lfloor\frac{W_{K}}{2}\rfloor)\) space to preserve the dimension in the output.  
**Example 1**.: _The following specific TCNN has proved to be the most suitable for integration into Algorithm A1 for the selections of hyperparameters we have investigated. The architecture is given as an \(L=6\) layer TCNN with \(C_{\text{in}}=3\) input channels, \(C_{out}^{1}=15\) for \(1<l<5\) hidden channel, \(C_{out}^{6}=1\) output channel and kernel size \(H_{K}=W_{K}=3\) as well as trained weights described by \(\theta\in\mathbb{R}^{8806}\) which determine the mapping by_  
\[\mathcal{N}_{\text{CNN}}(\ \cdot\ ;\theta):\mathbb{R}^{3\times 201\times 101} \rightarrow\mathbb{R}^{1\times 201\times 101}, \tag{18}\]  
_with activation function \(\sigma^{6}(x):=\min\{\max\{x,0\},1\}\). In contrast to many standard architectures, only the activation function of the output layer is not the identity. We chose \(\mathbb{R}^{3\times 201\times 101}\) as input space in anticipation of the setting in Example 2, reflecting our mesh choice to discretise domain \(D=[-1,1]\times[0,1]\) with \(201\times 101\) nodes, which for first-order finite elements then is the dimension of the discrete functions \(u\) and \(\varphi\). The architecture is depicted in Figure 1._