# Computational Acceleration of Topology Optimization Using Deep Learning
## 4 Model Design
### U-Net  
U-Net is a special type of CNN designed for solving image segmentation tasks. It consists of two main stages: contracting and expanding. During the first stage, the model generates N small, filtered versions of the image. In the next stage, all obtained images are upsampled and filtered for receiving the final solution. The solution for this task is the binary image, where 0's stands for void and 1's for the material.  
U-Net consists of the following layers: convolution, pooling, dropout, and upsampling.  
The convolutional layer is a set of learnable filters, which have a small receptive field but extended through the full depth of the input volume. In the scope of our experiment, at each step of the first stage, the total size of the frame is going to be decreased twice, and the number of filters will be increased twice. For the next stage, we apply reverse logic: increase the frame size and decrease the filter size at each step.  
Figure 5: Visual representation of CNN modules for 2D and 3D objects.  
Pooling layer: The process of sample-based discretization, where the goal is to down sample the input matrix. In the scope of this experiment, we have used a stride of two (N = M = 2) and the maximum value of each region has been assigned to the resulting matrix:  
\[f^{\prime}(x,y)=\max_{i,j\in\{0,1\}}\left[f(N*x+i,M*y+j)\right] \tag{10}\]  
Dropout (dilution) layer. This is a regularization technique used in artificial neural networks for preventing overfitting. Basically, it sets random values of the dataset to zero and forces machine learning algorithms to learn in a more efficient way.  
Upsampling is the process of increasing the width and height of the input matrix by N, M times, correspondingly. In the experiments we have used N = 2, M = 2 and filled the matrix using the following formula:  
\[f^{\prime}(x,y)=f\Big{(}int\Big{(}\frac{x}{N},\frac{y}{N}\Big{)}\Big{)} \tag{11}\]  
where \(f^{\prime}(x,y)\) is a new matrix and \(f(x,y)\) is the initial matrix.  
The architecture of the U-Net looks like a symmetrically mirrored convolutional neural network. It consists of two stages: contracting (encoder) and expanding (decoder).  
Stage 1: Contracting. This stage follows the regular architecture of CNN. It consists of N repeated applications of two 3 \(\times\) 3 convolutions each followed by rectified linear unit (ReLU) and a 2 \(\times\) 2 max pooling operation with stride 2. In our work, the initial number of feature channels is taken to be 8 and the number of channels is increased in a sampling operation.  
Stage 2: Expanding. This stage is the reverse of the first one with only one difference: at the beginning of each step, the corresponding feature map is cropped from the contracting path and concatenated with the processed one. The resulting feature map passes through two 3 \(\times\) 3 convolutions followed by ReLU activation function. Moreover, the generated tensor is upsampled, where the dimensions of the frame are increased twice. In the end, the resulting tensor passes through the last convolutional layer with only one feature channel followed by the sigmoid activation function.  
In our experiments, we use the following loss function for the training process:  
\[L_{\triangle}(e)=\left\{\begin{array}{c}\frac{1}{e}e^{2}\text{ for }|e|\leq\ \triangle\\ \triangle*\Big{(}|e|-\frac{1}{2}\triangle\Big{)}\text{ otherwise}\end{array}\right. \tag{12}\]  
where, \(e\) is the value of error and \(\triangle\) is a predefined coefficient which is equal to 0.5 in the scope of our experiments.