# Extracting Weld Bead Shapes from Radiographic Testing Images with U-Net
## 3 Data and Training
### Deep Learning Model Training  
U-Net was used to extract the features from the weld images and predict the shape and location of the weld beads. For training, U-Net received weld images and mask images that would become the training target images. The original weld images and mask images that would become the training target images were combined into the Tfrecord files, and these were used as the training dataset. High-quality images reduce the training speed, making it difficult to achieve good performance as there are many parameters that must be learned. Therefore, to perform training efficiently, the image pixel sizes were reduced to 256 \(\times\) 256 before training was performed. Additionally, the input images were normalized to (0,1), and the activation function of each layer was set up so that ReLU [20] was used to apply the normalized values. The training was performed with a batch size of 1 and a learning rate of 0.00001; 18,000 iterations were performed. Binary cross entropy [21] was used as the loss function, and Adam [22] was used as the optimization function. Algorithm 1 shows a detailed algorithm. Table 1 presents hardware and software for the experimental environment. Depending on the hyperparameters and the experimental environment, the time required for the experiment varies. The model was trained using only the training dataset, and the trained model was tested using the test dataset. Figure 6 shows the results of testing the model that trained 2500 images in which CLAHE and data augmentation are performed: (a) is the actual weld image used as the input, (b) is the actual mask image, and (c) is the mask image that was predicted via the training process. Figure 7 shows the loss functions of the training and test data. Figure 7 shows the loss function when training 500 raw images, Figure 7 shows the loss function when training 500 images in which CLAHE is performed, Figure 7 shows the loss function training 2500 images in which CLAHE and data augmentation are performed. From the loss function, it can be determined that overfitting did not occur in the model.  
\begin{table}
\begin{tabular}{c c} \hline \hline  & **Value** \\ \hline OS & Window 10 64 bit \\ CPU & Intel(R) Core(TM) [EMAIL_ADDRESS] GHz \\ RAN & 32 GB \\ GPU & NVIDIA TITAN Xp \\ \hline \hline \end{tabular}
\end{table}
Table 1: Experimental environment.  
```
Input: Image pixel feature \((\mathbf{X}_{i},\mathbf{y}_{i})\), \(i=1,\ \dots,\ N\), and training step \(t\). Output: U-net model parameters \(W\), Predicted values \(\mathbf{\hat{y}}\). Loss function: \(\begin{array}{l} BCE=-\frac{1}{N}\sum_{i=1}^{N}\mathbf{y}_{i}\times log(p( \mathbf{y}_{i}))+(1-\mathbf{y}_{i})\times log(1-p(\mathbf{y}_{i}))\end{array}\) Optimization: Adam  Require: \(\alpha\): Step size  Require: \(\beta_{1},\beta_{2}\ \in[0,1]\)  Require: Model parameters \(W\) \(m_{0}=0\) (Initializer initial 1 st moment vector) \(v_{0}=0\) (Initializer initial 2 nd moment vector) \(t=0\) (Initializer timestep) While\(W_{t}\) not converged do \(t=t+1\) \(g_{t}=\
abla_{t}f_{t}(W_{t-1})\) \(l_{t}=lr_{t-1}\times\sqrt{1-\beta_{2}^{t}}/\left(1-\beta_{1}^{t}\right)\) \(m_{t}=\beta_{1}\times m_{t-1}+(1-\beta_{1})\times g_{t}\) \(v_{t}=\beta_{2}\times v_{t-1}+(1-\beta_{2})\times g_{t}^{2}\) \(W_{t}=W_{t-1}-lr_{t}\times m_{t}/(\sqrt{v_{t}}+\epsilon)\) End While Return\(W_{t}\) (Resulting parameters)
```  
**Algorithm 1** Learning for image segmentation.  
Figure 6: Results of the testing of 2500 images model in which CLAHE and data augmentation are performed: (**a**) input images; (**b**) true mask images; (**c**) predicted mask images.