## 2 Machine Learning and Defect Size: Algorithms
### Neural Networks Architecture  
In this Section, the architectures of the ML algorithms developed to estimate the probability of a specific defect (Section 2.3.1) and the location and the scale parameters of the defect size distribution, i.e., the LEVD distribution (Section 2.3.2), are described.  
For both approaches, supervised feed-forward neural networks (FFNNs) have been employed. FFNNs are the simplest type of artificial neural network and aim at learning a function that maps the input data to the output variables. This function is estimated through a learning process based on the training dataset, which is composed of an input object, generally a vector of input data, and by the expected outputs, i.e., the results experimentally assessed given the input data. In detail, a FFNN is composed of \(N\) layers, each one containing \(n\) operational blocks that are the so-called neurons. The hidden layers are intermediate layers between the input and the output layers, and within them, the following operation is carried out:  
\[y=A\cdot\left(\sum w_{i}\cdot x_{i}+b\right) \tag{2}\]  
where \(x_{i}\) is the output of the previous layer or the input variable of the NN, \(w_{i}\) is the weight that multiplies each input variable, \(b\) is the bias of the neuron, \(A(\cdot)\) is the activation function and \(y\) is the output of the layer, with \(i=1\ldots n\), where \(n\) is the number of neurons. \(w_{i}\) and \(b\) are the learnable parameters, whereas the number of neurons \(n\), the number of layers \(N\) and the activation functions are the hyperparameters of the model. During the training process, the learnable parameters are randomly initialized and their value is upgraded with a back propagation scheme to optimize the loss function, which compares the NN predicted output with the target output available in the training dataset. Practically, the loss function measures the capability of the NN of modelling the training data. The back propagation is carried out iteratively in each epoch over a subset of data (i.e., the batch), to minimize the loss function.  
The above-described architecture should be designed to assess the set of trainable parameters that minimize the loss function, i.e., that maximize the predictive capability of the FFNN. In particular, the number of layers, the number of neurons for each layer, the number of epochs, and the batch size should be defined to maximize the performance of the FFNN. For both FFNN described in Sections 2.3.1 and 2.3.2, the hyperparameters have been optimized through an iterative process to achieve the best ML performances. The Mean Square Error (MSE) loss function has been considered in both ML algorithms:  
\[\text{MSE}=\frac{\sum_{i}^{N_{b}}\left(\overline{y}_{i}-y_{i}^{2}\right)}{N_{ b}} \tag{3}\]  
where \(N_{b}\) is the batch size, \(\overline{y}\) is the real output and \(y\) is the predicted output. Different loss functions can be considered during the training and the validation process. However, the MSE loss function has been selected for its regularization capability and since it \"amplifies\" high errors and is less affected by small differences and deviations. For this application, this is important since small deviations are accepted, due to the stochastic nature of the phenomenon, whereas higher deviations are to be avoided and are properly \"amplified\" by the MSE loss function.