# Machine Learning Techniques in Structural Wind Engineering: A State-of-the-Art Review
## 2 ML Methods Used in Structural Wind Engineering
### Artificial Neural Network (ANN)  
The concept of ANN is derived from biological sciences, where it mimics the complexity of the human brain in recognizing patterns through biological neurons, and thus imitates the process of thinking, recognizing, making decisions, and solving problems [42; 43]. ANN was the most popular method found in the reviewed literature to predict wind-induced pressures compared to other neural network methods (e.g., CNN or RNN). ANN is robust enough to solve multivariate and nonlinear modeling problems, such as classification and predictions. ANN is a group of layers that comprise multiple neurons at each layer and is also known as a feed-forward neural network (FFNN). It is composed of input layers, where all the variables are defined and fed into the hidden layers which are weighted and fed into the output layers that represent the response of the operation. The ANN architecture could be written as x-h-h-y which defines x number of inputs (variables), h number of hidden layers, and y number of outputs (responses) as shown in Figure 1. Each hidden layer comprises a certain number of neurons that gives a robust model, and this could be achieved by training and trials. The hidden layers are composed of activation functions that apply different weights to the input layer and transfer them to the output layers. The most common activation functions are the nonlinear continuous sigmoid, the tangent sigmoid, and the logarithmic sigmoid [44]. The weights are multiplied with the inputs and calibrated through a training process between the input and output layers to reduce the loss. The training process is applied using the Levenberg-Marquardt backpropagation algorithm, which belongs to the family of the Multi-Layer Perceptron (MLP) network [45] and was originally proposed by [PERSON] et al. [46]. It consists of two steps: feed-forward the values to calculate the error, and then propagate back the error to previous layers [47,48]. The repeated iteration process (epochs) of backpropagation network error continues and it keeps adjusting the interconnecting weights until the network error is reduced to an acceptable level. Once the most accurate solution is formed during the training process, the weights and biases are fixed, and the training process stops. The Levenberg-Marquardt is a standard numerical method which achieves the second-order training speed with no need to compute the Hessian matrix and was demonstrated to be efficient with training networks up to a few hundred weights [47,49]. Figure 2 shows the output signal for a generic neuron j in the hidden layer \(h\) defined in Equation (1), where \(w_{ij}^{h}\) is the weight that connects the \(i\)th neuron of the current layer to the \(j\)th neuron of the following layer, \(x_{i}\) is the input variable, b is the bias associated with the \(j\)th neuron to adjust the output along with the weighted sum, and \(f\) is the activation function that is usually adapted as either a tangent sigmoid or a logarithmic sigmoid, Equations (2) and (3), respectively. The (RBF-NN) that was used first by [50] is a function whose response either decreases or increases with the distance from a center point [51,52]. \[y_{j}^{h}\;=\;f\;(\sum_{i\;=\;1}^{n}w_{ij}^{h}x_{i}^{h}+b_{j}^{k}) \tag{1}\]  
\[f\;(u)\;=\;-1+\frac{2}{(1+e^{-2u})} \tag{2}\]  
\[f\;(u)\;=\;\frac{1}{(1+e^{-u})} \tag{3}\]  
Figure 1: Feed-forward neural network architecture. During the training process of BPNN, usually, the training is terminated when one of the following criteria is first met: (i) fixing the number of epochs to a certain number, (ii) the training error is less than a specific training goal, or (iii) the magnitude of the training gradient is less than a specified small value (i.e., \(1.0\times 10^{-10}\)).