## 2 Materials and Methods
### Machine Learning Methods
#### 2.1.3 Artificial Neural Networks  
ANNs are a set of useful machine learning techniques based on biological neural networks and are used to model complex relationships between inputs and outputs or to find patterns. The simplest processing element of a neural network is the neuron. Each neuron \(i\) may have multiple inputs, \(x_{1},x_{2},\ldots,x_{d}\), and synaptic weights, \(w_{i1},w_{i2},\ldots,w_{id}\) from which it produces a single output. More specifically, an input or propagation function \(g_{i}\) combines these inputs with their synaptic weights, and an activation function \(f_{i}\) is then subsequently applied to the obtained result to generate the corresponding output. Figure 2 graphically illustrates the model of a neuron \(i\), in which the input function is defined as \(g_{i}(x_{1},x_{2},\ldots,x_{d},w_{i1},w_{i2},\ldots,w_{id})=\sum_{j=1}^{d}w_{ ij}x_{j}+\theta_{i}\), and \(\theta_{i}\) is a bias value. This value is treated as a weight whose input is one. The output value of neuron \(i\) can therefore be expressed as \(y_{i}=f_{i}(\sum_{j=1}^{d}w_{ij}x_{j}+\theta_{i})\), where \(f_{i}\) is some activation function. Although a single neuron can perform some simple information processing functions, the power of ANNs arises from the connections between many of these simple and robust  
Figure 1: KNN algorithm for \(d=2\) predictor or dependent variables and \(K=7\). Figure 2: Diagram of an artificial neuron. units. Two typical network architectures, which differ in the types of connections between neurons, are the feedforward and recurrent networks. In a feedforward ANN, there are no cycles; that is, there is no closed path from any neuron to itself, meaning that the flow of information has a single direction. In a recurrent ANN, however, cycles are present, meaning that information can flow in any direction between the different layers, including the input layer and the output layer. An MLP network is a type of fully connected feedforward ANN in which backpropagation is used for training the network [49]. There are three types of layers in this neural network: the input layer, the hidden layers, and the output layer. Neurons in the input layer receive inputs and propagate them to the next layer. The hidden layers form the intermediate layers, while the output layer transfers the information from the network to the outside and gives the response for each input pattern. Figure 3 shows an example of an MLP network with a single hidden layer, where \(\{x_{1},x_{2},\ldots,x_{d}\}\) is the set of inputs accessing the network, \(w_{ij}\) denotes the synaptic weight between input neuron \(j\) and hidden neuron \(i\), and \(\hat{w}_{ki}\) denotes the synaptic weight between hidden neuron \(i\) and output neuron \(k\). As can be seen from this figure, the activation propagates in the network through the weights from the input layer to the intermediate layer. In this hidden layer, a certain activation function is applied to the inputs that reach it. The activation then propagates through the synaptic weights to the output layer. To train the network, two sets of weights must be updated: those between the input layer and the hidden layer, and those between the hidden layer and the output layer. Various activation functions can be used in an MLP network depending on the characteristics of the problem. Some of the most commonly used for regression problems are the logistic sigmoid function, the hyperbolic tangent function (tanh), and the rectified linear unit (ReLU) function [50]. To create the predictive model, a supervised and iterative learning process is followed, starting with an initial configuration of synaptic weights. These weights are adjusted iteratively according to the inputs of the training set, and a loss function is used to compare the output with the data to be predicted. One widely used loss function for regression problems is the mean squared error (MSE). The learning algorithm attempts to find the weights of the neural network that minimise this function.