## 2 Materials and Methods
### Development of ANN Model for Predicting HM Concentrations
#### 2.6.3 Artificial Neural Networks  
Artificial neural networks are frequently used for modelling different phenomena. Although many more complex and less complex ANN have been developed, one of the most popular is the unidirectional multi-layer network--the multi-layer perceptron (MLP) (Figure 4). Figure 3: Graphic interpretation of the trend function. In ANN model, individual input signals (x\({}_{i}\)) that reach the input layer are multiplied by the values of weights (w\({}_{ij}\)), then by products xi\({}_{i}\)w\({}_{ij}\). Next, they are transmitted to the hidden layer neurons, where they are summed up (Figure 4). The sums obtained undergo transformation by the activation function \(f\) (linear or nonlinear), and then they are transmitted to the output layer neurons. In the MLP model, the values of weights (w\({}_{ij}\)) are estimated at the training stage. The estimation is intended to minimise the following function [23]:  
\[\mathrm{E}(\mathrm{RMSE})=\frac{1}{2n}\mathrm{\sum}_{i=1}^{n}\left(y_{m}-y_{s }\right)^{2}, \tag{2}\]  
where E(RMSE)--target function, in the case under consideration equivalent to RMSE, n--number of data in a set, y\({}_{s}\)--value y obtained through iteration and y\({}_{m}\)--value y obtained through measurements. That is achieved with appropriate numerical methods, for instance, that of the Broyden-Goldfarb-Shanno. The values of the dependent variable outputs (y) are determined using the following formula [23]:  
\[\mathrm{y}=\mathrm{\sum}_{i=1}^{J}w_{i}f\left(\mathrm{\sum}_{i=1}^{1}w_{ij}x_{ i}+b_{j}\right), \tag{3}\]  
where I--number of model inputs, J--number of neurons in the hidden layer, w\({}_{ij}\)--the values of weights between inputs and neurons in the hidden layer, b\({}_{j}\)--loads of neurons in the hidden layer, w\({}_{j}\)--the values of weights between neurons in the hidden layer and the neuron in the output layer and f--the activation function. For the set of independent variables (x\({}_{i}\)), the Fisher-Snedecor test was used to find the optimal structure of the MLP model for the number of the hidden layer neurons ranging (i-2)(i+1) [24]. In the simulation tests, the following activation functions were taken into account: linear, exponential, tangent-hyperbolic and sine. The Broyden-Goldfarb-Shanno algorithm [23] was employed to determine the values of weights in the model. Due to substantial impact of the initial values of weights on the simulation results, and also problems with their optimisation, each ANN (having a specified number of neurons and activation function) was generated many times (5000 times) for different boundary conditions (initial values of weights for individual neurons). The above-mentioned algorithm was computed using the MATLAB software. The program automatically generates initial values [25], and the user can only choose the decay values for the hidden and output layers, which were assumed to be 0.001 and 0.0001 [8], respectively. The assessment of the predictive capacities of the models was based on coefficient determination (\(R^{2}\)), mean absolute error (MAE) and root-mean-square error (RMSE), as discussed in [26]. For the assumed values of neuron numbers and activation functions, the optimum values of weights were sought. The computations were performed using the method of successive approximations (for various numbers of iterations). In succession, for a number of neurons of the hidden layer, assumed  
Figure 4: The structure of the multi-layer perceptron artificial neural networks (ANN) model used for the study. a priori, activation functions were selected (hidden and output layers). With adopted assumptions, the computations of weight values were made using MATLAB software. Assuming successive values of neuron numbers to range from 1 to 2i+1 (1 neuron difference between successive models of concern), ANN models were determined for different combinations of the activation function. At the analysis stage, measures of fit of the computational results to measurements were obtained for the models produced.