## 2 Materials and Methods
### Basic and Hybrid Machine Learning Algorithms
#### 2.5.2 Training ANN  
* Basic ANN  
In this research, the training was conducted with 70% of the available data provided in the DOE phase. Then, 15% of the available data was used as a validation dataset and the rest, 15% as a test dataset. In the first step, the weights were imported to the network randomly. Applying the gradient descent method causes these weights to be continuously updated during successive iterations. To train the network, the mean square error (_MSE_) of the predicted values should move towards minimization. Equations (10)-(12) represent this method [9; 20]:  
\[E=MSE=\sqrt[1]{N}\sum_{i=1}^{N}\left(y_{p}-y_{t}\right)^{2} \tag{10}\]  
\[\Delta w_{ij}=-\eta\frac{\partial E}{\partial w_{ij}}\times O_{j} \tag{11}\]  
\[w_{ij}^{m}=w_{ij}^{m-1}+\Delta w_{ij} \tag{12}\]  
where \(\eta\) is the learning rate and controls the network convergence (a number between 0 and 1), \(E\) represents the _MSE_, \(N\) for the number of inputs, \(y_{t}\) desired output, predicted output by the ANN, and \(m\) indicates the iterations counter. * ANN + GA  
This method is similar to the basic ANN method, except that instead of using the gradient descend function used in basic ANN, the appropriateness of the weights assigned to each link is determined through the GA process, which is the selection of the best, the crossover, and the mutation, so that the MSE is minimized. In this algorithm, the probability of selecting each parent is assessed through Equations (13) and (14) [9]:  
\[f_{i}=\sqrt[k]{m_{i}} \tag{13}\]  
\[p_{i}=\sqrt[k]{i}\sum_{j=1}^{N}f_{i} \tag{14}\]  
where \(k\) is a coefficient, \(m_{i}\) symbolizes the fitness of \(i_{\text{th}}\) input, \(N\) substitutes for the number of generation population, and \(j\) indicates the number of generations. Equation (15) is applied to combine the two chromosomes of \(C_{i}\) and \(C_{h}\) to produce the next generation [9]. \[\left\{ \begin{matrix}C_{ij}-C_{ij(1-b)}+C_{ijb}\\ C_{ij}-C_{ij(1-b)}+C_{ijb}\end{matrix} \right. \tag{15}\]  
where \(b\) substitutes for a random number between 0 and 1 and indicates the chromosome's intersection point. In addition, the Equations (16) and (17) are used to model the mutation in this algorithm [9]. \[C_{mn}=\left\{ \begin{matrix}C_{mn}+(C_{mn}-C_{mn})\times f(g),r>0.5\\ C_{mn}+(C_{mn}-C_{mn})\times f(g),r>0.5\end{matrix} \right. \tag{16}\]\[f(g)=r_{2}(1-\sqrt[g]{G_{\max}}) \tag{17}\]  
where \(C_{mn}\) represents the gene \(C_{mn}\), \(C_{\min}\) and \(C_{\max}\) stands for the higher and lower bounds for genes, and \(r\) and \(r_{2}\) are random numbers while \(r\) is between 0 and 1, \(g\) indicates the number of present generation population and \(G_{max}\) is the maximum generations considered to iterate. So, the algorithm finally reaches a generation with perfectly fitting responses. A flowchart of the ANN + GA is shown in Figure 5 and its pseudocode is reported in Appendix A. * ANN + PSO  
The PSO algorithm is a method established on swarm intelligence inspired by the behavior of a bird flock. The behavior of the particles, their speed, and the direction of their movement are influenced by the best experience of the particle itself in matching the goal and the best experience among all members of the population. The objective of this algorithm is to reduce the average fitness of all members of the population. In Equations (18)-(20), the best personal position of the particle is presented as _Pbest_, and the best position of the whole particle swarm is symbolized by _Gbest_. If the particles are located in an N-dimensional space, the vector \(X_{i}\) = (\(X_{i1}\), \(X_{i2}\),..., \(X_{N}\)) and the vector \(V_{i}\) = (\(V_{i1}\), \(V_{i2}\),..., \(V_{iN}\)) represent the position and velocity of the particle \(i\), respectively. In each generation, the values of these vectors must be updated and compared to the previous generation. The Equations (19)-(21)are used to update these values.