## 3 Optimization Algorithms
### Grasshopper Optimization Algorithm (GOA)
An MLP has an input layer (IL), which receives inputs from the patterns, a number of hidden layers (HL), which is responsible for processing, and an output layer (OL) that returns the output. IL and OL include a number of neurons equal to inputs and outputs, respectively. Figure 2 illustrates a schematic MLP. Every node is linked to the nodes in the next layer with linking weights. The IL' nodes just receive the inputs and give them to the next layer. HLS' neurons take the outputs of the past layer's neurons, sum them, and put them in the activation function. The OL's nodes take their inputs from the previous layers' nodes, sum them, and return the answer as the network's outputs. There are various activation functions for ANNs such as sigmoid, tangent sigmoid, linear, and hyperbolic tangent sigmoid to name a few. In fact, the main problem of an ANN is to obtain the best weights for links where there are a number of algorithms for solving this problem. The most used problem-solving algorithm for ANNs is feed-forward back-propagation (FFBP) [82; 83]. In FFBP algorithm, weights are predicted from the IL to OL, but improved from the OL to IL by learning algorithms. Among learning algorithms, Levenberg-Marquardt (LM), due to its better speed and performance, is the best one to predict concrete behavior [40; 84]. In the LM algorithm, the updated weights and bias are calculated as follows:

\[wb_{n+1}=wb_{l}-[J]+\mu\,I]^{-1}\,J^{-1}\,e \tag{12}\]

where \(wb_{n+1}\) and \(J\) are the updated weights and biases and the Jacobian matrix that includes the first derivatives of the network errors concerning the weights and biases. \(\mu\), \(I\), and \(e\) are a positive real number damping factor, identity matrix, and vector of ANNs error, respectively. A schematic neuron is shown is Figure 3.

Figure 2: A schematic MLP. The performance of an ANN is affected by its structure, and trial-and-error is the most commonly used method for obtaining the best structure. Using this method can cost a huge amount of time and energy, although there is a possibility of not gaining the best architecture. Hence, having a systematic method to reach the ANN's best architecture is necessary. In the present study, an ANN is modified using GA, SSA, and GOA, separately. The gene of an individual and the position of a salp and grasshopper are divided into two parts. The first part is allocated to the number of HL. According to the Figure 4, the \(i\)th value of the first part can be zero or one. If it is zero, the \(i\)th HL is deactivated, and it is activated if the \(i\)th value of the first part is one. The second part, on the other hand, is related to the number of neurons in each HL. For instance, if the \(j\)th value of the second part is 11, it means there is 11 neurons in the \(j\)th HL. The process of each model is almost the same as its optimization algorithm, except that their fitness are calculated by ANN's error, and ANN uses RMSE to compute the error. \[RMSE=\sqrt{\frac{1}{Q}{\sum_{j=1}^{Q}(M_{q}-V_{q})^{2}}} \tag{13}\]

where \(Q\) is number of records, \(M_{q}\) is the Measured results and \(V_{q}\) is predicted values related to the \(q\)th record. ### M5P Tree

The M5P tree is the improved version of M5 tree that consist of three phases: dividing the input space, building the tree and deriving the knowledge [85]. The M5P tree, on the other hand, includes four Phases. The first phase includes building a full tree by splitting the data using the selected parameters and a division criterion. By using standard deviation reduction parameter (SDRP), the search space is split. SDRP is represented as follows:

\[SDRP=sd(D)-\sum_{j}\frac{|d_{r}|}{|D|}\times sd(d_{r}) \tag{14}\]

where \(||\) is the set's number elements, \(d_{r}\) is \(r\)th sub-space gained by splitting the node according to selected divided parameter, \(sd\) is the standard deviation, and \(D\) is the dataset

Figure 4: Position of a salp and a grasshopper and chromosome of an individual. Figure 3: A schematic neuron. which reaches the node.