# Innovative Data-Driven Machine Learning Approaches for Predicting Sandstone True Triaxial Strength
## 5 Model Building and Training  
The settings of hyperparameters in the MLP model significantly influence its performance. Reasonable hyperparameter settings enable the MLP model to adapt to the training data more accurately during the training process. HHO, as a multi-strategy optimization algorithm, can adjust its optimization strategies based on the remaining number of iterations, effectively preventing the model from becoming prematurely trapped in local optima. HHO has already been proven in numerous studies to be a highly effective method for hyperparameter optimization [48; 49; 50]. In this study, the HHO was chosen as the optimization method for finding suitable hyperparameter settings for the MLP model. To effectively establish and evaluate the HHO-MLP model, the sandstone database was randomly divided into training and testing datasets in a 7:3 ratio. Considering that in numerical analysis, models need to be converted into explicit expressions for further operations, the complexity of the explicit expressions of model outputs should be minimized to improve the convenience of further applications. Therefore, the number of hidden layers in the model was set to one, and the activation function of the output layer was set to linear. Additionally, to ensure the differentiability of the expressions, continuous and differentiable functions were selected as activation functions for the hidden layer. Through analysis of existing activation functions, four activation functions were selected for model establishment. Table 2 shows the specific forms of the activation functions. During the model training process, the parameters for the optimizer HHO were 20 for the number of Harris hawks and 50 for the number of iterations. The hyperparameters optimized include the batch size, the number of nodes per layer, and the learning rate of the MLP model. These parameters were set within specific ranges during the optimization process (batch size [1; 50], number of nodes per layer [1; 100], learning rate [1 \(\times\) 10\({}^{-6}\), \(1\times\) 10\({}^{-3}\)]). Additionally, to enhance the model's generalization performance, 20% of the training set was allocated as a validation set to evaluate the model's performance during training. The Mean Squared Error (MSE) was selected as the fitness function to assess the model's fitness, which can be calculated by Equation (41). \[MSE=\frac{\sum_{i=1}^{n}(f_{i}-y_{i})^{2}}{n} \tag{41}\]  
where \(f_{i}\) denotes the predicted value and \(y_{i}\), denotes the actual value. The iteration process for the models employing four different activation functions is depicted in Figure 6. From the figure, it is evident that HHO significantly optimizes all four MLP models. Most models exhibit convergence of MSE within 30 iterations, with complete convergence observed after 40 iterations. In terms of training results, the model using tanh as the activation function performed the best among the four, followed by sigmoid, softplus, and swish. Following the conventional pattern for selecting ML models, it is apparent that the model with tanh activation function would be chosen for further research. However, as mentioned earlier, the shape of the envelope corresponding to the rock strength model is also a crucial indicator of model performance. Therefore, further comparison of the four models obtained was necessary. \begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Equation** \\ \hline sigmoid & \(sigmoid(x)=\frac{1}{\Gamma+\epsilon^{x}}\) \\ softplus & \(softplus(x)=\log(1+\epsilon^{x})\) \\ swish & \(swish(x)=x\times sigmoid(\theta^{1}\times x)\) \\ tanh & \(tanh(x)=\frac{\epsilon^{x}-\epsilon^{x}}{\epsilon^{x}+\epsilon^{x}}\) \\ \hline \hline \end{tabular} \({}^{1}\)\(\beta\): trainable variable.  
\end{table} 
Table 2: Expressions for four activation functions. Figure 7 illustrates the compressive and tensile meridians obtained by the four models when predicting the behavior of sandstone No. 10.