# Artificial Neural Network-Based Model for Prediction of Frost Heave Behavior of Silty Soil Specimen
## 4 Prediction of Frost Heave Ratio Using the Artificial Neural Network Model
### Application of an ANN to Frost Heave Ratio Predictions  
In this study, an ANN model was developed to predict the frost heave ratio \(\zeta\) for the frozen soil. In the ANN model, two parameters-hydraulic conductivity in the unfrozen zone (\(k_{0}\)) and the thermal conductivity of the soil particle (\(\lambda_{s}\))-were considered as input parameters. The training data included input-target pairs: 197 pairs for training and 49 pairs for testing. Bayesian regularization was applied to a back-propagation neural network. The architecture of an ANN is usually determined via trial and error. Generally, the input-target pairs scale in the range of [\(-\)1, 1] before training. Thus, the minimum and maximum values of the original input-target pairs are scaled to \"\(-\)1\" and \"1\", respectively. After the training procedure, the weights matrix and bias vectors are applied to any future inputs, which should be scaled based on the minimum-maximum pairs of the original inputs and targets. Once the network is trained, the predicted value falls within the range [\(-\)1, 1]. The predicted value should be converted back into the same units by vector contains the minimum and maximum of the original input-targets pair. In this study, the tangent sigmoid transfer function \(f(x)=2/\left(1+e^{-2x}\right)^{-1}\) is adopted for all layers except for the input layer where the linear transfer function \(f(x)=x\) is used instead. Additionally, the learning rate (in Equation (28)) plays a vital role in the ANN network. If the learning rate is too low, the weight matrix updates at an inadequate rate and the local minimum may take a long time to achieve. In contrast, an overly large learning rate may result in the network overreaching and missing the local minimum optima. Traditionally, many studies adopted learning rate of 0.1 or 0.01. In our study, we investigated the effect of both learning rates on the ANN model. Figure 8 shows the relationship between the coefficient of determination R\({}^{2}\) and the number of neurons in the hidden layer for Bayesian Regularization and Levenberg Marquardt according to both learning rates (\(\eta\) = 0.01 and \(\eta\) = 0.1). The R\({}^{2}\) converged to a high value (R\({}^{2}\)\(\geq\) 0.95) when the ANN model had more than three neurons and six neurons in the hidden layer for Bayesian Regularization and Levenberg Marquardt, respectively. Although the Levenberg Marquardt algorithm achieved a higher converged coefficient compared to those of Bayesian Regularization at eight and ten neurons in the hidden layer but it requires a high computational cost for computing Jacobian matrix. Thus, the Bayesian Regularization was adopted in this study. With a learning rate of 0.01, the R\({}^{2}\) value of the ANN model based on the Bayesian Regularization peaked highest at five neurons in the hidden layer. Therefore, the learning rate and the number of neurons in the hidden layer were set as 0.01 and 5, respectively. Figure 9 shows the relationship between the converged coefficient R\({}^{2}\) and the number of neurons in the hidden layer of the ANN model based on the Bayesian Regularization according to the performance functions that consist of Mean Absolute Error (MAE), Mean Square Error (MSE), Sum Absolute Error (SSA), and  
Sum Square Error (SSE). The convergence coefficient peaked at five neurons in the hidden layer when the performance function was mean square error (MSE). Other performance functions were similar trending but they had a lower value of converged coefficient R\({}^{2}\). In this study, the Mean Square Error (MSE) was adopted to evaluate the performance of the ANN model. Figure 10 illustrates a comparison of the frost heave ratio predicted by the ANN model and simulation model. The model exhibited R\({}^{2}\) value of 0.9538 for training data and 0.8929 for the testing data.