## 4 Result and Discussion  
### Assessment of Accuracy and Stability of the Models  
In this section, the accuracy and stability are compared between the conventional ANN model without using the HS algorithm and the HS-ANN hybrid model. Both models were run 50 times, and the statistical analyses were conducted for the model results. Each of the HMCR and PAR of the HS algorithm were chosen to vary from 0.1 to 0.9 at intervals of 0.2, so a total of 25 HS-ANN models were tested. The models were used to estimate the stability number of rock armor for the experimental data of [PERSON] [26] for which the input and output variables are given as in Table 2. The 579 experimental data excluding the data of low-crested structures were used, as done by [PERSON] _et al._[3]. As described in Section 3.1, a hundred data sampled by the stratified sampling method were used to train the ANN, whereas the remaining 479 data were used to test the model. Figure 5: Pseudo-code of HS algorithm (modified from [PERSON] [33]). The correlation coefficient (\(r\)) and index of agreement (\(I_{a}\)) between model output values and target values of the 479 test data are used to evaluate the performance of the models. First, to compare the accuracy of the developed models with those of previous studies (see Table 1), the maximum value of correlation coefficient among 50 runs of each model is presented in Table 4. For the results of the HS-ANN models, the rank is indicated by a superscript, and the largest two values are shaded. The largest correlation coefficient of the HS-ANN model is only slightly larger than that of the ANN model, but both of them are much greater than those of previous studies (see Table 1), probably because the stratified sampling was used in the present study. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & HMCR\(\backslash\)PAR & 0.1 & 0.3 & 0.5 & 0.7 & 0.9 \\ \cline{2-7}  & 0.1 & 0.957 & 0.971 & 0.961 & \(\mathbf{0.973^{1}}\) & 0.964 \\ HS-ANN & 0.3 & 0.959 & 0.967 & 0.970 & \(0.972^{3}\) & 0.960 \\  & 0.5 & 0.961 & 0.954 & 0.961 & 0.957 & 0.968 \\  & 0.7 & 0.968 & \(\mathbf{0.973^{2}}\) & 0.959 & 0.967 & 0.970 \\  & 0.9 & \(0.971^{5}\) & 0.970 & \(0.972^{4}\) & 0.970 & 0.960 \\ \hline ANN & & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Maximum values of correlation coefficient. Figure 6: Flow chart of HS-ANN model. Even though we used the correlation coefficient to compare the accuracy of our model results with those of previous studies, it is not a good indicator for model accuracy because it merely evaluates the linearity between observation and prediction but not the agreement between them. Hereinafter, we use the index of agreement introduced by [PERSON] [34] as a measure of the degree to which a model's predictions are error-free but not a measure of correlation between the observed and predicted variates. The index of agreement is given as  
\[I_{a}=1-\frac{\sum_{i=1}^{N}{(p_{i}-o_{i})^{2}}}{\sum_{i=1}^{N}{\left[{\left|{p _{i}-\overline{o}}\right|+\left|{o_{i}-\overline{o}}\right|}\right]^{2}}}, \tag{15}\]  
where \(p_{i}\) and \(o_{i}\) denote the predicted and observed variates, and \(\overline{o}\) is the mean of the observed variates. The values for \(I_{a}\) vary between 0 and 1.0, where 1.0 indicates perfect agreement between observations and predictions, and 0 connotes complete disagreement. The statistical parameters used to measure the predictive ability and stability of the models are the average, standard deviation, and the minimum value of \(I_{a}\). The larger the average, the better the overall predictive ability of the model. The smaller the standard deviation, the higher the stability of the model, that is, the less variability among the model outputs from different runs of the model. Lastly, the larger the minimum value of \(I_{a}\), the larger the lower threshold of the predictive ability of the model. In summary, a large average and large minimum value of \(I_{a}\) signify that the predictive ability of the model is excellent.