## 4 Results and Discussions
### Design of Experiments  
The first process of constructing models is generating sample points, also called design of experiments (DoEs) which is the computational strategy to produce sample points for computer simulations and surrogate modeling. In this section, Latin hypercube sampling (LHS) with better space-filling and projective properties are used to generate sample points. In this work, we use the built-in Matlab function lhsdesign. Three samples sizes, i.e., \(5n\), \(8n\) and \(10n\) (where \(n\) is the dimension of a function), respectively, are chosen to investigate the effect of different sample sizes on the performance of MOA-CV and LOO-CV. For each function and each sample size, ten random sets of DoEs are generated. The abovementioned surrogate models are used to build models; MOA-CVerror and LOO-CVerror are used to rank models based on the calibrated criterion \(R^{2}\). Table 6 lists the percentages of choosing the best surrogate models for the four functions, with a higher value in italic bold. Almost for any function, the accurate rates of best model selection of MOA-CV and LOO-CV rise with the increase in samples sizes from \(5n\) to \(10n\). MOA-CV performs much better than LOO-CV in selecting best models, especially when the sample size is larger. In terms of HD functions, when the sample size is \(5n\), MOA-CV and LOO-CV perform quite poor. That is, with small samples neither the MOA-CV nor the LOO-CV method can find the best model. When the sample size increases to \(10n\), MOA-CV is more likely than LOO-CV to find the best model. The average result over four functions also shows that it is a little easier for the MOA-CV method to discover the best model than the LOO-CV method. Table 7 lists the accurate rate of finding the worst surrogate models for the four functions, also with a higher value in italic bold. MOA-CV has the best accurate rate when the sample size is \(8n\). When the sample sizes are \(5n\) and \(10n\), LOO-CV performs better than MOA-CV in terms of removing worst models. Similar to selecting the best models, MOA-CV and LOO-CV also perform quite poor with \(5n\) sample points in cases of HD functions. From the average result over four functions, it is seen that the MOA-CV method performs slightly worse than the LOO-CV method in finding worst models. As described above, for each function and each sample size, 10 sets of DoEs are generated randomly. Here, we compare the averaged results of selecting best and worst models over these 10 sampling plans. Figures 4-6 show the model ranks by using \(R^{2}\), LOO-CVerror and MOA-CVerror under \(5n\), \(8n\) and \(10n\) sample points. Tables 8 and 9 summarize the average accurate rates of selecting best and worst \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multicolumn{1}{c}{**Size of Samples**} & \multicolumn{2}{c}{\(5n\)} & \multicolumn{2}{c}{\(8n\)} & \multicolumn{2}{c}{\(10n\)} \\ \hline
**Functions** & **MOA-CV** & **LOO-CV** & **MOA-CV** & **LOO-CV** & **MOA-CV** & **LOO-CV** \\ \hline LD & 1 & **40\%** & 10\% & **40\%** & 10\% & 20\% & 20\% \\ functions & 2 & 10\% & **20\%** & **50\%** & 10\% & _60\%_ & 30\% \\ HD & 3 & 0\% & 0\% & **50\%** & 0\% & _80\%_ & 0\% \\ functions & 4 & 0\% & 0\% & **30\%** & 0\% & _90\%_ & 30\% \\ Average for all functions & **12.5\%** & 7.5\% & **42.5\%** & 5\% & **62.5\%** & 20\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Accurate rate of selecting best models.