## 3 The Proposed MOA-CV Method
### Demonstration of MOA-CV Method  
First, a 1D mathematical function as shown in Equation (3) is taken as an example to demonstrate the operation processes of the MOA-CV method. \[y=6-\frac{1}{\left(x-0.3\right)^{2}+0.01}-\frac{1}{\left(x-9\right)^{2}+0.04} \tag{3}\]  
Five sample points are generated randomly and a KRG surrogate model is built, as shown in Figure 2. Five corresponding virtual sample points is generated by the MOA-CV method based on the KRG surrogate model. Figure 2a shows that the each validation point is moved around a normal distribution, as the origin line shows. Figure 2b-f depicts the process of obtaining the MOA-CVerror over five iterations. Five errors for five iterations are obtained, and finally, the MOA-CVerror is calculated by averaging the five errors. To compare the model selecting performance of MOA-CV and LOO-CV, four surrogate models are constructed based on five sample points, as shown in Figure 3. The prediction accuracy \(R^{2}\) is listed in Table 3. It is obvious that the four surrogate models have different prediction accuracy. From the results of \(R^{2}\) in Table 3 we can know that the real accuracy order of those four models is KRG, RBF-MQ, RBF-TPS and PRS. The right order of surrogates is in italic bold. Although the magnitudes of MOA-CV and LOO-CV are different, we can still compare their performance according to the accurate rate of model selection. MOA-CVerrors of the four surrogate models are highly consistent with \(R^{2}\) and can accurately select the best and worst surrogate model. However, LOO-CVerrors are extremely distinct from \(R^{2}\). The best surrogate model selected by the LOO-CV method is the PRS model, which happens to be the worst one among the four surrogate models in actual situation. The reason may be that unlike the LOO-CV method which is model-independent, the MOA-CV method is a model-dependent method that can learn information from constructed surrogate models. That is, the MOA-CVerror are largely positively correlated with the accuracy of the built surrogate model itself. \begin{table}
\begin{tabular}{c l} \hline \hline
**No.** & \multicolumn{2}{c}{**Process**} \\ \hline Step 1 & Generate original sampling points \(\left(\mathbf{x},\mathbf{y}\right)\), and based on \(\left(\mathbf{x},\mathbf{y}\right)\), build a \\  & Surrogate model \(\mathbf{S}_{0}\) \\  & Split \(\left(\mathbf{x},\mathbf{y}\right)\) into two sets, i.e., training set \\ Step 2 & \(\left(\mathbf{x_{tr}},\mathbf{y_{tr}}\right)=\left\{\left(x_{1},y_{1}\right), \ldots,\left(x_{j-1},y_{j-1}\right),\left(x_{j+1},y_{j+1}\right),\ldots, \left(x_{n},y_{n}\right)\right\}\) and \\  & validation set \(\left(x_{v},y_{v}\right)=\left(x_{j},y_{j}\right)\) \\ Step 3 & Calculate the minimum distance \(d_{\min}\) among \(\left(\mathbf{x},\mathbf{y}\right)\) \\ Step 4 & Determine the mean and the variance of a random normal distribution \\  & at each validation point \(\left(x_{v},y_{v}\right)=\left(x_{j},y_{j}\right)\), \(\mu=y_{j}\) and \(\sigma=0.02d_{\min}\), \\ Step 5 & Obtain a virtual training point, i.e., \(\left(x_{\leftrightarrow j},\vec{y}_{\leftrightarrow j}\right)\) on the model \(\mathbf{S}_{0}\) \\ Step 6 & Generate a new training set \(\left(\overset{\star}{\mathbf{x_{tr}}},\overset{\star}{\mathbf{y_{tr}}}\right)\) and build a surrogate model \(\mathbf{S}_{1}\) \\  & based on \(\left(\overset{\star}{\mathbf{x_{tr}}},\overset{\star}{\mathbf{y_{tr}}}\right)\), get the prediction \(\vec{y}_{\leftrightarrow j}\left(x_{j}\right)\) of \(\mathbf{S}_{1}\) at \(x_{j}\) \\ Step 7 & Calculate the MOA-CVerror \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Operation Steps of LOO-MCV Algorithm. Figure 3: Comparison of four surrogate models.