## 3 The Proposed MOA-CV Method  
### The MOA-CV Method  
Regarding the problem above, the move-one-away cross validation (MOA-CV) method is proposed. The reason why the LOO-CV method cannot always select the best and the worst models may be that each validation point, which is eliminated calculating the LOO-CVerror, is different in importance to different surrogate techniques. That is, in some cases, a surrogate model constructed from the entire sample point has better accuracy than another one, but the results are reversed when uses the LOO-CVerror to assess the accuracy of models. Therefore, to preserve the information of validation points as much as possible, rather than eliminating the validation point as the LOO-CV method, MOA-CVerror of the proposed MOA-CV method are calculated by moving validation points around random normal distributions of samples. Pick a set of sample points \((\mathbf{x},\mathbf{y})=\Big{\{}(x_{1},y_{1}),\dots,\Big{(}x_{j},y_{j}\Big{)}, \dots,\Big{(}x_{n},y_{n}\Big{)}\Big{\}}\), and use the sample points to construct a surrogate model \(\mathbf{S}_{0}\). Select the \(j\)th sample \((x_{v},y_{v})=\Big{(}x_{j},y_{j}\Big{)}\) as the validation points, and the remaining \(n-1\) points as training points, i.e., \((\mathbf{x_{tr}},\mathbf{y_{tr}})=\Big{\{}(x_{1},y_{1}),\dots,\Big{(}x_{j-1}, y_{j-1}\Big{)},\Big{(}x_{j+1},y_{j+1}\Big{)},\dots,\Big{(}x_{n},y_{n}\Big{)}\Big{\}}\). Move the validation point around a random normal distribution with parameter \(\mu=x_{j}\) and \(\sigma=\lambda d_{\min}\), where \(d_{\min}\) is the minimum distance among sampling points \((\mathbf{x},\mathbf{y})\) and \(\lambda\) is set by hand, i.e., \(\lambda=0.02\). Then generate a virtual training point \(\Big{(}x_{\leftrightarrow j},\hat{y}_{\leftrightarrow j}\Big{)}\), where \(\hat{y}_{\leftrightarrow j}\) means the prediction of model \(\mathbf{S}_{0}\) at \(x_{\leftrightarrow j}\). Then the training points can be updated as \(\Big{(}\overset{\leftrightarrow}{\mathbf{x_{tr}}},\overset{\leftrightarrow} {\mathbf{y_{tr}}}\Big{)}=\Big{\{}(x_{1},y_{1}),\dots,\Big{(}x_{j-1},y_{j-1} \Big{)},\Big{(}x_{\leftrightarrow j},\hat{y}_{\leftrightarrow j}\Big{)}, \Big{(}x_{j+1},y_{j+1}\Big{)},\dots,\Big{(}x_{n},y_{n}\Big{)}\Big{\}}\), and use the virtual training set \(\Big{(}\overset{\leftrightarrow}{\mathbf{x_{tr}}},\overset{\leftrightarrow} {\mathbf{y_{tr}}}\Big{)}\) to build a surrogate model \(\mathbf{S}_{1}\). The validation point is unchanged. The main form of the MOA-CVerror is as follows:  
\[MOA-CVerror=\frac{1}{n}\sum_{j=1}^{n}\Big{(}y_{j}\Big{(}x_{j}\Big{)}-\hat{y}_ {\leftrightarrow j}\Big{(}x_{j}\Big{)}\Big{)}^{2} \tag{2}\]  
where MOA-CVerror means the error generated by the MOA-CV method, \(\leftrightarrow j\) means moving the validation point \((x_{v},y_{v})=\Big{(}x_{j},y_{j}\Big{)}\) to be a virtual training point \(\Big{(}x_{\leftrightarrow j},\hat{y}_{\leftrightarrow j}\Big{)}\), and \(\hat{y}_{\leftrightarrow j}\Big{(}x_{j}\Big{)}\) denotes the  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Term** & **PRS** & **RBF-MQ** & **RBF-TPS** & **KRG** \\ \hline \(R^{2}\) & 0.32 & 0.83 & 0.85 & 0.80 \\ Rank of \(R^{2}\) & 4 & 2 & 1 & 3 \\ _CVerror_ of LOO-CV & 12.3 & 13.1 & 14.0 & 10.9 \\ Rank of CV & 2 & 3 & 4 & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Rank of models by \(R^{2}\) and leave-one-out CV (LOO-CV).  
Figure 1: Comparison of different surrogate models.  
prediction of \(\mathbf{S}_{1}\) at \(x_{j}\). Similar to the LOO-CV method, the model with the smallest MOA-CVerror is the most accurate, and vice versa.  
The operation steps of the MOA-CV algorithm are briefly described in Table 2.