# An Effective Surrogate Ensemble Modeling Method for Satellite Coverage Traffic Volume Prediction
## 4 Experimental Results
### Effect of the Training Sample Size  
To investigate the effect of the training sample size on the proposed PWTA method and the candidate surrogates, 30, 40, 50, 60, 70, and 80 training points are sampled in each sub-domain respectively using full-factorial design. PRS, OK, RBF, and PWTA are constructed for each sub-domain with those training sets, respectively. From the experiments we observed that when training points are less than 30, PWTA is less accurate for most sub-domains. With more than 80 training points, the accuracy of PWTA becomes stable and cannot be improved obviously. Thus, the range between 30 to 80 training points is selected to better show the growth trends of model accuracy. Figure 4 presents R\({}^{2}\) of different surrogates vary with the training sample size in the eight sub-domains. The R\({}^{2}\) values of the different models in the global domain are shown in Table 4. The best value in each column is shown in bold for ease of comparison. From Figure 4, it can be easily seen that PRS performs poorly for the eight sub-domains. When the training sample size is small, R\({}^{2}\) of RBF are greatly lower than those of OK. With the enlarged training sample size, OK performs poorer than RBF for North America, Western Europe, East Asia, and Caribbean Sea regions, while for Russia and South America regions, the accuracy of OK are better. It suggests that the performances of different surrogates vary greatly in different sub-domains with different training sample sizes. However, for PWTA, it can be observed that for all the sub-domains and for all the different training sample number conditions, PWTA can perform robustly with top one or two accuracy level. In Table 4, the results show that with small training sample sizes, PWTA performs secondary but comparably well to OK (with less than 0.177% relative difference) which has the highest accuracy in terms of R\({}^{2}\) in global domain. When the number of training points is greater than 40, PWTA universally outperforms the other surrogates. This indicates that for different sub-domains as well as different training sample sizes, PWTA is a more robust choice. Furthermore, it also can be observed that the overall performance of all the surrogates become better and better as the training sample size increases. However, after 60 training points the trends of growth become smaller. Thus, the numbers of the training sample points are set to 60 in the eight sub-domains for the subsequent experiments. \begin{table}
\begin{tabular}{l c c c c c c c c c c} \hline \hline
**Performance** & **Surrogate** & **North** & **Western** & **Caribbean** & **South** & **Russia** & **Middle** & **East** & **Asia** & **Oce-ania** & **Global Domain** \\
**Metric** & **Model** & **America** & **Europe** & **Sea** & **America** & **East** & **Asia** & **Oce-ania** & **Domain** \\ \hline \multirow{2}{*}{**R\({}^{2}\) of the models using FFD**} & **PRS** & 0.6425 & 0.7029 & 0.5712 & 0.6067 & 0.7023 & 0.4787 & 0.7968 & 0.6469 & 0.6760 \\  & **OK** & 0.9622 & 0.9918 & 0.9877 & **0.9873** & 0.9918 & 0.9507 & 0.9869 & **0.9816** & 0.9837 \\  & **RBF** & **0.9671** & 0.9931 & 0.9884 & 0.9836 & 0.9872 & 0.9532 & **0.9890** & 0.9795 & 0.9830 \\  & **PWTA** & 0.9667 & **0.9937** & **0.9892** & 0.9866 & **0.9920** & **0.9541** & 0.9885 & 0.9814 & **0.9846** \\ \hline \multirow{2}{*}{**Mean R\({}^{2}\) of the models using LHS**} & **PRS** & 0.6266 & 0.7394 & 0.5591 & 0.6426 & 0.7307 & 0.4534 & 0.8254 & 0.7141 & 0.7021 \\  & **OK** & 0.9111 & 0.9635 & 0.9170 & 0.9342 & 0.9439 & 0.8522 & 0.9500 & 0.9259 & 0.9332 \\ \cline{1-1}  & **RBF** & 0.9225 & 0.9582 & 0.9376 & 0.9521 & 0.9507 & **0.9214** & **0.9825** & **0.9594** & 0.9564 \\ \cline{1-1}  & **PWTA** & **0.9324** & **0.9735** & **0.9389** & **0.9545** & **0.9675** & 0.9107 & 0.9798 & 0.9579 & **0.9593** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparisons of R\({}^{2}\) for different surrogates using LHS and FFD, respectively.