# Application and Evaluation of Surrogate Models for Radiation Source Search
## 5 Surrogate Performance Results  
For each of the surrogate models discussed in Section 3, we plot the goodness of fit for the first 50 test points in Figure 6 for detector 1. Note that we construct these surrogate models by employing Gauss-Legendre training points. Figure 6a,b depicts the natural log response of the physical model plotted versus the Legendre polynomial-based surrogate models solved for via discrete projection in Equation (21) and LASSO from Equation (26). Additionally, we plot in Figure 6c-f the natural log of the physical model response versus the response of the surrogate models based on multivariate adaptive regression splines, radial basis functions, Gaussian processes, and neural networks, respectively. We note that each of these surrogate models provides an accurate approximation to the physical model. For each of the 10 detector models and for each of the considered surrogate models, we plot the relative root mean squared error (\(rRMSE_{i}\)) for test points \(\{q^{s}\}_{s=1}^{s}\), \(s=1,...,S\) in Figure 7. We also compile the total \(rRMSE\), defined in Equation (10), for each surrogate model in Table 6 along with the computation and evaluation times of the surrogate models. Note that, in Figure 6a,b, both methods for computing the Legendre surrogate models produce good approximations to the physical model. However, the results in Table 6 show that the surrogate models computed by employing discrete projection provide better accuracy and efficiency than the surrogate models employing sparsity controlled regression. Whereas the Legendre polynomial-based surrogate models require the least time to compute, they require substantially more time to evaluate due to the large number of basis functions required to accurately approximate the non-smooth behavior of the physical model. The MARS surrogate models do not perform as well as other surrogate models in terms of accuracy, but outperform many of them in terms of computational speed. This is especially apparent in some of the large deviations of the surrogate model for detector 1 from the test data in Figure 6c as well as by the scale of the \(rRMSE_{i}\) in Figure 7. We explain this decrease in accuracy by the fact that MARS is meant to be used on high-dimensional problems. We would expect these surrogate  
Figure 5: Quadrature points for levels \(l=0,...,4\) for (**a**) Gauss–Legendre and (**b**) Clenshaw–Curtis algorithms. \begin{table}
\begin{tabular}{l c c c c c c} \hline \hline
**M** & **Legendre** & **Legendre Reg.** & **MARS** & **RBF** & **GP** & **NN** \\ \hline \(9^{3}\) & 26.950 & 0.4244 & 0.4803 & 0.3828 & 0.3804 & 0.4306 \\ \(11^{3}\) & 2.0417 & 0.4225 & 0.4341 & 0.3753 & 0.3661 & 0.4713 \\ \(13^{3}\) & 0.5061 & 0.3574 & 0.4190 & 0.3220 & 0.3109 & 0.5311 \\ \(15^{3}\) & 0.3351 & 0.3390 & 0.4307 & 0.2825 & 0.2738 & 0.3563 \\ \(17^{3}\) & 0.2947 & 0.3344 & 0.4025 & 0.2375 & 0.2422 & 0.4164 \\ \(19^{3}\) & 0.2427 & 0.3135 & 0.3908 & 0.2397 & 0.2173 & 0.3636 \\ \(21^{3}\) & 0.2128 & 0.3048 & 0.4175 & 0.2075 & 0.1975 & 0.2624 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of surrogate model accuracy bounds in \(rRMSE\) employing increasingly more Gauss–Legendre training points where Legendre refers to the model constructed via Equation (21) and Legendre Reg. refers to the model constructed via Equation (26). models to be competitive in terms of accuracy with the other surrogate models when approximating a higher dimensional problem. The ability of MARS to accommodate high-dimensional parameter spaces makes it an ideal candidate for problems with a large number of parameters. The radial basis functions presented in Section 3.3 are isotropic, but the surrogate models based on radial basis functions have low \(rRMSE\) in comparison with the other surrogate models, as depicted in Figure 7 and compiled in Table 6. This isotropy is a concern for our non-smooth model when less training points are available, since the model has different responses in each parametric direction, mainly between the \((x,y)\) and \(I\) parameters.