# Application and Evaluation of Surrogate Models for Radiation Source Search
## 3 Surrogate Models
### Multivariate Adaptive Regression Splines  
Multivariate Adaptive Regression Splines (MARS) were first proposed by [24] as a procedure for performing adaptive nonlinear regression using piecewise linear (spline) basis functions. MARS follows from recursive partitioning regression, which is also outlined in Ref. [24]. The MARS linear \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline
**N** & **2** & **4** & **8** & **16** & **32** & **64** & **128** & **256** & **512** & **1024** & **2024** \\ \hline \(rRMSE(w^{N})\) & 1.598 & 1.382 & 1.113 & 0.825 & 0.633 & 0.468 & 0.387 & 0.331 & 0.281 & 0.236 & 0.213 \\ \(rRMSE(w^{N})\) & 0.865 & 0.805 & 0.741 & 0.767 & 0.739 & 0.827 & 0.855 & 0.849 & 0.840 & 0.903 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Convergence of Legendre surrogate model \(rRMSE\) as \(N\) increases. Figure 3: **(a) Sum of squares error and (b) likelihood for Legendre-based surrogate models computed via Equation (21) with maximum polynomial degree \(P=1,...,30\). The dashed lines correspond to each of the \(d=10\) surrogate models and the solid line is their mean. **  
basis functions, often termed hinge functions, are introduced in pairs on either side of a \"knot\" \(t\), where there is an inflection point in a particular parametric direction; e.g.,  
\[\psi_{k}(q)=\max(0,q_{i}-t)\text{ and }\psi_{k+1}(q)=\max(0,t-q_{i}) \tag{27}\]  
are introduced concurrently. Here, \(q_{i},\ i=1,\ldots,p\) denotes the \(i\)th component of \(q\), where \(p=3\) in our model, since \(q=(x,y,I)\). In this way, the domain is divided so that \(\psi_{k}(q)\) is zero for \(q_{i}<t\) and \(\psi_{k+1}(q)\) is zero for \(q_{i}>t\). We employ an adaptive regression algorithm to generate knot locations using data \(\{q^{m},y^{m}\}_{m=1}^{M}\) from the statistical model defined in Equation (12) for the 10 detectors. We take the knot locations to be \(t\in\{q_{i}^{m}\}_{i=1,m=1}^{p,M}\). An additive MARS model would take the basis functions to be \(\Psi_{k}(q)=\psi_{k}(q)\), but we consider products of the linear basis functions, thus we take the basis functions to be  
\[\Psi_{j}(q)=\prod_{k=1}^{K_{j}}\psi_{k}(q)=\prod_{k=1}^{K_{j}}[s_{jk}(q_{v(j) }-t_{jk})]_{+}. \tag{28}\]  
Here, \(v(j,k)\in\{1,...,p\}\) labels the component of the parameter vector or \"predictor variable\" \(q=\{q_{i}\}_{i=1}^{p}\), \(K_{j}\leq p\) is the level of interaction between the linear basis functions, and \(t_{jk}\) is the \(k\)th knot employed by the \(j\)th basis function. Furthermore, the positive subscript means take the maximum of zero and the argument, as in Equation (27), and we take \(\Psi_{0}(q)=1\) and \(s_{jk}=\pm 1\). For simplicity, we consider only piecewise linear basis functions \(\psi(q)\), although this algorithm can also be performed with other piecewise basis functions, such as cubic. Additionally, we consider up to second-order interactions of the linear basis functions \(\psi\); i.e., \(K_{j}\leq 2\), and first-order interactions of the linear basis functions that are a function of the same variable. We found that employing second-degree interactions--i.e., products of linear basis functions of the same variable--did not increase the performance of the surrogate models significantly. The surrogate model is  
\[u^{N}(q,w)=\sum_{j=0}^{N}w_{j}\Psi_{j}(q), \tag{29}\]  
where \(\Psi_{0}(q)=1\) and the coefficients \(w_{j}\) are estimated using least-squares regression. Note that this is the same form as Equation (11), for \(w_{0}=1\) and \(\Psi_{0}(q)=P(q)\). The construction of the MARS model takes place in two steps. In the forward step, basis functions are added to the model to reduce a \"lack of fit\" value, which we take to be the least-squares error of the surrogate model to the physical model evaluated at the training points. This is performed until a user defined maximum number of terms is reached. The maximum number of terms we use in the forward process of the MARS algorithm to construct this surrogate model is \(N=200\).