# Application and Evaluation of Surrogate Models for Radiation Source Search
## 3 Surrogate Models
### Gaussian Process Regression  
In Gaussian process- or kriging-based surrogate models, one treats the high-fidelity simulations as realizations of a Gaussian process  
\[u^{N}(q)\sim\text{GP}\left(m(q),c(q,q^{\prime})\right). \tag{43}\]  
The mean and covariance functions \(m(q)\) and \(c(q,q^{\prime})\) are constructed to reflect the trends and correlation structure of the physical model \(u\). \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Basis Function** & **Mathematical Form** & **Comp. Time (s)** & **rMSE** \\ \hline Multiquadric & \(\Psi(h_{j}(q))=\sqrt{h_{j}^{2}(q)+\sigma^{2}}\) & 131 & 0.2094 \\ Inverse Multiquadric & \(\Psi(h_{j}(q))=1/\sqrt{h_{j}^{2}(q)+\sigma^{2}}\) & 132 & 0.2075 \\ Thin-plate Spline & \(\Psi(h_{j}(q))=h_{j}^{2}(q)\log(h_{j}(q)/\sigma)\) & 159 & 0.2089 \\ Gaussian & \(\Psi(h_{j}(q))=e^{-h_{j}^{2}(q)/2\sigma^{2}}\) & 135 & 0.2093 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of radial basis functions for surrogate model construction. We employ a constant mean \(m(q)=P(q)=\beta_{0}\mathbf{1}\), where \(\beta_{0}\) is a hyperparameter that we infer and \(\mathbf{1}\) is an \(M\times 1\) vector of ones, since we employ observations for \(M\) parameter values \(q^{m}\) to compute the surrogate model. Whereas universal kriging employs a polynomial trend function \(m(q)\), we limit our analysis to ordinary kriging by employing a constant mean \(m(q)\). Additionally, we employ covariance functions of the form  
\[c(q,q^{\prime})=\sigma^{2}r(q,q^{\prime}), \tag{44}\]  
where \(\sigma^{2}\) denotes the model variance and \(r(q,q^{\prime})\) is a parametric correlation function or kernel. By the definition of parametric given in Section 3.1, Gaussian processes are semi-parametric. This is because the class of basis functions employed is not fully determined, but follow from the predefined parametric correlation function. A fully nonparametric Gaussian processes model is discussed in Ref. [31]. A common choice for \(r(q,q^{\prime})\) is the squared exponential function  
\[r(q,q^{\prime})=e^{-\frac{1}{2\ell^{2}}\sum_{i=1}^{p}\left[q_{i}-q_{i}^{ \prime}\right]^{2}}, \tag{45}\]  
which can be expressed as  
\[r(h_{j}(q))=e^{-h_{j}(q)^{2}/2\ell^{2}} \tag{46}\]  
with \(h_{j}(q)\) defined in Equation (39) if we consider \(q^{\prime}=q^{j}\). Comparison with Equation (40) illustrates that this is comparable to employing radial basis functions as a correlation function. We note that this kernel is isotropic in the sense that the length scale \(\ell\) is the same for each of the \(p=3\) scaled components of the parameter \(q=(x,y,I)\). For inputs \(q^{1},\ldots,q^{M}\), the associated covariance and correlation matrices have entries \(C_{ij}=c(q^{i},q^{j})\) and \(R_{ij}=r(q^{i},q^{j})\). For the statistical model defined in Equation (12), it follows that  
\[u\sim\mathcal{N}(\beta_{0}\mathbf{1},C+\sigma_{0}^{2}I), \tag{47}\]  
where \(u=[u(q^{1}),\ldots,u(q^{M})]\). Note that \(\sigma_{0}^{2}\) serves as a nugget, which results in an emulator that does not interpolate the data and attaches a non-zero uncertainty bound around the data. Whereas use of the correlation function from Equation (45) facilitates comparison with radial basis surrogate models, it is overly smooth for the urban source applications with discontinuous derivatives. This motivates consideration of the less smooth Matern correlation functions [32; 33]. As summarized in Table 3, we consider three isotropic correlation functions  
\[r_{1/2}(h) =e^{-h/\ell}, \tag{48}\] \[r_{3/2}(h) =\left(1+\frac{\sqrt{3}h}{\ell}\right)e^{-\sqrt{3}h/\ell}, \tag{49}\]  
and Equation (45), where \(h=||q^{i}-q^{j}||\) denotes the Euclidean distance between samples \(q^{i}\) and \(q^{j}\) and the subscripts denote half integer choices with explicit representations. We also consider three anisotropic kernel functions where the characteristic length scale \(\ell_{i}\) is allowed to vary for each component of \(q=(x,y,I)\). As detailed in Ref.