# Application and Evaluation of Surrogate Models for Radiation Source Search
## 3 Surrogate Models
### Radial Basis Functions  
Here, we consider expansions  
\[u^{N}(q,w)=\sum_{j=1}^{N}w_{j}\Psi_{j}(q) \tag{37}\]  
with radial basis functions  
\[\Psi_{j}(q)=\Psi(||q-q^{j}||_{2})=\Psi(h_{j}(q)) \tag{38}\]  
defined in terms of the Euclidean distance  
\[h_{j}(q)=||q-q^{j}||_{2}=\left[\sum_{i=1}^{p}(q_{i}-q_{i}^{j})^{2}\right]^{1/2}. \tag{39}\]  
We remind the reader that \(p=3\) for our problem. A common choice of \(\Psi\) is  
\[\Psi(h_{j}(q))=e^{-h_{j}^{2}(q)/2\sigma^{2}},\ \ j=1,\ldots,N. \tag{40}\]  
Here, the hyperparameter \(\sigma\) is a scale factor, which is typically inferred when constructing the surrogate model. Details regarding the manner in which \(\sigma\) affects conditioning and stability are provided in Ref. [29]. Gaussian radial basis functions defined in Equation (40) have the advantage of physical interpretation and super-spectral convergence; i.e., errors decrease as \(O(e^{-CN})\). Additionally, multiquadric, inverse multiquadric, and thin plate splines are described in Table 2.  
Figure 4: Simple additive MARS example with two knot locations represented by the dashed vertical lines.  
To compute the coefficients \(w=[w_{1},\ldots,w_{N}]^{T}\), we formulate Equation (37), with observations given by Equation (39) as the matrix system  
\[u=Aw+\varepsilon, \tag{41}\]  
where \(u=[u(q^{1}),\ldots,u(q^{M})]^{T}\), \(A_{jm}=\Psi_{j}(q^{m})\), and \(\varepsilon=[\varepsilon^{1},\ldots,\varepsilon^{M}]^{T}\) are the errors from Equation (12). For \(M>N\), the least squares estimate is given by  
\[w=A^{\dagger}u, \tag{42}\]  
where \(A^{\dagger}=\left[A^{T}A\right]^{1/2}A^{T}\) is the pseudo-inverse of \(A\). To compute \(w\), we use the MATLAB backslash command which employs a QR factorization. Other solution techniques are discussed in Ref. [29; 30]. For the purpose of comparing with Legendre surrogate models, we employ the same number of basis functions \(N=2024\) by randomly selecting center points \(q^{j}\) for the basis functions \(\Psi_{j}(q)\) from the training data, such that \(\{q^{j}\}_{j=1}^{N}\in\{q^{i}\}_{j=1}^{M}\).  
In Table 2, we observe that the Gaussian radial basis functions surrogate model does not perform as well as the inverse multiquadric and thin-plate spline radial basis function surrogate models. This decrease in performance is likely due to the smoothness of the Gaussian radial basis functions as compared with the other basis functions. The inverse multiquadric function provides the best accuracy and low computational cost in comparison with the other basis functions. Therefore, we employ inverse multiquadric radial basis functions to develop surrogate models that we compare with the other surrogate modeling methods in Sections 4 and 5. We note that radial basis functions are often employed in the development of neural networks as the activation functions of the neural network nodes, which is discussed further in Section 3.5.  
Whereas the shape parameter \(\sigma\) can be treated as a hyperparameter to be optimized for each problem, we set \(\sigma=40\). We chose this by testing multiple values of \(\sigma\) and choosing the value that provided the smallest \(rRMSE\), defined in Equation (10). Because we randomly sample the basis function center points, optimizing the value of \(\sigma\) is difficult, whereas, if we used all or a constant set of the training points as center points for the basis functions, we could optimize this hyperparameter. The value of \(\sigma\) can affect the conditioning of the problem and, if too large, cause the Runge phenomenon to introduce large errors, as discussed in Ref. [29]. However, decreasing the value of \(\sigma\) improves conditioning and decreases accuracy, so these two effects must be considered when setting this hyperparameter.