# Spatial Autocorrelation Incorporated Machine Learning Model for Geotechnical Subsurface Modeling
## 2 Study Area and Methodology
### Gaussian Process Regression (GPR)  
The Gaussian processes regression (GPR) algorithm, as visualized in Figure 5c, is a form of Bayesian nonlinear regression and nonparametric algorithm [24]. One of the main components of the Gaussian process regression is the kernel or covariance function that defines the proximity or similarity of the target values (\(t\)) between two close inputs \(i\)[25]. Therefore, covariance functions \(k\) need to be calculated to prepare for the GPR. Using the presumptions of the Gaussian distribution, and assuming that the observed data points (\(i\),\(t\)) originate from noiseless distributions, the unseen target values (\(t^{*}\)) can be predicted. The aim is to predict \(t^{*}\) for new input data \(i^{*}\) based on the Gaussian process prior and the observed data points (\(i\),\(t\)). The joint prior distribution of the observed target values (\(t\)) and \(t^{*}\) is presented in Equation (4) [26].  
\[\begin{bmatrix}\mathbf{t}\\ \mathbf{t^{*}}\end{bmatrix}\ \sim\ \mathcal{N}\bigg{(}0,\begin{bmatrix}K&K_{T}^{*}\\ K^{*}&K^{**}\end{bmatrix}\bigg{)}. \tag{4}\]  
In Equation (4), \(\mathcal{N}\) is a Gaussian normal distribution with mean vector 0 and covariance matrix \(\begin{bmatrix}K&K_{T}^{*}\\ K^{*}&K^{**}\end{bmatrix}\), and \(T\) is the matrix transpose. The covariance matrix for the GPR (\(K\), \(K^{*}\) and \(K^{**}\)) is explained in Rasmussen and Williams [25]. By assessing the mean and covariance matrix from the conditional probability in Equation (5) given the data \(i\), \(i^{*}\), and \(t\), the \(t^{*}\) can be sampled from the joint posterior distribution. The \(t^{*}\) realizations from the posteriors are generated multiple times, and the best estimate for \(t^{*}\) acquired from the mean in this distribution (\(\overline{t^{*}}\)) is shown in Equation (6).  
Figure 5: Visualization of the applied machine learning algorithms; (**a**) support vector machine (SVM) for regression and classification; (**b**) artificial neural network (ANN) for regression and classification; (**c**) Gaussian process regression (GPR) for regression; (**d**) \(k\)-nearest neighbor (kNN) for classification.  
\[\mathbf{t}^{*}|\mathbf{t},\mathbf{i}^{*},\mathbf{i}\ \sim\ \mathcal{N}\Big{(}K^{*}K^{-1}\mathbf{t},K^{**}-K^{*}K ^{-1}K_{T}^{*}\Big{)}, \tag{5}\]  
\[\overline{\mathbf{t}^{*}}=K^{*}K^{-1}\mathbf{t}. \tag{6}\]  
Details on the GPR and its kernel or covariance function can be found in [PERSON] and Williams [25]. The application of GPR in civil engineering can be found in [PERSON] et al. [27].  
The \(k\)-nearest neighbor (kNN) algorithm is used for the classification model. This is a non-parametric algorithm that can identify classes of the \(k\)-nearest data points, which is closest in distance to the new data point [28; 29]. The algorithm performs classification based on the most common or majority class assigned to its \(k\)-closest data points [30]. The kNN algorithm (Figure 5d) has four steps [31]. The distance between the new data and entire data is calculated in the first step, commonly by using the Euclidean distance calculation method, as shown in Equation (7). Variables \(a\) and \(b\) are two points in Euclidean \(n\)-space, \(d_{am}\) and \(d_{bm}\) are Euclidean vectors. The distances are then ranked in the second step. In the third step, the smallest \(m\) values are taken and the class for the new data is determined in the final step.  
\[Euclidian_{i,j}=\sqrt{\sum_{m=1}^{\overline{n}}{(d_{am}-d_{bm})^{2}}}. \tag{7}\]