# Truncated Newton Kernel Ridge Regression for Prediction of Porosity in Additive Manufactured SS316L
## 3 Applied Regression Models and Modelling
### Kernel Ridge Regression (KRR)  
_KRR_ addresses nonlinearity in data by applying a mapping function \(\phi(.)\) that maps the data into a higher-dimensional space. The kernel function uses dot products such that  
\begin{table}
\begin{tabular}{c l} \hline \hline
**Symbols** & **Remarks** \\ \hline \(\mathbf{\beta}\) & Coefficient vector in \(RR\) and \(KRR\) \\ \hline \(\epsilon\) & Random error vector in \(RR\) and \(KRR\) \\ \hline \(\lambda\) & Regularization parameter in \(RR\) and \(KRR\) \\ \hline \(I_{d}\) & \(d\times d\) identity matrix in \(RR\) and \(KRR\) \\ \hline \(\gamma\) & Tuning parameter in the \(KRR\) radial basis function (\(=\frac{1}{2\sigma^{2}}\)) \\ \hline \(\sigma\) & Tuning parameter in the \(KRR\) radial basis function \\ \hline \(\mathbf{\kappa}\) & Dual variable vector in \(KRR\) \\ \hline \(\xi_{i},\xi_{i}\)\({}^{*}\) & Slack variables in \(SVR\) \\ \hline \(\mathbf{C}\) & Regularization parameter that adds a penalty in \(SVR\) \\ \hline \(\epsilon\) & Maximum error in \(SVR\) \\ \hline \(\alpha\), \(\mathbf{\alpha}^{*}\) & Lagrange multipliers in \(SVR\) \\ \hline \hline \end{tabular}
\end{table}
Table 3: List of symbols used in the models.  
\(K=k(\mathbf{x}_{i},\mathbf{x}_{j})=\left\langle\mathbf{\phi}(\mathbf{x}_{i}),\ \mathbf{\phi}(\mathbf{x}_{j})\right\rangle\)[39, 40]. The most used kernels are the linear, polynomial, and radial basis function kernels. For this study, the radial basis function kernel is used for both SVR and KRR and can be written as:  
\[k\big{(}\mathbf{x}_{i},\ \mathbf{x}_{j}\big{)}=e^{-\gamma\|\mathbf{x}_{i}-\mathbf{x}_{j}\|^{2}}=e^{ -\frac{1}{2\sigma^{2}}||\mathbf{x}_{i}-\mathbf{x}_{j}||^{2}} \tag{6}\]  
where \(\gamma=\frac{1}{2\sigma^{2}}\) is a tuning parameter that represents the width of the radial basis function.  
The vector \(\mathbf{\beta}\) in Equation (1) can be expressed as a linear combination of the data points such that \(\mathbf{\beta}=\mathbf{X}^{T}\mathbf{\alpha}\). Rewriting the model in Equation (1) results in  
\[\mathbf{y}=\mathbf{X}\mathbf{X}^{T}\mathbf{\alpha}+\mathbf{\epsilon}=\mathbf{G}\mathbf{\alpha}+\mathbf{\epsilon} \tag{7}\]  
where the matrix \(\mathbf{G}=\mathbf{X}\mathbf{X}^{T}\) is a Gram matrix.  
When the kernel \(K\) replaces the Gramian matrix \(\mathbf{G}\) in Equation (7), the \(KRR\) model becomes  
\[\mathbf{y}=\mathbf{K}\mathbf{\alpha}+\mathbf{\epsilon} \tag{8}\]  
The \(KRR\) function to be minimized with regard to the dual variable \(\mathbf{\alpha}\in\) becomes  
\[f(\mathbf{\alpha})=\frac{1}{2}(\mathbf{y}-\mathbf{K}\mathbf{\alpha})^{T}(\mathbf{y}-\mathbf{K}\mathbf{ \alpha})+\frac{\lambda}{2}||\mathbf{\alpha}||^{2} \tag{9}\]  
to which the solution is  
\[\mathbf{\alpha}=(\mathbf{K}+\lambda\mathbf{I}_{N})^{-1}\mathbf{y} \tag{10}\]  
If the matrix \((\mathbf{K}+\lambda\mathbf{I}_{N})\) is dense, the computational time has a complexity of \(O\big{(}N^{3}\big{)}\). Hence, iterative methods are the most suitable to compute \(\mathbf{\alpha}\). Thus, the model in Equation (10) can be rewritten as  
\[\mathbf{y}=(\mathbf{K}+\lambda\mathbf{I}_{N})\mathbf{\alpha} \tag{11}\]  
which is a linear system of equations and can be solved using iterative methods. [PERSON] and [PERSON] [40] have applied the linear conjugate gradient to the Newton step, creating a truncated regularized kernel ridge regression (_TR-KRR_) algorithm. The _TR-KRR_ algorithm is much faster than \(SVR\) without compromising accuracy. In this research, the _KRR_ method is implemented as _TR-KRR_. Details about the _TR-KRR_ algorithm can be found in [40].