# Multi-Angle Fast Neural Tangent Kernel Classifier
## 3 Background Techniques
### Multi-Kernel Learning  
Compared with a single-kernel function, the multi-kernel model has higher flexibility. The high-dimensional space after the mapping of multiple kernel functions is a combined space made by combining various feature spaces, which can connect different feature mapping capabilities of subspaces and map different feature components of heterogeneous data by the most suitable single kernel function, respectively. The data can then be more accurately and reasonably expressed in the new space and improve the prediction accuracy of sample data.  
The different construction methods and characteristics of multi-kernel functions can be divided into three categories: synthetic kernels, multiscale kernels, and infinite kernels. We replace the traditional kernel functions such as linear kernel, Gaussian kernel and polynomial kernel with NTK with deep structure as the base kernel functions of the multikernel learning method, thus enhancing the representation capability of the multi-kernel learning method.  
The linear combinatorial synthesis method is used to perform convex combinations of multiple basis kernel functions, and the final combined kernel function \(K_{final}\) is expressed as:  
\[K_{final}=\sum\limits_{i=1}^{n}w_{i}K_{i},\ w_{i}\geq 0,\ \sum\limits_{i=1}^{n}w_{i}=1, \tag{3}\]  
where \(K_{i}\) is the basis kernel function, \(w_{i}\) is the weight coefficient of the basis kernel function, and \(n\) is the number of basis kernel functions. Then, the decision function of the SVM-based multi-kernel learning method can be transformed into:  
\[f(x)=\text{sgn}(\sum\limits_{j=1}^{o}a_{j}y_{j}K(x_{i},x)+b)\Rightarrow f(x)= \text{sgn}(\sum\limits_{j=1}^{o}a_{j}y_{j}\sum\limits_{i=1}^{n}w_{i}K_{i}(x_{i},x)+b) \tag{4}\]  
where \(0\leq a_{j}\leq C\) is a Lagrangian multiplier.  
The linear combinatorial multi-kernel model has no basis for parameter selection and combination, which cannot satisfactorily solve the uneven distribution of samples and limits the representation capability of the decision function. In contrast, multi-scale kernel fusion is more flexible and provides a more complete choice of scales.  
Among the widely used kernel functions, the Gaussian radial basis kernel is a typical multiscale kernel with generalized approximation capability. Take this kernel as an example and multiscale it (assuming it has translation invariance):  
\[\begin{split} K(x,z)&=\text{exp}(-\frac{\|x-z\|^{2 }}{2\sigma^{2}})\\ K_{i}(x,z)&=[\text{exp}(-\frac{\|x-z\|^{2}}{2\sigma _{1}^{2}}),\cdots,\text{exp}(-\frac{\|x-z\|^{2}}{2\sigma_{m}^{2}})]\end{split} \tag{5}\]  
where \(\sigma_{1}<\cdots<\sigma_{m}\), when \(\sigma\) is small, the model is prone to classify the drastically varying samples. When \(\sigma\) is large, it can classify those samples that vary gently, resulting in a better generalization capability. The decision function of the SVM-based multi-kernel learning method can be transformed into:  
\[\begin{split} f(x)&=f_{1}(x)+f_{2}(x)+\cdots+f_{m} (x)\\ f_{1}(x)&=\sum\limits_{i=1}^{N}a_{i}K_{1}(x_{i},x) +b_{1}\\ \vdots&\\ f_{m}(x)&=\sum\limits_{i=1}^{N}a_{i}K_{m}(x_{i},x) +b_{m}\end{split} \tag{6}\]