# Kernel Learning by Spectral Representation and Gaussian Mixtures
## 3 Approximating Stationary Kernels
### Approximating Locally Stationary Kernel
Given this representation, we only need to compute \(\boldsymbol{\psi}\) once, avoiding the use of total Gram Matrix representation. Now, it is necessary to remark an interesting property of using this representation. It is possible to say that \(|\boldsymbol{\varphi}^{T}(\boldsymbol{x})\boldsymbol{\varphi}^{-}(\boldsymbol {x}^{\prime})-\kappa(\boldsymbol{x},\boldsymbol{x}^{\prime})|\leq C\) (using the [PERSON]'s inequality [23]) almost everywhere. Given this, it is possible to obtain the following inequality: given any \(\epsilon>0\), and taking samples \(M_{1}\) and \(M_{2}\) from \(\kappa_{2}\) and \(\kappa_{1}\) respectively; then  
\[P\left(|\boldsymbol{\varphi}^{T}(\boldsymbol{x})\boldsymbol{\varphi}( \boldsymbol{x}^{\prime})-\kappa(\boldsymbol{x},\boldsymbol{x}^{\prime})| \geq\epsilon\right)\leq 2\exp\left(-\frac{M_{1}M_{2}e^{2}}{2(\sigma^{2}+2 \epsilon^{2}/3)}\right).\]  
Therefore, the proposed representation of the kernel allows to obtain a good approximation to \(\boldsymbol{\varphi}^{T}(\boldsymbol{x})\boldsymbol{\varphi}^{-}(\boldsymbol {x}^{\prime})\). Furthermore, the following theorem gives a tighter bound making possible to say: Given a larger \(\epsilon\), the less likely is the possibility of having a larger \(|\boldsymbol{\varphi}^{T}(\boldsymbol{x})\boldsymbol{\varphi}(\boldsymbol{x} ^{\prime})-\kappa(\boldsymbol{x},\boldsymbol{x}^{\prime})|\). **Theorem 1**. : _Approximation of a locally stationary kernel._  
Let \(\mathcal{M}\) be a compact subset of \(\mathbb{R}^{D}\) with diameter \(Diam(\mathcal{M})\), and \(\sigma^{2}>\epsilon/2\) then the approximation of the kernel is given by:  
\[P\left(\sup_{\boldsymbol{x},\boldsymbol{x}^{\prime}}\left|\boldsymbol{\varphi }^{T}(\boldsymbol{x})\boldsymbol{\varphi}(\boldsymbol{x}^{\prime})-\kappa( \boldsymbol{x},\boldsymbol{x}^{\prime})\right|\geq\epsilon\right)<\frac{2 \sigma^{2}}{\epsilon}\left(1+4 Diam(\mathcal{M})\exp\left(-\frac{M_{1}M_{2} \epsilon^{2}}{4D(\sigma^{2}+2\epsilon^{2}/3)}\right)\right)\]  
**Proof of Theorem 1. ** Define \(s(\boldsymbol{x},\boldsymbol{x}^{\prime})\equiv\boldsymbol{\varphi}^{T}( \boldsymbol{x})\boldsymbol{\varphi}^{-}(\boldsymbol{x}^{\prime})\), a locally stationary kernel \(\kappa(\boldsymbol{x},\boldsymbol{x}^{\prime})\equiv\kappa_{1}\left(\frac{ \boldsymbol{x}+\boldsymbol{x}^{\prime}}{2}\right)\kappa_{2}(\boldsymbol{x}- \boldsymbol{x}^{\prime})\) and \(f(\boldsymbol{x},\boldsymbol{x}^{\prime})=|s(\boldsymbol{x},\boldsymbol{x}^{ \prime})-\kappa(\boldsymbol{x},\boldsymbol{x}^{\prime})|\leq 2\). Then, it is possible to say that \(\mathbb{E}[f(\boldsymbol{x},\boldsymbol{x}^{\prime})]=0\). Given that \(\kappa_{2}\) is shift invariant, it is possible to define \(\Delta_{-}\equiv\boldsymbol{x}-\boldsymbol{x}^{\prime}\in\mathcal{M}_{-}\). Now, given that \(\kappa_{1}\) can be interpreted as the mean, it is possible to define \(\Delta_{+}\equiv\frac{\boldsymbol{x}+\boldsymbol{x}^{\prime}}{2}\in\mathcal{ M}_{+}\). Consequently, it is possible to define \(\kappa(\Delta_{-},\Delta_{+})=\kappa_{1}(\Delta_{+})\kappa_{2}(\Delta_{-})\). Let \(\mathcal{M}\subset\mathbb{R}^{D}\) a compact bounded subset, it is known that \(Diam(\mathcal{M}_{-})\leq 2 Diam(\mathcal{M})\) and \(Diam(\mathcal{M}_{+})\leq 2 Diam(\mathcal{M})\). With this in mind, it is possible to define \(e\)-net that covers \(\mathcal{M}_{-}\times\mathcal{M}_{+}\) at most \(T=\left(\frac{4 Diam(\mathcal{M})}{r}\right)^{2D}\) balls of radius \(r\). Let \(\{\Delta_{-,i},\Delta_{+,i}\}_{i=1}^{T}\) denote the center of these \(T\) balls, and let \(L_{J}\) be the Lipschitz constant of \(f\). Therefore, we have that \(|f(\Delta_{-},\Delta_{+})|<\epsilon\) for all \(\Delta_{-},\Delta_{+}\in\mathcal{M}_{-}\times\mathcal{M}_{+}\). Then, \(f(\Delta_{-,i},\Delta_{+,i})<\frac{\epsilon}{2}\) and \(L_{f}<\frac{\epsilon}{2r}\) for all \(i\). Now, \(L_{f}=\|\
abla f(\Delta_{+,i}^{*},\Delta_{-,i}^{*})\|\), where \((\Delta_{+}^{*},\Delta_{-}^{*})=\max_{\Delta_{+},\Delta_{-}\in\mathcal{M}_{-} \times\mathcal{M}_{+}}\|\
abla f(\Delta_{+},\Delta_{-})\|\).