# Kernel Learning by Spectral Representation and Gaussian Mixtures
## 3 Approximating Stationary Kernels
### Approximating Locally Stationary Kernel  
As we know, \(\kappa_{2}\) is a stationary kernel which allows to approximate \(\kappa_{2}\) as presented in Section 3. Now, to obtain the locally stationary kernel, we would like to approximate \(\kappa_{1}\). For this, we define \(\zeta_{\mathbf{\varphi}}^{\prime}(\mathbf{x})=exp(i\mathbf{\upsilon}^{T}\mathbf{x}/2)\):  
\[\kappa_{1}\bigg{(}\frac{\mathbf{x}+\mathbf{x}^{\prime}}{2}\bigg{)}=\int_{\mathbb{R}^{ D}}exp\bigg{(}i\mathbf{\upsilon}^{T}\bigg{(}\frac{\mathbf{x}+\mathbf{x}^{\prime}}{2} \bigg{)}\bigg{)}f_{2}(\mathbf{\upsilon})d\mathbf{\upsilon}=\mathbb{E}_{\mathbf{\varphi}} \big{[}\zeta_{\mathbf{\upsilon}}^{\prime}(\mathbf{x})\zeta_{\mathbf{\upsilon}}^{\prime}( \mathbf{x})\big{]},\]  
where \(\mathbf{\upsilon}\sim f_{2}\). Using Monte Carlo integration and taking \(\mathbf{\upsilon}_{k}\sim f_{2}\), for \(k=1,2,\ldots,M_{2}\), it is possible to approximate \(\kappa_{1}\) as:  
\[\kappa_{1}\bigg{(}\frac{\mathbf{x}+\mathbf{x}^{\prime}}{2}\bigg{)}\approx\frac{1}{M_ {2}}\sum_{k=1}^{M_{2}}\zeta_{\mathbf{\upsilon}_{k}}^{\prime}(\mathbf{x})\zeta_{\mathbf{ \upsilon}_{k}}^{\prime}(\mathbf{x}^{\prime}). \tag{5}\]  
To approximate the output of the locally stationary kernel, we can use Equations (3) and (5) as follows:  
\[\kappa(\mathbf{x},\mathbf{x}^{\prime}) =\kappa_{1}\bigg{(}\frac{\mathbf{x}+\mathbf{x}^{\prime}}{2}\bigg{)}\kappa _{2}(\mathbf{x}-\mathbf{x}^{\prime})\] \[\approx\frac{1}{M_{1}M_{2}}\bigg{(}\sum_{k=1}^{M_{2}}\exp\bigg{(} i\frac{\mathbf{\upsilon}_{k}^{T}\mathbf{x}}{2}\bigg{)}\exp\bigg{(}i\frac{\mathbf{\upsilon}_{k}^{T} \mathbf{x}^{\prime}}{2}\bigg{)}\bigg{)}\bigg{(}\sum_{n=1}^{M_{1}}\exp\Big{(}i\mathbf{ \omega}_{n}^{T}\vec{x}\Big{)}\exp\Bigl{(}-i\mathbf{\omega}_{n}^{T}\mathbf{x}^{\prime} \Big{)}\bigg{)}\] \[=\frac{1}{M_{1}M_{2}}\sum_{n=1}^{M_{1}}\sum_{k=1}^{M_{2}}\exp \bigg{(}i\Big{(}\frac{\mathbf{\upsilon}_{k}}{2}+\mathbf{\omega}_{n}\Big{)}^{T}\mathbf{x} \bigg{)}\exp\bigg{(}i\Big{(}\frac{\mathbf{\upsilon}_{k}}{2}-\mathbf{\omega}_{n}\Big{)} ^{T}\mathbf{x}^{\prime}\bigg{)}\]  
where \(\mathbf{\omega}_{n}\sim f_{1}\) and \(\mathbf{\upsilon}_{k}\sim f_{2}\). In particular, if our kernel is real-valued, then previous equation becomes  
\[\kappa_{1}\bigg{(}\frac{\mathbf{x}+\mathbf{x}^{\prime}}{2}\bigg{)}\kappa_{2}\big{(}\bm {x}-\mathbf{x}^{\prime}\big{)}\approx\frac{1}{M_{1}M_{2}}\mathbf{\varphi}(\mathbf{x})^{T} \mathbf{\varphi}^{-}(\mathbf{x}^{\prime}) \tag{6}\]where  
\[ \begin{array}{c}\boldsymbol{\varphi}\phi\rangle_{-}\left[\omega\left(\left( \frac{\boldsymbol{v}_{1}}{2}+\boldsymbol{\omega}_{1}\right)^{T}\right)_{-} \ldots,\ldots,\omega\left(\left(\frac{\boldsymbol{v}_{M_{2}}}{2}+\boldsymbol{ \omega}_{M_{1}}\right)^{T}\right)_{-}\ldots\left(\left(\frac{\boldsymbol{v}_{ M_{2}}}{2}+\boldsymbol{\omega}_{M_{1}}\right)^{T}\right)_{-}\ldots\left(\left(\frac{ \boldsymbol{v}_{M_{2}}}{2}+\boldsymbol{\omega}_{M_{1}}\right)^{T}\right) \right]\\ \boldsymbol{\varphi}\cdot\boldsymbol{\psi}_{-}\left[\omega\left(\left(\frac{ \boldsymbol{v}_{1}}{2}-\boldsymbol{\omega}_{1}\right)^{T}\right)_{-}\ldots, \ldots,\omega\left(\left(\frac{\boldsymbol{v}_{M_{2}}}{2}-\boldsymbol{\omega}_ {M_{1}}\right)^{T}\right)_{-}\ldots\left(\left(\frac{\boldsymbol{v}_{M_{2}}}{2 }-\boldsymbol{\omega}_{M_{1}}\right)^{T}\right)_{-}\ldots\left(\left(\frac{ \boldsymbol{v}_{M_{2}}}{2}-\boldsymbol{\omega}_{M_{1}}\right)^{T}\right) \right]\end{array} \]  
and \(\boldsymbol{\omega}_{n}\sim f_{1},\boldsymbol{\psi}_{k}\sim f_{2}\). Thus, the advantage of representing the locally stationary kernel as Equation (6) is the possibility of computing \(f(\boldsymbol{x})\) as:  
\[f(\boldsymbol{x})=\sum_{j=1}^{N}\alpha_{i}\kappa(\boldsymbol{x}_{j}, \boldsymbol{x})\approx\frac{1}{M_{1}M_{2}}\sum_{j=1}^{N}\alpha_{i}\boldsymbol {\varphi}^{T}(\boldsymbol{x}_{j})\boldsymbol{\varphi}^{-}(\boldsymbol{x})= \boldsymbol{\psi}^{T}\boldsymbol{\varphi}^{-}(\boldsymbol{x})\]  
where \(\boldsymbol{\psi}=\frac{1}{M_{1}M_{2}}\sum_{j=1}^{N}\alpha_{i}\boldsymbol{ \varphi}^{T}(\boldsymbol{x}_{j})\).