# A Kernel Least Mean Square Algorithm Based on Randomized Feature Networks
## 2 The KLMS Algorithm Based on Random Fourier Features
### Randomized Feature Networks-Based KLMS Algorithm  
In this paper, the random Fourier feature is used to construct an online kernel adaptive filtering algorithm. Based on the conclusions made by the authors of [25], the feature map of \(\varphi_{w}(\mathbf{x})=[cos(\mathbf{x}^{T}\mathbf{x})~{}sin(\mathbf{w}^{T}\mathbf{x})]\) is better than the map \(\psi_{w}(\mathbf{x})=\sqrt{2}cos(\mathbf{w}^{T}\mathbf{x}+b)\) for Gaussian kernel. Therefore, the randomized feature network is constructed based on the mapping function \(\varphi_{w}(\mathbf{x})\). Without special explanation, the Gaussian kernel is the default kernel function. Thus, the vector \(\mathbf{w}\) is randomly sampled form the Gaussian probability distribution. For the Gaussian kernel with mean 0 and covariance \(2\sigma I\), \(p(\mathbf{w})\sim N(0,2\sigma I)\) can be obtained, where \(\sigma\) is the kernel bandwidth. Before introducing the randomized feature networks-based KLMS algorithm, the unified kernel-based least mean square scheme based on the explicit feature mapping method is given. Distinct from the implicit mapped KLMS algorithm proposed by the authors of [4], the explicit-mapped KLMS algorithm has similar simplicity to the original LMS algorithm because of its ability to work directly with the weights in the relatively low-dimensional space. For the given training sample, set \(\{\mathbf{x}(n),d(n);n=1,2,...,N\}\), and consider the learning problem of a nonlinear function \(f(\cdot)\):  
\[d(n)=f(\mathbf{x}(n)),\mathbf{x}\in R^{m},d\in R \tag{10}\]where \(\mathbf{x}\) is the input vector and \(d\) is the output signal. For the kernel function \(K(\mathbf{x},\mathbf{y})\) that can be approximated by  
\[K(\mathbf{x},\mathbf{x}^{\prime})=<\Phi(\mathbf{x}),\Phi(\mathbf{x}^{\prime})>\approx\varphi(\bm {x})^{T}\varphi(\mathbf{x}^{\prime}), \tag{11}\]  
the corresponding nonlinear adaptive algorithm can be constructed using the following steps. First, for the given explicit mapping function \(\varphi(\cdot)\), the transformed input vector can be represented by \(\varphi(\mathbf{x}(n)),n\in{1,2,...,N}\), where \(\varphi(\mathbf{x}(n))\) denotes that the input vector \(\mathbf{x}(n)\) is transformed into the feature space \(\mathbf{F}\), which is a relatively low-dimensional space. Second, for the nth iteration, substituting \(\Phi(\mathbf{x}(n))\) into Equation (1) with the feature vector \(\varphi(\mathbf{x}(n))\), the filter output can be computed by  
\[y(n)=\mathbf{\Omega}(n-1)^{T}\varphi(\mathbf{x}(n)) \tag{12}\]  
where \(\mathbf{\Omega}(n-1)^{T}\) is the weight vector obtained in the \(n-1\)th iteration, and the feature vector \(\varphi(\mathbf{x}(n))\) can be pre-calculated by an approximate mapping function. The update equation of weight vector in the feature space \(\mathbf{F}\) can be expressed as  
\[\mathbf{\Omega}(n)=\mathbf{\Omega}(n-1)+\mu e(n)\varphi(\mathbf{x}(n)), \tag{13}\]  
where \(\mu\) is the step size and \(e(n)\) is the prediction error computed by  
\[e(n)=d(n)-y(n). \tag{14}\]  
Based on the kernel adaptive filter theory, the above equations, including (12)-(14), constitute the basic framework of the explicit-mapped kernel least mean square algorithm. Given the approximate kernel mapping function \(\varphi(\cdot)\), the corresponding kernel-based least mean square algorithm can be constructed. In contrast to the weight updating function applied for the implicit map, the weight vector updating function (13) can be directly achieved by computing the feature mapping function, and the feature information of historical samples can be saved in the weight vector during the iteration process instead of storing the important samples. The real-valued random Fourier feature vector is introduced into the explicit mapped KLMS framework in this paper; the corresponding KLMS-RFN algorithm is shown in Algorithm 1.