# An Insightful Overview of the Wiener Filter for System Identification
## 2 System Model and the Conventional Wiener Filter  
Let us consider a linear single-input/single-output (SISO) system [14], whose output at the discrete-time index \(k\) is given by  
\[d(k) =\mathbf{h}^{T}\mathbf{x}(k)+w(k) \tag{1}\] \[=y(k)+w(k),\]  
where \(d(k)\) is also known as the reference or desired signal,  
\[\mathbf{x}(k)=\left[\begin{array}{cccc}x(k)&x(k-1)&\cdots&x(k-L+1)\\ \end{array}\right]^{T}\]is a vector containing the most recent \(L\) time samples of the zero-mean input signal \(x(k)\), the superscript \({}^{T}\) denotes the transpose operator, \(\mathbf{h}\) is the unknown system impulse response of length \(L\), and \(w(k)\) is the zero-mean additive noise. It is assumed that the signals take real values and that \(x(k)\) and \(w(k)\) are uncorrelated. Then, the objective of the so-called SISO system identification problem is to identify or estimate \(\mathbf{h}\), in the best possible way, given the input signal vector, \(\mathbf{x}(k)\), and the output signal, \(d(k)\). For this purpose, the celebrated Wiener technique is the most appropriate [15; 16; 17].  
From (1), it is clear that the variance of \(d(k)\) is  
\[\sigma_{d}^{2}=\mathbf{h}^{T}\mathbf{R}\mathbf{h}+\sigma_{w}^{2}, \tag{2}\]  
where \(\mathbf{R}=E\left[\mathbf{x}(k)\mathbf{x}^{T}(k)\right]\) is the \(L\times L\) covariance matrix of \(\mathbf{x}(k)\), with \(E[\cdot]\) denoting mathematical expectation, and \(\sigma_{w}^{2}\) is the variance of \(w(k)\). It can be deduced from (2) that the signal-to-noise ratio (SNR) of the studied SISO system as presented in (1) is  
\[\mathrm{SNR}(\mathbf{h})=\frac{\mathbf{h}^{T}\mathbf{R}\mathbf{h}}{\sigma_{w }^{2}}. \tag{3}\]  
Even though this SNR depends explicitly on the channel impulse response of the system, in many practical identification problems, this SNR is roughly known even if \(\mathbf{h}\) is not available [18].  
Let \(\widehat{\mathbf{h}}\) be a real-valued linear filter of length \(L\), which is also an estimate of \(\mathbf{h}\), so that \(\widehat{y}(k)=\widehat{\mathbf{h}}^{T}\mathbf{x}(k)\) is an estimate of \(d(k)\). Then, from the minimization of the popular mean-squared error (MSE) criterion  
\[\mathcal{J}\left(\widehat{\mathbf{h}}\right) =E\bigg{\{}\Big{[}d(k)-\widehat{\mathbf{h}}^{T}\mathbf{x}(k)\Big{]} ^{2}\bigg{\}} \tag{4}\] \[=\sigma_{d}^{2}-2\widehat{\mathbf{h}}^{T}\mathbf{p}+\widehat{ \mathbf{h}}^{T}\mathbf{R}\widehat{\mathbf{h}},\]  
the conventional Wiener filter [3; 5] is found  
\[\widehat{\mathbf{h}}_{\mathrm{W}}=\mathbf{R}^{-1}\mathbf{p}, \tag{5}\]  
where \(\mathbf{p}=E[\mathbf{x}(k)d(k)]\) represents the cross-correlation vector (of length \(L\)) between \(\mathbf{x}(k)\) and \(d(k)\). The block diagram of the conventional Wiener filter is depicted in Figure 1. While \(\widehat{\mathbf{h}}_{\mathrm{W}}\) leads to satisfactory results in many important applications [18], there is still room for improvement.  
It should be outlined that in different real-world applications, the presence of outliers (in the input signal and/or external noises) could bias the MSE criterion and, consequently, the solution from (5). This is due to the fact that the basic theory behind the conventional Wiener filter relies on the assumption of zero-mean and stationary signals. In order to cope with different outliers, the conventional approach should further involve additional control  
Figure 1: General block diagram of the conventional Wiener filter.  
and robustness mechanisms, e.g., see [19] and the references therein. The investigation of such particular cases is beyond the scope of this paper.