# An Insightful Overview of the Wiener Filter for System Identification
## 5 Wiener Filter via Kronecker Product Decomposition
Then, the MSE criterion is  
\[\mathcal{J}\left(\widehat{\mathbf{h}}_{1}|\widehat{\mathbf{h}}_{2}\right) = E\bigg{\{}\Big{[}d(k)-\widehat{\mathbf{h}}_{1}^{T}\widehat{ \mathbf{H}}_{2}^{T}\mathbf{x}(k)\Big{]}^{2}\bigg{\}}\] \[= \sigma_{d}^{2}-2\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{p} }_{2}+\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{R}}_{2}\widehat{\mathbf{ h}}_{1},\]  
where  
\[\underline{\mathbf{p}}_{2} = \widehat{\mathbf{H}}_{2}^{T}\underline{\mathbf{p}},\] \[\underline{\mathbf{R}}_{2} = \widehat{\mathbf{H}}_{2}^{T}\mathbf{R}\widehat{\mathbf{H}}_{2}.\]  
The minimization of \(\mathcal{J}\left(\widehat{\mathbf{h}}_{1}|\widehat{\mathbf{h}}_{2}\right)\) with respect to \(\widehat{\mathbf{h}}_{1}\) leads to  
\[\widehat{\mathbf{h}}_{1,W}=\underline{\mathbf{R}}_{2}^{-1}\underline{\mathbf{ p}}_{2}. \tag{36}\]  
In the same way, assume that \(\widehat{\mathbf{h}}_{1}\) (or, equivalently, \(\widehat{\mathbf{H}}_{1}\)) is fixed. Then, the MSE criterion is  
\[\mathcal{J}\left(\widehat{\mathbf{h}}_{2}|\widehat{\mathbf{h}}_{1}\right) = E\bigg{\{}\Big{[}d(k)-\widehat{\mathbf{h}}_{2}^{T}\widehat{ \mathbf{H}}_{1}^{T}\mathbf{x}(k)\Big{]}^{2}\bigg{\}}\] \[= \sigma_{d}^{2}-2\widehat{\mathbf{h}}_{2}^{T}\underline{\mathbf{p} }_{1}+\widehat{\mathbf{h}}_{2}^{T}\underline{\mathbf{R}}_{1}\widehat{\mathbf{ h}}_{2},\]where  
\[\underline{\underline{\mathbf{p}}}_{1} =\widehat{\underline{\mathbf{H}}}_{1}^{T}\underline{\mathbf{p}},\] \[\underline{\mathbf{R}}_{1} =\widehat{\underline{\mathbf{H}}}_{1}^{T}\mathbf{R}\widehat{ \underline{\mathbf{H}}}_{1}.\]  
The minimization of \(\mathcal{J}\left(\widehat{\underline{\mathbf{h}}}_{2}|\widehat{\underline{ \mathbf{h}}}_{1}\right)\) with respect to \(\widehat{\underline{\mathbf{h}}}_{2}\) leads to  
\[\widehat{\underline{\mathbf{h}}}_{2,W}=\underline{\mathbf{R}}_{1}^{-1} \underline{\underline{\mathbf{p}}}_{1}. \tag{38}\]  
By alternatively iterating between \(\widehat{\underline{\mathbf{h}}}_{1,W}\) and \(\widehat{\underline{\mathbf{h}}}_{2,W}\), an iterative Wiener filter is obtained, that converges to the true solution [10]. This approach, as shown in [10], performs better than the conventional Wiener filter in various contexts of noise and available data for the estimation of the involved second-order statistics. A more robust iterative Wiener filter must include regularization based on the \(\ell_{2}\) norm, i.e.,  
\[= \sigma_{d}^{2}-2\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{p} }_{2}+\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{R}}_{2}\widehat{\mathbf{ h}}_{1},\]0 \[= \sigma_{d}^{2}-2\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{p} }_{2}+\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{R}}_{2}\widehat{\mathbf{ h}}_{1},\]1  
from which the following optimal solutions are found:  
\[= \sigma_{d}^{2}-2\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{p} }_{2}+\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{R}}_{2}\widehat{\mathbf{ h}}_{1},\]2 \[= \sigma_{d}^{2}-2\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{p} }_{2}+\widehat{\mathbf{h}}_{1}^{T}\underline{\mathbf{R}}_{2}\widehat{\mathbf{ h}}_{1},\]3  
which is equivalent to regularize the covariance matrix \(\mathbf{R}\). Again, by alternatively iterating between \(\widehat{\underline{\mathbf{h}}}_{1,W,\delta}\) and \(\widehat{\underline{\mathbf{h}}}_{2,W,\delta}\), a robust iterative Wiener filter is obtained, that converges to the true solution. In order to avoid any potential numerical issues, a very small positive constant should be added to the diagonal of the matrices in (41) and (42) before inversion.