# Transient Analysis of a Selective Partial-Update LMS Algorithm
## 3 Stochastic Modelling of the Proposed Algorithm
### Second-Order Analysis  
The usage of the independence assumption implies that the expectation value of the matrix  
\[\mathbf{\Phi}(k)=\hat{\mathbf{v}}(k)\hat{\mathbf{w}}^{T}(k) \tag{22}\]  
has a major rule in a second-order stochastic model of adaptive algorithms [20]. Using Equation (19), a recursion for \(\mathbf{\Phi}(k)\) can be obtained as  
\[\begin{array}{rcl}\mathbf{\Phi}(k+1)&=&\mathbf{\Phi}(k)-\beta\mathbf{\Phi}(k)\mathbf{x}(k) \mathbf{x}^{T}(k)\mathbf{\Gamma}(k)\\ &&-\beta\mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{T}(k)\mathbf{\Phi}(k)\\ &&+\beta^{2}\mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{T}(k)\mathbf{\Phi}(k)\mathbf{x}(k)\mathbf{x}^{T }(k)\mathbf{\Gamma}(k)\\ &&+\beta^{2}\
u^{2}(k)\mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{T}(k)\mathbf{\Gamma}(k)+ \mathcal{O}[\
u(k)],\end{array} \tag{23}\]  
where \(\mathcal{O}[\
u(k)]\) contains first-order noise components, which are irrelevant to the following analysis.  
Recursion (23) can be rewritten in a more adequate formulation by applying the relationship  
\[\text{vec}(\mathbf{X}\mathbf{Y}\mathbf{Z})=\Big{(}\mathbf{Z}^{T}\otimes\mathbf{X}\Big{)}\text{vec }(\mathbf{Y}), \tag{24}\]  
where \(\text{vec}(\mathbf{A})\) is an operator that stacks the columns of matrix \(\mathbf{A}\) in order to generate a column-type vector and \(\otimes\) denotes the Kronecker product. Employing such a formulation, applying the expectation operator and using the independence assumption, leads to the following time-invariant state space equation:  
\[\mathbf{v}(k+1)=\mathbf{A}\mathbf{v}(k)+\mathbf{b}, \tag{25}\]  
where  
\[\mathbf{v}(k)\triangleq\mathbb{E}\{\text{vec}[\mathbf{\Phi}(k)]\}, \tag{26}\]  
\[\begin{array}{rcl}\mathbf{A}&\triangleq&\mathbf{I}-\beta\mathbb{E}\Big{\{}\Big{[} \mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{T}(k)\Big{]}\otimes\mathbf{I}\Big{\}}\\ &&-\beta\mathbb{E}\Big{\{}\mathbf{I}\otimes\Big{[}\mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{ T}(k)\Big{]}\Big{\}}\\ &&+\beta^{2}\mathbb{E}\Big{\{}\Big{[}\mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{T}(k) \Big{]}\otimes\Big{[}\mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{T}(k)\Big{]}\Big{\}}, \end{array} \tag{27}\]  
and  
\[\mathbf{b}\triangleq\beta^{2}\sigma_{v}^{2}\mathbb{E}\Big{\{}\text{vec}\Big{[} \mathbf{\Gamma}(k)\mathbf{x}(k)\mathbf{x}^{T}(k)\mathbf{\Gamma}(k)\Big{]}\Big{\}}, \tag{28}\]  
where \(\sigma_{v}^{2}\) is the variance of the additive noise.  
Since \(\text{vec}[\mathbf{X}]\) is a bijection operator, all information of the mismatch covariance matrix \(\mathbf{R}_{\mathbf{w}}(k)\triangleq\mathbb{E}[\mathbf{\Phi}(k)]\) can be extracted from \(\mathbf{v}(k)\). From both IA and NA stochastic hypotheses, the mean square deviation (MSD) and mean square error (MSE) performance metrics can be predicted for the \(k\)-th iteration using [20]  
\[\text{MSD}(k) \triangleq \mathbb{E}\Big{[}\|\mathbf{w}^{\star}-\mathbf{w}(k)\|^{2}\Big{]}\approx \text{Tr}[\mathbf{R}_{\mathbf{w}}(k)], \tag{29}\] \[\text{MSE}(k) \triangleq \mathbb{E}\Big{[}e^{2}(k)\Big{]}\approx\sigma_{v}^{2}+\text{Tr}[ \mathbf{R}_{\mathbf{x}}\mathbf{R}_{\mathbf{w}}(k)], \tag{30}\]  
where \(\text{Tr}[\mathbf{Y}]\) denotes the trace of matrix \(\mathbf{Y}\) and \(\mathbf{R}_{\mathbf{x}}\triangleq\mathbb{E}\big{[}\mathbf{x}(k)\mathbf{x}^{T}(k)\big{]}\) is the input autocorrelation matrix.  
**Remark 4**.: _It is noteworthy that Equation (25) is able to yield a closed-form estimate for both asymptotic MSD and MSE, provided the algorithm is stable. In this case, the steady-state vector \(\mathbf{v}_{\infty}\triangleq\lim_{k\to\infty}\mathbf{v}(k)\) can be computed by [20]_  
\[\mathbf{v}_{\infty}=(\mathbf{I}-\mathbf{A})^{-1}\mathbf{b}. \tag{31}\]