## 3 Discrete-Time 2 DOF Robot Manipulator Model
### Extended Kalman Filtering Training Algorithm  
In previous research, we demonstrated that Kalman filtering (KF) estimates the state of a linear system with an additive state and output white noise [12]. However, as the neural network mapping is a nonlinear system, then it is necessary to use an extended-KF (EKF) algorithm as described in [24]. The EKF is used for non-stationary discrete systems to ensure the accuracy of the online training process, and the mentioned method is shown below.  
\[\begin{split}& w_{j}^{l}(k+1)=w_{j}^{l}(k)+\eta_{j}^{l}K_{j}^{l}(k)e _{j}^{l}(k)\\ & K_{j}^{l}(k)=P_{j}^{l}(k)H_{j}^{l}(k)M_{j}^{l}(k)\\ & P_{j}^{l}(k+1)=P_{j}^{l}(k)-K_{j}^{l}(k)H_{j}^{l\top}(k)P_{j}^{ l}(k)+Q_{j}^{l}(k)\end{split}\, \tag{5}\]  
with  
\[\begin{split}& M_{j}^{l}(k)=[R_{j}^{l}(k)+H_{j}^{l\top}(k)P_{j}^{l}(k) H_{j}^{l}(k)]^{-1}\,\\ & e_{j}^{l}(k)=[x_{j}^{l}(k)-\chi_{j}^{l}(k)]\,\end{split} \tag{6}\]  
where \(l=1,2\) is the number of joints, \(j=1,2\) is the number of states of each joint, \(K_{j}^{l}(k)\in R^{L_{j}^{l}\times m}\) is the Kalman gain matrix, \(L_{j}^{l}\) is the respective number of neural network weights, \(\chi_{j}^{l}(k)\in R^{m}\) is the \(j\)-th neuron state corresponding to the \(l\)-th joint, \(x_{j}^{l}(k)\in R^{m}\) is \(j\)-th plant state of the \(l\)-th joint, \(\eta_{j}^{l}\) is the learning parameter, \(w_{j}^{l}(k)\in R^{L_{j}^{l}}\) is the synaptic weight vector calculated online; \(Q_{j}^{l}(k)\in R^{L_{j}^{l}\times L_{j}^{l}}\) is the measurement covariance matrix; \(R_{j}^{l}(k)\in R^{m\times m}\) is the state noise covariance matrix; \(e_{j}^{l}(k)\) is the prediction identification error; \(H_{j}^{l}(k)\in R^{L_{j}^{l}\times m}\) is a matrix in which each input \(H_{j}^{l}(k)\) is the derivative of \(j\)-th neural network state as is explained in more detail in [25]. For the proposed time-varying learning algorithm, the covariance matrices \(P_{j}^{l}(k)\), \(Q_{j}^{l}(k)\) and \(R_{j}^{l}(k)\) are initialized as diagonal matrices with random entries, and \(Q_{j}^{l}(k)\) and \(R_{j}^{l}(k)\) are composed of a time-varying coupled covariance between the \(j\)-th plant state, which allows the identification of interactions associated to the plant states and helps with the neural convergence. This technique is  
Figure 2: Neural identifier based on recurrent high-order neural network (RHONN) architecture.  
proposed in [12] and requires the efficient computation of \(Q_{j}^{l}(k)\) and \(R_{j}^{l}(k)\) in time-varying form such that it minimizes the identification error:  
\[\begin{split}& min_{Q_{j}^{l}(k)}\left(x_{j}^{l}(k)-x_{j}^{l}(k) \right),\\ & min_{R_{j}^{l}(k)}\left(y_{j}^{l}(k)-\bar{y}_{j}^{l}(k)\right). \end{split} \tag{7}\]  
This can be done in such a way that minimizes the variance (\(\sigma\)):  
\[\begin{split}&\sigma(x(k))=\Big{[}E\Big{(}[x(k)-x_{m}(k)]^{2} \Big{)}\Big{]}^{1/2},\\ &\sigma(y(k))=\Big{[}E\Big{(}[y(k)-y_{m}(k)]^{2}\Big{)}\Big{]}^{1 /2}\,\end{split} \tag{8}\]  
where \(x_{m}(k)=E(x(k))\) and \(y_{m}(k)=E(y(k))\) are expressed in terms of the recursive expectation value, \(E(\bullet)\), which represents the instantaneous mean value of the signal. For the proposed formulation, the state \([\mathbf{x}_{1}(k)\;\;\mathbf{x}_{2}(k)]^{\top}\) and output \(\mathbf{x}_{1}(k)=\mathbf{y}(k)\) are considered as available measurements [14]. To get the best identification process, the algorithm calculates the prediction covariance matrices with a recurrent on-line feeding process.