## 3 Fully Decoupled Extended Kalman Filter
### Fully Decoupled Extended Kalman Filter  
In the EKF for training neural networks, the computational requirements are mainly determined by the storage and updating needs of the estimation error covariance matrix. If the network has \(M\) weights and \(N_{o}\) outputs, the computational complexity is O(\(N_{o}M^{2}\)) and the storage requirements are O(\(M^{2}\)) [15]. Although the EKF stands out for its accuracy and faster convergence in terms of epochs, it is impractical in large neural networks [21]. To reduce the computational burden, [PERSON] and [PERSON] [11] introduced a modified EKF approach in 1991. This variant simplifies computations by disregarding statistical correlations between network parameters across different groups, operating under the assumption that each parameter solely correlates with others within the same group. Therefore, the complexity is O(\(N_{o}^{2}M+N_{o}\sum_{i=1}^{g}M_{i}^{2}\)), where \(M_{i}\) is the number of weights in the group \(i\), and the storage need is O(\(\sum_{i=1}^{g}M_{i}^{2}\)). The computational complexity and computational requirements are significantly lower than with the EKF. On the other hand, although the EKF calculations are performed for each group, the scaling matrix, whose inverse is required for calculating the Kalman gain, is unique and must be previously calculated in each iteration. Different forms of grouping can be assumed: a group for each layer of the network or layer-decoupled (LDEKF), a group for each node, or node-decoupled (NDEKF), and groups with only one element, or fully-decoupled (FDEKF); at the other extreme, there is a single group approach, which preserves the assumption of the EKF, referred to by the authors as the global group (GEKF). These authors mention the advantage of significantly reducing the computational load with little sacrifice of the performance of the trained network. The NDEKF approach appears to be the most natural, is consistent with the original EKF, and offers a good balance between computational complexity and accuracy. NDEKF has exhibited exceptional convergence characteristics and generalization capabilities, even outperforming GEFK, according to an experiment described in [34]. With decoupling, the estimation error covariance matrix becomes block diagonal for NDEKF and LDEKF, while for FDEKF it is absolute diagonal. Figure 2 shows a representation, similar to one shown in [11], of four different levels of decoupling. If we think about the error covariance matrix, we see that the white positions contain zeros and the gray positions contain non-zero values. On the other hand, taking advantage of the computational processing speed of currently available hardware, the FDEKF algorithm is now feasible for the complete training of multilayer neural networks with a large number of weights. Despite the disregard of statistical correlation between network parameters, FDEKF offers very good performance with much lower computational complexity. To apply any of the variants of the EKF to neural networks, it is assumed that there is a stationary neural model with ideal weights. These make up the state of the system that is attempted to be estimated through the application of the EKF. The response of the model is the output of the neural network. The ideal weights, which are kept constant, are those that minimize the error between the network output and the target output for the entire data set. In practice, it is argued that the introduction of noise into the process favors the performance of the filter [35].