# Whitening CNN-Based Rotor System Fault Diagnosis Model Features
## 3 Proposed Improvement for Fault Diagnosis Models
### Whitening CNN Inputs  
ND, as proposed in [31], employs the matrix multiplication composition of a convolutional layer, as explained in Section 2.1. ND performs the whitening of the layer input \(\mathbf{X}\) by subtracting its mean, and then by multiplying it with the inverse square root of the corresponding covariance matrix. Subtracting the mean centres the layer input, and multiplying with the inverse square root of the covariance matrix decorrelates the layer input local area-wise and channel-wise. Ideally, after ND the layer inputs have a mean of zero and the covariance matrix is approximately an identity matrix. The computations related to this whitening operation are shown in the following equations:  
\[\mathbf{Cov}=\frac{1}{N}(\mathbf{X}-\boldsymbol{\mu})^{T}(\mathbf{X}- \boldsymbol{\mu}) \tag{12}\]  
\[\mathbf{D}\approx(\mathbf{Cov})^{-\frac{1}{2}} \tag{13}\]  
\[\mathbf{y}_{j}=(\mathbf{X}-\boldsymbol{\mu})\cdot\mathbf{D}\cdot\mathbf{w_{j} }+\mathbf{b_{j}}. \tag{14}\]  
In Equation (12), the covariance of layer input data is computed. \(\mathbf{X}\) is the transformed input data matrix and \(\boldsymbol{\mu}\) are the mean of \(N\) values in the columns of \(\mathbf{X}\), i.e., the mean of values multiplied by an arbitrary weight. \(N\) depends on the number of samples in a mini-batch and the number of local areas. \(\mathbf{D}\) is the deconvolution kernel in Equation (13), which is an approximation of the inverse square root of the covariance matrix. Equation (14) shows the computation of the output values \(\mathbf{y}_{j}\) of the \(j\)-th filter with the centered and transformed input \(\mathbf{X}-\boldsymbol{\mu}\), the deconvolution kernel \(\mathbf{D}\approx\mathbf{COV}^{-\frac{1}{2}}\), the filter weights \(\mathbf{w}_{j}\) and the filter bias terms \(\mathbf{b}_{j}\). The whitening computations, as presented above, may decelerate the optimisation of a deep neural network excessively. Therefore, computation acceleration techniques for ND were also proposed [31]. These techniques include subsampling the layer input matrix \(\mathbf{X}\) for lighter covariance matrix computation, coupled Newton-Schulz iteration for the inverse square root of the covariance matrix approximation, and implicit decorrelation of layer inputs. This work optimised these acceleration algorithms for the 1D-CNN-based fault diagnosis models. The remainder of this subsection explains the practical implementation of ND and the required acceleration techniques. After the layer input formulation to matrix \(\mathbf{X}\), as shown in Figure 1, the covariance matrix is computed as expressed in Equation (12). Since the covariance matrix computation is performed for every layer and during every forward pass, subsampling the layer input matrices \(\mathbf{X}^{l}\) likely decreases the required training time. That is, the number of rows \(N\) in a layer input matrix \(\mathbf{X}\), consisting of local areas from all mini-batch samples, decreases by subsampling the rows. Subsampling is likely to have a small effect on the covariance matrix because the number of input values \(N\) is relatively high compared to the covariance matrix dimensions. To whiten the layer inputs, the inverse square root of the covariance matrix needs to be computed. Several techniques exist for computing the inverse square root of a matrix. However, the coupled Newton-Schulz iteration was shown to be both a numerically stable and fast algorithm for approximating the inverse square root of the covariance matrix [31]. The iteration starts by initialising matrices \(\mathbf{Y_{0}}=\mathbf{Cov}+\epsilon\cdot\mathbf{I}\) and \(\mathbf{Z}_{0}=\mathbf{I}\). The matrices \(\mathbf{Y}_{k}\) and \(\mathbf{Z}_{k}\) are updated every iteration with Equations (15) and (16), respectively. The matrices converge to approximate values of the square root and the inverse square root of the covariance matrix, as shown in Formulas (17) and (18).