## 4 Modified Sparse Filtering and Two-Stage Learning Method  
In this section, a novel method known as modified sparse filtering is proposed, which aims to resolve the problem of overfitting. From the above discussion, the nature of the overfitting phenomenon is that different vectors of the weight matrix exhibit similar patterns. To suppress the similarity of the row vector, a constraint condition is applied to the cost function, which is shown as follows:  
\[\text{minimize}\;\mathsf{J}(\mathbf{W})=\sum_{i=1}^{M}\|\mathbf{\dot{y}}^{i}\|_{1}+ \lambda\sum_{i\
eq j}\Bigl{|}\mathbf{\omega}_{i}\mathbf{\omega}_{j}^{T}\Bigr{|} \tag{8}\]  
where \(\lambda\) is the tuning parameter, \(\mathbf{\omega}_{i}\mathbf{\omega}_{j}\) are the vectors of the weight matrix. The constraint condition represents the sum of the inner product among different row vectors.  
A two-stage learning method is proposed for intelligent fault diagnosis of machines based on the modified sparse filtering. The illustration and flowchart of the method are shown in Figure 10. In the first learning stage, modified sparse filtering is used to extract local discriminative features from raw vibration signals and the learned features of the signals are obtained by averaging these local features. In the second stage, softmax regression is applied to classify mechanical health conditions using the learned features.  
1. **Collect signals.** The vibration signals of machines are obtained under different health conditions. These signals compose the training set \(\left\{\mathbf{x}^{i},\mathbf{y}^{j}\right\}_{i=1}^{M}\), where \(\mathbf{x}^{i}\in\Re^{N\times 1}\) is the \(i\)th sample containing \(M\) vibration data points and \(\mathbf{y}^{i}\) is the health condition label. We collect \(N_{s}\) segments from each sample to compose the training set \(\left\{\mathbf{s}^{j}\right\}_{j=1}^{N_{s}}\) by an overlapped manner, where \(\mathbf{s}^{j}\in\Re^{N_{in}\times 1}\) is the \(j\)th segment containing \(N_{in}\) data points. The set \(\left\{\mathbf{s}^{j}\right\}_{j=1}^{N_{s}}\) is rewritten as a matrix form \(\mathbf{S}^{j}\in\Re^{N_{in}\times N_{s}}\).
2. **Whitening.** It is necessary to pre-process \(\mathsf{S}\) by whitening. Whitening uses the eigenvalue decomposition of the covariance matrix \[\text{cov}(\mathbf{S}^{T})=\mathbf{E}\mathbf{D}\mathbf{E}^{T}\] (9)  
Figure 9: Row vectors of \(\mathbf{W}\): (**a**) Vectors of \(N_{in}=100\) in the time domain; (**b**) Vectors of \(N_{in}=100\) in the frequency domain; (**c**) Vectors of \(N_{in}=200\) in the time domain; (**d**) Vectors of \(N_{in}=200\) in the frequency domain.  
where \(E\) denotes the orthogonal matrix of eigenvectors of the covariance matrix \(\text{cov}(S^{T})\), and \(D\) is the diagonal matrix of the eigenvalues. The whitened training set \(S_{w}\) can then be obtained as follows: \[S_{w}=E\mathbf{D}^{-1/2}\mathbf{E}^{T}\mathbf{S}\] (10)
3. **Train sparse filtering.**\(S_{w}\) is employed to train the modified sparse filtering model; as a result, the weight matrix \(\mathbf{W}\) is obtained by minimizing Equation (8).
4. **Calculate the local features.** The training sample \(x^{i}\) is alternately divided into \(K\) segments, where \(K=N/N_{in}\). These segments constitute a set \(\left\{x_{i}^{k}\right\}_{k=1}^{K}\), where \(x_{i}^{k}\in\Re^{N_{in}\times 1}\). The local features \(\hat{f_{k}}\in\Re^{1\times N_{in}}\) can be calculated from each training sample \(x_{i}^{k}\) by the weight matrix \(\mathbf{W}\).
5. **Obtain the learned features.** The local features \(\hat{f_{i}}\) are combined into a feature vector \(\hat{f}\) by averaging, and \(\hat{f}\) is the learned feature vector: \[\hat{f^{\prime}}=\left(\frac{1}{K}\sum_{k=1}^{K}\hat{f_{k}}\right)^{T}\] (11)
6. **Train softmax regression.** Once the learned feature set \(\left\{\hat{f^{\prime}}\right\}_{i=1}^{M}\) is obtained, we combine it with the label set \(\left\{\mathbf{y}^{i}\right\}_{i=1}^{M}\) to train softmax regression.