# Model-Free Data Mining of Families of Rotating Machinery
## 3 Statistical Significance and Reduction  
Now that we have a meaningful feature vector we want to identify the statistically most significant dimensions and components and discard the rest. This is important because the harmonic feature space likely contains a significant amount of irrelevant information caused by dependencies and redundancies. Our goal is to identify the most influential dimensions and components, the ones with a very high variance [10,11]. Higher dimensionality feature spaces also lead to exponentially more complex classification [21]. We find that the reduction is not only wanted but also necessary for the performance of linear classifiers and clustering algorithms. We first compute the _dimension-wise_ (each dimension separate) mean \(\mu(X)\) and _dimension-wise_ standard deviation \(\sigma(X)\) over all the recordings of each machine. Bins with a small variance reflect features that are generally the same across all recordings and will not assist with further discrimination of the type/class of machine. Similarly bins with a large variance reflect features that vary across recordings and might be particularly useful for such discrimination. We eliminate all bins whose variance falls below some set threshold. This results in a very significant reduction in the dimension of our data. Figure 3 shows the bins that remain to have a variance larger than the threshold. Immediately a pattern emerges: our recordings exhibit a fingerprint of the quarter, half, and full harmonics. For a 3-axis system with an average of \(N/2=2000\) relevant FFT data points per axis we get a reduction to about 50-80 data points as can be seen in Figure 3.  
Next we examine the _component-wise_ dependencies which are the dependencies between the dimensions, interpreted as the dependencies between the harmonics, e.g., possible links between the second and fourth harmonic. This is accomplished by _principal component analysis_ (PCA). Again, we examine this from a variance point of view. We compute a transform \(T\) to change the basis of our feature space such that it minimizes the reconstruction error:  
\[\min_{u_{m}}\sum_{k=0}^{K}||x^{k}-u_{m}u_{m}^{\intercal}x^{k}||\]  
having \(M\) orthogonal vectors \(u_{m}\). In other words, we choose the \(u_{m}\) to minimize the information lost when reconstructing the features \(x^{k}\) in the transformed space. The total variance is hence explained by a set of components, each contributing a certain amount to the total. Conveniently, as per PCA, the \(u_{m}\) are the eigenvectors of the covariance matrix \(\Sigma\). The components can be computed by eigenvalue decomposition. The question is, how many of these components do we need? After normalizing all eigenvectors and sorting them by the magnitude of the corresponding eigenvalue \(\lambda_{m}\), we have to decide how much of the original variance is important to us as the total variance is explained by a sum of eigenvalues \(\sigma_{x}^{2}=\sum\lambda_{m}\). The _principal component_\(u_{1}\) will preserve the most information as it is the direction of the data's largest variance, \(u_{2}\) represents the second principal component, and so on. In our application using the Kaiser rule and choosing only components with eigenvalues larger than 1 works very well, resulting in the selection of the first 5-10 principal components of the total 50. In other words, only about 10-20% of the data meets the [PERSON] criterion. Less than half of the components are needed to preserve 95% of the variance. Using the selected principal components, the data are, hence, transformed into the new _feature space_ by computing:  
\[x^{\text{PS}}=T^{\intercal}x,\ T= \begin{bmatrix}u_{1}&u_{2}&\ldots&u_{M}\end{bmatrix} \]  
The transformations are computed and stored. Note that the transformations are not label-dependent as in multi-class SVM. The stored transformations are used to transform new data that needs to be analyzed or classified.