# Model-Free Data Mining of Families of Rotating Machinery
## 4 Clustering and Classification  
After the harmonic feature space transform and reduction we begin the analysis of our set of _meaningful_ feature vectors. Classically there are two approaches: _supervised learning_ where the number of groupings (classes) are interpreted from the labels and _unsupervised learning_ where the number of groupings (clusters) are determined from some measurement in the feature space. While we have labels, we are interested in an unsupervised learning approach as we aim to find a relation between samples. As stated, clustering is the method of creating a partition of the data using some measurement criterion [15]. Once we know the number of clusters, we can easily compute the clustering. The key question is, how many clusters do we want? The \"best\" number of clusters is not necessarily equal to the number of existing labels as we also would like to understand differences between machines with the same label, so a recall-accuracy is not an applicable measure. We approach the problem using an _entropy-based_ measurement that represents the tradeoff of two points of view. We have labels that allow us to express ideal clustering conditions such as two machines with different drive types should be in two different clusters or machines with the same serial number should be in the same cluster. We express this mathematically in the next paragraphs. In supervised learning we are given labels \(y^{k}\), e.g., the serial numbers of the machines. This means we have a set of \(N\) disjoint classes \(\Omega=\{\Omega_{1},\cdots,\Omega_{N}\}\), \(\Omega_{n}\) being the set of the data points belonging to the class \(n\) so \(\Omega_{n}=\{x^{k}|y^{k}=n\}\). In clustering we compute a partition into K-clusters \(C=\{C_{1},\cdots,C_{K}\}\), again disjoint, with each \(x^{k}\) assigned to exactly one of the clusters. We prefer clusterings to have recordings of the same class (e.g., type of machine) in the same cluster. This can be satisfied trivially by putting all recordings into one cluster. We also prefer clusterings that have recordings of different classes (type of machines) in different clusters, which is trivially satisfied by putting all recordings into their own cluster. Hence we propose a trade-off technique by combining these two goals. We choose a measurement based on entropy. The entropy of a partition \(S=\{S_{i}\}\) is given by  
\[\mathcal{E}(S)=-\sum p(S_{i})\log(p(S_{i}))\]  
where \(p(S_{i})\) is the probability of samples belonging to \(S_{i}\) so \(p(S_{i})=\frac{|S_{i}|}{\sum|S_{i}|}\). \(\mathcal{E}(S)\) is 0 if there is only one class in the partition. \(\mathcal{E}(S)\) is at its maximum if there are as many classes as samples in the partition: \(-\frac{1}{N}\sum\log(\frac{1}{N})=\log(N)\). Now given two partitions and using the Bayesian probability  
\[p(X|Y)=\frac{p(X\cap Y)}{p(Y)}\]  
the _conditioned entropy_ is  
\[\mathcal{CE}(C|\Omega)=-\sum_{k}p(\Omega_{k})\sum_{i}p(C_{i}|\Omega_{k})\log (p(C_{i}|\Omega_{k})).\]  
Note, if \(\Omega\) is one single set then this is just \(\mathcal{E}(C)\) and if \(C\) is one single set this is 0. So, as desired, having all samples of the same class in the same cluster means \(\mathcal{CE}(C|\Omega)=0\) and having all samples of different classes in different clusters means \(\mathcal{CE}(\Omega|C)=0\). We define the _symmetric conditioned entropy_ as  
\[\mathcal{SCE}(\Omega,C)=\frac{\mathcal{CE}(\Omega|C)+\mathcal{CE}(C|\Omega)}{ \mathcal{E}(\Omega)+\mathcal{E}(C)}.\]The _symmetric conditioned entropy_ now presents a trade-off, by keeping \(\Omega\) fixed and varying the clustering \(C\) the measurement proves to be convex. If \(C\) is a renaming of \(\Omega\), i.e., same partition indexed differently, then \(\mathcal{SCE}(\Omega,C)=0\). The other two border cases are given above. More generally, the measure is designed for the case where the number of classes and clusters differ, i.e., non-square confusion matrices.