# IoT System for Detecting the Condition of Rotating Machines Based on Acoustic Signals
## 4 Experimental Results
### Comparative Analysis with Hyperparameter Tuning  
The preliminary analysis is performed on only one mill with a fixed set of hyperparameters. In order to verify the ability of this algorithm to maintain its performance on different machines, further experiments are performed. The mill from a previous example (Mill A) is observed again, but with the microphone located in different positions (Position 1--to the side of the mill in Figure 12: Position 2--near the back of the mill; and Position 3--at the front of the mill), to test whether the different type and level of ambient noise affect classification. The algorithm is tested on different industrial coal mills with different rotary frequencies (denoted as Mill B and Mill C). Seeing how both the rotary frequency and the number of impact plates of the mills differ, characteristic frequencies in the specter are different as well, so this test is used to demonstrate that the algorithm should be applicable for different cyclostationary rotary machines. The number of testing and training samples used in all of these cases is given in Table 5. \begin{table}
\begin{tabular}{c c c c} \hline \hline  & & \multicolumn{2}{c}{**PREDICTED**} \\ \cline{3-4}  & & **NEGATIVE** & **POSITIVE** \\ \hline \multirow{2}{*}{ACTUAL} & NEGATIVE & TN = 213 & FP = 7 \\ \cline{2-4}  & POSITIVE & FN = 2 & TP = 73 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Confusion matrix when class 4 is denoted as positive. Figure 14: Classification results for different partitions of data into train and test sets. Circles denote the classification median, while horizontal lines denote 25 th and 75 th percentile. The graph shows an average correct classification with low variability for each partition, which indicates high accuracy and efficiency of the algorithms. The approaches mentioned here are double tested: with default values of hyperparameters (used in the initial example--Section 4.2) and tuned hyperparameters. The tuning is conducted using GA, as described in Section 2. In this experiment, the number of DWT levels is limited between 4 and 10, bearing in mind that the frequency band of the approximation coefficients should not be too large because dominant frequency components will be averaged out, and it should not be too small because this increases dimensionality of feature space without adding any useful information. The number of NN hidden layers is fixed to 2, and the number of neurons in each layer is bonded to an integer number between 3 and 15. It is not advisable to increase the complexity of NN further because, in order to avoid overfitting, the number of training samples would need to exponentially increase as well. The performance of the trained neural network is used as an optimization function. Because the search grid is not very large, the population size is lowered to 10 individuals and the maximum number of generations is 10. This ensures that the optimization time is manageably low. The hyperparameters obtained in all these cases are given in Table 6, while the performance results using described metrics are given in Table 7. It should be noted that the accuracy of the algorithm is calculated considering all the classes, while precision, recall, and F1 score are calculated with respect to class 4. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **DWT\_LVL** & **NN\_LAYERS** \\ \hline
**DEFAULT** & 6 & [5; 5] \\ \hline
**MILL A, POSITION 1** & 8 & [8; 7] \\ \hline
**MILL A, POSITION 2** & 6 & [9; 3] \\ \hline
**MILL A, POSITION 3** & 6 & [8; 5] \\ \hline
**MILL B** & 5 & [5; 9] \\ \hline
**MILL C** & 6 & [8; 4] \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Hyperparameter values after optimization.