# Deep Learning for Anomaly Detection in CNC Machine Vibration Data: A RoughLSTM-Based Approach
## 4 Experimental Results and Discussion
### Performance Analysis
Figure 8: Time series representation of vibration data of OP00, OP01, OP04, OP07, and OP09 operations of M02 machine. Figure 9: Comparison of confusion matrices for LSTM (**a**) and RoughLSTM (**b**) models using OP00, OP01, OP04, OP07, and OP09 operations of M02 machine. Table 5 compares the performance of the LSTM and RoughLSTM models on the M03 machine. LSTM (a) shows low performance, while RoughLSTM (b) delivers higher and more balanced metrics. RoughLSTM particularly excels in AUC and F1-score. Figure 10: Comparison of ROC curves for LSTM (**a**) and RoughLSTM (**b**) models using OP00, OP01, OP04, OP07, and OP09 operations of M02 machine. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multicolumn{1}{c}{**(a)**} & & & & & & \\ \multicolumn{1}{c}{} & **AUC** & **AUC 95\%CI** & **F1-Score** & **Precision** & **Recall** & **Specificity** & **Std (AUC)** \\ \hline x axis & 0.4950 & 0.483–0.507 & 0.751 & 0.600 & 1.000 & 0.000 & 0.006 \\ y axis & 0.4930 & 0.480–0.505 & 0.756 & 0.607 & 1.000 & 0.000 & 0.006 \\ z axis & 0.4930 & 0.480–0.505 & 0.756 & 0.607 & 1.000 & 0.000 & 0.006 \\
**(b)** & & & & & & \\ x axis & 0.8450 & 0.833–0.857 & 0.8200 & 0.8200 & 0.9560 & 0.3320 & 0.006 \\ y axis & 0.9520 & 0.946–0.958 & 0.8920 & 0.8570 & 0.9300 & 0.7300 & 0.003 \\ z axis & 0.8950 & 0.885–0.906 & 0.8530 & 0.7630 & 0.9670 & 0.4600 & 0.005 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Cross-validation metrics for (a) LSTM and (b) RoughLSTM using OP00, OP01, OP04, OP07, and OP09 operations of M02 machine. Table 6 compares the LSTM and RoughLSTM models on the M03 machine. LSTM (a) shows low AUC (0.489-0.512) and zero specificity despite perfect recall, indicating poor negative class identification. RoughLSTM (b) performs significantly better, with higher AUC (0.912-0.943), balanced recall-specificity, and improved F1-scores (0.869-0.911), demonstrating superior overall performance. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multicolumn{1}{c}{**(a)**} & & & & & \\ \multicolumn{1}{c}{} & **AUC** & **AUC 95\%CI** & **F1-Score** & **Precision** & **Recall** & **Specificity** & **Std (AUC)** \\ \hline x axis & 0.512 & 0.500–0.524 & 0.770 & 0.626 & 1.000 & 0.000 & 0.006 \\ y axis & 0.489 & 0.477–0.501 & 0.766 & 0.624 & 1.000 & 0.000 & 0.006 \\ z axis & 0.495 & 0.483–0.508 & 0.767 & 0.622 & 1.000 & 0.000 & 0.006 \\
**(b)** & & & & & & \\ x axis & 0.943 & 0.933–0.951 & 0.911 & 0.861 & 0.967 & 0.664 & 0.006 \\ y axis & 0.918 & 0.908–0.927 & 0.869 & 0.831 & 0.912 & 0.599 & 0.003 \\ z axis & 0.912 & 0.901–0.923 & 0.900 & 0.852 & 0.954 & 0.648 & 0.005 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Cross-validation metrics for (a) LSTM and (b) RoughLSTM using OP01, OP02, OP04, OP07, and OP10 operations of M03 machine. Figure 13: Comparison of ROC curves for LSTM (**a**) and RoughLSTM (**b**) models using OP01, OP02, OP04, OP07, and OP10 operations of M03 machine. In this study, the accuracy values of the LSTM and RoughLSTM models were compared through experiments conducted on MO1, MO2, and MO3 machines, and the results are given in Table 7. The results indicate that RoughLSTM achieved higher accuracy across all machines and axes compared to LSTM. While the accuracy of LSTM models generally remained in the 60-70% range, RoughLSTM demonstrated superior performance, ranging from 73% to 90%. When analyzed on a machine basis, RoughLSTM achieved 17%, 18%, and 23% higher accuracy on the x, y, and z axes, respectively, for the MO1 machine. For the MO2 machine, the most significant improvement was observed on the y-axis, where RoughLSTM outperformed LSTM by 24%. For the MO3 machine, RoughLSTM demonstrated a performance advantage of 15-20% over LSTM across all axes, further highlighting its superiority. Additionally, the highest values are indicated in dark color and in all other tables, dark color represents the highest value.