# Deep Learning for Anomaly Detection in CNC Machine Vibration Data: A RoughLSTM-Based Approach
## 4 Experimental Results and Discussion
### Performance Analysis
Furthermore, the AUC distributions show a clear separation from the random guessing threshold, with mean AUC values consistently above 0.93 and well-defined confidence intervals. These findings strongly confirm that RoughLSTM significantly outperforms the standard LSTM model, providing a robust and reliable classification approach for this dataset. Table 4 presents the cross-validation metrics for the LSTM and RoughLSTM models using different operations (OP02, OP05, OP08, OP11, OP14) of the M01 machine. The table is divided into two sections: (a) for the LSTM model and (b) for the RoughLSTM model. (a) LSTM model: The performance metrics for this model are generally low. The AUC values range between 0.503 and 0.507, and the F1-scores are between 0.731 and 0.733. Although the recall values are 1.000, the specificity values are 0.000, indicating that the model fails to correctly identify negative classes. (b) RoughLSTM model: This model shows significantly better performance compared to LSTM. The AUC values range from 0.929 to 0.979, and the F1-scores are between 0.852 and 0.942. The recall and specificity values are more balanced and higher, indicating that the model better identifies both positive and negative classes. In this section, as the next step, the experimental studies conducted for the M01 machine were similarly applied to the M02 and M03 machines. However, at this stage, only the confusion matrix and ROC values are presented, instead of the learning curves of the methods. Figure 8 shows an example of the combination of vibration data obtained in OP00, OP01, OP04, OP07, and OP09 operations on the MO2 machine. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**(a)** & & & & & & \\  & **AUC** & **AUC 95\%CI** & **F1-Score** & **Precision** & **Recall** & **Specificity** & **Std (AUC)** \\ \hline x axis & 0.503 & 0.491–0.515 & 0.731 & 0.576 & 1.000 & 0.000 & 0.006 \\ y axis & 0.507 & 0.495–0.518 & 0.732 & 0.577 & 1.000 & 0.000 & 0.006 \\ z axis & 0.493 & 0.482–0.504 & 0.733 & 0.578 & 1.000 & 0.000 & 0.006 \\
**(b)** & & & & & & \\ x axis & 0.935 & 0.927–0.943 & 0.852 & 0.800 & 0.911 & 0.676 & 0.004 \\ y axis & 0.929 & 0.920–0.937 & 0.860 & 0.822 & 0.902 & 0.722 & 0.004 \\ z axis & 0.979 & 0.974–0.983 & 0.942 & 0.928 & 0.956 & 0.894 & 0.002 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Cross-validation metrics for (a) LSTM and (b) RoughLSTM using OP02, OP05, OP08, OP11, and OP14 operations of M01 machine. In the confusion matrices (See Figure 9) for the LSTM model, the false negative (FN) rates are significantly high, indicating that the model fails to distinguish between classes and misclassifies almost all instances. This poor performance is evident across the x, y, and z axes, where the model predicts almost all instances incorrectly. In contrast, the RoughLSTM model reduces false negatives and false positives considerably, leading to a significant increase in correct predictions. This improvement is clearly visible in the confusion matrices, where the RoughLSTM model demonstrates a better balance between actual and predicted values. Additionally, the F1-scores in Figure 9 highlight another critical difference between the models. The LSTM model has low precision and recall values, leading to poor F1-scores, which confirm its inability to correctly identify class instances. However, the RoughLSTM model achieves substantially higher F1-scores, reinforcing its ability to balance precision and recall effectively. This suggests that RoughLSTM not only improves classification accuracy but also ensures better overall model robustness. In Figure 10, examining the ROC curves, the LSTM model's curve follows the random guess line, with AUC values (0.495, 0.493, 0.493) being very low, confirming its inefficiency. On the other hand, the RoughLSTM model's ROC curves are closer to the upper-left corner, achieving AUC values of 0.845, 0.952, and 0.895, which indicate strong classification performance.