# Deep Learning for Anomaly Detection in CNC Machine Vibration Data: A RoughLSTM-Based Approach
## 4 Experimental Results and Discussion
### Performance Analysis
A lower validation loss indicates that the model is less likely to be overfitting. RoughLSTM shows lower variance than LSTM, which proves that the model offers more stable learning. In Figure 6, examining the confusion matrices and performance metrics of both models across three axes (x, y, z), significant differences are revealed. The LSTM model (Figure 6a) shows extreme overfitting to class \"1\" across all axes (x, y, z) and misclassifies all examples belonging to class \"0\" as false negatives. It incorrectly classifies 2281 samples on the x-axis, 2284 on the y-axis, and 2290 on the z-axis as false positives. This indicates that the model failed to achieve balanced learning and is biased towards a single class. Additionally, the precision, recall, and specificity metrics for the LSTM model reflect this imbalance, with recall values being notably high due to the model's bias toward class \"1\" but specificity being considerably low. In contrast, the RoughLSTM model (Figure 6b) demonstrates more balanced classification performance. On the x-axis, it achieves 1105 true negatives and 2118 true positives, while on the y-axis, it shows 1181 true negatives and 2096 true positives. The z-axis performance reveals 1462 true negatives and 2222 true positives, the model successfully making reasonable predictions for both classes. Unlike the LSTM model, RoughLSTM does not suffer from extreme class bias, as seen in the more evenly distributed precision, recall, and specificity values. The additional performance metrics (precision, recall, F1-score, and specificity) further validate RoughLSTM's superiority. While the LSTM model achieves high recall due to class bias, RoughLSTM maintains a better trade-off between precision and recall, ensuring that both classes are correctly identified. The specificity scores also confirm RoughLSTM's ability to avoid misclassifications of negative cases, which is crucial for real-world anomaly detection applications. In conclusion, the RoughLSTM model shows a more balanced and reliable classification performance. Unlike the LSTM model, it can recognize both classes and has a lower misclassification rate. This indicates that RoughLSTM is more suitable for practical applications. The model's ability to maintain good performance across all axes while correctly identifying both positive and negative cases demonstrates its superior generalization capability and robustness compared to the standard LSTM model. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & **Average** & **Average** & \multirow{2}{*}{**Average**} & **Average** & **Standard** & **Standard** \\  & **Training** & **Validation** & & **Training Loss** & **Validation** & **Deviation of** \\  & **Accuracy** & **Accuracy** & & **Loss** & **Accuracy** & **Loss** \\ \hline LSTM & \(\sim\)\%85 & \(\sim\)\%78 & \(\sim\)0.35 & \(\sim\)0.42 & 4.2 & 3.8 \\ RoughLSTM & \(\sim\)\%89 & \(\sim\)\%84 & \(\sim\)0.28 & \(\sim\)0.34 & 2.1 & 2.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Statistical performance evaluation of models. Figure 7 compares the ROC curves and AUC distributions of the LSTM model and the RoughLSTM model. In Figure 7a, the ROC curve of the LSTM model nearly follows the 45-degree line (random guess line), with AUC (area under the curve) values of 0.503, 0.507, and 0.493, respectively. These values indicate that the model's classification performance is at the level of random guessing, making it ineffective for this dataset. Additionally, the AUC distributions at the bottom further confirm the model's poor performance, as the bootstrap distributions are tightly clustered around 0.50, reinforcing its lack of discriminative power. Conversely, Figure 7b demonstrates that the RoughLSTM model significantly improves classification performance. The ROC curves for the x, y, and z axes are much closer to the upper-left corner, resembling an ideal classifier. The AUC values of 0.935, 0.959, and 0.979 indicate that the RoughLSTM model effectively differentiates between the two classes.