# A Robust Conformal Framework for IoT-Based Predictive Maintenance
## 6 Experimental Results and Performance Evaluation: A Case Study on CMAPSS Datasets
### Detailed Performance Analysis on FD004  
A broader perspective on the training process can be obtained by examining the aggregated loss curves, as presented in Figure 9.  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Method** & **Hyperparameters** \\ \hline
**Naive Margin** & \(\alpha=0.05\); Margin = \((1-\alpha)\) \\ \hline
**Weighted Exponential Margin** & \(\alpha=0.05\); Weights: \(w_{i}=\exp\!\left(\frac{i}{n/5}\right)\) \\ \hline
**Bootstrap Margin** & \(\alpha=0.05\); \(B=200\) bootstrap samples; \\ \hline \hline \end{tabular}
\end{table}
Table 3: Conformal methods and their key hyperparameters.  
The hybrid architecture CNN-GRU surpasses the other metrics in terms of training loss by continuing to decline smoothly, effectively handling the noisy pattern in the sensor through its convolutional masks acting as a feature engineering layer and managing the slight non-stationarity in degradation trajectories.  
Moving to the final predictive performance, Figure 10 compares the S-score, MAE, RMSE, and \(R^{2}\) for LSTM, BiLSTM, N-BEATS, CNN-GRU, and XGBoost. These lower performances despite DL approaches are possibly due to the combination of its simpler learning approach and the large number of features coming from the preprocessing stage, presented in Section 2, which limits its overall capabilities. In detail, the BiLSTM achieves consistently low MAE and RMSE, reflecting its stability and accuracy in modeling sensor-driven degradation signals. CNNGRU demonstrates resilience to outliers, as evidenced by its favorable S-score. Thus, both appear to be strong choices in this particular PdM scenario.  
Figure 10: Final FD2 performance metrics for LSTM, BiLSTM, NBEATS, CNN-GRU, and XGBoost.  
Figure 9: Aggregated FD2 loss curves for LSTM, BiLSTM, N-BEATS, and CNN-GRU, illustrating differences in convergence behaviors.  
Moving to the uncertainty scores assessment, the results regarding the calibration metrics are presented in Table 4. The conformal scores highlight that the bootstrap approach attains coverage probabilities closest to the desired (\(1-\alpha\)) while retaining moderate interval widths, indicating a practical compromise between excessive conservatism and insufficient coverage, with bootstrap showing similar reliability but producing marginally broader intervals. These findings underscore the advantages of the bootstrapping conformal strategy in balancing interval tightness with robust coverage, which refines the naive's ability to adapt to a noisy environment.  
The further analysis conducted analyzing the CP metrics on the validation set, presented in Figure 11, shows a coverage distribution that spans from near 0.0 to 1.0, with a moderate median value that is particularly noteworthy given a uniform margin selection approach yielding moderate overall performance. Similarly, the EoL Proximity distribution also presents a moderate alignment between predicted and actual EoL indices, with a subset showing considerable deviations collectively showing that even without strict unit-wise calibration, the model attains reasonably moderate metrics.