# Predictive Maintenance Based on Identity Resolution and Transformers in IIoT
## 5 Evaluation
### Experimental Settings  
An instantiation of ITPM is presented in this section. Then, the authors evaluate the performance of the state-aware and sequence prediction models on segmented datasets. The datasets are divided into certain sequence lengths to construct sample datasets for model training, validation, and testing purposes. This paper ensured the randomization of the samples by shuffling the entire sample datasets and creating corresponding subsets by random sampling in a ratio of 7:2:1. This allocation uses 70%, 20%, and 10% of the sampling for training, validation, and testing of learning capabilities, respectively. The authors trained the state awareness and sequence prediction models with 300 epochs on the training set, assessing their learning effects after each epoch. Once the models were well trained, the authors evaluated their performances on the testing sets. The ITPM framework comprises two parts, as shown in Figure 1. Therefore, two-part experiments were set up to validate the efficiency of ITPM. Firstly, the authors assessed the state awareness capability of the SAM on the CWRU dataset. In Figure 2, the authors illustrate that the SAM takes a set length of sample vibration signal data as the input and sets a value to represent the corresponding equipment condition, e.g., 0, 1, 2, etc. The main function of the SAM is to identify the corresponding real-time equipment condition based on the signal trend in the data. Furthermore, to test the ability of the SAM to recognize sensor sequences of different lengths, the authors partitioned the dataset into five different lengths, i.e., [256; 512; 1024; 4096], which were used to train the state awareness model, leading to six well-trained state awareness models. Finally, to assess the model performances comprehensively, the authors utilized the precision, accuracy, recall, F1-score, receiver operating characteristic (ROC) curve and area under the curve (AUC) values of the prediction results as evaluative indicators. Formulas for calculating these metrics are provided as follows. \[\text{Accuracy}=\frac{TP+TN}{TP+TN+FP+FN} \tag{16}\]  
\[\text{Precision}=\frac{TP}{TP+FP} \tag{17}\]  
\[\text{Recall}=\frac{TP}{TP+FN} \tag{18}\]  
\[F1\text{-score}=\frac{2\cdot\text{Precision}\cdot\text{Recall}}{\text{ Precision}+\text{Recall}} \tag{19}\]  
\[\text{AUC}=\frac{1}{2}\sum_{i=1}^{m-1}{(x_{i+1}-x_{i})(y_{i}+y_{i+1})} \tag{20}\]  
where \(\{(x_{1},y_{1}),(x_{2},y_{2}),...,(x_{m},y_{m})\}\) indicates the sequentially connected points of the ROC curve with \(x_{1}=0\), \(x_{m}=1\). Additionally, costs of time, including training time and testing time, are taken into account. Regarding classification tasks, the computation of associated metrics typically relies on positive and negative sample classification outcomes that would be presented as a confusion matrix comprising four different components. These encompass True Positives (TPs), False Positives (FPs), True Negatives (TNs), and False Negatives (FNs). Secondly, the authors present an evaluation of the performance of sequence prediction on the CWRU dataset. As previously mentioned, the SPM is specifically proposed to predict sequence-to-sequence mapping, where the input sequence and output sequence are of equal length. As shown in Figure 3, where the input to the model consists of a sample pair, a sample from the current sampling period (on the left) and a sample from the subsequent sampling period (on the right), which is arranged sequentially. During the training stage, both these samples are simultaneously fed into the model, allowing it to learn the mapping relationship between them. In the testing phase, only the current samples are fed for the model, which is expected to forecast the next/future sequence, given the correlationbetween them. To test the ability of the SPM to predict sensor sequences of different lengths, the authors partitioned the dataset into five different lengths [256; 512; 1024; 4096; 256].