# Predictive Maintenance Based on Identity Resolution and Transformers in IIoT
## 4 Proposed Framework for Predictive Maintenance
### Sequence Prediction Model  
The proposed SPM, as shown in Figure 3, comprises four fundamental modules: Inputs, Encoder, Decoder, and Outputs. The Inputs module primarily involves signal embedding. The Encoder and Decoder modules contain similar components in the SAM. The Outputs module includes a Linear Layer. Each of these components will be detailed in the following section. **(1) Signal Embedding in SPM**  
The signal embedding in the SPM consists of two distinct phases, namely the formulation of sequence pairs and the segmentation of segments. Typically, the task of sequence production emphasizes the production of similar sequences using randomization, without necessarily having any connection among the sequences of inputs and outputs. Conversely, when making predictions about sequences, previous time series signals play a crucial role in predicting future segments, creating an inherent correlation between these sequences. Both the encoder and decoder require two different sequences as the input, and the input to decoder \(I_{enc}\) is largely dependent on the results relayed by encoder \(I_{dec}\). The authors therefore refer to the \([I_{enc},I_{dec}]\) combination of sequences as a sequence pair. In order to build sequence pairs, the primary dataset needed to be first divided into unique and equivalent segments of different sequence lengths. This was performed to ensure that each sequence had an equal chance of being paired with any other sequence in the dataset. By having these distinct segments, the authors were able to create a robust set of sequence pairs that were representative of the entire dataset. The authors then proceeded to analyze these pairs in order to gain insights into their properties and relationships. Overall, our approach allowed us to generate a comprehensive set of sequence pairs that could be used for various applications such as machine learning, data analysis, and more. The data sequence is formulated as Equation (11). \[\text{Data}_{\text{seq}}=\{seq_{1},seq_{2},...,seq_{n-1},seq_{n}\} \tag{11}\]  
The authors then shift and truncate the corresponding data fragments in the primary dataset to obtain \(I_{enc}\) and \(I_{dec}\), as shown in Equations (12) and (13). \[I_{enc} =\text{Data}_{\text{seq}}[1:(n-1)] \tag{12}\] \[=\{seq_{1},seq_{2},...,seq_{n-1}\}\] \[I_{dec} =\text{Data}_{\text{seq}}[2:n]\] \[=\{seq_{2},seq_{3},...,seq_{n}\} \tag{13}\]  
Figure 3: The architecture of the sequence prediction model. Ultimately, to achieve congruity between two inputs, it is possible to acquire the amalgamated input of the sequence prediction model. This resultant input can be expressed as Equation (14). \[I_{spm}=\{(seq_{1},seq_{2}),(seq_{2},seq_{3})...,(seq_{n-1},seq_{n})\} \tag{14}\]  
where \(I_{spm}\) represents the input formulation of the sequence prediction model. The index of the sequence in the formula is denoted by \(n\). The authors have already introduced the operation for segmentation in Section 4.1--see the relevant contents in the signal embedding in the SAM. **(2) Encoder to Decoder**  
Transformer architecture has been widely used for sequence modeling, natural language processing, and machine translation. In the sequence-to-sequence architecture, it primarily utilizes a series of self-attention blocks and layers that are fully connected to compose the encoder part and decoder part, and it operates in parallel. This design enables a more efficient computation in handling sequential data, providing an effective resolution on the processing and analysis of sequences. Both the encoder part and decoder part are made up of \(N\) indistinguishable blocks, which are illustrated in Figure 3, where \(N=3\). This kind of architecture serves to enhance the learning capacity of the model. However, it is noteworthy that the decoder block, unlike the encoder structure, integrates a masked multi-head attention sub-layer to safeguard position subsequent segment during prediction.