## 3 Materials and Methods
### Architecture  
The Transformer model has been successfully applied in various domains such as time series prediction and classification. With the help of the Transformer model, significant breakthroughs have been achieved in weather and fault prediction. In this study, we propose a risk prediction model for elevators based on a Transformer and a temporal compression mechanism. This method leverages the powerful processing capabilities of the Transformer model and combines them with elevator data to effectively analyze and predict the maintenance needs of elevators, providing scientific and rational data support for elevator maintenance. This approach enables a transition from reactive maintenance to predictive maintenance, greatly improving the safety and reliability of elevator operations. This study is based on time series analysis of historical IoT elevator data, which were collected by IoT devices and include various features such as temperature, number of operations, installation time, and elevator type. We generated features in chronological order and added labels indicating the occurrence of faults. The historical data were divided into dense features, sparse features, and static features. Sparse features, such as the number of operations, were represented using one-hot encoding, while dense features, such as temperature, were represented as vectors. Static features, such as the elevator type, do not have temporal relationships. Our Risk Prediction Model for Elevators based on a Transformer and Self-Temporal Compression Mechanism (RPM-ETC) consists of four layers: the Data Encoding Layer, Feature Fusion Layer, Feature Extraction Layer, and Prediction Layer, as illustrated in Figure 1.  
In the Data Encoding Layer, the model converts sparse discrete features into dense feature vectors. For the sparse and static features, we utilized an embedding layer for mapping. The embedding layer represents one-hot encoded features as dense vectors, effectively reducing the dimensionality of the original data and better capturing relationships and patterns within the data. The embedding layer takes one-hot encoding as input. When processing discrete elevator data, we first constructed a feature matrix, which records the latent feature representations corresponding to each value of every attribute field. Then, by multiplying the one-hot encoded discrete data with the feature matrix, the corresponding dense feature vectors for the discrete data were obtained. In the Feature Fusion Layer, the model first extracts the associative information of encoded sparse and static features through convolutional layers. Subsequently, linear layers are employed to map the extracted features to the same dimensionality and feature space as dense features, ensuring unified processing of different feature types. These mapped sparse, static, and dense features are then concatenated to form an initial feature combination. Next, the model applies the partial autocorrelation function (PACF) to compute correlations among the concatenated features. The correlation information obtained is then fused with the original features to generate temporal features. Finally, the system organizes the mapped sparse, static, and dense features in chronological order to form the complete feature input. This approach not only ensures consistent processing of different feature types but also achieves comprehensive analysis of feature relationships, thereby significantly enhancing the model's accuracy and reliability in elevator fault prediction tasks. In the Feature Extraction Layer, features are extracted using a Transformer module equipped with the Self-Temporal Compression Mechanism and Enhanced Positional Encoding. Finally, in the prediction layer, the model utilizes an MLP (multilayer perceptron) composed of two fully connected layers and the Sigmoid function to predict elevator faults.