## 3 Methodology and Materials
### Applying with Attention Mechanisms
#### 3.2.3 Autoencoder Preprocess with Cycling Layer  
Within the autoencoder's encoding phase, we have innovatively integrated the cycling layer and self-attention layer. This integration is critical as it allows the encoder to reduce the data dimensionality and recognize and encode the cyclical patterns inherent to time series datasets. In Figure 2, the data after cleaning and sequence creation will be moved to the autoencoder process. The dataset with 14 \(\times\) 16 (original attributes with timestamp and order) dimensions allows these models to efficiently learn and compress data into lower-dimensional spaces, preserving the crucial sequential information that would be lost in flattened 2D representations. The encoding process commences may encompass various time series variables recorded over time. Initially, the data are routed through the cycling layer, which incorporates cyclical temporal information into the data representation. Subsequently, the enriched data are processed by the self-attention layer, which enables the network to emphasize information based on the cyclical context introduced by the preceding layer. As the data advance through the subsequent layers of the encoder, they are progressively compressed into a latent space--the core of the autoencoder. This latent space embodies the most salient features of the input data in a condensed format. At this bottleneck, the data, now represented as a compressed version of the original dataset, encapsulate the most crucial patterns and trends discerned by the network. The decoding process mirrors the encoding process. The compressed representation obtained from the latent space is fed into the decoder. The decoder aims to reconstruct the original high-dimensional data from this compressed representation. This reconstruction occurs through a sequence of layers that progressively increase in dimensionality. In the initial stages, the decoder aims to re-expand the encoded features toward a format that approximates the original input data. Subsequent layers further refine this reconstruction by adjusting the output according to a loss function, which quantifies the discrepancy between the original data and the reconstructed data. The final output layer of the decoder is designed to align with the dimensions of the original input data, yielding the reconstructed data. These reconstructed data are then compared against the original dataset to evaluate the accuracy of the reconstruction and the efficacy of the encoding process. In our experiment, the entire autoencoder process, encompassing both encoding and decoding, undergoes optimization and training. A loss function, such as the Mean Squared Error for continuous data, is employed to guide the adjustment of network weights to minimize reconstruction error. This process ensures that the autoencoder learns the most effective input data representation. The autoencoder refines its parameters through iterative training until the reconstruction loss is minimized, indicating that the network has acquired a robust representation of the time series data. The proposed autoencoder preprocessing method excels at extracting essential features from complex datasets, allowing models to concentrate on the most significant elements. Through Table 1, the cycling layer demonstrates its value when compared to traditional self-attention in an autoencoder. The comparison in the table is grounded in the inherent design strengths and weaknesses of positional and cycling self-attention layers based on the analysis and summary in the methodology section. Based on the methodology analysis, positional self-attention captures short-term dependencies well but struggles with long-term cyclical patterns, while cycling self-attention is designed to handle periodic structures, excelling at capturing long-term dependencies. Positional self-attention treats each time step rigidly, which can make it less efficient at focusing on key cyclical moments, whereas cycling self-attention prioritizes relevant cyclical events.