## 3 Methods
### Overview
```
Input: Input sample \(x_{in}\) Output: Output value \(x_{out}\)
1 for\(i=1\) to epochdo
2 for\(j=1\) to batchdo
3\(x_{0}^{j}=\text{Embedding}(\text{UMAP}(x_{in}^{j}))\);
4 for\(k=1\) to \(K\)do
5\(x_{k}^{j}=\text{Time-Block}(x_{k-1}^{j})\);
6\(x_{k}^{j}=\text{Variable-Block}(x_{k}^{j})\);
7
8 end for
9 Update model parameters;
10
11 end for
```

**Algorithm 2**Training algorithm

In Algorithm 1, the term epoch refers to the number of iterations of model training. In each epoch, the data are divided into several equal-sized batches, and the data in each batch are used to compute the loss value and update the model parameters once using an optimizer. Here, \(K\) represents the number of Time Blocks and Variable Blocks in STformer. During the testing phase, the process of anomaly detection algorithm is as shown in Algorithm 2. In Algorithm 2, \(x_{train}\) represents training data, \(x_{val}\) represents validation data, and \(x_{test}\) represents test data. Initially, the training data are fed into the model to obtain model-generated training data. Subsequently, the error between the training data and the generated data is computed. Following this, the same process is repeated with validation data to calculate the error between the validation data and the generated data. Next, the errors of the training and validation data are proportionally divided, with a threshold (thresh) set as the boundary between the top 1% and the remaining 99% of error values. Finally, the error of the test data is calculated and classified according to Equation (7): data with errors greater than the threshold (thresh) are classified as abnormal, while data with errors less than or equal to the threshold are classified as normal. Figure 2: The architecture of STformer primarily includes the UMAP module, Time Block, and Variable Block. \[ \begin{split}\|pred_{\text{test}}-X_{\text{test}}\|_{F}^{2}>\textit{ thresh}\\ \|pred_{\text{test}}-X_{\text{test}}\|_{F}^{2}\leq\textit{thresh}\\ \end{split}  \tag{7}\]

```
Input: Training data \(x_{train}\), validation data \(x_{val}\), testing data \(x_{test}\). for\(i=1\) to batchdo \(pred_{train}=\text{STformer}(x_{train})\); \(loss_{train}=pred_{train}-x_{train}\);
1 end for for\(i\)=1 to batchdo \(pred_{val}=\text{STformer}(x_{val})\); \(loss_{val}=pred_{val}-x_{val}\);
2 end for
3 thresh = np.percentile(\(loss_{train}\), \(loss_{val}\), 99); for\(i=1\) to batchdo \(pred_{test}=\text{STformer}(x_{test})\); \(loss_{test}=pred_{test}-x_{test}\); if\(loss_{test}>\textit{thresh}\)then
4 Anomalous
5 else
6 Normal
7
8 end for
```

**Algorithm 2**Test algorithm

### Preliminaries

Attention mechanisms mimic how humans allocate attentional resources, focusing on key areas while reducing attention to other areas, in order to extract more relevant information and filter out distracting information. Single-headed attention mechanisms tend to focus on a single point, limiting their ability to capture multiple important fragments of information in the input and process diverse contextual information simultaneously. In contrast, multi-headed attention mechanisms concentrate attention across different dimensions, enabling them to capture more comprehensive and nuanced feature representations. In this mechanism, each attention head acts like a feature detector, concentrating on capturing different local features [29]. In our study, we adopted the scaled dot-product attention mechanism. This mechanism uses a dot-product function to map queries (Q), keys (K), and values (V) to outputs, where Q, K, V, and the output are all represented in matrix form. Specifically, this mapping mechanism can be described by the following Formula (8):

\[ \begin{split} Q&=X_{in}W_{Q}\\ K&=X_{in}W_{K}\\ V&=X_{in}W_{V}\\ \end{split}