## 2 Materials and Methods
### Transformer Encoder  
Transformers were introduced by [PERSON] et al. in 2017 [45] as an attention-based architecture to process sequential data. Unlike recurrent models such as long short-term memory (LSTM), transformers enable parallel computation and capture long-range dependencies using self-attention mechanisms [45]. The model has become foundational in natural language processing, time-series modeling, and multivariate data analysis due to its scalability, effectiveness in learning contextual relationships, and fast training convergence. The transformer encoder operates by applying self-attention to model the relationship between all positions in an input sequence. This is particularly beneficial in domains such as intrusion detection, where the underlying patterns in system logs or network packets are often dependent on global temporal context. In edge and IIoT systems, sequences of sensor readings, system calls, or traffic data are typically complex, high-dimensional, and asynchronous. The encoder's capability to capture cross-temporal and cross-feature interactions makes it ideal for learning structured representations useful for detecting anomalies or cyber threats [46; 47; 48]. The internal structure of the encoder is shown in Figure 3.  
The process begins by embedding the input sequence and injecting positional encodings to preserve order information. Each encoder block comprises two main components: a multi-head self-attention layer and a position-wise feed-forward neural network (FFNN). These sub-layers are followed by residual connections and normalization. The self-attention mechanism enables the model to attend to different positions in the sequence simultaneously, while the feed-forward layer refines the intermediate representations. The stacked  
Figure 3: The transformer encoder architecture. encoder layers allow for hierarchical modeling of complex dependencies over the sequence. Figure 3 also expands the core building blocks within the encoder. The scaled dot-product attention mechanism computes similarity scores between all pairs of input positions. These scores are then normalized using a softmax function to form attention weights, which are used to compute weighted combinations of the input values. This operation is applied across multiple heads in the multi-head attention module. Each head focuses on a different subspace of the input representation, and their outputs are concatenated and linearly transformed. This flexible and parallel structure enhances the model's ability to capture diverse patterns and long-range correlations, which is critical in anomaly-rich environments such as IIoT and edge deployments [47; 48; 49]. The transformer encoder begins by converting input tokens into dense vector embeddings, followed by the addition of positional encodings to retain sequential information. Since the architecture does not inherently model sequence order, sinusoidal positional encodings are applied to inject relative and absolute position information. These encodings are calculated using sine and cosine functions of different frequencies, as shown in Equations (5) and (6) [46]:  
\[PE_{(pos,2i)}=sin\Big{(}\frac{pos}{1000^{2i/d}}\Big{)}, \tag{5}\]  
\[PE_{(pos,2i+1)}=cos\Big{(}\frac{pos}{1000^{2i/d}}\Big{)} \tag{6}\]  
Here, \(pos\) is the position index, \(i\) is the dimension index, and \(d\) is the embedding size. After positional encoding, the embedded sequence is passed through a series of attention layers. In each layer, the input vector is linearly projected into three matrices (queries, keys, and values), which are used in the attention calculation. These projections are computed as shown in Equations (7)-(9) [47]:  
\[Q=ZW^{Q}, \tag{7}\]  
\[K=ZW^{K}, \tag{8}\]  
\[V=ZW^{V} \tag{9}\]  
where \(W^{Q},~{}W^{K}\), and \(W^{V}\) are learned projection weights, and \(Z\) is the input from the previous encoder layer (or the embedding if it is the first layer).