# Magnetic Localization Method for Vehicles Based on Transformer
## 3 Methods
### Transformer Model Structure
#### 3.2.1 Encoder  
The structure of the encoder is shown in Figure 5a. First, shallow magnetic features are extracted from the given magnetic sequence by passing through a convolutional layer and a positional embedding layer [30]. Thus, the dimensionality of the original magnetic sequence is expanded from 3 to \(D\). Then, the shallow features are fed into \(N_{1}\) Transformer encoder layers and become deep features with shapes of \([B,W,D]\). Each Transformer encoder layer possesses the same structure as that of the standard Transformer [30], which is shown in Figure 5b, including one self-attention layer, one FeedForwarD (FFD) layer, two layer normalization operations, and several residual connections.  
**Self-Attention Layer:** The self-attention layer is the core component of the encoder layer; it enables the encoder to extract useful temporal information. The self-attention layer is based on a multihead attention mechanism, which is an advanced version of the attention mechanism formulated in Equation (2):  
Figure 4: The structure of the proposed Transformer model.  
Figure 5: The structures of the (**a**) encoder and (**b**) encoder layers.  
\[\text{Attention}(Q,K,V)=\text{softmax}(\frac{QK^{T}}{\sqrt{d_{k}}})V \tag{2}\]  
where \(Q\in R^{Q_{t}\times d_{k}}\), \(K\in R^{K_{t}\times d_{k}}\), and \(V\in R^{V_{t}\times d_{k}}\) are the query, key, and value matrices, respectively. \(Q_{t}\), \(K_{t}\), and \(V_{t}\) are the numbers of queries, keys, and values, respectively, and \(d_{k}\) is the second dimension. \(Q\), \(K\), and \(V\) can be easily computed by a combination of linear transformation and concatenation for every timestamp.  
The multihead attention mechanism can be expressed as shown in Equation (3):  
\[\text{MultiHead}(Q,K,V)=\text{concat}(\text{Attention}(Q_{t},K_{t},V_{i}))W ^{O}_{D\times D} \tag{3}\]  
where \(Q_{i}=QW^{Q}_{D\times(D/h)}\), \(K_{i}=KW^{K}_{D\times(D/h)}\), and \(V_{i}=VW^{V}_{D\times(D/h)}\). \(W^{Q}_{D\times(D/h)}\), \(W^{K}_{D\times(D/h)}\), and \(W^{V}_{D\times(D/h)}\) represent the weight matrices for \(Q\), \(K\), and \(V\), respectively, with shapes of \(D\times(D/h)\), where \(h\) is the number of heads, and concat(*) is the concatenation operation. \(W^{O}_{D\times D}\) is the final output matrix with a shape of \(D\times D\).  
Therefore, the self-attention mechanism can be written as shown in Equation (4):  
\[\begin{cases}\text{SelfAtt}(X)=\text{MultiHead}(Q,K,V)\\ Q=\text{concat}(W^{Q}X_{i})\qquad i\in[1,W]\\ K=\text{concat}(W^{K}X_{i})\qquad i\in[1,W]\\ V=\text{concat}(W^{V}X_{i})\qquad i\in[1,W]\\ \end{cases} \tag{4}\]  
where \(X\) is the input of the self-attention layer, which can be viewed as the magnetic features extracted from the given sequence, and \(X_{i}\) is the \(i\)-th magnetic feature in the sequence.  
**FFD Layer**: Among the above-described layers, only the FFD layer is time-independent, which means that \(M_{i}\), the \(i\)-th feature, is determined only by the \(i\)-th input. The other layers evaluate all the inputs of the entire sequence to compute the layer outputs.  
**Layer Normalization**: Layer normalization is a common type of normalization operation used in deep learning [31]; it is applied after implementing a simple addition operation involving the current data stream and the data stream connected with a residual connection.