# Enconv1d Model Based on Pseudolite System for Long-Tunnel Positioning
## 3 Methodology
### Model Framework
#### 3.2.1 Temporal Feature Extraction Module  
The correlation of pseudorange measurements across consecutive time steps encompasses both pseudorange data and positional data. Pseudorange measurement correlation arises from the continuous variation in geometric distance and clock bias between the receiver and base station over time, resulting in the pseudorange measurements exhibiting a certain level of correlation over a continuous period. On the other hand, the positioning results of the receiver over consecutive time intervals are related to the motion state of the vehicle, meaning that at any given time, the receiver's position is correlated with its previous and subsequent positions. However, the correlation between single-differenced pseudoranges over consecutive periods may lead to the model getting trapped in a local optimum during training. This occurs because the model repeatedly learns from similar data, affecting its generalization ability and slowing down its convergence speed. The Transformer effectively captures long-term dependencies within sequences using its self-attention mechanism, while the introduction of the multi-head attention mechanism further enhances its capability by enabling the parallel processing of multiple subspaces, thus allowing for the extraction of more profound and more complex mappings and improving the representational power of the model. Therefore, we choose to utilize the encoder structure of the Transformer and its self-attention mechanism as the time feature extraction module for single-differenced pseudoranges. The goal is to leverage the self-attention mechanism and encoder structure to focus on critical time steps and the long-term dependencies between time steps, thereby improving positioning accuracy from both local and global perspectives. The temporal feature extraction module consists of an input, encoder, and output layer (the gray section in Figure 2). The input layer generates input matrix \(X\) by sliding input vector \(\Delta\rho\) through a sliding window and slicing it according to time step T. \(X=\{\Delta\rho(t_{1}),\Delta\rho(t_{2}),\ldots,\Delta\rho(t_{T})\}\), \(\Delta\rho(t)=\{\Delta\rho^{(12)}(t),\Delta\rho^{(13)}(t),\ldots,\Delta\rho^{ (ij)}(t),\ldots,\Delta\rho^{(n(n-1))}(t)\}\), \(\Delta\rho(t)\in\mathbb{R}^{m}\), and \(X\in\mathbb{R}^{T\times m}\). The encoder layer includes multi-head attention layers and feed-forward network layers. The self-attention mechanism is the core component of the encoder, utilizing the scaled dot product attention mechanism to compute the output mapping between the query vector and a set of key-value input pairs, thus modeling long-term dependencies within the data. For input vector \(X\), the core of the scaled dot product attention mechanism lies in learning the weight matrices to compute Q, K, and V. The dot product of Q and K is divided \(\sqrt{d_{k}}\) to obtain the score function. The score values are then normalized using the softmax function, producing weight coefficients ranging from 0 to 1 with a total sum of 1. These weights are subsequently multiplied with V to compute the weighted sum. The output matrix is represented as follows:  
\[\left\{ \begin{array}{l}Q=XW^{Q}\\ K=XW^{K}\\ V=XW^{V}\end{array} \right. \tag{5}\]  
\[Attention(Q,K,V)=\ softmax\bigg{(}\frac{QK^{T}}{\sqrt{d_{k}}}\bigg{)}V \tag{6}\]  
where \(X\in\mathbb{R}^{T\times m}\), \(W^{Q}\in\mathbb{R}^{m\times d_{k}}\), \(W^{K}\in\mathbb{R}^{m\times d_{k}}\), and \(W^{V}\in\mathbb{R}^{m\times d_{V}}\). \(d_{k}\), \(d_{k}\), and \(d_{v}\) represent the dimensions of the query, key, and value, respectively. Multi-head attention (Figure 3) involves learning multiple sets of distinct weight matrices \(W^{Q},W^{K},W^{V}\) for the same input vector \(X\), which are then used to compute different Q, K, and V representations for each attention head. Each head independently calculates the \(Attention(Q,K,V)\), and the outputs from all heads are concatenated and projected to generate the final output matrix.