# LIransformer: A Transformer-Based Framework for Task Offloading in Vehicular Edge Computing
## 3 Method
### Optimization Process 1: Generate Weight Coefficients Based on Local Linear Features
#### 3.2.2 Input Module  
The input module consists of three components: location embedding, position encoding, and temporal encoding, which encode geographic, sequential, and temporal information, respectively. The input datum of this module is a set consisting of a multiple vehicle trajectory set, \(T\). We use a matrix, \(M\), to represent the input data, i.e.,  
\[M= \begin{bmatrix}s_{11}&\cdots&s_{1q}\\ \vdots&\ddots&\vdots\\ s_{p1}&\cdots&s_{pq}\end{bmatrix}  \tag{7}\]  
where \(M\in\ \mathbb{R}^{p\times q\times d_{in}}\), \(p\) denotes the quantity of trajectories (batch size) at each training epoch, \(q\) denotes the length of the longest trajectory, and \(d_{in}\) denotes the dimension of each node. In the input module, the matrix \(M\) can be divided into two parts according to \(d_{in}\), the matrix \(M_{l}\) (\(d_{in}=1\)), containing serial information, and the matrix \(M_{s}\) (\(d_{in}=1\)), containing temporal information. The input of location embedding process is the matrix \(M_{l}\). It is generated by the results of pre-training process. Specifically, two embedded matrices, \(M_{l}\) and \(M_{g}\), are generated by replacing the serial numbers in \(M_{l}\) with the corresponding embedded vectors in the trainable embedded matrix \(W_{t}\) and the geographic embedded matrix \(W_{g}\), where \(M_{t}\in\mathbb{R}^{p\times q\times 512}\), \(M_{g}\in\ \mathbb{R}^{p\times q\times 512}\). Finally, the matrix \(M_{t}\) and the matrix \(M_{g}\) are summed by weights to obtain the matrix \(M_{L}\), which represents the location information. \[M_{L}=\theta_{1}M_{t}+\theta_{2}M_{g} \tag{8}\]where \(\theta_{1}\) and \(\theta_{2}\) are the weights of the matrix \(M_{t}\) and the matrix \(M_{g}\). The weights will be updated in the backpropagation process. The positional information determines the order of the nodes in the trajectory, so position encoding is critical. The LTransformer follows the Transformer's position encoding method; i.e., position information is represented by trigonometric functions. The encoding results are computed for each position and dimension, i.e.,  
\[PE(t,i)= \begin{cases}\sin\Bigl{(}\frac{t}{10000^{(i-1)/d}}\Bigr{)},\ \ if\ i\ is\ even\\ \cos\Bigl{(}\frac{t}{10000^{(i-1)/d}}\Bigr{)},\ \ if\ i\ is\ odd\end{cases}  \tag{9}\]  
where \(t\) denotes the position number, \(i\) denotes the dimension, and \(d\) denotes the dimension after embedding. For each element in \(M\) and each of its dimensions, \(M_{P}\) is calculated according to Equation (7). The calculation is shown in Equation (8). \[M_{P}= \begin{bmatrix}PE(1,i)&\cdots&PE(q,i)\\ \vdots&\ddots&\vdots\\ PE(1,i)&\cdots&PE(q,i)\end{bmatrix} ,\ i=1,2,3,\ldots 512 \tag{10}\]  
The result of position encoding is the matrix \(M_{P}\), representing the sequential information. In VEC, there is a temporal pattern in the vehicle trajectories. For example, traffic volumes and overall direction of movement in the morning are different from those in the evening. Therefore, the LTransformer framework uses the time encoding in Informer, which is able to express the temporal pattern. First, the LTransformer selects five aspects of temporal pattern expression based on the training results, which are hour, day, week, month, and year. Then, we calculate each expression of the timestamp in the matrix \(M_{s}\). The calculation of each aspect has its own characteristic; e.g., in the hour aspect, we calculate the proportionality of the current minute in an hour. The specific formulation of each aspect is as follows:  
\[ \begin{array}{l}h=minute/60\\ d=hour/24\\ w=day/7\\ m=day/30\\ y=day/365\end{array}  \tag{11}\]  
After the calculation, the results are concatenated and embedded into a 512-dimensional space using a linear layer to generate \(M_{T}\), where \(M_{T}\in\ \mathbb{R}^{p\times q\times 512}\). The matrix \(M_{T}\) represents the temporal information.