## 4 Model Building
### Main Model for Trajectory Prediction  
The main model structure for trajectory prediction is shown in Figure 4. The model consists of four parts. Firstly, during training, the vehicle trajectory data are divided into historical trajectory and endpoint. And they are input to two encoders, respectively, E\({}_{\text{past}}\) and E\({}_{\text{end}}\), to generate corresponding embedding representations. Secondly, the map road  
Figure 3: Overall structure of the framework of the trajectory prediction model. data are handed over to the VGAE sub-model for specialized processing and obtains the embedding representation of the road. Thirdly, the three feature vectors obtained are concatenated and used as inputs to the endpoint prediction module. The encoder Elatent of the CVAE model learns the distribution of the driving endpoint latent space, and the most likely trajectory endpoint is generated by the decoder Dlatent. Effective improvements are made to the encoder structure of Elatent and Dlatent, including the addition of multi-head self-attention mechanisms. During the prediction phase, the generated trajectory endpoint is input to Eend to generate the embedding representation corresponding to the predicted endpoint. Finally, outputs of the third part, together with the vehicle relationship matrix and the embedding representation of past trajectories, are input to the Social Pooling layer structure of PECNet for learning the vehicle relationships. Finally, the Predictor layer generates the complete predicted trajectory. This paper has made several significant modifications to the endpoint prediction module. Firstly, the tensors of all encoders and decoders have been enhanced with an additional \"seq_len\" dimension. This enhancement helps the model learn relationships between different sequence frames and also caters to the data requirements of the multi-head self-attention mechanism. Secondly, during training, the environment feature tensors processed by VGAE are integrated. Finally, after the decoder processing, a fully connected layer is added to reduce the dimensionality of the data and obtain the final predicted destination coordinates. The complete details of the training process are shown in Figure 5.  
Figure 4: The structure of the main model for trajectory prediction. The original encoder and decoder of PECNet were both MLP models. However, the number and duration of targets in expressway scenes have significantly increased compared to the pedestrian scenes, and the MLP alone may not be able to achieve good prediction accuracy. For this reason, the coding and decoding structure in this topic is completely revamped to include the multi-head self-attention mechanism of Transformer. The original MLP structure is used as the final part of the new autoencoder to obtain the ultimate output. Three additional layers are added before it, including the positional embedding layer, the multi-head self-attention layer, and the dropout layer. The encoder-decoder structures before and after the modification are shown in Figure 6.  
Figure 5: Training details of the improved endpoint prediction module. Figure 6: Structural details of the improved endpoint prediction module. The position encoding layer adds the position vectors computed for each position in the input sequence to the feature vectors at the same position to better capture the dependencies between positions in the sequence. The calculation equation is as follows:  
\[Position_{k}^{i}= \begin{cases}\begin{array}{c}sin\frac{i}{10000^{\frac{M}{d}} },\;\;k=2n,\;n\in\mathbb{N}\\ cos\frac{i}{10000^{\frac{M}{d}}},\;\;k=2n+1,\;n\in\mathbb{N}\end{array},\;i \in[0,N)\end{cases}  \tag{7}\]  
In Equation (7), \(i\) represents the position, \(k\) represents the index of the feature dimension, and \(d\) represents the dimension. \(N\) represents the length of the feature vector. The position encoding layer calculates position embedding vectors using trigonometric functions and adds them to the feature vectors to encode positional information.