# MTGNet: Multi-Agent End-to-End Motion Trajectory Prediction with Multimodal Panoramic Dynamic Graph
## 1 Introduction
Firstly, a panoramic dynamic traffic graph is constructed based on multi-dimensional state information such as vehicle position and speed, and the high-interaction regions and sparse-interaction regions are dynamically divided. This dynamic scene division strategy can automatically adjust the scene size according to the traffic density. In high-interaction regions, the perception radius is reduced to decrease redundant calculations, and in sparse-interaction regions, the scope is expanded to capture potential long-distance interactions. The model adopts an encoder-decoder architecture to achieve  
Figure 1: Illustration of dynamic interaction. end-to-end prediction output. The encoder is responsible for handling the features of the multimodal traffic graph, and the decoder directly generates trajectory predictions. The model employs a Transformer-based encoder to perform temporal alignment and noise filtering on the multimodal data. Compared with using time models like LSTM [3,10] to process temporal data, the self-attention mechanism of the Transformer can effectively capture the long-term dependencies in drivers' behaviors, filter out the noise of atypical operations, and retain the features of basic driving behaviors. We uniformly process the time offsets of different targets. Subsequently, the positions are obtained, respectively, through a linear transformation and by adding a position encoder (pos_encoder). For local interaction modeling, MTG introduces an encoder based on a Graph Convolutional Network (GCN) to process the data after the noise reduction treatment by the Transformer encoder. Compared with the method of directly using fully connected layers [4], MTG uses a GCN in high-interaction regions to capture the local features among agents. The features of neighboring nodes are aggregated through the message-passing mechanism, and the key interaction relationships are dynamically focused on by combining the attention weights. MTG reduces the computational overhead in sparse-interaction regions while maintaining high-precision modeling in high-interaction regions. At the same time, in order to ensure the acquisition of the offset position representation after position encoding as accurately as possible, we use residual superposition to fuse it. The main contributions of this paper are as follows:  
* A Multimodal Transformer Graph Convolutional Neural Network (MTGNet) is proposed. This network dynamically constructs an agent-centric scene based on vectorization. While capturing global features, it utilizes graph convolutional neural networks to focus on local features and employs a Res_Decoder layer to decode the constructed panoramic vector, which better confirms the established panoramic vehicle connection network;
* A panoramic agent-based fully connected dynamic traffic graph is proposed to solve the resource waste problem caused by building a fixed scene size based on agent-centric methods [4,19,20], and to dynamically divide the traffic scene size. By updating the node features at the current time step through the message-passing mechanism, the changes in the relationships between each agent and other agents at the current time step can be captured, thereby better adapting to the changes in the dynamic environment and local features;
* We propose using a Transformer-based encoder to capture the long-range dependencies among features and filter out noise by leveraging its self-attention mechanism. This approach circumvents the drawbacks of directly using the direct distance between agents as a weight factor to construct the adjacency matrix. It not only reduces the computational overhead of the model but also makes the adjacency relationships of nodes in the panoramic agent fully connected dynamic traffic graph more consistent with the interaction relationships among agents in real traffic scenarios;
* The Argoverse 2.0 dataset [24] was used for testing. Argoverse 2.0 is a fully upgraded version of Argoverse 1.0. It has richer features than Argoverse 1.0.