# MTGNet: Multi-Agent End-to-End Motion Trajectory Prediction with Multimodal Panoramic Dynamic Graph
## 1 Introduction
Furthermore, [PERSON] et al. [10] improved upon the work of [PERSON] et al. [12] by enabling the model to output a continuous, multimodal probability distribution for predicting the vehicle's location range within the next 5 s. However, this approach has a critical limitation: it assumes that traffic scenes conform to grid structures to capture temporal dependencies, while the spatial relationships between agents are non-Euclidean [16]. This makes it challenging for LSTM to handle complex spatial relationships in dynamic traffic scenarios. To address these challenges, some studies have proposed using vectorization [17; 18] methods to model traffic scenarios. However, it is difficult to perform real-time predictions in traffic environments with high-speed intelligent agent flows. To address the issue that vectorization methods are usually not robust to translation and rotation of the reference frame, recent work has normalized the scene to an agent-centric coordinate system for reasoning [4; 19; 20], thereby maintaining rotation invariance relative to the agent's global position and orientation [4]. When there are a large number of agents in the scene, re-standardizing the scene and calculating the characteristics of each agent incur extremely high computational overhead. The method proposed in [4] exploits the symmetry and hierarchical structure in the multi-agent motion prediction problem, divides the motion prediction task into multiple stages, and models these relationships hierarchically. It uses a Transformer to capture dependencies in long time series. However, when processing traffic scenes, this method divides the scene using a fixed size, which can easily result in unnecessary resource waste. Additionally, in the study of trajectory prediction based on graphs, some scholars have proposed using a GCN (graph convolutional neural network) [21; 22] to model interactions between multiple agents. However, in these initial attempts, the connection intensity is regarded as a function determined by the distance between nodes in the graph [23]. Most studies only assign edge weights based on the distance between nodes. Moreover, the strength of interactions between agents also changes dynamically at different time steps. As shown in Figure 1, at Time Step 1, Agent 2 has a stronger influence on Agent 3 compared to Agent 1. In terms of distance, the distance between Agent 2 and Agent 3 is greater than that between Agent 1 and Agent 3. However, at Time Step 2, the distances between Agent 1 and Agent 3 and between Agent 2 and Agent 3 become almost identical. Notably, the influence of Agent 1 on Agent 3 increases significantly at this stage. Therefore, defining edge weights solely based on distance is clearly inconsistent with real-world scenarios [16]. In addition, within the same traffic scenario, there are significant differences in the degree of influence exerted by agents in the left and right lanes, as well as those ahead and behind in the same lane, on the trajectory of the target agent. Compared to the agent ahead in the same lane, the influence of the agent behind is significantly smaller. In real-world traffic scenarios, this method is clearly unsuitable for multi-agent future trajectory prediction and will also introduce noise interference [16]. At the same time, ref. [23] proposed an attention-based spatiotemporal graph convolutional network to construct a hierarchical structure consisting of spatial GCN and temporal GCN to address this issue. However, it employs GRU (gated recurrent unit) to predict the future trajectory of the agent. When learning the long-term dependencies of the agent's historical trajectory, GRU is prone to gradient vanishing or gradient exploding problems, making it difficult for the model to accurately capture key information over long time spans in the historical trajectory. To address these issues, we propose Multimodal Transformer Graph Convolutional Neural Network (MTGNet) for multimodal trajectory prediction in complex urban traffic scenarios.