# MTGNet: Multi-Agent End-to-End Motion Trajectory Prediction with Multimodal Panoramic Dynamic Graph
## 3 Methodology
### Motion Trajectory Encoder  
As shown in Figure 3, the Motion Trajectory Encoder is composed of Linear_Embedding, Pos_Encoder1D, and Transformer_Layer. Figure 2: Multimodal Transformer Graph Convolution Neural Network (MTGNet). When preprocessing data, although using absolute position [42] or relative position with the initial position as the coordinate origin to predict future trajectories [43] can achieve trajectory prediction within a certain period of time, we argue that using the angle of position transformation and the offset at each time step to predict the next position can provide richer information for our model compared to directly using position coordinates. In the application of position transformation, our model design still requires selecting an initial point as the benchmark for distance calculation [44]. According to the concept of this paper, the movement transformation of the first 50 time steps is used to predict the movement of the next 60 time steps. Although it is technically feasible to set the starting position of the time step as the coordinate origin, we still choose to use the 49 th time step as the coordinate origin. Such a setting not only helps the model understand the change process of each step more deeply but also provides strong guidance for subsequent predictions. Linear_Embedding is used to perform a linear transformation on the input data, followed by the ReLU [45] activation function to introduce nonlinearity and map the data to the latent space. Specifically, in terms of the model design, we first standardized the temporal offsets of different agents. We transformed the input sequence of position coordinates of agents into a vectorized sequence:  
\[f_{t}=[\Delta p_{t},\theta_{t}] \tag{1}\]  
Here, \(\Delta p_{t}\) represents the displacement vector between adjacent time steps, which captures the changes in motion speed and direction. \(\theta_{t}\) represents the heading angle, which is defined as the angle between the displacement vector and the coordinate axis:  
\[\Delta p_{t}=p_{t}-p_{t-1} \tag{2}\]  
\[\theta_{t}=arctan2(\Delta p_{t}[1],\Delta p_{t}[0]) \tag{3}\]  
Figure 3: (**a**) Structure of Motion Trajectory Encoder; (**b**) structure of Transformer_Layer. Then, we obtain the position information of the agent through a linear transformation and the addition of a position encoder (Pos_Encoder1D). At the same time, to ensure the offset position representation after position encoding as accurately as possible, we further use the residual superposition method to fuse information, thereby optimizing the effect of position representation. Among them, Pos_Encoder1D uses a fixed trigonometric function form for encoding. First, the number of channels is processed to ensure that it is an even number:  
\[\text{C}^{\prime}=[\text{C}/2]\times 2 \tag{4}\]  
Here, C is the original number of channels. \(\text{C}^{\prime}\) is the number of channels after processing. \([\cdot]\) represents rounding down. Then, the inverse frequency factor \(inv\_freq_{i}\) is calculated [32]:  
\[\text{inv\_freq}_{i}=\frac{1}{10000\frac{2i}{\text{C}^{\prime}}},i\in\{0,2, \ldots,C^{\prime}-2\} \tag{5}\]  
Here, \(i\) represents the index for calculating the inverse-frequency factor. The value-range of \(i\) starts from 0, increments with a step size of 2, and goes up to \(C^{\prime}-2\). \(\text{inv\_freq}_{i}\) represents the \(i\)-\(th\) inverse-frequency factor. \(\frac{2i}{\text{C}^{\prime}}\) is the exponent part, which determines the variation pattern of the inverse-frequency factor. The position index p is scaled by the inverse frequency, and the position encoding matrix is constructed \(PE_{p}\), The final shape is (x, C\({}^{\prime}\)):  
\[\text{sin\_}inp_{p,i}=p*inv\_freq_{i} \tag{6}\]  
Here, \(\text{sin\_}inp_{p,i}\) represents the intermediate value calculated from the position index p and the inverse frequency factor \(inv\_freq_{i}\).