# MTGNet: Multi-Agent End-to-End Motion Trajectory Prediction with Multimodal Panoramic Dynamic Graph
## 3 Methodology
### Decoder  
The Res_Decoder layer plays a pivotal role in the model as a decoder, tasked with decoding the constructed panoramic vectors and transforming the results from a local coordinate system to a global coordinate system. This process provides an accurate global reference for the connections and relative positions between vehicles, thereby optimizing the performance of the entire network. The Res_Decoder layer achieves this by processing the input panoramic vectors in multiple stages through Residual Prediction Modules, leveraging residual connections and feature transformations to ensure precise capture of inter-vehicle relationships and positional information.  
* Panoramic Vector Decoding and Multi-Stage Processing  
In the workflow of the Res_Decoder layer, the input panoramic vectors are first processed in stages by multiple Residual Prediction Modules. Each module refines the decoding process through residual connections and feature transformations, ensuring that the output at each stage captures increasingly detailed feature information. This multi-stage design not only enhances the model's representational capacity but also makes the decoding process more robust and efficient. The core of each Residual Prediction Module is the residual connection, expressed as:  
\[h_{i}=f(h_{i-1})+h_{i-1} \tag{16}\]  
where:  
\(\rm{h_{i-1}}\) is the input to the i-th module.  
\(\rm{f(h_{i-1})}\) represents the feature transformation obtained through linear transformations, normalization, and activation functions.  
\(\rm{h_{i}}\) is the output of the ii-th module.  
Each Residual Prediction Module includes a Group Normalization layer to stabilize and accelerate training. The normalization process is defined as:  
\[y=\frac{x-\mu}{\sigma}\cdot\gamma+\beta \tag{17}\]  
where:  
\(\rm{x}\) is the input feature.  
\(\rm{\mu}\) and \(\sigma\) are the mean and standard deviation computed over groups.  
\(\rm{\gamma}\) and \(\beta\) are learnable scaling and shifting parameters.  
\(\rm{y}\) is the normalized output.  
* Multi-Stage Processing  
Assuming there are N Residual Prediction Modules, the decoding process can be described as:  
\[h_{0}=x_{input} \tag{18}\]\[h_{i}=\text{ResidualPredictionModule}(h_{i-1}),\text{ i}=1,2,...\text{N} \tag{19}\]  
where:  
\(\text{h}_{0}\): Represents the initial input, which is the panoramic vector received by the Res_Decoder layer.  
\(\text{h}_{i}\): Represents the output of the ii-th Residual Prediction Module.  
N: Represents the total number of Residual Prediction Modules.  
During the decoding process, the DecoderResidual layer not only extracts and decodes features but also maps the decoded results from the local coordinate system to the global coordinate system by incorporating agent-specific transformations (e.g., translation and rotation). This mapping allows the model's output to be directly aligned with the ground truth data in the original dataset, simplifying subsequent analysis and evaluation. By transforming the coordinates into a global reference frame, the DecoderResidual layer provides a unified framework for comparing and analyzing predictions directly in the global space. This design not only improves the model's accuracy but also enhances the overall performance of the network, particularly in multi-agent collaborative tasks, where it enables precise descriptions of inter-vehicle relative positions and connections.  
Furthermore, the output shape of the DecoderResidual layer is adjusted to facilitate downstream processing. In short, although we pass the transformation information of each agent during the decoding process, the goal of the loss function is to directly produce end-to-end results. Therefore, after recalculating the global positions, the results are aligned with the original positional data in the dataset. Leveraging the computational power of neural networks, the model can be efficiently deployed on edge devices without the need for post hoc global coordinate fitting.