# _M\({}^{2}\)Tames_: Interaction and Semantic Context Enhanced Pedestrian Trajectory Prediction
## 3 Proposed Approach
### M2 Tea: Multi-Head Mask and Temporal Attention
#### 3.2.2 Multi-Head Mask Attention  
Previous models [5; 26] on pedestrian trajectory prediction using transformer models often overlooked the issue of losing the social identity information of agents and their surroundings. To address this problem, _M2 Tea_ is employed, the principle of which is explained in Figure 4, where the mask is assigned 0 or 1. If the \(i\)-th query \(q_{i}\) and the \(j\)-th key \(k_{j}\) belong to the same type of agent, the mask equals 1; otherwise, it is 0. This differentiation allows _M2 Tea_ to focus on an individual agent's motion characteristics while analyzing interactions with other agents or the environment. The mask mechanism ensures that _M2 Tea_ can accurately capture and learn the agent-owned behavioral patterns and trajectory characteristics while understanding how agents interact within a group. Such an approach enhances the comprehensiveness and accuracy of trajectory prediction.  
The multi-head mechanism constructs a model with multiple attention heads, and each attention head can learn different spatio-temporal relationships and feature representations. The weighted features calculated by each attention head are fused to form a richer and more diverse overall feature representation. To address complex interactions between multiple agents, given a simplified scenario with agents A and B, their interaction can be influenced by various factors, such as their current positions, relative velocity vectors, and goals.  
However, traditional single attention mechanisms can not effectively capture these diverse interaction modes. Multiple heads are created in the multi-head attention mechanism, each focusing on different aspects of interaction information. One head can concentrate on capturing agent A's potential destination and movement direction, while another could focus on agent B's velocity and position. This multi-head structure allows \(M^{2}\)_Tames_ to simultaneously learn features from these different aspects; thereby, \(M^{2}\)_Tames_ can understand the complex interactions between A and B more comprehensively.  
Introducing timestamps and a masking mechanism enable \(M^{2}\)_Tames_ to clearly distinguish between the identity and temporal information of the multiple agents during the learning process. The advantage of the multi-head mechanism lies in its ability to simultaneously process the information of different scales [23], making \(M^{2}\)_Tames_ more flexibly adaptable to the diverse spatio-temporal scales in pedestrian trajectory prediction. This design enhances the adaptability of \(M^{2}\)_Tames_, allowing it to capture the complete dynamics of agent movement more effectively. The specific computational formulas are defined as follows:  
\[A^{h}_{ij}(t)=softmax((Q^{h}_{i}(t)\cdot(K^{h}_{j}(t))^{T})/\sqrt{d_{k}}) \tag{11}\]  
\[\Lambda^{h}_{ij}(t)=A^{h}_{ij}(t)\cdot Mask_{ij}(t) \tag{12}\]  
\[O_{M^{2}Tea,i}(t)=\sum_{h=1}^{\mathcal{H}}\sum_{j=1}^{N}\Lambda^{h}_{ij}(t) \cdot V^{h}_{j}(t) \tag{13}\]  
where \(Q^{h}_{i}(t)\) and \(K^{h}_{j}(t)\) represent the projections of the \(i\)-th query and \(j\)-th key in the \(h\)-th attention head, respectively. The \(d_{k}\) is the scaling factor to ensure the stability of the attention mechanism. \(\Lambda^{h}_{ij}(t)\) is the attention weight considering the temporal mask. \(\mathcal{H}\) is the number of attention heads, and \(V^{h}_{j}(t)\) is the projection of the \(j\)-th value in the \(h\)-th head.