# TGN: A Temporal Graph Network for Physics Prediction
## 3 Methodology
### Spatial Encoding  
After representing the mesh as a graph \(G\), GSE combines the topology information in \(G\) and the attributes of the nodes in \(S_{t}\) to capture the spatial features at each time step \(t\) and represent them with a latent vector \(\mathbf{z}_{t}\). In the discussion on encoding spatial information, we ignore the time variable \(t\) to simplify the subsequent expression. GSE is a graph neural network model with an Encode-Process-Decode architecture [9; 41]. It initially extracts node features and edge features from the graph \(G\) using Multi-Layer Perceptrons (MLPs). \[\mathbf{v}_{i}^{0}=\text{mlp}_{v}(\mathbf{S}[i],\mathbf{n}_{i}),\quad\mathbf{e}_{ij}^{0}=\text{ mlp}_{e}(\mathbf{p}_{ij},|\mathbf{p}_{ij}|), \tag{2}\]  
where \(\mathbf{S}[i]\) denotes the physical quantity of node \(i\) and \(\mathbf{n}_{i}\) denotes the one-hot encoding of the type of node \(i\). \(\mathbf{p}_{i}\) is the spatial coordinate of node \(i\). We encode the relative position vector \(\mathbf{p}_{ij}\) and its norm \(|\mathbf{p}_{ij}|\) as the edge feature. After that, GSE uses \(L\) GraphNet blocks to capture spatial features through the message-passing mechanism to further represent the nodes. Each GraphNet block contains an independent set of parameters, and in Figure 3, we briefly demonstrate the updating process of the nodes of the \(l\)-th GraphNet block. For the \(l\)-th GraphNet block we use \(B\) basis functions \(\text{mlp}_{l,b}^{e}\) implemented by MLPs to capture different edge features  
\[\mathbf{e}_{ij}^{l,b}=\text{mlp}_{l,b}^{e}(\mathbf{e}_{ij}^{l-1},\mathbf{v}_{i}^{l-1},\mathbf{ v}_{j}^{l-1}),\quad b=1,\dots,B, \tag{3}\]  
and then we use the concatenation operator \(\parallel\) to concatenate the results of the basis functions for updating the edge features:  
\[\mathbf{e}_{ij}^{l}=\mathop{\parallel}_{b=1}^{B}\mathbf{e}_{ij}^{l,b}. \tag{4}\]  
We update the node features using the output of the previous block and the aggregation result of the edge features \(\mathbf{e}_{ij}^{l,b}\), where the aggregation of the edge features can be divided into three steps. First, we aggregate each set of edge features from different basis functions \(\text{mlp}_{l,b}^{e}\) separately. Second, based on the features of each node \(i\in\mathcal{V}\) we compute  
Figure 3: Visualization of node updates. We employ 3 MLPs (B = 3) to update edge features and combine the aggregated results of the edge features through the node weights \(W^{l,i}\). This simplified diagram does not depict multiple heads or aggregators. the combined weight coefficient \(W^{l,i}=\mathbf{\Phi}v_{i}^{l-1}+\mathbf{b}\in\mathbb{R}^{B}\), where \(\mathbf{\Phi}\) and \(\mathbf{b}\) denote the learnable weights and bias parameters. Third, we weight the aggregation results of each set of edge features:  
\[\hat{\mathbf{v}}_{i}^{l}=\sum_{b=1}^{B}W_{b}^{l,i}\sum_{j\in\mathcal{N}(i)}\mathbf{e}_{ ij}^{l,b}. \tag{5}\]  
Here \(\mathcal{N}(i)\) denotes all neighbors of node i. Inspired by the work of [PERSON] and [PERSON] et al. [42], we add \(H\) heads and combine multiple aggregators \(\oplus\in\mathcal{A}\) (\(\mathcal{A}\) denotes a set of different aggregators such as mean, sum, min, or max). We apply different weighting coefficients to each header and aggregator and extend Equation (5) as  
\[\hat{\mathbf{v}}_{i}^{l}=\big{\|}_{h=1}^{H}\sum_{\oplus\in\mathcal{A}}\sum_{b=1}^{ B}w_{h,\oplus,b}^{l,i}\bigoplus_{j\in\mathcal{N}(i)}\mathbf{e}_{ij}^{l,b}. \tag{6}\]  
We then combine the aggregation result \(\hat{\mathbf{v}}_{i}^{l}\) of the edge features with the node features and encode them via MLPs to update the node features:  
\[\mathbf{v}_{i}^{l}=\text{mpl}_{i}^{p_{i}}(\mathbf{v}_{i}^{l-1},\hat{\mathbf{v}}_{i}^{l}). \tag{7}\]  
After \(L\) GraphNet blocks updating the graph \(G\), we obtain a new set of node representations \(V=(v_{i}^{l}:i\in\mathcal{V})\).