# Non-Autoregressive Sparse Transformer Networks for Pedestrian Trajectory Prediction
## 4 Experiments
### Ablation Study
#### 4.4.2 Contribution of Spatial and Temporal Sparsity  
In order to evaluate the contribution of the sparsity to the transformers, we conducted studies on the different extent of sparsity on transformer encoder. We designed experiments for sparsity hyper-parameters through a searching protocol. The spatial and temporal \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multicolumn{4}{c}{**Components**} & \multicolumn{6}{c}{**Performance (ADE/FDE)**} \\ \hline  & **STE** & **TTE** & **TD** & **ETH** & **HOTEL** & **UNIV** & **ZARA1** & **ZARA2** & **AVG** \\ \hline (a) & \(\surd\) & - & \(\surd\) & 0.38/0.65 & 0.26/0.42 & 0.29/0.59 & 0.38/0.67 & 0.19/0.43 & 0.30/0.55 \\ (b) & - & \(\surd\) & \(\surd\) & 0.37/0.64 & 0.18/0.37 & 0.35/0.67 & 0.28/0.58 & 0.20/0.44 & 0.28/0.54 \\ (c) & \(\surd\) & \(\surd\) & - & 0.37/0.63 & 0.16/0.35 & 0.29/0.57 & 0.27/0.58 & 0.20/0.42 & 0.26/0.51 \\ NaST & \(\surd\) & \(\surd\) & \(\surd\) & 0.35/0.62 & 0.15/0.35 & 0.27/0.56 & 0.25/0.56 & 0.19/0.41 & 0.24/0.50 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Ablation Study on ETH and UCY datasets of ADE/FDE metrics (the lower numerical result is better). sparsity was evaluated at intervals of 0.1 in the range of 0 to 1. Representative results are shown in Table 3. We can receive different variants of our model by setting different values in the spatial sparse threshold \(\gamma\) and the temporal sparse threshold \(\delta\). First, we set the temporal sparse threshold \(\delta\) fixed with 0.5 and observe the effect of different spatial sparsity. Spatial sparsity \(\gamma\) was evaluated at intervals of 0.1 in the range of 0 to 1. In NaST-sp0, the spatial sparse threshold \(\gamma\) was set to 0, which meant dense spatial interaction. It modeled the social interaction among the specific pedestrian with all neighbors in a scene. The performance of NaST-sp0 degraded further on the complex scenes such as UNIV, since dense interactions might make the model suffer from overfitting. When the spatial sparse threshold \(\gamma\) was set to 1 in NaST-sp1, it meant no interaction between pedestrian pairs. Evidently, the performances degraded, which illustrated the importance of social (spatial) interaction in predicting trajectories. In NaST-sp0.5, it worked well on the relative dense scenes such as UNIV and ZARA2, but it was not satisfactory on other simple scenes when the spatial sparse threshold \(\gamma\) was set to 0.5. The reason may be that the sparse mask might filter the important neighbors who interact with the specific pedestrian in the less crowded scenes. When the spatial sparse threshold \(\gamma\) is set to 0.1, the results are shown for NaST in Table 3.  
Second, we fixed the spatial sparsity to \(\gamma=0.1\) and then evaluated the effect of temporal sparsity. Temporal sparsity \(\delta\) is evaluated at intervals of 0.1 in the range of 0 to 1. Representative results are shown in Table 3. NaST-tp0 referred to the temporal sparse threshold \(\delta=0\), which meant dense temporal dependency. The prediction of the current time step depended on all the time steps before. In NaST-tp1, \(\delta\) was set to 1, which indicated that no temporal interaction among different time steps. The model obtains the best average performance in the original NaST when \(\gamma=0.1\) and \(\delta=0.5\). This indicated that the proper sparsity was conducive to making precise predictions. Different sparse spatial and temporal interaction matrices are obtained according to different spatial and temporal sparsity. Figure 4 shows the sparse spatial interaction matrices of different spatial sparsity in part of ETH scene. In Figure 4a, spatial sparsity \(\gamma=0\), representing dense spatial interaction; each pedestrian has interactions with others. In Figure 4b, the spatial sparsity \(\gamma=0.1\), representing sparse spatial interaction; each pedestrians have interactions with neighbors. In Figure 4c, the spatial sparsity \(\gamma=0.5\); the pedestrians have interactions with fewer neighbors.