# Non-Autoregressive Sparse Transformer Networks for Pedestrian Trajectory Prediction
## 3 Methods
### Non-Autoregressive Transformer Decoder  
Traditional time series prediction methods often predict future location \(\hat{y}_{t}\) based on \(\hat{y}_{t-\tau}\). This kind of autoregressive fashion is prone to the propagation of errors in future predictions and is computationally expensive in practice. As the individual steps of the decoder must run sequentially rather than in parallel, autoregressive decoding prevents architectures such as the transformer from fully realizing their train-time performance  
Figure 3: The illustration of message passing mechanism of each pedestrian in direct temporal graph.  
advantage during inference. An appropriate solution is to adopt a non-autoregressive inference pattern in the encoder-decoder model. Hence, we address these limitations by modelling the problem in a non-autoregressive pattern, as described in the following.  
As illustrated in Figure 1, the transformer decoder comprises \(L\) layers, which receives the output of the sparse transformer encoder and a query sequence to produce the output embedding of the predictions. Similar to the transformer encoder, the decoder stacks are composed entirely of feed-forward networks (MLPs) and multi-head attention modules. Since no RNNs are used, there is no inherent requirement for sequential execution, making non-autoregressive decoding possible. Before decoding starts, the transformer decoder needs to know how many time steps the predicted trajectory will take in order to generate all the locations in parallel. As discussed in the machine translation work [28], the non-autoregressive decoding shows less conditional dependency between predicted elements \(\hat{y}_{t}\). Therefore, the input of the transformer decoder should contain as much as temporal dependency. The output of the transformer encoder contains spatial and temporal features, which are extracted by the sparse spatial transformer and sparse temporal transformer separately. It provides a reliable dependency on which to make predictions, especially to the relation of the locations at different times.  
Additionally, given the observed sequence \(X_{1},\dots,X_{T_{obs}}\), the last observation \(X_{T_{obs}}\) is the most relevant to the following next time steps. Inspired by [28], we simply copy the input embedding of the transformer encoder at the last observed time step and fill the query sequence with it. Then, we can obtain the query sequence \(q_{T_{obs+1}},\dots,q_{T_{pred}}\). The transformer decoder generates the future trajectories in a non-autoregressive pattern and outputs the predictions of \(T_{obs+1},\dots,\ T_{pred}\) at one time. A random Gaussian noise is concatenated to the output of the transformer decoder to generate various future predictions [9]. The concatenated features are then embedded by a fully connected layer. The final predictions are obtained by adding the last time input of the transformer encoder with a residual connection. Given the residual connection, it can predict the location offsets from the last observed location \(X_{T_{obs}}\) to each predicted \(\hat{y}_{t}\), \(t=T_{obs+1},\dots,\ T_{pred}\).