  Although Transformer has made breakthrough success in widespread domains
especially in Natural Language Processing (NLP), applying it to time series
forecasting is still a great challenge. In time series forecasting, the
autoregressive decoding of canonical Transformer models could introduce huge
accumulative errors inevitably. Besides, utilizing Transformer to deal with
spatial-temporal dependencies in the problem still faces tough difficulties.~To
tackle these limitations, this work is the first attempt to propose a
Non-Autoregressive Transformer architecture for time series forecasting, aiming
at overcoming the time delay and accumulative error issues in the canonical
Transformer. Moreover, we present a novel spatial-temporal attention mechanism,
building a bridge by a learned temporal influence map to fill the gaps between
the spatial and temporal attention, so that spatial and temporal dependencies
can be processed integrally. Empirically, we evaluate our model on diversified
ego-centric future localization datasets and demonstrate state-of-the-art
performance on both real-time and accuracy.
