## 2 Related Work
### Attention Mechanism and Transformer  
Even though LSTM can resolve the long sequence dependence issue, the model must receive each input data set separately, which is not practicable for complex time series computing. The latest attention mechanism significantly improves the above issue. The attention mechanism gained notoriety in 2014 when Google Mind published \"Recurrent Models of Visual Attention\"; this work used the RNN model and incorporated the attention mechanism for picture categorization [32]. The Seq2 Seq&attention methodology for machine translation was first applied to the NLP area in 2015 when [PERSON] published the article \"Neural Machine Translation by Jointly Learning to Align and Translate\" [33]. The 2017 publication \"Attention is All You Need\" by the Google machine translation team ultimately rejected network architectures such as RNN and CNN. Machine translation jobs only used the attention mechanism [34]. Better outcomes were obtained, and the attention mechanism immediately gained the interest of many academics. The reduced performance of long sentence translation in natural language was the primary problem that the attention mechanism was intended to address. The attention mechanism has undergone ongoing improvement and has been applied extensively for temporal prediction [34; 35; 36]. The attention process illustrates two key points. The weights and weighted averages of changeable context vectors are calculated using the softmax function. The attention mechanism has been widely employed in many models, including LSTM, Seq2 Seq, the encoder-decoder framework, etc. because it is an effective method of obtaining information that enables the encoder to actively seek relevant information at each step and temporarily ignore irrelevant information. [PERSON] developed a multichannel LSTM neural network that uses an attention layer to connect model outputs to input sequences in order to further increase prediction accuracy [37]. [PERSON] also used a sequence-to-sequence (\"Seq2 Seq\") model to predict influenza epidemics; the findings revealed that their method performed better than ARIMA and LSTM-based models [38]. The Seq2 Seq model's basic premise is to read an input sentence using a recurrent neural network, compress the information of the entire sentence into a fixed-dimensional encoder and then read this encoder using a second recurrent neural network to \"de-compress\" it into a sentence in the target language. The recurrent neural network that reads the input sentence and decompresses the sentence in the target language is referred to as the encoder and decoder, respectively, in [PERSON]'s 2014 proposal of the encoder-decoder architecture, based on Seq2 Seq [39]. The widely used encoder-decoder structure allows for selecting various recurrent neural networks per various requirements and usage scenarios. Transformer's foundation is a framework for encoders and decoders. Transformer, a multi-head self-attention mechanism developed by Google and based on the encoder-decoder, was widely adopted in natural language processing after it was submitted in 2017. The position of the input language is encoded using the encoder then converted into a vector by the decoder, utilizing the attention method. Transformer has recently started to be used for time series forecasting. It largely relies on its multi-head attention mechanism and layer stacking to capture dynamic hierarchical patterns in time series data, and to predict lengthy time series of challenging situations. Similarly to natural language processing, traffic timing issues research must concentrate on networks' temporal and spatial aspects. As a result, the transformer model has some promise for use in network and traffic flow prediction. Transform models may be used to address long-term correlation and multidimensional dynamic dependence of temporal traffic data, since the data in the traffic domain has greater dimensional features than natural language data.