# Long-Term Forecasting Using MAMTF: A Matrix Attention Model Based on the Time and Frequency Domains
###### Abstract  
There are many time series forecasting methods, but there are few research methods for long-term multivariate time series forecasting, which are mainly dominated by a series of forecasting models developed on the basis of a transformer. The aim of this study is to perform forecasting for multivariate time series data and to improve the forecasting accuracy of the model. In the recent past, it has appeared that the prediction effect of linear models surpasses that of the family of self-attention mechanism models, which encourages us to look for new methods to solve  
information from the previous step for the next step, it is easy to have the problem of gradient disappearance or gradient explosion, which is not suitable for long-term prediction. The TCN model [12] consists of inflated causal convolution and a residual structure, and it is easy to lose a large amount of information with inflated convolution, which leads to a poorer prediction accuracy. However, with the proposal of a transformer [13], a series of models for long-term multivariate time series prediction began to emerge, and the problem of long-term time series prediction was gradually solved. On the basis of the self-attention mechanism, this evolution has produced a variety of different attention mechanisms, including ProbSparse self-attention [14], the Auto-Correlation Mechanism [15], and Frequency-Enhanced Attention with the Fourier Transform (FEA-f) [16]. The self-attention mechanism was originally proposed for natural language processing (NLP), which is effective for semantically rich NLP problems, but the temporal information contained in time series data is easily corrupted when using the self-attention mechanism. The good prediction results of the DLinear model [17] beat all models based on the self-attention family before FEDformer [16]. This confirms that in addition to using the self-attention mechanism to build the model, some other methods are also effective for long-term multivariate time series forecasting.  
In order to overcome the drawbacks of a high computational complexity and the lack of global consistency associated with self-attentive mechanisms, we therefore try to seek a breakthrough with a new methodology. Consider that time series are used to predict future changes in time series data using past time series data. We consider the data at a future point in time as the result of weighting the past time series and thus propose a matrix attention mechanism. The superiority of the matrix attention mechanism is verified in ablation experiments by comparing it with other self-attention families. The main contributions of our work are as follows.  
First, we propose a matrix attention mechanism. This attention mechanism will weight the previous data equally for prediction, which solves the problem that recurrent neural networks find it difficult to extract long distance information. This avoids the destruction of temporal information and simplifies the computational complexity compared to self-attentive families.  
Second, we design a frequency domain block to extract data features with high randomness and volatility and a time domain block to extract the overall trend change in the time series. Together, the two constitute our forecasting model.  
Third, we validate the accuracy of our model using three public datasets and one private dataset. It is demonstrated experimentally that our model achieves the best prediction results on all four datasets. The superiority of the matrix attention mechanism and the superiority of our model architecture are also demonstrated in the ablation experiments.  
The rest of the paper is organized as follows. Section 2 describes the related work. Section 3 describes in detail the matrix attention mechanism model based on the time and frequency domains. Section 4 describes the experimental procedure and results. Section 5 summarizes the paper.