## 4 Method
### Multi-Scale \(SAMA\) Bilinear Fusion  
In this study, we use multiple self-attention heads [9] to compute the \(SAMA\) indicators of different scales, and further combine them to forecast the future trend of time series. Specifically, the 5-step, 20-step, and 60-step moving averages are simultaneously modeled, i.e., \(SAMA_{5}\), \(SAMA_{20}\), and \(SAMA_{60}\) indicators, which capture the past weekly, monthly, and seasonal dynamic patterns of day-frequency time series, respectively.  
Generally speaking, we predict the moving direction of time series by comparing different scales' moving average indicators. For example, in the stock market, the short-term moving average crosses upward or falls below the long-term moving average, which are usually regarded as rising and falling signals, respectively. However, such heuristic strategies are highly subjective and do not make full use of the characteristics of different scales' moving average indicators. In this paper, we resort to the bilinear fusion [10] to make the different dimensions of different scales' indicators interact with each other and obtain richer representations. To be specific, we input the three moving averages \(SAMA_{5}\), \(SAMA_{20}\), and \(SAMA_{60}\) into the linear transformation layer, and perform dimensional expansion operations on them, respectively, to obtain vectors \(\mathbf{a}_{5}\), \(\mathbf{a}_{20}\), and \(\mathbf{a}_{60}\). Then, we send them to the bilinear fusion layer and the indicators of different scales are aggregated to a matrix through the outer product. As such, the interaction and fusion between each pair of indicators \(\mathbf{a}_{i}\) and \(\mathbf{a}_{j}\) are carried out, and the bilinear vector \(\mathbf{z}_{\{i,j\}}\) is finally obtained:  
\[\mathbf{z}_{\{i,j\}}=flat(\mathbf{a}_{i}\otimes\mathbf{a}_{j}), \tag{8}\]where \(\otimes\) and \(flat\) denote the outer product and flattening operation, respectively. \(i\), \(j\in\{5,20,60\}\) and \(i\
eq j\). Finally, we adopt the idea of ensemble learning to make time series prediction. We input the obtained bilinear fusion vectors into three weak learners to adaptively learn the corresponding weight of the bilinear fusion vectors. Then, we send them to a strong learner with a fully connected layer for future trend prediction:  
\[\hat{y}=\mathbf{W}_{FC}[\mathbf{W}_{1}\mathbf{z}_{\{5,20\}},\mathbf{W}_{2}\mathbf{z}_{\{5,60\}},\bm {W}_{3}\mathbf{z}_{\{20,60\}}], \tag{9}\]  
where \(\mathbf{W}_{1}\), \(\mathbf{W}_{2}\), \(\mathbf{W}_{3}\), and \(\mathbf{W}_{FC}\) are the weight matrices to be learned.  
We consider three movement directions of time series, namely the rising, falling, and steady trends. We use cross entropy as the loss function to penalize the deviation of the prediction from the ground-truth. The \(\hat{y}\) donates the predicted probability distribution over different trends at the next time step, and \(y\) is a one-hot vector indicating the true trend label. The cross entropy loss can be computed as  
\[l(\hat{y},y)=\sum_{i}y_{i}\log\hat{y}_{i}, \tag{10}\]  
where \(i\) is the dimension index. The optimal \(SAMA\) indicator can be determined by minimizing the loss over all time series.