## 3 Incremental Pyraformer-DCCA
### Pyraformer
#### 3.1.1 Pyramid Attention  
The core of Pyraformer lies in its PAM that captures long-range temporal dependencies by decomposing the input sequence into multiple resolutions. This multi-resolution structure enables long-range dependencies at lower resolutions and the significant reduction of computational complexity.  
The structure of the PAM is shown in Figure 5. Pyraformer employs a pyramidal graph structure to perform self-attention by extracting features at different resolutions through inter-scale and intra-scale connectivity, modeling dependencies at multiple scales [21]. In this structure, nodes at the bottom level represent the observed values at each moment, while nodes at the upper levels aggregate and extract features from the nodes below them. By connecting nodes across different levels through inter-scale connections, the relationships between nodes at different resolutions can be effectively captured. These  
Figure 4: Pyraformer Module. N* denotes the number of repeated layers in the module.  
Figure 3: Framework of the proposed Incremental Pyraformer–DCCA Model.  
connections enable the model to share and integrate information across various time scales, allowing for the modeling of long-term dependencies and more complex temporal patterns.  
Consider a pyramidal graph where each node represents an element in the input sequence at different temporal scales. Let the input sequence be represented as \(X\in\mathbb{R}^{L\times D}\), where \(L\) is the sequence length, and \(D\) is the feature dimension. In the attention mechanism, the input \(X\) is first transformed into three distinct matrices: query (\(Q\)), key (\(K\)), and value (\(V\)), given by:  
\[Q = XW_{Q} \tag{10}\] \[K = XW_{K}\] (11) \[V = XW_{V} \tag{12}\]  
where \(W_{Q},W_{K},W_{V}\in\mathbb{R}^{D\times D_{k}}\) are learnable weight matrices.  
In the PAM, for a node at scale \(s\) and position \(\ell\), its neighborhood \(N_{\ell}^{(s)}\) includes the following nodes: adjacent nodes (\(A_{\ell}^{(s)}\)), child nodes (\(C_{\ell}^{(s)}\)), and a parent node (\(P_{\ell}^{(s)}\)). \(A_{\ell}^{(s)}\) refer the nodes at the same scale that are adjacent to the current node. This helps capture local temporal dependencies. \(C_{\ell}^{(s)}\) refer the nodes at the finer resolution level, capturing more detailed temporal information. \(P_{\ell}^{(s)}\) refer the node at the coarser scale, representing broader temporal trends. These connections form a pyramid-like hierarchical structure, allowing nodes to aggregate information from different temporal resolutions without requiring global attention.  
The attention calculation for a node \(n_{\ell}^{(s)}\) in the pyramid graph can be expressed as follows:  
\[y_{i}=\sum\limits_{\ell\in N_{i}(s)}\frac{\exp(q_{i}k_{\ell}^{T}/\sqrt{q_{K}}) v_{\ell}}{\sum\limits_{t\in N_{i}(s)}\exp(q_{i}k_{t}^{T}/\sqrt{d_{K}})} \tag{13}\]  
where \(q_{i}\) represents the query vector for the current node, \(k_{\ell}\) is the key vector of the neighboring node, and \(v_{\ell}\) is the corresponding value vector. The term \(d_{K}\) is the dimensionality of the key vectors, used to normalize the dot product. The neighborhood node \(N_{\ell}^{(s)}\) is  
Figure 5: Pyramid Attention Module. Blue dashed lines: indicate self-attention within each node. Green dashed lines: represent connections between nodes at different levels (inter-scale connectivity). Orange dashed lines: show connections between nodes within the same level (intra-scale connectivity). Red dotted line: marks the longest information propagation path between any two nodes.  
defined as the union of adjacent nodes (\(A_{\ell}^{(s)}\)), child nodes (\(C_{\ell}^{(s)}\)), and the parent node (\(P_{\ell}^{(s)}\)), effectively capturing dependencies at multiple temporal scales:  
\[N_{\ell}^{(s)}=A_{\ell}^{(s)}\cup C_{\ell}^{(s)}\cup P_{\ell}^{(s)} \tag{14}\]