## 3 Incremental Pyraformer-DCCA
### Incremental Learning  
Incremental learning allows models to adapt to new data or features as they become available without the need for retraining from scratch. This method is particularly effective in dynamic environments where data continuously evolve, as it helps maintain the efficiency and scalability of the model. Incremental learning involves updating the model with new information while preserving previously acquired knowledge, thereby ensuring the model remains relevant and retains previously acquired knowledge. We utilized the concept of incremental learning to add new mapped features into the Pyraformer-CCA framework without retraining the entire model. This incremental learning approach based on the Broad Learning System (BLS) is crucial in scenarios with continuous introduction of new data or features [22; 23]. It ensures that the Pyraformer-DCCA model efficiently adapts to changes.  
Given the input training samples \(X\), the objective is to generate a complete model consisting of mapped features and enhancement nodes, ultimately resulting in the output weight matrix W. First, for each feature mapping node \(i\) (where \(i=0,1,\ldots,n\)), weights \(W_{ei}\) and biases \(\beta_{ei}\) are randomly initialized. The input \(X\) is then mapped using an activation function to generate feature mappings, as expressed by the following formula:  
\[Z_{i}=\phi(XW_{e_{i}}+\beta_{e_{i}}) \tag{17}\]  
The activation function \(\phi\) is typically chosen as ReLU or another nonlinear function to ensure that the model has strong representational capabilities. Once all feature mappings are computed, they are concatenated to form the feature mapping group \(Z^{n}\):  
\[Z^{n}=[Z_{1},Z_{2},\ldots,Z_{n}] \tag{18}\]  
To further improve the representation capacity, the algorithm introduces enhancement nodes. Each enhancement node \(j\) also has randomly initialized weights \(W_{h_{j}}\) and biases \(\beta_{h_{j}}\), and the concatenated feature mapping group \(Z^{n}\) is processed to obtain enhancement features \(H_{j}\):  
\[H_{j}=\xi(Z^{n}W_{h_{j}}+\beta_{h_{j}}) \tag{19}\]  
The activation function \(\xi\) is used here to enhance the nonlinear expressiveness of the features. All enhancement nodes are then concatenated together to form the enhancement node group \(H^{m}\):  
\[H^{m}=[H_{1},H_{2},\ldots,H_{m}] \tag{20}\]  
Denote \(A^{m}=[Z^{n}|H^{m}]\) and \(A^{m+1}\equiv\left[A^{m}\middle|\xi\left(Z^{n}W_{h_{m+1}}+\beta_{h_{m+1}} \right)\right]\) as \(W_{h_{m+1}}\in\mathbb{R}^{nk\times p}\) and \(\beta_{h_{m+1}}\in\mathbb{R}^{p}\). The connecting weights and biases from mapped features to the p additional enhancement nodes are randomly generated.  
The formula for calculating the pseudoinverse is as follows:  
\[A^{+}=\lim_{\lambda\to 0}(\lambda I+AA^{T})^{-1}A^{T} \tag{21}\]To accommodate the expansion, the pseudoinverse of the updated matrix is deduced using an efficient incremental update rule, avoiding the computational burden of recalculating the pseudoinverse from scratch. The updated pseudoinverse is expressed as:  
\[(A^{m+1})^{+}=\begin{bmatrix}(A^{m})^{+}-DB^{T}\\ B^{T}\end{bmatrix} \tag{22}\]  
where \(D=(A^{m})^{+}\xi(Z^{n}W_{h_{m+1}}+\beta_{h_{m+1}})\), and \(B^{T}\) is defined by the condition:  
\[B^{T}=\begin{cases}(C)^{+}&\text{if C}\
eq 0\\ (1+D^{T}D)^{-1}B^{T}(A^{m})^{+}&\text{if C}=0\end{cases} \tag{23}\]  
with \(C=\xi(Z^{n}W_{h_{m+1}}+\beta_{h_{m+1}})-A^{m}D\).  
Finally, the output weights \(W^{m+1}\) are updated using the updated pseudoinverse:  
\[W^{m+1}=\begin{bmatrix}W^{m}-DB^{T}Y\\ B^{T}Y\end{bmatrix} \tag{24}\]  
where \(Y\) represents the output values. Once all iterations are complete, the final output weight matrix \(W\) is given by:  
\[W=W^{m+1} \tag{25}\]