# Multi-Scale CNN-Transformer Dual Network for Hyperspectral Compressive Snapshot Reconstruction
## 3 Materials and Methods
### DualCT Module
#### 3.2.1 Parallel Complementary Fusion  
As shown in Figure 3, the parallel complementary fusion structure consists of a spatial domain CNN branch (abbreviated as Spa-CNN), a spectral domain Transformer branch (abbreviated as Spe-Trans), and bidirectional cross-linking between these two branches. In the Spa-CNN branch, it cascades three consecutive convolution operations with kernel sizes of \(1\times 1\), \(5\times 5\), and \(1\times 1\). The GELU layer is used after each convolution. The Spe-Trans branch exploits the Transformer to capture the global correlation between spectral bands, and its detailed design will be described in the next sub-section. Bidirectional cross-linking facilitates feature fusion between two branches, effectively capturing spatial-spectral correlations in the hyperspectral image. Specifically, the cross-linking is composed of spatial interaction and channel interaction. The channel interaction directs feature information from the Spa-CNN branch to the Spe-Trans branch, enhancing the feature modeling capability in the channel dimension. At the same time, the spatial interaction directs features from the Spe-Trans branch to the Spa-CNN branch, prompting the model to capture the spatial dependencies between different locations. The calculation flow of bidirectional interactions can be expressed as follows:  
\[ \begin{split} F_{c}&=Spa-CNN(LN(X^{l})),\\ A&=CI(F_{c}),\\ F^{{}^{\prime}}_{t}&=Spe-Trans(X^{l},A),\\ F^{{}^{\prime}}_{c}&=SI(F^{{}^{\prime}}_{t})\odot F _{c},\end{split}  \tag{5}\]  
where \(X^{l}\) is the input of the parallel complementary fusion structure, \(F_{c}\) is the feature output of the Spa-CNN branch, \(CI(\cdot)\) is the channel interaction function, SI is the spatial interaction function, and A is the channel attention calculated from \(F_{c}\) by \(CI(\cdot)\). The spectral-Trans branch takes the \(X^{l}\) and attention weight A as input and uses A to weight the value matrix. The detailed formulation of spectral-Trans branch will be presented in Section 3.2.2. Usingthe output \(F_{t}^{{}^{\prime}}\) of the spectral-Trans branch as input, we calculate the spatial attention map and use it to weigh \(F_{c}\). Therefore, this parallel complementary structure can promote the interaction and fusion of spatial and spectral features. **With regard to channel interaction**, we first use a global average pooling layer to obtain the global average feature vector of \(F_{c}\). Then, the global average feature is processed through two consecutive \(1\times 1\) convolutional layers, and the GELU activation function [43] is used for non-linear transformation between the two convolutional layers. Finally, the channel attention map is generated along the channel dimension by applying the sigmoid function. The specific operations for channel interaction are defined as follows:  
\[A=CI(F_{c})=\sigma(Conv(GL(Conv(GP(F_{c}))))), \tag{6}\]  
where \(\sigma\) represents the Sigmoid function, \(Conv(\cdot)\) represents the \(1\times 1\) convolution, GL represents the GELU activation function, and GP represents the global average pooling layer. The channel attention, A, is used to weight each spectral band of the value matrix in the spectral Transformer. **With regard to spatial interaction**, it calculates the spatial attention map from \(F_{t}^{{}^{\prime}}\) to weight the output \(F_{c}\) of the spatial-domain CNN branch. Specifically, it consists of two \(1\times 1\) convolutional layers, and the GELU activation function [43] is used between these two convolutional layers. Finally, the spatial attention map is generated by applying the sigmoid function. The specific operations for spatial interaction are defined as follows:  
Figure 3: Parallel complementary fusion structure inside DuaCT module. \[SI(F_{t}^{{}^{\prime}})=\sigma(Conv(GL(Conv(F_{t}^{{}^{\prime}})))), \tag{7}\]  
where \(SI(\cdot)\) represents the spatial interaction process, \(\sigma\) represents the Sigmoid function, \(Conv(\cdot)\) represents \(1\times 1\) convolution, and GL represents the GELU activation function.