# DBMF: A Novel Method for Tree Species Fusion Classification Based on Multi-Source Images
## 2 Materials and Methods
### Classification Models
#### 2.3.4 Double Branch Multi-Source Fusion Network  
Meaningful features are extracted by double branch multi-source fusion (DBMF) from the HSI bands that are exploited via the interpolation algorithm concerning MSI at the same pixel. The expected features of the HSI are extracted so that all the HSI bands can jointly reconstruct the feature map, which minimizes spectral distortion. Compared with other networks, DBMF has lower complexity because it requires less computation. The proposed DBMF method is illustrated in Figure 3. The spectral information of a pixel is determined by its average reflectance spectrum, while the spatial features are associated with the surrounding pixels. In this framework, the spectral branch extracts spectral features as the spatial branch extracts spatial features; they are then fused for subsequent operations. To minimize the complexity of tree categories and maintain sample imbalance, we fused two single-source datasets by the framework of spatial and spectral branches to complete the classification process. The HJ-1A HSI data are input to the HS branch, which is resampled by the bilinear interpolation algorithm to make its spatial resolution the same as that of the MSI captured by Sentinel-2. The HS input is composed of different spectral sizes of the HSI paths to fully exploit the structural information of the images. The Bi-LSTM first extracts the HSI data characteristics, then assigns multiple weights to each spectral feature through the triple-attention mechanism. Dense networks then perform residual calculations for numerous shallow features to generate representative residual features. The spectral branch implementation is described in Table 1. The kernel size of the bidirectional layer is (\(1\times 1\times 7\)). Feature maps in the shape of (\(7\times 7\times 47\), \(24\)) are obtained, then permuted to the triple-attention block to make a size of (\(7\times 7\times 47\), \(24\)). A feature map in the shape of (\(7\times 7\times 1\), \(60\)) is then obtained, and the BN-Mish-Conv block calculates a (\(1\times 1\times 47\)) kernel. The BN-Lfu-Conv block consists of a batch normalization (BN) layer  
Figure 2: Sandglass block with bottleneck structure. Note: The bottleneck falls into the middle of the depth-wise convolutions, which have high-dimensional representations, so that the module encodes expressive spatial information. and an active function with an Lfu unit (inspired by [PERSON] [39]), as well as an individual convolution layer. The MSI data captured by the MS Sentinel-2 satellite are input into the MS branch. The MSI has a much higher spatial resolution than the HSI, and therefore they are subject to very different spatial scales. Each convolutional layer is followed by a BN-Mish-Conv block. The sandglass block is applied to eliminate redundant spatial features, the output of which is a combination of deep and shallow learning CNNs that mine and merge deep spatial characteristics. The red dotted branch of the network employs the expected bands grouped by Bi-LSTM and the triple-attention model to extract spectral characteristics. The green dotted branch of the network employs the convolution and sandglass block to extract spatial characteristics. In the right architecture, the spectral information and spatial information are combined to form a new layer for spectral-spatial classification. The details of the spatial branch are similar with the spectral branch, which is shown in Table 1. The layer is a basic structure composed of a BN and Mish activation function plus a 3D-CNN with a kernel size of (3 \(\times\) 3 \(\times\) 1) and 24 channels. A sandglass block is applied to the network to change the feature map shape into (7 \(\times\) 7 \(\times\) 1, 24). These two steps are repeated to finally bring a spatial shape of (1 \(\times\) 60) to the next fusion layer. The spectral and spatial feature maps are obtained by spectral and spatial branches, respectively.