## 2 Materials and Methods
### Classification Models  
#### 2.3.1 Feature Extraction Based on MVAE  
AE consists of an encoder network \(h=f_{encode}(x)\) and a decoder network \(x^{\prime}=f_{decode}(h)\). The aim of AE is to make \(x\) and \(x^{\prime}\) as similar as possible (e.g., by mean squared error). Sometimes the number of latent samples is more than the input, and the sparsity is constrained. Because of the plenty of computation for AE, a variational autoencoder (VAE) was proposed to reduce the computation by optimizing the distribution of latent codes, where KL divergence and reparameterization methods are applied to make random samples decoded into simple distribution [34] for data augment of multi-source remote sensing images. Based on the VAE model, MVAE is designed to extract features from them (composed of H-VAE and M-VAE). The dimension of HSI/MSI is reduced by PCA in h-MVAE, as shown in Figure 3. Then, the image is segmented into '\(a\times a\times c\)' shape by a sliding window, where, '\(a\)' denotes the size of the sliding window and '\(c\)' denotes the image channel after preprocessing. The encoder structure adopts a mixed convolution mode to extract feature information. In this paper, multiple 3D-Conv and 2D-Conv are used for extracting the spatial and spectral information. Mixed ConvTranspose3d and ConvTranspose2d layers are used to construct the decoder part. In addition, to avoid the risk of overfitting, h-MVAE used a batch normalization layer after the convolutional operation. Each linear layer is bound to Smish [35], except the output layer. In the module, for the purpose of the size of reconstruction before and after being consistent, an average pooling layer is used to fix the shape of the feature map.  
\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & **Birch** & **Larch** & **Mongolia** & **Poplar** & **Spruce** & **Willow** \\ \hline Dataset (1) & 130,124 & 39,216 & 57,620 & 3019 & 15,330 & 3492 \\ Dataset (2) & 150,771 & 58,829 & 11,412 & 2175 & 17,048 & 1067 \\ Dataset (3) & 99,082 & 82,746 & 38,114 & 1013 & 13,460 & 15,486 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Tree species samples from three datasets of the study area.  
Figure 2: Map of three tree species datasets from the study area.  
Based on the original VAE algorithm, two variables \(\mu\) and \(\
u\) obtained by the encoder, and then the latent code \(z\), were calculated follows:  
\[z=\mu+\gamma\times\
u \tag{1}\]  
here, \(\gamma\) denotes the random value drawn from the normal distribution.  
The loss function of the VAE module consists of distribution loss of latent codes and reconstruction loss, which is represented by Equation (2), where \(L_{Z}\) denotes loss of latent codes and \(L_{tr}\) denotes loss of reconstruction, \(N\) denotes the number of samples, and \(L_{Z}\) calculates the KL divergence between the latent codes in the normal distribution. In Equation (3), the shape of the input image \(I\) is a \(\times\) a \(\times\) c, and the shape of the output image for decoder \(\hat{I}\) is the same as \(I\).  
\[L_{Z}=\frac{1}{2}\sum_{i=1}^{N}\left(\mu_{i}^{2}+\
u_{i}^{2}-log\
u_{i}^{2}-1\right) \tag{2}\]  
\[L_{tr}=\sum_{i=1}^{N}\left(\frac{1}{\text{a}\times\text{a}\times\text{c}} \sum_{x}^{a-1}\sum_{y}^{a-1}\sum_{z}^{c-1}\left(I-\hat{I}\right)^{2}\right) \tag{3}\]  
The whole loss function \(L\) can be presented by Equation (4), which is the sum of the loss of latent codes and the loss of reconstruction.  
\[L=L_{Z}+L_{tr} \tag{4}\]  
In MVAE, the feature extracted from MSI is similar to HSI as a pretext task. The difference between them was that HSI was preprocessed before that. HSI and MSI are used as input to calculate \(L_{VAE}^{H}\) and \(L_{VAE}^{M}\), respectively. After the VAE module is trained, all patches are mapped into the VAE, which provides data augmentation for downstream tasks of tree species classification.