# IFF-Net: Irregular Feature Fusion Network for Multimodal Remote Sensing Image Classification
## 3 Results
### Experimental Setup  
(1) Evaluation metrics: In this study, we assess the performance of a classifier for multimodal remote sensing data using three commonly employed metrics: overall accuracy (OA), average accuracy (AA), and kappa coefficient (\(\kappa\)). Generally, higher values of these metrics indicate better performance in classifying remote sensing images. OA, AA, and \(\kappa\) provide quantitative evaluations of classification performance, with their respective formulas as follows:  
\[OA=\frac{N_{c}}{N_{a}} \tag{18}\]  
\[AA=\frac{1}{C}\sum_{i=1}^{C}\frac{N_{c}^{i}}{N_{a}^{i}} \tag{19}\]  
and  
\[\kappa=\frac{OA-P_{e}}{1-P_{e}} \tag{20}\]  
In this context, \(N_{c}\) represents the count of accurately classified instances, \(N_{a}\) represents the total count of instances, \(N_{c}^{i}\) signifies the count of instances belonging to a specific category \(i\) within those that are accurately classified, and \(N_{a}^{i}\) indicates how many instances belong to category \(i\) out of all instances. Within \(\kappa\), \(P_{e}\) refers to an anticipated prior probability, which can be computed by  
\[P_{e}=\frac{N_{r}^{1}\times N_{p}^{1}+\ldots N_{r}^{i}\times N_{p}^{i}+\cdots+ N_{r}^{C}\times N_{p}^{C}}{N_{a}\times N_{a}} \tag{21}\]  
where \(N_{r}^{i}\) denotes the number of true samples for each category and \(N_{p}^{i}\) denotes the number of predicted samples for each category. (2) Implementation details: The IFF-Net model experiments proposed in this paper are implemented on the TensorFlow 2.0 platform, and the libraries used for model computation are CUDA10.0 and CuDNN10.0, using a Dell tower server, which the server is branded Dell and purchased from Luzhou, Sichuan, China, with Inter Xeon Silver 4210 2.60 GHz CPU, 128 GB of RAM, and a GPU of NVIDIA GeForce GTX 3080 Ti. In this paper, the square field for \(r\times r\) represents the spatial information of the center pixel, due to the rich spatial information fused with multimodal information the performance of IFF-Net on small image blocks has a great advantage compared to that of larger image blocks. Through experimental comparative analysis, the experimental parameter in this paper set the block size to \(r=12\). Regarding the regularization factor \(\lambda\), which is used to balance the weights \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Class No** & **Class Name** & **Training Set** & **Testing Set** \\ \hline C1 & Forest & 8243 & 46,711 \\ \hline C2 & Residential Area & 40,296 & 228,346 \\ \hline C3 & Industrial Area & 2935 & 16,631 \\ \hline C4 & Low Plants & 8887 & 50,368 \\ \hline C5 & Soil & 2614 & 14,812 \\ \hline C6 & Allotment & 1996 & 11,309 \\ \hline C7 & Commercial Area & 3724 & 21,100 \\ \hline C8 & Water & 1001 & 5671 \\ \hline  & Total & 69,696 & 394,948 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Describes the sizes of training and test sets for the Berlin dataset. between the two loss values, the loss function in this paper is defined in Formula (1), and the \(L_{1}\) paradigm acts on the judgment factor to achieve the sparsity constraint. A larger \(\lambda\) degrades the classification performance, and the performance starts to decay when \(\lambda\) approaches 0. When \(\lambda\) > 0.1, the classification performance drops on both datasets. The experimental results show that \(\lambda\) = 0.05 is chosen as the regularization factor for the IFF-Net network. Empirically, an appropriate increase in the number of convolutional layers can enhance classification performance and accuracy; however, it incurs a higher computational cost and leads to a slower network convergence speed. In the case of insufficient training samples, it can easily lead to overfitting. At the same time, a larger \(n\) will lead to an increase in the use of random-access memory (RAM) for the model parameters during training. Meanwhile, a larger n value will also lead to a decrease in the OA value. The selection of two residual blocks with feature extraction and propagation, where \(n\) = 2, is made after comprehensive consideration.