# An Adaptive Weighted Method for Remote Sensing Image Retrieval with Noisy Labels
## 4 Experiments and Analysis  
### Datasets and Experimental Setup  
Three widely used public remote sensing image datasets: UC-Merced dataset (UCMD) [43], aerial image dataset (AID) [44] and Northwestern Polytechnical University dataset (NWPU) [45] are used to evaluate the effectiveness of our proposed method. The UCMD dataset [43] consists of 21 categories, and each category contains 100 images with a size of 256 \(\times\) 256 pixels. The AID dataset [44] contains 30 categories and a total of 10,000 images, with 220-420 images per category. The image size is 600 \(\times\) 600 pixels. The NWPU dataset [45] contains 31,500 images and 45 categories. There are 700 images per category, and each image is 256 \(\times\) 256 pixels in size. These three datasets can be used to validate the robustness of the method, as they have different levels of complexity and intra-class diversity. Examples of all categories in the three datasets are shown in Figures 2-4.  
In the experiments, we randomly select 60% of the images as the training set, 20% of the images as the validation set and 20% of the images as the test set. To evaluate the effectiveness of the methods against noisy labels, we add different proportions of symmetric noise (e.g., 5%, 10%, 20% and 30%) to simulate noisy labels in the training and validation sets. Specifically, the label is flipped uniformly across all the classes with probability \(p\), regardless of the similarity between the classes. In this case, the label transition matrix E has the entries \(1-p\) in the diagonal and \(p\) in the off-diagonal elements.  
To comprehensively evaluate the effectiveness of our method, we compare it with manual weighting methods on different backbone models, including ResNet [46], DenseNet [47] and MobileNet [48]. The compared methods are MAE [13]; GCE [14]; SCE [15]; RCE [15]; ACE [16]; AUL [31]; AEL [28] and four APL combinations, namely \(\alpha\)NCE (normalised cross-entropy loss) + \(\beta\)MAE, \(\alpha\)NCE + \(\beta\)RCE, \(\alpha\)NFL (normalised focal loss) + \(\beta\)MAE and \(\alpha\)NFL + \(\beta\)MAE with 12 different combinations of the weighting hyperparameters \(\alpha\) and \(\beta\)[17]. These 12 weights are recommended in the literature [17]. In addition, mAP (mean average precision) is used as a metric to evaluate the retrieval performance.  
All experiments are repeated three times to ensure the reliability of the results. All methods are performed with ImageNet pretrained classifiers and the Adam optimiser. The initial learning rate and weight decay are both set to 0.00015. To avoid unreliable noisy sample evaluation results at the beginning of training, \(\alpha\) and \(\beta\) are set to 1 for the first three epochs. Our method's feature extraction module uses ResNet50 as the basic backbone. AWNet also uses Adam as the optimiser, with an initial learning rate of 0.001 and a weight decay of 0.0001. All models are trained using PyTorch 1.8.2 on a single NVIDIA GeForce RTX 3090 GPU with a batch size of 128 for 20 epochs.  
Figure 3: Examples of the AID dataset: (1) airport; (2) bare land; (3) baseball field; (4) beach; (5) bridge; (6) centre; (7) church; (8) commercial; (9) dense residential; (10) desert; (11) farmland; (12) forest; (13) industrial; (14) meadow; (15) medium residential; (16) mountain; (17) park; (18) parking; (19) playground; (20) pond; (21) port; (22) railway station; (23) resort; (24) river; (25) school; (26) sparse residential; (27) square; (28) stadium; (29) storage tanks; (30) viaduct.  
Figure 2: Examples of the UCMD dataset: (1) agricultural; (2) airplane; (3) baseball diamond; (4) beach; (5) buildings; (6) chaparral; (7) dense residential; (8) forest; (9) freeway; (10) golf course; (11) harbour; (12) intersection; (13) medium density residential; (14) mobile home park; (15) overpass; (16) parking lot; (17) river; (18) runway; (19) sparse residential; (20) storage tanks; (21) tennis courts.