## 3 Materials and Methods
### Proposed Method
#### 3.3.2 Class-Based Data Filtering  
The IP102 dataset, and consequently the obtained _clean dataset_, is remarkably unbalanced; some classes are even without images. Figure 3 shows the composition of the _clean dataset_; in particular, the number of samples belonging to each class are indicated, which highlights how the distribution is skewed towards some classes.  
Nevertheless, 36% of the _clean dataset_ classes are composed of less than 35 images. A dataset filtering phase is therefore required to ensure that the next data balancing step has a real validity to overcome the unbalanced dataset issue. In fact, augmentation techniques applied to a small set of original images (compared to the median) would produce a large number of synthetic samples which do not provide models with learning features for a proper classification of these pest categories due to overfitting. A fixed threshold for class filtering is therefore employed at this stage with the aim of both maintaining the _balanced dataset_ as close as possible to the original one and maximizing the classification  
Figure 3: Images per class distribution of the clean IP102 dataset. The graph underlines the high degree of imbalance that leads the classification model to be biased towards classes with a bigger cardinality. The plot does not show classes without images.  
Figure 2: Examples of some of the challenges introduced by the IP102 dataset. In (**a**), pests belonging to different categories which demonstrate very similar visual characteristics are shown. In (**b**), images of the dataset which contain copyright notices or other symbols that do not pertain to the instance in question and add noise are shown.  
performance. The process of selecting the appropriate threshold has been supported by several experiments.  
The objective of this stage is the removal of classes for which a \"significant augmentation\" is needed so as to avoid the introduction of biases in the dataset. However, a high number of classes in the dataset is composed of few images, and removing all these classes would bias the experiment too. For this reason, several tests have been carried out according to the following approach: \((i)\) removing classes below a varying threshold, \((ii)\) checking how many classes and images are remaining, \((iii)\) applying the augmentation pipeline, and \((iv)\) checking the classification results. Figure 4 shows the variation of the dataset image distribution with a threshold value as a consequence of the filtering process. An analysis of the figure shows that as the filtering value increases, more classes are removed from the dataset, thus reducing the number of instances for the training phase. As a consequence of the elimination of less populated classes, the median and mean values of the cardinality of the remaining classes increase.  
The study carried out has highlighted that a filtering threshold equal to 25 images per class offers an optimal trade-off. In fact, the aforementioned value provides the following: \((a)\) the removal of classes for which an oversampling procedure would result in the generation of too many synthetic samples; \((b)\) the maintenance of the majority of classes in the dataset; \((c)\) the preservation of the scientific validity of the experiment. All classes with a cardinality below 25 are therefore not included in the _filtered dataset_ that undergoes to the next balancing phase.