# A Closer Look at Few-Shot Classification with Many Novel Classes
## 5 Experiments
### Experimental Setup  
#### 5.1.1 Datasets  
This paper introduces ImageNet-MNC, a novel dataset tailored for FSL-MNC. The base class set is derived from ImageNet-1k (ILSVRC 2012-2017) [7], a benchmark widely utilized in image classification comprising 1000 categories. Furthermore, the recently released ImageNet-21K (winter '21 version) [7] provides the novel class set, featuring a vast and diverse collection of over 21,000 categories.  
To assemble ImageNet-21K-MNC, the study initially removed all 1000 categories that overlapped with ImageNet-1k from ImageNet-21K, preserving the uniqueness of the new dataset. Subsequently, categories containing fewer than 20 samples were excluded, amounting to 1455, to ensure each category had adequate data for effective training and testing.  
After refining the dataset, a total of 16,712 viable categories remained. For the meta-learning experiments, 15,000 of these categories were randomly chosen for meta-testing to evaluate the model's generalization capabilities on unseen classes. The remaining 1712 categories served as a validation set in the meta-training phase, facilitating the tuning and optimization of model parameters.  
The dataset's partition strategy ensures a strict separation between training and testing categories and enables performance assessment of various models within a cohesive and intricate framework. The design of ImageNet-MNC caters to the demands of large-scale, challenging classification tasks prevalent in modern machine learning, particularly deep learning, and aims to propel advancements in few-shot learning, transfer learning, and related fields.  
In addition to the proprietary dataset, the experiments also employed several publicly available few-shot learning datasets, including miniImageNet [15], CIFAR-FS [27], and Meta-Dataset [28]. These datasets were strictly non-overlapping in training and testing phases, adhering to the setup where base and novel classes do not overlap.  
miniImageNet, a subset of ImageNet-1k, contains 100 categories, each with 600 samples. Following established protocols [17], 64 categories were utilized as seen classes, with the remaining 16 and 20 categories serving as unseen classes for model validation and evaluation, respectively. CIFAR-FS is organized into 64 training categories, 16 validation categories, and 20 testing categories, with images sized at 32 \(\times\) 32.  
Additionally, this study explored the cross-domain few-shot generalization capabilities of SHA-Pipeline using the Meta-Dataset, which encompasses 10 public image datasets across various domains: ImageNet-1k, Omniglot, FGVC Aircraft, CUB-200-2011, Describable Textures, QuickDraw, FGVCx Fungi, VGG Flower, Traffic Signs, and MSCOCO. Eachdataset includes distinct training, validation, and testing splits. To prevent novel class leakage during pre-training, the experimental protocol set by previous research [29] was adhered to considering only the training split of ImageNet-1k for meta-training, with testing sets from all datasets used for meta-testing. For additional details on the Meta-Dataset, refer to Appendix 3 of the related study [28].