# A Closer Look at Few-Shot Classification with Many Novel Classes
## 5 Experiments
### Analysis of Computation Overhead  
The SHA-Pipeline can be explicitly divided into two phases for analysis: the meta-training phase and the meta-testing phase. Firstly, the computational overhead during the meta-training phase is analyzed. As shown in Figure 6, our lightweight parallel framework effectively reduces computation time, achieving up to a 150-fold parallel acceleration ratio (when \(N=2560\)). Additionally, traditional parallel frameworks often face limitations due to exceeding GPU memory capacity when \(N>40\), but our framework can smoothly scale meta-training up to \(N=2560\). \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c} \hline \hline ID & Arch & Pre-Train & MetaTr & Shot & \multicolumn{8}{c}{**Way**} & \multicolumn{8}{c}{**Average**} \\ \cline{6-13}  & & & & 5 & 10 & 20 & 40 & 80 & 160 & 320 & 640 & 1000 & 1280 & 2560 \\ \hline
1 & & & & 5 & 10 & 20 & 40 & 80 & 160 & 320 & 640 & 1000 & 1280 & 2560 \\  
In the meta-testing phase, the experiment focuses on analyzing the computation time and memory overhead of SHA-Pipeline when handling sets of categories of different sizes. As the number of categories increases, so does the computational demand; however, by employing a lightweight parallel framework, SHA-Pipeline effectively reduces this overhead. Specifically, when the number of categories \(N\geq 160\), SHA-Pipeline achieves a super-linear speedup in meta-testing computation time, as shown in Figure 7. This is because the computation time is masked by the time taken to load the dataset into the GPU memory. Furthermore, the parallel processing of SHA-Pipeline not only optimizes computation time but also significantly reduces memory pressure by efficiently allocating and managing GPU resources. This efficient resource management ensures that the system remains stable even in scenarios with a very large number of categories, avoiding performance bottlenecks due to resource limitations. The memory overhead of the SHA-Pipeline during the meta-testing phase is illustrated in Figure 8. It is noted that ProtoNet, without the lightweight parallel framework of this experiment, exceeds the 40 G memory limit of a single GPU when the number of categories \(N\) surpasses 80. However, the distributed architecture adopted in this experiment effectively mitigates the memory costs associated with larger values of \(N\). Data from Figure 8 show a consistent upward trend in memory overhead as the number of categories increases. Notably, when \(N\) reaches 320, the memory consumption becomes extremely large, exceeding 2560 G, which is significantly beyond the practical  
Figure 6: Computation overhead (1 shot, ViT-S, DINO). Figure 7: Computation Overhead of SHA-Pipeline during meta-testing (DINO, ViT-S, 15 query samples). operational limits of this experiment (40 G/GPU multiplied by 64 GPUs equals 2560 G), making it challenging to continue the experiment without special measures. To address this challenge, the experiment further divides the query set into minibatches for forward propagation on each GPU. This approach allows for incremental backward propagation and synchronization updates on these smaller minibatches, effectively reducing memory requirements during training. This strategy not only improves memory utilization efficiency but also ensures that experiments during the meta-testing phase can continue efficiently even with a very large number of categories. The size of these minibatches is 256. Thus, the memory overhead is approximately equal when \(N\in 320,640,1000,1280,2560\). Moreover, this innovative lightweight parallel processing framework not only optimizes memory consumption but also significantly enhances computational efficiency, ensuring stability and high performance for the SHA-Pipeline even in demanding FSL-MNC scenarios. Through this technique, SHA-Pipeline not only addresses the issue of high memory costs but also enhances the model's scalability and flexibility when handling large datasets, providing robust support for broader practical applications in the future.