## 4 Experiments
### Ablation Study
This pattern of dispersion is crucial, as it reflects the model's adeptness at upholding the distinctiveness of each sample, thereby safeguarding its generalizability. Such a delicate equilibrium between ensuring the specificity of individual samples and their aggregation into coherent clusters underscores the sophistication of our method. It affirms the model's robustness and adaptability in managing complex datasets, achieving a commendable harmony between the twin pillars of specificity and generalizability. \begin{table}
\begin{tabular}{l l c c c c c} \hline \hline
**Measurement** & **Method** & **CIFAR-100** & **TinyImageNet** & **CUB-200-2011** & **Stanford Dogs** & **MIT67** \\ \hline \multirow{9}{*}{Top-5 \(\downarrow\)} & Cross-entropy & 6.91\({}_{\pm 0.09}\) & 22.21\({}_{\pm 0.29}\) & 22.30\({}_{\pm 0.08}\) & 11.80\({}_{\pm 0.27}\) & 19.25\({}_{\pm 0.53}\) \\  & AdaCos & 9.99\({}_{\pm 0.20}\) & 22.24\({}_{\pm 0.11}\) & 15.24\({}_{\pm 0.06}\) & 11.02\({}_{\pm 0.22}\) & 19.05\({}_{\pm 0.23}\) \\  & Virtual-softmax & 8.54\({}_{\pm 0.11}\) & 24.15\({}_{\pm 0.17}\) & 13.16\({}_{\pm 0.20}\) & 8.64\({}_{\pm 0.21}\) & 19.10\({}_{\pm 0.20}\) \\  & Maximum-entropy & 7.29\({}_{\pm 0.11}\) & 21.53\({}_{\pm 0.08}\) & 19.80\({}_{\pm 1.12}\) & 10.90\({}_{\pm 0.31}\) & 20.47\({}_{\pm 0.00}\) \\  & Label-smoothing & 7.18\({}_{\pm 0.08}\) & 20.74\({}_{\pm 0.03}\) & 22.40\({}_{\pm 0.05}\) & 13.41\({}_{\pm 0.04}\) & 19.53\({}_{\pm 0.75}\) \\  & CS-KD & 5.69\({}_{\pm 0.03}\) & **19.21\({}_{\pm 0.04}\)** & 13.07\({}_{\pm 0.26}\) & 8.55\({}_{\pm 0.07}\) & 17.46\({}_{\pm 0.38}\) \\  & BAKE & 7.45\({}_{\pm 0.06}\) & 20.06\({}_{\pm 0.34}\) & 13.00\({}_{\pm 0.04}\) & 10.23\({}_{\pm 0.13}\) & 18.43\({}_{\pm 0.39}\) \\  & CSD (ours) & **5.34\({}_{\pm 0.09}\)** & 19.39\({}_{\pm 0.23}\) & **9.49\({}_{\pm 0.24}\)** & **7.09\({}_{\pm 0.27}\)** & **16.77\({}_{\pm 0.21}\)** \\ \hline \multirow{9}{*}{R@1 \(\uparrow\)} & Cross-entropy & 61.38\({}_{\pm 0.04}\) & 30.59\({}_{\pm 0.02}\) & 33.92\({}_{\pm 1.70}\) & 47.51\({}_{\pm 1.02}\) & 31.42\({}_{\pm 1.00}\) \\  & AdaCos & 67.95\({}_{\pm 0.02}\) & 44.66\({}_{\pm 0.05}\) & 54.86\({}_{\pm 0.24}\) & 58.37\({}_{\pm 0.43}\) & 42.39\({}_{\pm 1.91}\) \\  & Virtual-softmax & 68.35\({}_{\pm 0.08}\) & 44.69\({}_{\pm 0.08}\) & 55.56\({}_{\pm 0.24}\) & 59.71\({}_{\pm 0.56}\) & 44.20\({}_{\pm 0.09}\) \\  & Maximum-entropy & 71.51\({}_{\pm 0.29}\) & 39.18\({}_{\pm 0.07}\) & 48.66\({}_{\pm 2.10}\) & 60.05\({}_{\pm 0.45}\) & 38.06\({}_{\pm 3.32}\) \\  & Label-smoothing & 71.44\({}_{\pm 0.03}\) & 34.79\({}_{\pm 0.07}\) & 41.59\({}_{\pm 0.04}\) & 54.48\({}_{\pm 0.68}\) & 35.15\({}_{\pm 1.54}\) \\  & CS-KD & 71.15\({}_{\pm 0.15}\) & 47.15\({}_{\pm 0.04}\) & 59.06\({}_{\pm 0.38}\) & 62.67\({}_{\pm 0.07}\) & 46.74\({}_{\pm 1.48}\) \\  & BAKE & 71.24\({}_{\pm 0.66}\) & 45.23\({}_{\pm 0.34}\) & 62.55\({}_{\pm 0.01}\) & **64.72\({}_{\pm 0.43}\)** & 46.12\({}_{\pm 0.65}\) \\  & CSD (ours) & **72.22\({}_{\pm 0.21}\)** & **48.05\({}_{\pm 0.47}\)** & **65.85\({}_{\pm 0.32}\)** & 63.97\({}_{\pm 0.62}\) & **50.65\({}_{\pm 1.00}\)** \\ \hline \multirow{9}{*}{Mstd} & Cross-entropy & 17.24\({}_{\pm 1.23}\) & 11.84\({}_{\pm 0.06}\) & 9.26\({}_{\pm 0.06}\) & 8.96\({}_{\pm 0.35}\) & 6.52\({}_{\pm 0.72}\) \\  & Label-smoothing & 5.54\({}_{\pm 0.45}\) & 8.49\({}_{\pm 0.41}\) & 5.68\({}_{\pm 0.38}\) & 4.93\({}_{\pm 0.26}\) & 3.95\({}_{\pm 0.65}\) \\ \cline{1-1}  & CS-KD & 5.91\({}_{\pm 0.34}\) & 5.92\({}_{\pm 0.64}\) & 4.18\({}_{\pm 0.29}\) & 3.64\({}_{\pm 0.66}\) & 2.91\({}_{\pm 0.31}\) \\ \cline{1-1}  & BAKE & 3.85\({}_{\pm 0.32}\) & 5.42\({}_{\pm 0.48}\) & 3.83\({}_{\pm 0.29}\) & 2.70\({}_{\pm 0.19}\) & 2.19\({}_{\pm 0.11}\) \\ \cline{1-1}  & CSD (ours) & 5.84\({}_{\pm 0.16}\) & 7.17\({}_{\pm 0.7}\) & 4.87\({}_{\pm 0.09}\) & 4.47\({}_{\pm 0.29}\) & 3.36\({}_{\pm 0.33}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Top-5 error, recall at 1 (R@1) rates (%), and Mstd of ResNet-18 on various image classification tasks. The arrow on the right side of the evaluation metric indicates ascending or descending orders of the value.