## 4 Experiments
### Quantitative Results
\begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**Model** & **Method** & **CIFAR-100** & **TinyImageNet** & **CUB-200-2011** & **Stanford Dogs** & **MIT67** \\ \hline \multirow{8}{*}{ResNet-18} & Cross-entropy & 24.71\({}_{\pm 0.34}\) & 43.53\({}_{\pm 0.19}\) & 46.00\({}_{\pm 1.03}\) & 36.29\({}_{\pm 0.32}\) & 44.75\({}_{\pm 0.80}\) \\ \cline{2-6}  & AdaCos & 23.71\({}_{\pm 0.36}\) & 42.61\({}_{\pm 0.20}\) & 35.47\({}_{\pm 0.07}\) & 32.66\({}_{\pm 0.34}\) & 42.66\({}_{\pm 0.40}\) \\  & Virtual-softmax & 23.01\({}_{\pm 0.42}\) & 42.41\({}_{\pm 0.20}\) & 35.03\({}_{\pm 0.51}\) & 31.48\({}_{\pm 0.16}\) & 42.86\({}_{\pm 0.71}\) \\  & Maximum-entropy & 22.72\({}_{\pm 0.28}\) & 41.77\({}_{\pm 0.13}\) & 39.86\({}_{\pm 1.11}\) & 32.41\({}_{\pm 0.20}\) & 43.36\({}_{\pm 1.62}\) \\ \cline{2-6}  & DDGSD & 23.85\({}_{\pm 1.97}\) & 41.48\({}_{\pm 0.12}\) & 41.17\({}_{\pm 1.28}\) & 31.53\({}_{\pm 0.54}\) & 41.17\({}_{\pm 2.62}\) \\  & BYOT & 23.81\({}_{\pm 0.11}\) & 44.88\({}_{\pm 0.46}\) & 40.76\({}_{\pm 0.39}\) & 34.02\({}_{\pm 0.14}\) & 44.88\({}_{\pm 0.46}\) \\ \cline{2-6}  & Label-smoothing & 22.69\({}_{\pm 0.28}\) & 43.09\({}_{\pm 0.34}\) & 42.99\({}_{\pm 0.39}\) & 35.30\({}_{\pm 0.66}\) & 44.40\({}_{\pm 0.71}\) \\  & Center-loss & 22.89\({}_{\pm 0.46}\) & 42.43\({}_{\pm 0.14}\) & 37.21\({}_{\pm 0.13}\) & 36.07\({}_{\pm 0.28}\) & 44.41\({}_{\pm 0.56}\) \\  & CS-KD & 21.99\({}_{\pm 0.13}\) & 41.62\({}_{\pm 0.38}\) & 33.28\({}_{\pm 0.99}\) & 30.85\({}_{\pm 0.28}\) & 40.45\({}_{\pm 0.45}\) \\  & Bake & 21.28\({}_{\pm 0.15}\) & 41.71\({}_{\pm 0.21}\) & 29.74\({}_{\pm 0.70}\) & 30.20\({}_{\pm 0.11}\) & 39.95\({}_{\pm 0.11}\) \\ \cline{2-6}  & CSD (ours) & **21.19 \({}_{\pm 0.05}\)** & **40.98 \({}_{\pm 0.23}\)** & **28.03 \({}_{\pm 0.08}\)** & **28.84 \({}_{\pm 0.58}\)** & **37.96 \({}_{\pm 0.82}\)** \\ \hline \multirow{8}{*}{DeseNet-121} & Cross-entropy & 22.23\({}_{\pm 0.04}\) & 39.22\({}_{\pm 0.27}\) & 42.30\({}_{\pm 0.44}\) & 33.39\({}_{\pm 0.17}\) & 41.79\({}_{\pm 0.19}\) \\ \cline{2-6}  & AdaCos & 22.17\({}_{\pm 0.24}\) & 38.76\({}_{\pm 0.23}\) & 30.84\({}_{\pm 0.38}\) & 27.87\({}_{\pm 0.65}\) & 40.25\({}_{\pm 0.68}\) \\ \cline{1-1}  & Virtual-softmax & 23.66\({}_{\pm 0.10}\) & 41.58\({}_{\pm 1.58}\) & 33.85\({}_{\pm 0.75}\) & 30.55\({}_{\pm 0.72}\) & 43.66\({}_{\pm 0.30}\) \\ \cline{1-1}  & Maximum-entropy & 22.87\({}_{\pm 0.45}\) & 38.39\({}_{\pm 0.33}\) & 37.51\({}_{\pm 0.71}\) & 29.52\({}_{\pm 0.74}\) & 43.48\({}_{\pm 1.30}\) \\ \cline{1-1} \cline{2-6}  & Label-smoothing & 21.88\({}_{\pm 0.45}\) & 38.75\({}_{\pm 0.18}\) & 40.63\({}_{\pm 0.24}\) & 31.39\({}_{\pm 0.46}\) & 42.24\({}_{\pm 1.23}\) \\ \cline{1-1}  & CS-KD & 21.69\({}_{\pm 0.46}\) & 37.96\({}_{\pm 0.09}\) & 30.83\({}_{\pm 0.39}\) & 27.81\({}_{\pm 0.13}\) & 40.02\({}_{\pm 0.91}\) \\ \cline{1-1}  & Bake & 20.74\({}_{\pm 0.19}\) & 37.07\({}_{\pm 0.24}\) & 28.79\({}_{\pm 1.30}\) & 27.66\({}_{\pm 0.05}\) & 39.15\({}_{\pm 0.37}\) \\ \cline{1-1} \cline{2-6}  & CSD (ours) & **20.27 \({}_{\pm 0.39}\)** & **37.05 \({}_{\pm 0.29}\)** & **27.23 \({}_{\pm 0.40}\)** & **26.57 \({}_{\pm 0.82}\)** & **36.08 \({}_{\pm 0.46}\)** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Top-1 error rates (%) on various image classification tasks and model architectures. The best results are in bold font. fuses the sample knowledge as the teacher. They outperform the above methods but also introduce complex sampling mechanisms. Our CSD explores the underlying reasons why they work and simplifies such mechanisms accordingly, outperforming them on all data sets. Comparing the three methods, CS-KD learns from another sample, BAKE learns from multiple samples at the same time, while our method goes a step further by learning from all the samples within a class at the same time, greatly exploiting the potential of samples to learn from each other. (4) _Label smoothing_[18]. Label smoothing is effectively a subset of our method, where class centers are considered static.