## 4 Experiment
### Experimental Results  
**Transfer learning. ** We carried out experiments that froze particular model parameters during transfer learning to investigate how to effectively use the representations pretrained on a large-scale dataset. The comparative results for each dataset are listed in Table 1. The evaluation protocols of \(k\)-NN and linear probing were used to examine the quality of the representations. As shown in Table 1, \"Full FT\" demonstrates a degraded performance than \"w/o FT\" on most of the datasets. Specifically, linear probing performance decreased by an average of \(18.7\%\). However, we observed a slight performance improvement in the Food dataset, which had a relatively sufficient amount of training data. These results confirmed that updating all parameters during SSL with a small-scale dataset might degrade the performance. However, the overall \(k\)-NN and linear probing accuracy are slightly improved when only the layer normalization is fine-tuned using the target dataset. In the Food dataset, there was a performance improvement of \(2.1\%\)\(k\)-NN accuracy over \"w/o FT\". Additionally, we examined updating the last block, and the results indicated better performance than fine-tuning only the layer normalization. In particular, the accuracy on Cars increased by \(2.4\%\). These results demonstrate that updating only the last block is more effective \begin{table}
\begin{tabular}{l c c c c c c} \hline
**Transfer Learning (SSL)** & **Aircraft**[16] & **Car**[15] & **Flower**[39] & **Food**[40] & **CUB**[14] & **Dog**[41] \\ \hline w/o FT & 61.0 (36.7) & 65.8 (22.5) & 96.3 (86.5) & 79.8 (67.7) & **80.8 (69.5)** & **83.7 (77.3)** \\ Full FT & 14.0 (3.5) & 50.1 (14.5) & 84.5 (72.7) & 80.9 (72.7) & 56.9 (42.2) & 69.0 (62.2) \\ LayerNorm FT [13] & 60.0 (36.4) & 66.3 (231) & 96.9 (88.1) & 80.8 (69.8) & 76.9 (60.8) & 82.1 (74.0) \\ Lastblock FT & 62.3 (37.8) & 68.7 (25.8) & **97.2** (90.7) & **82.8** (75.7) & 76.1 (60.5) & 80.4 (73.6) \\ Lastblock FT + \(L_{\text{con}}\) & **63.6** (**39.2) & **68.9** (**26.8) & 96.9 (**90.9) & 82.6 (**75.9) & 77.1 (60.8) & 81.8 (74.2) \\ \hline \end{tabular}
\end{table} 
Table 1: Test accuracy on each dataset under linear probing (\(k\)-NN) evaluation. Bold font denotes the best-performing transfer learning method. The “w/o FT” refers to an initial model (i.e., the pretrained model with ImageNet), and the “Full FT” is a model that is fully fine-tuned via SSL on the target FGVC dataset. Note that the transfer learning process is conducted in a self-supervised manner, and the ground-truth labels are only used in the evaluation process. when fine-tuned with a small-scale dataset in SSL scenarios. However, the pretrained DINO performed best on the CUB and Dog datasets. We conjecture that because ImageNet contains many dog and bird images, the model can learn adequate representations without further fine-tuning. We applied the consistency loss term to fine-tune the last block to verify the effectiveness of the proposed consistency loss. The performance can be further enhanced when the consistency loss is combined with the DINO loss, as shown in the last row of Table 1. By considering patch-level consistency, the performance is increased over the absence of the consistency loss in every case except for the linear evaluation on the Flower and Food datasets. In addition, it is possible to mitigate the performance decrease caused by naive fine-tuning on the CUB and Dog datasets. Furthermore, a comparison of the confusion matrices of \"LayerNorm FT\" and the proposed method on the Aircraft dataset is presented in Figure 3. Positive values in the off-diagonal elements indicate that the misclassification of \"LayerNorm FT\" is greater than that of the proposed method. Similarly, negative values for the diagonal elements indicate that our proposed method achieves a higher correct classification rate compared with \"LayerNorm FT\".