# Advancing Model Explainability: Visual Concept Knowledge Distillation for Concept Bottleneck Models
## 6 Conclusions  
This study proposes a novel knowledge distillation method, VICO-KD, designed for training CBMs. Our approach harnesses the potential of visual concepts to bolster the transfer of knowledge from a teacher model to a student model, thereby enhancing task accuracy, concept prediction performance, and model interpretability. By explicitly integrating visual concepts into the knowledge distillation framework, VICO-KD facilitates the student model's acquisition of more meaningful and discriminative features, aligning them more closely with human-interpretable concepts. Our proposed VICO-KD method exhibits substantial enhancements over Vanilla-KD when applied to benchmark datasets annotated with concepts. Specifically, it achieves performance improvements of 10.65%, 2.9%, and 0.3% on the CUB, AwA2, and CelebA datasets, respectively. By visualizing and comparing saliency maps, we verify that the CBM trained using VICO-KD acquires visual concepts closely resembling those learned by the teacher CBM. This observation underscores the capacity of our method to capture not only simple explicit concepts but also more intricate visual concepts, thereby further validating its efficacy. Through thorough experimentation on benchmark datasets, we substantiate the effectiveness of VICO-KD when compared to existing concept models and logit-based knowledge distillation. Our method attains state-of-the-art performance while ensuring a consistent learning process and fostering more resilient and interpretable visual representations. Consequently, VICO-KD constitutes a considerable advancement in the evolution of interpretable and efficient computer vision models, laying the groundwork for more transparent and dependable AI systems. While our VICO-KD method demonstrates promising results, future research should focus on addressing potential limitations. Specifically, we aim to develop more robust techniques for mitigating dataset biases and enhancing model fairness. This includes implementing advanced data augmentation strategies, creating comprehensive bias detection frameworks, and designing methodologies that can systematically identify and correct representational skews in concept-based machine learning models. Moreover, we recognize the critical need to improve model resilience against real-world challenges. Future work will explore adversarial robustness training, noise tolerance mechanisms, and uncertainty quantification techniques. These efforts will not only refine the VICO-KD approach but also contribute to developing more transparent, reliable, and interpretable artificial intelligence systems. Conceptualization, J.-H.L. ; methodology, J.-H.L. ; software, J.-H.L. ; investigation, J.-H.L. and I.-H.S. ; resources, J.-Y.K. ; data curation, J.-H.L. ; writing--original draft preparation, J.-H.L. ; writing--review and editing, D.T.V. and J.-Y.K. ; visualization, J.-H.L. ; project administration, N.-K.L. ; funding acquisition, N.-K.L. All authors have read and agreed to the published version of the manuscript. This work was partly supported by Innovative Human Resource Development for Local Intellectualization program through the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (IITP-2024-00156287, 30%) and Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2021-0-02068, Artificial Intelligence Innovation Hub, 30%), Korea government (MSIT) (2022-0-00215, 40%). Not applicable. The CUB-200-201 dataset can be found at [[https://www.vision.caltech.edu/datasets/cub_200_2011/](https://www.vision.caltech.edu/datasets/cub_200_2011/)]([https://www.vision.caltech.edu/datasets/cub_200_2011/](https://www.vision.caltech.edu/datasets/cub_200_2011/)) (accessed on 1 March 2024).