Title: Keypoint Counting Classifiers: Turning Vision Transformers into Self-Explainable Models Without Training
Primary area: interpretability and explainable AI
Keywords: self explainability, vision transformers, keypoints, vision language

Abstract: Current approaches for designing self-explainable models (SEMs) require complicated training procedures and specific architectures that makes them impractical. With the advance of general purpose foundation models based on Vision Transformers (ViTs), this impracticability becomes even more problematic. Therefore, new methods are necessary to provide transparency and reliability to ViT-based foundation models. In this work, we present a new method for turning any well-trained ViT-based model into a SEM without retraining, which we call Keypoint Counting Classifiers (KCCs). Recent works have shown that ViTs can automatically identify matching keypoints between images with high precision, and we build on these results to create an easily interpretable decision process that is inherently visualizable in the input. We perform an extensive evaluation of quantitative measures and human user studies, which show that KCCS improve the human-machine communication compared to recent baselines. We believe that KCCs constitute an important step towards making ViT-based foundation models more transparent and reliable. Code is available at _anonymised_ and in the supplementary.

TL;DR: Turning vision transformers into self-explainable models without training and with improved input visualisation through key points.