ICML 2025

Enhancing Performance of Explainable AI Models
with Constrained Concept Refinement
Geyu Liang 1 Senne Michielssen 2 Salar Fattahi 1
Abstract
The trade-off between accuracy and interpretabil-
ity has long been a challenge in machine learning
(ML). This tension is particularly significant for
emerging interpretable-by-design methods, which
aim to redesign ML algorithms for trustworthy in-
terpretability but often sacrifice accuracy in the
process. In this paper, we address this gap by
investigating the impact of deviations in concept
representations—an essential component of in-
terpretable models—on prediction performance
and propose a novel framework to mitigate these
effects. The framework builds on the principle
of optimizing concept embeddings under con-
straints that preserve interpretability. Using a gen-
erative model as a test-bed, we rigorously prove
that our algorithm achieves zero loss while pro-
gressively enhancing the interpretability of the
resulting model. Additionally, we evaluate the
practical performance of our proposed framework
in generating explainable predictions for image
classification tasks across various benchmarks.
Compared to existing explainable methods, our
approach not only improves prediction accuracy
while preserving model interpretability across var-
ious large-scale benchmarks but also achieves this
with significantly lower computational cost.
1. Introduction
ML algorithms are often caught in a dilemma between in-
terpretability and performance. Models such as linear re-
gression (Hastie, 2009) and decision trees (Quinlan, 1986)
provide straightforward interpretability through parameter
weights and rule-based predictions. However, they fre-
1Department of Industrial and Operations Engineering, Univer-
sity of Michigan, Ann Arbor, MI, US. 2Department of Computer
Science, Princeton University, Princeton, NJ, US. Correspondence
to: Salar Fattahi <fattahi@umich.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning...