Title: CHiQPM: Calibrated Hierarchical Interpretable Image Classification
Venue: NeurIPS 2025
Accepted: yes
Keywords: XAI

Global Interpretability

Local Interpretability

Conformal Prediction

Image Classification
Abstract: Globally interpretable models are a promising approach for trustworthy AI in safety-critical domains. Alongside global explanations, detailed local explanations are a crucial complement to effectively support human experts during inference. This work proposes the Calibrated Hierarchical QPM (CHiQPM) which offers uniquely comprehensive global and local interpretability, paving the way for human-AI complementarity. CHiQPM achieves superior global interpretability by contrastively explaining the majority of classes and offers novel hierarchical explanations that are more similar to how humans reason and can be traversed to offer a built-in interpretable Conformal prediction (CP) method. Our comprehensive evaluation shows that CHiQPM achieves state-of-the-art accuracy as a point predictor, maintaining 99% accuracy of non-interpretable models. This demonstrates a substantial improvement, where interpretability is incorporated without sacrificing overall accuracy. Furthermore, its calibrated set prediction is competitively efficient to other CP methods, while providing interpretable predictions of coherent sets along its hierarchical explanation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a globally and locally interpretable image classification model that builds upon QPM with improved contrastive class representations and hierarchical explanations. CHiQPM encourages class-wise feature sharing and introduces a built-in interpretable conformal prediction mechanism by traversing explanation trees formed from shared feature activations. Experiments on CUB, Stanford Cars, and ImageNet-1K demonstrate that CHiQPM maintains high accuracy while offering calibrated predictions.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The paper aims to provide global explanations that are encouraged, especially the design of interpretable models rather than relying on post-hoc explanations.\n2. The idea of inducing hierarchical explanation paths by enforcing shared features between similar classes is novel.\n3. The paper is easy to follow, making the objectives accessible to a broad audience.\n\n**Weaknesses**\n1. The model heavily relies on learned mid-level features, which are not guaranteed to correspond to human-interpretable concepts. While the authors introduce a contrastive objective and grounding loss, these only encourage sparsity but not semantic alignment. This leaves the interpretation paths inherently fragile and possibly misleading. Moreover, relying on a small set of local features per class may hurt performance, especially if global context is important.\n2. The proposed framework assumes that similar classes will share a few features. However, for more abstract or complex classification tasks, a larger number of features per class may be necessary. As the feature pool expands, feature overlap between classes may decrease, undermining the construction of hierarchical structures and weakening the interpretability claims.\n3. The paper lacks analysis of how the depth and width of the explanation trees vary under different configurations (e.g., feature pool size or class granularity). Such ablations are crucial to understand how robust or shallow the constructed explanation paths truly are.\n\nQUESTIONS:\n1. How is the interpretability of local features ensured? While contrastiveness is enforced, it does not imply semantic consistency. How do we know the activated features correspond to concepts meaningful to humans?\n2. What is the generalization ability of the model structure? If applied to a complex dataset with highly abstract or noisy classes, how would CHiQPM perform? What happens if class pairs do not share enough features to construct a meaningful hierarchy?\n3. How does CHiQPM compare with prototype methods like ProtoTree [Nauta et al., 2021]? What are the key advantages of your approach?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nQPM introduce a way to make convolutional networks globally interpretable by representing each class as the logical \"or\" of a very small, fixed set of feature maps. \n\nThis yields ultra‐compact, class‐wise explanations but lacks fine‐grained, instance‐level detail and makes no provision for calibrated uncertainty in predictions. CHiQPM, builds directly on QPM:\n\n(a) The discrete quadratic program is augmented to force many class pairs to share nwc – 1 features, creating a tree‐like hierarchy of explanations rather than a flat, per‐class OR.\n\n(b) After solving the QP, the model is fine-tuned with a novel Feature Grounding Loss and ReLU-based regularization that:\n\n(i) equalizes feature activation strength within each class representation and \n\n(ii) suppresses spurious activations, dramatically improving how well features align to human‐understandable concepts.\n\n\n(c) By traversing the learned hierarchy and using a tailored nonconformity score, CHiQPM produces calibrated prediction sets whose size meets a desired coverage target while keeping the returned classes semantically coherent (e.g. birds of similar shape), a capability missing in both QPM and flat set predictors.\n\nExperiments on CUB‑2011, Stanford Cars and ImageNet‑1K across four backbones show (a) accuracy kept similar to a dense CNN (b) higher structural grounding than QPM.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. CHiQPM builds on QPM’s adds a hierarchical tree structure, giving both a concise global summary and sample-specific decision paths.\n2. The discrete QP formulation forces most class pairs to share nwc – 1 features, yielding an explanation hierarchy rather than a flat feature assignment.\n3. Introducing L₍feat₎ plus a ReLU-based fine-tuning step sharply improves alignment between learned features and human concepts.\n4. Strong experiments\n\nWeaknesses:\n1. Point‐accuracy improvements over QPM are very small and even negative on some backbones, limiting appeal outside interpretability‐focused audiences.\n2. Dense notation (e.g. reusing “M” for multiple matrices) and scattered rationale for key design decisions (why ReLU vs. soft-threshold) hinder readability.\n\nQUESTIONS:\n1. Are there other hierarchy construction choices? \n2. Grounding is measured via concept annotations; however, do humans actually find the hierarchical paths helpful? A small-scale user study (e.g 20 domain experts evaluating 100 samples) could validate the claimed usability of the explanations.\n3. What is the sensitivity to λ_feat and maximum depth of the hierarchy? Please add a grid.\n4. Are there limitations regarding class imbalance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This work introduces the Calibrated Hierarchical QPM (CHiQPM). The strength of CHiQPM lies in its ability to provide both global and local explanations. The model employs contrastive explanations for majority classes and offers hierarchical explanations that align with human reasoning; a unique feature that also enables interpretable conformal prediction (CP) through traversable decision paths. Empirical results demonstrate that CHiQPM achieves near-state-of-the-art accuracy (retaining 99% of non-interpretable models' performance) while maintaining interpretability.

STRENGTHS AND WEAKNESSES:
Pros:
1. the idea of integrating global, local XAI and CP apears to be novel.
2. the paper is technically sound.

Cons:
1. while the overall idea is novel, the work is incremental to the original QPM work.
2. the writing is a bit hard to follow if without prior knowldege of QPM (i.e., not self-contained), I have to quick read the QPM paper as well.
3. the evaluation miss some common metrics for local XAI, e.g., delete/insert scores, Pointing Game;

QUESTIONS:
1. Given the proposed method can do both global and local explanations; if the author consider to compare with some common XAI benchmarks, e.g., OpenXAI.

2. Please provide some discussions around scalability of the proposed approach.

3. In practice, can the authors give some examples of how a conformal set of features is better than, say, the use of top-1 or top-k features for decision-making.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 1
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces Calibrated Hierarchical QPM (CHiQPM), an interpretable image classification framework that merges the compact global explanations of Quadratic Prototype Models (QPM) with *hierarchical* local explanations and built-in *conformal prediction* (CP).\nAt training time, a dense backbone network is first diversified, after which a discrete quadratic program with new hierarchy constraints selects a small, fixed set of concept-like features per class. Those features are then fine-tuned with a Feature Grounding Loss and an additional ReLU so that each activates only on a single, human-aligned visual concept. Finally, the model is *calibrated* by CP, enabling it to traverse its sample-specific hierarchy and output coherent class sets.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The proposed pipeline is easy to follow.\n- Non-trivial empirical gains while keeping backbone accuracy.\n- Interpretable hierarchy visualizations and qualitative examples. Figures illustrate that the learned tree edges correspond to semantic super-/sub-classes and that prediction sets indeed trace these edges upward when uncertainty is high.\n\n## Weaknesses\n- Essential steps lack any formal analysis. There is no approximation bound, convergence argument, or sample-complexity discussion. Theoretical novelty is therefore thin.\n- Since the proposed method appears to be somewhat sensitive to a hyper-parameter $\\rho$, theoretical discussions on the optimality of hyper-parameter selection strengthen this study. Moreover, the authors set $\\lambda_{\\text{feat}} = 3$ in their numerical experiments, but there are no discussion on how to choose this value, and no justification.\n- Exchangeability is assumed in the manuscript, but there is no discussion as to when this assumption is satisfied and when it is violated (how realistic an assumption it is). At least there should be a discussion of the assumptions made with respect to the experimental setup reported by the authors.\n\nQUESTIONS:\n## Questions\n- Can you provide formal statements for the proposed method as raised in the above?\n- Justification of hyper-parameter selection is needed. Can you give more discussion?\n- Can you state whether the reported experimental settings satisfy the assumption? and can you provide several counterexamples?\n\n## Additional suggestions\n- Matrix \\textit{M} denotes a feature map, but the letter $M$ is reused for class-class similarity, risking confusion.\n- L655: “calibrated cohrerent set predictions” → coherent\n- L839: “These visuzalizations demonstrate…” → visualizations\n- “we temporarily save every model … to speed up metric **calculation**.” The preceding sentence says *metrics* (plural). Singular/plural mismatch.\n- “auxiliary Feature Diversity Loss $\\mathcal{L}_{\\text{div}}$ [33]” – no definition or formula is given in the current manuscript. Readers must look up ref. [33].\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}