Title: ProtoPairNet: Interpretable Regression through Prototypical Pair Reasoning
Venue: NeurIPS 2025
Accepted: yes
Keywords: deep learning

interpretability

prototype-based neural network

case-based reasoning
Abstract: We present Prototypical Pair Network (ProtoPairNet), a novel interpretable architecture that combines deep learning with case-based reasoning to predict continuous targets. While prototype-based models have primarily addressed image classification with discrete outputs, extending these methods to continuous targets, such as regression, poses significant challenges. Existing architectures which rely heavily on one-to-one comparison with prototypes lack the directional information necessary for continuous predictions. Our method redefines the role of prototypes in such tasks by incorporating prototypical pairs into the reasoning process. Predictions are derived based on the input's relative dissimilarities to these pairs, leveraging an intuitive geometric interpretation. Our method further reduces the complexity of the reasoning process by relying on the single most relevant pair of prototypes, rather than all prototypes in the model as was done in prior works. Our model is versatile enough to be used in both vision-based regression and continuous control in reinforcement learning. Our experiments demonstrate that ProtoPairNet achieves performance on par with its black-box counterparts across these tasks. Comprehensive analyses confirm the meaningfulness of prototypical pairs and the faithfulness of our model’s interpretations, and extensive user studies highlight our model's improved interpretability over existing methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose ProtoPairNet as an interpretable approach to regression through prototypical pair reasoning. Experiments show the utility of the proposed approach in both computer vision and reinforcement learning settings. Several human studies are conducted that further highlight the benefits of the pair based approach.\n\nSTRENGTHS AND WEAKNESSES:\nPrototype based approaches are uncommon in regression scenarios and solutions would be a welcome and useful contribution. Modernising case-based solutions are one of Rudin’s 10 Grand Challenges for interpretable machine learning.\n\nThe approaches and comparisons are reasonably well grounded in the literature. However, there is no discussion of seminal work on prototypes and criticisms [1]. This is especially relevant as the user study in this prior work also shows the benefits of using a prototype and another example (a criticism in the case of [1]). \n\nThere is a theoretical contribution for both prediction and pair relevance. The design is (for the most part) intuitive and I liked the geometric interpretation. \n\nWhile the inclusion of a user study is important, there are several and design questions and flaws. How was the number of participants determined? It does not seem like there was a power analysis conducted to determine this? It does not seem like the experiment was controlled for the number of items presented to users. Would it not have made more sense to compare a prototypical pair against either/both (i) a single prototype and a random prototype (ii) two random prototypes. The problem is that with the current design, the number of items presented to users is not fixed and could therefore be considered a confounding variable. Were there attention checks? This is typically considered best practice in user study design. The figures in the appendix that show the set-up (e.g., figure 11) should probably go in the main text for clarity. In fact, the appendix was quite a big document but it was rarely referred to in the main text. \n\n\n[1] Kim, B., Khanna, R. and Koyejo, O.O., 2016. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems, 29.\n\nQUESTIONS:\n- Can you provide some more clarity on the assumptions of the prediction branch? It would be nice to know of scenarios when these assumptions would break so I can better understand the assumptions themselves. Would the labels not always vary linearly along the k-th ProtoPair axis?\n\n- Is one prototype from the pair always on either side of the projected input image? \n\n- What is the motivation for the diversity criterion? Do you not want the prototypes to be always similar to the input image? Was training quite difficult given the number of terms in the loss function?\n\n- What were the limitations of the user study? \n\n- Why do you not use a part-based framework, given the success of this approach in other works.\n\nLIMITATIONS:\n- Authors could go into more detail about the computational advantages or drawbacks of the proposed approach. There is related work on prototypes and criticisms, I also wonder if using maximum mean discrepancy would have advantages over the k-means based approach used here?\n\n- Not using a part-based framework is a notable weakness. The authors mention this as a weakness in the limitations but I think some rationale is needed here given that most developments in the field recognise the benefits of part-based approaches. \n\n- Authors mention code upon acceptance but I think it would actually have been useful to provide the anonymised code in the supplement or on one of the anonymous archives as this would give further clarity on methodological approaches. \n\n- User study design (see weaknesses)", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-20"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes **Prototypical Pair Network (ProtoPairNet)**, a novel model belonging to the class of prototype-based models, which offers the promise of interpretability versus standard black-box models.\n\nProtoPairNet tackles the regression problem instead of the classification problem, which has been the dominant focus of the community. While some prior works have explored regression with prototype-based models, this paper argues that those approaches suffer from one of the following limitations: (1) they rely on similarity-weighted averaging, which introduces interference from unrelated prototypes, or (2) they sacrifice interpretability.\n\nTo address this, the paper introduces two key innovations:\n\n(1) Instead of using single instances as prototypes, the model uses **pairs** of instances, such that the regression output varies linearly along the line connecting the two instances.\n\n(2) Given a test input, the model computes relevance scores with respect to a library of prototype pairs, selects one using a soft mechanism (Gumbel-Softmax), and applies a linear interpolation formula to compute the output from the labels of the selected prototype pair.\n\nIn experiments, the method performs competitively with black-box models and outperforms other previous prototype-based models. User studies also suggest that the model’s reasoning process aligns with that of human participants.\n\nSTRENGTHS AND WEAKNESSES:\n## Pros\n\n1. The paper is well-written and easy to follow.\n2. The paper is appropriately scoped.\n3. The idea of using prototype pairs is novel.\n\n## Cons\n\n1. The motivation for using pairs versus single prototypes is somewhat unclear. The issue of interference from irrelevant prototypes could be addressed using Gumbel-Softmax alone. So, what necessitates the use of *pairs* of prototypes?\n2. If the intent is to increase expressiveness, it would be nice to have an ablation that varies the tuple size from 1 to 2 and to perhaps even higher-order tuples. Additionally, the following aspects are unclear to me:\n    \n    (1) How are the pairs chosen to form the prototypes?\n    \n    (2) Are these prototypes decided during training, or are they provided at test time?\n    \n    (3) Does the method explicitly enforce the “linearity” of the target function with respect to the latent representations of the prototype pairs?\n    \n\nSee also the “Questions”.\n\nQUESTIONS:\n## Questions\n\nQ. Why is now a timely moment for this paper? All the ingredients motivating this work seem to have existed already a few years ago.\n\nQ. How are prototype pairs decided? How is the pairing done?\n\nQ. I tend to believe that interpretability is usually gained at the expense of flexibility, so one would expect prototype models (which are interpretable) to perform worse than the black-box models -- which isn't a bad thing, but just something I would expect. However, Table 1 suggests the opposite, and also, it appears that ProtoPairNet even outperforms black-box PPO. I was wondering if the authors have an explanation about this? Would including additional benchmarks help here?\n\nQ. Have the authors tried using any pre-trained backbones?\n\nQ. If the perpendicular drop from the input latent to the prototype axis lies outside the segment formed by the prototype pair, how is this case handled?\n\n## Suggestions for Improvement\n\nAn ablation of the three “metrics” and the various loss terms would be informative.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ProtoPairNet, a prototype-based framework for interpretable regression. Unlike traditional prototype models that rely on one-to-one comparisons, ProtoPairNet compares inputs to pairs of prototypes, enabling predictions based on the input’s position along the axis between two reference examples. This approach provides directional information needed for accurate continuous predictions. The model also uses a relevance mechanism to select the most appropriate prototype pair for each input, ensuring sparse and interpretable decision-making. Additionally, the authors propose a three-stage training strategy, including a projection of prototypes to real data samples for better visualization. ProtoPairNet is evaluated on age prediction and reinforcement learning-based control tasks, demonstrating competitive performance and improved interpretability over prior methods, as confirmed by multiple user studies.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Interesting approach for prototypical regression: Prototype-based models have been widely used for classification, but very few extend to regression. ProtoPairNet addresses this by introducing a structured, interpretable framework for continuous prediction tasks. This makes it a novel and valuable contribution to the field of explainable AI.\n- Interpretable Prototypes: Prototypes in ProtoPairNet are projected onto actual training instances, making them easy to visualize and understand. Unlike abstract latent vectors, these prototypes can be shown as real images. This adds transparency and helps users intuitively understand how the model arrived at its prediction.\n- Backbone compatibility: ProtoPairNet is compatible with various standard architectures like ResNet, EfficientNet, and Vision Transformers. This flexibility makes it easy to apply to various domains without redesigning the core model from scratch. This modular design also allows for future extensions.\n- Selection mechanism for explanation clarity: ProtoPairNet uses a selection mechanism to select the most relevant prototype pair for each prediction. This reduces noisy explanations and ensures that the reasoning remains focused and interpretable. \n\nWeaknesses\n- Limited benchmarking: ProtoPairNet is only tested on two tasks: age prediction and a car racing control task. These are relatively constrained problems that may not reflect the challenges of more complex, noisy, or high-dimensional regression domains. Without broader evaluation, the model’s generalizability remains uncertain.\n- Global explanations: Unlike part-prototype models that highlight specific input regions (e.g. facial features), ProtoPairNet uses global prototypes (entire images). This means it cannot indicate which parts of an input were most important for the prediction. As a result, it lacks fine-grained, spatial explanations, which limits its usefulness and interpretability, as we cannot be sure which features are contributing to the prototypical reasoning. It also makes the interpretation subjective to the user, which might be misaligned with how the model really works.\n- Limited use of multiple pairs per prediction: The model predicts using only one prototype pair per input. While this improves interpretability, it may not capture all relevant aspects of the input when multiple semantic factors are relevant. This reasoning could reduce prediction quality in more complex scenarios.\n\nQUESTIONS:\n1. Table 2: ProtoPariNet seems to significantly outperform other architectures. Are those SOTA architectures, and were they fully fine-tuned for this task?\n2. Could you propose more complex benchmark for SOTA comparison? Age prediction seems not sufficient to decide the generalizability of this architecture.\n3. MAE and R^2 seem like a limited performance metrics. It would be nice to see a distribution of prediction error, or more exhaustive metrics that describe it (along with comparison to other models).\n4. Would be nice to see some attribution method (like LRP or Integrated Gradients) to justify the explanation reasoning (Is the model really focusing on age, or maybe something else?)\n\nLIMITATIONS:\nThe authors have noted the limitation concerning global prototypes rather than prototype-part based ones (or attribution-based). However, my concern lies with the benchmarking, specifically the datasets and evaluation metrics employed, as well as the generalizability of the approach to more complex scenarios.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ProtoPairNet, a novel interpretable architecture that combines deep learning with case-based reasoning to predict continuous targets, extending prototype-based models from solving classification tasks with discrete outputs to continuous targets such as regression tasks. \nSpecifically, ProtoPairNet leverages prototypical pairs as ProtoPair axis in the latent space: given an input, it makes predictions based on the relative positioning along the ProtoPair axis defined by its most relevant prototype pair. The architecture comprises 1) a prediction branch that uses geometric projection onto axis between prototypical pairs, and 2) a pair relevance branch that selects the most relevant axis for a given sample via Gumbel-softmax. \nProtoPairNet is evaluated on two domains: age prediction (a supervised learning task) and car racing (a behavioral cloning task with a reinforcement learning expert), achieving performance competitive with black-box baselines, while providing transparent, case-based reasoning for predictions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Quality\n1. The paper provides very concrete and detailed mathematical derivations as proof, demonstrating strong theoretical support for the proposed method.\n2. The paper provides very concrete and detailed mathematical derivations as proof, demonstrating strong theoretical support for the proposed method.\n3. The approach is rigorous, including a three-stage training process and a non-trivial prototype projection algorithm to ensure semantic and geometrical coherence.\n-Clarity\n1. The paper is well written, with clear and thorough explanations.\n2. All figures and tables contribute positively to readers’ understanding.\n- Significance\n1. The paper tackles an important problem, namely extending prototype-based models from discrete classification to interpretable regression tasks using reasoning-based approaches.\n- Originality\n1. The model architecture and training paradigm can be considered novel.\n\nWeaknesses\n- Quality\n1. The experimental results, while competitive, are only evaluated on a limited set of regression tasks (face age, car racing in paper). Vision is heavily emphasized, no \"classic\" tabular regression or high-dimensional time-series regression tasks are included.\n2. There is a lack of discussion of potential failure modes or cases where one-to-one or traditional regression models might outperform (e.g., highly nonlinear target relationships, out-of-distribution scenarios).\n\n- Clarity\n1. Some descriptions, such as the prototype projection (Section 2.4), are mathematically dense and could be made more accessible, 2. The description of baselines in Table 1 is not sufficiently clear.\n- Significance\n1. While user studies are carefully executed, all evaluation hinges on just two datasets. It is not fully clear how well this method generalizes to hard regression tasks (e.g., non-vision or highly nonlinear problems).\n2. ProtoPairNet’s interpretability hinges on whether its selected prototype pairs are semantically coherent and representative. If the nearest prototype pair for an input does not correspond to a meaningful axis or concept in the data space, the resulting explanation may lack clarity and fail to provide useful human insight.\n- Originality\n1. Underlying notions are well established, though the extension to pairs and projection-based reasoning is new.\n\nQUESTIONS:\n1.\tGeneralization Beyond Vision: Have the authors considered testing ProtoPairNet on non-visual regression tasks (e.g., tabular, timeseries)? If so, what difficulties arise, and how can pair-axes be made interpretable in domains without natural prototypes?\n2.\tOut-of-Distribution Reasoning: How does the model handle inputs whose latent representation falls far outside any prototype pair axis? Is there any mechanism for cases when the prototype pairs are insufficient to cover the data manifold?\n3.\tPotential Failure Modes: This paper would benefit from a more thorough discussion and analysis of potential failure modes.\n4.\tBaseline Clarification: The description of baseline in Table 1 is not sufficiently clear.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}