Interpretable Mesomorphic Networks for Tabular Data
Authors: Arlind Kadra
Sebastian Pineda Arango
Josif Grabocka
Venue: NeurIPS 2024 poster (2024)
Keywords: explainability
deep neural networks
tabular data
hypernetwork
interpretability
explainable benchmark
xai.
Primary area: interpretability_and_explainability
TL;DR: Explainable deep networks that are not only as accurate as their black-box deep-learning counterparts but also as interpretable as state-of-the-art explanation techniques.

Abstract: Even though neural networks have been long deployed in applications involving tabular data, still existing neural architectures are not explainable by design. In this paper, we propose a new class of interpretable neural networks for tabular data that are both deep and linear at the same time (i.e. mesomorphic). We optimize deep hypernetworks to generate explainable linear models on a per-instance basis. As a result, our models retain the accuracy of black-box deep networks while offering free-lunch explainability for tabular data by design. Through extensive experiments, we demonstrate that our explainable deep networks have comparable performance to state-of-the-art classifiers on tabular data and outperform current existing methods that are explainable by design.

Interpretable Mesomorphic Neural Networks For Tabular Data
Arlind Kadra
Sebastian Pineda Arango
Department of Representation Learning University of Freiburg kadraa@cs. uni mni-ffeiburg... de
Department of Representation Learning
University of Freiburg
pineda@cs. uni uni-freiburg. de
Josif Grabocka
Department of Machine Learning
University of Technology Nuremberg josif, grabocka@utn.c
Abstract
Even though neural networks have been long deployed in applications involving tabular data, still existing neural architectures are not explainable by design. In this work, we propose new class interpretable neural networks for tabular data that are both deep and linear at the same time (i.e. mesomorphic). We optimize deep hypernetworks to generate explainable linear models on a per-instance basis. As result, our models retain the accuracy of black-box deep networks while offering free lunch explainability for tabular data by design. Through extensive experiments, we demonstrate that explainable deep networks have comparable performance to state-of-the-arr classifiers on tabular data and outperform current existing methods that are explainable by design.
Introduction
Tabular data arguably the most widely spread traditional data modality arising in plethora of real-world application domains (Bischl et al., 2021 Borisov et al., 2022). There exists recent trend to deploy neural networks for predictive tasks tabular data (Kddra et al. 2021 Gorishnly et al. 2021: (Somepalli et al.] [022]] Hollmann et al. 2023).
series of such application realms, is important to be able to explain the predictions of deep learning models humans (Ras et al. 2022), especially when interacting with human decision-makers. such as in healthcare (Gulum et al. 202T), or the financial sector (Sadhwani al. 2020). Heavily parametrized models such as deep neural networks can fit complex interactions in tabular datasets and achieve high predictive accuracy, however, they are not explainable. that context, achieving both high predictive accuracy and explainability remains an open research question for the Machine Learning community.
2021 Tjoa & Guan
In this work, introduce mesomorphic neural architectures! new class of deep models that are both deep and locally linear at the same time, therefore, offering interpretability by design. In a nutshell, propose new architecture that is simultaneously (i) deep and accurate, well (ii) linear and explainable on a per-instance basis. Technically speaking, we learn deep hypernetworks that generate linear models that are accurate concerning the data point are interested in explaining.
Our interpretable mesomorphic networks for tabular data (dubbed IMN) are classification or regression models that identify the relevant tabular features by design. It is important to highlight that this work
'The etymology of the term mesomorphic is inspired by Chemistry "pertaining to intermediate phase of matter". For instance, a liquid erystal qualifics as both solid and liquid.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

tackles explaining predictions for a single data point (uuddbeg & Lee| 2017), instead of explaining a model globally for the whole dataset (Ras et al., 2022). Similarly to existing prior works (Alvarez) Melis & Jaakkola 2018, Chen et al. 2018), we train deep models that generate explainable local models for a data sample interest. In contrast, we train hypernetworks that generate linear models in the original feature space through purely supervised end-to-end optimization.
We empirically show that the proposed explainable deep models are both as accurate as existing black- box classifiers for tabular datasets and achieve better performance compared to explainable end-to-end prior methods. At the same time, IMN as interpretable as explainer techniques. Throughout this work, explainers can be categorized into two groups: i) interpretable surrogate models that are trained to approximate black-box models (Lundberg Lee, 2017). and ii) end-to-end explainable methods by design. Concretely, we show that our method achieves comparable accuracy to competitive black-box classifiers and manages to outperform current state-of-the-art end-to-end explainable methods on the tabular datasets of the popular AutoMI. benchmark (Gijsbers et al.) (019). In addition, we compare our technique against state-of the-art predictive explainers on the recent XAI explainability benchmark for tabular data (Liu et al., 2021) and empirically demonstrate that our method offers competitive interpretability. As a result, our method represents a significant step forward in making deep learning explainable by design for tabular datasets. Overall, this work offers the following contributions:
We present a technique that makes deep learning explainable by design via training hyper- networks to generate instance-specifty linear models.
We offer ample empirical evidence that our method is as accurate as black-box classifiers, with the benefit of being as interpretable as statc-of-the thr-rrrrtt prediction explainers.
2 Proposed Method
2.1 Shallow Interpretability through Deep Hypernetworks
Let us denote a tabular dataset consisting of N instances of M M-dimensional features as X E R and the C-dimensional categorical target variable as Y E {I...., C.*. A model with parameters E W estimates the target variable as f RM x W + RC and is optimized by minimizing the empirical risk arg min Σm= C (yn: f(rn w)). where L {l..... . C} RCC + R. is loss function. An explainable model f is one whose predictions Un. = f(rn; w) for data point In are interpretable by humans. For instance, linear models and decision trees are commonly accepted be interpretable by Machine Learning practitioners (Ribeiro et al. 2016 Lundberg & Lee| 2017).
In this work, rethink shallow interpretable models fxx;; w) by defining their parameters w E W to be the output of deep non-interpretable hypernetworks wxx; 0) RM x θ - W, where the parameters of the hypernetwork are 0 E O. We remind the reader that a hypernetwork (a.k.a. meta- network, or "network of networks") is a neural network that generates the parameters of another network (Ha et al. 2017). In this mechanism, we train deep non-interpretable hypernetworks to gen- erate interpretable models f in an end-to-end manner as arg mingge EN L (yn, (In w(r f))).
2.2 Interpretable Mesomorphic Networks (IMN)
Our method trains deep Multi-Layer Perceptron (MLP) hypernetworks that generate the param- eters of linear models. For the case of multi-class classification, we consider linear models with parameters w E IR (M++1) denoting one set of weights and bias terms per class, as (In; w), = e=(#niww). / / ==== e=(x,;w),, with = (Xni w), >m= We.mXn.m + We.0 represent- ing the logit predictions for the e-th class. For the case of regression the linear model simply If (wn:w) = EM= Wm + Wo with w E M+1
.M
Let present our method IMN by starting with the case of multi-class classification following the hypernetwork mechanism explained in Section 2.1 The hypernetwork w(r,; : x x -
R.
(MM++) with parameters 0 E ② is a function that given data point Hn R generates the predictions as:
2

---

M Σ m=1
Won:: -= (=(xniu((sn00))
= ((nn:
(Ini 0rm Zn,m + (Tn 0).0
(1)
Instead of training weights w in standard linear classification,
the output of MLP
network as the linear weights w(In; 0). We illustrate the architecture of our mesomorphic network
in Figure [ In the case of regression, our linear model with hypernetwrkks is fIn, ( 0))
Em=1 w (Iz O),,, Zn.m + (I,,i 0) We highlight that our experimental protocol (Section includes both classification and regression datasets.
Ultimately, we train the optimal parameters of the hypernetwork to minimize the following loss in an end-to-end manner: arg min Σ C (yn, (xni (xx 0))) + A|[w (In; Il1-
8
Our hypernetworks generate interpretable mod- els that are accurate concerning a data point of interest (e.g. "Explain why patient 2n is esti- mated to have cancer f( w< f)) > 0.5 by analyzing the impact of features using the generated linear weights."), "). We stress that our novel method IMN does not simply train linear model per data point, contrary to prior work (Ribeiro et al., 2016). Instead, the hyper- network learns to generate accurate linear mod- els by a shared network across all data points. As a result, generating the linear weights demands single forward pass through the hypernetoork, rather than separate optimization procedure. Furthermore, our method intrinsically learns to generate similar linear hyperplanes for neighbor- ing data instances. The outputted linear models are accurate both in correctly classifying the data point Tn, but also for the other majority of training instances in the neighborhood (see soof-of--cncepp experiment below). The out- come is linear model with parameters w(xx;
Interpretable Weights W
TabResNet Backbone
Input Layer
TebResNee Block
Input Foatures
MCCMM
ResNet with w(a; - $ xf ((=
Figure 1: The IMN architecture.
that both interprets the prediction, but also serves as an accurate local model for the neighborhood of points.
2.3 Explainability Through Feature Attribution
The generated linear models w (Ir 0) be used explain predictions through feature attribution (i.e. feature importance) (Liu et al. (021). I). It is important to re-emphasize that our method offers interpretable predictions for the estimated target [(x,; w (Im, 0)) of particular data point 2n. Con- cretely, we can analyse the linear coefficients (wi(r,, (0)=+,;)), ...,, (x,, 0)Ml to distill the importances of (In,1..... In, M. by measuring the residual impact on the target. The impact of the m-th feature In.m in estimating the target variable, is proportional to the change in the estimated target if we remove the feature (Hooker et al. 2019). Considering our linear models, the impact of the m-th feature is proportional to the change of the predicted target if we set the m-th feature to zero. In terms of notation, we multiply the feature vector element- wise with a Kronecker delta vector 8 1 m
f(In w (xn; 0)) fxn 8"; (In, 0)) w w(x.; 0) Fn.m
(2)
As result, feature attribution strategy is that the m-th feature impacts the prediction of the target variable by a signed magnitude of w(x,;; f) Fn,m- In our experiments, all the features are normalized the same mean and variance, therefore. the magnitude w(x,; 0) Zn.m can be di- rectly used to explain the impact of the m-th feature. In cases where the unsigned importance is required, a practitioner can use the absolute impact [w 0 0) m. as the attribution. Further- more, to measure the global importance of the m-th feature for the whole dataset, we can compute N >
[w(xn; f) l.

---

Globally Accurate
Locally Interpretable
(x) x (+wb(() 0) (or [ x --(()))=
0-
2
Figure 2: Investigating the accuracy and interpretability of IMN. Left: The global decision boundary of our method that separates the classes correctly. Right: The local hyperplane pertaining to an example r' which correctly classifies the local example and retains a good global classification for the neighboring points.
2.4 Proof-of-concept:: Globally Accurate and Locally Interpretable Classifiers
As a proof of concept, we run our method on the half-mon malf-moon toy task that consists of 2-dimensional tabular dataset in the form of two half-moons that are not linearly separable.
Initially, we investigate the global accuracy our method. As shown in Figure 2 (left), our method correctly classifies all the examples. Furthermore, our method learns an optimal non-linear decision boundary that separates the classes (plotted in green). To determine the decision boundary, we perform fine-grid prediction all possible combinations of and Subsequently, we identify the points that exhibit the minimal prediction distance to a probability prediction of 0.5. Lastly, in Figure 2 (right) we investigate the local interpretability of our method, by taking point x' and calculating the corresponding weights (w (x'), w (x'))) generated by our hypernetwork, where we omited the dependence 0 for simplicity. The black line shows all the points that reside the hyperplane w(xx) as {xr w (x²'² x + wi (x') 0}. It is important highlight that the local hyperplane does not only correctly classify the point x', but also the neighboring points, retaining an accurate linear classifier for the neighborhood of points.
To validate our claim that the per-example (local) hyper- plane correctly classifies neighboring points, we conduct the following analysis: For every datapoint Tn, we take a specific number of nearest neighbor examples from every class, and we evaluate the classification accuracy of the hyperplane generated for the datapoint 2,, on the set of all neighbors. We repeat the above procedure with varying neighborhood sizes and we present the results in Table The results indicate that the mesomorphic neural network generates hyperplanes that are accurate in the neighbor- hood of the point whose prediction we are interested in explaining.
Table I: Accuracy of local hyperplanes for neighboring points.
Number of Neighbors
Accuracy
0.84
25
0.82
50
0.78
100
0.77
200
0.77
3 Related Work
Interpretable Models by Design: There exist Machine Learning models that offer interpretability by default. A standard approach is to use linear models (Tibshirani) 1996] Efron et al. 20044) Berkson 1953) that assign interpretable weights each of the input features. On the other hand, decision trees Loh 2011 Craven & Shavlik 1995) use splitting rules that build up Ieaves and intermediate nodes. Every leaf node associated with a predicted label, making it possible to follow the rules that led to a specific prediction. Bayesian methods such as Naive Bayes (Murphy et al. 2006) or Bayesian Neural Networks (Friedman et al. 1997) provide a framework for reasoning on the interactions of prior beliefs with evidence, thus simplifying the interpretation of probabilistic outputs. Instance- models allow experts to reason about predictions based on the similarity to the train samples. The prediction model aggregates the labels of the neighbors in the training set, using the average of the top-k most similar samples (Freitas) 2014 Kim al. 2015), or decision functions extracted from
prototypes Martens et al. (2007). Attention- based models like TabNet (Arik & Plister 2021 make

---

use of sequential attention to generate feature weights on per-instance basis, while, DANet (Chen al., 2022) generates global importance weights for both the raw input features and higher order concepts. Neural additive models (NAMs) (Agarwal et al.) 2021) use a neural network per feature to model the additive function of individual features to the output. However, these models trade-off the performance for the sake of interpretability, therefore challenging their usage on applications that need high performance. A prior similar work also trains heper-networks to generate local models by learning prototype instances through an encoder model Alvarez-MMs & Jaakkola (2018). In contrast, we directly generate interpretable linear models in the original feature space.
Interpretable Model Distillation: Given the common understanding that complex models are not interpretable, prior works propose to learn simple surrogates for mimicking the input-output behavior of the complex models (Burkart& & Huber 202 1 p. Such surrogate models are interpretable, such as linear regression or decision trees (Ribeiro et al. 2016). The local surrogates generate interpretations only valid in the neighborhood of the selected samples. Some approaches explain the output by computing the contribution of each attribute (Lundberg & Lee 2017) to the prediction of the particular sample. An alternative strategy is to fit globally interpretable models, by relying on decision trees (Frosst & Hinton (017)] Yang et al.) 2018), or linear models (Ribeiro et al. 2016). Moreover, global explainers sometimes provide feature importances (Goldstein al. 2015] Cortez & Embrechts 2011), which can be used for auxiliary purposes such as feature engineering. Most of the surrogate models tackle the explainability task disjointly, by first training black box model, then learning a surrogate in a second step.
Interpretable Deep Learning via Visualization: Given the success of neural networks in real- world applications in computer vision, a series of prior works (Ras et al.] (222) introduce techniques aiming at explaining their predictions. A direct way to measure the feature importance is by evaluating the partial derivative of the network given the input (Simonyan et al. 2013). CAM upscales the output of the last convolutional layers after applying Global Average Pooling (GAP), obtaining map of the class activations used for interpretability (Zhou et al.) 2016). Deepl.ift calculates pixel-wise relevance scores by computing differences with respect a reference image (Shrikumar et al. 2017). Integrated Gradients use a baseline image to compute the cumulative sensibility of a black-box model f to pixel-wise changes (Sundararajan et al. 2017). Other methods directly compute the pixel-wise relevance scores such that the network's 's output equals the sum of scores computed via Taylor Approximations (Montavon et al. 2017).
4 Experimental Protocol
4.1 Predictive Accuracy Experiments
Baselines: In terms of interpretable white-box classifiers, we compare against Logistic Regression and Decision Trees, based on their scikit-learn library implementations (Pedregosa et al., 2011). On the other hand, we compare against two strong classifiers on tabular datasets, Random Forest and CatBoost.
We use the scikit-learn interface for Random Forest, while for CatBoost we use the official implementation provided by the authors (Prokhorenkova al.) 2018). Lastly, in terms of interpretable deep learning architectures, we compare against TabNet (Arik & Pfister 2021), transformer architecture that makes use of attention for instance-wiss feature-selection and NAM (Agarwal et al., 202T), a neural additive model which learns an additive function for every feature. For TabNet we use a well- melntaeint public implementation while, for NAM we use the official public implementation from the authors
Protocol: We run our predictive accuracy experiments on the AutoML benchmark that includes 35 diverse classification problems, containing between 690 and 539 383 data points, and between 5 and 7 201 features. For more details about the datasets included in our experiments, we point the reader to Appendix C In our experiments, numerical features are standardized, while we transform categorical features through one-hot encoding. For binary classification datasets we use target encoding, where a category is encoded based shrunk estimate of the average target values for the data instances belonging to that category. In the case of missing values, we impute numerical features with zero
5

---

and categorical features with a new category representing the missing value. For CatBoost and TabNet we do not encode categorical features since the algorithms natively handle them. For all the methods considered we tune the hyperparameters with Optuna (Akiba et al., 2019). well-knoww hyperparameter optimization (HPO) library. We use the default HPO algorithm (TPE) from the library and we tune every method for 100 HPO trials or wall-time limit of 1 day, whichever condition gets fulfilled first. The HPO search spaces of the different baselines were taken from prior work (Gorishniy) et al. 2021; Hollmann et al. 2023). For more detailed description, we kindly refer the reader to Additionally, we use the area under the ROC curve (AUROC) as the evaluation metric. Lastly, the methods that offer GPU support run a single NVIDIA RTX2080Ti, while, the rest of the methods are run on an AMD EPYC 7502 32-core processor.
Appendix C
4.2 Explainability Experiments
First, we compare against Random, a baseline that generates random importance weights. Furthermore, BreakDown decomposes predictions into parts that can be attributed to certain features (Staniak & Biecek 2018). TabNet offers instance-wise feature importances by making use of attention. LIME is local interpretability method (Ribeiro et al.) (2166) that fits an explainable surrogate (local model) to single instance predictions of black-box models. On the other hand, L2X is method that applies instance-wise feature selection via variational approximations of mutual information (Chen et al., 2018) by making use of a neural network to generate the weights of the explainer. MAPLE is method that uses local linear modeling by exploring random forests as a feature selection method (Plumb et al. 2018). SHAP is an additive feature attribution method (Lundberg & Leel (017) that allows local interpretation of the data instances. Last but not least, Kernel SHAP offers reformulation of the LIME constrains (Lundberg & Lee 2017).
Baselines:
Metrics and Benchmark: As explainability evaluation metrics we use faithfulness (Lundberg & Lee, 2017), monotonicity (Luss et al. (021) (including the ROAR variants (Hooker et al.) 2019)),
infidelity Yeh et al. (219)) and Shapley correlation (Lundberg & Lee 2017). For detailed description of the metrics, we refer the reader to XAI-Bench, a recent explainability benchmark (Liu et al.) 2021).
For our explainability--elaiee experiments, we use three datasets (Gaussian Linear, Gaussian Non- I. Linear, and Gaussian Piecewise) available in the XAI-Bench (Liu et al. 2021). For the state-of-the-art explainability baselines, we use the Tabular ResNet (TabResNet) backbone as the model for which the predictions are to be interpreted (same as for IMN). We experiment with different versions of the datasets that feature diverse p values, where P corresponds to the amount of correlation among features. All datasets have a train/validation set ratio of 10 to 1.
Implementation Details: We use PyTorch as the main library for our implementation. As
backbone, we use TabResNet where the convolutional layers are replaced with fully-connected layers as suggested by recent work (Kadra et al. (02T). For the default hyperparameters of our method, we use 2 residual blocks and 128 units per layer combined with the GELU activation (Hendrycks) & Gimpel| 2016). When training our network, we use snapshot ensembling (Huang et al. (017) combined with cosine annealing with restarts (Loshchilov & Hutter 2019). We use learning rate and weight decay value of 0.01, where, the learning rate is warmed up to 0.01 for the first 5 epochs, dropout value of 0.25, and an LI penalty of 0.1 on the weights. Our network is trained for 500 epochs with batch size of 64. We make our implementation publicly available
Experiments and Results
Hypothesis 1: IMN outperforms interpretable white-box models in terms of predictive accuracy.
We compare our method against decision trees and logistic regression, two white-box interpretable models. We run all aforementioned methods the AutoML benchmark and we measure the predictive performance in terms of AUROC. Lastly, we measure the statistical significance of the results using the autorank package Herbold (2020) that runs Friedman test with a Nemenyi post-hoc test, and a 0.05 significance level. Figure 3 presents the average rank across datasets based on the AUROC performance. As observed IMN achieves the best rank across the AutoMI. benchmark
"Source code at battts::/gittie wtw.tthhb..omm/raaaaaaaaaa wtt./////nnnnna//////

---

datasets. Furthermore, the difference is statistically significant against both decision trees and logistic regression. The detailed per- per-dataset results are presented Appendix
The explainability of IMN does not have statistically significant negative impact on predictive accuracy. Additionally, it achieves comparable performance against state-of of-the-art methods.
Hypothesis 2:
This experiment addresses a simple question: Is our explainable neural network as accurate as a black-box neural network counterpart, that
Average Rank
CD
has the same architecture and same capacity?. Since our hypernetwork is slight modification of the TabResNet Kadra et al. (202T), we com- pare it against TabResNet as classifier. For completeness, we also compare against four other strong baselines, Gradient-Boo Boosted Deci- sion Trees (CatBoost), Random Forest, TabNet, and NAMs. Since the official implementation of NAMs only supports binary classification and regression, we separate the results into: i) results over 18 binary classification datasets (Figure Top), and ii) results over all datasets (Figure Bottom).
Decision Tree Logistic Regression
IMN
Figure 3: The critical difference diagram for the white-box interpretable methods. A lower rank indicates better performance over datasets.
The results of Figure 4 demonstrate that IMN achieves a comparable performance to state-of- the-art art tabular classification models, while sig- nificantly outperforming explainable methods by design. IMN achieves a comparable perfor-
Average Rank
mance to TebResNet, while outperforming Tab- Net and NAMs, indicating that its explainability does not harm accuracy in significant way. There is no statistical significance of the differ- ences between IMN, TabResNet and CatBoost. However, the difference in performance between IMNs, Random Forest, TabNet and NAMs is sta- tistically significant.
Random Forest
Tablet
ToblesNet
Average Rank
Random Forest IMN
CatBoost
Figure 4: Black-box methods comparison with crit- ical difference diagrams. Top: The average rank for the binary datasets present in the benchmark. Bottom: The average rank for all datasets present in the benchmark. A lower rank indicates a bet- ter performance. Connected ranks via a bold bar indicate that performances not significantly dif- ferent (p > 0.05).
Additionally, we investigate the runtime perfor- mance of the different baselines (NAM ex- cluded since it cannot be run on the full bench- mark). We present the results in Table 2 As expected, deep learning methods take longer time to train, however, both IMN and TabRes- Net are the most efficient during inference. We observe that TabResNet takes longer to converge compared to IMN however, both methods demand approximately the same inference time. As result, the explainability of our method comes free-lunch benefit. Lastly, IMN is 64x faster in inference compared to TabNet, an end-to-end deep- learning interpretable method. Hypothesis 1 and 2 are valid even when default hyperparameters are used, for more details we kindly refer the reader to Appendix
Hypothesis 3: IMNs offer competitive levels of interpretability compared state-of-the-an explainer techniques.
Table 2: Aggregated training and inference times for all methods.
We compare against 8 explainer baselines in
Method
Median Training
Medlan Inference
terms of 5 explainability metrics the 3 datasets of the XAI benchmark (Liu et al.] 2021). fol-
(CPU)
lowing the protocol we detailed in Section 4.2 The results of Table 3 demonstrate that IMN
CatBoost (GPU)
Random Fores:
2.20
Logisie Regression
0007
competitive against all explainers across the indicated interpretability metries. We tie in per-
Drcision
0.06
'The number of training epochs is hyperparameter.
7

---

Table 3: Investigating the interpretability of IMNs against state-of-the-art interpretability methods.
The results are generated from the XAI Benchmark (Liu et al. 2021 datasets (with = 0).
Metrie
Dataset
Random
Breakd.
Maple
LIME
1.2X
SHAP
K. K.SHAP
TabNet
IMN
Linear
0.645
0.882
0.974
0.981
0.987
Faithfulacss (1)
Gaussian Non-Lincar
-0.079
40.001
0.487
0.796
0.155
0.926
0.970
0.161
0.621
Guussian Piecewise
0.091
0.634
0.967
0.929
0.016
0.981
0.990
0.058
0.841
Gaussion Linear
00039
0494
0.548
0.544
0.049
0.549
0.550
0.041
0.639
Fuithfulmess (ROAR) (1)
Gaussiun Non-Linear
0,050
0.006
0.040
40.04
-0.66
-0,100
-0.036
4.001
0.027
Gaussion Piecewise
0.055
0.372
0.450
0.015
0.409
0.426
0.072
0.404
Guussian Linear
0.219
0,041
0,007
0.007
0.034
0.007
0.007
0,049
0.007
Infidelity (-)
Gaussian Non-Lincar
0.075
0.086
0.021
0.071
0.089
0.030
0.022
0.047
0.018
Gaussian Piecewise
0.132
0.047
0.014
0.019
0.070
0.016
0.019
0.046
0.008
Gaussion
0.605
0.652
0.680
0.667
0.785
Monotonicity (ROAR) (1)
Gaussiun Line-r
0.497
0.542
0.645
0.587
0.457
0.670
0.632
0.493
0.637
Gaussion Piecewise
0.485
0.665
0.427
0.442
0.717
0.797
0.542
0.682
Gaussian Linear
00.06
0.246
0.999
0.942
-0,214
0.993
0.999
0.095
0.999
Shapley Correlation (t)
Gaussian Non-Lincar
-0.069
0.872
0.974
0.999
0.741
Gaussian Piccewise
-0.778
0.099
0.983
0.959
0.157
0.991
0.999
0.070
0.875
Total Wins
7
7
0.02
G. ..innar
0.75
0.01
0.50
0.00
0.5 Correlation
0.5 Correlation
0.5
0
0.5
Correlation
Correlation
IMN
SHAP
Maple
BreakD.
Random
K. SHAP
LIME
L2X
TabNet
Figure 5: Performance analysis of different interpretability methods over a varying degree of feature correlation p. We present the performance of all methods faithfulness (ROAR), monotonicity (ROAR), faithfulness, and infidelity (ordered from left to right) on the Gaussian Linear dataset for values ranging from [0, l].
formance with the second-best method Kemel-SHHA (Lundberg & Lee 2017) and perform strongly against the other explainers. It is worth highlighting that in comparison to all the explainer techniques, the interpretability of our method comes as a free-lunch. In contrast, all the rival methods except TabNet are surrogate interpretable models to black-box models. Moreover, IMN strongly outperforms TabNet, the other baseline that offers explainability by design, achieving both better interpretability (Table B) and better accuracy (Figure
As result, for all surrogate interpretable base- lines we first need to train a black-box model. Then, for the prediction of every data point, additionally train a local explainer around that point by predicting with the black-box model multiple times. In stark contrast, our method combines prediction models and explainers an all-in-one neural network. To generate an ex- plainable model for data point v,, IMN does not need to train a per-point explainer. Instead, IMN requires only forward pass through the
Table 4: Interpretable method inference times. All the methods are run on the GPU and the time is reported in seconds.
Method Name
Credit-g
Adult
Christine
IMN
0.01
0.02
0.02
TabNet
0.11
1.30
0.43
SHAP (TabResNet)
17.69
565.11
228.31
SHAP (CatBoust)
4.55
66.89
4317.61
trained hypernetwork to generate a linear explainer. To quantify the difference in runtime between our method and other interpretable methods we compare the runtimes on few datasets from the benchmark with a varying number of instances/features such as Credit-g (1000/21), Adult (48842/15), and Christine (5418/ 8/1637). presenns the results, where, as observed IMN has the fastest infer- ence times, being 11-65x faster compared to TabNet which employs attention, 111001100x faster compared to SHAP that uses the same (TabResNet) backbone, and 155-20 faster compared to SHAP that uses CotBoost as backbone.

---

Lastly, we compare all interpretability methods on 4 of metrics the presence of varying p factor, which controls the correlation of features on the Gaus- sian Linear dataset. Figure F presents the comparison, where IMN behaves similarly to other interpretable methods and has a comparable performance with the top methods in the majority of metrics. The re- sults agree with the findings of prior work (Liu et al. 2021), where the performance in the interpretability metrics drops in the presence of feature correlations. Although our work focuses tabular data, in Ap- pendix A we present an application of IMN in the vision domain.
Table 5: The feature rank importances for the Census dataset. A lower rank is associated with a higher feature importance.
Decision Tree
CatBoost
Ape
Capital Cain
Capiial Loss
10
Danagraphic
Education
Education sum.
Hour week
Race
Oxcapation
Relationhip
Hypothesis 4: IMN offers global (dataset-wide) interpretability of feature importance.
The purpose of this experiment is to showcase that IMN can used to analyze the global inter- pretability of feature attributions, where the dataset-wide wide importance of the m-th feature is aggregated aware of public benchmark offering ground-truth
as >>> |u( m) T..mm. Since we global interpretability of features, we experiment with the Adult Census Income (Kohavi et al., 1996). a very popular dataset, where the goal is to predict whether income exceeds S50K/yr based on census data. We consider Decision Trees, CatBoost, TabNet, and IMN as explainable methods. Additionally, we use SHAP to explain the predictions of the TabResNet backbone.
We present the importance that the different methods assign to features in Table [ To verify the feature rankings generated by the models, we analyze the top 5 features of every individual method by investi- gating the drop in model performance we remove the feature. The more important a feature is, the more accuracy should drop when removing that fea- ture. The results of Figure 6 show that IMNs have
Percentual decrease AUROC
IMN
3
CatRoost
TabNec
Decision Tree SHAP
higher relative drop in the model's accuracy when the
most important predicted feature is removed. This
shows that the feature ranking generated by IMN proportional to the predictive importance of the fea-
ture and monotonously decreasing. In contrast, in the case of CatBoost, TabNet, SHAP, and Decision Trees, the decrease in accuracy is not proportional to the order of the feature importance (e.g. the case of Top-1 for Decision Trees, TabNet, SHAP or Top-2 for CatBoost).
Without top-k
Figure 6: Investigating the decrease in AU- ROC when removing the k-th most important feature.
We additionally consider the task of predicting mush- room edibility (Lincoff) 1997). The odor feature allows one predict whether mushroom is edi- ble or not and basing the predictions only on odor would allow a model to achieve more than 98.5% accuracy (Arik & Pfister 202T). We run IMNs on the mushroom edibility task and we achieve perfect test AUROC of 1. Furthermore, in Figure we in- vestigate the impact of every feature described in Section 2.3] where, as observed, our method correctly identifies odor as the feature with the highest impact in the output. Based on the results, we conclude that IMNs offer global interpretability.
Feature Impacis
gil-attachment
vell-color
staikshapp pe
cap-8 pa
stalk-color-be lin-iigg
Stallrroot
gil-size
stalk-color-bbdie
po of
Btalk-surace ab-vbe ng
color
spore print-color
odor
10
10
10"
Figure 7: Feature impacts for the mushroom- edibility task.

---

Conclusion
In this work, propose explainable deep networks that comparable in performance to their black-box counterparts but also as interpretable as state-of-the-ar explanation techniques. With extensive experiments, we show that the explainable deep learning networks outperform traditional white-box models in terms of performance. Moreover, the experiments confirm that the explainable deep-learning architecture does not include significant degradation in performance or an overhead on time compared to the plain black-box box counterpart, achieving competitive results against state- of-the-an art classifiers in tabular data. Our method matches competitive state-of state-of-the- of-the-art explainability methods on recent explainability benchmark in tabular data, offering explanations of predictions as a free lunch.
Limitations and Future Work
One potential limitation of our method is that although interpretable. the per-instance models are linear. A potential future work can focus on generating other types of non-linear interpretable models, such as decision trees. More concretely, the hypernetwork can generate the parameters of the decision splits and the decision value at each node, as well as the leaf weights. Another potential strategy is to generate local Support Vector Machines, by expressing the prediction for a data point as a function of the similarity of the informative neighbors.
Acknowledgements
JG, AK and SBA would like to acknowledge the funding by the Deutsche Forschungsgemeinsccaatt (DFG, German Research Foundation) under grant number 417962828 and grant INST 39/963-1 FUGG (bwForCluster NEMO). In addition, JG and AK acknowledge the support of the BrainLinks- BrainTools center of excellence. Moreover, the authors acknowledge the support and HPC resources provided by the Erlangen National High Performance Computing Center (NHR @ FAU) of the Universität Brlangen-Nürnberg (FAU) under the NHR project v00lbe. NHR funding is provided by federal and Bavarian state authorities. NHR @ FAU hardware is partially funded by the German Research Foundation (DFG) 440719683.
Sniedrich-Alexadder Aeexander-Unnieessttää
References
Agarwal, R., Melnick, L., Frosst, N., Zhang, X., Lengerich, B.. Caruana, R., and Hinton, G. E. Neural additive models: Interpretable machine learning with neural nets. Advances in neural information processing systems, 34:4699-477 4711, 2021.
Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2623 2631, 2019.
Alvarez-Melis, D. and Jaakkola, T. S. Towards robust interpretability with self-explaining neural networks. Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 18, pp. 7786-7795, Red Hook, NY, USA, 2018. Curran Associates Inc.
Ancona, M., Ceolini, E., Öztireli, C., and Gross, M. Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv preprint arXiv:1711.06104 2017.
Arik, S. Ö. and Pfister, T. Tabnet: Attentive interpretable tabular learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 6679-6687, 2021.
Berkson, J. A statistically precise and relatively simple method of estimating the bio-assay with quantal response, based on the logistic function. Journal of the American Statistical Association, 48(263)::65-599 1953.
Bischl, B., Casalicchio, G., Feurer, M., Gijsbers, P., Hutter, F., Lang, M., Mantovani, R. G., van Rijn, J. N., and Vanschoren, J. OpenML benchmarking suites. In Thirty-fifth Conference Neural Information Processing Systems Datasets and Benchmarks Truck (Round 2), 2021. URL https://openreviit http://wwwwweeeeoooooooooooottaa
10

---

Borisov, V., Leemann, T., Seßler, K., Haug. J., Pawelczyk, M., and Kasneci, G. Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems, pp. 1-21, 2022. doi: 10.1
Burkart, N. and Huber, M. F. A survey on the explainability of supervised machine learning. Journal of Artificial Intelligence Research, 70:245-317, 2021.
2021.
Chen, J. Song, L., Wainwright, M., and Jordan, M. Learning to explain: An information- theoretic information-theorrticc perspective on model interpretation. In Dy, J. and Krause, A. (eds.), Proceedings of the 35th International Conference on Machine Learning, volume of Proceedings of Machine Learning Research, pp. 883-892. PMLR, 10-15 Jul 2018. URL httph: pettps:/proceedde v80/chen18) .html
mlr. press/
Chen, J., Liao, K., Wan, Y., Chen, D. 7., and Wu, J. Danets: Deep abstract networks for tabular data classification and regression. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pp. 5930-3938, 2022.
Cortez, P. and Embrechts, M. Opening black box data mining models using sensitivity analysis. In Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining, CIDM 2011, part of the IEEE Symposium Series on Computational Intelligence 2011, April 11-15, 2011, Paris, France, pp. 341-348. IEEE, 2011. doi: 10.1109/IDM..011.5500 1.5949442. URL https
10. 1100/CIDM..../ 2011. 5949423
Craven, M. and Shavlik, J. Extracting tree-structured representations of trained networks. Advances in neural information processing systems, 8, 1995.
Efron, B., Hastie, T., Johnstone, I., and Tibshirani. R. Least angle regression. The Annals of Statistics, 32(2):407 - 499, 2004. doi: 10.1114009553440000007, URL https://doo.. org 10. 1214/ 0090536040000000677
Freitas, A. A. Comprehensible elassification models: a position paper. ACM SIGKDD explorations newsletter, 15(1):1-10, 2014.
Friedman, N., Geiger, D., and Goldszmidt, M. Bayesian network classifiers. Machine learning, 29: 131-163, 1997.
Frosst, N. and Hinton, G. E. Distilling a neural network into a soft decision tree. In Besold, T. R. and Kutz, O. (eds.), Proceedings of the First International Workshop on Comprehensibility and Explanation in Al and ML 2017 co-located with 16th International Conference of the Italian Association for Artificial Intelligence (AI*TA 2017), Bari, Italy, November 16th and 17th, 2017, volume 2071 of CEUR Workshop Proceedings. LEUR-WS.org, 2017. URL https:tpp.. : //curr- - ww org/ 2tt/Voo-2071// CEIAIIA www11111aapp____1111111 3. pdf
Gijsbers, P., LeDell, E., Poirier, S., Thomas, J., Bischl, B., and Vanschoren, J. An open source automl
benchmark. arXiv preprint arXiv:1907.0000 0009000999991 [cs. LG), 2019. URL htttt:t//axxxii org/ abs/ 1907. Accepted at AutoML Workshop at ICML 2019.
Goldstein, A., Kapelner, A.. Bleich, J., and Pitkin, E. Peeking inside the black box: Visualizing statistical learning with plots individual conditional expectation. journal of Computational and Graphical Statistics, 24(1):44-66 65, 2015.
Gorishniy, Y., Rubachev, I., Khrulkov, V., and Babenko, A. Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems, 34: 18922-1843, 2021.
Gulum, M. A., Trombley, C. M., and Kantardzic, M. A review of explainable deep learning cancer detection models in medical imaging, Applied Sciences, 11(10):4573, 2021.
Ha, D., Dai, A. M., and Le, Q. V. Hypernetworks. International Conference on Learning Representations, 2017. URL
Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelus). arXiv preprint 1606.0006 2016.
11

---

Herbold, S. Autorank: A python package for automated ranking of classifiers. Journal of Open Source Software. 5(48):2173, 2020. doi: 10.21 10.21105/joss.01111 021733 URL Inttps://ooi. org/10 21105/joss 02173
Hollmann, N., Müller, S., Eggensperger, K., and Hutter, F. TabPFN: A transformer that solves small tabular classification problems a second. In The Eleventh International Conference on Learning Representations, 2023, URL https: petttts://opeee /wtp/eeeeeevvieeeeeeeeeeeeeeeeee net/forunnr - cpsvvc
Hooker, S., Erhan, D., Kindermans, P.-J., and Kim, B. A Benchmark for Interpretability Methods in Deep Neural Networks. Curran Associates Inc., Red Hook, NY, USA, 2019.
Huang, G., Li, Y., Pleiss, G., Liu, Z., Hoperoft, J. E., and Weinberger, K. Q. Snapshot ensembles: Train 1, get m for free. In International Conference on Learning Representations, 2017. URL .et/ www/ttetuuuueeeee htte///oommmidddmwwwe""
Kadra, A., Lindauer, M., Hutter, F., and Grabocka, J. Well-tuned simple nets excel tabular datasets. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021.
Kim, B., Shah, J. A., and Doshi-Velez, Mind the gap: A generative approach to interpretable feature selection and extraction. Advances in neural information processing systems, 28, 2015.
Kohavi, R. et al. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid. In Kdd, volume 96, pp. 202-207, 1996.
Lincoff, G. H. Field guide to North American mushrooms. Knopf National Audubon Society, 1997.
Liu, Y., Khandagale, S., White, C., and Neiswanger, W. Synthetic benchmarks for scientific research in explainable machine learning. In Advances in Neural Information Processing Systems Datasets Track, 2021.
Loh, W.-Y. Classification and regression trees. Wiley interdisciplinary reviews: data mining and knowledge discovery, 1(1):14-23, 2011.
Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL httpps://opeureeieew.net hwt.://eeettttooormmmaaaeeeee
Lundherg, S. M. and Lee, S. A unified approach interpreting model predictions. In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H. M., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems 30: Annual Confer- ence on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA. USA. pp. 4765-4774, 2017. URL/httss: http:ttt////rroccciinggg newrips cc/ hwtcc/appe////// /01ee///11///// /00h//hahh// -rl--bbsttrt..... html
Luss, R., Chen, P.-Y., Dhurandhar, A., Sattigeri, P., Zhang, Y., Shanmugam, K., and Tu, C.-C. Leveraging latent features for local explanations. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '21, pp. 1139-1149, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450383325. 3467265. URL UtL/https: //doi. org/ 10. 11555554444488836665 3467265
doi: 10.1145/3447548
Martens, D., Baesens, B., Gestel, T. V., and Vanthienen, J. Comprehensible credit scoring models using rule extraction from support vector machines. Eur. J. Oper. Res., 1836))14466-1476 2007. doi: 10. 10.1011/
WtRps:tts:://doo org/10. 1016/0011eje. ejjor 2006. 051
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., and Müller, K. Explaining nonlinear clas- sification decisions with deep taylor decomposition. Pattern Recognit. 65:211-222, 2017. doi: 10.10166/j 2016. 10.6.11.008$ 1.08. /ttp::/:://d....rg//..0o.../
2016. 008
Murphy, K. P. et al. Naive bayes classifiers. University of British Columbia, 18(60):1-8, 2006.
Pedregosa, Fabian acnd Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825-2830, 2011.
Plumb, G., Molitor, D., and Talwalkar, A. S. Model agnostic supervised local explanations. Advances in neural information processing systems, 31, 2018.
12

---

Prokhorenkova, L.. Gusev, G., Vorobev, A., Dorogush, A. V., and Gulin, A. Catboost: unbiased boosting with categorical features. Advances neural information processing systems, 31, 2018.
Ras, G., Xie, N., van Gerven, M., and Doran, D. Explainable deep learning: A field guide for the uninitiated. J. Artif. Intell. Res., 73:329-396, 2022. doi: 10.161 10.1613/jair.I. 1.13200. URL https
//doi.or/100 1613/jair.... 13200
Ribeiro, M. T., Singh, S., and Guestrin, C. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16. pp. 1135-1144, New York, NY. USA, 2016. Association for Computing Machinery. ISBN 45044323.
doi: 10.1 10.1145/9999772.29377788 URL
fittps://oot orgeiiorg111111 10. 1145/2939677 2939778
Sadhwani, A., Giesecke, K., and Sirignano, J. Deep Learning for Mortgage Risk*. Journal of Financial Econometrics, 19(2):313-368, 07 2020, ISSN 1479-8409. doi: 10. 1..1099/fiinee/baa025....
URL https://doi. org/10. 1ts.///jfaaee///aaaeeeee
Shrikumar, A., Greenside, P., and Kundaje, A. Learning important features through propagating activation differences. In Precup, D. and Teh, Y. W. (eds.), Proceedings of the 34th Interna- fional Conference on Machine Learning, ICMI. 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 3145-3153. PMLR, 2017. URL http://proceediing. mlr, pTwss//70//hrriummattt h/rrshuaarrr/e/ 7aa html
Simonyan, K., Vedaldi, A., and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint 131.6:3332.6034 2013.
Somepalli, G., Schwarzschild, A., Goldblum, M., Bruss, C. B., and Goldstein, T. SAINT: Improved neural networks for tabular data via row attention and contrastive pre-training, 2022. URL httpps://peenrevier net/forumiid n1221111sr2
Staniak, M. and Biecek, P. Explanations of model predictions with live and breakdown packages. arXiv preprint arXiv:1804.01955 2018.
Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In Precup, D. and Teh, W. (eds.), Proceedings of the 34th International Conference on Muchine Learning, ICMI. 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Research, pp. 3319-3328. PMLR, 2017. URL http:tttt:::oceeee/eeee
mlr prrsp/
/70/sundararajann77... html
Tibshirani, R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267-288, 1996.
Tjoa, E. and Guan, C. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE Transactions on Neural Networks and Learning Systems, 32(1 1):4993-4813, 2021. doi: N0.1109/NNNSS.2000 30223314..
Wydmański, W., Bulenok, O.,, and Smieja, M. Hypertab: Hypernetwork approach for deep learning on small tabular datasets. In 2023 IEEE 10th International Conference on Data Science and Advanced Analytics (DSAA), pp. 1-9. IEEE, 2023.
Yang, C.,, Rangarajan, A., and Ranka, S. Global model interpretation via recursive partitioning.
In 20th IEEE International Conference on High Performance Computing and Communications,
16th IEEE International Conference on Smart City; 4th IEEE International Conference on Data Science and Systems, HPCC/SmarCCit/DDS 2018, Exeter, United Kingdom, June 28-30, 2018, URL https
pp. 1563-1570. IEEE, 2018. doi: 10.1 SWW.H//PCCCSmmCCCCIISSSSSSS. //doi. //doi.org/10 org/10. /00. Sttps//CCCSSmatiiiiee /DS.. 2018. 00256
Yeh, C.-K., Hsieh, C.-Y., Suggala, A. S., Inouye, D. I., and Ravikumar, P. On the (in)Fidelity and Sensitivity of Explanations. Curran Associates Inc., Red Hook, NY, USA, 2019.
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. Learning deep features for discriminative localization. In 2016 IEEE Conference Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 2921-2929. IEEE Computer Society, 2016. doi: 10. 10.1109/CVPR22016.313 URL https: : //doi org/ 10. 1109/CVPR. 200.9///............ 319
13

---

A
IMN can be extended to image classification backbones
Original
SmoothGrad
DeepLift
1.0
Original
Grads.
1. Grads.
SmoothGrad
DecpLift
IMN
0.0
0.5
1.0
-1
0
0.0
0.5
1.0
-1
0
0.0
0.5
1.0
Original
Grads.
Grads.
SmoothGrad
DeepLift
IMN
0.0
Figure 8: Comparison of IMN against explainability techniques for image classification.
We use IMN to explain the predictions of ResNet50, broadly used computer vision backbone. We take the pre-trained backbone <(.) " + R from PyTorch and change the output layer to fully-con colnecconee layer w
# that generates the weights for multiplying the input image æ R #x with channels, and finally obtain the logits se for the class c. this experiment, use À 10 as the LI penalty strength.
We fine-tuned the ImageNet pre-trained ResNet50 models, both for the explainable (eM-ReeNNe)) and the black-box (ResNet) variants for 400 epochs on the CIFAR- 10 dataset with learning rate of 10-4, To test whether the explainable variant as accurate as the black-box model, we evaluate the validation accuracy after 5 independent training runs. IMN-ResNet achieves accuracy of 87.49 1.73 and the ResNet 88.76 + 1.50, with the difference being statistically insignificant.
We compare our method to the following image explainability baselines: Saliency Maps (Gradi- ents) Simonyan al. 2013), DeepLift (Shrikumar al. 2017), Integrated Gradients (Ancona et al.) 2017) with SroothGrad. All of the baselines are available via the captum library rival explainers
(6) We compare the by visually interpreting the pixel-wise weights selected images in Figure 8 The results confirm that IMN- ResNee generates higher weights for pixel regions that include descriptive parts of the object.
B Plots
Average Sank Bww.://Ddouuutttppppeeeeooo
Deolan Tim
Figure 9: The critical difference diagrams that represent average rank over datasets based on the AUROC test performance for: left) The white-box methods and IMN, middle) The black-box methods and IMN for the binary classification datasets, right) The black-box methods and IMN for the entire benchmark of datasets.
In Figure 9 we repeat the experiments from Hypotheses and 2, however, without performing hyperparameter optimization. Moreover, we consider two additional baselines, DANet (Chen et al.
14

---

2022) and HyperTab (Wydmañski et al. 2023). As observed, our findings are consistent and both hypotheses are validated even when default hyperparameters are used for all the methods considered.
To further investigate the results on individual datasets, in Figure TO we plot the distribution of the gains in performance of all methods over single decision tree model (with default hy- perparameters). The gain G' of a method m run on dataset for single run calculated as shown in Equation
2.00
1.75
1.50
Gain
1.25
1.00
AUROC D)
(m, DTree, D)
0.75
AUROC A Derre, D)
0.50
IMN
TabResNet
CatBoost
R. Forest
TabNet
NAM
The results indicate that all methods except NAM achieve a comparable gain in performance across the AutoML benchmark datasets, while, the latter achieves a worse performance overall. We present detailed results in Appendix C
Figure 10: The gain distribution of the state-of-the- art models. The gain is calculated by dividing the test AUROC against the test AUROC of decision tree.
Additionally, in Figure II] we present the perfor- mance of the different explainers for the differ- ent explainability metrics. We present results for the Gaussian Non-Linear Additive and Gaussian Piecewise Constant datasets over a varying presence of correlation between the features. The results show that our method achieves competitive results against Kernel Shap (K. SHAP) and LIME, the strongest baselines.
Faithfulness (R)
Monotonicity (R)
Faithfulness
Infidelity
G. Lin-Linear
0.02
0.00
0.6
0.4
-0.55
0.00
0
0.5
0.5 Correlation
0.5 Correlation
Piecewise Constant
Correlation
Correlation
0.02
0.75
0.25
0.01
0.50
0.00
0
0
0.5 Correlation
1
0.5 Correlation
1
0.5 Correlation
0.5
Correlation
IMN
SHAP
Maple
BreakD.
Random
K. SHAP
LIME
L2X
TabNet
Figure 11: Performance analysis of all explainable methods on faithfulness (ROAR), monotonicity (ROAR), faithfulness, and infidelity. The results are shown for the Gaussian Non-Linear Additive and Gaussian Piecewise datasets where, correlation (p) ranges from [0, 1].
Lastly, investigate how sensitive IMN is the controlling hyperparameter configuration, we compare IMN and CatBoost (a method known for being robust to its hyperparameters in the commu- nity). Specifically, for every task, we plot the distribution configurations for every method. We present the results in Figure T2 where, observed, IMN has a comparable sensitivity to CatBoost with regard to the controlling hyperparameter configuration. Moreover, in the majority of cases, the IMN validation performance does not vary significantly.
the performance of all hyperparameter
15

---

Distribution of Task Performances
1.0
Validation AUROC
0.9
o
0.8
0.7
Mereed
CatBoost
0.6
IMN
0.5
4142
1007
43433
etyeer
#21es
42366
dster
22022
43380
40081
41100
41163
2351
2ide
4bopd
140
1468
2486
41288,
41G77
Task /D
Figure 12: The distribution of the validation performance of the different hyperparameter configura-
tions per task for CatBoost and IMN.
C Tables
To describe the 35 datasets present in our accuracy-related experiments, summarize the main descriptive statistics in T The statistics show that our datasets are diverse, covering both binary and multi-class classification problems with imbalanced and balanced datasets that contain diverse number of features and examples.
Additionally, provide the per-dataset performances for the accuracy-related experiments of every method with the default configurations. Table 7 summarizes the performances on the train split, where, observed Random Forest and Decision Trees overfit the training data excessively compared to the other methods. Moreover, Table 8 provides the performance of every method on the test split, where, IMN, TabResNet, and CatBoost achieve similar performances. We provide the same per-dataset performances of every method with the best-found hyperparameter configuration during HPO for the train split in Table and test split in Table I0.
Lastly, we provide the HPO search spaces of the different methods considered our experiments in Table
16
16

---

Table 6: Statistics regarding the AutoMI. benchmark datasets.
Dataset
Dausec Nane
Iistunces
Feutures
Number Clisses
Majocit Class Procerage
Minoriy Clas Percestage
1i-ss-p dacttt
3196
47:778
2000
2
1080
846
2109
KDDCrpp9 appetency
599383
benk-meekering
1454
748
13
1468
1080
1486
34465
119
28.562 562
1489
5404
70.65
349
1590
adult
458-2
76.072
4135
Amazon employee
32769
10
3.789
23512
higss
95250
29
23517
96330
40585
dtumle
58300
10
78,
0.017
40981
Australian
690
15
55. 507
44.493
40384
2310
20
7
14.286
14.785
40395
Faehimn-MNSS
70000
765
10
10,000
10.003
41507
jungle these
44819
7
$1,456
9.672
41138
APSFailuz
76000
171
98.
1,509
41142
cleisine
5415
1637
50,000
$0,003
41143
Jasmine
2984
145
50,000
50,003
41146
olvise
5124
21
50,000
50,000
41147
abbrt
425240
79
50,000
50,000
41150
MiniBonNB
130054
51
71,938
28 28.062
41159
guilerno
20000
4251
50 08.
40.015
41161
nocondo
20000
4257
2
75. 75,0000
25 25/0
41163
diben
10000
3001
20, 40.
19.1 190
41166
fabent
8237
801
23. 39.
6.004
41165
nbasi
10000
7201
10
10.430
9.580
41166
ollat
58310
181
10
21. 962
2.534
41168
junis
83733
55
66,005
2,015
41169
bokera
65196
28
100
5.143
0.170
Table 7: The per-dataset train AUROC performance for all methods in the accuracy experiments with default hyperparameter configurations. The train performance is the mean value from 10 runs with different seeds. A dashed line '.' represents a failure of running on that particular dataset.
Dataset ID
Decision Tree
Logistic Regression
Random Forest
TabNet
TabResNet
CatBoost
IMN
1.000
0.990
1.000
0.980
1.000
1.000
1.000
12
1.000
1.000
1.000
1.000
1.000
1.000
1.000
31
1.000
0.795
1.000
0.514
1.000
0.963
1.000
54
1.000
0.955
1.000
0.492
1.000
1,000
1.000
1067
0.998
0.818
0.997
0.825
0.928
0.971
0.920
1111
1.000
0.822
1.000
0.966
0,899
0.895
1169
0.994
0.680
0.994
0.705
0.697
0.733
0.697
1.000
0.908
1.000
0.947
0.945
0.948
0.942
1464
0.983
0.757
0.978
0.490
0.830
0.934
0.834
1468
1.000
1.000
1.000
0.493
1,000
1,000
1.000
1486
L.000
0.988
1.000
0.995
0.999
0.997
0.998
1489
1.000
0.813
1.000
0.947
0.974
0.982
0.977
1590
1,000
0.903
1,000
0.920
0.920
0.935
0.920
1596
1,000
0.951
1,000
0.949
0.992
0.997
0.995
4135
1.000
0.839
0.998
0,890
0.981
0.877
23512
1.000
0.683
1.000
0.820
0.863
0.831
0.853
23517
1.000
0.533
1.000
0.529
0.587
0.703
0.546
4068.5
1,000
0.999
1.000
0.988
0.999
0.994
40981
1.000
0.932
1.000
0.472
1.000
0.996
1.000
40984
1.000
0.983
1.000
0.990
0.999
1,000
0.999
40996
1.000
0.989
1.000
0.997
1.000
0,999
1.000
41027
1,000
0.799
1.000
0.980
0.982
0.989
0.982
41138
1,000
0.992
L.000
0.999
0.999
1,000
0.996
41142
L.000
0.942
1.000
0.951
1.000
0,999
1.000
41143
L.000
0.868
1.000
0.874
1.000
0,992
41146
L.000
0.967
1.000
0.989
1,000
1,000
1.000
41147
1,000
0.746
1,000
0.769
0.827
0.763
41150
1.000
0.938
1.000
0.896
0.985
0,988
0.985
41159
L.000
0.826
1.000
0.840
1,000
0.977
1.000
41161
I.000
1,000
1.000
0.999
1,000
1.000
1.000
41163
L.000
1,000
1.000
1.000
1,000
1.000
1.000
41164
1.000
0.994
1.000
0.968
1,000
0.983
1.000
41165
1.000
1,000
1.000
0.876
1,000
1,000
1.000
41166
1.000
0.889
1.000
0.943
0.978
0.992
0.995
41168
1.000
0.804
1.000
0.911
0.915
0.971
0.921
41169
1.000
0.854
1.000
0.867
0.938
0.998
0.986
17

---

Table 8: The per-dataset AUROC performance for all methods in the accuracy experiments with default hyperparameter configurations. The test performance is the mean value from 10 runs with different seeds. A dashed line ' represents a failure of running on that particular dataset.
Dataset ID
Decision Tree
Logistic Regression
NAM
Random Forest
TabNet
TabResNet
CatBoost
IMN
3
0.987
0,990
0.977
0.998
0,983
0.999
0,999
0.999
12
0.938
0.998
0.995
0,999
0,999
0.999
31
0.643
0.775
0.717
0.795
0.511
0.756
0.790
0.751
54
0.804
0.938
0.927
0,501
0.968
0.934
0.957
1067
0.620
0,802
0.659
0.801
0.789
0.808
0.800
0.805
1111
0,793
0.843
1169
0.592
0.679
0.588
0.692
0.699
0.695
0,718
0.695
1461
0,703
0.908
0.827
0.930
0,926
0.931
0.937
0.930
1464
0,599
0.749
0.738
0.666
0.516
0.740
0.709
0.742
1486
0.935
0.987
0.934
0.993
0,991
0.994
0,994
0.993
1489
0.842
0.805
0.806
0.962
0.928
0.949
0.948
0.950
1590
0.752
0.903
0.874
0.917
0.908
0.915
0,930
0.915
0.951
0.997
0.996
0.994
4135
0.639
0.853
0.838
0.846
0.855
0.883
0.858
23512
0.626
0.683
0.583
0.794
0.803
0.825
0.804
0.823
23517
0.501
0.530
0.505
0,515
0.522
0.529
0.526
0.530
40685
1.000
0.993
40981
0.817
0.930
0.918
0.945
0.463
0.919
0.935
0.908
40984
0.946
0.980
0.995
0,985
0.994
0.995
0.994
40996
0.886
0.984
0.991
0.989
0.994
0.993
0.992
41027
0.792
0.797
0.931
0.979
0.974
0.978
41138
0.861
0.974
0.558
0.989
0.970
0.972
0.992
0.980
41142
0.626
0.742
0.724
0.796
0,713
0.782
0.822
0.775
41143
0.749
0.850
0.831
0,880
0.823
0.860
0.870
0.865
41146
0,910
0.966
0,983
0.974
0,982
0,988
0.981
41147
0.606
0.748
0.675
0.762
0.765
0.779
0.762
41150
0.867
0.938
0.912
0.981
0.896
0.984
0.984
0.984
41159
0.730
0.712
0.618
0.892
0.754
0.871
0.897
0.841
41161
0.857
0,995
0.972
0.999
0.998
1.000
0.998
41163
0,873
0.994
0.999
0.998
1,000
1.000
1,000
41164
0.786
0,898
0.925
0,888
0.913
0.935
0.902
0.579
0.835
0.838
41166
0.699
0.882
0.927
0,918
0.952
0.949
0.943
41168
0.633
0.798
0.831
0,813
0,868
0.862
0.856
41169
0.554
0.841
0.800
0.842
0,883
0.866
0.865
18

---

Table 9: The per-dataset train AUROC performance for all methods in the accuracy experiments parametrized with the best hyperparameter configuration found during HPO. dashed line 1-' represents a failure to run on that particular dataset.
Dataset ID
Decision Tree
Logistic Regression
Random Forest
TabNet
TabResNet
CatBoost
IMN
0.999
0.995
0.997
1.000
1,000
1.000
12
0.986
1.000
0,999
1.000
1.000
1,000
1.000
31
0,820
0.785
0.945
0.982
0.806
0.888
0.813
54
0,930
0,960
0.971
0.988
0.990
1.000
0.968
1067
0.831
0.807
0.819
0.891
0.813
0.875
0.810
1111
0.836
0.829
0.885
0.825
0.842
0,682
0.690
0.714
0.696
0.754
0,900
0.907
0,922
0.946
0.947
0.954
0.796
0.765
0.867
0.947
0.763
1468
0.972
1,000
0.996
1,000
1,000
1,000
1,000
1486
0.981
0.988
0.987
0.997
0.998
0.999
0.997
1489
0.908
0.815
0.938
0.993
0.995
1.000
0.998
1590
0,904
0.903
0.91
0.920
0.943
0.926
1596
0,931
0,951
0,946
0.952
0.998
4135
0.833
0.826
0.865
0.843
0.995
0.850
23512
0.737
0.684
0.766
0.831
0.869
0.900
0.859
23517
0.529
0.532
0.562
0.523
0.531
0,587
0.535
40685
1.000
0,999
1,000
1.000
1.000
1,000
0.996
40981
0.931
0.932
0.978
0.949
0.943
0.947
40984
0.989
0.988
0.996
0.999
0.999
1,000
0.999
40996
0.956
0.988
0.976
0.998
1.000
0,994
41027
0,873
0.801
0.905
0.999
0.996
0.989
0.996
41138
0,979
0.989
0,989
0.996
0.992
1.000
0.991
41142
0.786
0.874
0.860
0.976
0.952
1.000
41143
0.849
0.872
0.934
0.877
0.977
0,997
0.941
41146
0.966
0.968
0.986
1.000
1.000
0,999
41147
0.727
0.745
0.746
0.768
0.865
41150
0,946
0.956
0,961
1,000
0.989
41159
0.775
0.823
0.878
0.552
1,000
1,000
1.000
41161
0,860
0,999
0,961
0.999
1,000
1,000
1.000
41163
0.901
1,000
0.976
1.000
1,000
1,000
1,000
41164
0.707
0.975
0,901
0.998
0.982
0.999
0.999
41165
0.750
0.936
0.840
0.960
0.995
0.988
41166
0.822
0.892
0.867
0.986
0.984
1,000
0.999
41167
0.941
0.997
1.000
41168
0.787
0.804
0.823
0.908
0.900
0.944
0.908
41169
0.791
0.853
0.846
0.879
0.926
0.976
0.961
19

---

Table 10: The per-dataset test AUROC performance for all methods in the accuracy experiments parametrized with the best hyperparameter configuration found during HPO. A dashed line represents a failure to rum on that particular dataset.
Dataset ID
Decision Tree
Logistic Regression
NAM
Random Forest
TabNet
TabResNet
CatBoost
IMN
3
0.999
0.997
0.977
0.998
1,000
1.000
1.000
12
0.960
0.998
0,998
0.998
0.998
0,998
1.000
31
0.765
0.852
0.717
0.826
0.741
0.844
0.826
0.863
54
0.844
0.959
0.924
0,955
0.966
0.935
0.959
1067
0.777
0.804
0.659
0.813
0.784
0.795
0.834
0.805
1111
0,829
0.801
1169
0.679
0.678
0.588
0.686
0,702
0.693
0.724
0,680
1461
903
0.907
0.827
0.919
0.917
0.932
0.939
0.933
1464
0.647
0.729
0.738
0.679
0.668
0.697
0.678
0.710
0.994
0.999
1486
0.977
0.986
0.934
0.985
0.989
0.994
0,996
0.992
1489
0.890
0.802
0.806
0.920
0.945
0.958
0.955
0.956
1590
0.901
0.901
0.874
0.908
0.913
0,930
0.913
0.931
0.951
0,946
0.952
4135
0.845
0.854
0.838
0.870
0.861
0,909
0.865
23512
0.725
0.681
0,583
0.754
0,803
0.817
0.808
0.817
23517
0.522
0.530
0.505
0.529
0.525
0.527
0,532
0.531
40685
1.000
1.000
40981
0.905
0.916
0.918
0.933
0.916
0.920
0.909
40984
0.981
0.989
0.991
0,995
0.994
0.996
0.993
40996
0.954
0.985
0.976
0.989
0.994
0.991
41027
0,800
0,900
0.993
0.980
0.991
41138
0.974
0.986
0.558
0.985
0.987
0.986
0.986
0.986
41142
0.744
0.811
0.724
0.794
0.814
0,819
0.808
41143
0.843
0.831
0.871
0.846
0.860
0.871
0.856
41146
0.966
0.962
0.975
0.991
0,984
0.987
0.984
41147
0.728
0.748
0.675
0.748
0.767
0,786
0.754
41150
0.942
0.954
0.912
0.958
0.967
0.985
0.986
0.984
41159
0.753
0.725
0.618
0,847
0.543
0.871
0.912
0.868
41161
0.849
0.997
0.972
0,950
0.999
0,999
0.998
41163
0.885
0.996
0.969
0.997
1,000
1.000
1,000
41164
0.697
0.914
0.877
0.869
0.919
0,937
0.919
0.804
0.870
41166
0.811
0.886
0.862
0,929
0.954
0,956
0.947
41168
0.768
0.797
0.809
0.808
0.866
0.871
0.856
41169
0.762
0.845
0.823
0.839
0,884
0.869
0.873
20

---

Table 11: The hyperparameter search space for IMN. TabResNet has the same search space without
weight normalization.
Hyperparameter
Type
Range
Log scale
nr_epocks
Integer
[10, 500)
Starning_rate
Continuous
[1c-5, Io-1]
bateh_sise
Categorical
(32, 64, 128, 256, 512)
weight_deeay
Continuous
[|e 5, 1]
weight weight_norm norm
Continuous
[le - 5, le 1]
dropout_rak
Continuous
[0.0.5]
Table 12:
The hyperparameter search space for logistic regression.
Hyperparameter
Type
Range
Log scale
C
Continuous
[1e 5,5]
perallyy
Categorical
(12, none)
mar maz_iterations
Integer
[50, 500)
fil intercepl
Categorical
(True, False]
Table 13:
The hyperparameter search space for decision tree.
Hyperparameter
Type
Range
Log scale
criterion
Categorical
{Gini, Entropy]
mar deppis
Integer
[1]
min sato_ssmppeessse sppit
Integer
[2, 11]
max
Integer
[3, 26]
splitter
Categorical
[Best, Random)
21

---

Table 14: The hyperparameter search space for CatBoost.
Hyperparameter
Type
Range
Log scale
learning_rate
Continuous
[lc 5.
random sandom_strength
Integer
[1,20]
12 leaf ree
Continuous
[1, 10
Begging_tempereee
Continuous
[lc 6.
lea Leeessmmationntteraaaaee steratsons
Integer
[1,20]
sterations
Integer
[100, 4000)
Table 15:
The hyperparameter search space for Random Forest.
Hyperparameter
Type
Range
Log scale
crilorion
Categorical
{Gini, Entropy)
max mar_depth
Integer
1, 21]
min_samplee split
Integer
2,
maz_ maazlees_nodes
Integer
3, 26]
cestimattooss
Integer
[100, 4000]
Table 16: Hyperparameter search space for the TabNet model.
Hyperparameter
Type
Range
Log scale
n_a
Categorical
{8, 16, 24, 32, 64, 128}
learning rale
Categorical
(0.005, 0.01, 0.02, 0.025)
gamma
Categorical
{1.0, 1.2, 1.5, 2.0)
n_steps
Categorical
73,4,5,6,7,8, 9, 10)
lambda sparse
Categorical
(o, 0.000001, 0.0001, 0.001, 0.01,
batch_size
Categorical
(256, 512, 1024, 2048, 4096, 8192, 16384, 32768)
virtual Bartual_batch____ batch_size
Categorical
[256, 512, 1024, 2048, 1096)
decay_rate
Categorical
{0.4, 0.8, 0.9, 0.95)
secay_iteraaiona
Categorical
(500, 2000, 8000. 10000, 20000)
momentum
Categorical
(0.6, 0.7, 0.8, 0.9, 0.95, 0.98}
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The results in Section 2.4 and Section 5 validate our claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the proposed method are mentioned Section 7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
23

---

Justification:
There is no theory in this paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
provide all the information regarding our method/baselines and the preprocessing of the data. We additionally open-source the code. Lastly, the results are reproducible the experiments were seeded.
Justification: In Sections and
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
24

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
The code is open-souee open-sourced (Section 4 and all of the necessary information regarding the datasets provided in Table 6 combined with their online identifier where they can be easily accessed from OpenML.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//abps.cc/public/uuiiee
swsssssssinyyyyyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The information is provided in Section The code is additionally open- sourced.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Critical difference diagrams that provide statistical significance information are provided in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar then state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
All the necessary information is provided in Section g and
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The impact our work has been mentioned in the Introduction Section and in Section
Guidelines:
The answer NA means that there is no societal impact of the work performed.
26

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best. faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Everything that used from previous work be it method or dataset has been properly cited in the manuscript.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
27

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The code is provided and documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
28

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
29


Reviews:
[{"review_id": "abhm6QoUFX", "reviewer": "Reviewer_csKM", "summary": "The paper introduces a new class of neural networks designed to be both deep and interpretable. These networks, referred to as Interpretable Mesomorphic Networks (IMN), utilize deep hypernetworks to generate linear models on a per-instance basis. This approach aims to retain the high accuracy of traditional black-box neural networks while providing explainability by design. The authors demonstrate through extensive experiments that their models perform comparably to state-of-the-art classifiers and outperform existing explainable methods on tabular data.", "questions": "See above", "limitations": "N/A", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed method offers interpretability by generating linear models specific to each data instance, making it easier to understand individual predictions.\n\n- The IMN achieves accuracy comparable to black-box models, ensuring high performance while providing explainability.\n\n- The approach is applicable to both classification and regression tasks, enhancing its utility across different types of tabular data problems.\n\n- The method has been validated through extensive experiments, demonstrating its efficacy compared to state-of-the-art classifiers and explainable methods.", "weaknesses": "I am not an expert on interpretability issues, but I am fairly familiar with the topic of tabular data prediction. Overall, I didn't notice any significant shortcomings in the paper. However, the author overlooked DANet `[1]`, a model that outperforms TabNet and provides a better interpretable framework. This could be included for comparison.\n\n`[1]` Danets: Deep abstract networks for tabular data classification and regression", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the valuable feedback. Below we will address the concerns of the reviewer:\n\n- Regarding: **“However, the author overlooked DANet [1], a model that outperforms TabNet and provides a better interpretable framework. This could be included for comparison.”:**\n\n    We thank the reviewer for the valuable suggestion. We would like to point out that there exist 2 main differences between TabNet/IMN and DANet. TabNet/IMN provides local explanations, which are in the original feature space. While, on the other hand, DANet provides global explanations and it groups/abstracts higher-level features from the original feature space. This in turn makes DANet incompatible with our experimental protocol regarding interpretability.\n\n\n   However, we believe DANet falls under methods that are explainable by design, and as such, we provide a comparison between all baselines and DANet in terms of performance in Figure 1 at the Global Response. As the reviewer suggested, DANet outperforms TabNet, however, it still performs worse compared to our proposed method in terms of accuracy. \n\n    Our results are consistent with the results of previous work [1]. We will update the camera-ready version of our work to include DANet as an additional interpretable baseline.\n\n\n[1] McElfresh, D., Khandagale, S., Valverde, J., Prasad C, V., Ramakrishnan, G., Goldblum, M., & White, C. (2024). When do neural nets outperform boosted trees on tabular data?. Advances in Neural Information Processing Systems, 36.\n\nWe believe to have addressed all the concerns raised by the reviewer and we welcome any additional questions the reviewer might have.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XAi3yNUWOZ", "reviewer": "Reviewer_hm6c", "summary": "This paper introduces a new neural network architecture called Interpretable Mesomorphic Networks (IMN) for handling tabular data. IMN combines the high accuracy of deep learning models with the interpretability of linear models by generating instance-specific linear models through deep hypernetworks, achieving both local linearity and global depth.", "questions": "Q1: How scalable is IMN in practical applications (i.e., in time complexity)? While IMN performs well in experiments, how does it fare in large-scale, complex tabular datasets in real-world applications?\n\nQ2: How sensitive is IMN to hyperparameters? Does IMN's performance vary significantly with different hyperparameter settings, requiring extensive tuning?\n\nQ3: Comparison with other emerging interpretability methods: How does IMN compare with the latest interpretability methods, such as those based on attention mechanisms, in terms of advantages and disadvantages?", "limitations": "check weakness and questions.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "S1: Innovative Design: IMN combines the advantages of deep learning and linear models, achieving a balance between high performance and interoperability.\n\nS2: Extensive Experimental Evidence: IMN's effectiveness and advantages are demonstrated through experiments on various datasets.\n\nS3: Interpretability: IMN provides both instance-level explanations and global feature importance analysis.\n\nS4: Efficiency: IMN shows excellent inference time performance, particularly on large-scale datasets.", "weaknesses": "W1: The implementation and training process of IMN is relatively complex, which might pose challenges for deployment and maintenance in practical applications.\n\nW2: IMN relies on the design and optimization of deep hypernetworks, potentially requiring extensive hyperparameter tuning and computational resources.\n\nW3: While extensive, the experiments do not cover all possible application scenarios, especially for more challenging unstructured data. what is the significant of the proposed method?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the valuable feedback. Below we will address the concerns raised by the reviewer:\n\n- Regarding: **W1**\n\n    **Implementation:**\n\t\n    We would like to point out to the reviewer that we reuse simple feed-forward backbones from previous work [1]. The only extra operation is implemented by a single line (Line 58 in module hypernetwork.py at the provided code). \n\n    **Training:**\n\n   Our code offers a fit/predict interface similar to the well-known and widely used scikit-learn library [2] which should facilitate usage for the practitioners. Additionally our method uses the stochastic gradient descent training procedure which is the standard. We would like to remind the reviewer that our approach is end-to-end, in this regard there is only one model class and not a series of components that have to be fit individually.\n\n    Lastly, we would like to point the reviewer to Table 2 in the manuscript, where as observed, our method has a faster median training time compared to the tabular ResNet. (As a result of the epoch being a hyperparameter, which translates to IMN having a faster convergence)\n\n    **Deployment:**\n\n    After training the model, a practitioner would only need to instantiate the class and load the weights. We kindly point the reviewer to Table 2 in the main manuscript, where as observed, IMN has a comparable inference time compared to the tabular ResNet. IMN can be run on the CPU and for inference/deployment the model can additionally be quantized.\n    \n    **Based on the above information, we believe the aforementioned aspects are trivial for our proposed method.**\n\n\n- Regarding: **W2**\n\n    **When HPO is used:**\n\n    To verify that IMN does not need extensive hyperparameter tuning and computational resources when HPO is applied, we compare the time it takes IMN to find the incumbent configuration versus the time it takes the plain counterpart, the tabular ResNet. \n   We provide a comparison of the incumbent performance of both methods versus the number of trials. As in the main manuscript (Table 4), we select 3 datasets with distinctive characteristics (number of instances/ nr features): Credit-g (1000/21), Adult (48842/15), and Christine (5418/1637). We present the results in Figure 2 at the attached document to the Global Response, where as observed, it takes only a few trials to find a well-performing hyperparameter configuration for both IMN and the tabular Resnet. Additionally, it takes IMN the same number of trials as the tabular ResNet to find the optimal hyperparameter configuration. \n\n    **When HPO is not used:**\t\n\n    We would kindly point the reviewer to Line 305, **“Hypothesis 1 and 2 are valid even when default hyperparameters are used, for more details we kindly refer the reader to Appendix B”**. \n\n- Regarding: **W3**\n\n    We agree with the reviewer that other data modalities are important, however, we believe tabular data are also very important and ubiquitous [3], due to the numerous application domains involving tabular data. In that context, our focus is on tabular data which in contrast to the other mentioned modalities is structured. We believe investigating other modalities merits a separate investigation of its own and falls outside the scope of our work. \n\n   Despite the above, we would like to point the reviewer to Line 342 in our main manuscript: “Although our work focuses on tabular data, in Appendix A we present an application of IMN in the vision domain.”\n\n- Regarding: **Q1**\n\n   We would like to point out to the reviewer that we run our method on the AutoML benchmark [4], a benchmark widely used by the community that features a diverse set of real problems. This includes large-scale tabular datasets from real-world applications. \n\n    Moreover, Table 2 provides time information regarding the time complexity of our method and the baselines over all datasets.  Additionally, in Table 4, we provide a more specific analysis for hand-picked datasets that have different characteristics showcasing the scalability of IMN regarding inference time.\n\n   Lastly, as an example, on the largest dataset (airlines, circa 583k training instances) IMN takes only 6.9 hours for training and has an inference time of 0.195 seconds.\n\n- Regarding: **Q2**\n\n   We investigate the sensitivity of IMN and CatBoost with regard to the hyperparameter configuration used. For every task included in our experiment, we generate a distribution of the validation AUROC performances for all the explored hyperparameter configurations per method.\n\n   Figure 3 in the attached document to the Global Response presents the results, where as observed, IMN has a comparable sensitivity to CatBoost with regard to the controlling hyperparameter configuration. Moreover, in the majority of cases, the IMN performance does not vary significantly.\n\n- Regarding: **Q3**\n\n   We would like to kindly point the reviewer to Hypothesis 2, 3, and 4, where we compare IMN with TabNet, an interpretable method that employs attention. IMN manages to outperform TabNet: in terms of performance with a statistically significant difference (Hypothesis 2, Figure 4), in terms of training/inference time (Hypothesis 2, 3 and Table 2, 4), in terms of local interpretability (Hypothesis 3) and in terms of global interpretability (Hypothesis 4, Figure 6).\n\n    Our results are consistent with the results of previous work [5].\n\n**We believe we have clarified all the concerns raised by the reviewer. We would kindly ask the reviewer to increase the score and recommend acceptance based on the provided clarifications.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mE9NkxEVCN", "reviewer": "Reviewer_qRx1", "summary": "The paper presents a hypernetwork approach to build an interpretable neural network for tabular data. Deep hypernetwork takes an input and returrns the weights for a linear model, which classify a given point. The example-baed interpretability follows from the interpretability of a linear models. The model is evaluated on typical benchmark dataset.", "questions": "-The authors should explain the connection with the paper [1] and use other deep models for comparison.\n-The global classification rule is given by the hypernetwork. How the classification changes for nearby points. The authors could illustrate it with an example.", "limitations": "yes", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The model is simple, very intuitive and theoretically sound.\n2. This is another work which show a wide range of applicability of hypernetworks. Here, the authors use hypernetworks for interpretability problem.\n3. The authors propose local classification rule (individual model for each point) using linear model (target network), while the global rule is given by the deep model (hypernetwork). This is a very interesting idea and has not been invistigated before for hypernetworks", "weaknesses": "1. The authors restrict their baseline to interpretable classifiers. There are a lot of recent studies on tabular data that authors should compare with.\n2. This is not the first work on uisng hypernetworks on tabular data. The authors of [1] use hypernetworks to build an ensemble of neural networks. The auhtors should at least refer to this work in their paper\n\n[1] Wydmański, Witold, Oleksii Bulenok, and Marek Śmieja. \"Hypertab: Hypernetwork approach for deep learning on small tabular datasets.\" 2023 IEEE 10th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable feedback. Below we will address the concerns raised by the reviewer:\n\n- Regarding: **“The authors restrict their baseline to interpretable classifiers. There are a lot of recent studies on tabular data that authors should compare with.”**\n\n   The main focus of our work lies in interpretability. We compare with state-of-the-art tabular models (Hypothesis 2), to verify that our method is comparable in accuracy and it does not suffer a significant degradation in accuracy. Based on recent work [3] which extensively compares tabular models on a wide range of datasets (circa 176 datasets), we selected the top-performing methods for different model types (Neural Networks and Gradient Boosted Decision Trees).\n\n   We believe the existing set of included baselines is representative of the most frequent models used in the tabular domain and which achieve competitive performance.\n\n   Moreover, we add two additional baselines to our experimental protocol, DANet [1] and HyperTab [2] as the reviewer suggested. The results are presented in Figure 1 at the Global Response. We describe the results in more detail in our next response to the reviewer.\n\n\n- Regarding: **“This is not the first work on uisng hypernetworks on tabular data. The authors of [1] use hypernetworks to build an ensemble of neural networks. The auhtors should at least refer to this work in their paper”.**\n\n   We thank the reviewer for providing a valuable reference. Following the reviewer’s suggestion, we additionally add HyperTab to our experiments. We provide the results on our benchmark for all methods in Figure 1, where as observed, HyperTab achieves a similar performance to IMN, with a slightly lower rank. \n\n   To validate the HyperTab results, we additionally compare all methods for datasets that have circa 1k instances and less (5 datasets), the results are presented in the table below:\n\n\n   |Method | Average Rank |\n   |-------------|:----------------------:|\n   |TabNet      |        7.0  |\n   |DANet       |        5.4  |\n   |IMN           |      4.0  |\n   |Random Forest   |    3.2 |\n   |TabResNet     |      3.0  |\n   |CatBoost        |    2.8  |\n   |HyperTab        |     2.6  |\n\n   The results are consistent with the results from the original authors [2], where the authors advocate that **“HyperTab performs strongly for small datasets however, it is comparable with other methods for medium and large-scale datasets”**. This in turn validates our results.\n\n   We will make sure to include the provided reference in the related work and describe the differences between HyperTab and ours. For e.g.:\n\n   - Our work takes as input the full view of original features, while HyperTab takes as input a binary mask depicting the subset of selected original features.\n   - Our work makes a single forward pass through the hypernetwork, while HyperTab does multiple forward passes through the hypernetwork to build an ensemble of target networks.\n   - The target network of IMN is a single output layer of 1 unit in the case of a binary classification task and $k$ units in the case of multi-class classification where $k$ represents the number of classes. While HyperTab has a target network of a hidden layer of multiple units and an output layer (2 layers in total).\n   -  The referenced work focuses on accuracy while ours focuses on interpretability.\n\n- Regarding: **“The global classification rule is given by the hypernetwork. How the classification changes for nearby points. The authors could illustrate it with an example.”**\n\n   We would like to kindly point the reviewer to Section 2.4, Figure 2 left, where we have provided an example of how the global classification boundary looks based on the hypernetwork prediction as the reviewer requests. The section additionally provides information on the classification accuracy of the locally generated hypernetworks regarding neighboring points.\n\nWe believe to have addressed the concerns raised by the reviewer and we welcome any other questions/uncertainties that the reviewer might have.\n\n[1] Chen, J., Liao, K., Wan, Y., Chen, D. Z., & Wu, J. (2022, June). Danets: Deep abstract networks for tabular data classification and regression. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 36, No. 4, pp. 3930-3938).\n\n[2] Wydmański, Witold, Oleksii Bulenok, and Marek Śmieja. \"Hypertab: Hypernetwork approach for deep learning on small tabular datasets.\" 2023 IEEE 10th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 2023.\n\n[3] McElfresh, D., Khandagale, S., Valverde, J., Prasad C, V., Ramakrishnan, G., Goldblum, M., & White, C. (2024). When do neural nets outperform boosted trees on tabular data?. Advances in Neural Information Processing Systems, 36.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pY7Ac02087", "reviewer": "Reviewer_uYeq", "summary": "This work explores interpretable models with a focus on tabular data and introduces a novel method that is locally linear while retaining a non-linear global decision boundary. The authors achieve this by expressing their model as a linear model that takes the input features and produces a dot product with weights that depend non-linearly on the features. Thus for every point, a linear decision surface is produced but the surface remains parameterized as a non-linear function of the inputs. This makes the method more interpretable as feature importance attribution can be performed, while at the same time preserving the accuracy benefits. The authors perform a very broad evaluation of their method on the AutoML benchmark, showing that there method indeed outperforms white box models such as logistic regression, remaining on par with state-of-the art models for tabular data, while offering better and more efficient interpretability.", "questions": "1. The authors focus on tabular data but in principle I don’t see anything stopping the methodology from also working on vision. Did the authors try any experiments on simple vision datasets? I would be very interested in seeing if the method can also keep high accuracy scores there.\n2. How does the method compare to a simple first order Taylor expansion of the black-box model of interest, around a given input datapoint x? This strategy should also preserve global accuracy scores while offering interpretability locally. It is of course more expensive to compute compared to the authors model, but ignoring this, I would like to understand the differences better.", "limitations": "See above.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The evaluation that the authors perform is very thorough and the results seem impressive. The formulated hypotheses are very sensible and very convincingly demonstrated. \n2. The method is very intuitive and easy to understand. The toy task with the half moons is a really good illustration to gain intuition! The paper is also very well written and easy to follow.", "weaknesses": "1. Input-dependent weights reminds me lot of a (single layer) Transformer, where the attention matrix (which is built from the inputs and weights in a non-linear fashion) serves as a weight matrix to then linearly combine the inputs. The method here uses more involved networks to produce this set of weights (not just a single layer) but still I feel like this similarity is really worth highlighting more. How would for instance a single-layer Transformer perform? Would the attention matrix also serve as a useful interpretability tool? The baseline TabNet seems to leverage attention scores to some degree to offer interpretability, it would be nice if the authors could further comment on the similarities.\n2. While the method is linear by design around a fixed datapoint x, the decision surface could still rapidly change as a function of x (especially if the classification task requires the function to do so). In such cases, does the method still offer high interpretability? The feature importance metric proposed here only zeroes out the input feature applied to the hyperplane, but the input feature used for constructing the weights is not modified. In such a situation, I would expect that the feature importance metric would fail. I guess this is why the authors employ a regularizer in the loss? How does changing its strength affect the results? In general it would also be nice to get an understanding of where this method might fail to offer explainable results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for providing valuable feedback. Below we address the concerns raised by the reviewer:\n\n- Regarding: **“Weakness 1**\n\n   Supposing a single input example $x$ with $d$ features, one would need to project each of the $d$ features to a fixed dimensionality to represent tokens, in this scenario the context length would be the number of features $d$. Then, most commonly, related work in the domain appends a classification token [1]. \n   Strictly speaking, a transformer consists of projection weight matrices, like $w_q$, $w_k$, and $w_v$. The attention matrix of the transformer gives a similarity score to the different $w_q$ and $w_k$ representations, and not to the original features. \n\n   In the transformer layer, one would use the value of the classification token which would be a combination of the different $w_v$ of the other feature representations based on the similarity scores. This would be a combination of the other feature representations which after the transformer layer would then pass through an output layer to generate the class predictions as in [1]. On the contrary, our work does not use a combination of the different feature representations to generate the output. One could consider the output weights to the linear model generated from our hypernetwork to be an analogous operation.\n\n   In the case of attention over instances, it would be different from our method since our method takes as an input only one example at a time.\n\n    TabNet has a different architecture that employs sequential attention, it features a series of steps where it learns sparse matrices $M$ of the different original features for every decision step. Based on the mask of a step, the decision step will have an impact on the final output. The masks are then combined based on the decision step importance $\\eta$ to the final output. On the contrary, our work does not feature multiple decision steps, and after generating the weights, the output is linearly dependent on them.\n\n- Regarding: **“Weakness 2”**\n\n   The aforementioned scenario by the reviewer would not impact our method, as the target linear model is generated per example. The hypernetwork will learn to fit every point and it will retain a global accuracy as shown in Section 2.4. The interpretability experiments in Section 5, Hypothesis 3 include piece-wise functions, and based on the results we argue that our method has competitive performance with the other interpretability baselines.\n\n   The hyperplane based on a given example generates the weights which combined with the example features provide the impact of the individual features. Zeroing out an input feature to the hyperplane would generate predictions for a different example.\n   The regularizer in the loss helps to induce sparsity in the generated weights, however, it only marginally improves the prediction quality of the interpretability. We have experimented with removing the regularizer in the loss and we observed that the predictions from our method were robust.\n\n\n- Regarding: **“Question 1”**\n\n   The reviewer is correct in his\\her understanding. We would like to kindly point the reviewer to Line 342 in the main manuscript, where we link to Appendix A, where we have provided a proof-of-concept example on a vision task.\n\n- Regarding: **“Question 2”**\n\n   The coefficients of the hyperplane from the Taylor expansion of the first order are given by the derivative of the output with regard to the example features. As for IMN, we perform the multiplication of the derivative with the corresponding feature and we compare both methods over the interpretable benchmark. We provide the results below:\n\n                                                   \n   |Dataset              |             Metric         |         IMN           |          Taylor|\n   |----------------------|--------------------------|-----------------------|----------------|\n   | |  | | |\n   | | Faithfulness  |                     0.987          |          **0.989**|\n   | |ROAR Faithfulness  |         **0.639**        |            0.558|\n   | Gaussian Linear  | Infidelity (Lower better) |    **0.007**        |           0.009 |\n   | | ROAR Monotonicity  |       **0.785**      |            0.757 |\n   | | Shapley Correlation      |  **0.999**        |            **0.999** |\n   | | | | |\n   | | Faithfulness |              **0.621** |                  0.427 |\n   | |ROAR Faithfulness    |      0.027           |         **0.050** |\n   | Gaussian NonLinearAdditive | Infidelity (Lower better) |  **0.018**         |           0.030 |\n   | | ROAR Monotonicity  |       **0.637**        |            0.625 |\n   | | Shapley Correlation   |     **0.741**          |         0.618 |\n   | | | | |\n   | |Faithfulness    |           **0.841**           |        0.823 |\n   | | ROAR Faithfulness       |   0.404      |              **0.458** |\n   | Gaussian Piecewise | Infidelity (Lower better) | **0.008**       |           0.082 |\n   | | ROAR Monotonicity     |     **0.682**       |             0.605 |\n   | |Shapley Correlation      |  **0.875**         |         0.560 |\n   | Wins |   |                      11             |           3 |\n\n   As observed, IMN outperforms the Taylor approximation and wins 11-3 in the provided benchmark.\n\nWe believe to have correctly addressed all the questions raised by the reviewer. If the reviewer has additional questions, we are more than happy to answer them.\n\n[1] Gorishniy, Y., Rubachev, I., Khrulkov, V., & Babenko, A. (2021). Revisiting deep learning models for tabular data. Advances in Neural Information Processing Systems, 34, 18932-18943.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for providing valuable feedback regarding our work. Below we will summarize the main concerns raised by the reviewers:

- Reviewer csKM: **“The authors overlooked DANet”**

   While DANet is an interpretable method by design, compared to IMN/TabNet, it does not provide local explanations and it additionally does not provide explanations in the original feature space given that it abstracts higher notion features from the original features. As such, DANet is incompatible with our experimental protocol regarding interpretability.

   We include DANet in our accuracy-related experiments and we compare against all baselines for the datasets present in our benchmark. We present the results in Figure 1 of the attached document, where, as the reviewer suggests, DANet outperforms TabNet. However, it has a worse performance compared to our proposed method IMN. Our results are consistent with the results from prior work [1].


- Reviewer hm6c: **“IMN requires extensive tuning and extra computational resources”**

   To address the reviewer’s concern, we compare IMN against the Tabular ResNet (TabResNet). To prove that IMN does not require extensive tuning, we compare the validation incumbent performance (best validation performance observed) during the HPO trials for both methods. As in the main manuscript (Table 4), we select 3 datasets with distinctive characteristics (number of instances/ nr features): Credit-g (1000/21), Adult (48842/15), and Christine (5418/1637). 

   We present the results in Figure 2 of the attached document, based on which:
   - IMN has a similar convergence compared to TabResNet with respect to the incumbent hyperparameter configuration.
   - IMN finds a well-performing hyperparameter configuration in a few trials in the majority of cases, similar to the TabResNet, demonstrating that tuning its HPs is not a demanding task.

- Reviewer hm6c: **“How sensitive is IMN to hyperparameters”**

   To address the reviewer’s concern, we compare IMN and CatBoost (a method known for being robust to its hyperparameters in the community). Specifically, for every task, we plot the distribution of the performance of all hyperparameter configurations for every method. We present the results in Figure 3 of the attached document. As observed, IMN has a comparable sensitivity to CatBoost with regard to the controlling hyperparameter configuration. Moreover, in the majority of cases, the IMN validation performance does not vary significantly.

- Reviewer qRx1: **”The authors restrict the comparison to interpretable classifiers and the authors do not reference HyperTab [2] as a prior work with hypernetworks for tabular data”.**

   Following the reviewer’s suggestion, we included DANet [2] and HyperTab [3], two deep learning baselines in our experiments. We present the results in Figure 1, where IMN achieves a better rank compared to both methods over all datasets. Our results are consistent with the results from prior work [1] and with the original work [3].  

- Reviewer uYeq: **“How would a first-order Taylor expansion compare with the proposed method”**

   We compare IMN against a first-order Taylor expansion on our explainability benchmark. Based on the results, we argue that IMN outperforms the first-order Taylor expansion. For more details, we kindly refer to the respective answer of the reviewer.

We would like to again thank the reviewers and we look forward to a productive rebuttal.

[1] McElfresh, D., Khandagale, S., Valverde, J., Prasad C, V., Ramakrishnan, G., Goldblum, M., & White, C. (2024). When do neural nets outperform boosted trees on tabular data?. Advances in Neural Information Processing Systems, 36.

[2] Chen, J., Liao, K., Wan, Y., Chen, D. Z., & Wu, J. (2022, June). Danets: Deep abstract networks for tabular data classification and regression. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 36, No. 4, pp. 3930-3938).

[3] Wydmański, Witold, Oleksii Bulenok, and Marek Śmieja. "Hypertab: Hypernetwork approach for deep learning on small tabular datasets." 2023 IEEE 10th International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 2023.

Meta-review: This paper proposed an interpretable architecture for tabular data, where a hypernetwork learns a local linear classifier for the input data point. The weights of this classifier provide a measure of feature importance both locally and globally. The paper is well written and its efficacy is supported by suitable experiments. All reviewers recommended the acceptance, the AC also recommends acceptance.
Decision: Accept (poster)