Explainability as statistical inference
Authors: Hugo Henri Joseph Senetaire
Damien Garreau
Jes Frellsen
Pierre-Alexandre Mattei
Venue: Submitted to ICLR 2023 (2023)
Keywords: Interpretability
Explainability
Statistical Learning
Imputation

Abstract: A wide variety of model explanation approaches have been proposed in recent years, all guided by very different rationales and heuristics. In this paper, we take a new route and cast interpretability as a statistical inference problem. We propose a general deep probabilistic model designed to produce interpretable predictions. The model’s parameters can be learned via maximum likelihood, and the method can be adapted to any predictor network architecture, and any type of prediction problem. Our method is a case of amortized interpretability models, where a neural network is used as a selector to allow for fast interpretation at inference time. Several popular interpretability methods are shown to be particular cases of regularised maximum likelihood for our general model. We propose new datasets with ground truth selection which allow for the evaluation of the features importance map. Using these datasets, we show experimentally that using multiple imputation provides more reasonable interpretation.

Under review as a conference paper at ICLR 2023
EXPLAINABILITY AS STATISTICAL INFERENCE
Anonymous authors
Paper under double-blind review
ABSTRACT
A wide variety of model explanation approaches have been proposed in recent years, all guided by very different rationales and heuristies. In this paper, we take a new route and cast interpretability as statistical inference problem. We propose general deep probabilistic model designed to produce interpretable predictions. The model's 's parameters can be learned via maximum likelihood, and the method can be adapted any predictor network architecture, and any type of prediction problem. Our method is case of amortized interpretability models, where neu- ral network is used as a selector allow for fast interpretation at inference time. Several popular interpretability methods are shown to be particular cases of regu- larized maximum likelihood for our general model. We propose new datasets with ground truth selection which allow for the evaluation of the features importance map. Using these datasets, we show experimentally that using multiple imputation provides more reasonable interpretation.
INTRODUCTION
Fueled by the recent advances in deep learning, machine learning models are becoming omnipresent in society. Their widespread use for decision making or predictions in critical fields leads growing need for transparency and interpretability of these methods. While Rudin (2019) argues that we should always favor interpretable models for high-stake decisions, in practice, black-box methods are used due to their superior predictive power. Researchers have proposed a variety of model explanation approaches for black-box models, and we refer Linardatos al.] (202T) for a recent survey. Finding interpretable methods is hard. The multiplicity of evaluation methods (Afchar) jet al., 202T] Jethani et al. 2021a| Liu al., 202T Hooker et al., 2019) makes it difficult to assess the qualities of the different methods.
In this paper, we will focus on methods that offer an understanding of which features are important for the prediction of given instance. These types of methods are called instance-wise feature selection and quantify how much prediction change when only a subset of features is shown to the model. Ribeiro et al. (2016) fit interpretable linear regression locally around given instance, and Seo et al. (2018) create saliency maps by back-propagating gradient through the model. Lundberg & Lee (2017 approximate Shapley values (Shapley 1953) for every instance. |oog & Vedaldi (2017), Slack et al. (2020) propose to create saliency map by evaluating the change in performance of the model when exposed to different selections of features. practice, these methods show very good results but focus on local explanations for single instance. An evaluation of the selection for single image requires an exponential number of passes through the black-box model.
It is of particular interest to obtain explanations of multiple instances using amortized explanation methods. The idea of such methods is to train a selector network that will generalize the selection given for a single instance. While there is higher cost of entry due to training an extra network, the interpretation at test time much faster. Jethani et al. (2021b) proposed to obtain Shapley values with selector network. [Cenn et al.) (2018). Yoon et al. (2018) both proposed to train selector that selects a minimum subset of features while maximizing an information- -heerriiiccetecccalll threshold. Jethani al. (2021a) showed that the selector in such models was really constrained to select the important features, but it would encode target information to facilitate the prediction, and proposed method that trains surrogate predictor alleviate this issue.
In this paper, we propose LEX (Latent Variable as Explanation) a modular self ieterpeteeeeee probabilistic model class that allows for instance-wist wise feature selection. L.X is composed of three

---

Under review as a conference paper at ICLR 2023
3 5
(x)
(=(2/2)
お p( ووz ا)ع
fo(
y p(y
Figure 1: The LEX pipeline allows to transform any prediction model into an explainable one. In supervised learning, standard approach uses function fo (usually a neural network) to param- eterize prediction distribution pg- In that framework, we would feed the input data directly to the neural network Ja- Within the LEX pipeline, we obtain distribution of masks Py parameterized by a neural network 94 from the input data. Samples from this mask distribution are applied to the original image z to produce incomplete samples Zz. We implicitly create the missingness by sam- pling imputed samples ğ given the masked image using a generative model conditioned on both the mask and the original image. These samples are then fed classifier to obtain prediction. As opposed to previous methods, multiple imputation allows us to minimise the encoding happening in the mask and get a more faithful selection.
different modules: a predictor, a selector, and an imputation scheme. We show that up to different optimization procedures, other existing amortized explanation methods (1.2X Chen al. (2018) Invase Yoon et al. (2018), and REAL-X Jethani et al. a222 optimize an objective that can be framed as the maximization of LEX model. LEX can used either "In-Situ," where the selector and predictor are trained jointly, or "Pos"-Hoc."" to explain an already learned predictor. We propose
two new datasets evaluate the performance of instance wise feature selection and experimentally show that using multiple imputation leads to more plausible selection.
Notation Random variables are capitalized, their realizations not. Exponents correspond to the index of realisations and indices correspond to the considered feature. For instance, z} corresponds to the 5h realization of the random variable Xj. which is the 5th feature of the random variable X. i E [o, DI. x-, defined as the vector (Fa. ...,,,,,1111 #141.. 1, ...,,...,,III.. ID), i.e., the vector with 7t dimension removed. Let {0, 1]. then X₂ is defined the vector (za)))) where only select the dimensions where 1, and - denotes the vector (l where we only select the dimension where 2 0. In particular, X- is Ileel-dimensional and æ1-2 is (D - ||=|])- dimensional.
2 CASTING INTERPRETABILIT AS STATISTICAL LEARNING
Let X'= =TP X' be D-dimensional feature space and y be the target space. We consider two random variables X = (XI....,
and E y following the true data generating distribution Pama(I: y). y). We have access to N i..dd realisations of these two random variables, x² , x. X' and labels y!....,y' y² E V. We want to approximate the conditional distribution the labels Paa (3)z) and discover which subset of features are useful for every local prediction.
2.1 STARTING WITH A STANDARD PREDICTIVE MODEL
To approximate this conditional distribution, a standard approach would be to consider a predictive model @ y) fz)). where fo IR" - H is neural network and (#(-I)) (m) is a parametric family of densities over the target space, here parameterized by the output of fo. Usually, & is a categorical distribution for classification task and normal distribution for regression task. The model being
2

---

Under review as a conference paper at ICLR 2023
posited, various approaches exist to find the optimal parameters such maximum likelihood or Bayesian inference. This method is just a description of the usual setting in deep learning. This is the starting point for models, from which we will derive a latent variable as explanation model.
2.2 LATENT VARTABLE AS EXPLANATION (LEX)
As discussed in Section the prediction model ((y]Jo())) is not interpretable by itself in general. Our goal is embed it within general interpretable proba- bilistic model. In addition, we want this explanation to be easily understandable by human, thus we propose to have score per feature defining the importance of this feature for prediction. We propose create a latent Z E {0, 1]. ? that corresponds to a subset of selected fea- tures. The idea that if Za 1, then feature used by the predictor, and conversely.
I
We endow this latent variable with a distribution p., (z]xr. This distribution Py is parametrized by neural network gr X' + J0, 1l called the selector with weights e T. To obtain the importance feature map of an input æ, we look at its average selection E,, [Z|z]] or estimate if not readily available (see Fig. 7 A common parametriza- tion is choosing the latent variable Z be distributed a product of independent Bernoulli variables p,, (=(ær) =
y
N
N
Left panel: graphical model standard predictive model. We pro- pose embed this model in latent plainer model using the construction of I1a ((zllg, (x)a). With that parametrization, the im- the right panel.
Figure 2:
portance feature map of an input x is directly given by the output of the selector 9y, For instance, et al. (2021a) use parametrization with independent Bernoulli vari- ables and obtain the feature importance map directly from L2X (Chen et al.) 2018) uses a neural network 9r called the selector but since the parametrization of p.y not an independent Bernoulli, they obtain their importance feature map by ranking the features' importance with the weights of the output of 9-- FastSHAP (Jethani et al. (222b) also uses a similar network 9, but this network's output holds different meaning. Indeed, it outputs the Shapley value of the feature for given instance, hence framing different notion of interpretability.
Yoon al. (2018) and
In the next sub-section, we will define how feature turn-off should be incorporated in the model, ie., we will define po xr) 2), the predictive distribution given that some features are ignored. With these model assumptions, the predictive distribution will be the average over likely interpretations,
Σ sE(0,11
Po(u|0, x)),
(1)
2.3 TURNING OFF FEATURES
Turning off features for given predictor is very common problem in interpretable Al (Chang al. 2019 Frye al., 202 Aas al. 2021 Covert et al. 2021 Rong et al.! 2022). We want explanation to be
to embed any kind of predictor into latent explainer. To that end, want make use of fa the same way we would in the setting without selection in Section 2.1 with the same input dimension. Hence we implicitly turn off features by considering an imputed vector X. The imputed vector takes the same values as z for 1 and follows an imputing scheme when z = O. Given ø and X is defined by the following generative process:
1. We sample the imputation part according to the imputing scheme X (X
2. Then, we define X, = x, if 2j and X if =0
For instance, a simple imputing scheme is constant imputation, iee, X put to c E R whenever Zj = 0. Thus, O-imputation corresponds to replacing the missing features with 0. One could also
the true conditional imputation p. (X1=2(012) (11=((111 Note that constant imputation
3

---

Under review as a conference paper at ICLR 2023
is case of single imputation, meaning that for given mask and incomplete instance 2z. the reconstruction I is entirely determined. On the contrary. multiple imputation allows for multiple options for reconstruction. Denoting the density for the generative process above p() define predictive distribution given that some features are ignored as
pe 2) -xx-p(R11.2 (ullfo())
Fig. ! shows how the model unfolds.
This construction allows us to define the following
factorisation of the complete model in Fig.
2
Py.0 x, T, 2)
(3)
2.4 STATISTICAL INFERENCE WITH LEX
Now that LEX is cast statistical model, it is natural infer the parameters using maximum likelihood estimation. The ligeliikl function is
((0,7) Σ log Po, (u/=)
Σ
Iog|Ez-p, Z-p, (11e (1-
(4)
Maximizing the previous display is quite challenging since have optimize the parameters of an expectation over a discrete space inside a log. Leveraging discrete latent variable model literature Appendix [E] we are able formulate good estimates of Eq. equation gradients and optimize the model using stochastic gradient descent.
(2(x)) can be any conditional distribution, then the model can just refuse to learn something explainable by setting (*|x) 1==1,- All features are always turned on. Experimentally, appears that some implicit regularization or differences the initialisation the neural networks may prevent the model from selecting everything. Yet without any regularization, we leave this possibility to chance. Note that this regularization problem appears other model explanation approaches. For instance, LIME (Ribeiro et al. 2016) fits a linear regression locally around given instance we want to explain. For the linear regression to be interpretable, they add a regularization measure of complexity for this model.
The first type of constraint we can add is an explicit function- based regularization R: f0, 1) - R. This function is to be strictly increasing with respect to inclusion of the mask that the model reaches a trade-off between selection and prediction score. The regularization strength is then con- trolled by positive hyperparameter X. For instance, Yoon al. (018 proposed used an L1- regularization on the average selection map R(z)= = || While this allows for a varying number of difficult to find in practice. Another method consid-
feature selected per instance, the optimal ered by [Chen et al. (2018) is to enforce the selection within the parametrization of py, Indeed, they only consider distributions such that any sampled subset have a fixed number of selected features k i.e., E/= 2a k. This work was extended by Xie & Ermon 2019) who proposed a continuous relaxation of weighted reservoir sampling.
2.5
LEX IS ALL AROUND
LEX is a modular framework for which we can compare elements for each of the parametrizations:
the distribution family and parametrization for the predictor pa;
the distribution family and parametrization for the selector Py;
the regularization function R {0, 1) + R to ensure some selection happens. This regularization can implicit in the distribution family of the selector;
the imputation function P., probabilistic deterministic, that handles feature turn-off.
By choosing different combinations, we can obtain a model that fits our framework and express interpretability as the following maximum likelihood problem:
max
log Ez (-lam) E Ex Pa ( AE ((2=p,((e>)) ([=))
(5)
4

---

Under review as a conference paper at ICLR 2023
Table 1: Existing models and their parametrization in the LEX framework.
Model
Sampling (p-)
Regularization (R)
Imputation (p,)
Training regime
L2X
Subset sampling
Implicit in Py
imputation
Surrogate PostHoc
Invase
Bernoulli
Ll-regularization
imputation
InSitu
Surrogate
Fixed 0 InSitu Surrogate PostHoc
REAL-X
Bernoulli
Ll-regularization
imputation
Many models, though starting from different formulations of interpretability, minimize a cost func- tion that is a hidden maximum likelihood estimation of a parametrization that fits our framework. Indeed, 1.2X (Chen al. 2018) frames their objective from a mutual information point of view, Invase (Yon et al. 2018) (and REAL-X (Jethani et al., (021a)) whose objective is derived from Invase's ' objective) frame their objective from Kullback-Leibe
divergence but we can cast the optimization as the maximization of the log-likelihood of a LEX model. We refer to Table for an overview, and Appendix A for more details.
2.6 LEX UNIFIES POST-HOC AND IN- SITU INTERPRETATIONS
While our model can be trained from scratch as self-interpretable model (we call this setting In-Situ), we can also use to explain given black-box model in the same statistical learning framework (this setting is called Post-Hoc). In the In-Situ regime, the selection learns the minimum amount features to get a prediction closer to the true output of the data while the Post-hoc regime, the selection learns the minimum amount of features to recreate the prediction of the fully observed input of a given classifier pm (2(zz) to explain. The distinction between the "In-Situ" regime and the "Post-hoc" regime is mentioned, for instance, in Watson & Floridi (2021) crucial. We distinguish four types regimes:
Free In-Situ regime training interpretable model from scratch using the random variable Y
Pdata as a target.
Fix-0 In-Situ regime training only the selection part of the interpretable model using fixed clas- sifier but still using the random variable Y Paata (Y|X) as target. In that setting, we do not get an explanation how pe predict its output but an explanation map for the full LEX model pe.y.
Self Post-Hoc regime training only the selection part of the model using a fixed classifier po, but the target is given by the output of the same fixed classifier when using the full information ~ (=)) This can be understood as Fix-0 In-Situ regime where the dataset is generated from Pdata (ttaa(x)pu (z)pe(ylzz.
Surrogate Post-Hoc regime training both the selection and the classification part but the target Y is following the distribution of given fixed classifier (m]ly).) The full model pe,y is trained to mimic the behaviour of the model Pm, (y(z). This can be understood as a Free In LntSitu regime where the dataset generated from Paaa Pm (y)x).
HOW TO TURN OFF FEATURES?
We want to create an interpretable model where an un-selected variable is not "seen" by the classifier of the model. For instance, for a given selection set 2 {0, 13 and given predictor pe. variable is not "seen" by the predictor when averaging all the possible outputs by the model over the unobserved input given the observed part for given data distribution p.. This allows us define the restricted predictor with respect to the distribution p.:
/
Pp. (yl x2)
p(y|æ1-2, x)), (od
(6)
#1-2
Covert et al. 2021 advocates the use of the true conditional imputation Pdata (21-=|22) for p.- This motivates the use of a multiple imputation schemes that should mimic the behaviour of the true conditional imputation p. Paata for model.
5

---

Under review as a conference paper at ICLR 2023
Chen al. (2018) and Yoon al. (2018) proposed to use 0 imputation in their parametrization as a very rough approximation of the true conditional imputation. Using such imputation can lead to issues. Notably, Jethani et al. (202 la) showed that, with such an imputation. training the selector and predictor jointly can lead to the selector encoding the output target distribution for the classifier making the interpretation incorrect. Instead of using an imputation scheme that would approximate the true conditional imputation, Jethani et al. (2021a) proposed to separate the optimization of 0 and 7. py is first trained to approximate the restricted predictor Pdata (y) xx)) for any E {0, 1 Jheee, the selector part of the model Py is optimized using a fixed 8. If po (y|z, 2) approximates correctly Paun (3)22)) the selector should not encode the output of the prediction.
However, training pe (y/az, 2) to approximate the true conditional imputation for any mask patter very complicated. Indeed, Le Morvan et al. (202T) showed that the optimal function with single imputation data suffers from discontinuities which makes it very difficult estimation problem. If the model is not correctly approximating Pasta [y)) there is no guarantee that the selection will be meaningful. Jethann et al. 2021a) advocates the use of a constant that is outside the support. Yet, all the experiments are made using O imputation which is inside the support of the input distribution. Having constant imputation inside the domain of the input distribution may lead to further dis- crepancy between the surrogate and the restricted predictor. Indeed. [psen et al., (2022) showed that using constant imputation inside the domain to learn a purely discriminative models with missing data leads to some artefacts in the prediction.
On the other hand, propose approximate the true conditional distribution by using a multiple imputation scheme. This generative model should allow for fast sampling of the quantity p. (ā) (ā] z). Depending on the difficulty of the dataset, obtaining an imputing scheme allowing for fast and effi- cient masked imputation can be complicated. We show that we can come up with simpler imputing schemes that perform as well or better than the fixed constant ones. For instance, we propose train a mixture of Diagonal Gaussian to sample the missing values or to randomly sample instances from the validation dataset and replace the missing features with values from the validation sample. We describe many different methods for imputing with multiple imputation in Appendix B
4 EXPERIMENTS
There are many different sets of parameters to choose from in the LEX model. In this section, we want to study the influence of the imputation method in the LEX models as this was the discussion of previous recent papers (Covert et al. 202T Jethani et al.] 2021a). We show the benefits of multiple imputation and that surrogate constant imputation can still lead to encode the target output in the
masks.
For all these existing methods, we fix the Monte-Carlo gradient estimator to REBAR Tucker et al. (2019) as it provides an unbiased and low variance estimation of the gradients. We use an implicit regularization constraining the distribution p.y sample fixed proportion of feature (note that this is a very similar setting as the one considered in L,X except for the Monte- -arlo gradient estimator). We call this proportion the selection rate. This choice of regularization allows to compare all sets of parameters on the same "credit" for selection (see F22 for more details)
Evaluation of features' importance map is hard. These evaluation methods rely very often on syn- thetic datasets where a ground truth is available (Liu et al.) 2021] Afchar et al.) 2021) or retrain a full model using the selected variable (Hooker et (019). We consider more complicated datasets than these synthetic datasets. We create three datasets using MNIST, FashionMNIST, and CelebA as starting points. These datasets provide information only on subset of features that should not be selected since it is not informative for the prediction task. It allows us to consider any selection outside this remaining subset as an error in selection. We call this type of selection ground truth the maximum selection ground truth. Since we want to evaluate the selection on ground truths from the dataset, all our experiments are made in the In-Situ regime.
To compare our selections to these ground truths, we look at the true positive rate (TPR), false positive rate (FPR), and false discovery rate (FDR) of the selection map. To that end, we sample at test time a 100 mask samples from p.,, and calculate the three measures for each one of the 100 masks. We then show the average of the selection performance over these 100 masks. To compare the prediction performance, we consider the accuracy of the output averaged over the mask samples.

---

Under review as a conference paper at ICLR 2023
S
S2
S3
0.78
6.70
0.700
0.75
ACC
6.65
0.670
ACC
9.29
0/600
0.65
0.625
0.1
6.1
0.1
Effective Selection Rate
Effective Selection Rate
Effective Selection Rate
0.6
C.4
0.4
FDR
FDR
FDR
02-
6.2-
0.0
0.2
2.4
0.2
0.4
26
08
0.2
0.4
08
2k
Effective Selection Rate
Effectivve Selection Rate
Efflective Selection Rate
1.0
10-
0.8
TPR
IPRR
0.6-
6.6
6.4
0.2
2.1
066
0.
0.2
0.1
366
08
02
0.4
0.5
9.8
Effective Selection Rato
Effective Selection Rate
Effective Selection Rate
Surrogate 0 Imputation
Impatation
Standard Gaussian
Figure 3: Performances of LEX with different imputations. 0 imputation (solid orange line) cor- responds to the imputation method of Invase/L2X, Surrogate 0 imputation (blue dashed line) is the imputation method of REAL-X. The standard Gaussian is the true conditional imputation method from the model (green dotted curve). Columns correspond the three synthetic datasets (S1, S2, S3) and lines correspond to the different measure of quality of the model (Accuracy, FDR, TPR). We report the mean and the standard deviation over folds/generated datasets.
4.1 ARTIFICIAL DATASETS
Datasets We study synthetic datasets (S1, S2, S3) proposed by Chen et al. (2018) where the features are generated with standard Gaussian. Depending on the sign of the control flow feature (feature 11), the target will be generated according to a Bernoulli distribution parameterized by one among the three functions described in Appendix C These functions use different number of features to generate the target. Thus, depending on the sign of the control flow feature, S1 and $2 either need 3 or 5 features to fully generate the target while S3 always requires 5.
For each dataset, we generate 5 different datasets containing each 10,000 train samples and 10, 000 test samples. For every dataset, we train different types of imputation with a selection rate in 1 3 ......: 1. i.e. we select 2, features. We then report the results in Fig. 3 with their standard deviation across each 5 generated datasets. In the following experiments, we compare a multiple imputation based on standard Gaussian and constant imputation with and without surrogate using a constant (as used by Yoon et al. 2018 Chen al. 2018 Jethani et al. 2021a).
Selection evaluation The ground truth selection for S1 and have two different true selection rates depending on the sign of the 11 feature (shown as two grey lines in Fig.3). 3). In Fig. 3 for S1, using multiple imputations outperforms other methods as soon as the number of selected variables approaches the larger of the two true selection rates. For S2, LEX performs better most of the time when the selection rate higher than the larger of the two true selection rates but has a higher variance in performance. LEX outperforms both 0-imputation and surrogate with O-imputation on S3 as soon as the selection rate is close to the true selection rate (shown as the grey line) while still maintaining a better accuracy.
7

---

Under review as a conference paper at ICLR 2023
0.73
8
6.90
FDR
TPR
0.5-
5
1-6--------- 6766
-J0-9-8-7
7-1-0--------1-00 1
Imputotion Constant
Insutation Constant
Imputation Constant
Figure 4: Performances of LEX with varying constant imputation (orange solid line) and surrogate constant imputation (blue dashed line) on S3 using the true selection rate for the selection distri- bution. Though Invase/L.2X (resp. REAL-X) uses constant imputation (resp. surrogate constant imputation), all these methods used only 0 as imputation constant. We report the mean and standard deviation over folds/generated datasets.
1.00
Surrogate 0 Impulation
0.4
Impatation
Mixture of logistics
Effective Selection
Elfective Selection
Figure 5: Performances of LEX on the Switching Panel MNIST dataset with different imputations. Surrogate imputation corresponds to the parametrization of REAL-X, imputation corresponds to the parametrization Invase/L2X We report the mean and standard deviation over 5 folds/generated datasets.
Dependency on the constant of imputation We now focus on S3 and consider the same experiment with 5 generated datasets but with a fixed selection rate ii corresponding to the true number of features, without a surrogate varying the constant from - 10 to 9. In Fig. 4 the quality of both the selection and the accuracy depends on the value of the imputation constant. Both the surrogate constant imputation and constant imputation perform better when the constant within the domain of the input distribution. The performance drops drastically outside the range [-1, 1]. Jethani et al. (202 suggested using constant outside the domain of imputation which is clearly sub-optimal. Further results in Fig. 10 explicit this dependency on all three synthetic datasets.
5. We train a single model for both the constant imputation with and
4.2 SWITCHING PANELS DATASETS
We want to use more complex datasets than the synthetic ones while still keeping some ground truth explanations. To create such a dataset, we randomly sample a single image from both MNIST and FashionMNIST (Xiao al., (017)) and arrange them in dom order to create new image. The target output will be given by the target of MNIST for Switching panels MNIST (SP-MNIST) and by the target of FashionMNIST for Switching panels Fashion- MNIST (SP- FashioonNNISTT)
4
Figure 6: Samples from SP- MNIST and their LEX expla- nations. The class of the top sample is O, the class of the bottom sample
Given enough information from the panel from which the target generated, the classifier should need see any informa- tion from the second panel. If selection uses the panel from the dataset that is not the target dataset, it means that the predictor leveraging information from the mask. We consider that the maximum ground truth selection is the panel corresponding to the target image.
We train every model on 45,000 images randomly selected the train dataset from MNIST (the remaining 15,000 images are used the validation dataset). For each model, we make 5 runs the model on 5 generated datasets (i.e. for each generated dataset, the panels are redistributed at random) and report the evaluation of each model on the same test dataset with their standard deviation over the 5 folds. All method uses U-NET (Ronneberger convolutional neural network for the classification. Note that in practice for MNIST, only 19% of pixels are lit average per image. The true minimum effective rate for our image should therefore
al., 2015) for the selection and fully
8

---

Under review as a conference paper at ICLR 2023
0.015
ACC
Surrogate O Impetation
0.016
VAEAC
Marginal
0.96
Imputation
0.05
0:35
0:15
0.00
0:15
Effectivve Selection Rate
Effective Selection Rate
Figure 8: Performances of LEX on the CelebA Smile dataset with different methods of approxima- tion for the true conditional distribution.
be around 10% of the pixels. We evaluate the results of our model around that reasonable estimate of the effective rate. The selected pixels around this selection rate should still be within the ground truth panel.
In Fig. [ we can see that LEX using multiple imputation with a mixture of logistics to approx- imate the true conditional imputation outperforms both using constant imputation (1.2X/Invase) and a surrogate constant imputation (REAL-X). Indeed, around the estimated true selection rate (10%), less selection occurs in the incorrect panel while still maintaining high predictive perfor- mance. Fig. 6 shows that LEX uses information from the correct panel of SP-MNIST. We highlight further results and drawbacks in Appendix
4.3 CELEBA SMILE
The CelebA dataset (Liu et al., 2018) is a large-scale face attribute dataset with more than 200, 000 images which provides 40 attributes and landmarks or positions of important features in the dataset. Using these landmarks, we able to create dataset with minimal ground truth selection. The assignment associated with the CelebA smile dataset a classification task to predict whether a person is smiling. We leverage the use of the landmark mouth associated with the two extremities of the mouth to create a ground truth selection in a box located around the mouth. We make the hypothesis that the model should look at this region to correctly classify if the face is smiling or not in the picture (see Appendix SirendxxCandd Fig. 9 for details and examples).
In Fig. 8 evaluate two methods of multiple imputation. The VAEAC Ivanov et al.] (2018) which is a generative model allow- ing for the generation of imputed samples given any conditional mask and a marginal multiple imputation. Both our multiple imputation methods perform similarly compared to the constant imputation method both in selection and accuracy. In Fig. / we see that LEX uses the information around the mouth to predict whether the face is smiling or not which is a sensible selection. (See Appendix for further results and samples)
CONCLUSION
Figure 7: Samples from CelebA smile and their LEX explanations with 10% rate. Using constant imputation leads to less visually com- pelling results (Appendix F) compared to other constant im-
We proposed: framework, LEX, casting interpretability max- imum likelihood problem. We have shown that LEX encompasses several existing models. We provided 2 datasets on complex data with ground truth to evaluate the feature importance map. Using these, we compared many imputation methods to remove features and showed experimentally the advantages of multiple imputation putation methods. These new datasets can used for other explanation methods than the amortized ones we focused on here.
The framing of interpretability as statistical learning problem allowed us to use maximum likeli- hood to find optimal parameters. One could explore other methods for optimizing the parameters, such as Bayesian inference. Interpretation maps are more easily readable when they provide smooth segmentations for images. Another avenue for future work could be to study the of different parametrizations or regularization that favors connected masks (in the sense that neighbouring pix- els have more incentives to share the same selection) allow for smoother interpretability maps.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Kjersti Aas, Martin Jullum, and Anders Loland. Explaining individual predictions when features are dependent: More accurate approximations shapley values. Artificial Intelligence, 293:0035,,, 2021.
Darius Afchar, Vincent Guigue, and Romain Hennequin. Towards rigorous interpretations: for- malisation of feature attribution. In International Conference on Machine Learning, pp. 76-86. PMLR, 2021.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 2013..
2013.
Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Importance weighted autocncoders. arXiv 2015.
preprint
Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image clas-
sifiers by counterfactual generation. In International Conference Learning Representations, 2019.
Jianbo Chen, Le Song, Martin Wainwright, and Michael Jordan. Learning to explain: An information-theorees perspective on model interpretation. International Conference on Ma- chine Learning, pp. 883-892. PMLR, 2018.
Ian Covert, Scott Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22(209):1-90, 2021.
Justin Domke and Daniel R Sheldon. Importance weighting and variational inference. Advances in neural information processing systems, 31, 2018.
Ruth C. Fong and Andrea Vedaldi. Interpretable explanations of black boxes by meaningful pertur- bation. In Proceedings of the IEFE international conference on computer vision, pp. 3429-3437, 2017.
Christopher Frye, Damien de Mijolla, Tom Begley, Laurence Cowton, Megan Stanley, and Ilya Feige. Shapley explainability on the data manifold. In International Conference on Learning Representations, 2021.
Sara Hooker, Dumitru Erhan, Pieter-Jan Kindermans, and Been Kim. A benchmark for interpretabil- ity methods in deep neural networks. Advances neural information processing systems, 32, 2019.
Niels Ipsen, Pierre-Alexandre Mattei, and Jes Frellsen. How to deal with missing data in supervised deep learning? In ICLR 2022-10th International Conference on Learning Representations, 2022.
Oleg Ivanov, Michael Figurnov, and Dmitry Vetrov. Variational autoencoder with arbitrary condi- tioning. arXiv preprint
2018.
Neil Jethani, Mukund Sudarshan, Yindalon Aphinyanaphongs, and Rajesh Ranganath. Have we learned to explain?: How interpretability methods can learn to encode predictions their inter- pretations. In International Conference on Artificial Intelligence and Statistics, pp. 1459-1467. PMLR. 2021a.
Neil Jethani, Mukund Sudarshan, Ian Connick Covert, Su-In Lee, and Rajesh Ranganath. FastSHAP: Real-time shapley value estimation. International Conference on Learning Representationss 2021b.
Marine Le Morvan, Julie Josse, Erwan Scornet, and Gaël Varoquaux. What's good imputation predict with missing values? Advances in Neural Information Processing Systems, 1530- 11540, 2021.
Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. Explainable Al: A review of machine learning interpretability methods. Entroyy 23(1):18, 2021.
10

---

Under review as a conference paper at ICLR 2023
Yang Liu, Sujay Khandagale, Colin White, and Willie Neiswanger. Synthetic benchmarks for sci- entific research in explainable machine learning. arXiv preprint arXiv:2006.12543, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale celebfaces attributes (celeba) dataset. Retrieved August, (5(0188):1, 2018.
Scott M Lundberg and Su-In Lee. A unified approach interpreting model predictions. Advances in neural information processing systems, 30, 2017.
Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables. arXiv preprint arXiv:16/ 1ete:6//1.000 100000 2016.
Shakir Mohamed, Mihaela Rosca, Michael Figurnov, and Andriy Mnih. Monte carlo gradient esti- mation in machine learning. J. Much. Learn. Res., 21(132):1-62, 2020.
Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Couraaeea, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn; Machine learning in Python. Journal of Machine Learning Research, 12:2825-28366 2011.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "Why should trust you?" Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, 2016.
Yao Rong, Tobias Leemann, Vadim Borisov, Gjergji Kasneci, and Enkelejda Kasneci. Evaluating feature attribution: An information-theortii perspective. arXiv preprint arXiv:2202.00449, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi-
cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234 241. Springer, 2015.
Cynthia Rudin. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, (5):206-215, 2019.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelcnn++: Improving the pixelenn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.05517, 2017.
Junghoon Seo, Jeongyeol Choe, Jamyoung Koo, Seunghyeon Jeon, Beomsu Kim, and Taegyun Jeon. Noise-adding methods of saliency map as series of higher order partial derivative. arXiv preprint arXiv:1806.0888 03000, 2018
Lloyd S. Shapley. A value for n-person games. Contributions to the Theory of Games, number 28 in Annals of Mathematics Studies, pages 307-317, II, 1953.
Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM Conference on Al, Ethics, and Society, pp. 180-186, 2020.
Richard Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient meth- ods for reinforcement learning with function approximation. Advances in neural information processing systems, 12, 1999.
G. Tucker, D. Lawson, S. Gu, and C. Maddison. Doubly reparameterized gradient estimators for monte carlo objectives. In International Conference on Learning Representations, 2019.
David S Watson and Luciano Floridi. The explanation game: formal framework for interpretable machine learning. Ethics, Governance, and Policies in Artificial Intelligence, pp. 185-219. Springer, 2021.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmark- ing machine learning algorithms. arXiv preprint arXiv: 177.007777 2017.

---

Under review as a conference paper at ICLR 2023
Sang Michael Xie and Stefano Ermon. Reparameterizable subset sampling via continuous relax- ations. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, CAI'19, pp. 3919-3925, AAAI Press, 2019.
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. Invase: Instance-wise variable selection using neural networks. In International Conference on Learning Representations, 2018.
Jinsung Yoon, James Jordon, and Mihaela van der Schaar. INVASE:
Instance wise variable selection using neural networks. In International Conference Learning Representations, 2019. URL https://openreviii net Bet roAcK?
12

---

Under review as a conference paper at ICLR 2023
A OTHER METHODS FALLING INTO THE LEX FRAMEWORK
In this section, we show how to cast three existing models our statistical learning setting. Though the original papers framed their wanted interpretabiliy through different points of view, will show that what they actually optimize can be seen as likelihood lower bounds of specific LEX models.
A.1 LEARNING TO EXPLAIN (L2X, CHEN ET AL. 2018)
Let us consider fixed classifier Pm- For a fixed number of features k, we want train a selector p. (this corresponds to V(0,-) in the original paper) that will allow selection for this exact number of features (using subset sampling) only and predictor pa (this corresponds to go in the original paper) such that the full model po,r will mimic the behaviour of pm- We use 0-imputation to parameterize imputation distribution. We can rewrite Eq. (6) Chen al. (2018) using our notation as
Ex Ex-pua.eePh
Pm (vlx)) po
Σ pm (u/()) lo po .x)
-Expumesp
Ex Pmp. log (YIx)
Y [log 2 Xwp, log po (Y [X]],
where used Jensen inequality to obtain this last inequality. Thus the L2X objective is lower bound of the likelihood the post-hoc setting as the target distribution comes from the fixed classi- fier Pm we want explain. The objective described is shed under the light of the Surrogate Post- Hoc regime. By considering Pm, as the true data distribution Pdata this can be extended to the Free In-Situ regime.
A.2 INVASE (YOON ET AL. 2018)
We consider selector p-, (this corresponds to (2. (x..) 8) in the original paper) and classifier po (this corresponds f in the original paper) using O-imputation to parameterise the imputation module. We consider a regularization function controlled by such that R(>) = Ep, (II=1lo). They also train baseline classifier, since this classifier is simply used to reduce variance in the estimated gradient and does not change the objective for the optimization in and y.
We can rewrite Eq. (5) in Yoon al. (2018) using notation
i(2, x, v) Σ
yb log (0(
The loss for the selector network is then
/
xxy Pdata (I, 3)
Σ (22(2)))(2, + A||=|lo]drdd
=E 2000111
Ex. Pun
E ~xp,
[1(2, B) + A||2||o]
Ex,Y Para
E Ez~p.
log (m( Y X, 2) + {||=|||]
Ex
E2
Eznp,
loggoo( (Y Z) + A||a|lbl
Ex,Y~
E Zz-p, [- log Px (Y|)) + A|]=|[o]
EX.Y Egap, [- log E3 P (Y (Y] [12=,,11122|0
EX,Y~ Pays E Ez [- log E 8o [Y] + AR(,,))
> Ex.Y~Pial - log Ez E Pa Y X + A
Though the loss for the optimisation of the selector with & and the loss for the optimisation separated in the original article, they only differ by the regularization Ez-p, [*[[]][o] which is
13

---

Under review as a conference paper at ICLR 2023
independent of 0. Thus, minimizing both 8 and y using the loss for the selector network is equivalent to the minimization set up in Invase.
The last equation is exactly the negative log-likelihood of the LEX model with parametrization described above. Hence the minimisation target of INVASE a minimisation of an upper bound of the negative log-likelihood.
A.3 REAL-X (EETAAN ET AL. 2021A)
Let us consider a fixed classifier Pm- Similarly to INVASE, we consider a selector Py (this corre- sponds (sel) and classifier pe (which correspond qpeed) and function controlled regularizer R.
We can rewrite Eq. (3) from Jethani et al. (2021a) in our framework as
Egwapan Byr-pm. Ezop, Po z)] + XR(())
< Lewpan Ey-pp [log Es-p,, (o 2)] + AB(()
The last equation a lower bound on the log- log-likelihood of LEX model parametrized as described above.
As opposed to the previous method, the predictor trained on a different loss than the selector:
Pup B,,5(0055 [log po x 2)]
where (00.5) is distribution of independent Bernoulli for the mask.
This loss is completely independent of the selector's parameters. Thus, training first the predictor network Ps until convergence and then training the selector py equivalent the alternating training in Algorithm in Jethann et al. (202aa).
Thus, we can consider that the associated LEX model always trained with a fixed and REAL-X is maximizing negative log-likelihood of LEX parametrization in a fixed 0 setting.
B MULTIPLE IMPUTATION SCHEME
B.1 VAEAC
VAEAC (Ivanov al. 2018) arbitrary conditioned variational autoencoder that allows to sample from an approximation of the true conditional distribution. To impute a single example, we first sample latent h according to Pprior using a prior network, then sample the unobserved features Pgen using the generative network
p. ( = J. Pprior (1l ( (81 2,)1,,
We only use the VARAC for the CelebA Smile experiment and leverage the architecture and weights provided in the paper.
B.2 GAUSSIAN MIXTURE MODEL (GMM)
The Gaussian mixture model allows for fast imputation of masked samples for low to medium size dataset when using spherical Gaussians. Before training. fit the mixture model the fully observed dataset by maximizing the log likelihood of the model Eq. (TO) using the expectation maximization algorithm.
practice, we use the Gaussian Mixture Model library from [Pedregosal et al. (2011) to obtain the parameters (n** 1 Σ²)x- In the case of spherical Gaussians, Σ₁ is diagonal matrix. thus Vk, (4g, Σσ are the size of the input dimension
LGMM(x)= = Σ T N 5TK).
(10)
To obtain sample imputing the missing variable, start by calculating p(k]a,, 2). In the particular case of spherical Gaussian, this allows us to consider only the unmasked features when calculating this quantity.
14

---

Under review as a conference paper at ICLR 2023
)z, (E*)=)
2k) Ex (i )2))
(11)
Finally, we sample center from the previous conditional distribution and sample from the associated Gaussian:
(=( ( ,
(12)
In practice, train the Gaussian mixture model on discrete image, we add some uniform noise to the input data to help the learning of the input data.
B.3 MEANS OF GAUSSIAN MIXTURE MODEL (MEANS OF GMM)
We propose an extension of the previous GMM model to get more in distribution samples from the dataset. After sampling center from the conditional distribution Eq. (II), instead of resampling the imputed values from the Gaussian distribution, we can use directly the means of the centers of the sampled center as imputed data.
())
Σ 2
(13)
B.4 DATASET GAUSSIAN MIXTURE MODEL (GMM DATASET)
In a second extension of the previous GMM imputation we make use of the validation dataset to create imputations. To that end, we store the quantity Presampling (r"malls [k) for every center and every data point aiaa in the validation set, where
N( (a luk, Σx) N Hux Es)
Presampling ( (Iiaa (k)
(14)
After sampling center from the conditional distribution, we sample one example according to the distribution in Eq. (1).
Hence
(T( Σ p(k|22, 2)
Σ Presampling (I &
(15)
B.5 KMEANS AND VALIDATION DATASET (KMEANS DATASET)
By using the KMeans instead of the GMM for the GMM Dataset, we can obtain simpler multiple imputation methods. To that end, we calculate the minimum distance between a masked input and the masked centers of the clusters 14k. We select cluster ie* closest to the input data and we can sample uniformly any image from the validation dataset belonging to this cluster.
B.6 MIXTURE OF LOGISTICS
We propose to use a mixture of discretized logistic as an approximation for the true conditional distribution. For each center of the mixture among K centers, we consider set of D center p and scale parameters s which allows the creation of discretized logistic distribution for each pixel similar to the construction in Salimans et al. (2017). We obtain the parameters by maximizing the lig-likeldhoo of the model Eq. (16:
p(x)
Σ k
Σ
logistic(zaallūjkkk st))
(16)
d)
where logistic( ( > o(xa + 0.5 --
/=)))))))
and is the logistic ( / -
sigmoid function. In the edge cases of pixel value equal to 0, we replace 0.5 by -oo and for 255 we replace 0.5 by +00.
15

---

Under review as a conference paper at ICLR 2023
We initialise the model means and weights by using the K-Means algorithm from Skleam Pe- dregosa et al. 2011). We then learn the model either by stochastic gradient ascent on the likelihood or by stochastic EM (both methods leads to similar choice of parameters).
Similarly to the GMM, we sample an imputation by first sampling a center from the mixture using the distribution p(k(ar, z), where
Ed logistic(=1uu; +))
=)
(17)
EN ThK Ede(o.D) logistic(( 1 $
We can then sample the imputed data using the parameters obtained from the subset restricted sam- ple, that is:
p,(F(a, = Σ E(kk),, Σ
logistic(zallá. s5)
(18)
d|aa=1
B.7 MEANS MIXTURE OF LOGISTICS
Instead of sampling from the logistic distribution, after sampling center & from the conditional distribution Eq. (II) we can use directly the means of the sampled centers as imputed data, that is,
2) Σ p(k|2z, Σ E1am=
(19)
B.8 VALIDATION DATASET (MARGINAL)
We can sample randomly from the validation dataset to replace the missing value from the unob- served dataset. This corresponds to approximating the conditional true imputation Pdota (x1-a/11 by the unconditional marginal distribution Pdata (x1-2). This type of multiple imputation were al- ready use for
(x(r) 2) Pdata (i1=))
(20)
C DATASET DETAILS
C.1 SYNTHETIC DATASET GENERATION
The input features for the synthetic datasets follow the generation procedure
( ~ 1),, y~ ~~B (+](a))
(21)
We consider different functions for different situations:
fa(x) exp(x1122)
fa(z) = exp (i x -4. 4).
fo(z) exp(-10 (0.2x7) + [as) + 2g + c" - 2.4).
This leads to the following datasets:
S1:
f(x) = { fA(z)
if < 0 if 0.
(22)
[B(L)
S2:
{
fa(x)
if <0 if 2.
f(x)
(23)
So(x)
S3
[n(x)
if <0
f(x)
fc(x)
if if 2 0.
(24)
16

---

Under review as a conference paper at ICLR 2023
C.2 PANEL MNIST AND FASHIONMNIST
Both dataset MMNIST and FashionMNIST are transformed in the same fashion as Jethani et al. (2021a) by dividing the imput features by 255. Note that this transformation will affect the choice of the optimal constant of imputation for LEX. We create the train set and validation set of the panel dataset by using only images from the train datasets of MNIST and FashionMNIST and the test set by using only images from the test datasets. The split between train and validation is split randomly with proportion 80%, 20%. Hence, the train dataset of the switching panels input contain 48,000 images, the validation dataset contains 12,000 images and the test dataset 10,000 images.
C.3 CELEBA SMILE
The CelebA dataset consists of 162, 770 train, 19867 val- idation and 19962 test color images of faces of celebri- ties of size 178 218. Before training any model, we crop the image to keep 128 x 128 pixels size square in the center of the image (using Torchvision's Center- Crop function and we normalize the channels in datasets. Note that we use this center crop to be able to use directly the weights and parametrization provided by the author of Ivanov et al. (2018) with CelebA. We define the target of this dataset using the attribute given for smiling in the dataset.
We can create the maximum ground truth boxes by using the landmarks position of the mouth. Given the position of both extremities of the mouth, direction and the lenght of the mouth. Supposing not only the mouth but also the region around is useful for the clas- sifier, we create the maximum ground truth boxes using rectangle oriented following the direction of the mouth, centered at the center of both mouth extremities and with height corresponding to two times the lenght of the mouth and width corresponding to the length of the mouth.
obtain both the
Two examples from the CelebA smile dataset associated with their ground truth selection.
Figure 9:
D EXPERIMENT DETAILS
For every experiment, use importance weighted lower bound with 10 importance samples for the mask and single important sample for the imputed values. We estimate the monte carlo gradient using REBAR with the weighted reservoir sampling and the relaxed subset sampling distribution as control variate.
Synthetic Dataset The predictor pe is parameterized with a fully connected neural network fo with 3 hidden layers while the selector p, parameterized with fully connected neural network 9y with 2 hidden layers. The hidden layers are dimension 200 and use ReLU activations. The predictor has softmax activation for classification while the selector uses sigmoid activation. We trained all the methods for 1000 epochs using Adam for optimisation with learning rate 10 and weight decay 10-3 with batch size of 1000. Both selector and predictor are free during training for experiment with constant imputation and multiple imputation. When using a surrogate imputation, the surrogate is trained the same time as the selector according to the algorithm Jethani et al. a02
Switching Panels The predictor is composed by 2 sequential convolution block that outputs re- spectively 32, 64 filters. Each block is composed with 2 convolutional layers and an average pooling layer. We fed the output of the last convolutional block to fully connected layer with softmax activation. The selector is a U-Net Romnebbrerrrr et al. (2015) with down sampling and up sampling blocks and sigmoid activation. The U-Net outputs 28x56x1 image mask corresponding to the parameters for each pixel in the image. We trained all the methods for 100 epochs using Adam for optimisation with learning rate 10-1 and weight decay 10-3 with batch size of 64. Both selector
17

---

Under review as a conference paper at ICLR 2023
and predictor are free during training for experiment with constant imputation and multiple impu- tation. When using a surrogate imputation. the surrogate is trained at the same time as the selector according to the algorithm in Jethani et al. (2021a).
CelebA the predictor neural network fa is composed by 4 sequential convolution block that out- puts respectively 32, 64, 128 and 256 filters. Each block is composed with convolutional layers and an average pooling layer. We fed the output of the last convolutional block to a fully connected layer with softmax activation. The selector is a U-Net (Ronneberger et al. 2015) with down sampling and up sampling blocks and a softmax activation. We lower the number of possible mask by creating x 32 grid of 4 pixels over the whole image. The U-Net outputs a 32 32 x image where each feature corresponds to the parametrization of 4 x 4 pixel square in the original image.
For experiments with constant imputation multiple imputation, train the predictor for 10 epochs. We then train the selector using the pre-trained fixed classifier for 10 epochs. For experiments with surrogate constant imputation, independent Bernoulli distribution to mask every pixel (this corresponds to the objective of EVAL- X in Eq (5) in Jethani et al. (2021a)). We then train the selector using the pretrained surrogate for 10 epochs. We optimize every neural netowrk using Adam with a learning rate 10-4 and weight decay 10-3 with batch size of 32
train the surrogate for 10 epochs using
E A DIFFICULT OPTIMIZATION PROBLEM
E. AN IMPORTANCE WEIGHTED LOWER BOUND
To maximise Eq. 5 inequality, we can get a lower bound on this log-likelihood.
one difficulty resides in the two expectations inside the log. Using Jensen
((0,7)
log Ez Z- (12 Rxpp (e
n=l
Ez (12") P
Xxxt())))))))11))111 log(po(v((X))
This bound may be too loose and give poor estimate of the likelihood of the model. Instead, propose to use importance weighted variational inference (IWAE) Burda et al. (2015) so we can have a tighter lower bound than with Jensen inequality. Note that have apply two times the IWAE, for the expectation on masks and on the imputed features. For each data point Fn, we sample L mask importance samples and L.K imputations importance samples.
The IWAE lower the log-likelihood on the mask with mask samples
[(,,7) (0)
(25)
where
N
L
(0,7)) Σ E Am ( 8 8
log L
Σ
(v (xnl))
For fixed mask imputations, the IWAE lower bound with K imputation samples is
Lr(0,)) 2 (0,
(26)
where
N
K
LL.K (0,7) =
Σ
E Ez ..... Zo (12 (iapp..
log LL LK
Σ
Σ
Po (xm(()
18

---

Under review as a conference paper at ICLR 2023
Theorem 3 of Domke & Sheldon (2018) ensures that when L t00,
Lr(0,)) [(0, [(0,7) + (
and, for a fixed L,
1 K
(0,k( +0
Thus, using a large number of importance samples for both the mask and the imputation ensures that we are maximizing tight lower bound of the log-likelihood of the model.
Note that other models falling into the LEX framework are optimizing a lower bound of the log-likelihood (see Appendix A for details) with the Jensen inequality. However, casting their method into the statistical learning framework motivates to choose a tighter lower bound of this log-likelihood which improves results in classification and selection.
E.2 GRADIENT MONTE CARLO ESTIMATOR
We want to train the maximum likelihood model with stochastic gradient descent. Using Monte Carlo gradient estimator for is straightforward. Finding Monte Carlo Gradient estimator for y is more complicated because the expectation on masks depends on and we sample from discrete space {0. 1)P. A simple way of getting an estimator for this gradient by using policy gradient estimator (uutoon et al. 1999) or Score Function Gradient estimator (Mohamed al. 2020).
On the other hand, by relaxing the discreteness of the distribution, it is possible to reparametrize the expectation in and use pathwise monte carlo gradient estimatorMMohame These estimators introduce some bias but lower the variance of the estimation. For instance, Yoon et al. (2088) proposed use the concrete distribution (Maddison al., (2016) which is continuous relaxation of the discrete Bernoulli distribution. Similarly, Xie & Ermon] (2019), [Chen et al. (2018)
al. 2020).
used some forms of continuous relaxations of subset sampling distribution.
RealX (Jethani et al. 2021a) proposed to use REBAR (Tucker et al., (019) to further reduce the
variance of these gradient estimators while still keeping an unbiased estimator thereof by using the relaxation of the discrete distribution as a control variate for a score function gradient estimator.
When using multiple imputation, there is possibility to use continuous mask as the imputa- tion distribution p. 2, x)) would not be properly defined. To that end, we leverage the allowed reparametrization the continuous relaxation of the discrete distribution but still apply a straight- through estimator [engio et al. (213).. When using independent Bernoulli for pr, we consider a thresholded straight through estrmator with threshold 0.5. For relaxed subset sampling, we use top K function for the traight-through estimator with l: being the number of features to be selected. Using this new straight-through estimator for the different continuously relaxed distribution, we can either use pathwise Monte Carlo gradient estimator or variation of REBAR where the relaxation modified by the straight-througe function.
F EXTENDED RESULTS
F.1 EXTENDED RESULTS WITH A FIXED SAMPLING RATE
SP-MNIST In Fig. II we observe results for different constants of imputation using a surrogate on SP-MNIST. We see that changing the constant of imputation for the surrogate may drastically change the performance of the selection.
On Fig 20 we can observe the average of 100 samples from a model trained with different constant imputation and a surrogate function and an average rate of selection of 5%. We can see that the constant imputation drastically changes the shape of the imputation even though we are using surrogate function. When using 0, the selection model seems to try to recreate the full image instead of selecting the correct panel. Using constant 1 and seems recreate the negative of the shape of the two on the correct panel. Finally, when using constant -1, the selection recreates the target number in both panels to facilitate classification. These samples using surrogate function can
19

---

Under review as a conference paper at ICLR 2023
6.7
ACC
ACC
6.00-
0.-
0.50
6.8
0.2
0.1
08
0.2
0.4
26
066
022
04
26
Effectiie Selection Rate
Effectiiv Selection Rate
Efffective Selection Rate
0.8
6.8
FDR
6.6-
FDR
FDR
0.8
6.4
6.0
26
Effective Selection Rate
Effective Selection Rate
Effective Selection Rate
1.00
1.6
0.70
TPR
TPRR
0.50
6/2
0.1
0.
0.8
3/5
0.
0a
0.
Effective Selection Rate
Effective Selection
Elfective Selection Rate
Surrogate - Imputation
Surrogate -3 Imputation
Surrogate Imputation
Surrogute Imputation
Surrogate Imputation
Figure 10: Performances of LEX with different imputation constants using surrogate constant imputation the synthetic datasets. This corresponds to the REAL-X parametrization with different constant imputation. Columns corresponds to the three synthetic datasets (S1, S2, S3) and lines corresponds to the different measure of quality of the model (Accuracy, FDR, TPR). [mean + std over folds/generated dataset]
20

---

Under review as a conference paper at ICLR 2023
be considered cheating the selection is used to improve the classification results. As opposed, samples using multiple imputation on Fig. 21 are less dependent on the type of multiple imputation used.
Surrogate 0 Imputation
Surrogate Imputation
Surrogate Impotation
Surrogate - Imputation
0.6
Surrogate -3 Imputation
0.
5.1
0.2
033
0.
0.0
0.1
0.2
0.3
0.4
Effective Seleectio Rate
Efffective Selection Rate
Figure 11: Performance LEX using a surrogate constant imputation for different imputation con- stants on the SP-MNIST dataset.
10:
Krocans Dataset
0.3
GMM
ACC
Means GMM
Latent GMM
Means of Mixture of Logistics
0.0
Mixture of Logistics
Effective Selection
0.1
0.2
Effective Selection
0.3
GMM dataset
Figure 12: Performances of the LEX method with different methods of multiple imputation on the SP-MNIST dataset.
SP-FashionMNIST
Following the same procedure as Switching panels MNIST, we consider the dataset Switching panels FashionMNIST. Since on average 50% of the pixels are lit in FashionM- NIST, we expect the true selection rate to be around 25 % of the total number of pixels in Switching Panels FaShionMNIST.
In Fig. 13 compare LEX with two methods of multiple imputation, the Mixture of Logistics and GMM Dataset. These two multiple imputations outperforms their constant imputation counterparts with and without surrogate near the expected true rate of selection. Using the mixture of logistics allows to maintain a strong accuracy, similar to the accuracy of both constant imputation methods. observe that the selections obtained with the surrogate constant imputation is more robust to the change in constant imputation compared to the Switching Panels MNIST dataset but the variations are still higher that the variations obtain with many different multiple imputation in Fig.
In Fig. 14
O Imgulation
0.4
ACC
3.5
Mixture of Logistics
Surrogate Imputation
GMM dataset
00
0.
022
04
0.0
Effective Selection Rate
Effecttive Selection Rate
Figure 13: Performances of LEX on the SP-Fashion dataset with different methods of approximation for the true conditional imputation.
CelebA Smile On Fig. 16 as opposed to the results on SP-MNIST and the synthetic datasets, that the performance of L.EX using surrogate constant imputation does not depend on the choice of a constant.
21

---

Under review as a conference paper at ICLR 2023
0.5
Surrogate 0 Imputation
Surrogate Imputation
0.4
Surrogate Imputation
0.3
Surrogate - Impptatiion
Surrogate -3 Impatation
0.0
0.2
0.4
0.0
0.2
044
0.6
Effective Seloction Rate
Effective Selection Rate
Figure 14: Performance of LEX using a surrogate constant imputation for different imputation con- stants on the SP-MNIST dataset.
Mixture of Logistics
Means of Mixture of Logistics
ACC
0.55
Kmeans Dataset
0.2
0.80
GMM datasel
0.0
0.2
64
Effective Selection Rate
Edfective Selection Rate
Figure 15: Performances of the LEX method with different methods of multiple imputation the Jp-FaahionMNIIT dataset.
Surrogate Impulation
EVR
Surrogate Impulation
Surrogate Impulation
Surrogate 1 Imputation
Surrogate 3 Imputation
0.05
0.02
8:15
0.20
Eflective Selection Rate
Figure 16: Performances on the CelebA smile dataset with different constant for the surrogate con- stant imputation.
F.2 EXTENDED RESULTS USING L E.I-RGUULARIZATIO
To provide a better analysis the influence the imputation method on the performance of LEX, we fixed all the other parameters hence slightly changing the original methods (Invase and REAL- X). We differed from the original implementations in two ways, the choice of the regularization method and sampling distribution Py as well as the choice of the Monte-Carke gradient estimator. Finding an optimal with LI LI-regularization is difficult as different ranges lead different per- formances and different rates of selection depending the datasets. the imputations (see Appendix F2.Ip. While we can estimate an adequate selection rate with intuition from the dataset, no such intuition available for À which requires long exploration. In that section, will study how these choices might affect the performances of LEX and compare to the original implementations using the SP-MNIST dataset.
F.2.1 ON THE DIFFICULTY OF TUNING
We proposed in Section 4 to study LEX with fixed selection rate because finding an optimal requires an extensive search and makes it difficult to compare the results between sets parameters. In this section, study how different lead to very different rates of selection depending the sets parameters considered.
We fix the regularization to L1 R-guularizztion and the distribution p., be independent Bernoulli (this is the same setting for distribution and regularization as Invase and REAL-X) and fix the Monte- Carlo gradient estimator REBAR. We study the evolution of the rate of selection with parameter
22

---

Under review as a conference paper at ICLR 2023
1.00
Selection
Imputation
Logistics
0.25
Surrogate Imputation
lng(A)
Figure 17: Effective selection rate depending on the value of À of the same LEX model for different types of imputation trained on the SP-MNIST dataset.
varying in [0.01, 0.05, 0.1, 0.3, 0.5, 2.0, 5.0, 10.0, 100.0). We observe in Fig. 17 that depending on the imputation, the evolution of the rate of selection is very different. Since we want to compare different methods on the same "credit" of selection (ie the same average rate of selection), we have to search very large range of À in practice.
F,2.2 INFLUENCE OF THE MONTE-CARE GRADIENT ESTIMATION
In their original implementation, L2X, Invase and REAL-X X use different timators for the optimization. L2X uses Gumbel-Softmax Maddison et al. (016)) continuous relaxation of the discrete Bernoulli distribution which is a biased but low variance estimator. Invase uses the REINFORCE Sutton al. (1999) estimator, an unbiased but high variance estimator: They proposed to control the variance using baseline network, trained without selection, as control variate. REAL-X uses the REBAR Tucker et al. 2019) estimator, a REINFORCE estimator using Sufbel-Softmax control variate. We chose to focus on REBAR as it can be considered an improvement over REINFORCE and Gumbel-Softmax.
gradient es-
1.0
3,0
100
0.8
ACC
ACC
0.6
0.7
0.7
0.4
0.2
055
Effectivv Selection rate
Effective Selection rate
Effective Selection rate
1.00
0.5
0.02
179
0.50
0.03
0.2
0.35
0.00
Rfffective Selection rate
Effective Selection
Effective Selection
REBAR PATHWISE REINFORCE
Figure 18: Comparison of the performance of LEX models with Ll-reuularization with 3 different Monte-Carlo gradient estimator on the SP-MNIST dataset. À varies in 00.1, 0.5. 1.0. 3.0, 5.0. 10.0,]- First column corresponds to 0-imputation mimicking the be- haviour of INVASE for different MC gradient estimator, second column corresponds to surrogate O-imputation mimicking the behaviour of REAL-X and third column corresponds to multiple impu- tation with a Mixture of Logistics
In this section, we fix the regularization to L.-Regularizztion and the distribution p,, to be an in- dependent Bernoulli and we compare the difference in performance depending the Monte-Carlo gradient estimator (REINFORCE, Sufmeel-Softax, REBAR) for different methods imputation. Note that plain REINFORCE slightly differs from Invase (because of the baseline control variate) but they admitted in the reviews for their paper Yoon et al. (019) that using the control variate did not improve the results. In Fig. 18 we observe that changing the Monte-Carlo gradient estimator
23

---

Under review as a conference paper at ICLR 2023
leads to similar results in prediction and selection. However, changing the Monte-Carlo gradient estimator changes how the choice of À impact the selection rate.
F.2.3 COMPARISONS WITH THE SET-UPS L2X, INVASE AND REAL-X
Here we propose a comparison of the original set-ups of 1.2X, Invase and REAL-X to two parame- terisations of LEX models with multiple imputation. The first one is the same set-up in I using REBAR and an implicit regularization with a fixed rate selection. For the second set-up, we train using REBAR but consider Independent Bernoulli for the selection distribution regularized by an explicit LI-Regularisation (note that this is the same regularization and distribution of Invase and REAL-X). À varies in [0.1, 0.3, 0.5, 1.0, 2.0, 5.0, 10.0] for the method with L1 Re-ulularizztiin.
1,2
Invase
ACC
Ll-Regglarization
L2X
0.0
Ours Fixed Selection Rate
0.5
0.1
112
03
0.4
0.0
0.1
0.3
0.4
Effective Selection rule
Effective Selection rale
Figure 19: Performances of LEX trained on the SP-MNIST dataset using the same sets of param- eters as the original implementation of L2X, Invase, and REAL-X. We compare them to two sets of parameters with Mixture Of Logistics imputation (denoted Ours) using types of regular- ization: L1 L1-Regularization and implicit regularization with a fixed selection rate. The models with LI Ll-regularization were trained on the same grid X [0.1, 0.3, 0.5, 1.0, 2.0, 5.0, 10.0.].
we see that our method (i.e. using multiple imputation with Mixture of Logisties) provides best performances in the vicinity of the expected selection rate. On the other hand, even with À very close 0, Invase still selects very small subset of the features making it difficult to compare to the other methods. Morcover, the accuracy Invase, REAL-X, and L2X do not decrease with the effective selection rate which suggests that these methods encode the target output the mask selection.
On Fig. 19
24

---

Under review as a conference paper at ICLR 2023
Lee
2
/
0 0
C
0
6
2
6
6
Each figure corresponds to the average of 100 mask samples from the selector trained using a surrogate constant of imputation for different constant. From top to bottom, we have the input data, and constants -3, -1, 1,3. The selector was parametrized by subset sampling with a rate of selection of 5%.
Figure 20:
Lee
2
/
0
2
2
2
Figure 21: Each figure corresponds to the average of 100 mask samples from the selector trained using different multiple imputation. From top to bottom, the different multiple imputation are KMeans Dataset, Means of GMM, Mixture of Logitics, means of mixture of logistics. The selector was parametrized by subset sampling with rate of selection of 5%.
25

---

Under review as a conference paper at ICLR 2023
Input Image
Marginal Imputation
Surrogate
Constant imputation
Figure 22: Selection obtained averaging over 100 samples from the distribution P., parametrized by
a subset sampling distribution selecting 10% of the pixels.
26


Reviews:
[{"review_id": "_V_MO0n99Zc", "reviewer": "Reviewer_DxL9", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper introduces a new method for jointly learning a predictor and an interpretability model. The method\noffers a general framework that has LEX, INVASE, and REAL-X as special cases. The paper also introduces two more\ndatasets synthetically created from FashionMNIST and CelebA with ground truth selected features.\n", "summary_of_the_review": " Interesting and significant contribution to the interpretability\n literature. Paper would benefit from clarifying its unique\n contributions.\n", "strength_and_weaknesses": "The paper is clearly written and makes a real contribution. The\nframework outlined really does seem to add flexibility by allowing for\na more generic imputation strategy to be used. I am curious, why\ncouldn't the existing methods be adapted to just use multiple\nimputations?\n\nMy main concern is that the main benefit is the ability to use\namortized imputation methods than necessarily any other portion of the\nframework. This comes across a bit incremental, as all other methods\ntrivially allow changing the selector network and predictor network.\n\nI have some quibbles with the title as it's hardly the first paper that\nlearned an explainable model as a statistical inference task, but perhaps\nthis is best cleared up with clarifications of the ways this paper is statistical\ninference and Chen et al are not.\n\nIt is mentioned that selector when using LEX should still end up encoding the predictor,\nbut I didn't really see any experiments demonstrating that.\n\nUpdate: given the contribution seems to be the framework, I think the paper would greatly benefit from some rewriting to better highlight the strengths of that framework. Though I should stress I think the imputation methods discussed in this work are very interesting and would be a great contribution on their own.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "Zt1EHtkTxv", "reviewer": "Reviewer_Hv7W", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes to frame the model interpretability problem as a statistical inference problem and develops a general probabilistic model that trains a predictor and a selector network via maximum likelihood to produce interpretable predictions. The proposed framework LEX is modular and able to encompass existing models including L2X, Invase, and REAL-X. The paper also introduces two datasets with ground truth selection for evaluating the proposed method.", "summary_of_the_review": "This paper proposes a unified probabilistic framework to solve the interpretability problem as a statistical inference problem, which could be inspiring to the community. However, the authors need to clarify their main contribution and justify why the current baseline comparison experiments are sufficient to demonstrate the proposed method's effectiveness.", "strength_and_weaknesses": "Strengths: The proposed method offers a general framework for casting the interpretability problem as a statistical inference problem, encompassing existing methods such as L2X, Invase, and REAL-X. In addition, the insights on using multiple imputation in the proposed procedure could be useful to others.\n\nWeaknesses: The evaluation is not sufficiently convincing. The paper mainly demonstrates the effectiveness of multiple imputation over other imputation variants under the LEX framework, instead of directly comparing LEX with the other existing methods. Moreover, the technical contribution of their proposed method over existing ones has not been stated clearly in the paper. For example, though it does provides a unified framework, it is not the first paper framing the interpretability problem as a statistical inference problem.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "M2rjTrCqgW", "reviewer": "Reviewer_sxVY", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies amortized instance-wise feature-selection methods. It proposes Latent Variable as Explanation (LEX), a new framework that frames interpretation as a statistical learning method using a unified probabilistic likelihood, and shows how various existing methods can be understood under the LEX framework. The paper proposes two new datasets, a switching panels dataset constructed using MNIST and FashionMNIST, and the CELEBA SMILE dataset, to help evaluate various amortized evaluation methods. Using the proposed LEX framework, the paper demonstrates the advantage of multiple imputation over the combination of constant imputation with surrogate predictors.", "summary_of_the_review": "While the proposed LEX framework is conceptually appealing, and the newly proposed datasets are well-designed and useful, the paper in its current form is poorly organized and lacks clarity. Additionally, the baseline setups seem problematic. In my opinion the paper is not ready for publication in its current form, but I encourage the authors to improve the writing and organization and include some additional baseline comparisons and resubmit in the future.\n\n------------------\n\nUpdate after rebuttal: I thank the authors for revising the paper and taking into account my comments. However, after discussing with other reviewers and the AC, we still feel that the paper should either rephrase as presenting the benefits of multiple imputation, or put more efforts into demonstrating the usefulness of the general LEX framework. As a result I am keeping my score.", "strength_and_weaknesses": "### Strength\n\n- The LEX framework is conceptually appealing, and on a high level is presented in a way that is easy to follow.\n- The two new datasets are well-designed and make a lot of sense of evalaution of instance-wise feature-selection methods.\n- The observation on the advantage of multiple imputation is valuable and intuitive.\n\n### Weaknesses:\n\n- The main weakness lies in the clarity of the presentation.\n    - While on a high level, the LEX framework is presented in a way that is easy to follow, the paper seems to be carelessly written, and is filled with typos, grammar errors, undefined terms and inconsistent notations. A non-exhaustive list of examples include:\n      - Figure 1 caption, a standard approaches uses\n      - In Section 2.3, why is f_theta is a classifier? It's just a function mapping the input to the parameters of some distributions.\n      - In equation 4, why aren't we optimizing the parameter iota? Does the imputation involve any parameters?\n      - First sentence of last paragraph in Section 2.4, missing a product in the definition of R?\n      - The regularization function R is not consistent between Section 2.4 and Section 2.5. What exactly is the domain of function R?\n      - Many terms don't seem to be clearly explained. What is 0 imputation? What is surrogate posthoc?\n      - Section 3, data distribution q, why do we call it a data distribution? It seems data distribution should be p_data. And how is q different from p_{\\iota}? Why do we change notation here again?\n      - What is multiple imputation? Seems like just using a generative model to approximate the true imputation distribution derived from the joint data distribution? Need to be defined in the main text.\n      - Section 4 2nd paragraph, want to use of → want to use.\n      - Maximum/minimum ground truth selection, used without definition.\n      - Section 4 2nd paragraph last sentence, provides → provide\n      - Section 4 3rd paragraph, features k s? Typos?\n      - InSitu or In-Situ?\n      - Section 4 says all the experiments are conducted in the In-Situ regime, but there are two In-Situ regimes according to Section 2.6. What exactly is the setup? Also according to Table 1, several existing methods are not in the In-Situ regime, which seems to suggest the paper is not including any of those methods as baselines.\n      - Section 4 mentions a 100 feature importance maps. It's not immediately clear what it means. Also in the next sentence it becomes 100 features instead of 100 feature?\n      - Section 4 4th paragraph, three measures, not measure.\n      - Section 4.1, selection evaluation, it's not clear what selection rates mean.\n    - Too much materials are pushed into the appendix, to the point that it becomes challenging to evaluate the paper based just on the materials in the main text. Some examples include:\n        - Section 2.4 talks about different regularization approaches. But which regularization does this paper use exactly?\n        - Section 2.5 there should be brief discussions on how existing methods fit into the LEX framework in the main text.\n        - Section 3, since multiple imputation is an important point that this paper tries to make, there should at least be brief discussions on what multiple imputation means, what are some of the methods this paper is using in the main text.\n        - Section 4.1, just from the main text it's close to impossible to understand what the 3 synthetic datasets look like. There should be at least a brief description. But some of the existing detailed setups can potentially be moved into the appendix.\n- Additionally, the way the baselines are set up seems problematic:\n    - The paper frames three existing methods (L2X, Invase and REAL-X) under the LEX framework. However, in the experiments the paper only compares multiple imputation with variants in the LEX framework that only loosely correspond to L2X, Invase and REAL-X. For example the paper uses a customized gradient estimator to handle the discrete latent variables, while existing methods adopt other ways of obtaining gradients estimation (e.g. gumbel-softmax and rebar). This makes the comparison difficult to interpret due to multiple moving parts. It would be helpful to also include performance of the original methods in the comparison (e.g. can we use multiple imputation similarly in the original methods? How does that compare with the optimal setup under the LEX framework? How do the original methods perform on the two newly proposed, more complicated datasets?)\n- Finally, while the LEX framework is conceptually appealing, the paper does not do a very good job at demonstrating how the framework can benefit the study of instance-wise feature-selection methods. In the experiments the paper mainly demonstrates the benefit of multiple imputation. While intuitive, it seems to me even without the LEX framework we can similarly try to incorporate multiple imputation and study its benefits. It would be beneficial if the authors can more clearly illustrate the benefit of the LEX framework for understanding and improving existing methods. For example, does it lead to insights with which we can design new methods to improve upon existing methods? The lack of fair comparison with existing methods (as explained above) makes it hard to access this point.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject