Ultra-marginal Feature Importance
Authors: Joseph Janssen
Vincent Guan
Venue: NeurIPS 2022 Submitted (2022)
Keywords: Feature Importance
AI Fairness
Interpretable Machine Learning
Removing Dependencies

Abstract: Scientists frequently prioritize learning from data rather than training the best possible model; however, research in machine learning often prioritizes the latter. Marginal contribution feature importance (MCI) was developed to break this trend by providing a useful framework for quantifying the relationships in data in an interpretable fashion. In this work, we aim to improve upon the theoretical properties, performance, and runtime of MCI by introducing ultra-marginal feature importance (UMFI), which uses preprocessing methods from the AI fairness literature to remove dependencies in the feature set prior to measuring predictive power. We show on real and simulated data that UMFI performs better than MCI, especially in the presence of correlated interactions and unrelated features, while partially learning the structure of the causal graph and reducing the exponential runtime of MCI to super-linear.

14 Scientists often seek to determine the true relationships between a set of characteristics and some outcome of interest. These relationships ideally determined by performing carefully controlled experiments so that causality can be established. However, experiments can be difficult and costly Ultra-marginal Feature Importance
Anonymous Author(s)
Affiliation Address email
Abstract
Scientists frequently prioritize learning from data rather than training the best possible model; however, research in machine learning often prioritizes the lat- ter. Marginal contribution feature importance (MCI) developed to break this trend by providing a useful framework for quantifying the relationships in data in an interpretable fashion. In this work, aim to improve upon the theoretical properties, performance, and runtime of MCI by introducing ultra-marginal fea- ture importance (UMFI), which uses preprocessing methods from the AI fairness literature to remove dependencies in the feature set prior to measuring predictive power. We show on real and simulated data that UMFI performs better than MCI, especially in the presence of correlated interactions and unrelated features, while partially learning the structure of the causal graph and reducing the exponential runtime of MCI to super-linear.
2
11
Introduction
13
16
17 to pursue, unethical perform, or impossible to control [51 [41,4), leaving only observational data available. The relationships that are hidden within vast quantities of observational data are often difficult determine, statistical tools, such feature importance, have been explored.
18
19
20
Recently, feature importance methods such as Shapely-valuee 40 [ 33 SAGE [4], accumulated
local effects (ALE) [3), permutation importance (PI) [8], and conditional permutation importance (CPI) [16] have been used in sighiimpctt journal papers by scientists who want to explain the mechanisms within data
22
42
29
38
[9]
26 However, these methods may not adequately ALE can only easily show first order effects [36]. and
24
explain data in certain circumstances 12
11.
although CPI improves upon some limitations of PI, CPI has the property that two perfectly correlated features with significant predictive power would both be deemed unimportant [14. Further, only one model is trained in ALE, CPI, and PI. Thus, correlated features, which can alter the model assembly process, could be given artificially low importance if the goal is to explain the data 24) Instead of exploring single model, the developers of SAGE, SPVIM, and marginal contribution feature importance (MCI) evaluate the difference in accuracy between a model trained with the feature of
26
27
28
29
30
31 interest and a model trained without it, across all feature subsets [4 49]. though these methods
32 are prevented from being accepted by wider scientific audience because of their high computational
33 cost. In particular, we note that MCT is the current mate-of-the--e method for explaining data as
34 it was shown in extensive experiments to have better quality and robustness when compared to
35 Shapely-values, SAGE, ablation, and bivariate methods
36 Though MCI can be seen as the current state-of-the-at method for explaining the data, it has three 37 key shortcomings. First, the exact computation of MCI requires an exponential number of model trainings, which makes MCI ineffective at interpreting large datasets (e.g., gene expression studies).
38
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

39
Second, although it can handle complex feature interactions and data with correlated features, MCI underestimates the importance of correlated features that form interaction effects because MCI usually ignores features that share information with the feature of interest zy. Even if and xj form an interaction effect, the additional predictive power offered by x; on top of subset S would be diminished by the presence of x; E S, provided that the correlation between 2;j and is strong enough. Third, MCI can give non-zero importance to features that are completely unrelated to the response variable, as experimentally shown in Catav et al. [Σ Figure S3] and theoretically shown in Harel et al. [23]. We hypothesize that constructing independent and iiformation-preeerve representations of the data could resolve these three issues. With this in mind, we introduce ultra- marginal feature importance (UMFD, a new variable importance method that can better explain the data while drastically reducing runtime.
40
41
42
43
44
45
46
48
50
The rest of this paper is organized as follows. Axioms for explaining the data are proposed in Section The framework for UMFI is then formally presented in Section alonn with its theoretical properties and its simple algorithm. In Section [ we conduct experiments on simulated and real data to assess the quality, robustness, and time complexity of UMFI compared to MCL. Finally, an overview of the work, its limitations, and ideas for future work are discussed in Section 5
Baction
51
52
54
55
Related work
This paper is greatly inspired by the development of marginal contribution feature importance (MCI) by Catav et al. [TT]. Although other methods, such as SAGE (14). have been retooled to better explain data [1]. up until this point, MCI had been the only feature importance method developed
57
58
59
specifically to explain data. Let F {.1....,TTr) Jpp be the set of features used to predict the response
60
variable, Y. Recall that the universal predictive power of a set of features S C F is given by
v(S) = min ((((((0), Y)] - min [EI((((),))) Y),, SEG(S)
(1)
SEG(D)
where / is a specified loss function and G(S) is the set of all predictive models restricted to using 62 features in S C F. v is closely related to mutual information, with equality under ideal conditions
61
63 [1], and in practice, v is often approximated by machine learning evaluation functions. Using this,
64
Catav et al. 11 defined the marginal contribution feature importance (MCI) of a feature I; E F by
I (xx)
max v(S U (x:1) v(S).
(2)
SCF
85
To achieve goal of improving upon the shortcomings of MCI, we evaluate the importance of feature of interest I; after preprocessing the data to remove dependencies on z₁. Finding independent representations of predictors for creating improved feature importance methods is a novel objective, though similar ideas have been suggested as future work in König et al. (30 and Chen et al. [12]. The weaker concept of finding orthogonal representations of data has been discussed previously 18 though the discussion has been limited to relative importances measures for multiple linear regression, mostly in the domain of psychology (6] 52). While orthogonalizing predictors can be done easily with simple techniques, methods which can not only remove correlations between features, but also remove more general dependencies, have seen great progress within the domains of Al fairness and privacy. Some examples of these techniques include regression 77. optimal transport
66
68
70
71
72
73
74
75
28), neural networks [10 convex optimization [IO], and principal inertial components (45
78
Linear regression and optimal transport were implemented for UMFI in this paper.
77
2
Axioms for explaining data
78
Any attempt to build a method that explains the data should begin by rigorously defining what explaining the data truly means. Different definitions and goals have been formulated by Chen et al.
79
80 [22 and Catav et al. [ Inspired by these definitions, we provide three intuitive, justified, and
81 rigorous axioms for true-to-data feature importance methods. Given a feature set F, response Y,
82 and a feature of interest x; E F, the feature importance of x; is defined as ImpFF (a;) E Rzo- We define the following three axioms as vital for any method that claims to explain the data:
83
1. Elimination axiom: Eliminating a feature 2; from the feature set F can only decrease the importance of the feature of interest:
Vz, E F [tss).mp ( Mazii (xx)) S. Imp F.Y (x:).
2

---

require that the evaluation function 2. Duplication invariance and symmetry axiom: Adding a duplicate copy of feature å = 3; already in the feature set F will not change the importance of the other features in
F, and the duplicated feature will have importance equal to the original feature:
Va; E F. Imp" F.Y (x:) = Imp (U (xx) and Imp FU( Y (+) Imp Fupa)) (1z))
3. Blood relation axiom: If data is generated from a causal graph, feature I; will be given non-zero and positive importance if and only if it is blood related to the response Y in the causal graph. Two vertices in causal graph are said to be blood related if there is a directed path between them if there is backdoor path between them via a common ancestor.
Imp P.y (x:) > 0
E BR(Y).
84
The elimination axiom comes directly from Catav et al. m.. Once a feature observed to be significantly related to the response, the relationship strength between the feature and response should not drop, regardless of the additional features added. In fact, often times the importance should increase since adding features could reveal further synergistic information about the response Y.
85
86
87
88
The duplication invariance and symmetry axiom separates feature importance methods that are for data explanation from methods intended for model optimization [ A model may use the two identical features equally often and therefore spread the importance equally between them (random forests), or only one of the features may be given importance (lasso) [12.. However, from the data's perspective, both features should be equally related to the response and the original importance found before duplication should still be true. Further, after duplication, additional interaction capability is available 222, so the importance of all other features should remain the same.
90
92
94
The blood relation axiom asserts that feature importance scores intended for data explanation should extract reliable knowledge about the underlying causal graph and data generating process. statistical association between a feature and the response, which is a quality of interest for many applications (e.g., genome-widd wide association studies), exists precisely when the two features are blood related, or equivalently, when there is an open path between them (see Greenland et al. 20) and Williams et al. 48) for a more in-depth explanation of this definition as well as other relevant concepts about causal 100
95
96
96
graphs). Thus, feature importance metric satisfying this axiom would give non-zer importance to a feature if and only there is a statistical association between that feature and the response. Additionally, if the goal is to construct causal graph to represent the relationships in the data, then a feature importance metric satisfying this axiom can partition the feature set into features that are blood related to the response and features that not blood related to the response. Although it does not enable us to immediately recover the full causal graph, this partitioning may be a helpful supplemental tool for other causal discovery methods. See Supplement B for further discussion.
102
104
105
106
107
3
Ultra-marginal feature importance
108
109
Let F = {-1,..,, Fp be a set of p features of arbitrary type used to predict the response Y. We note that features may be viewed as random variables, or as realizations of random variables according to their joint distribution, in the form of dataset.
110
111
112
In order to define ultra-marginal feature importance,
113
v. which measures the predictive power of a group of features [I]] and which approximates Equation ( is also defined for transformations of the feature set following the removal of dependencies. We therefore define the space of information subsets of a feature set F as I(F) (F) g any function defined F). We call these information subsets of F because T((:0(F)) S (Y;F) holds for any function g by Theorem A.3
115
116
117
Definition We denote SE. as a preprocessed feature set after dependencies on the feature of interest I; have been removed from F. An optimally preprocessed feature set is denoted by SP and we say that preprocessing S is optimal if it obeys the following properties:
118
119
120
1. SF = [F(F) for some function
2. SF 11. x4.
122
3. 1(;;SF SE, x.)) 1(Y;F)
3

---

1: Let Y be the response variable of the set of predictors F. Choose feature x; F. The first property ensures that ST E I(F), and hence, no information from outside of F is gained during the transformation. The second property upholds that the random vector Sr. is independent of 24, and the last property affirms the optimality of St; in the sense that there is no unnecessary
124
126
information loss incurred during preprocessing. Given that it exists, an optimal preprocessing $F, is not unique, since scaling g( (P) by a constant does not affect the last two properties. In practice, the last two properties can be difficult to guarantee, but
127
128
later in Section 4 that non-optimal
130
preprocessings are good enough in many circumstances.
131
Definition Given an evaluation function v : I(F) - Rzo and a feature set F, we define the ultra-mant marranaarainal feature importance (UMFI) of feature I, E F as
132
Ug (x.)= = = U (x, v(S..).
(3)
UMFI obeys the three axioms given in Section 2 under certain assumptions proven in Appendix Mainly, we assume that vx-) = -): Under ideal conditions, this relationship holds when y satisfies Equation ① [14), but in practice, the accuracy of the approximation depends on the quality of the method, the specified loss function, and the response variable's ' distribution [15] See Covert [.5 and Appendix A..3 for more thorough overview.
134
135
136
137
al.
138
Since UMFI is model-agnostic, we provide a general algorithm for computing the ultra-marginal feature importance of feature z, E F, which can be applied using any pair of preprocessing and modeling techniques. We note that Vy is not restricted to the domain of machine learning models or even models in general. For example, one could also implement UMFI with measures of dependence such as the Salbert-Schmidd independence criterion [21] or non-ML estimates of mutual information BI. Furthermore, if machine learning modeling techniques are used for UMFI, we advise that the
140
142
144
median score over multiple iterations of the algorithm is used to account for the variance of Vy-
Algorithm I:
Algorithm for computing UMFI
2: Obtain Sf. by using technique that optimally removes dependencies on x; from F.
3: Specify method f and a corresponding evaluation function v/j.
4: Estimate the predictive power, Byy (S ), that S., has about Y.
5: Estimate the predictive power, (S, {x,)), that ST, {z;} has about Y.
6: return U/ (z:) = (S, U (z+t)) (S..
4 Experiments
145
146
We perform experiments to compare UMFI and MCI with respect to quality, robustness, and time complexity. To implement UMFI, we consider optimal transport (28 (UMFI_OT) and linear regres- sion 7 (UMFI_LR) as methods to remove dependencies from the data. A detailed overview of these implementations is shown in Appendix and experiments comparing these methods appear in F For all experiments, we use random forests' out-of-bag accuracy (R²2 OOB- DOB-accuracy for regression tasks and OOB classification accuracy for classification tasks) as the evaluation metric V H. We use the ranger R package to implement random forests with default hyperparameters and 100 for the number of trees [50]. All experiments were run in Microsoft R Open Version 4.0.2 35 Appendix Oopeedix(GGootainn Contains additional experiments comparing UMFI and MCI with other feature importance metrics including ablation,, permutation importance, and conditional permutation impor- tance. In the same section,
147
149
Appendix
151
153
155
156
rerun the experiments comparing MCI and UMFI using extremely randomized trees instead of random forests and do an additional comparison on a real dataset from hydrology [ Code for all experiments can be found in the Supplement.
157
158
159
4.1
Experiments on simulated data
160
We run UMFI on simulated data to verify that it performs well compared to MCI. The data in all simulation studies contains one response variable Y. four explanatory features 31, I2, 33, 24. and
161
162
1000 randomly generated observations. Each study is repeated 100 times to test stability.

---

163
4.1.1 Nonlinear interactions
164
Interaction effects are common in many scientific disciplines where assessing feature importance
185
prevalent, including hydrology [27 B2), genomies [T] B7), and glaciology [7 4 39].
So, as was done in Catav et al. [II we assess the ability of MCI and UMFI to detect nonlinear
167
interaction effects the data 33 We consider:
3 31, 23, 344 (0,
Y 21
x2 + (1gn(11 12) 03 24-
168
Feature importance metrics should ideally conclude that x1 and z2 have higher importance compared to 33 and 24 because of the extra interaction term, sign(11 * x2). Figure lajshows consistently good performance across all methods. Each method gave high relative importance scores to 21 and Z2. while Ty and 24 received less, but still substantial importance. All methods show similar variability.
189
170
171
4.1.2
Correlated interactions
Interacting features often correlated 25 27 So, this simulation study aims to repeat the nonlinear interactions study, except now z and 22 are highly correlated with eachother. In the same way, Z3 and x4 are highly correlated with eachother. Let A, B, C, D, E, ~ (O 1). We consider:
174
175
I A + B, 22 = B C, x3 D + E, x4 E+G
Y = b1 + 52 sign(21 x22 + xiz + 2:4-
Just with the interaction experiment with independent features, we would expect 21 and 2i be more important than and I4 because of the extra interaction term, sign(rı * z2). The results in Figure[[[clearlyy [bcclearly show that UMFI provides better estimations of feature importance compared MCI when correlated interactions are present. MCI estimates that all features have approximately the same feature importance scores, while both UMFI methods show significantly greater importance for 21 and compared to Iy and T4.. MCI fails in this experiment because it penalizes feature subsets that share information with the feature of interest z; when evaluating the importance of x; via Equation 2 For example, if we are assessing the MCI score for 21, since D2 is strongly correlated with 21, then the predictive power offered by Ti on top of subset S would be diminished by the presence of E S. Therefore, 22 is not utilized in the MCI score for 21, which prevents the detection of the interaction term sign( (x1 * x2)) UMFT able to detect this interaction because it can extract the information from 22 that interacts with 2 while keeping this extracted feature independent of z1- Although not yet tested, we suspect that similar results would be demonstrated in the presence of dependent, but uncorrelated interactions.
177
178
179
180
182
184
186
188
190
4.1.3
Correlation
191
Feature importance methods that seek to explain data, such as MCI and UMFI, should not change the measured importance of features in the presence of highly correlated or duplicated variables according to the duplication invariance and symmetry axiom. To test this, we implement simulation study similar to the ones found in Catav et al. [1]. Let (10 0.01). We consider:
192
193
31 U2, 34 N(O, 1), x3 x +
y = D1 T22
195
The addition of 23. which is approximately duplicate of 1 should not alter the importance of xi and #1 should remain equally as important as z2, since they have the same influence on the response Y. The results shown in Figure Ic show that both MCI and UMFI work reasonably well. As with the previous simulation experiment, the variability is consistent across methods. As was desired, UMFI with linear regression shows equal relative importance scores for 21 and xz. The importance given to
196
198
200
D2 was slightly greater than D1 according to MCI and UMFI with optimal transport. Interestingly,
MCI assigns some importance to IA, which was independent of the response, while both UMFI methods assign importance scores close to zero, Because of this, we conclude that UMFI with linear regression performs the best in this simulated scenario.
202
203
204
4.1.4
Blood relation
205
To ensure that UMFT true to the data and could be used learn part of the structure of the causal graph in theory as well as in practice, implement the blood relation simulation experiment. In this
5

---

30
24
20-
x1
x3
X2
X4
x1
Vartacle
Variable
(a) Nonlinear interactions
(b) Correlated interactions
UC1
UNPT
40
Verndee Ieportaroo
50
20-
25
0-
x1
x3
X1
x3
12
14
x2
X22
24
x1
Correlation
(d) Blood relation
Figure 1: Results for the experiments simulated data from Subsection Feature importance scores are shown as percentage of the total for each of d1 to 24 from 100 replications. Results are shown for marginal contribution feature importance (MCI), ultra-marginal feature importance with linear regression (UMFI_I.), and ultra-marginal feature importance with pairwise optimal transport (T)._OT).
207
study, data is generated from the causal graph in Figure 7 from the Supplement, which was inspired by the collider causal graph found in Harel al. (23. The feature S is unobserved, thus only zs and
208
209
Ia are blood related to the response Y. Because of this, according to the blood relation axiom, 23 and T4 should given high and positive importance while Ii and 22 should receive zero importance.
210
In Section 3, we proved that in ideal scenarios, UMFI will only give non-zero importance to blood related features. We hypothesize that we can extend this to real-world scenarios where non-Gaussian features and interaction information appear. To test this, we consider:
212
1 S ~ 10, 8 ((-1,1)), e e~ u(1.5005,0..
Exp(1)
32
33 x2 S
Y = 5 +
F4 Y +
214 The results shown in Figure Id indicate that MCI fails to distinguish the blood related features, since most of the importance is given to 21, I2 & BR(Y). In contrast, UMFI LR and UMFI OT detect 216 that In and I2 should have zero importance while giving most of the importance to 24 and the rest of the relative importance to 23.
4.2
BRCA experiments
219
use the same breast cancer (BRCA) classification dataset 4 used in previous feature importance studies including Catav et al. [I]] and Covert et al. [4 to test the quality and robustness of UMFI

---

on real data. The original data contains over 17, 000 genes and 571 anonymous patients that have been diagnosed with one of 4 breast cancer sub-types. We consider the same subset of 50 genes as in Catav et al. [1]] and Covert et al. [14] for easier computation and result visualization. Of the 50 selected genes, 10 are known to be associated with breast cancer, while the other 40 genes
222
224
225
are randomly sampled. This data was downloaded from https: //ggttttttuu.... com/ com/TAU TAU-MLwe11/ citoortancettreemmnnne parree/main/ee dataset (MIT License). In Catav et al. [1]] and Covert et al. [14. these 40 randomly sampled genes are assumed to be unassociated with breast cancer. However, to ensure a more definitive ground truth, we also randomly permute the values of these 40 genes across their respective 571 observations to further reduce the chance that these genes have any association with breast cancer. Quality is then measured with the true positive and true negative rates: the 10 BRCA associated genes should have some non-zero importance (positive), and the other 40 genes should have exactly zero importance (negative). These experiments were 200 times on different seeds and with different random sample of 500 patients for each iteration. Robustness is measured using the standardized interquartile range (SIQR) from the repeated experiments, which is calculated by dividing the average IQR across the 50 features by the average median. This experiment is too computationally intensive for MCI to be calculated exactly, so we implement MCI assuming soft 2-size submodularity.
226
Marginal-Contributta -ettoo-eeeuree--ee
228
230
232
238
234
236
237
(N)
0.00
BRCA Associatad
MCI Importance
0.15
Not BRCA Associated
0.10
0.00
8L0224
520
IOFIR
BRCK
BR042
02001
MARDHE
VTHOOOL
NER 1
0000090
CSTH
POUN
100000
TTCH
HRHL
ADMRC
coco
R_-1000
orseen
088P2
NG01
RL,0000
0.20
UMFI LR importance
BRCA Associated
0.15
Not BRCA Associated
0.05
SCLTTA
cone
GRGAS
COND1
SHCAT
TDKM
MAHOWL
PSL ME
PORERILS
0009
SNTR
CoMEL
crase
SSTPO
HONDTOL
SOMH
putit
UMFI OT Importace
BRCA Associated
0.15
Not BRCA Associated
0.26
BOL
JNG
COU01
TEXM
CSIRE
8OMPL,
TRASO
NOP
MSH
ADRASC
C1s-212
SATRU
BRMSS
ADOYE
P
HRGLL
Figure 2: Median feature importance scores provided by (a) MCI, (b) UMFI with linear regression,
and (c) UMFI with pairwise optimal transport, for each gene in the BRCA dataset after 200 iterations. Genes colored in blue are known to be associated with breast cancer while genes colored in grey random permutations of randomly selected genes, which we assume to be unassociated with breast cancer. The first and third quantiles of the scores are visualized for each gene.
We found that MCI and UMFI (UMFI_LA LL and UMFL_OT [O) correctly gave significant importance 239 to the 10 genes that are known to be associated with breast cancer (Figure []. Interestingly, the ordering of important features was similar across methods, with BCLI and SLC22A5 always
7

---

being the most important and TEX14 always being the least important of the 10 BRCA- -ssooiiase genes. However, MCI consistently gives non-zero importance all features, while UMFI correctly gives zero importance to the majority of the randomized genes. Furthermore, UMFT's performance in this experiment improves with increased iterations. After running the experiment 5000 times, both UMFI methods have a perfect overall accuracy when distinguishing between important and permuted features (Appendix G.2.I). I). Although UMFI scores have higher variability than MCI (Table I is clear from Figure 2 that UMFI separates the 10 associated genes from the 40 unassociated genes better than MCI does.
242
244
246
248
Table 1:
The standardized interquartile range (SIQR), true positive rate (TPR), true negative rate (TNR), overall accuracy (OA), and the number of features for which feature importance can be calculated within 1, 15, and 60 minute(s) are displayed after running the methods on the BRCA data.
Method
SIQR
TPR
TNR
OA
@ Imin
@ 15min
@ lhr
MCI (k=2)
6.6 %
0
0.20
35
80
130
UMFI (LR)
41.9%
1
0.975
0.98
500
2000
4010
UMFI (OT)
28.5%
0.775
0.82
300
1500
3000
249
4.3
Computational complexity
250
MCI must train and evaluate a model for each element of the power of the feature set, which implies ((2)) model trainings if there are features. the evaluation function y obeys soft k-size submodularity, then the maximizing subset has no more than & elements, which reduces the number of model trainings to 11p*+1) [II. UMFI circumvents the exponential training time since it can be evaluated immediately after removing the dependencies of x, from the feature F. To confirm the above statements, and to show that the extra model trainings required for MCI dominate the computation time for removing dependencies
251
253
254
255
256
UMFI, we ran a simple experiment. For range of dataset sizes from the BRCA data, we evaluate the computation time for calculating the feature importance scores of all features using MCI and UMFI. We ran this experiment for a dataset with
257
258
259
features, and then slowly added features until our given time budget of 1 hour ran out. Once all 50
260
BRCA features were used, more features were randomly generated. All datasets had 571 observations. These experiments were run using an Intel Core 19-9980HK CPU 2.40GHz with 32GB of RAM. Code was parallelized in R, and 12 of the 16 available threads were used.
1.0
0.8
Time (Hours)
0.8
g.
MCI
0.2
Approximate MCI
UMFI LT UMF
0.0
0
1000
2000
3000
4000
of Features
Figure 3: Computation time for a single iteration of each method including: MCI (dark red), MCT with the soft Assizesubmoduulrity assumption (pink), UMFI OT (light blue), and UMFI_L _IR (dark blue), plotted against the number of processed features from the BRCA data.
262
263 From Figure 3
we can observe that UMFI is approximately superlinear, with UMFL OT incuring more computational cost compared to UMFI LR. Giving each method I hour to run. MCI processed
8

---

285 19 features, MCI with the soft 2-size submodularity assumption processed 130 features, UMFI_OP
286
processed about 3000 features, and UMFI LR processed about 4000 features [Table[].
5
Conclusion
267
In this study, we introduced ultra-marginal feature importance (UMFI) a new method that uses preprocessing techniques, originally developed in the domain of AI fairness, to provide fast and accurate feature importance scores for the purposes of explaining data. We introduced three ideal axioms that feature importance measures should satisfy if they claim to explain the data, which are all satisfied by UMFI under some basic assumptions (Appendix (.. Optimal transport and linear 272
268
270
278
regression were explored as preprocessing techniques to remove dependencies from data. When compared with MCI, the previous state-of-the-art method for explaining data, experimental results showed that UMFI was able to provide faster and more accurate estimates of feature importance on real and simulated data, particularly in the presence of correlated interactions and unrelated features. UMFI's s superior time complexity could be leveraged run feature importance on larger datasets or to achieve more accurate results by utilizing its median scores after many iterations.
274
275
276
277
278
Throughout the work on this paper, several shortcomings appeared. First, we only considered two simple methods for removing dependencies, linear regression and pairwise optimal transport. Other methods certainly exist in the literature, including optimal transport with chaining (28) neural networks [0, 4T or principal inertial components [46]. Though our two methods performed fairly well on the real and simulated datasets in Section 4 optimal transport and linear regression failed to find representations of the data that were independent of the protected attribute when we tested the
280
282
283
284
methods on hydrology dataset with more shared information compared to BRCA (I] (Appendix
G.4. However, neural nets or principal inertial components certainly could have given better results. 286
287
Also, despite requiring significantly more computational cost, better methods for estimating the conditional CDF, or using optimal transport with chaining, should give better estimates for S.P when implementing UMFI OT. Even though dependencies were not removed optimally for the hydrology dataset, the estimates of feature importance were still reasonably accurate. Second, UMFI scores are less robust than MCI since they have higher variability, however, because of the significantly lower computational cost, UMFI can be run multiple times and averaged to increase robustness. Third, it is not clear how closely Vy approximates mutual information in practice. Finally, though UMFI can work for any arbitrary feature type, in this paper, we have only considered datasets with continuous explanatory variables.
288
289
290
291
292
293
294
295
296
In future work, we would like to test how well other methods, such as neural networks, pair with UMFI while further testing on wider variety of random variable types such as binary, categorical, and ordinal features. Further, we would like to explore how well dependence can be removed and UMFI can be estimated on real data as the number of features increases to sizes much larger than 50.
297
298
299
300 To reiterate, UMFI is a powerful tool for detecting and explaining the relationships hidden within data. We emphasise that UMFI is just framework. A variety of other methods can be used to estimate the universal predictive power v/ including, but not limited to, XGBoost, neural networks, or Gaussian processes. Even non- m----ooddddeeede methods such as Hilbert-Schmidt independence criterion could be explored in future applications. Furthermore, new preprocessing techniques for dependence removal
302
303
304
305
still being developed in the AI fairess community, these, in addition other existing methods, can be used in future applications of UMFI for additional improvements.
306
Broader Impact
307
308
We hope that UMFI will be a useful tool in a variety of disciplines including bioinfomatics, ecology. earth sciences, and health science for discovering scientific processes and relationships hidden within data. Though we think that our contributions can only lead to positive social and environmental impacts by aiding scientific discoveries in domains like earth science and bioinformatics, statistical methods, especially those that are aimed at genetics research, have historically been used to justify
309
310
311
harmful and misleading claims. If such claims arise using our methods, then they should be dismissed since direct causal effects cannot be concluded after using our methods alone.
313
9

---

References
315
318 [1] Nans Addor, Andrew Newman, Naoki Mizukami, and Martyn P Clark. The camels data set: catchment attributes and meteorology for large-sample studies. Hydrology and Earth System Sciences, 21(10):5293-53)) 5313, 2017.
317
318
319 [2] Nans Addor, Grey Nearing, Cristina Prieto, AJ Newman, Nataliya Le Vine, and Martyn Clark. A ranking of hydrological signatures based on their predictability in space. Water Resources Research, 54(1 1):8792-8812, 2018.
321
322
[3] Daniel W Apley and Jingyu Zhu, Visualizing the effects of predictor variables in black box supervised learning models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 82(4):1059-1086, 2020.
324
325
[4] Eviatar Bach, Valentina Radié, and Christian Schoof. How sensitive are mountain glaciers to climate change? insights from a block model. Journal of Glaciology, 64(244)2247-588, 2018.
328
327
[5] Adrián Bazaga, Dan Leggate, and Hendrik Weisser. Genome-wide investigation of gene-cancer associations for the prediction of novel therapeutic targets in oncology. Scientific reports, 10(1):
329
1-10, 2020.
330
[6] Jian Bi. A review of statistical methods for determination of relative importance of correlated predictors and identification of drivers of consumer liking. Journal of Sensory Studies, 27(2): 87-101, 2012.
331
332
333
[7] Sarah Bird, Miro Dudík, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. Fairleam: A toolkit for assessing and improving faimess in ai. Microsoft, Tech. Rep. MSR-7R-2020-32, 2020.
334
335
336
[8] Ieo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
337
[9] Alexander Brenning and GF Azócar. Statistical analysis of topographic and climatic controls and multispectral signatures of rock glaciers in the dry andes, chile (27-33 s). Permafrost and Periglacial Processes, 21 21(1):54-66, 2010.
338
339
340
[10] Flavio Calmon, Dennis Wei, Bhanukiran Vinzamuri, Karthikeyan Natesan Ramamurthy, and Kush R Varshney. Optimized pre-pon rrepprocessing for discrimination prevention. Advances in neural information processing systems, 30, 2017.
342
343
Amnon Catav, Boyang Fu, Yazeed Zoabi, Ahuva Libi Weiss Meilik, Noam Shomron, Jason Ernst, Sriram Sankararaman, and Ran Gilad Bachrach. Marginal contribution feature impor- tance an axiomatic approach for explaining data. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning. volume 139 of Pro- ceedings of Machine Learning Research, pages 1324-1335. PMLR, 18-24 Jul 2021. URL https://proceedingg mlr ppesssvv39/cart wwt.w/aa..aaaaaa...... html
344
345
347
349
[12] Hugh Chen, Joseph D Janizek, Scott Lundberg, and Su-In Lee. True to the model or true to the data? arXiv preprint 1iXiv:2006.1623: 2020.
350
[13] Shay Cohen, Gideon Dror, and Eytan Ruppin. Feature selection via coalitional game theory. Neural Computation, 1937) 2007.
352
353
[14] Ian Covert, Scott M Lundberg, and Su-In Lee. Understanding global feature contributions with additive importance measures. Advances in Neural Information Processing Systems, 33: 17212-17223, 2020.
354
355
356
[15] Ian Covert, Scout M Lundberg, and Su-In Lee. Explaining by removing: A unified framework for model explanation. J. Mach. Learn. Res., 22:209-1, 2021.
357
358
[16] Dries Debeer and Carolin Strobl. Conditional permutation importance revisited. BMC bioinfor- matics, 21(1):1-30, 2020.
10

---

380
[17] Tamsin L Edwards, Sophie Nowicki, Ben Marzeion, Regine Hock, Heiko Goelzer, Hélène Seroussi, Nicolas C Jourdain, Donald A Slater, Fiona E Turner, Christopher J Smith, al. Projected land ice contributions to twent--irst-century sea level rise. Nature, 593(7857):74-88, 2021.
361
362
363
364
[18] WA Gibson. Orthogonal predictors: A possible resolution of the hoffman-wart controversy.
365
Psychological reports, 1 111):332-3, 1962.
388
[19] David A Gill, Michael B Mascia, Gabby N Ahmadia, Louise Glew, Sarah E Lester, Megan Barnes, Ian Craigie, Emily Darling. Christopher M Free, Jonas Geldmann, et al. Capacity shortfalls hinder the performance of marine protected areas globally. Nature, 543(7647): 665-669, 2017.
367
368
369
370
[20] Sander Greenland, Judea Pearl, and James M Robins. Causal diagrams for epidemiologic research. Epidemiology, pages 37-48, 1999.
371
372
[21] Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Measuring statistical dependence with hilbert-schmidd norms. In International conference on algorithmic learning theory, pages 63-77. Springer, 2005.
373
374
375
[22] Virgil Griffith and Christof Koch. Quantifying synergistic mutual information. arXiv preprint
376
1 1205, 2012.
377
[23] Nimrod Harel, Ran Baaahaahacrach,,,, and Uri Obolski. Inherent inconsistencies of feature importance. arXiv preprint 0rXiv:2206.08204, 2022.
378
379
[24] Giles Hooker, Lueas Mentch, and Siyu Zhou. Unrestricted permutation forces extrapolation: variable importance requires at least one more model, or there is no free variable importance. Statistics and Computing, (1))::1-16, 2021.
380
381
382
[25] Aleks Jakulin and Ivan Bratko. Quantifying and visualizing attribute interactions: An approach based on entropy. 2003.
383
384
[26]
Alexander Janssen, Mark Hoogendoorn. Marjon H Cnossen, Ron AA Mathôt, OPTI- CLOT Study Group, SYMPHONY Consortium, MH Cnossen, SH Reitsma, FWG Leebeek, RAA Mathôt, K Fijnvandraat, et al. Application of shap values for inferring the optimal func- tional form of covariates in pharmacokinetic modeling. CPT: Pharmacometrics & Systems Pharmacology, 2022.
385
386
387
389
[27] Joseph Janssen and Ali A Ameli. A hydrologic functional approach for improving large- sample hydrology performance in poorly gauged regions. Water Resources Research, 57(9): e2021 22031WR0 2021.
391
382
[28] James E Johndrow and Kristian Lum. An algorithm for removing sensitive information: application to race-independent recidivism prediction. The Annals of Applied Statistics, 13(1):
393
1 189-220, 2019.
395
[29] Päl Johnsen, Signe Ziemer-Sprensser Andrew Thomas WaWan, Megan E Cahill, and Mette Langaas. A new method for exploring gene-gene and gene-environment interactions in gwas with tree ensemble methods and shap values. BMC bioinformatics, 22(1):1-29, 2021.
397
[30] Gunnar König, Timo Freiesleben, Bernd Bischl, Giuseppe Casalicchio, and Moritz Grosse- Wentrup. Decomposition of global feature importance into direct and associative components (dedact). arXiv preprint arXiv: 002.110.0000 08086, 2021.
399
401
[31] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physicul Review E, 69(6), jun 2004. doi: 10. 10. 103/physreve.66.0666
htlpst //doi
403
org/10. 1103//2ppyyseeee....mm 69. 066138
[32] Edward Le, Ali Ameli, Joseph Janssen, and John Hammond. Snow persistence explains stream high flow and low flow signatures with differing relationships by aridity and climatic seasonality. Hydrology and Earth System Sciences Discussions, pages 1-22, 2022.
405
11

---

407
[33] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances neural information processing systems, 30, 2017.
408
409
[34]
Alexander Marx, Arthur Gretton, and Joris M Mooij. A weaker faithfulness assumption based on triple interactions. In Uncertainty in Artificial Intelligence, pages 451 1600. PMLR, 2021.
410
[35]
R Core Team Microsoft. Microsoft R Open. Microsoft, Redmond, Washington, 2017. URL https://mran: mi microsoft. com/
412
413
[36]
Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.
[37]
Alena Orlenko and Jason H Moore. A comparison of methods for interpreting random forest models of genetic association in the presence of non-additive interactions. BioData mining, 14 (1):1-17, 2021.
415
416
417
[38] Lennart Schmidt, Falk Heße,, Sabine Attinger, and Rohini Kumar. Challenges in applying machine learning models for hydrological inference: A case study for flooding events across germany. Water Resources Research, 5665)e20199R005592,, 2020.
419
420
[39]
Heidi Sevestre and Douglas Benn. Climatic and geometric controls on the global distribution of surge-type glaciers: implications for unifying model of surging. Journal of Glaciology, 61 (228):646-663) 662, 2015.
422
423
[40] Lloyd Shapley, A value for n-person games, contributions to the theory of games, 2, 307-317, 1953.
424
425
[41] Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon. Learning controllable fair representations. The 22nd International Conference on Artificial Intelligence and Statistics, pages
426
427
PMLR, 2019.
428
[42]
Lina Stein, Martyn P Clark, Wouter JM Knoben, Francesca Pianosi, and Ross A Woods. How do climate and catchment attributes influence flood generating processes? large-sample study for 671 catchments across the contiguous usa. Water Resources Research, 57(4):22020WR2283000,
429
430
431
2021.
432
[43]
Katarzyna Tomezak, Patrycja Czerwińska, and Maciej Wiznerowiez. The cancer genome atlas (tega): an immeasurable source of knowledge. Contemporary oncology, 19(1A):A68 2015.
[44] Matthew J Vowels, Necati Cihan Camgoz, and Richard Bowden. D'ya like dags? survey on structure learning and causal discovery. ACM Computing Surveys (CSUR), 2021.
435
436
[45]
Hao Wang and Flavio P Calmon. An estimation-theoredl view of privacy. 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 886-893.
438
IEEE, 2017.
439
[46]
Hao Wang, Lisa Vo, Flavio P Calmon, Muriel Médard, Ken R Duffy, and Mayank Varia. Privacy with estimation guarantees. IEEE Transactions on Information Theory, 65(12))8025-80)) 8042, 2019.
440
441
442
[47] Hui Wang. David Bennett, Philip I. De Jager, Qing- Zhang, and Hong- Yu Zhang. Genome- wide epistasis analysis for alzheimer's s disease and implications for genetic risk prediction. Alzheimer's 's research & therapy, 13(1):1-13, 2021.
444
445
[48] Thomas C Williams, Cathrine C Bach, Niels Matthiesen, Tine Henriksen, and Luigi Gagliardi. Directed acyclic graphs: tool for causal studies in paediatrics. Pediatric research, 84(4):487-493, 2018.
447
448
[49] Brian Williamson and Jean Feng. Efficient nonparametric statistical inference on population feature importance using shapley values. In International Conference on Machine Learning, pages 10291. PMLR, 2020.
450
451
[50] Marvin N Wright and Andreas Ziegler. ranger: A fast implementation of random forests for high dimensional data in c++ and r. arXiv preprint http:://58.000000. 2015.
[51] Sewall Wright. Correlation and causation. 1921.
12

---

454
[52] Lee H Wurm and Sebastiano A Fisicaro. What residualizing predictors in regression analyses does (and what it does not do). Journal of memory and language. 72:37-48, 2014.
455
Checklist
456
457
1. For all authors...
458
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] After reading the abstract and introduction please read
459
460
Section
+ to see quantitative support for our claims.
481
(b) Did you describe the limitations of your work? [Yes] Please see Section S for a detailed description of limitations.
463
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
S [
484
465
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
486
467
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] Please see Section
489
470
(b) Did you include complete proofs of all theoretical results? [Yes] Please see Section 3 and the Supplementary material.
471
472
3. If you ran experiments..
479
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)? [Yes] Please see Section #and the Supplementary material.
474
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] Please see Section Iastionnaane and the Supplementary material.
477
478
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] In most cases we did, see Section the Supplementary material, but for the computational complexity experiment, we did not because of the higher computational cost and the fact that repeated experiments would not significantly change the results (this was tested but not shown).
479
480
481
482
489
(d) Did you include the total amount of compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] Please see Section 4 and the Supplementary material.
494
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
487
(a) If your work uses existing assets, did you cite the creators? [Yes] Please see Section
488
4
489
(b) Did you mention the license of the assets? (Yes] Please see Section 4.2 The BRCA
490
dataset is available on Github via the MIT License.
(c) Did you include any new assets either in the supplemental material as URL? [Yes] Please see the supplemental material for code.
492
493
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes] Please see Section
494
495
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [Yes] Please see Section 4.2
497
5. If you used crowdsourcing or conducted research with human subjects...
498
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
489
500
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
501
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
503
13


Reviews:
[{"review_id": "qQqno9Q_V28", "reviewer": "Reviewer_sCQt", "summary": "This work considers how to quantify the importance of different features in a ML model (for each feature $f$ from the set of all features $F$) to the response variable $Y$. To do so, they modify a method known as \"marginal contribution feature importance\" (MCFI). In simple terms, rather than measuring the maximum contribution that a feature makes to a subset of other features (where contribution roughly means how much it improves predictive performance), \"ultra marginal feature importance\" (UMFI) measures the maximum contribution to any random variable that is a function of the full feature set. \n\nThis sounds like it could make computation harder, but in fact the authors prove that it can be computed efficiently by removing $f$'s dependencies/signal from $F$ (in a sense that isn't precisely defined, in my view), and then measuring how the newly trained model's performance degrades. Thus, the computational cost is significantly lower than computing the maximum contribution over $2^{d-1}$ feature subsets (where there are a total of $d$ features).\n\nThe authors' proof rests on several assumptions that won't hold in practice, but the method should still give reasonable results. And there is no gold-standard way of removing dependencies, but the authors demonstrate the method with a couple viable options (e.g., residualizing out with linear regression).", "questions": "On line 120, is it important that $g$'s domain is $A \\subseteq F$ rather than simply $F$? I don't see why it can't simply be $F$. And is it important that the co-domain be $\\mathbb{R}^{|A|}$ rather than some higher- or lower-dimensional Euclidean space? \n\nA couple points from above that would be helpful to comment on:\n- Non-uniqueness of dependency-removing transformation\n- How MCI handles correlated features wrong, and how UMFI is supposed to fix the problem\n- Why MCI gives non-zero importance to unrelated features but UMFI somehow does not\n- Various points about the experiments", "limitations": "The authors included a nice discussion of limitations towards the end of the paper. One additional point that I would have liked to see covered, besides the main challenge of properly removing dependencies: whether there are any downsides to finding the maximum contribution. For example, are there cases where finding the max would yield equal importance for two features $x_1$ and $x_2$, where $x_1$ is useful on its own but $x_2$ is only useful when present along with a complementary feature $x_3$? This seems like a potential issue for both MCI and UMFI (depending on what one wants out of their importance values). ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n- This work presents a new perspective on how to define feature importance in ML models. Rather than removing/corrupting subsets of features, it considers the maximum contribution that each feature $f$ makes to any random variable that's a function of the full feature set. Leave-one-out or Shapley value approaches wouldn't make sense in the context of arbitrary functions of the features (the value function $\\nu$ is no longer a set function, but a function of any random variable), but it's a neat result that the random variable to which a feature $f$ contributes most is somewhat easy to approximate.\n- UMFI seems to provide reasonable results in the experiments.\n\n### Weaknesses\n\nAbout the method:\n- The idea of removing dependencies of one random variable from another does not seem to be precisely defined anywhere in the paper. It seems like there is not a unique way to do this - for example, I can remove $f_1$'s dependencies from $f_2$ by setting $f_2$ to a constant. If the only requirement in this work is that the modified $f_2$ variable becomes independent from $f_2$, and the non-uniqueness of the transformation isn't a problem, that should be discussed more prominently.\n- The name \"information subsets\" for $I(F)$ was a bit confusing, it doesn't seem like there are subsets involved here (the way they are in MCI or SAGE). It seems to mean the set of all possible functions applied to the features/input variables. I guess I get the spirit of the name, but it's confusing.\n- On lines 55-57, one of the stated issues with MCI is that it underestimates the importance of correlated features. I didn't quite understand how MCI gets correlated features wrong, or how UMFI is supposed to fix the problem - can you elaborate on this? This point doesn't seem very well supported in theory, and it's only supported in a limited sense by the experiments.\n- On lines 57-58, one of the stated issues with MCI is that it can give non-zero importance to features unrelated to the model. As UMFI yields strictly larger importance than MCI (this is mentioned later in the paper), UMFI should share this flaw rather than fixing it. The results in Figure 2 don't seem to reflect this, which is very strange.\n- I believe the authors are potentially confusing two different meanings of the term \"marginal.\" The authors labeled both MCI and UMFI as \"marginal methods\", but I don't think they are if you look at what the \"marginal vs. conditional divide\" is about. The divide is about whether a feature importance method handles held-out features with their marginal or conditional distribution; e.g., KernelSHAP often uses the marginal distribution, and SAGE advocates for using the conditional. Because MCI retrains models for each subset of features, it's actually a *conditional* method, at least approximately (if it's helpful, I can point to a paper that proves how retraining and sampling held-out features from their conditional distribution are approximately equivalent). UMFI on the other hand is *neither* - the models are trained on completely different features so there's no question of re-using a model and sampling replacement values for held-out features. For MCI, \"marginal\" refers to \"marginal contribution\" from the game theory context, which has nothing to do with the marginal vs. conditional label. UMFI is also roughly about marginal contributions (even though $\\nu$) isn't a cooperative game, so I could see calling it a \"marginal contribution\" method. But calling both of these \"marginal methods\" isn't very helpful given the ambiguity.\n\nSome potential issues with the experiments:\n- The experiments don't seem to use any real datasets. Even the BRCA experiment synthetically modifies certain features to ensure that they're unimportant to the response variable. Would it be possible to use a real dataset, or at least present additional results with the unmodified BRCA data?\n- Related to the request for a real dataset, another type of metric would be to measure the predictive performance of models trained with the most important features - and this would be simple to run even with real datasets. Would the authors consider adding something like this?\n- Would it be possible to include other baselines in the experiments? For example, the methods examined in MCI or SAGE? \n- MCI giving large importance to randomized features in Figure 2a is a bit hard to believe, it's very counterintuitive. Can you verify that there isn't a mistake here, or explain why this occurs? \n\nThe notation in this work was confusing at times. A couple choices that made this paper difficult to read were:\n- $f$ denotes a function in most ML papers, but here it denotes a feature. Many papers instead represent the features as $x = (x_1, \\ldots, x_d)$ when there are $d$ features, and the model inputs/outputs are usually written as $(x, y)$. $y$ is retained here, but not $x$ for some reason\n- It would be helpful to write explicitly that $F$ is simultaneously used as both a set (e.g., $S \\subseteq F$) and a random variable (e.g., $I(Y; F)$). Similar to the above, the notation that's often used is $S \\subseteq D = ${$1, \\ldots, d$} to denote the feature indices and $x_S$ to represent a random variable that concatenates the set of features\n- It would be helpful to say explicitly that $g(A)$ represents a random variable resulting from applying a function $g$ to a subset of features (towards the beginning of section 3)\n- The symbol used to denote independence (see theorem 3.1) is never defined and likely unfamiliar to some readers\n- $g$ is overloaded to denote two different types of functions: one is a predictive model for $Y$ (equation 2) and one is an arbitrary function on the feature space (definition of $I(F)$)\n\nNits: \n- On line 49, there are methods that suggest training models with many feature subsets (another one is SPVIM, Williamson & Feng 2020), but several do not - including SAGE and permutation tests. There are a variety of tricks for handling the held-out features including setting to the mean, sampling replacement values (from the dataset or from a generative model), training a model to handle missing features, etc (see \"Explaining by removing: a unified framework for model explanation\" by Covert et al.)\n- On lines 29-30, marginal (rather than conditional) methods are not typically thought of as being for interpreting the data. See \"True to the model or true to the data\" by Chen et al., it discusses how marginal methods are better suited for understanding the model rather than the data\n- I think there's a typo in the subscript of the union operator on line 134 - shouldn't it be $f$ rather than $F$?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "ffXFHK96AyV", "reviewer": "Reviewer_qdSP", "summary": "The problem of estimating feature importance is considered, specifically\nestimating marginal contribution importance (MCI). In essence, MCI estimates the\nvalue of a feature based on the performance drop when the feature is excluded.\nThis paper studies an extension which the authors dub ultra-marginal feature\nimportance (UMFI) that maximises over \"information subsets\" rather than the all\nsubsets of features (like the original MCI).\n\nIt turns out this slight change provides large computational benefits and\nimproved performance to boot. The authors prove a key result that allows the\nmaximising information subset to be approximated using dependency removal\ntechniques such as those from the fair learning literature. This reduction in\ncomplexity allows scaling to previously unobtainable number of features.\n", "questions": "It's clear many methods could be used as ν, and is discussed in the conclusion, but\nno alternative is explored in the empirical results. It's quite natural to wonder how\ncritical the choice of supervised learning methods is: have this been explored?\n\nThe choice of ν can also be more generic dependency measures, so what about\neasy to estimate measures of dependence like HSIC?\n", "limitations": "It's clear many methods could be used as ν, and is discussed in the conclusion, but\nno alternative is explored in the empirical results. It's quite natural to wonder how\ncritical the choice of supervised learning methods is: have this been explored?\n\nThe choice of ν can also be more generic dependency measures, so what about easy\nto estimate measures of dependence like HSIC? Exploration of supermodularity\nmight be more straightforward.\n", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This is a great extension to the MCI framework as it is simple, novel, and has a\nlarge practical impact. The presentation is excellent, theorem proof\nstraightforward, and empirical experiments well chosen. Limitiations of the\nstudy are well discussed.\n\nOne weakness is the authors have found a dataset where their choice of optimal\ntransport/linear regression for dependence removal failed, but it is relegated\nto supplementary materials.\n\nminor: line 301 is missing the word \"features\"\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "OoWGp8aDyUP", "reviewer": "Reviewer_rqHi", "summary": "This paper proposes an ultra-marginal feature importance (UMFI) method by extending marginal conditional importance (MCI) methods for evaluating feature importance. Some experiments have been conducted on simulated and real-world data sets.\n", "questions": "See above. ", "limitations": "Limitation statements are given. \n", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This paper studies an important problem. The authors have reviewed the related work well. \n\nThe extension of the work from MCI methods is by introducing an information subset in feature importance evaluation. The authors do not justify why an information subset is necessary. I have a few questions about the proposed method.  \n\nWhen evaluating the feature impotence of f, there needs to find the largest information subset S*, such that f and S* are independent. My question is if there is no such an S*, how the importance of f is evaluated? This is real when all features are correlated. \n\nI need to be convinced to use the largest information subset S* in evaluating the importance of f. I use a causal effect estimation to understand the solution since feature importance is related to the causal effect of f on Y. When S* is independent of f, it does not ``confound\" f in a viewpoint of causal effect estimation. Why should we remove the effect of S* on Y when estimating the effect of f on Y?   \n\nBy reading the algorithm, the importance of a feature is related to a model, see Lines 3-5. Do authors mean to explain feature importance in the model? This is not clear in the paper. \n\nThe predictions of a model depend on the parameters in practice. Based on Lines 4 and 5 in Algorithm 1, the importance of a feature can be dependent on the parameters. How do authors avoid the effect of parameter variation on evaluating the importance of a feature? \n\n---\n\nAfter discussions.\n\nThe authors answered my queries, and their revisions have addressed my major concerns, I raise my overall rate.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "BGO8ffog3jG", "reviewer": "Reviewer_3eYw", "summary": "The authors provide a simple yet effective extension of the recently proposed marginal feature importance criterion that allows a fast and accurate estimate of the marginal importance of each feature in the presence of feature correlation and non-linear interactions.", "questions": "In the proof of Theorem 3.1, you seem to assume that the maximizer of eq. 3 is unique. But what if \\tilde{S}^* is also a maximizer of 3? I guess for your scope it is enough that S^* is one of the maximizers, but this should be clarified for the sake of soundness.\n\n", "limitations": "The limitations of the approach are exhaustively discussed in the conclusion on the paper.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nA simple but clever extension of the marginal feature importance criterion allows to circumvent the need for enumerating all potential feature subsets, substantially increasing the practical applicability of the approach.\n\nWeaknesses:\n\nThe main ingredients were already all in the original paper introducing marginal feature importance. From this perspective the novelty of the work is not dramatic.\n\nThere is a lack of intuition in introducing the ultra-marginal feature importance criterion. It would be useful to discuss the role of g and its use in dependency removal when introducing it, possibly providing an illustrative example. Overall, the methodological section is very shallow and should be expanded to better highlight the rationale and relevance of the contribution.\n\nMinor: \n\nIn theorem 3.1, S should be orthogonal to f, not to F.\n\nI suggest to better contextualize the related work or move it to the end of the paper, as it's unclear why certain topics are being discussed (e.g. orthogonal predictors).\n\nAFTER REBUTTAL\nThe authors improved the clarity of manuscript and better highlighted the significance of the contribution. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject