Ultra-marginal Feature Importance
Anonymous Author(s)
AffiliationAddress email
Abstract
Scientists frequently prioritize learning from data rather than training the bestpossible model; however, research in machine learning often prioritizes the latter.
Marginal feature importance methods, such as marginal contribution feature importance (MCI), attempt to break this trend by providing a useful framework forquantifying the relationships in data in an interpretable fashion.
In this work, wegeneralize the framework of MCI while aiming to improve performance and runtime by introducing ultra-marginal feature importance (UMFI). To do so, we provethat UMFI can be computed directly by applying preprocessing methods from the AI fairness literature to remove dependencies in the feature set.
We show on realand simulated data that UMFI performs at least as well as MCI, with significantlybetter performance in the presence of correlated interactions and unrelated features,while substantially reducing the exponential runtime of MCI to super-linear.
Introduction
Scientists often seek to determine the true relationships between a set of characteristics and someoutcome of interest.
These relationships are ideally determined by performing carefully controlledexperiments so that causality can be established.
However, experiments can be difficult and costly topursue, unethical to perform, or impossible to control [44], leaving only observational data available.
The relationships that are hidden within vast quantities of observational data are often difficult todetermine, so statistical tools, such as feature importance, have been explored.
Feature importancequantifies and, more importantly, ranks how related explanatory features are to a response.
There is no consensus on how define feature importance, so choosing an appropriate method stronglydepends on the questions that the user seeks to answer [42, 19, 9].
The most widely referenceddistinction between feature importance methods is the conditional versus marginal divide, whichdefines the two extremes of feature importance methods [15].
This division is sometimes also framedas true to the model versus true to the data [11].
The distinction between these two types of methodsis only evident in the presence of correlated features. Indeed, if all explanatory features in a datasetare independent, conditional and marginal importances are exactly the same [15, 30]. Conditionalmethods are typically used for feature selection, dimensionality reduction, and improving predictionperformance. On the other hand, marginal methods are specifically developed to explain and interpretdata [19], seeking to avoid the tendency of conditional methods to explain the model and givecorrelated features too little importance [11].
For example, if a scientist wants to build a databaseof genes associated with some disease, then a marginal approach could be preferred [15, 9, 11].
Alternatively, if an engineer wants to determine the smallest number of features to obtain good modelpredictions, a conditional importance metric would be more suitable [19, 9].
Recently, feature importance methods such as Shapely-values [33, 12, 26], SAGE [14], accumulatedlocal effects (ALE) [3], permutation importance [7], and conditional permutation importance (CPI)
[15] have been used in high-impact journal papers by scientists who want to explain the mechanismswithin data [2, 4, 35, 24, 32, 17].
However, these methods do not adequately account for correlated features and feature interactions.
ALE can only easily show first order effects [29], and although CPI improves upon some limitations of permutation importance, CPI has the property that twoperfectly correlated features with significant predictive power would both be deemed unimportant[14].
Further, only one model is trained in ALE, CPI, and permutation importance.
Thus, correlatedfeatures, which can alter the model assembly process, could be given artificially low importanceif the goal is to explain the data [20].
Developers of feature importance methods like SAGE andmarginal contribution feature importance (MCI) have attempted to address these issues by evaluatingthe difference in accuracy between a model trained with the feature of interest and a model trainedwithout it, across all feature subsets [10, 14].
In particular, MCI was shown to have better quality androbustness when compared to Shapely-values, SAGE, ablation, and bivariate methods [10].
However,subset-based feature importance methods usually require an exponential number of model trainings,which makes these methods ineffective for interpreting large datasets (e.g., gene expression studies).
In this paper, we introduce ultra-marginal feature importance (UMFI), a new subset-based marginalfeature importance method that performs at least as well as previous methods while drasticallyreducing run-time.
UMFI generalizes the framework of MCI, developed by Catav et al.
[10], in orderto overcome some key shortcomings.
First, MCI is given by a maximization problem that requiressearching across all subsets of the feature space F . Second, although it can handle complex featureinteractions and data with correlated features, MCI underestimates the importance of correlatedfeatures that interact in the expression for the response variable. Third, MCI can give non-zeroimportance to features that are completely unrelated to the response variable.
The rest of this paper is organized as follows.
A brief overview of MCI is given in Section 2.
Thetheoretical framework for UMFI is then formally presented in Section 3 along with an algorithm tocompute the UMFI of a feature f .
In Section 4, we conduct experiments on simulated and real data toassess the quality, robustness, and time complexity of UMFI compared to MCI.
Finally, an overviewof the work, its limitations, and ideas for future work are discussed in Section 5.
Related work
This paper is greatly inspired by the work of Catav et al.
[10], whose marginal contribution featureimportance was developed to handle complex feature interactions and correlated features, so webuild upon their strong framework.
To do so, we generalize the central maximization problem for
MCI to also consider transformations of the feature set F . We show that the new maximizationproblem, which maximizes over all "information subsets" of F , is attained by the largest informationsubset that is independent of the feature of interest f .
Therefore, the maximizer can be computedimmediately after preprocessing the data to remove dependencies on f .
Ideally, this would be donein a manner that preserves as much information from the original features as possible. Finding orthogonal predictors for resolving the controversies of feature importance in the presence ofmulticollinearity is not strictly new [16].
However, the discussion of orthogonal predictors for relativefeature importance methods has been seemingly limited to applications to multiple linear regression,mostly in the domain of psychology [5, 45].
While the techniques for orthogonalizing predictors hasbeen limited to fairly simple linear algebra, more advanced and more general dependency removalmethods have seen great progress within the AI fairness and privacy literature.
Some examples ofthese techniques include linear regression [6], optimal transport [23], neural networks [8, 34], convexoptimization [8], and principal inertial components [39].
Linear regression and optimal transportwere implemented for UMFI in this paper.
Besides MCI and SAGE [10, 14], other feature importance methods have recognized the difficulty inproviding accurate scores when complex interactions are present and when predictors are dependent.
Likely owing to the simplicity and popularity of permutation-based methods, the most commonapproach has been to retool permutation importance [7] and conditional permutation importance[36] to better handle these issues.
For example, the conditional subgroup permutation has beenproposed, such that the subgroups are constructed by decision trees, in which the distribution of thefeature becomes less dependent on other features [30].
However, this method only performs wellwhen the response is dependent on just a few features [30].
Despite their popularity and myriad ofvariations, a thorough overview by Hooker et al.
[20] has shown that permutation-based methods arenot suitable when trained on a feature set exhibiting statistical dependencies, as they can producehighly misleading results.
Hooker et al.
[20] instead advocate for feature importance methods thatachieve their score by 1) permuting or generating new values of the feature using its distribution conditional on the remaining features, or 2) removing the feature and examining the drop in accuracywhen a new model is trained. We note that MCI and UMFI both fall under the second category.  Marginal contribution feature importance
Let F be the set of features used to predict the response variable, Y . Given an evaluation functionν : P ( F ) → R + , Catav et al. [10] defined the marginal contribution feature importance (MCI) of afeature f ∈ F as [Equation]
Let ˆ y be a constant predictor and G ( S ) be the set of all predictive models restricted to using featuresin S ⊆ F . Given a loss function l , the "idealized" evaluation function satisfies [Equation] With this definition, ν ( ∅ ) = 0 since both terms will cancel out, and ν is monotonically increasingsince if S 1 ⊆ S 2 then G ( S 1 ) ⊆
G ( S 2 ) . Though these properties are not strictly guaranteed whenfaced with real data and an imperfect model, they are reasonable and follow the intuition that morefeatures give a model more information about the response variable. In practice, one can for exampleuse machine learning models such as elastic-net or random forests for G and negative log-loss crossvalidation or out-of-bag accuracy for the loss function l [10].
Given a monotonically increasing evaluation function ν , satisfying ν ( ∅ ) = 0 , Catav et al. [10] provedthat I ν ( f ) is the only function that satisfies three desirable axioms that they believe a method shouldhave for explaining data properly.
First, the marginal contribution axiom states that the importance ofa feature should be at least as large as the increase in the evaluation function after adding the featureto the model containing all other features, i.e. I ν ( f ) ≥ ν ( F ) − ν ( F \ { f } ) .
Second, the eliminationaxiom states that the importance of a feature can only decrease if features from the feature set areremoved. Third, the minimalism axiom states that we should take the minimum feature importancevalue that satisfies the first two axioms, thus ensuring its uniqueness.
Ultra-marginal feature importance
We hypothesize that the maximization function formulated by Catav et al.
[10] is too restrictive. Instead, we introduce ultra-marginal feature importance (UMFI) by proposing a maximization overinformation subsets of the feature set F
[Equation]
We define the space of information subsets of F by I ( F ) =
{ g ( A ) : A ⊆ F and g : R | A | → R | A | } .
We call these information subsets because I ( Y ; g ( F )) ≤ I ( Y ; F ) is a property of mutual informationfor any function g [13, p. 33].
By considering transformations of feature subsets, we may usedependency removal methods from the AI fairness literature, which were developed to achieveunbiased predictions [34, 23].
Given this expanded framework, U ν ( f ) obeys the marginal contributionand elimination axioms laid out by Catav et al.
[10] (Supplement C.1).
In addition to ν ( ∅ ) {+= 
+} and ν being monotonically increasing, we also assume that ν is supermodularunder mutual independence and that ν is invariant under duplicate information, i.e. ν ( A )
=
ν ( A ∪ B )if B ⊂ A . Both of these conditions are satisfied when ν ( S )
=
I ( Y ; S ) ([18], Supplement A.1).
Theevaluation function is equivalent to mutual information in the idealized setting where we know the Bayes classifier and ν is defined via (2) with the cross entropy loss function [14]. In practice, theaccuracy of the approximation ν ( S )
≈
I ( Y ; S ) depends on the quality of the machine learning modeland the specified loss function. Assuming these properties, we can solve directly for the maximizinginformation subset S ∗ for Equation 3.
Theorem 3.1.
The maximizing information subset S ∗ for (3) is given by S ∗ = (cid:83) S ∈I ( F ): S ⊥⊥ F S , whichis the largest information subset of F such that S ∗ ⊥⊥ f .
Proof.
First, suppose for contradiction that S ∗ solves (3) and that S ∗ ̸⊥⊥ f . Let ˜ S ∗ represent theremaining information of S ∗ after having removed dependencies of f from S ∗ .
Then, ν ( S ∗ ∪ f ) =
ν ( ˜ S ∗ ∪ f ) from invariance under duplicate information, but ν ( S ∗ ) ≥ ν ( ˜ S ∗ ) from monotonicity. Hence, ν ( ˜ S ∗ ∪ f ) − ν ( ˜ S ∗ ) ≥ ν ( S ∗ ∪ f ) − ν ( S ∗ ) , which contradicts the fact that S ∗ is the maximizinginformation subset, so we may assume that S ∗ ⊥⊥ f . Now, suppose that S ∗ , X, f are mutually independent information subsets of F .
Then, by thesupermodularity of ν under mutual independence, X must be included in S ∗ . By invariance underduplicate information, the maximizer for (3) is satisfied by S ∗ =
(cid:83)
S ∈I ( F ): S ⊥⊥ f
S . Note that the union of all information subsets that are independent of f is equivalent to the informationsubset obtained by removing dependencies on f from the feature set F because of invariance underduplicate information. This immediately implies a fast algorithm for computing the ultra-marginalfeature importance of a feature f ∈ F (Algorithm 1).
Experiments
We perform experiments to compare UMFI and MCI with respect to quality, robustness, and timecomplexity.
To implement UMFI, we consider optimal transport [23] (UMFI_OT) and linear regression [6] (UMFI_LR) as methods to remove dependencies from the data.
A detailed overview of theseimplementations is shown in Supplement D and experiments comparing these methods appear in
Supplement E. For all experiments, we use random forests’ out-of-bag accuracy ( R 2 OOB-accuracyfor regression tasks and OOB classification accuracy for classification tasks) as the evaluation metricν [7].
We use the ranger R package to implement random forests with default hyperparameters and100 for the number of trees [43].
All experiments were run in Microsoft R Open Version 4.0.2 [28].
Supplement G contains additional experiments comparing feature importance metrics.
4.1 Experiments on simulated data
We run UMFI on simulated data with correlated interactions, correlated features, and non-linearinteractions to verify that it performs well compared to MCI.
The data in all simulation studies containsone response variable Y , four explanatory features x 1 , x 2 , x 3 , x 4 , and 500 randomly generatedobservations.
Each of the simulation studies are repeated 100 times so that we can also test thestability of each method.
4.1.1 Correlated interactions
Interacting features are often correlated [21, 22], so it is vital to test if feature importance metricswork as desired in these settings.
Let A, B, C, D, E, G ∼ N (0 , 1) .
We consider
[Equation]
In this simulation, x 1 is correlated with x 2 , and x 3 is correlated with x 4 in the same way. Wewould expect x 1 and x 2 to be more important than x 3 and x 4 because of the extra interaction term,sign ( x 1 ∗ x 2 ) .
The results in Figure 1a clearly show that UMFI provides better estimations offeature importance compared to MCI when correlated interactions are present.
The variability of allestimates are approximately the same across methods.
However, MCI estimates that all features have approximately the same feature importance scores, while both UMFI methods show significantlygreater importance for x 1 and x 2 compared to x 3 and x 4 .
MCI fails in this experiment becausex 2 contains most of the important information coming from x 1 , which implies that I ( Y ; x 1 , x 2 ) −
I ( Y ; x 1 ) ≤ I ( Y ; x 1 ) − I ( Y ; ∅ ) .
Therefore, when computing the importance for x 1 , the maximizingsubset that solves Equation 1 cannot contain x 2 if ν ( S ) ∼ I ( Y ; S ) .
UMFI is able to detect thisinteraction because it can extract the information from x 2 that interacts with x 1 while keeping thisextracted feature independent of x 1 .
4.1.2 Correlations
Marginal feature importance methods such as MCI and UMFI should not change the measuredimportance of features in the presence of correlated variables.
To test this, we implement a simulationstudy similar to the ones found in Catav et al.
Let ϵ ∼ N (0 , 0 . 2) .
We consider
[Equation]
The addition of x 3 should not alter the importance of x 1 , and x 1 should remain equally as importantas x 2 , since they have the same influence on the response Y .
The results shown in Figure 1b showthat both MCI and UMFI work reasonably well.
As with the previous simulation experiment, thevariability is consistent across methods.
As was desired, MCI and UMFI with linear regression showequal relative importance scores for x 1 and x 2 .
The importance given to x 2 was slightly greater thanx 1 in the MCI and UMFI optimal transport scenario, but the difference is not significant.
Interestingly,
MCI assigns some importance to x 4 , which was independent of the response, while both UMFImethods assign importance scores close to 0 .
Because of this, we conclude that UMFI with linearregression performs the best in this simulated scenario.
4.1.3 Non-linear interactions
Interaction effects are common in many scientific disciplines where assessing feature importance isprevalent, including hydrology [22, 2, 25], genomics [9, 41, 31], and finance [46, 37].
So, as was donein Catav et al.
[10], we assess the ability of MCI and UMFI to detect non-linear interaction effects inthe data.
This simulation study provides an XOR-type causal relationship where individually, theexplanatory features are unrelated with the response, but the response is influenced by the interactionof two features [27].
Let ϵ ∼ Exp (1 / √ 2) .
We consider
[Equation]
Ideally, the results of a feature importance metric would conclude that x 1 and x 2 have equally highimportance while x 3 and x 4 have no importance. Figure 1c shows consistently good performanceacross all methods.
Each method gave high relative importance scores to x 1 and x 2 , while x 3 and x 4received scores near 0 . This is perfect as the response was only a function of x 1 and x 2 , while it wasindependent of x 3 and x 4 . The only difference across methods is that the variability of estimates wasslightly greater when using UMFI with optimal transport compared with the other methods.
4.2 BRCA experiments
We use the same breast cancer (BRCA) classification dataset [38] used in previous feature importancestudies including Catav et al.
[10] and Covert et al.
[14] to test the quality and robustness of UMFIon real data.
The original data contains over 17 , 000 genes and 571 anonymous patients that havebeen diagnosed with one of 4 breast cancer sub-types.
We consider the same subset of 50 genesas in Catav et al.
[10] and Covert et al.
[14] for easier computation and result visualization.
Ofthe 50 selected genes, 10 are known to be associated with breast cancer, while the other 40 genesare randomly sampled.
This data was downloaded from https://github.com/TAU-MLwell/
Marginal-Contribution-Feature-Importance/tree/main/BRCA_dataset (MIT License).
In Catav et al.
[10] and Covert et al.
[14], these 40 randomly sampled genes are assumed to beunassociated with breast cancer.
However, to ensure a more definitive ground truth, we also randomlypermute the values of these 40 genes across their respective 571 observations to further reduce thechance that these genes have any association with breast cancer. Quality is then measured with the [Figure]
Each box plot summarizes the results of the 100 replications of each simulation. The first and third quantiles (thin lines), the median (thick line), and outliers (points) of the 100 replications are shown. true positive and true negative rates: the 10 BRCA associated genes should have some non-zeroimportance (positive), and the other 40 genes should have exactly zero importance (negative).
Theseexperiments were run 200 times on different seeds and with a different random sample of 500 patientsfor each iteration.
Robustness is measured using the standardized interquartile range (SIQR) of eachfeature’s importance from repeated experiments, which is calculated by dividing the average IQRacross the 50 features by the average median.
We found that MCI and UMFI (UMFI_LR and UMFI_OT) correctly gave significant importance tothe 10 genes that are known to be associated with breast cancer (Figure 2).
However, MCI consistentlygives non-zero importance to all features, while UMFI gives zero importance to the majority of therandomized genes as desired. Of the 40 randomized genes, the few that UMFI gives a non-zero scoreto have a score close to zero, and importantly, their scores are much smaller compared to the scores ofany of the 10 BRCA-associated genes.
During these experiments, we noticed that a greater proportionof permuted genes are given zero importance as the number of iterations grows. To confirm theconvergence of these importance scores we ran the UMFI experiments 5000 times and noted that both
UMFI methods have a perfect overall accuracy when distinguishing between important and permutedfeatures (Supplement G).
Interestingly, the ordering of important features was similar across methods,with BCL11A and SLC22A5 always being the most important and TEX-14 always being the leastimportant of the 10 BRCA-associated genes.
Although UMFI scores have higher variability than MCI (Table 1), it is clear from the feature importance graph that UMFI separates the 10 associatedgenes from the 40 unassociated genes better than MCI does even after considering UMFI’s higher
SIQR.
Even though UMFI should theoretically dominate MCI ( U ν ( f ) ≥ I ν ( f ) ), we observed that virtuallyall of the genes’ MCI scores were larger than their UMFI scores.
This could be due to a multitudeof factors.
First, we hypothesize that MCI can give incorrectly high scores when implemented withcertain machine learning models, such as random forest and xgboost.
In particular, random forestscan be inaccurate when trained on only a few features.
When trained on one or two features, thecorrelation between trees may increase dramatically, therefore giving a poor representation of anidealized evaluation function, ν [7]. Because the maximizing subset in (1) may be very small or evenempty, MCI is more susceptible to this fault compared to UMFI, which uses at least | F | − 1 modifiedfeatures for predictions when running Algorithm 1. Second, the evaluation function ν ( S ) may notactually behave like I ( Y ; S ) and hence may not obey supermodularity under mutual independenceand invariance under duplicate information. This could lead to UMFI giving inaccurate results.
Third, our implementations of linear regression and optimal transport may have been inadequatefor removing dependencies with minimal information loss.
Experimental results in Supplement
E indicate that optimal transport may have removed more information than necessary, and linearregression failed to completely remove dependencies.
[Table]
4.3 Computational complexity
MCI must train and evaluate a model for each element of the power set of the feature set, whichimplies O (2 p ) model trainings if there are p features.
If the evaluation function ν obeys soft k -sizesubmodularity, then the maximizing subset has no more than k elements, which reduces the numberof model trainings to O ( p k +1 ) [10].
UMFI circumvents the exponential training time since we mayimmediately estimate the maximizing set of features after removing the dependencies of f from thefeature set F .
To confirm the above statements, and to show that the extra model trainings requiredfor MCI dominate the computation time for removing dependencies in UMFI, we ran a simpleexperiment.
For a range of dataset sizes from the BRCA data, we evaluate the computation time forcalculating the feature importance scores of all features using MCI and UMFI.
We ran this experimentfor a dataset with 5 features, and then slowly added features until our given time budget of 1 hour ranout.
Once all [-
-]50 BRCA features were used, more features were randomly generated.
All datasetshad 571 observations.
These experiments were run using an Intel Core i9-9980HK CPU 2.40GHzwith 32GB of RAM.
Code was parallelized in R, and 12 of the 16 available threads were used.
Ourexperimental results are graphed in Figure 3 and indicate that UMFI is approximately superlinear,with UMFI_OT incuring more computational cost compared to UMFI_LR.
Giving each method 1hour to run, we found that MCI could process 19 features, MCI with the soft 3-size-submodularityassumption could process 130 features, UMFI_OT could process about 3000 features, and UMFI_LRcould process about 4000 features (Table 1).
Conclusion
In this study, we built upon the framework laid out by marginal contribution feature importance (MCI)and introduced a new feature importance framework called ultra-marginal feature importance (UMFI). We showed that UMFI can be estimated directly for a feature f by finding the largest informationsubset of the feature set F that is independent of f .
This signified the importance of removingdependencies for our method, which we explored using optimal transport and linear regression.
Whencomparing UMFI and MCI, not only does UMFI satisfy the key axioms used to define MCI, but italso has the ability to obtain faster and more accurate estimates of marginal feature importance onreal and simulated data, particularly in the presence of correlated interactions and unrelated features.
Throughout the work on this paper, several shortcomings appeared.
First, we only considered twosimple methods for removing dependencies, linear regression and pairwise optimal transport.
Othermethods certainly exist in the literature, including optimal transport with chaining [23], neuralnetworks [8, 34], or principal inertial components [40].
Though our two methods performed fairlywell on the real and simulated datasets in Section 4, optimal transport and linear regression failed tofind representations of the data that were independent of the protected attribute when we tested themethods on a hydrology dataset with more shared information compared to BRCA [1] (Supplement
E).
However, neural nets or principal inertial components certainly could have given better results.
Also, despite requiring significantly more computational cost, better methods for estimating theconditional CDF or using optimal transport with chaining should give better estimates for S ∗ whenimplementing UMFI_OT.
Second, UMFI scores are less robust than MCI since they have muchhigher variability, however, because of the significantly lower computational cost, UMFI can be runmultiple times and averaged to increase robustness.
Third, it is not clear precisely how close ν followsthe invariance under duplicate information, monotonicity, and supermodularity under independenceassumptions in practice.
Finally, though UMFI can work for any arbitrary feature type, in this paper,we have only considered datasets with continuous explanatory variables.
In future work, we would like to test how well other methods, such as neural networks, pair with
UMFI while further testing on a wider variety of random variable types such as binary, categorical,and ordinal features.
Further, we would like to explore how well dependence can be removed and
UMFI can be estimated on real data as the number of increases to sizes much larger than 50 .
Itwould also be beneficial to determine conditions that guarantee supermodularity of ν under mutualindependence, as well as invariance under duplicate information, since these properties cruciallyenable the direct computation of UMFI via Algorithm 1.
This can likely be done by investigating therelationship between the mutual information of a feature set and the prediction accuracy of a varietyof models, so that we can quantify the approximation ν ( S )
∼ I ( Y ; S ) . To reiterate, UMFI is a powerful tool to explain relationships in data using feature importance.
Weemphasise that UMFI is just a framework.
A variety of other methods can be used to estimate theevaluation function including, but not limited to, XGBoost, neural networks, or Gaussian processes.
Furthermore, new preprocessing techniques for dependence removal are still being developed inthe AI fairness community, so these, in addition to other existing methods, can be used in futureapplications of UMFI for additional improvements.
Broader Impact
We hope that UMFI will be a useful tool in a variety of disciplines including bioinfomatics, ecology,earth sciences, and health science for discovering scientific processes and relationships hiddenwithin data.
Further, we hope our work provides further justification and encouragement for thedevelopment of better AI fairness preprocessing methods and better machine learning models thatmore closely resemble the idealized ν .
Though we think that our contributions can only lead topositive social and environmental impacts by aiding scientific discoveries in domains like earthscience and bioinformatics, statistical methods, especially those that are aimed at genetics research,have historically been used to justify harmful and misleading claims.
If such claims arise using ourmethods, then they should be dismissed since our work can only provide associative measures. Strongcausal statements cannot be concluded after using our methods alone.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] After reading the abstract and introduction please readSection 4 to see quantitative support for our claims. (b) Did you describe the limitations of your work?
[Yes] Please see Section 5 for a detaileddescription of limitations.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See
Section
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] Please seeSection
(b) Did you include complete proofs of all theoretical results?
[Yes] Please see Section 3and the Supplementary material.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)?
[Yes] Please seeSection 4 and the Supplementary material. (b) Did you specify all the training details (e
g., data splits, hyperparameters, how theywere chosen)?
[Yes] Please see Section 4 and the Supplementary material.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
[Yes] In most cases we did, see Section 4 and the Supplementary material, but for the computational complexity experiment, we did not because of the
higher computational cost and the fact that repeated experiments would not significantly
change the results (this was tested but not shown).
(d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)?
[Yes] Please see Section 4 and theSupplementary material.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] Please see Section4.2.
(b) Did you mention the license of the assets?
[Yes] Please see Section 4.2.
The BRCA dataset is available on Github via the MIT License.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
Please see the supplemental material for code.(d)
Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[Yes] Please see Section 4.2.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[Yes] Please see Section 4.2.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]