Top-$k$ Feature Importance Ranking
Authors: Eric Chen
Tiffany Tang
Genevera I. Allen
Venue: Accepted by TMLR ()

Abstract: Accurate ranking of important features is a fundamental challenge in interpretable machine learning with critical applications in scientific discovery and decision-making. Unlike feature selection and feature importance, the specific problem of ranking important features has received considerably less attention. We introduce RAMPART (Ranked Attributions with MiniPatches And Recursive Trimming), a framework that utilizes any existing feature importance measure in a novel algorithm specifically tailored for ranking the top-$k$ features. Our approach combines an adaptive sequential halving strategy that progressively focuses computational resources on promising features with an efficient ensembling technique using both observation and feature subsampling. Unlike existing methods that convert importance scores to ranks as post-processing, our framework explicitly optimizes for ranking accuracy. We provide theoretical guarantees showing that RAMPART achieves the correct top-$k$ ranking with high probability under mild conditions, and demonstrate through extensive simulation studies that RAMPART consistently outperforms popular feature importance methods, concluding with two high-dimensional genomics case studies.

Published in Transactions on Machine Learning Research (12/2025)
Top-k: Feature Importance Ranking
Yuxi Chen Cornegie Mellon University
ericc30andrew. edu
Tiffany Tang University of Notre Dame
Ulang4 and edu
Genevera Allen Columbia University
annewer.allen@eooaue edu
Reviewed on OpenReview: https: //
// openreview. net/ forum? id= 20SHpccsa?
Abstract
Accurate ranking of important features is fundamental challenge in interpretable machine learning with critical applications in scientific discovery and decision-making. Unlike feature selection and feature importance, the specific problem of ranking important features has received considerably less attention. We introduce RAMPART (Ranked Attributions with MiniPatches And Recursive Trimming), a framework that utilizes any existing feature importance measure in a novel algorithm specifically tailored for ranking the top-k features. Our approach combines an adaptive sequential halving strategy that progressively focuses computational resources on promising features with an efficient ensembling technique using both observation and feature subsampling. Unlike existing methods that convert importance scores to ranks as post-processing, our framework explicitly optimizes for ranking accuracy. We provide thcoretical guarantees showing that RAMPART achieves the correct top-k ranking with high probability under mild conditions, and demonstrate through extensive simulation studies that RAMPART consistently outperforms popular feature importance methods, concluding with two high-dimensional genomics case studics. Our code available at httpp://ggtabb..oo/ttttiiigggg/
1 Introduction
A key challenge in interpretable machine learning is determining not just which features influence model predictions but their relative importance ranking. In particular, accurately ranking the top- most important features would fundamentally change the decision-making and scientific discovery process in numerous high-stakes applications [Bhatt al. 2020 Jaxa-Rozen and Trutnevyte 2021 For example, in many social science surveys, there are far more questions that. could, in principle, be asked than respondents can answer without incurring fatigue or disengagement. Researchers therefore often conduct small pilot studies with a larger question bank to identify the most important questions to retain in the final survey instrument (Jeong et al. 2023 DeVellis 2017]. In genomics, genome-widk wide association studies (GWAS) (Visscher et al. 2017) are by far the most common approach to identify important genes or genetic variants that are associated with disease risk. These data-driven studies often identify hundreds of important genetic variants. However, to translate these findings into tangible therapeutic targets and clinical practice, wet-lab validation is necessary, but typically limited to few dozen genetic variants, if not fewer due to its high cost (Fu et al.] 2020 Wang et al. 2025 Pashaei et al. 2025). More generally, given resource constraints, the need to rank or prioritize a small number top-ranked candidates for costly downstream decision-making is a common theme among many clinical and scientific pipelines.
Although feature importances have been extensively studied in machine learning, methods specifically designed for ranking the top-k most important. features remain underdeveloped.. Current approaches for top-k feature

---

Published in Transactions on Machine Learning Research (12/2025)
ranking typically rely on the heuristic of first estimating feature importance values for all features, sorting them, and then subsetting to the top-k features with the largest importance (Lundberg and Lee 2017 Neuhol and Benjamiiii 2024 Goldwasser and Hooker 2025). However, the first step of this paradigm is particularly limiting as valuable data and computational resources are being used to estimate the importances of all features, including those that, are irrelevant or far outside of the top-k that are of primary interest. This issue further exacerbated in realistic settings with correlated and high-dimensional data (e.g., genomics), where existing feature importance estimates are known to be highly unstable and unreliable NNcoodemus and Malley 2009 Nicodemmus 2011 Hooker et al. 2021). These challenges highlight the need for paradigm that directly targets the top-k ranking task, which focuses computational and statistical resources on the most. important features and can effectively handle correlated, higl-dimensional data.
1.1 Our Contributions
Motivated by these challenges, we focus on the problem of directly ranking the top-k features with the highest global importance and develop a model-agnostic framework tailored this task. To effectively handle correlated features, we first introduce RAMP (Ranked Attributions with MiniPatches),, an ensembling strategy that aggregates feature importances from models trained random subsamples (or "minipatches") of both observations and features. This minipatch-based approach breaks harmful correlation patterns among features while maintaining statistical power (Gan et al., 2022]. Building on RAMP, we then develop RAMPART (RAMP And Recursive Trimming), which leverages an adaptive strategy to progressively focus computational resources on promising features while eliminating suboptimal ones. Unlike existing approaches that allocate cqual resources to all features, this adaptive strategy enables RAMPART to grow incrcasingly precise in distinguishing between similarly-ranked top features as the candidate pool shrinks. Importantly, both RAMP and RAMPART are agnostic frameworks for top-k feature importance ranking that can serve as wrappers around any existing feature attribution method. Finally, we provide thcoretical guarantees on recovering the correct top-k feature importance ranking under mild assumptions, establishing explicit sample complexity bounds that may be of independent interest.
1.2 Related Works
Feature Importance Although not directly designed for feature importance ranking, numerous model- specific and model-agnostic feature importance measures have been developed to quantify the contribution of each predictor feature the model's predictions and performance (Molnar 2025). Popular model-specific approaches include regression coefficients for linear models, Mean Decrease in Impurity for tree-based methods (Briiman 2001), and neural network attributions like DeepLift and Integrated Gradients (Shrikumar et al. 2019 Sundararajan et al. 00 Model-agnostic methods include occlusion-based (Lei and Wasserman] 2014] permutation- based (Breiman 2001,, and Shapley-base based techniques (Lundberg and Lee [017 Lundberg et al. 2020 In Section 4 we will demonstrate the shortcomings of simply ranking these fcature importances obtain the top-k.
Ranking from Pairwise Comparisons On the other hand, viewing this problem from the lens of the ranking literature, many previous works have directly estimated rankings from pairwise comparisons. This literature includes tournament methods (Mohajer et al. 2017), spectral techniques (Negahban et al.) 2017
Chen and Suh 2013 Chen et al.] 2019), adaptive selection paradigms (Heckel et al. 2016 2018 and weighting strategies (Shah and Wainwright, 2018 Wauthier et al. 2013 Ammar and Shah 2012). Despite their theoretical appeal, these methods struggle to capture multivariate feature dependencies and face computational barricrs in high dimensions, limiting their applicability to top-k feature importance ranking.
Feature Importance Ranking More recently, several works have focused on feature importance ranking specifically. Kariyappa et al. (2023) developed sampling algorithms to identify lop-k features by Shapley values without addressing their ordering. Teneggi and Sulam (2024) employed statistical independence testing with betting principles, primarily for semantic concept validation in vision rather than tabular data. NNuhhof and Benjamini (2024) introduced a framework for quantifying uncertainty of feature importance rankings through sinultaneous confidence intervals. Their approach focuses primarily on post-hoc interpretation of
2

---

Published in Transactions on Machine Learning Research (12/2025)
pre-computed importance scores rather than providing an efficient algorithmic framework for large-scale feature ranking. Model-free, dependence-baseet ordering offers an orthogonal approach to ranking variables via a nonparametric coefficient of conditional dependence and yields tuming-free ranking without fitting predictive models (Azadkia and Chatterjoe 2021 closely related approach by Azadkia and Roudaki 2002
leverages an integraled N2 dependence measure and a greedy forward-selection procedure.
Most relevant to our work, Goldwasser and Hooker (0225) developed a sequential pairwise hypothesis testing framework for assessing the statistical significance of the top-k most important features using resampled attribution scores. This approach, however, requires normality and independence assumptions that are rarely satisfied in practice and violated by correlated estimators. Computational overhead from repeated pairwise lesting also limits scalability to high dimensions. Their subsequent rank verification method (Goldwasser et al. 2025) similarly assumes Gaussian distributions, constraining applicability to real-world data with Gan-Gausiian distributions and complex dependencies.
Best Arm Identification To avoid the current limitations of existing feature importance ranking ap- proaches, we introduce a recursive trimming strategy, which draws inspiration from multi-armed bandits research on best. arm identification, including UCB approaches (Audibert et al. 2010 Chen al. 2017).
Thompson Sampling (Russo| ), and halving algorithms (Zhao et al 2023). Particularly relevant is research on best-k-arm identification (Chen et al. 2008 Gao and Chen 2015 You et al. 2023) with Liu and Ročková
(2023) successfully applying Thompson Sampling to variable selection. However, direct application of bandit algorithms to feature ranking faces key challenges: (1) assumed arm independence -violated by correlated features and (2) unknown distributions of importance measures. Our work addresses these limitations with a novel approach that efficiently produces statistically robust feature rankings while accounting for feature interdependencies.
2
Adaptive Feature Importance Ranking
2.1 Problem Setup
Suppose we observe a dataset D= [(11, 01) ..., (XN, yN)} where x; E R-M and y; E R denote the features and response respectively. We assume observations are independent and identically distributed draws from an unknown joint distribution. We also assume that each feature j e [M] := (I....,M. possesses an inherent global feature importance Qj- There are many existing metrics that can be used to quantify Øy: see Lundberg and Lee (2017) Molnar (2025); Fisher et al. (2019) for instance. We note that the feature importance Φ) depends on the specific predictive model and importance metric used, with each method potentially defining a distinct ground truth. We define the rank of the j-th feature as ry = EM1 1 {lØy] < [oil} and the j-th best feature as Tj- Additionally, for some pre-speeified lo & M, we assume that the top-k features do not contain any ties: Tr, * Try for any j.j' E [k] where j # je. Our goal is to correctly estimate these top ranks such that Pr = TT, for j E [k].
2.2 Motivation
As mentioned previously, one naive approach to feature importance ranking is to first estimate importance scores M, and then sort the features accordingly. For instance, in a high-dimensional regression model, one may often rank the features according to the magnitude of their standardized coefficients. However, realistic, high-dimensional settings, this approach faces several fundamental challenges, which motivate our proposed framework.
First, obtaining accurate feature importance estimates in high dimensions is problematic. These estimates often suffer from bias due to inherent correlations in high-dimensional data, with well-known metrics performing poorly in correlated settings (Chamma el al. 2023a). The estimates also exhibit. high variance and unreliability, as even classical methods such as ordinary least squares coefficients are known to become unstable when the number of features approaches or excceds the sample size (Hastic et al.) 2009). The computational burden compounds these issues: computing importances for large number of features can be expensive, forcing popular approaches to resort to approximations. For instance, methods like Shapley
3

---

Published in Transactions on Machine Learning Research (12/2025)
values are computationally infeasible without substantial approximations that compromise their theoretical guarantees (Mitchell et al. 2022 Ghorbani and Zou, 2019).
Second, this approach fundamentally misaligns computational and statistical resources with our objective. Our focus is not on ranking all features but only the top-k, a crucial distinction that should guide algorithm design. Analogous to sparse modeling in high-dimensional statistics, we expect many features to be noise and are uninterested in learning their ranks (Hastie al.) 2015). However, estimating and sorting importance metrics expends valuable statistical resources (e.g., data samples) and computational effort on all features, including noise features. This misallocation is particularly problematic when distinguishing between features of similar importance levels, where more precise estimation is needed.
For example, consider predicting credit risk using various financial and demographic features, which are often highly correlated. Lenders using machine learning models to model credit risk are required by law to disclosc the principal reasons when denying an application or taking other adverse action (Consumer Financial Protection Bureau 2011). While standard feature importance methods may struggle to distinguish between similarly important features due to correlations, our framework is designed to specifically address these challenges by focusing resources on the most relevant features and improving feature ranking accuracy in high-dimensional, correlated settings.
Motivated by these considerations, we develop two-stage approach that combines efficient feature importance estimation with adaptive refinement. We first introduce RAMP (Ranked Attributions with MiniPatches), which leverages ensemble learning principles through random subsampling of both features and observa- tions. We then extend this to RAMPART (RAMP And Recursive Trimming), which progressively focuses computational resources the most promising features.
2.3 Ranked Attributions with MiniPatches (RAMP)
We begin by assuming access to a feature importance ranking procedure M. This procedure M: gaxm xR" H {o..... m - }} takes as input a subset of the data (i.c., "minipatch") (X1,F, Y,) where I S [N, |7| = n is a subsample of observations and F C [M]]P]] = m is a subsample of features, and returns rank estimates fy for features j E [F]. In practice, this ranking procedure M typically involves: (i) fitting a predictive model to the minipatch, (ii) computing feature importance scores using a specific attribution method, and (iii) sorting these scores by magnitude output ranks. For example, a simple ranking procedure M could entail fitting a linear regression model and ranking features by the magnitude of their standardized coefficients, or altcrnatively, fitting decision tree and ranking fcatures by their mcan decrcase in impurity (MDI).
Given M, we formalize RAMP in Algorithm [ The procedure operates by generating mumerous minipatches, with each consisting of a different random subsample of both observations and features. For each minipatch, RAMP applics M (c.g., linear regression coefficients or decision tree with MDI) to obtain feature importance rank estimates r), which are then averaged across all minipatches where each feature appears. This minipatch ensembling approach reduces variance while breaking harmful correlation patterns between features (Gan ct al. 2022 The final step sorts the averaged ranks F, using order statistics F(I)....;F(M) to obtain the final rankings.
Importantly, RAMP serves as a meta-algorithm that improves the accuracy of feature rankings regardless of the specific attribution method employed. More specifically, each ranking procedure M produces its own importance measure 6y and hence ranks ry, but ordinary estimates of these quantities typically suffer from instability in high-dimensions due to feature correlations and sampling variability (Chamma et al., 2023b Kelodjou et al. 2024). Given the choice of ranking procedure M, RAMP improves the estimation of the corresponding ranks ry by ensembling across diverse minipatches, effectively reducing variance while maintaining the statistical properties of the underlying importance measures (Gan et al. 2022 Yao and Allen 2020). This approach provides more stable and accurate approximations of the true feature importance ranking, while maintaining the flexibility to accommodate any feature importance procedure. Since decisions are made based on ranks instead of raw attribution magnitudes, RAMP can ensemble heterogeneous per- minipatch ranking procedures (c.g., MDI, SHAP, Integrated Gradients) without modification, ochoing the model-class view that genuinely important variables should remain important across a broad family of predictive models (Fisher et al. 2019).
4

---

Published in Transactions on Machine Learning Research (12/2025)
Algorithm Ranked Attributions with MiniPatches (RAMP)
Input: Ranking procedure M, dataset D = [(x;, vi) = (x, Y), number of minipatches B, data subsample
size n < N, feature subsample size m < M
Output: Estimated feature ranks fI,..., fM
for b [B] do
I randomly subsample observations [N]
F. < randomly subsample m features C [M]
4: r < MM Fo I1); for j Fb.
5: end for
B)JER 3
6: For all j E [M], set 8g - E1E[B] 1 { E Fb}
return f (i f()) =ry)-
2.4 RAMP And Recursive Trimming (RAMPART)
Though RAMP provides an improved foundation for feature ranking, the uniform treatment of all features within RAMP is not ideal when our primary interest is in the lop-k ranked features. To address these limitations, we develop RAMPART (Ranked Attributions with MiniPatches And Recursive Trimming), an adaptive framework that builds upon the minipatch sampling from RAMP while incorporating ideas from the sequential halving literature. Our approach is inspired by the pioneering work of Karnin et al. (2013) successive halving algorithms, as well as recent advances in batched sequential halving (Jun et al. 2016 Koyamada et al. 2024) for the fixed batch setting. While these methods were originally developed for best-arm identification in multi-armed bandits, we adapt their core insight of progressive resource allocation to the feature ranking context.
RAMPART operates by iteratively applying RAMP an increasingly focused set of features. In each round, it identifies and retains the more promising half of the features while eliminating those less likely to be in the top-k set. This adaptive strategy, formalized in Algorithm recursively trims the feature pool size, enabling more accurate rank estimation in later rounds where fine-grained distinctions become crucial. The number of iterations is carefully chosen to ensure the final feature pool size aligns with our target k. By concentrating computational resources on the most relevant features, RAMPART more efficiently spends its resources, distinguishing between similarly-ranked important features. This adaptive resource allocation is a critical advantage over traditional approaches that uniformly evaluate all fcatures since features that survive later rounds in RAMPART evaluated more frequently, enabling increasingly precise rank estimates where they matter most.
Algorithm Ranked Attributions with MiniPatches And Recursive Trimming (RAMPART)
Input: Ranking procedure M, dataset D ((x,, 14) W number of minipatches B, data subsample size n N, feature subsample size < M, top features k. Output: Estimated feature ranks f
1: + log, M - [log, k]
C1 + [M]
for E (l...., T) do
PI...., c. RAMP n, (C.)
5: C++1 - ( 1/,112
end for
return f, = 2J j E Cr otherwise f, -==
5

---

Published in Transactions on Machine Learning Research (12/2025)
3 Theoretical Analysis
In this section, we show thcoretical guarantees for RAMP and RAMPART. We also prove that RAMPART achieves performance superior to that of RAMP under mild assumptions on the properties of the ranking procedure- .M.
Assumption 1. Unique Top-k Ranks. For any two features j.j' E [M] where at least one feature has true rank smaller than k either Tj > Ty or ry > Tj.
While this assumption might appear restrictive at first glance, only requires distinct ranks among the top-k features of interest. Ties are permitted among features outside this set. This reflects real-world settings where we need to distinguish among the most important features but can tolerate ambiguity in the ordering of less relevant or null features.
Assumption 2. Rank Consistency. For any two features j.J' S [M, m with ry < ry that are sampled in the samn minipatch,
1 2p> 2
P(β,
where the probability is taken over all minipatches F of size m in S such that j and j' are sampled together.
This consistency assumption requires that our ranking procedure M performs better than random guessing when comparing features within the same minipatch on average. If a model cannot reliably order features when evaluated together, cannot be expected to produce meaningful relative rankings. In particular, we only require probabilistic consistency, allowing for errors individual comparisons.
Assumption Unbiased Ordering. For any two features j.j' E [M] 2 m with ry < <r',
E E Biijer # < <( E
where expectations are taken over minipatches in S that sample one feature but not the other.
This assumption says that rank comparisons should remain informative across different minipatches. It requires that features of higher importance tend to receive better ranks relative less important features, even when they appear in separate samples rather than being directly compared. This property is especially relevant for minipatch approach, since aggregate rank estimates across many different subsamples where not all pairs of features appear together.
Assumption 4. Bounded Deviation. There exists universal constant C 0 such that for any two features j,j E S S (M), 2 m with rj < <ry that are sampled in the same minipatch,
Eliy - f,ly > Fy] - Eli, - fylf > fy >C
where expectations are taken over all minipatches F' of size m in S such that j and j' are sampled together.
This final assumption ensures that correctly ordered features are separated by a larger margin than incorrectly ordered ones. Specifically, when more important feature ranked above less important one, their expected rank difference exceeds the expected difference when incorrectly ordered by at least some small positive constant C. This property provides stability in our estimates where ranking errors have less impact than correct orderings, allowing to recover true feature ordering through ensembling. Together, these assumptions enable theoretical guarantees for our algorithms.
Theorem 5. Under Assumptions 14 if the number of minipatches salisfies
BRAMP = (M²
k.M
In
then with probability at least 1 5, RAMP will correctlyy rank all top-k features: Pr, = Try for j E fl....,k) k).
We defer the proof to Appendix A.2
Building on these results, we now show that RAMPART achieves stronger performance guarantees while requiring the same order of computational complexity.
6

---

Published in Transactions on Machine Learning Research (12/2025)
Theorem 6. Suppose T 2 3. Then under assumptions 1 4
there erist choices of {B.) such that RAMPART corvectly identifies the lop-k features with probubility af least 1 - 6 5/T using the same order
of total minipatches as RAMP: E B, BRAMP.
We defer the proof to Appendix A.3 While both algorithuss require the same order of total minipatches, RAMPART achieves the correct ranking with higher probability. Furthermore, our assumptions are sub- stantially weaker compared to existing work. Traditional multi-armed bandit approaches require rewards to be independent (Russo) 2020), while recent works on variable selection have imposed much stronger identifiability assumptions demanding each arm's optimal reward distribution be uniformly separaled from all other possibilities (Liu and Ročková, 2023). In contrast, our analysis does not assume independence among features nor parametric measurement noisc: we only require probabilistic consistency and better-than-chare pairwise ordering, allowing for substantial noise and correlation between features.
4
Empirical Studies
In this section, we demonstrate the empirical performance of RAMP and RAMPART through series of carefully designed experiments and two real-data case studies.
4.1 Comparative Simulation Studies
We design a comprehensive simulation framework to evaluate feature ranking methods across numerous settings. We first generate data from multivariate normal distributions x; E RM under two covariance structures: identity (Σ (E= 7) or auloregressive (Ei, = -pl--|l with p p= 0.5). For both covariance structures, we assign non-zero coefficients 3, (100 -i+ + 1) to the first ten features (i = l...., 10), with all others set to zero. The parameter Y E f0.03, 0.05, 0.10, 0.20, 0.50), referred to signal-to-noise ratio (SNR) in subsequent analyses, controls the separation between coefficient values, with smaller values creating more challenging ranking problems.
We construct four distinct scttings: linear regression, nonlinear additive regression, lincar classification, and nonlinear additive classification. For regression settings, we generate responses as y, = fxx) + 4 with G ~ (O, 1); for classification, y/ ~ Ber(1/(1 + e=()))). The function f distinguishes linear settings, where (x:) = x from nonlinear additive settings, where f(x.) = ΣØ= 8,95 (Xi,); with g;(x) cos (x) for E 51 and gz(x) sin'-4(>) for j {6...., 10). All features X, in the lincar settings and gj (X,) in the nonlinear settings are standardized to zero mean and unit variance, ensuring that coefficient magnitudes [8ml directly reflect the contribution of the features and function as the ground truth importance measures.
In each scenario, we employ task-appropriate prediction models: OLS and logistic regression for the lincar regression and classification settings, respectively, and random forests (100 trees) for the nonlinear regression and classification tasks. Additionally, we employ neural networks across all settings, configured as regressors for regression tasks and classifiers (with final sigmoidal activation) for classification tasks. All neural networks have a consistent two-layer architecture with M hidden units and ReLU activation trained to convergence.
As comparison methods, we first include a baseline (model-specific) feature importance (FT) method for each prediction model: absolute coefficients for OLS and logistic regression, Mean Decrease in Impurity (MDI) for random forests, and Integrated Gradients for neural networks (Sundararajan et al. 2017). We also evaluate two popular model-agnostic approaches. First, we apply SHAP with architecture--peciffe variants (HAP TreeSHAP, or GradientSHAP) (Lundberg and Lee 2017 Lundberg et al.] 2020), computing global scores by averaging local attribution scores across observations. Second, we assess permutation importance by measuring the average change in prediction error over 100 random permutations set of size N/2. For all methods, we obtain feature rankings by sorting importance scores by magnitude.
held-out
To ensure a fair and direct comparison, we implement RAMP and RAMPART using the same baseline (model- specific) feature importance method, described previously, for each prediction model: absolute coefficients for OLS and logistic regression, MDI for decision trees, and Integrated Gradients for small neural networks (two-layer neural network with 5m hidden units and ReLU activation trained for epochs on each minipatch). This implementation ensures direct. comparability, as performance differences stem solely from our algorithmic
7

---

Published in Transactions on Machine Learning Research (12/2025)
framework rather than from variations in feature importance attributions. Both RAMP and RAMPART use minipatches with = 125 observations and = 10 features. For our experiments with M = 500 dimensions and k = 10 target features, RAMPART requires 6 halving iterations with 2000 minipatches per iteration, while RAMP uses 10000 total minipatches, maintaining comparable computational budgets. Note we omit Goldwasser and Hooker (2025) as their approach requires repeatedly resampling Shapley estimates for top features when statistical tests fail, making it prohibitively expensive in high dimensions.
We evaluate ranking performance using Rank-Biased Overlap (RBO) with p 0.7 (Webber al. 2010 +
which naturally prioritizes accuracy at higher ranks through geometrically decreasing weights, making it appropriate for our top-k ranking task:
A RBO, (1 Σ
8
For each experimental sctting, we conduct 100 simulations with random seeds fixed across all methods, averaging the resulting RBO scores and reporting standard error bars in our plots. Figure | demoostraaes that RAMPART consistently yields the most accurate feature rankings across both regression and classification tasks, highlighting the benefits of adaptive resource allocation. RAMPART's advantage becomes particularly pronounced at higher signal-to-noise ratios (SNR 2 0.1) and under autoregressive covariance, showcasing robust performance even with correlated features. While performance naturally decreases in nonlinear additive settings for all methods, RAMPART maintains its relative advantage. These results demonstrate that RAMPART's adaptive allocation strategy provides substantial practical benefits across diverse modcling settings. Additional results for higher dimensions (M 1000 and M 2000) with fixed sample size (n 250) are provided in Appendix
B further showcasing RAMPART's robustness in increasingly challenging
high-dimenssonall settings.
To further demonstrate how accurate feature rankings translate to improved predictive performance, we also conducted ablation studies using the same simulation setup as in our identity covariance classification experiments from Figure [ Here, using the same predictive models and configurations as before, we selected two representative signal-to-noise ratios (SNR 0.06 and SNR 0.5), split the data into a 70/30 train-test split, and assessed the model's test prediction performance the top-ranked features are progressively added in the model in order of their estimated importance rankings. In Figure we illustrate how classification error decreases as the lop-ranked features are progressively added as predictors in the model. This comparison reveals the practical impact of feature importance ranking accuracy on prediction performance, with RAMPART's superior rankings consistently yiclding the lowest classification errors from models trained using only the top-ranked features. Additional simulation results for ablation studies with higher dimensions can be found in Appendix M which further verify RAMPART's robustness increasingly challenging high-dimensional settings.
4.2 Cancer Genomics Case Studies
We finally demonstrate RAMPART's effectiveness through two high-dimensional cancer genomics case studies: (i) a regression problem to predict the elficacy or response of a cancer drug on different cancer cell lines, and (ii) classification problem to predict breast cancer subtypes based upon gene expression profiles. Across both settings, our goal is not only to build accurate predictive models, but to rank the genes that most strongly drive these outcomes, thereby providing interpretable insight into underlying biology and suggesting hypotheses for future therapeutic development.
4.2.1 Case Study Drug Response Prediction
For our first case study, we predict response to PD-0325901 (an MEK inhibitor) across N = 259 human cancer cell lines using their RNASeq gene expression profiles (M = 1104 genes) from the Cancer Cell Line Encyclopedia (CCLE) (Baretina et al. 2012) (see Appendix C.1 for details). As before, we compare multiple feature ranking methods to identify the top 10 responso-driving genes. Our bascline approach uses a random forest regressor (200 trees) with Mean Decrease Impurity (MDI). We also compute TreeSHAP values on the same random forest. model and evaluate permutation importance by measuring prediction error changes
8

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Simulations
Logistic Regression
Neural Networks
Random Forests
Neural Networks
(Basline Fl: Coofficients)
(Baseline Fl: Integrated Gradients)
(Baseline Fl: MDI)
(Baseline Fl: Integrated Gradients)
0.5
0.-
0.4-
0.4
Classification
00
300
2.6
3.10
320
350
0.53
0.56
0.58
0.28
0.52
3.3
3.05
0.10
0.70
0.50
033
0.56
0.1
0.23
0.5
Autoragressive
Autorege ssaive Covatiance
Astoregressive Covariance
Autoregrasaive Uavariance
0:5
0.6
59
0.5
0:5
0.5
055
880
0.4-
0.4
0.4
8:4
0.5
0.2
3.03
3.0
3.5
0.10
0.22
0.00
0.05
0.10
0.0
0.23
544
200
581
20R
Ordinary Least Squares
Neural Networks
Random Forests
Neural Networks
(Baseline Fl: Cocfficients)
(Baseline Fl: Integrated Gradients)
(Baseline Fl: MDI)
(Baseline Fl: Integrated Gradients)
Covariance
0.5
6--
0.5
58-
100
0.4-
0.4
0.1
Regression
3.03
2.05
3330
320
3.50
0.23
0.28
0.2
0.20
0.52
0.03
3.06
0.10
6.0
0.50
0.5
0.28
0.1
0.23
0.5
Autorogressive
Autoregressive
Autorogressive Coveriance
Autoregressive Covariance
0.4-
0.4
0.1
0.5
303
2.20
2.50
0.18
0.25
0.03
0.00
0.50
0.22
SM1
530
581
3001
Figure 1: Feature importance ranking accuracy for classification (lop) and regression (bottom) (M= = 500)
9

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Additive Simulations
Logistic Regression
Neural Networks (Baseline Fl: Integrated Gradients) Gradieent
Random Forests (Baseline Fl: MDI)
Neural Networks
(Baseline Fl: Coefficients)
(Baseline Fl: Integrated Gradients)
SNR-0.06
SKR-0.06
SNR=0006
SNR=0.96
2.50
0.50
0.50
0.50
2.45
0.45
0.43
0.45
Carcflatin
6.40
0.40
0.00
0.40
0:35
0.35
0.25
Classification
0.20
0.30
0.30
0.20
SNR 5.5
S.R=0055
SNR
SNR+0.5
2.40
0.40
0.00
0.40
8.30
0.30
0.30
0.30
0.25
0.25
2.00
0.20
0.20
0.20
0.55
0.15
0.33
2.10
0.10
0.10
0.30
numbbr Kankcd Pooturts
Numri of kcd Poatorcs
Skarbor Panket Poaturcs
Narbzor of Rankod Poatancs
Saseline
SHAP
Permecaation Fl
RAMP Ne/Baeeir FiI
RAMPART (weBasseline Fi)
Figure 2: Classification error vs. number of lop-ranked features used as predictors in ablation simulation with identity covariance (M = 500)
over 100 random permutations on a 50/50 train-test split. For RAMP and RAMPART, we use MDI with regression trees as the minipatch ranking procedure M, with minipatch parameters m = 10 and n = 100.
RAMPART uses 4000 minipatches per iteration, while RAMP used 20000 total minipatches. Table 4 presents the resulting gene rankings from all methods.
Since real-world genomic studies lack ground-truth feature importance rankings, we validate our top-10 gene findings using established biological knowledge. This biological validation approach follows established precedent in the feature importance literature (e.g., see Lundberg et al. 2020 Covert et al. 2020) where feature importance on genomic data are validated through connections to prior medical research. Along these lines, there are several key biological findings from this case study. First, all 10 genes identified by RAMPART have boen proviously implicated in biological pathways involving PD-0325901 and related cancers (Table I In particular, the lop-ranked gene from RAMPART, TOR4A, is a known oncogene for glioma. (Wang el al. 2022), a cancer for which PD-0325901 is currently being tested in clinical trials (Vinitsky et al.) 2022]. Second, the main pathway affected by PD-0325901 is the MEK/ERL signaling pathway, which regulates cell proliferation, differentiation, and survival. Several of the genes identified by RAMPART, including ETV4.
SPRV2, and WNT5A, are key factors in the MEK /ERK pathway (Oh et al. 2012 Milillo et al. et al. 2021).
2015 Hasan
Finally, we perform a gene ontology (GO) enrichment analysis AAshburner et al., 2000 Aleksander et al. 2023), a standard bioinformatics approach for identifying the biological processes that are enriched (or over-represented) in a set of genes, to further validate the biological relevance of the top-10 genes identified by RAMPART. According to the GO enrichment analysis, the biological processes, "regulation of transmembrane receptor protein
kinase signaling pathway" and "regulation of cellular response to growth factor stimulus," " are significantly enriched (FDR 0.05) in the top-10 genes from RAMPART. Notably, the serine/threoning kinase signaling pathway plays an essential role in the activation of MEK (Zheng and Guan 1994). In contrast, no GO biological processes are significantly enriched in the top-10 genes from any of the other competing methods. Additionally, of the four genes (i.c., SPRY2, FERMT1, WNT5A, and NRROS) linked to the identified GO biological processes, two are uniquely identified by RAMPART and have not been ranked in the top 10 by any other hod This GO enrichment analysis not only supports
10

---

Published in Transactions on Machine Learning Research (12/2025)
Rank
Gene
Connection to PD-0325901
TOR4A
Torsin family gene and oncogene for glioma and other cancers (Wang et al.
2022
2
ETV4
Well-known ETS transcription factor regulated by MEK/ERK /ERK pathway OOh et al.
2012
SPRY2
MEK/ERK /ERK pallway (Zheng and Guan Gap junction gene associated with increased Lumor progression for various cancers et al. 2016)
Key inhibitor of MEK/
1994 Milillo al.
2015
4
GJB1
Aasen
PYCARD
Encodes key adaptor protein in infammatory and apoptotic signaling pathways, playing
dual roles in multiple cancers (Protti and De Monte 2020)
6
WNT5A
Wut signaling pathway gene which can enhance MEK/ERR pathway Zheng and Guan
1994
Hasan al. 2
7
FERMTI
Regulates Ser/ Thr kinase signaling pathway, activating MEK Zhheng and Guan
1994
8
NRROS
Regulates Ser/ 'I'r kinase signaling pathway, activating MEK (Zeng and Guan 1994
LYZ
Encodes lysogyme; shown to exhibit aberrant expression in tumor cells GGu al.
2023
10
NPASE
Regulator of cireadian rhythms and tumor suppressor involved in DNA damage response
(Hollman et al. 2008
Table 1: 1op-10 genes from RAMPART for drug response prediction and their connections to PD-0325901. Genes highlighted in blue were identified by the GO enrichment analysis be involved in the "regulation of transmembrane receptor protein serine/threonie threonine (Ser/Thr) kinase signaling pathway" and "regulation of cellular response to growth factor stimulus."
the biological relevance of RAMPART's identified genes, but also highlights the advantage of RAMPART over existing methods for fcature importance ranking and high-impact scientific discovery.
4.2.2 Case Study II: Breast Cancer Subtype Classification
We next consider a high-dimenniooall multi-class classification problem based on The Cancer Genome Atlas (TCGA) Breast Cancer (BRCA) cohort. Using RNA-seq data from the TCGA-BRCA study (The Cancer Genome Atlas Network 2012) and PAM50 subtype labels (Parker al. 2009), we obtain gene-expression matrix with N = 758 primary tumors and M 5000 genes after preprocessing (sce Appendix C.2 for details). Each sample is assigned one of five intrinsic subtype labels (Luminal A, Luminal B, HER2-enriched, Basal-like, Normal-like), yielding a five-class classification outcome. As in Section 4.2.1 we compare multiple feature-ranking methods and focus on the top 10 genes from each method as subtypo-disrriminativ markers. All hyperparameters are identical to Case Study 4.2.1 except that we replace decision-tree/ www.eeeeaeooooooreeeeeeeee regressors with their classífication counterparts. Table presents the resulting gene rankings from all methods.
Notably, the 10 RAMPART-selotted genes in Table have each been implicated and extensively studied in the previous breast cancer literature. Genes including ESR1, FOXA1, GATA3, and MLPH mark luminal, estrogen peceptor-divver discase and are tightly linked to endocrine response and prognosis (Brett et al. 2021
Toy et al.
2013 Fu et al.] 2016 Hurtado et al. 2011 Takaku et al. 2015 Kouros- Mehr et al. 2006 Thakkar FOXC1 captures the basal-like transcriptional programme and loss of luminal identity Hann
ct al.
2015).
et al.
2017 Yu-Rice al. 2016). On the other hand, TPX2, ASPM, CDK1, FOXM1, and UBE2C function
as core regulators of mitotic entry, spindle assembly, and chromosome sogregation whose overexpression marks highly proliferative, poor-prognosis lumors (Marugán et al. 2024 Tang et al., 2019 Enserink and
Chymkowitch 2022
Bergamaschi et al.) 2014 Hu et al., 2025).
We also performed GO Biological Process enrichment on the top-10 genes from each method and visualized the processes significantly enriched at FDR p < 0.05 (Table G). RAMPART's genes are enriched for 16 such biological processes, including transcriptional regulation, reproductive and gland-related development, and coll-cycle and proliferationnassoiiate programs, which are well-known components in cancer biology (Hanahan and Weinberg 2011). In contrast, RAMP, SHAP, and permutation importance identify a more limited set of 5, 10, and 6 enriched GO terms, respectively.
11

---

Published in Transactions on Machine Learning Research (12/2025)
Rank
Gene
Role in Breast Cancer Biology
1
E
Encodes estrogen receptor-o: ESRI mutations and dysregulation drive endocrine resistance in ER disease (Brett et al. 2021 Toy et al. 2013
2
FOXC1
Basal-like transcription factor opposing CATA3/ ERa; linked to ERa loss,
docrine resistance, and aggressive basal-like tumors (Yu-Rice et al. 2016
3
TPX2
Mitotic spindle assembly factor overexpressed CIN-high tumors; activates YAP
signaling and sensitizes cells to the SRC inhibitor dasatinib (Marugán et al. 2024
4
FOXAI
Pioneer transcription factor opening chromatin for ER DNA binding; reprograms
ER-driven transcription and promotes endocrine resistance (Fu et al. 2016)
5
ASPM
Spindle pole and centrosome protein; overexpression drives proliferation and marks
high-grade, prot-prognosis breast cancer (Tang et al, 2019)
6
MLPH
Component of estrogen-responsive luminal signature; high MLPH marks ER
tumors with good prognosis (Thakaar et al. 2015
7
CDK1
Core cyclin-dependent kinase controlling G2/M transition and mitotic entry;
hyperactivation marks highly proliferative tumors Enserink and Chymkowitch 2022)
8
FOXM1
Oncogenic transcription factor driving G2 G2/M cell-cycle genes; promotes prolifera- lion, invasion, and endocrine resistance in ER disease (Bergamaschi et al. 2014
9
GATAS
Master regulator of mammary luminal differentiation: recurrently mutated
luminal tumors and essential for luminal identity (Kouros-Mehr et al. 2006
Takaku et al. 2015)
UBE2C
Ubiquitin-conjuaaaln enzyme activating the APC/C complex; overexpressed
lighly proliferative lumors and associated with poor prognosis (Hu et al. 2025
Table 2: Top-10 genes from RAMPART for breast cancer subtype prediction and their functional roles.
Method
Case Study I
Case Study
RAMPART
21.83
36.97
RAMP
14.15
20.65
Baseline
12.48
3.62
TreeSHAP
13.79
58.46
Permutation 489.86
3637.54
Table 3: Wall-clock time (seconds) for computing feature rankings in Case Study (drug response, N 259, M 1104 genes) and Case Study 11 (breast cancer subtypes, N 758, M 5000 genes). All methods use the same tree-ensemble hyperparameters across both datasets; times exclude data loading and basic preprocessing. Experiments were run a 16-inch MacBook Pro (Apple M3 Max, 48 GB RAM, macOS 14.3).
4.3 Runtime and Scalability
Practical runtime, Table 3 shows that RAMPART computationally lightweight in the regimes we consider. In the smaller CCLE drug-response case study (M =1044, all non-permutation methods finish within 25 seconds: RAMPART takes 21.8 seconds, compared with 12 14 seconds for Baseline RF and TreeSHAP. Thus, this moderate-diimensional problem, the extra stability comes at only modest constant-factor overhead. When we move to the gigher-dimensional TCGA breast cancer subtype task (M = 5000) keeping all hyperparametors fixed, RAMP and RAMPART remain well under minute (20.7 and 37.0 seconds), and now RAMPART is act faster than Tre (58.5 seconds) while providing a more robust top-k ranking. In both case studies, permutation feature importance is one two orders of magnitude slower (from 8 minutes up to more than an hour), reflecting the need re-evaluate the trained forest on many permuted versions of each feature and thereby perform work that scales essentially linearly with the feature dimension and the number of permutations.
gain from recursive trimming
12

---

Published in Transactions on Machine Learning Research (12/2025)
Scaling with dimensionality. The scalability of RAMPART follows from how allocate compute across trimming rounds. Each round fits the same number of minipatch forests on patches of fixed size and a fixed subsample of observations, so its cost is approximately a constant multiple of the cost of fitting single random forest on a small feature subset. The only quantity that grows is the mumber rounds nceded to shrink from M candidates down to finalists: with a halving schedule, L R [log((M/k) runs only single such round (with more minipatches). so RAMPART incurs an additional logarithmic factor in runtime relative to RAMP while sharing the same per-round cost. Thus, the dominant term in the runtime of RAMPART is constant times (ok). In our experiments, increasing M by factor of R 4.5 (from 1104 to 5000) increases RAMPART's runtime by only factor of = 1.7, whereas TreeSHAP slows down by more than 4x and permutation by more than 7x. Taken together, these results illustrate the core advanlage of our design: RAMPART attains a statistically stronger and more interpretable lop-k ranking at the cost of only a small, logarithmically growing multiple of fitting a single forest, while remaining scalable to increasingly higher-dimensional scttings.
In contrast, RAMP
5 Discussion
In this paper, we introduced RAMPART, a novel framework that achieves accurate feature ranking in high-dimensional settings by combining minipatch ensembles with recursive trimming. Instead of developing new feature importance method, we adopt a complementary, model-agnostic view that can leverage any existing feature importance procedure importance procedures in a ranking algorithm that uses inexpensive subsampled fits to screen many features and concentrate computation on the candidates that plausibly belong in the top-k. As demonstrated through our genomics case studies, precise feature ranking can drive scientific discovery by identifying key drivers in complex biological processes and guiding future research. However, while our empirical and theoretical results are encouraging, they also highlight several challenges and opportunitics. Given that RAMPART is a wrapper method, it may inherit modeling assumptions and potential biases from the underlying attribution procedure, and our current guarantees are derived under stylized separation and consistency conditions that. may be difficult to verify in practice. These considerations motivate several directions for future work.
black-box primitive. RAMPART wraps these existing feature
First, adapting minipatch sizes and sampling schemes as the candidate feature pool shrinks could enable more precise comparisons while maintaining computational efficiency, and our minipatch ensemble framework offers a flexible foundation for exploring alternative adaptive sampling methods to further enhance top-k ranking performance. Second, extending the ideas developed here beyond tabular settings to other data modalities, such as graphs, images, and time series, via domain-appropriate base explainers would broaden the impact of this paradigm. Finally, refining our assumptions and sharpening thcoretical bounds in regimes that mirror high-stakes applications would further strengthen the reliability of RAMPART for scientific and decision- decision-making use.
Acknowledgments
GA and YC acknowledge support from NSF DMS-2516872.
References
Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jía, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. Explainable machine learning in deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 648-657, Barcelona Spain, January 2020. ACM. ISBN 978-1-4503-6936-77 doi: https://dl. acm..acm.co wwt.amorg//////oooe 111.:////11.1115151151155551 3375624
URL
Mare Jaxa-Rozen and Evelina Trutnevyte. Sources of uncortainty long-term global scenarios of solar photovoltaic technology. Nature Climate Change, :1(3):266 273, March 2021. ISSN 1758-678X, 1758-6798. doi: 10. 00003//411580011111111 URL http://wwwww.aatoo////////aaaaa/
nature.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Dahycon Jeong, Shilpa Aggarwal, Jonathan Robinson, Naresh Kumar, Alan Spearot, and David Sungho Park. Exhaustive or exhausting? evidence on respondent fatigue in long surveys. Journal of Development Economics, 161:102992, 2023. doi: (e0e
Robert F. DeVellis. Seale Development: Theory and Applications. Applied Social Research Methods Series. Sage Publications, Thousand Oaks, CA, 4th edition, 2017.
Peter M Visscher, Naomi R Wray, Qian Zhang, Pamela Sklar, Mark I McCarthy, Matthew A Brown, and Jian Yang. 10 years of gwas discovery: biology, function, and translation. The American Journal of Human Genetics, 101(1):5-22 2017.
Yuhua Fu, Jingya Xu, Zhenshuang Tang, Lu Wang, Dong Yin, Yu Fan, Haiyan Wang, Lilin Yin, Shilin Zhu, Mengjin Zhu, Mei Yu, Xinyun Li, Xiaolei Liu, and Shuhong Zhao. A gene prioritization method based on a swine multi-omics knowledgebase and a deep learning model. Communications Biology, 3:502, 2020. doi: 0040088//22000300-00122222
Qianru Wang, Tiffany M. Tang, Michelle Youlton, Chad S. Weldy, Ana M. Kenney, Omer Ronen, J. Weston Hughes, Elizabeth T. Chin, Shirley C. Sutton, Abhineet Agarwal, Xiao Li, Merle Behr, Karl Kumbier, Christine S. Moravec, W. H. Wilson Tang, Kenneth B. Margulies, Thomas P. Cappola, Atul J. Butte, Rima A. Arnaout, James B. Brown, James R. Priest, Victoria N. Parikh, Bin Yu, and Euan A. Ashley. Epistasis regulates genetic control of cardiac hypertrophy. Nature Cardiovascular Research, pages 1-21, 2025.
Elnaz Pashaei, Elham Pashaei, and Nizamettin Aydin. Biomarker identification for alzheimer's disease using multi-filter gene selection approach. International Journal of Molecular Sciences, 26(5):1816, 2025. doi: 10.3390/jmms260518866
Scott Lundberg and Su-In Lee. A Unified Approach to Interpreting Model Predictions. NewrIPS, 2017. URL 07874
https://arxiv: httpp/////aiivvorg//aaa/////////
Bitya Neuhof and Yuval Benjamini. Confident Feature Ranking. In Sanjoy Dasgupta, Stephan Mandt, and Yingzhen Li, editors, Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, volume 238 of Proceedings of Machine Learning Research, pages 1468-1476. PMLR, May 2024. html
URL htttps::/proceedinggg. mlr, http://ooeelnggg......essss////s
Jeremy Goldwasser and Giles Hooker. Statistical Significance of Feature Importance Rankings, January 2025.
URL http://arxiv.orge 20... arXiv:2401.15800 [stat]..
Kristin Nicodemus and James D Malley. Predictor correlation impacts machine learning algorithms: implications for genomic studies. Bioinformatics, 25(15):1884 1890, 2009.
Kristin K Nicodemus. On the stability and ranking of predictors from random forest variable importance measures. Briefings in bioinformatics, 12(4))369-37 2011.
Giles Hooker, Lucas Mentch, and Siyu Zhou. Unrestricted permutation forces extrapolation: variable importance requires at least one more model, or there is no free variable importance. Statistics and Compuling, 31:1- 16, 2021.
Luqin Gan, Lili Zheng, and Genevera Allen. AgnostAgnosstic Confidence Intervals for Feature Importance: A Fast and Powerful Approach Using Minipatch Ensembles, 2022.
Christoph Molnar. Interpretable Machine Learning. 3 edition, 2025. ISBN 978-3-911578-08-5. URL
https://christophhn git..t.tttutii.. httphab.iiointteeeetaaellelllela mloobbokk
Leo Breiman. Random forests. Machine learning, 45:5 32, 2001.
Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning Important Features Through Propagating Activation Differences, October 2019. URL http:://tx:::i wtt.aaawwww..aaraaaaaaaaaaaeaeae 1704.. 02685
[es).
14

---

Published in Transactions on Machine Learning Research (12/2025)
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic Attribution for Deep Networks, June 2017. URL
1ttps//11..00.66555 arXiv:1703.014. [es].
Jing Lei and Larry Wasserman. Distribution-free Prediction Bands for Non-parametric Regression. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76(1):71 96, Jamuary 2014. ISSN 1369- 7412, 1467-9868. doi: 10.1111/rssb.121 12021. URL https://accadmmii oup. 7075937
Scott M Lundberg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2(1):56-67, 2020. doi: 10.1038 (L.008//42256--- 001-266------------- 9.
Soheil Mohajer, Changho Suh, and Adel Elmahdy. Active Learning for Top-K Rank Aggregation from Noisy Comparisons. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 2488 2497. PMLR, August 2017. URL ltttps://proceediiiiii www.engg.m.......m..m......mee prress/770mooajerrrr.. html
Sahand Negahban, Sewoong Oh, and Devavrat Shah. Rank Centrality: Ranking from Pairwise Comparisons. Operations Research, 65(1):266-87, 2017.
Yuxin Chen and Changho Suh. Spectral MLE: Top-K Rank Aggregation from Pairwise Comparisons. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 371-380, Lille, France, July 2015. PMLR. URL httts:://proceeddnngg . mlr. www...rresse///eem chena15..ee......m httm
Yuxin Chen, Jianqing Fan, Cong Ma, and Kaizheng Wang. Spectral Method and Regularized MLE Are Both Optimal for Top-K Ranking. Annals of Stolislics, 47(4), 2019.
Reinhard Heckel, Nihar B. Shah, Kannan Ramchandran, and Martin J. Wainwright. Active ranking from pairwise comparisons and when parametric assumptions do not help. The Annals of Statistics, 2016. URL oarp
http::://app..smmatiiiii
Reinhard Heckel, Max Simchowitz, Kannan Ramchandran, and Martin Wainwright. Approximate Ranking from Pairwise Comparisons. In Amos Storkey and Fernando Perez-Cruse editors, Proceedings of the T Pientt-Pirrt International Conference on Artificial Intelligence and Statistics, volume 84 of Proceedings of Machine Learning Research, pages 1057-1066. PMLR, April 2018. URL https://pproceeddng/ mlr
Nihar Shah and Martin Wainwright. Simple, Robust and Optimal Ranking from Pairwise Comparisons. Journal of Machine Learning Research, 18:1-38, 2018. URL http::///ia...oo////pppessseeeee wwtpppapass/v11/11/222222 206. 206.html html
Fabian Wauthier, Michael Jordan, and Nebojsa Jojic. Efficiont Ranking from Pairwisc Comparisons. In Sanjoy Dasgupta and David McAllester, editors, Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 109-117, Atlanta, Georgia, USA, June 2013. PMLR. URL httpss:/pproceedings mlr.
htt.:edddigg.......rrraa//aaaaaa wtpppp//////aaaeierrrerreeeeeeee hhmll Issue: 3.
Ammar Ammar and Devavrat Shah. Efficient rank aggregation using partial data. In Proceedings of the 12th PER.T/////RFRRACCRRRR Joint International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS '12, pages 355 366, New York, NY, USA, 2012. Association for Computing Machinery. ISBN 978-1-4503-1097-0 doi: 4554422255755622577799 URL hntps: htttts://toi....c //doi.or///
ACM
10. 1541//2254466... 2254799 event-place: London, England, UK.
M. Kariyappa, H. Raj, and A. Shah. Shapkefficient: Sampling algorithms for top-k shapley feature ranking.
In NeurIPS Workshop on Explainable AT, 2023.
Jacopo Teneggi and Jeremias Sulam. Testing Semantic Importance via Betting. In A. Glober- son, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Ad- vances in Neural Information Processing Systems, volume 37, pages 76450-76499 Curran As- sociates, Inc., 2024. URL Bac25555accchaate777777666664422
15

---

Published in Transactions on Machine Learning Research (12/2025)
Mona Azadkia and Sourav Chatterjee. A simple measure of conditional dependence. Annals of Statistics, 49 -000000-100, 2021. doi: 2
Mona Azadkia and Pouya Roudaki. A new measure of dependence: Integrated R², 2025. URL https:
18146 arXiv:2505.1144 [math.ST].
Jeremy Goldwasser, Will Fithian, and Giles Hooker. Gaussian Rank Verification, February 2025. URL
http:://axxi..rr//bbs///// 14142
arXiv:2501.1414 [stat].
Jean- Audibert, Sébastien Bubeck, and Rémi Munos. Best Arm Identification in Multi-Armed Bandits. Conference on Learning Theory, 2010.
Lijie Chen, Jian Li, and Mingda Qiao. Nearly Instance Optimal Sample Complexity Bounds for Top-k Arm Selection. In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 101 110. PMLR, April 2017. URL https://proceeddnngs mlr wwl.l..reesvvv///v/ http://////eeaaataaa...
Daniel Russo. Simple Bayesian Algorithms for Best Arm Identification. Operations Research, 68(6), 2020. URL
Yao Zhao, Connor Stephens, Csaba Szepesvari, and Kwang-Sung Jun. Revisiting Simple Regret: Fast Rates for Returning a Good Arm. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 42110 42158. PMLR, July 2023. http:///gg......pee//////////aa 2ha.23................ html
URL https::/pproceedingg .ml....ssm......m............... wttpppessss/////epppppa
Chun-Hurg Chen, Donghai He, Michael Fu, and Loo Hay Lec. Efficient Simulation Budget Allocation for Select- ing an Optimal Subset. INFORMS Journal on Computing, 20(4):579 595, 2008. doi: 10. 10. 11w.1//////.//33333 10s........0026666.. 02668. URL httpp:://bb...rr///..11///////// http://11..107////o..ooo 1268 _eprnn: https:/ http:///ooo..oo10112/55.33 /11.:1/1228///1.00000000000
Siyang Gao and Weiwei Chen. A note the subset selection for simulation optimization. In 2015 Winter Simalation Conference (WSC), pages 3768 3776, 2015. doi:
Wei You, Chao Qin, Zihao Wang, and Shuoguang Yang. Information-Divecae Selection for Top-Two Algorithms. In Gergely Neu and Lorenzo Rosasco, editors, Proceedings of Thirty Sixth Conference on Learning Theory, volume 195 of Proceedings of Machine Learning Research, pages 2850 2851. PMLR, July 2023. URL https://proceeddngss mlr. serrss/v195/yourrs html
Yi Liu and Veronika Ročková. Variable Selection Via Thompson Sampling. Journal of the
2021.192514.. American Statistical Association, 118(541):287-304, 2023. doi: 10.1080/01621459 010800011621455222119995444
URL https://doi http:///0oo..//1...// 1080/00221449 2021. 1928514
Publisher: Taylor & Francis _eprint:
http::///ho..ogg/000000000111112 00t.:/orgg10.00000001611111-1-10 https:
Aaron Fisher, Cynthia Rudin, and Francesca Dominici. All Models Wrong, but Many are Useful: Learning a Variable's Importance by Studying Entire Class of Prediction Models Simultaneously, December 2019.
URL http://arxiv.org ab//aa/ss....///////////////aaaa .04499 arXiv:1801.0144 [stat].
Ahmad Chamma, Bertrand Thirion, and Denis A. Engemann. Variable Importance in High-Dimensione Set- 10858 arXiv:2312.10858
tings Requires Grouping, December 2023a. URL [cs].
T. Hastie, R. Tibshirani, and J.H. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer series in statistics. Springer, 2009. ISBN 9780387848846. URL. https://books google. google.com/boone hotpc///ooooseeeeesssoo.......
Rory Mitchell, Joshua Cooper, Eibe Frank, and Geoffrey Holmes. Sampling Permutations for Shapley Value Estimation, February 2022. URL httl::tt:///// //arviv./ cetrxiv.org/bbll org/abs//////e.oooooo...oo. 2204. 12199 arXiv:2104.12199 [stat].
16

---

Published in Transactions on Machine Learning Research (12/2025)
Amirata Ghorbani and James Zou. Data Shapley: Equitable Valuation of Data for Machine Learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2242 2251. PMLR, June 2019. URL httpps://ppoceeednngg mmr. wet.smllr..rrss. wwwpppwwssse// http:///ggwoobr.....ooo html
Trevor Hastie, Robert Tibshirani, and Martin Wainwright. Statistical Learning with Sparsily: The Lasso and Generalizations. Chapman Hall/CRC, 2015. ISBN 1498712169.
Consumer Financial Protection Bureau. Equal credit opportunity act (regulation b), 12 c.f.r. part 1002. http//hhaaaeeepaatttto00
https: /typs://wwwx ecfr. wwwp///wweee/tttteeeeeeeeee wttptt//ttilee1//hhppaaae
2011. Official staff commentary to (1)) specifies that creditors must disclose the principal reasons for adverse action and that disclosing more than four reasons is not likely to be helpful to the applicant.
Ahmad Chamma, Denis A. Engemann, and Bertrand Thirion. Statistically valid variable importance assessment through conditional permutations. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2023b. Curran Associales Inc.
Gwladys Kelodjou, Laurence Rozé, Véronique Masson, Luis Galárraga, Romaric Gaudel, Maurice Tchuente, and Alexandre Termier. Shaping Up SHAP: Enhancing Stability through Layer- Wise Neighbor Selection, June 2024. URL
abs/2000.08529, 2020. URL http://wwww..rraaa///2/...11 12115 arXiv:2312.12115 [cs].
Tianyi Yao and Genevera I. Allen. Feature Selection for Huge Data via Minipatch Learning, ArXie,
wtwp:oooo///o///iee/ee Ct:CCopppDID2222333777 223953471
Zohar Karnin, Tomer Koren, and Oren Somekh. Almost Optimal Exploration in Multi-Armed Bandits. In Sanjoy Dasgupta and David McAllester, editors, Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 1238-1246, Atlanta, Georgia, USA, June 2013. PMLR. URL htttp:://proceeeeeee .ml. mml wtt.speee/////e/ei111l1.. .tml Issue: 3.
Kwang-Sin Sung Jun, Kevin Jamieson, Robert Nowak, and Xiaojin Zhu. Top Arm Identification in Multi-Armed Bandits with Batch Arm Pulls. In Arthur Gretton and Christian C. Robert, editors, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, volume 51 of Proceedings of Machine Learning Research, pages 139 148, Cadiz, Spain, May 2016. PMLR. URL https://proceeedngs mlr press/v61/junte htmll
Sotetsu Koyamada, Soichiro Nishimori, and Shin Ishii. A Batch Sequential Halving Algorithm without Performance Degradation, June 2024. URL http:://aardv..r////aa///0000000
arXiv:2406.00424 [es,
stat].
William Webber, Alistair Moffat, and Justin Zobel. A similarity mcasure for indefinite rankings. ACM Transactions on Information Systems, 28(4):1-38, November 2010. ISSN 1046-8188, 1558-2868. doi: 11155/111212211521111 URL https://d/ acm. wwtp:///g////..ooo//// 1tg//bl//0..1145/1522222.8 1852106.
Jordi Barretina, Giordano Caponigro, Nicolas Stransky, Kavitha Venkatesan, Adam A Margolin, Sungjoon Kim, Christopher J Wilson, Joseph Lehár, Gregory V Kryukov, Dmitriy Sonkin, et al. The cancer cell line encyclopedia enables predictive modelling of anticancer drug sensitivity. Nalure, 483(1391) :603 -073, 2012.
Tan Covert, Scott M Lundberg, and Su-Tn Lee. Understanding global feature contributions with ad- ditive importance measures. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 17212-17223. Cur- ran Associates, Inc., 2020. URL Paper. pdf.
Qun Wang, Yaqiong Li, Jiamei Li, Zhigang Yao, Xiaochun Ma, and Ji-wei Ma. Delta and notch-like epidermal growth factor-related receptor suppresses human glioma growth by inhibiting oncogene tor Journal of Cancer Research and Therapeutics, 18(5):1372-13 1379, 2022.
17

---

Published in Transactions on Machine Learning Research (12/2025)
Anna Vinitsky, Jason Chiang, Asim K Bag, Olivia Campagne, Clinton F Stewart, Paige Dunphy, Barry Shulkin, Qian Li, Tong Lin, Mary Ellen Hoehn, et al. Tgg-22. sj901: Phase i/ii evaluation of single agent mirdametinib (pd-0325901), a brain-penetrant m inhibitor, for the treatment of children, adolescents, and young adults with low-grade glioma (Igg). Neuro- Seuro-Oncology, 24:192, 2022.
Sangphil Oh, Sook Shin, and Ralf Janknecht. Etv1, 4 and 5: an oncogenic subfamily of cts transcription factors. Biochimica et Biophysica Acla (BBA)-Revieus Cancer, (8826(1)1111 12, 2012.
Annamaria Milillo, Francesca La Carpia, Stefano Costanzi, Vanessa D' D'urbano, Maurizio Martini, Paola Lanuti, Gisella Vischini, Luigi M Larocca, Marco Marchisio, Sebastiano Miscia, et al. A spry2 mutation leading to mapk/erk pathway inhibition is associated with an autosomal dominant form of iga nephropathy. European Journal of Human Genetics, (31)::166 1678, 2015.
Md Kamrul Hasan, Emanuela M Ghia, Laura Z Rassenti, Goorge F Widhopff and Thomas Kipps. Wnt5a enhances proliferation of chronic lymphocytic leukemia and erk1/2 phosphorylation via rorl/dock2- dependent mechanism. Leukemia, 35(6):1621-1633 2021.
Michael Ashburner, Catherine A Ball, Judith A Blake, David Botstein, Heather Butler, J Michael Cherry, Allan P Davis, Kara Dolinski, Selina S Dwight, Janan Eppig, et al. Gene ontology: tool for the unification of biology. Nature genetics, 25(1):25-29, 2000.
Suzi A Aleksander, James Balhoff, Seth Carbon, J Michael Cherry, Harold J Drabkin, Dustin Ebert, Marc Feuermann, Pascale Gaudet, Nomi L Harris, et al. The gene ontology knowledgebase in 2023. Genetics, 224(1): 2023.
Chao-Feng Zheng and Kun-Liang Guan. Activation of mek family kinases requires phosphorylation of two conserved ser/thr residues. The EMBO journal, 13(5):1123 1131, 1994.
Trond Aasen, Mare Mesnil, Christian C Naus, Paul D Lampe, and Dale W Laird. Gap junctions and cancer: communicating for 50 years. Nature Reviews Cancer, 16(12):775-788, 2016.
Maria Pia Protti and Lucia De Monte. Dual role of inflammasome adaptor ase in cancer. Frontiers in cell and developmental biology, 8:40, 2020.
Zhiwen Gu, Lei Wang, Qian Dong, Kaikun Xu, Jingnan Ye, Xianfeng Shao, Songpeng Yang, Cuixiu Lu, Cheng Chang, Yushan Hou, et al. Aberrant lyz expression in tumor cells serves as the potential biomarker and target for hec and promotes tumor progression via csgrp78. Proceedings of the National Academy of Sciences, 120(29):2211544412, 2023.
Aaron E Hoffman, Tongzhang Zheng, Yue Ba, and Yong Zhu. The circadian gene npas2, a putative tumor suppressor, is involved in dna damage response. Molecular Cancer Research, 6(9):1461 1468, 2008.
The Cancer Genome Atlas Network. Comprehensive molecular portraits of human breast tumours. Nature 490(7418):61-70, 2012. doi: 10. 00.1038/nature11441
Joel S. Parker, Michael Mullins, Maggie C.U. Cheang, Samuel Leung, David Vodue, Tammi Vickery, Sherri Davies, Christiane Fauron, Xiaping He, Zhiyuan Hu, John F. Quackenbush, Inge J. Stijleman, Juan Palazzo, J.S. Marron, Andrew B. Nobel, Elaine Mardis, Torsten O. Nielsen, Matthew J. Ellis, Charles M. Perou, and Philip S. Bernard. Supervised Risk Predictor of Breast Cancer Based Intrinsic Subtypes. Journal of Clivical Oncoloyy, 27(8):1160 1167, March 2009. ISSN 0732-183X, 1527-7755. doi: N0.1200/JCO.2008. 18. 1370. URL https::/ wrg//oi/100 1200 2000 18. 1370
Jessica O. Brett, Laura M. Spring, Aditya Bardia, and Seth A. Wander. ESR1 mutation as an emerging clinical biomarker in metastatic hormone receptor-positive breast cancer. Breast Cancer Research, 23(1):85, 2021. doi: 10.118//13305-0021-001022
Woiyi Toy, Yang Shen, Helen Won, Bradley Green, Rita A Sakr, Marie Will, Zhiqiang Li, Kinisha Gala, Sean Fanning, Tari A King, Clifford Hudis, David Chen, Tetiana Taran, Gabriel Hortobagyi, Geoffrey Greene, Michael Berger, José Baselga, and Sarat Chandarlapaty. ESR1 ligand-binding domain mutations
18

---

Published in Transactions on Machine Learning Research (12/2025)
in hormone-resistant breast cancer. Nature Genetics, 45(12):1439- -445, December 2013. ISSN 1061-4036,
1546-1718. doi: 10. 10.1038//g 2822. URL https://www/ https://www.naturee nature. com/articles/ng 2822
Y YucRice, Y Jin, B Han, Y Qu, Johnson, T Watanabe, L Cheng, Deng, H Tanaka, B Gao, Z Liu, Z Sun, S Bose, A E Giuliano, and X Cui. FOXC1 is involved in ER silencing by counteracting GATA3 binding and is implicated in endocrine resistance. Oncogene, 35(41):5400-5441 October 2016. ISSN 0950-9232, 1476-5594. doi: 00.1038//ncc.2001
URL
Carlos Marugán, Natalia Banz-Gómez, Beatriz Ortigosa, Ana Konfort-Vengutt, Cristina Bertinctti, Ana Teijo, Marta González, Alicia Alonso De La Vega, María José Lallena, Gema Moreno-BBenne and Guillermo De Cárcer. Tpx2 overexpression promotes sensitivity dasatinib in breast cancer by activating yap transcrip- tional signaling, Molecular Oncology, 18(6):1531 1551, June 2024. ISSN 1574-7891, 1878-0261. doi: 10.1002/ 3878-0261.13605 URL http:://feob..ommmmmliiaaaa
www.oonnnnnnnnann wwtpaarrr.llll........... www.w. wwt.:ooccccooocoocoooooooo 10011000000000010010000 13602
Xiaoyong Fu, Rinath Jeselsohn, Resel Pereira, Emporia F. Hollingsworth, Chad J. Creighton, Fugen Li, Martin Shea, Agostina Nardone, Carmine De Angelis, Laura M. Heiser, Pavana Anur, Nicholas Wang, Catherine S. Grasso, Paul T. Spellman, Obi L. Griffith, Anna Tsimelzon, Carolina Gutierrez, Shixia Huang. Dean P. Edwards, Meghana V. Trivedi, Mothaffar F. Rimawi, Dolores Jopez-Terrada. Susan G. Hilsenbeck, Joe W. Gray, Myles Brown, C. Kent Osborne, and Rachel Schiff. FOXA) A1 overexpression mediates endocrine resistance by altering the ER transcriptome and IL-8 expression in ER-positive breast cancer. Proceedings of the National Academy of Sciences, 113(43), October 2016. ISSN 0027-8424, 1091-6490. doi: 10.1073/pnaa http:/aaas.....1111111 1612835113. URL https: //ppss///aaaaaaa http:/paass.o/o//////
org /o// http////fff/l//o/oo/// wttp:///....0/ppa...... 111.11111/.....11111.1111. 1612835113
Jianing Tang, Mengxin Lu, Qiuxia Cui, Dan Zhang, Deguang Kong, Xing Liao, Jiangbo Ren, Yan Gong, and Gaosong Wu. Overexpression of ASPM, CDC20, and TTK Confer a Poorer Prognosis in Breast Cancer Identified by Gene Co-expression Network Analysis. Frontiers in Oncology, 9:310, April 2019. ISSN 2234-943X. doi: (2889/fonn.20199
URL http://wwww.rttirrrii.iin www.riirrsnnnnnrrrrrrreeoeo atgg/rticle//1.... (33.///////..33//ooooo.
Arvind Thakkar, Hemanth Raj, Ravishankar, Bhaskaran Muthuvelan, Arun Balakrishnan, and Muralidhara Padigaru. High Expression of Three-Gern Gene Signature Improves Prediction of Relapse- Free Survival in
Estrogen Receptor- Possiive and Nodo-Positive Brcast Tumors. Biomarker Insights, 10:BMI.330599, January 2015. doi: 10.4137/BAA S335555 URL http::/joourna/// sagepub. wag.epub.com/oo////...... 1tddddii1111 4137 SS00559
Jorrit M. Enserink and Pierre Chymkowitch. Cell cycle-dependent transcription: The cyclin dependent kinase Cdkl is a direct regulator of basal transcription machineries. International Journal of Molecular Sciences, 23(3) 2022. doi: 10.3390 ht0//jimm230012900
Anna Bergamaschi, Zeynep Madak-EErogan, Yu Jin Kim, Yoon-La. Choi, Hailing Lu, and Benita S. Katzenel- lenbogen. The forkhead transcription factor FOXMI promotes endocrine resistance and invasiveness in estrogen receptor- propptor-oosiive breast cancer by expansion of stem-like cancer cells. Breast Cancer Research, 16 (5):436, 2014. doi: 04,1186//13058--01--433--
Hosein Mouros-Mehr, Fuan M. Slorach, Mark D. Sternlicht, and Zena Werb. GATA-3 maintains the differentiation of the luminal cell fate in the mammary gland. Cell, 127(5))1041-1065 2006. doi: 10.1016/j. ccll.2006.09.044
Motoki Takaku, Sara A. Grimm, and Paul A. Wade. GATA3 in breast cancer: tumor suppressor or oncogenc?
Genes Eapression, 16(4):163 168, 2015. doi:
Qin Hu, Kewn Wang, Chuanrong Chen, Jian Ding, Yang Hc, and Zhaoning Ji. Ube2c promotes cell
cycle progression and suppresses dna damage-induced apoptosis in triple-negative breast cancer. DNA
Repaor, 154:103901, 2025. ISSN 1568-7864 doi: http:://00..oo//00000///cccoopp
URL
https://www www.sseceeeeeeett.coooooo/o wtt.://wwcc..tt...o.ccccccccttaa
Antonio Hurtado, Kayleigh A. Holmes, Caryn S. Ross-Inncs, David Schmidt, and Jason S. Carroll. FOXA1 is a key determinant of estrogen receptor function and endocrine response. Nature Genetics, 43(1):27-33, 10 doi: 10.1038/ng.7300.
19

---

Published in Transactions on Machine Learning Research (12/2025)
Bingchen Han, Neil A. Bhowmick, Ying Qu, Shin-Jung Chung, Armando E. Giuliano, and Xiaojiang Cui. FOXC1: an emerging marker and therapeutic target for cancer. Oncolarget, (699):069558-06333 2017.
doi: 10.18632/oncottarge.22777
Douglas Hanahan and Robert A Weinberg. Hallmarks of cancer: the next generation. cell, 144(5):646 674, 2011.
20

---

Published in Transactions on Machine Learning Research (12/2025)
A Proofs
A.1 Auxiliary Lemmas
Lemma 7. Let = Eliy] be the expected rank of feature j under the ranking procedure M, where the expeclation laken over minipatches that sumple feature j. Neal, define
min id' Mar
to be the smallest difference between the expected rank estimates across all features in [M]. Then under assumptions and we have
(--1)
[m-])
(2p 1)
M
Proof. First, observe that given the above assumptions, necessarily have My My whenever rj <TT
we have
H Elř,lj E F)
= Eli,li E F,J' F BWW.:///EEEE./////R//RRR/EE
Elfy EFj - www.://en--iibaaaeerereeeeree
Elfyli'
My
where the incquality follows from knowing that Ei E F $ F E F,J' F] by assumption that P(j' P(j & F 1 '' E F), and that Bl E F E(ryli.j' E F] by assumptions 2 and
since
P > Pybis'' E F) (E > - > fyl)
P > fylj.j E F)- (C+ F > h
fyl)
and rearranging gives the desired result. It follows that for any j.j' E [M] with rj <Tz
My H1 El E F| P(j E F 1 E + e F,j
= j F) Bi F F) P
F
Elřy ( P(j
= (Elfy P(y
P0; P(,)));)) PÚEEFij FF)
(Effy
(B[Fy - -flīy > [y). (2p C((-p)) P
>
(2p D) Flj F)
(m-1) 1) (2p 1) (M-1)
where the first. equality holds by the law of total conditional expectation, the first inequality holds by assumption 3 the second inequality holds by assumption 4 and the second-to-last inequality holds since the difference between estimated ranks is at least and that C O.
Lemma 8. For iteration & of algorithm 2 define the mándmum separation gap
A
min i
where u² is the expected rank estimate of feature j with respect to the feature pool C.. Under Assumptions we have
1
A, >22 2' (2p 1).
M
for any ileration t.
21

---

Published in Transactions on Machine Learning Research (12/2025)
Intuitively, Lemma 8 shows that trimming increases the expected gap between adjacent ranks geometrically across rounds, enabling finer discrimination among near-tied features with the same overall sampling budget.
Proof. At any iteration t, we know from lemma that
m - A4+1 > (2p 12p-1) 1). |Ct+1| - = -2p-11) 12p-1) 1).
m -
m 2 (2p 1) M/2
M -
A.2 Proof of Theorem 5
Proof. Let T = (n..... Tk denote the set of top-k features. In addition, for any feature E [M] let S₃ denote the set of minipatches where feature j is included. Next, for any given feature j we order the minipatches in Sj arbitrarily as ,,....,Ky Kn, where ny = =Syl and define the martingale difference sequence indexed by
X, = r3 E E F+-1
where Fa-1 = "K,i,....,,.... 11- We see that X, is Fg Fe-maasurable and
B[X, El Elr 1=-
We also trivially see that Xx, < m - 1. Then, by Azuma-Hocfdinn's incquality we can write
ngE² 2n, (m 1)2 = exp (- n,²²
P r4 2 c) = PMMag 2 nyc) exp
(2mmm 1)2
where the first equality holds by observing that
n2 Mny
X,
Eli [F,11] = - 11))
Next, since n,- ~ Binomial(B, where q = m/M, we have by the Chernoff bound that
Bq s exp(- Bq
P(n, 1
Then, by the law of total probability we can write
By
Bq
P(7, - n4 2 = P
Fj - 14j
2
f,
S
+ P
13 11 2 6, 2
2
2
Bg
Bg
< exp
+ +P
Fi - 2 n, 2
8
2
Bq
(- Bqe²
< exp
+ exp
8
4(m 1)2
As before, we define
A =
min
F3 PJ
M
and set e = A/2. By assumption
1 this quantity is non-zero and well-defined. Then we can write for some
top-k feature j and other feature j where ry < <T that
P(r) 2 r.y P f3 2 M, +c) PPry S My e).
We see that the probability of any true top-k feature not being ranked correctly is upper bounded by
Pon <P
( UU62)))
U
WET J'r,,,1
22

---

Published in Transactions on Machine Learning Research (12/2025)
By the union bound, we see want to find B such that
Bq
Bqe²
)
Per S Σ Σ
P(r, 2ny) 2kM
exp
+ exp
4(m 1)2 sd. 8.
JET 2'ry2
One way choose B is such that each of the two exponentials - 8/4KM. Solving for these two conditions separately gives
88
4kM
4 4(m - 1)3
4k:M
B 2 max
In
In
6
qe²
4(m- 1)2
4kM
In
qe
8
(M
1)2
4kM
In
(2p 1)2m
8
where 6 = A/2 and A 2 (2p-1)
m
by lemma
Choosing
(2p-1) - 1)2 yields the result.
16 (2p
M
A.3 Proof of Theorem 6
Proof. Let N₂ = M/2 denote the feature pool size at iteration 6, 9r = m/NN, 4 5,/2, and Dr be the features ranked in the bottom half iteration & to be discarded. We trivially see that Ty ry for any j' D and T. Furthermore, any top-k feature j at iteration t $ T 1 will survive onto the next round by being ranked in the upper-half of the feature pool. Then defining Fyy, the event that feature is incorrectly climinated at iteration t, we can write for any t = T 1 that
P(E SP
U (1
J'ED;
Σ P(r) 2 rz.)
JED
N
$ 2
(P(F) 2 nr + Et) + +P(Fy 1 -())
- Biqt
(- (- B.q.e
No
exp
+ exp
8
4(m 1)2
where the second inequality holds by the subadditivity of measure, and the last two (in)equalities hold by a similar argument found in appendix A.2 For each iteration T - 1, we see that the probability of error is upper bounded by
P S P(Eg))
Whereas for the last iteration T, we need to correctly rank all top-k features. Similarly following appendix A.2
we have
D S P & Σ Σ P(r Fy).
JET
JET jirryrry
Next, we want to control each term P S 8/2T and PT S 8/T such that the total probability of error is bounded within Perr 5/2 5/T. To achieve this, and again following a similar argument outlined in appendix A.2
can choose
B, 2 4(m - -1)² In
4KTN,
9:
23

---

Published in Transactions on Machine Learning Research (12/2025)
for t S T
and
4(m - 1))
ATTNN
Br 2
In
9:5t
8
Set A1 (2p M-1
Then following lemma g
can write BRAMPART
16(m - 1)²
4KTN:
BRAMPART
B
In
8
16(m
1)2 mAi
N
4KTN
In
(2
8
16(m- 1)²
T
M
4KTM
In
(t
1) In2
mdi
22(1-1)41-1
t=1
16(m - 1)²
4KTM
T
T MIn2
M In
In
mAi
5
23r-3
23r-3
1=1
16M (m
1)2
6
4KTM
In
ln2 2
mAi
5
10
16M 1)2
6
4k:(logg
(
logg (k) + 1)M
In
mAi
5
8
16M 1)²
6
4kM
In
m 5
where in the second-to-last inequality follows from knowing that T 2 3.
B Additional Simulation Results
In this section, we further validate RAMPART's effectiveness and our theoretical analysis by examining how the probability of exactly recovering the true top-k features scales with the total number of minipatches. We then assess robustness in higher dimensions (M 1000, 2000) and analyze the impact of minipatch size on ranking accuracy.
B.1 Theory Validation
We first verify our theoretical results stated Theorem by examining how the number of minipatches affects ranking performance of RAMP and RAMPART. We generate synthetic data with N 1000 observations and M 160 features drawn independently from a standard normal distribution X N I) with Y 0.22-X83+,, where e is unit Gaussian noise. The coefficient vector / is constructed to have four non-zero features with coefficients 4, 3, 2, 1. Since features are on the same scale, the magnitude of these coefficients directly determines the feature importance ordering, providing a clear ground truth for evaluating ranking performance. We compare RAMP and RAMPART with minipatch parameters n = 80 and m 20 (the trimming process also terminates when there are 20 features remaining). To ensure a fair comparison, we allocate a total budget of B minipatches across the five halving iterations for RAMPART, while using 5B minipatches for RAMP to match the total computation. We evaluate the empirical success probability averaged over 500 random trials, where success is defined exactly recovering the ranks of the top four features.
Figure 3 shows the empirical success probability as function of the total number of minipatches. In general, both methods demonstrate increasing accuracy with more minipatches, with RAMPART consistently achieves superior ranking accuracy compared to RAMP, validating the consistency guarantees of Theorems 5 and
While our theory shows RAMPART and RAMP require the same order of total minipatches, empirically RAMPART achieves better performance, particularly in the regime of 100-1000 minipatches where its adaptive resource allocation proves beneficial. This suggests that RAMPART's strategy of progressively
24

---

Published in Transactions on Machine Learning Research (12/2025)
Empirical Success Probubality
0.6
0.4
012
RAMP
RAMPART
0.0
0
200
400
803
900
1000
1200
1400
1600
1800
2000
Total Number of Minipatches
Figure Number of Minipatches vs Ranking Success Probability
focusing computation on promising features provides practical advantages beyond what our theoretical analysis captures. Bridging this gap to obtain sharper theoretical guarantces that better align with empirical performance remains an exciting direction for future work.
B.2 Higher Dimensions
For M 1000 and 2000, extend the setup described in Section by adding nonzero features while preserving the same cocfficient structure. We make two implementation adjustments to accommodate higher dimensionality: (1) For baseline FI, SHAP, and feature permulation with random forests, scale the nunber of trees (200 trees for A = 1000 and 400 trees for M = 2000) to ensure adequate feature space coverage; (2) For RAMPART and RAMP, we set = 4000 and B 20000 minipatchos respectivcly, while keeping all other parameters unchanged.
The results for M 1000 (Figures & demonstrate RAMPARI's robust performance across both covariance structures, with its advantage becoming particularly pronounced under autoregressive covariance. RAMPART's sequential halving strategy proves especially effective in these higher- dimensional settings, showing remarkable stability compared to M 500 while other methods exhibit noticeable degradation. At extreme dimensionality (M = 2000, Figures & 7 RAMPART continues to excel, maintaining strong performance even at high signal-to-noise ratios under autoregressive covariance, while competing methods show significant accuracy drops. These results highlight RAMPART's unique ability to handle both high dimensionality and complex fcature interactions through its adaptive resource allocation stratogy.
We also extend our ablation studies to higher dimensions using the same methodology Section 4.1 focusing the identity covariance setting and examining columns 2 and 5 (SNR 0.06 and SNR 0.5) across both classification and regression tasks. As before, we maintain consistent experimental conditions by using the same predictive models, data generation processes, and hyperparameter configurations across all settings. For regression tasks, we measure the mean squared error (MSE) rather than classification error. Figures 12 demonstrate that RAMPART maintains its performance advantage across all dimensions, affirming that RAMPART's adaptive resource allocation strategy remains valuable even in digees-dimmniiionll settings where feature ranking becomes more challenging.
25

---

Published in Transactions on Machine Learning Research (12/2025)
B.3 Impact of Minipatch Size
We also investigate the effect of varying feature subsample size within the RAMP framework, returning to the M = 500 setting with experimental conditions from Section 4 Figures 13 and 14 demonstrate that while performance remains stable across minipatch sizes for linear models under identity covariance, smaller minipatches (m 10) consistently outperform larger ones (m = 50) in nonlinear additive and correlated settings. These results suggest that smaller minipatch sizes may better capture local feature interactions while avoiding noise from irrelevant features. While we fix m 10 for RAMPART in this work, exploring adaptive minipatch sizes that scale with feature pool size across halving itcrations remains an interesting direction for fulure research.
C Additional Case Study Discussions
C.1 Drug Response Prediction
Response Variable. To quantify the PD-0325901 drug response in our case study, we use the area under
the dosc-response curve (AUC) the primary outcome of interest. The AUC is widely-used measure of overall drug sensitivity, defined as the area between the dose-response curve and 0 (Barretina et al. 2012). A higher value indicates that the drug was more effective at killing the cancer cells. We refer to Barretina et al. (212 for details on how this dose-response data was collected and processed.
Data Preproccssing of Gene Exprcssion Data. The raw CCLE data used in this case study can be downloaded from the DepMap Portal | ott/portalldommloall (version 18Q3). Due to the heavy right-skewed distríbution the RNASeq gene expression values, we log-transformed (log(x+1)) the raw gene expression data. We also restricted our analysis to the 1200 genes with the highest empirical variance across the cell lines and sequentially removed genes which had > 0.95 Pearson correlation with another gene in the dataset, resulting in 1104 genes. The processed gene expression data was finally standardized to have zero mean and unit variance.
Gene
RAMPART
RAMP
Baseline
SHAP
Permutation
TORAA
2
ETV4
2
2
5
1
9
SPRY2
7
5
2
GJB1
4
5
1
3
7
PYCARD
WNT5A
FERMTI
5
NRROS
LYZ
4
2
8
10
NPAS2
10
6
7
6
RP11-290L1.3
6
9
10
ITGA6
6
3
ID3
8
3
4
DUSP6
8
9
TNFRSF14
10
TMEM181A
RP11-284F21.0
8
Table 4: Top-10 ranked genes according to each feature importance ranking method for predicting the PD-0325901 drug response.
26

---

Published in Transactions on Machine Learning Research (12/2025)
C.2 Breast Cancer Subtype Classification
Response Variable. We used the PAM50 intrinsic sublype label as a five-class outcome, which are annotated as Luminal A, Luminal B, -ERe-ennrchedd Basal-like, or Normal-like (Parker et al. 2009).
Data Preprocessing of Gene Expression Data. We obtained the data from the TCGAbiolinke inks R package. Starting from the original gene expression data with 19, 947 genes, we log-transformed the expression values (log(1 1)) reduce the impact of highly skewed counts. For cach gene, we computed its empirical variance across all tumors and retained the M 5000 most variable genes as candidate features. The selected genes were then standardized to have mean and unit variance.
Gene
RAMPART
RAMP
Baseline
SHAP
Permutation
FSR1
3
FOXC1
2
1
10
TPX2
3
FOXA1
4
6
ASPM
5
MLPH
6
2
CDK1
7
FOXM1
8
GATA3
9
7
UBE2C
10
TFF3
4
TBC1D9
XBP1
8
KIF2C
9
PRR15
10
SPDEF
MYBL2
AGR3
DEGS2
SIDT1
10
ANLN
PLK1
CCNB1
TTYHI
KPNA2
SL.C2533
MCM2
Table Top-10 ranked genes according to each feature importance ranking method for breast cancer subtype classification.
27

---

Published in Transactions on Machine Learning Research (12/2025)
GO Biological Process
RAMPART
RAMP
SHAP
Permutation
Posilive regulation of transcription regulatory region DNA
100.0
100.0
100.0
binding
Prostate glandular acinus development
100.0
100.0
100.0
Prostate gland epithelium morphogenesis
100.0
100.0
Uterus development
100.0
100.0
Lung epithelial cell differentiation
100.0
Regulation of miRNA transcription
82.32
82.32
82.32
Regulation of epithelial to mesenchymal transition
59.37
59.37
59.37
Positive regulation of mitotic cycle
49.79
Mitotic cell cycle phase transition
42.00
Male gonad development
41.44
Stem cell differentiation
31.82
Developmental growth
17.55
Cell division
15.62
Tube morphogenesis
14.70
Cell population proliferation
13.63
Negative regulation of transcription by RNA polymerase II
10.20
10.20
Positive regulation of transcription by RNA polymerase II Regulation of cell cycle
9.78
9.59
Positive regulation of nucleobuse-containnng compound
6.16
metabolic process
Positive regulation of macromolecule metabolic process
5.25
Double-strand break repair via break-induced replication
100.0
Regulation of DNA-templated DNA replication initiation
100.0
PosiLive regulation of mitotic metaphase/ anaphase transition
100.0
DNA replication initiation
100.0
G2/M transition of mitotic cell cycle Mitotic nuclear division
100.0
33.21
Table 6: GO biological processes significantly enriched (FDR < 0.05) and fold change (capped at 100) in top-ranked genes across feature selection methods. The baseline top-10 genes had processes significantly enriched at this level.
28

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Additive Simulations
Logistic Regression
Neural Networks
Random Forests
Neural Networks
(Basline Fl: Coefficients)
(Baseline Fl: Integrated Gradients)
(Baseline Fl: MDI)
(Baseline Fl: Integrated Gradients)
idankity Covariance
Identity Covariance
kieniity Covariance
Iderkity Covariance
100
0.4
0.2
0.2
0.2
0.2
0.0
0.-
00-
0.0
0.00
0.06
0.30
0.20
050
0.03
0.05
0.10
0.00
0.50
243
0.08
0.10
0.20
0.50
0.00
0.06
030
0.20
0.0
Autoregressive Covariance
Autoregressivee
Autoregnassivee
Autoregressiv va Covar
0.8
0.-
0.1
0.-
1010
0.4
0.2
0.2
0.2
0.2
Deseline F1
SHAP
Permutation F1
+
RAMP tw/Besseine Fi]
RAMPART IWiBaseliie Fill
Figure 4: Ranking accuracy (RBO with p 0.7) for classification tasks (M = 1000)
Simulations
Nonlinear Additive Simulations
Ordinary Least Squares (Baseline Coefficients)
Neural Networks Integrated Gradients)
Random Forests
Neural Networks
(Baseline MDI)
(Baseline Integrated Gradients)
Identiy Coveriance
Idenitty Coveriance
Identity Coveriance
Idertity Covariance
5.5
0.5
0.6
0.6
0.5
0.6
0.-
0.2
0.2
0.0
0.-
0.0
0.03
0.06
0.10
0.20
0.50
0.03
0.05
0.10
0:10
0.20
0.50
0.53
0.10
0.20
0.50
Autoregresoive
Autoregressive
Autoregressive
Autoregress
0.6
0.5
0.6
0.6
0.4
0.1
0.1
0.1
0.2
0.2
0.2
SMK
5MA
+
Boseline F1
SHAP
Permutation F1
+
RAMP twiBaseine Fij
RAMPART (w/Baseline Fi)
Figure 5: Ranking accuracy (RBO with 0.7) for regression tasks = (M=10000
29

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Additive Simulations
Logístic Regression
Random
(Basline Fl: Coofficients)
(Baseline Fl: Integrated Gradients)
(Baseline Fl: MDI)
(Baseline Fl: Integrated Gradlents)
Identty Covariance
Identity Covoriance
Ideniity Covariance
Identity Covariance
0.0
0.4-
0.4
0.4
0.4-
0.
02
0.2
0.2
0.0
0.
00-
0.0
0.00
0.30
0.20
0.08
Autoregressive Covoriance
Autoregressive Coveriance
Autoregressive Covarfance
Autorogressive Covariance
0.6
0.6
0.-
0.6
0.5
0.5
0.3
0.3
0.3
0.2
0.2
0.2
0.2
Basoline #1
SHAP
Nermutation Fl
+
RAMP (w/Saseline Fi)
RAMPART Iw/Saseline Fi)
Figure 6: Ranking accuracy (RBO with p = 0.7) for classification tasks (M 2000)
Linear Simulations
Nonlinear Additive Simulations
Ordinary
Forests (Baseline Fl: MDI)
(Baseline Fi: Coefficients)
(Baseline Fl: Integrated Gradients)
(Baseline Fl: Integrated Gradients)
Identry
Identity Covariance
Identity Coveriance
Identity Covariance
0.5
0.5
0.6
0.5
0.5
0.5
010
0.4
0.2
0.0
0.0
80-
0.0
0.03
0.06
0.00
0.20
0.0
0.03
0.05
0.10
0.28
0.50
303
0.66
0:10
0.20
0.50
0.53
0.56
0.10
0.20
0.50
Autoregressive Covariance
Autoregressive Coveriance
Autoregressive Covarfence
Autoregressive Covoriance
0.8
0.5
0.5
0.5
0.6
0.6
0.6
0.6
0.4
Basoline Fl
SHAP
Pormutationn Fl +
RAMP (wiBasoline Fij
RAMPART Iw/Saseline FI)
Figure 7: Ranking accuracy (RBO with p 0.7) for regression tasks (M = 22000
30

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Additive Simulations
Ordinary Least Squares
Neural Networks
Random Forests
Neural Networks
(Baseline Fl: Coefficients)
(Baseline Fl: Integrated Gradients)
(Baseline Fl: MDI)
(Baseline Fl: Integrated Gradients)
3.-
3.-
3.0
5.2
2.5
2.5
2.5
2.5
Hon Mannaard
1.5
1.5
1.0
1.0
1.0
1.0
SMRR
SMRR=0.5
ShR-0.5
SHR-OU5
100
100
100
100
Nean
40-
42-
20-
20
20-
20-
Baseline H1
SHAP
Permutation F1
HUDMP (w/lSassringg [1]
HAMHMAIL [witieseline P1)
Figure& 8: Mean-Squas Squan- error vs. number of top-ranked features used as predictors in ablation simulation with identity covariance (M- -00)
Linear Simulations
Nonlinear Additive Simulations
Logistic Regression
Neural Networks
Random Forests (Baseline Fl: MDI)
Neural Networks
(Baseline Fl: Coefficiants)
[Baseline Fl: Integrated Gradients)
(Baseline Fl: Integrated Gradients)
SNR-0.06
SNR-0.05
SNR-0.06
SNR-0.0G
Cleatrcator Ever
0.40
0.40
0.40
0.43
0.90
0.50
0.50
0.52
0.25
0.25
0.25
0.23
S.I1-0.5
SAIC-
SNR-=0.5
S24R=0.5
0.43
0.45
0.45
0.83
0.00
0.40
0.40
0.03
0.35
0.25
0.25
0.33
Confecttin
0.30
0.5
0.25
0.35
325
0.20
0.70
0.20
0.20
6.15
0.15
0.15
0.15
0.10
0.10
0.10
Dazeline
+
SHAPP
Permutation Fl
RAMP Uw/Baseline Fil
RAMPART Iw/Beseline Fi)
Figure 9: Classification error number of top-ranked features used predictors in ablation simulation with identity covariance (M 1000)
31

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Additive Simulations
Ordinary Least Squares
Neural Networks
Random Forests
Neural Networks
(Baseline Fl: Coefficients)
(Baseline Fl: Integrated Gradients)
(Baseline Fl: MDI)
(Baseline Fl: Integrated Gradients)
Shit=0.06
25
20-
2.-
2.-
2.0
1.5
1.5
1.5
1.5
1.0
1.0
1.0
1.0
SMRR
SARR=0.5
ShR-O
SHR-OU5
100
100
100
100
Nean
20-
20-
22-
Baseline 21
SHAP
Permutation F1
HUDMP (w//Bassiing [1]
HAMHMAIL [witieseline P1]
Figure 10: Mean-Squared error vs. number of top-ranked features used as predictors in ablation simulation with identity covariance(M 1000)
Linear Simulations
Nonlinear Additive Simulations
Logistic Regression
Neural Networks
Random Forests (Baseline Fl: MDI)
Neural Networks
(Baseline Fl: Coefficiants)
[Baseline Fl: Integrated Gradients)
(Baseline Fl: Integrated Gradients)
SNR-0.06
SNR-0.05
SNR-0.06
SNR-0.0G
0.50
0.5
0.50
0.45
Clentrator Erer
0.90
0.40
0.40
0.03
0.39
0.35
0.35
0.35
0.23
0.2
0.25
0.23
S.I5-0.5
SAIC-
SKR-=0.5
0.43
0.45
0.45
0.85
0.42
0.40
0.40
0.40
0.35
0.25
0.35
0.31
0.30
0.30
0.30
0.30
0.5
0.
0.25
5.25
0.20
0.20
0.20
0.20
0.15
0.15
0.15
0.0
0.20
0.10
0.30
Dazeline
SHAP
Permutation Fl
RAMP Ew/Baseline Fil
RAMPART (w/Boseline Fi)
Figure 11: Classification error number of top-ranked features used as predictors in ablation simulation with identity covariance (M 2000)
32

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Additive Simulations
Ordinary Least Squares
Neural Networks (Baseline Integrated Gradients)
Random Forests
Neural Networks
(Baseline Fl: Coefficients)
(Baseline MDI)
(Baseline Integrated Gradients)
Skit=0.06 06
Skit=0.06
SNRtm0.06
SMR+0.00
3.0
3.0
2.-
2.3
Hoan
1.5
120
100
100
102
Nean Ever
60-
60-
02-
02-
40-
40-
40-
42-
20
20
20-
20-
Nimbbr of 70p Ranoed Pestures
Ni: Kimbbr Top Ranoad Pastures
11. Nimbbrr Top Rauoord Peaturmss
Nimber Top Rayttrd Featurnes
Baseline #1
SHAP
Permutation 21
KADMP (w//Baseel !1]
HAMIMANL [witisseliee
Figure 12: Mean-Squared error vs, number of top-ranked features used as predictors in ablation simulation with identity covariance (M = 2000)
Linear Simulations
Nonlinear Additive Simulations
Logistic Regression
Neural Networks Idendity
Random Forests
Neural Networks
Identtly Covariance
Icentiy Covariance
Identity Coverience
0.5
08
0.5
0.6
06
0.6
0.-
0.4
0.1
0.1
0.4-
0.-
03
0.7
02-
0.0
50
0.0
0.0
0.03
0.66
0.10
0.20
0.50
803
0.00
0.15
0.20
0.50
0.05
0.05
0.30
0.20
0.30
0.03
0.30
0.10
0.20
0.50
Autoregressive Covarlance
Autoregressive Covariance
Autoregressive Coveriance
Autorogressive Covarlance
0.5
0.5
0.4
0.4
0.4
0.5
0.5
0.2
0.2
0.2
0.2
0.3
0.1
0.1
0.1
0.0
3.0
0.0
0.0
0.03
0.66
0.10
0.20
0.50
8.03
0.00
0.10
0.20
0.50
0.03
0.05
0.30
0.20
0.30
0.03
0.20
0.10
0.20
0.50
Figure 13: Effect (RBO with p = 0.7) of minipatch size for classification (M- = 500)
33

---

Published in Transactions on Machine Learning Research (12/2025)
Linear Simulations
Nonlinear Additive Simulations
Ordinary Least Squares
Networks
Random
Identiy Covariance
Identity Covariance
icentiy Covariance
Identity Covariance
0.8
0.
0.5
08
0.6
0.6
0.4
0.2
52
0.2
0.2
0.20
Autorogressive Covariance
Autoregressvve Covariance
Autorcaresdive Coveriance
Autorogresoive Covarfance
0.7
0.7
0.7
0.7
0.0
0.4
0.4
0.4
0.4
880
0.3
0.1
0.1
0.1
0.1
0.0
248
m 10
-4
m= 20
50
Figure 14: Effect (RBO with p 0.7) of minipatch size for regression (M 500)
34


Reviews:
[{"review_id": "c0KbspfKZT", "reviewer": "Reviewer_GoeQ", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces RAMPAR, a novel framework for accurately ranking the top-k most important input features. The method combines a minipatch ensemble approach with an adaptive sequential halving strategy to focus computational resources. The authors provide rigorous theoretical guarantees and demonstrate the method's superiority through simulations and a case study using a cancer genomics dataset.\n\nStrengths:\n\n1.\tThe method is clearly designed for the specific goal of top-k ranking, effectively leveraging the task's inherent properties. By combining ensembling and adaptive resource allocation, the framework is demonstrably more robust and yields consistently better performance than standard feature importance methods across diverse simulation settings.\n\n2.\tThe article contains substantial and rigorous theoretical derivation and analysis. The authors provide explicit sample complexity bounds and demonstrate that RAMPART achieves the correct ranking with a higher probability than RAMP while maintaining the same order of total computational complexity.\n\n3.\tThe experimental results, particularly the Rank-Biased Overlap (RBO) scores in the simulations and the biological validation in the genomics case study, effectively demonstrate the method's superior performance compared to strong baselines.\n\nWeaknesses:\n\n1.\tThe theoretical foundation relies on several data assumptions, such as Unbiased Ordering (Assumption 3), which may not hold in real-world scenarios characterized by complex, non-additive feature interactions (e.g., features that only contribute when combined). Given that the practical generalization is only tested on a single high-dimensional genomics case study, the method's broad applicability and stability across diverse real-world datasets remain uncertain.\n\n2.\tThe paper strongly argues that RAMPART‘s recursive trimming optimizes resource usage by avoiding the costly estimation of low-rank features. However, the reported runtime in the Section 4.2 case study does not fully support this claim. Traditional methods like Random Forest (Baseline FI) and TreeSHAP are executed in 6–13 seconds, which is faster than the RAMPART method (18–19 seconds). This discrepancy casts doubt on the practical significance of the claimed efficiency gains associated with avoiding low-rank feature calculations.", "claims_and_evidence_yn": "Yes", "claims_explanation": "See my strengths of summary.", "audience_yn": "Yes", "audience_explanation": "The topic is important, and the proposed method is interesting and supported by rigorous theoretical analysis.", "requested_changes": "More explanation of the resource consumption is needed. Additional experiments on more practical datasets would strengthen the generalization of the proposed method.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5vvDinx2AF", "reviewer": "Reviewer_3zmm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces RAMPART (Ranked Attributions with MiniPatches And Recursive Trimming), a new framework designed specifically for the task of accurately and efficiently ranking the top-k most important features in a dataset. \n\nThe authors argue that traditional methods, which first calculate importance scores for all features and then rank them, are inefficient and unstable in high-dimensional settings. RAMPART addresses this with a two-part strategy. First, it uses an ensembling technique called RAMP, which trains models on random subsamples (\"minipatches\") of both observations and features to produce more stable rank estimates. Second, it incorporates this into a \"recursive trimming\" process, which iteratively discards the least promising half of the features, progressively focusing computational resources on the most likely top candidates. Through theoretical analysis, simulations, and a cancer genomics case study, the paper demonstrates that RAMPART outperforms popular feature importance methods like SHAP and permutation importance, providing more accurate and meaningful rankings.\n\nI think one of the key strengths is the strong validation. The authors support their claims with a comprehensive evaluation, including theoretical guarantees for recovering the correct ranking, extensive simulation studies, and a compelling real-world genomics case study where RAMPART's findings were validated with established biological knowledge.\n\nOn the other hand, it's still dependent on base importance metric. RAMPART is a meta-algorithm that improves the ranking process for any given feature importance measure. However, its final output is still dependent on the choice of this base metric (e.g., MDI, Integrated Gradients), and it does not resolve the fundamental debate over which measure best represents \"true\" importance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "I think this paper mainly has two claims:\n1) RAMPART is more accurate for top-k feature ranking than existing methods.\n2) RAMPART's design (minipatch ensembling + recursive trimming) is what drives its performance.\n\n\nFor 1), this paper provides extensive simulation studies (Section 4.1, Figures 1, 4-7) across a wide variety of settings. In these controlled experiments where the ground truth is known, RAMPART consistently achieves the highest ranking accuracy (measured by Rank-Biased Overlap), especially in more challenging scenarios with correlated features and higher dimensions. \n\n\nFor 2), the authors cleverly isolate the effects of the two components. By comparing RAMPART to RAMP (which uses minipatches but not trimming), the results show that while RAMP improves over baseline methods, RAMPART's adaptive trimming provides an additional performance boost. This demonstrates that the progressive focusing of computational resources is a key contributor to its success.", "audience_yn": "Yes", "audience_explanation": "I think explainable AI researchers will be interested in this submission. This submission addresses a practical problem of ranking important features, which is a more specific and often more useful task than just assigning importance scores.", "requested_changes": "Overall, I think this paper is in good shape.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "gaNwPD2WD1", "reviewer": "Reviewer_YJRp", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces  general framework for top-k feature importance ranking RAMPART.\n\nPRos:\n- It identifies the limitation of existing feature importance approaches that only compute scores and sort them post-hoc, emphasizing the need for directly optimizing ranking accuracy for the top-k most important features.\n- It proposes a minipatch-based ensembling method that aggregates importance rankings obtained from random subsamples of observations and features to reduce variance and feature correlation bias.\n- It extends RAMP into RAMPART, which can narrows the candidate feature set, allocating computational resources adaptively to promising features.\n- The experimental section is fairly comprehensive and provides adequate evidence to support the paper’s claims.\n\nCons:\n- The technical motivation could be better articulated in introduction. The introduction moves quickly from the problem to the proposed solution, without clarifying the rationale for adopting this specific technique.\n- The paper lacks illustrative examples. Including a few examples would help readers better understand the proposed techniques.\n- The discussion section could be expanded to include limitations or potential extensions of the method. Or you should transform the section title into conclusion.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The theoretical analysis is sound, and the experiments are sufficient to support the main claims.", "audience_yn": "Yes", "audience_explanation": "The topic of feature importance ranking is relevant to researchers in interpretable machine learning and model explainability, which are active areas within TMLR’s audience.", "requested_changes": "- The introduction could better explain the reasoning behind the chosen approach. Providing a brief discussion of why this particular framework or design was selected would make the paper more accessible and logically coherent.\n- Consider adding one or two concrete examples to help readers intuitively understand how the proposed method works and what specific problems it addresses.\n- The discussion could be expanded to include key limitations or possible extensions of the method. Alternatively, if no further analysis is added, it may be clearer to rename this part as a conclusion section.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)