Testing Most Influential Sets
Authors: Lucas Darius Konrad
Nikolas Kuschnig
Venue: ICLR 2026 Poster (2026)
Keywords: attribution
robustness auditing
causal inference
fairness
least squares
extreme value
Primary area: interpretability and explainable AI
TL;DR: Theoretical foundations & procedures to test most infuential sets for excessive influence

Abstract: Small influential data subsets can dramatically impact model conclusions, with a few data points overturning key findings. While recent work identifies these most influential sets, there is no formal way to tell when maximum influence is excessive rather than expected under natural random sampling variation. We address this gap by developing a principled framework for most influential sets. Focusing on linear least-squares, we derive a convenient exact influence formula and identify the extreme value distributions of maximal influence – the heavy-tailed Fréchet for constant-size sets and heavy-tailed data, and the well-behaved Gumbel for growing sets or light tails. This allows us to conduct rigorous hypothesis tests for excessive influence. We demonstrate through applications across economics, biology, and machine learning benchmarks, resolving contested findings and replacing ad-hoc heuristics with rigorous inference.

TESTING MOST INFLUENTIAL SETS
Lucas D. Konrad
Nikolas Kuschnig"
Vienna University of Economics and Business 1020 Vienna, Austria
Monash University
3 5145 Caulfield, Australia
lucas. www.adenr.u.ccc.coo. ac
nikola wwwchhiggmonash..c.oommm
ABSTRACT
Small influential data subsets can dramatically impact model conclusions, with a few data points overturning key findings. While recent work identifies these most influential sets, there is no formal way to tell when maximum influence is excessive rather than expected under natural random sampling variation. We address this gap by developing a principled framework for most influential sets. Focusing on linear least-squares,
derive convenient exact influence formula and identify the extreme value distributions of maximal influence - the heavy-tailed Fréchet for constant-sizz sets and heavy-tailed data, and the well-behaved Gumbel for growing sets or light tails. This allows us to conduct rigorous hypothesis tests for excessive influence. We demonstrate through applications across economics, biology, and machine learning benchmarks, resolving contested findings and replacing ad-hoc heuristics with rigorous inference.
INTRODUCTION
Machine learning (ML) models and statistical inferences can be highly sensitive to small subsets of data. In many applications. just handful of samples can overturn key conclusions: two countries nullify the estimated effect of geography on development (Kuschnig et al., 2021), a single outlier flips the sign of treatment effect (Broderick et al., 2023), a small group of individuals drives disparate outcomes in algorithmic decision-making (Black & Fredrikson, 2021). These most inftuential sets lets-dal data subsets with the greatest influence on model predictions are central questions of interprtability, fairness, and robustness in modern machine learning (see, e.g.. Black & Fredrikson, 2021: Chen al., 2018; Chhabra et al., 2023; Ghorbani & Zou, 2019; Sattigeri et al., 2022).
Despite their practical importance, practitioners lack principled tools to assess whether set's influence is genuinely problematic. Current practice relies on heuristics, ad-hoc sensitivity checks, and domain expertise, while approximate methods such as influence functions (Koh & Liang, 2017; Fisher et al., 2023; Schioppa et al., 2023) systematically underestimate the impacts of sets and extreme cases (Basu et al., 2020; Koh al., 2019). Recent work highlights both the promises and challenges of most influential subsets - small sets can drive results even in randomized trials (Broderick et al., 2023; Kuschnig et al., 2021), heuristic algorithms can fail in simple settings (Hu et al., 2024; Huang et al., 2025), and influence bounds remain an active area of research (Moitra & Rohatgi, 2023; Freund & Hopkins, 2023; Rubinstein & Hopkins, 2025). What remains missing is principled method to distinguish natural sampling variation from genuinely excessive influence.
We develop statistical framework for assessing the significance of most influential sets. By focusing on linear regression - a tractable, interpretable, and widely-used setting that underlies many modern methods we derive the exact asymptotic distributions of maximal influence. We show that two distinct regimes emerge depending on the size of the influential set: when the size fixed, maximal influence converges to heavy-tailed Fréchet distribution; when the size grows with the sample, maximal influence converges to a behl-behaved Gumbel distribution. Our results enable principled hypothesis tests for excessive influence, replacing ad-hoc diagnostics with rigorous statistical procedures. We demonstrate their practical value via applications across economics, biology, and machine learning benchmarks, resolving ambiguous cases where influential sets drive contested findings.
Authhrrs listed in alphahetical order and contributed equally.

---

Contributions. We present comprchensive analysis of the influence of most influential sets, both theoretically and in practical applications. Our main contributions are:
1. Theoretical foundations. We derive distributions for the influence of most influential sets, establishing their extreme value behavior and enabling statistical testing.
2. Efficient implementation. We provide closed-form formula for evaluating set influence, making our approach practical for real-world applications.
3. Empirical validation. We demonstrate the utility of our framework across domains, re- solving the contested 'Blessing of Bad Geography' in economics, assessing robustness in biological data of sparrow morphology, and auditing fairness in ML benchmark datasets.
To summarize - we provide the first rigorous theoretical results that allow us interpret influence, and demonstrate practicality by resolving contested findings.
Outline. The remainder of the paper is structured as follows. Section introduces the problem of most influential sets and formalizes the setting. Section 3 presents our theoretical results on the distribution of maximal influence. Section demonstrates the practical merits our framework through simulations and empirical applications. Section 5 discusses implications, limitations, and future directions, and Section 6 concludes.
PRELIMINARIES AND BACKGROUND
Practitioners routinely encounter situations where small subsets of data points drive key conclusions. Consider the following scenarios:
Scientific discovery:
Rugged terrain generally hinders economic development, but not in Africa. What if this striking result driven by just two small island nations?
Fairness auditing: An algorithmic decision-making system produces different outcomes for a protected group. What if the disparity can be explained by only a handful of data points?
Data cleaning: single influential point among a thousand samples flips a strong correlatoo to a null result. Should we trust the original finding or the one without the outlier?
Data preprocessing: A microcredit experiment shows negligible outcome variations overall, except for few outliers. How should we prepare and analyze the sample?
At the core of these examples lie most infiuntial sets, which exert disproportionate influence an estimate or prediction. These sets are intuitive to interpret, directly tied to the quantity of interest, and provide new dimension for assessing estimates by highlighting their support in the data.
What has been unclear, however, is how to interpret and deal with influential sets. Existing influence methods quantify how much changes, but offer little guidance interpretation. Current practice relies heavily on domain expertise and ad-hoc rules, lacking statistically rigorous framework for judging influence. Heuristics, such as sign-flips or significance thresholds, can flag effects that are stable and miss genuinely problematic influence. We address this gap by quantifying whether observed maximum influence statistically compatible with natural sampling variation.
2.1 FORMAL PROBLEM STATEMENT
We consider supervised learning task with input space X' C RR. and target space y C RR. The goal is to learn function f(..) X + ) parameterized by 0 E R Given training data {In vn)) W
and a loss function L (..,), we learn parameters by solving
θ = arg min Σ (f)().
BERG ==1
Let [N] = {l..... N} and denote both an index set and its corresponding subsample as S C [N]. For any subset S, we use a subscript 0-g is to denote an estimate without S, i.e.
arg min Σ (5 (.))
BERR
ng
2

---

Definition (Most Influential Set). For positive integer & < N, the k-most influential set is
arg argmax max A (S;
SC
where A ( (((0))((f_s)) is the influence of subset & on the scalar target function Φ RRQ - I. denote the maximum inftuence as A maax A (Smmax Φ)
Research Question. What is the probability distribution of max and how can we distinguish excessive influence from natural sampling variation?
2.2 INFLUENCE FUNCTIONS VS. EXACT INFLUENCE
common and related approach to study influence is via influence functions (Fisher al., 2023; Hu et al., 2024; Koh & Liang, 2017). These are motivated by reweighing via the perturbation
((::3)
arg min
Σ Cf(0, 2.). + Σ (()(..))).))) MERQ N
1
SEB
Setting < = recovers 8, while c = -N yields p_g. The influence function is the first-order linear approximation 0.
While influence functions are computationally convenient, they are unreliable even for simple models (Basu et al., 2020; Hu al., 2024; Huang et al., 2025; Koh al., 2019). In particular, they systematically underestimate the impact of (a) sets of data points and (b) highly inftuential data points. This occurs because the first-order approximation cannot reflect higher-order effects from the interplay between data points or differential leverage scores.
Exact Maximum Influence. We therefore derive exact influence formula (in Section 3) and characterize the behavior of maximum influence (Section 3.1) to enable principled testing (Sec- tion 3.3). We focus on the tractable but ubiquitous linear setting, allowing to accurately portray most influential sets, for which extreme behavior dominates and first-order approximations fail most dramatically.
2.2.1 EXTREME VALUE THEORY
Our goal to characterize the behavior of max the influence of the most influential set. This quantity is defined through maximization over all possible subsets, and its distribution is governed by extreme value theory rather than classical asymptotics. See Figure for an illustration. When taking maxima random quantities, three possible limiting distributions emerge (Fisher & Tippett, 1928; Gmedenko, 1943): the well-behaved Gumbel (Type I), the heavy-tailed Fréchet (Type II), and the bounded Weibull (Type III).
Ai
max A
120
Observations
Observations
100
N 100
N 50
N N=50 50
60
N 25
18
N 25
0.0
0.1
0.2
0.3
0.0
0.1
0.2
0.3
Figure Illustration of the distribution of A ((()) for general observations, and the maximum influence A max over all possible i, for N E [25. 50, 100}. One can clearly discern an upward shift and substantial increase the density the tails.
3

---

We need to determine which extreme value distribution (EVD) attracts A A Specifically, distinguish between the Gumbel distribution with exponential tails, and the Fréchet distribution with polynomial tails allowing for potentially arbitrarily large influence. (The Weibull distribution can be ruled out since influence is unbounded.) Once we establish the asymptotic distribution, we show how it applies to finite samples.
2.3 SETTING
Consider the standard linear regression model, where ʃ is a linear function relating random features X the outcome Y via the parameter vector 0. The ordinary least-squares (OLS) population estimator of 0 is
1 = E[XX']
yielding fitted values y = X'O and the associated residuals R y - X'O. Stacking N observed training samples yields the design matrix X Nx and outcome vector y E TR For OL.S, we assume that X'X is invertible and remains after removing any subset." The sample OLS estimator then
i arg min X0 = (X'x)-
yielding predictions ý X0 and residuals r y.
For our main results, we consider least-squares estimates the regression coefficients, with and without penalization. For illustration, however, we will consider univariate model with one positive coefficient interest and target function ((0) = 01. This cases notation considerably and comes without loss of generality theoretical results apply to more general settings."
3 PROPOSED APPROACH
The influence single observation well-known (Belsley et al., 1980; Cook, 1979; Walker & Birch, 1988) to be
A ((i)) X (XXX) X
h;
where h; is the leverage score and the residual of observation i.
We extend this result be (i) applicable sets observations, and (ii) analytically convenient.
Proposition The influence of some set S the least-squares estimator Ô is
A where X > 0 is an optional penalization parameter.
= = x AIp) -1 X
(1)
Proof. Partition the full normal equations into S and -,, and subtract X XgX00
X x AI X'y
X A1) X'sy-s XLys
X'
x X 5 AI)Ō= - -X'sx- + X'(yss - Xs O).
The key element stems from r3 ys normal equations and inverting:
Xg 0. and the result follows from subtracting the leave-out
X s + 11 X x's + X
XX
X
X
+ AI)
( (0- X
A(S) (X' XX_ + -X,r
thank an anonymous reviewer pointing us towards this elegant proof.
'This is not necessary for results, they extend ridge regression, where the penalization parameter À > ( X'X + AI creates ridge that guarantees invertibility.
MMuttppee features can be factored out via the Frisch- Waugh- theorem under mild assumptions. leaving equivalent univariate regression. A different sign can be accommodated by simply sign-fiippng the feature of interest adjusting the target function ø accordingly.
4

---

Proposition 1 reveals the additive structure of individual contributions in the numerator, as well as the multiplicative adjustment from the denominator. provides an exact closed-form expression that avoids re- fitting the model for each candidate subset.
3.1 EXTREME VALUE DISTRIBUTIONS
We now turn to finding the distribution of (SS) for the most influential set, Sgax, Since this quantity defined by an extremal operation (maximization over all possible subsets), its asymptotic behavior is governed by extreme value theory. Specifically, we seek the limiting EVD H such that A max MDA (H) i.e., A max lies in the maximum domain of attraction of H.
Two canonical EVDs are of particular interest: variables, and the Gumbel (Type I) distribution A for light-tailed variables. We distinguish two practically relevant regimes based on how the subset size scales with sample size N:
the Fréchet (Type II) distribution 2 for heavy-tailed
Constant-size sets: remains fixed N 20.
2. Growing-size sets: k grows N + and k/N -
Both regimes have been considered in practical applications (see, e.g. Broderick et al., 2023; Kuschnig al., 2021), and as will show next they yield fundamentally different asymptotic behavior with important implications for the interpretation of influence.
3.1.1 CONSTANT-SIZE SETS
Theorem (EVD for constant-sin size sets). Suppose E [X3] < oc, and that the heavier tail of Xs; R decays al polynomial speed with coefficients E,, where min co. [sg [Smax] remains constant as - xo, then
A Amax Fréchet(a, b,E), N-000
lim
with location parameter a, scale parameter and shape parameter min(
Proof sketch. Let C' == Eies X,R: and D D= D m= X Notice that C and D are asymptotically independent. Since X; and R have polynomial tails with coefficients &n, &r, their product C MDA( with & = min(fz, {r}, since its upper tail behaves like the tail of max(X,R) for i E Smax Lemma shows that the inverse sum D MDA(A), and the product CD_ inherits the Fréchet behavior from C by Lemma 2.
This result shows that, for constant-size sets, A max inherits tail behavior from the heavier tail of R and X. one of them is sufficiently heavy-tailed, even small sets can exert extreme influence with non-negligible probability. Corollary simplifies Theorem 1 in absence of heavy tails.
Corollary 1. If the tail coefficients of both X₁ and Rd are infinite, then
lim A max Gumbel(a, b). N-000
3.2 GROWING SETS
When the most influential set grows 'slow enough' with the sample size, the central limit theorem (CLT) dominates the asymptotic behavior:
Theorem 2 (EVD for growing sets). If (X,, Bn N satisfies the conditions of a CLT and S
grows at o(N) but faster than O(1), then
lim
max Gumbel(a, b).
+00
By assumption, m. -+ oo and m. = o(N) as N xc. The numer- ator, C, can be written as a partial sum of my terms and grows at rate O(m)). Since {X, R, NN satisfies the conditions of CLT, we obtain (C B(CT) / /mrx ب NWu,, o²) as N + oc. Hence, following Lemma and Corollary 2, the product CD lies in the maximum domain of attraction of the Gumbel distribution.
Proof sketch. Let m = |
5

---

This reveals fundamental distinction: constant-size sets dominated by the heaviest tail, while, for growing sets, A max converges to a well behaved Gumbel distribution with exponentially decaying tails. This result holds regardless of the underlying distributions of X and R. as long the variance of X; Rg is finite.
3.3 IMPLEMENTATION
With the theoretical results established, turn towards practical implementation. Assuming there is a most influential set of interest, our procedure follows three steps:
1. Choose the EVD family, Our theoretical results guide the decision between the Gumbel and Fréchet families, which based on the hypothesized set size and the tail behavior of X and R. For the latter, can estimate tail coefficients using maximum likelihood estimation (MLE; Smith, 1985; Bücher & Segers, 2017). If 1/ is sufficiently close to zero, we default to the Gumbel distribution (per Corollary 1 and Theorem 2). Otherwise, distribution with shape parameter: e, following Theorem 1.a
the Fréchet
2. Estimate EVD parameters. With the EVD known, we estimate its location and scale parameters a, b c.g., using the block maxima method (Coles, 2002: de Haan & Ferreira, 2006). For this, we divide the sample (excluding S for robustness) into M blocks of size N/M, compute A max for each block, and use MLE based on these draws. Since selecting the maximum out of N/M observations reduces the expected maximum compared to the full sample, bias correction can be applied for the Gumbel distribution, leveraging that the densities for sizes N and N/M are related by
F (a)) 3 Gumbel(a, b) and [FN/M(2) (2:) 4 (umbella,
which yields the location correction = + b bogg(M), where is the MLE.
3. Perform hypothesis test. Finally, test the null hypothesis Ho that the observed influence
reflects natural sampling variation against the alternative H1 of excessive influence. Based the estimated parameters, we can simply compute the p-value as P(A max 8ebs) where Jabs is the observed maximum influence.
Computation. Thanks to Proposition 1, our procedure is practical and computationally convenient, allowing for application to large and varied datasets. While accurately estimating Extreme Value Distributions (EVDs) is recognized and separate challenge in its right, the maximum likelihood steps within our framework are simple and well-behaved, optimizing over only two parameters in the Gumbel case. The primary computational constraint instead stems from finding most influential sets we need to approximate A max for the M block maxima estimates. For computational tractability, we use an adaptive greedy algorithm (Hu et al.. 2024; Kuschnig al., 2021) with complexity (MM) and an efficient implementation based on our closed-fome influence formula for sets.
4 EXPERIMENTS
In this section, we validate our theoretical predictions, investigate convergence in small samples, and demonstrate their practical relevance and utility through real-world applications spanning economics, biology, and machine learning.
4.1 SIMULATION STUDY
We begin with controlled setting, where we (i) illustrate, (ii) investigate convergence for small samples, and (iii) evaluate empirical estimation.
"This set can be obtained with any of the methods in the literature (Broderick et al., 2023; Kuschnig al., 2021; Freund & Hopkins, 2023), and could, e.g., be the smallest set that achieves a sign-flip commonly considered heuristic cutoff. If no such heuristic is used and multiple tests are considered (c.g., for different coefficients or sets sizes), a multiple testing correction should be applied. Note that EVT controls the implicit search for the most influential over the (?) possible subsets.
'Small values of correspond to extremely heavy-tailed distributions where the variance (5 & 2) or even the mean (f & 1) become infinite. Such cases pose practical challenges for statistical inference, and imply that arbitrarily large influence possible case.

---

Illustration. Figure illustrates approach on simple linear regression with moderately influential point due to high leverage. Panel A visualizes the data, significance thresholds (at the 10, 5, and 1 1% significance levels) function of predictor and response values. Panel B presents the underlying extreme value analysis: block maxima inform the estimated Gumbel distribution, yielding p-value of 0.04 for the observation of interest.
(a) Effect an Influential Observation
(b) Extreme Value Analysis
maxima
Corrected Gumbel
Plain Cumbel
Density
Influencial Observation
20
Observation p 0.040
-20-
-40
without infuenial Fit with influendal 90, 95, 99% thresholds
X
0.00 0.01
0.02
4
0.04
0.05
0.06
Figure 2: Illustration of our methodology on simple linear regression with a moderately influential observation. Panel A depicts observations, estimated regression lines with and without the influential point, and conditional significance regions at the 10, 5, and 1% levels (dotted lines). Panel illustrates the extreme value analysis: histogram of block maxima in the background, fitted Gumbel distributions with (solid) and without (dashed) bias correction, and the resulting p-value for the observation of interest.
4.1.1 CONVERGENCE TO EXTREME VALUE DISTRIBUTIONS
Next, we verify that maximal influence converges to the predicted extreme value distributions. We consider four scenarios based combinations of the standard Normal and t(5) distributions for X, R. For each scenario, we simulate 1, 000 datasets of sizes between N = 20 and N 1000, and compare the empirical parameter estimates with the theoretical prediction. Overall, we find rapid convergence, implying that our theoretical predictions are applicable with small samples.
Figure 3 shows convergence of the scenarios to the predictions, which indicated by dashed horizontal lines. (Details are provided in Table Al of the Appendix.) All scenarios reliably converge for moderate sample sizes. The Normal-Normal scenario is insignificantly different from Gumbel behavior (g = 0) for N > 50, and the heavy-tailed cases also exhibit the predicted Fréchet behavior (5 0.2). Notably, the ((5)-Noraal case converges slower rates, likely due to the relative instability of the inverse (X'X) in small samples. Overall, the simulation results support the applicability of Theorem 1 in small samples.
Location and Scale Estimation. Next, we evaluate whether block maximum MLE accurately captures the location and scale parameters for empirical testing. Results are provided in Figure Al of the Appendix. We find that bias-corrected estimation the location parameter works well, while the scale estimate is consistent but exhibits minor downward bias that disappears asymptotically (consistent with known limitations of the MLE; see Dombry Ferreira, 2019). For our goal of hypothesis testing, the overall distribution and its quantiles are recovered effectively.
4.2 APPLICATIONS
We investigate several real-world datasets - two applications from economics and biology, and four machine learning benchmarks and provide the first conclusive investigation of influence.
Economic Development and Geography. We re-examine the controversial finding that rugged terrain benefits African economies when compared to the rest of the world (Nunn & Puga, 2012).
7

---

1(3)
0.3
Normal-t(S)
Estimated Inverse Shape
(55--Nrmal
0.0
20
100
200
500
1000
Sample Size
Figure 3: Convergence empirical samples to the limiting distribution with increasing sample sizes (N {20, 30, 50, 75, 100, 150, 200, 300, 500, 1000), note the non-linear scale) for four different scenarios. Dots indicate estimated means from 1000 repetitions, while the shaded area indicates 1SD. The Normal-Normal and the ((5) cases quickly converge to the theoretical predictions of E 0 and 5 0.2, while the mixed scenarios converge marginally slower.
Kuschnig al. (2021) call the influence of the Seychelles, which remove significance of the estimate of interest when coupled with any of Rwanda, Lesotho, Eswatini, and the Comoros. However, lacking the statistical framework that provide, they were not able to test whether this level of influence should be deemed excessive or not.
We
decisively resolve this controversy. Table reveals the Seychelles excessively influential Orugged. both individually (p 0.001) and in combination with other outliers, except for Lesotho. This substantiates the suspected confounding from the size of nations (Kuschnig et al., 2021), lending statistical rigor prior concerns and calling into question the differential relation between ruggedness and income in Africa.
Table 1: Influence of Ruggedness on log(GDP per capita in 2000)
Influential Set
A(S)
i
p-value
Seychelles
0.077
0.020
0.004
1e 16
Seychelles + Lesotho
0.046
0.036
0.007
0.216
Seychelles + Rwanda
0.070
0.028
0.006
0.001
Seychelles + Eswatini
0.091
0.027
0.006
1c-16
Seychelles + Comoros
0.061
0.028
0.006
0.004
Sparrow Morphology - Big Heads and Beaks. We analyze the relation between head and tarsus length in saltmarsh sparrows, based on measurements of N = 1, 295 sparrows with known outliers (Gjerdrum et al., 2008; Zuur et al., 2010). The baseline regression yields = 0.011 with a standard error of (.030), implying a relation that is statistically indistinguishable from zero.
However, curious data point moves the estimate to 0.219(.029) turning the estimate significantly positive. An additional data point further moves the estimate to 0. (.032. These extreme impacts from vanishing fraction of the sample are deemed excessive by approach at any conventional significance level (both p < 0.001).5
$ 'One possible explanation for this excessive influence are data entry errors: The first observation (an outlier in both head and tarsus size) may have the two (adjacent) features mixed when swapped, they would fit well into overall averages. The second observation (an outlier in one feature) stands out with both values being equal up the significant digit.
8

---

Machine Learning Benchmarks. We apply our framework to four widely-used regression bench- marks: Law School, Adult Income. Boston Housing, and Communities & Crime. For each dataset, we identify most influential set of interest and test for excessive influence.
Law School (N 20, 800): We examine the coefficient for the 'Other' race indicator, with 378 relevant samples. We consider two sets: 77 data points that move the estimate from -0.0412 (.0144) to 0.1117 (.0159), creating significant estimate with flipped sign, and 17 data points that reduce the estimate to -0.0223 (.0097). Our approach indicates that the larger set's influence falls within expected variation, while the smaller set exhibits statistically excessive influence (p = 0.019).
Adult Income (N = 32, 561): We investigate the top 1% most influential sets (325 points) that shift the 'Male' indicator from 0 = 0.062, either raising it to 0.0992 or decreasing it to 0.0214. Despite these considerable shifts from a small fraction of the data, neither is deemed excessively influential by our approach.
Boston Housing (NV 506): We focus the effect of crime rate on house values. The baseline (highly significant) coefficient -tffiiient-O110SS0 -0.1080 (.0329) is rendered insignificant at -0.0352 (.0556) after excluding just 6 observations. In this case, the underlying EVD is Fréchet with inverse shape {- = 0.29 due to the heavy tail of the crime variable. The set's influence is highly significant (p- 0.001), indicating excessive influence.
Communities & Crime (N 994): We investigate 2 and 2 data points with substantial influence on the relation between race and crime rates. The complete set is not extreme, the points cancel each other out. After exclusion, the first subset of two increases the coefficient by more than 22%, which is deemed excessive < 0.001. When re-estimating after their exclusion, the second set decreases the estimate by more than 10% and is deemed excessive at the 5% level (p- 0.014). (See Table A2 for details.)
DISCUSSION
We develop the first rigorous statistical framework for assessing when most influential sets represent genuine problems rather than natural sampling variation. By deriving the extreme value distribution of maximal influence, we allow practitioners to replace ad-hoe sensitivity checks with principled statistical decision rules.
Our key insight is how maximal influence fundamentally depends on size and tail behavior. For constant-size sets with polynomial tails, maximal influence follows a heavy-tailed Fréchet distribution, implying that extreme influence can be arbitrarily large. For growing sets exponential-tailed data, maximal influence converges to a well-behaved Gumbel distribution.
These results address critical gap in interpretable machine learning. While recent work has developed methods to identify influential sets (Broderick et al., 2023; Freund & Hopkins, 2023; Hu et al., 2024), no formal theory existed to determine when their influence is excessive. Our framework provides the missimg theoretical foundation that enables rigorous statistical inference for influential observations and sets (first discussed by Cook, 1979).
We can clarify the applicability of heuristics that are commonly used for identifying excessive influence, and provide conclusive answers when excessively influential sets are suspected (such as the Seychelles in the ruggedness example; see Kuschnig et al., 2021). The 2/ //V threshold for coefficient influence (Belsley et al., 1980), e.g., is asymptotically accurate for randomly selected observations, but too restrictive for most influential observations, where the selection procedure necessitates extreme value theory.
Practical Recommendations. In general, most influential sets hold valuable information for inference. Our test is deliberately conservative, controlling Type errors (false claims of excessive influence) at the cost of some Type II errors (failing to detect truly excessive influence). This reflects our view that influential sets are a natural feature of data and not a problem to be eliminated. When test identifies an excessively influential set, however, recommend:

---

1. Investigate mechanism. Document the set and investigate why it differs - it may convey genuine heterogeneity, data quality issues, unobserved confounding, or important edge cases not addressed by the model.
2. Handle appropriately. We argue that an excessively influential set warrants separate analysis; exclusion can be considered if it reflects measurement error outliers that irrelevant for the pattern of interest.
3. Report transparently. State the set, decision, and test outcome; if conclusions hinge on the set, report both and discuss why. We advise against trimming winsorizing to force alignment with the remaining data; these transformations create artificial data that may obscure and distort rather than illuminate underlying relations.
Limitations and Future Work. Our analysis focuses on linear regression - foundational for theory and modern ML methods, but limited contexts where interpretability is valued (Rudin, 2019). Extensions to generalized linear models, tree-based methods, or non-parametric estimators require further developments. Our asymptotic arguments leverage independence between features and residuals, which may be restrictive when dependence affects influence patterns. Explicitly addressing dependence between influence components through the selected set and across sets is possible through generalizations. While simulations show that small-sample behavior quickly converges a predictions (even at N 100), further investigation the
asymptotic gap is warranted.
Practicality of approach hinges estimating the extreme value null, and, hence, approximating maximum influence. Finite-sample estimation of tail behavior and EVD parameters can be delicate, and improving these estimators would directly sharpen p-values. Two options include leveraging comain-speciff information and improved bias correction methods (Dombry & Ferreira, 2019; Oorschot & Zhou, 2020). Advances in finding influential sets remain an active research area (Hu et al., 2024; Huang et al., 2025), and could substantially improve block-maxima, reduce runtime, and broaden applicability.
Broader Implications. Our framework enables more reliable decision decision-making across domains where linear models remain the method of choice. Principled tools for understanding data points that drive model behavior crucial for building trustworthy systems. Applications span fairness assessments - where influential sets can reveal algorithmic bias - to causal inference settings, such randomized controlled trials quasi-experimental econometric analyses where small data subsets can fundamentally alter estimates.
Importantly, we reframe influence as natural feature of data requiring appropriate treatment rather than a problem to be fixed. Influential sets can represent genuine heterogeneity or important edge cases that should inform model development. This perspective enables more nuanced approaches data analysis, where information is preserved and assessed through principled statistical inference rather than discarded based on rules of thumb.
CONCLUSION
We developed a statistical framework that transforms the assessment of most influential sets from art to science. By deriving the extreme value distributions of maximal influence, we enable rigorous hypothesis testing distinguish excessive influence from natural variation. Applications across economics, biology, and machine learning benchmarks demonstrate the practical utility of our approach.
Our method offers clear guidance to practitioners when small sets overturn results of interest, our tests reveal whether this influence is statistically excessive. This enables more robust and transparent decision-make making in settings where reliability matters, from medical trials to policy evaluation to algorithmic systems. By providing theoretical foundations for influential set analysis, this work advances both the theory and practice of interpretable machine learning.
10

---

REPRODUCIBILITY STATEMENT
Proofs are detailed in the Appendix, datasets are from the cited sources, and code to reproduce results are available https: https:://github.com.. http://www.oooo/oooacceee/eessee
STATEMENT ON LLM USE
Large language models used (i) aid and polish writing, (ii) discover and retrieve related work, (iii) check results for apparent mistakes.
REFERENCES
Samyadeep Basu, Xuchen You, and Soheil Feizi. On second-order group influence functions for black-box predictions. In Proceedings the 37th International Conference on Machine Learning, pp. 1139-1 1449. PMLR, 2020.
David A. Belsley, Edwin Kuh, and Roy E. Welsch. Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. John Wiley & Sons, 1980. doi: 10.1002/0471725153.
Emily Black and Matt Fredrikson. Leave-one-out unfairness. Proceedings of the 2021 ACM Conference Fairness, Accountabilityy and Transparency, pp. 285-295, ACM, 2021. doi:
(415/3442188.3445555
Tamara Broderick, Ryan Giordano, and Rachael Meager. An automatic finite-sample robustness metrie: When dropping little data make big difference?, 2023. URL pettttss://rxiii org/abs/ 2011 14999. arXiv:201 1. 14999 [stat.ME].
Axel Bücher and Johan Segers. On the maximum likelihood estimator for the generalized extreme- value distribution. Extremes, 20(4):839-87) 2017. doi: 10.100777/00877-077-222-666
Irene Chen, Fredrik Johansson, and David Sontag. Why my classifier discriminatory? In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
Anshuman Chhabra, Peizhao Li, Prasant Mohapatra, and Hongfu Liu. What data benefits my classi- fier? enhancing model performance and interpretability through influence-based data selection, In Advances in Neural Information Processing Systems, 2023.
Stuart Coles. An Introduction Statistical Modeling of Extreme Values, Journal of the American Statistical Association, 1044404 December 2002. ISSN 0162-1459, 1537-274X. doi: 10.1 1a.11999jaa..2022.2222
R. Dennis Cook. Influential observations linear regression, Journal of the American Statistical Association, 74(365):169-1777 1979. doi: 10.2307/2286747.
Laurens de Haan and Ana Ferreira. Extreme Value Theory: An Introduction. Springer, 2006.
Clément Dombry and Ana Ferreira. Maximum likelihood estimators based on the block maxima method. Bernoulli, 25(3): 1723, 2019. doi: 10.3150/18BEEJ1032.
Paul Embrechts, Claudia Klüppelberg, and Thomas Mikosch. Modelling Extremal Events. Springer, 1997.
Jillian Fisher, Lang Liu, Krishna Pillutla, Yejin Choi, and Zaid Harchaoui. Influence diagnostics under ceff-concordance. In Proceedings of the 26th International Conference on Artificial Intelligence and Statistics, pp. 10028-1007)) PMLR, 2023.
R. A. Fisher and I.. H. C. Tippett. Liimiting forms of the frequency distribution of the largest or smallest member of sample. Mathematical Proceedings of the Cambridge Philosophical Society, 2402):180-190, 1928. doi: 10. 10.100777033050041100155888.
D. Freund and S. B. Hopkins. Towards practical robustness auditing for linear regression. arXiv preprint, 2023. doi: 10.48550aaxx..220711155

---

Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In Proceedings of the 36th International Conference on Machine Learning, pp. 1242-22511 PMLR, 2019.
Carina Gjerdrum, Kira Sullivan-Wiley, Erin King, Margaret A. Rubega, and Chris S. Elphick. Egg and chick fates during tidal flooding of saltmarsh sharp-tailed sparrow nests. The Condor: Ornithological Applications, 110(3):579-5888) 2008. doi: 25555//ood.200.8855555
B. Gnedenko. Sur la distribution limite du terme maximum d'une série aléatoire. Annals of Mathematics, 44(3):423-455 1943.
Y. Hu, P. Hu, H. Zhao, and J. W. Ma. Most influential subset selection: Challenges, promises, and beyond. In Advances in Neural Information Processing Systems, 2024. NeurIPS.
Jenny Y. Huang, David R. Burt, Yunyi Shen, Tin D. Nguyen, and Tamara Broderick. Approximations to worst-case data dropping: Unmasking failure modes, 2025. URL https::/ttaaatt /ttpiv./aaxivvvvoa org/ abs/ /400 09008, arXiv:2408.0999 09008 [stat.ME].
Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Proceedings the 2017.
the 34th International Conference on Machine Learning, pp. 1885-1894. PMLR,
Pang Wei Koh, Kai- Kai-Siang Ang, Hubert H. K. Teo, and Percy Liang. On the accuracy of influence functions for measuring group effects. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 5254-5264. Curran Associates, Inc., 2019.
Nikolas Kuschnig, Gregor Zens, and Jesús Crespo Cuaresma. Hidden in plain sight: Influential sets in linear regression. Technical Report 8981, CESifo, 2021.
Ankur Moitra and Dhruv Rohatgi. Provably auditing ordinary least squares in low dimensions. In International Conference on Learning Representations, 2023. doi: 10.48550karxi.22255.. 142.4 ICLR 2023.
Nathan Nunn and Diego Puga. Ruggedness: The blessing of bad geography in Africa. Review of Economics and Statistics, 94(1):20-36, 2012. doi: 10. 10.1162/RET_aa_00066
Jochem Oorschot and Chen Zhou. All block maxima method for estimating the extreme value index,
2020. URL https: www.v..rvvvv/v.////v///..//a..a. org/ /2000////01.00 15950. arXiv:2010.15955 [math.ST].
Ittai Rubinstein and Samuel Hopkins. Robustness auditing for linear regression: To singularity and beyond. In International Conference Learning Representations, 2025.
Cynthia Rudin. Stop explaining black box machine learning models for high-stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5):206-215, 2019. doi: 10.1038//42256-019-0000
Prasanna Sattigeri, Soumya Ghosh, Inkit Padhi, Pierre Dognin, and Kush R. Varshney. Fair infinitesi- mal jackknife: Mitigating the influence of biased training data points without refitting, 2022. URL
fittps:/aaret org/ alos 2212. 0 06803. arXiv:2212.00 12.06803 [cs].
Andrea Schioppa, Katja Filippova, Ivan Titov, and Polina Zablotskaia. Theoretical and practical perspectives what influence functions do. Advances in Neural Information Processing Systems, 36:27560-2755 27581, 2023.
Richard L. Smith. Maximum likelihood estimation in a class of nonregular cases. Biometrika, 72(1): 67-90, 1985.
Esteban Walker and Jeffrey B. Birch. Influence measures in ridge regression. Technometrics, 30(2): 221-227, 1988.
Alain F. Zuur, Elena N. Ieno, and Chris S. Elphick. A protocol for data exploration to avoid common statistical problems. Methods in Ecology and Evolution, 1(1):3-14, 2010. doi: 10.1111/j. 2041-210X. 20..210000099.0...
12

---

A1 LEMMA FOR THE INVERSE SUM OF SQUARES
Lemma 1 (Asymptotic Normality of Inverse Sum of Squares). Let (Xi)i=) be a sequence of independent and identically distributed (i.i.d) random variables satisfying:
1. [[X]] 0o (finite fourth moment)
2. [[x]] μl 0 (positive second moment)
3. Var(Xi) d² (non-degenerate variance of squares)
Define Sn Di=1 X and Yn -. Then Yn is asymptotically normal with:
1/22 Yn
11
4
N
0, 0²
as + 0c.
mp
1
Proof. Define the sample mean of squares
X nnnns
Sn By the Central Limit Theorem (CLT);
Vn X p1) 4 NO
where μ = [ and o² Var(Xf) (finite by E[X < oo).
Consider the transformation g(x) x-1, which is differentiable at > 0 with derivative g'(x) = x 2 The Delta Method gives:
vn (1(x,)) - s(w)) S N (0,00--1'(()))
Substituting (xx = (X ( ----1//5 n/S,, and g(11 = 1:
1 S N (0, 2-)- -V (0. o²)
Vn
n1
Sn
Rewriting n/Sn nyn;
Vn - 11 $ N (0, 0²
Factoring the left side:
1
Vā - 1-1) = 1/2 n Yn
1
= 3/2
Ynn
nip4
Thus:
3/2
Yn 1
- NV
0,
a²
map
n1a
13

---

A2 LEMMATA FOR THE PRODUCT EVD
For notational simplicity let S = Eigs X; R4 and : D_g. where D= X This holds for any realization S = E IR and T = E RR. Further, let MDA(H) denote the maximum domain of attraction of an EVD H where we write Z E MDA(H). We specifically denote the Fréchet as P. and the Gumbel as A. We are interested in the EVD of A S.
Lemma 2. Let S MD)(A) and T E (DD((e)) with tail-coefficient > 0 and S and T being independent. then A (3) S- MD).
Proof. Recall that for Gumbel tails (S) the survival function decays faster than any polynomial, i.e..
3 3- (-33
P(S s) exp
oc,
while for the Fréchet tails (7) the survival function is regularly varying with index aa, i.e.,
P(T > ~>~("= as l + oo,
where (t) is slowly varying function. The density satisfies:
fr(t) at Lt(() as t +
We are interested in the EVD of A, i.e., P(A > 8). Since A = - and S, T are independent by assumption:
IP 8) = PPST T > 8) { P(S > 6/1)
Next, we split the integral at M > 0:
M P(S > (/t)sr(t)dt+
n P(S (1)
P(A>))
For fixed M, we have 7, - 0, as 8 - since 8/1 + and Gumbel tails decay faster than any polynomial, and the dominant term is Tz. Substitute = 8/1 (1 = 8/u, , di -65/ u2 du), and we have
P(S > 8/()sT(()dt 1 F(S > ()fT(ō/u) du.
12
u
M
Using the asymptotic form of fr:
[r(5/u) ~ a(5/u) - I8
we obtain
5
8
-a-1
5
Ig ~
P(S > u)
a
Ly
du
1
u²
5/M
8
ad
IPSS
> u) a-1 Lor
du.
11
As 8 - oc, by Lemma 4 in Appendix A3, we obtain
6//
PPS > u)u--1 Lor ( du ~ [r( / TPSS > u) du.
(A2.1)
The integral converges because:
1. near 0 we have IP(S > u) RJ 1 and a- is integrable for 0, and
2. the Gumbel decay dominates a1
14

---

Denote the constant
C)a, = ( IP(S u)u du E (0,
then
P(A> 8) aδ aō LL 8) 8-" ()L(())
The term in parentheses is slowly varying in 8 since (6) is slowly varying.
Thus, the survival function P(A 8) is regularly varying with index -a, and therefore, A has Fréchet tails with tail-coefficient a, which concludes the proof.
Corollary Following Lemma and assuming a tail coefficient oc follows that S Gumbel and thus A (S) S. E MD)(A).
Proof. The result follows directly from properties of the Fréchet distribution.
Lemma 3. S E MDA( and T MD)((ts) then A (S) E MDA @ (Pmin(a,)).
Proof. The proof of this follows directly from Lemma 1.3.1 the convolution closure of distribution functions with regularly varying tails in Embrechts et al. (1997).
Corollary (Conditional EVD). Further, if S E MDA(E) for some EVD E, it holds that
A (S) X MDA(E).
15

---

A3 LEMMA FOR ASYMPTOTIC EQUIVALENCE
Lemma (Asymptotic Equivalence Statement).
6/M
P(S > u) ( du ~ LT(0) / PPS u)u du,
where has Gumbel tails, Lr slowly varying, > is the tail coefficient, M > a fixed constant.
Proof. For clarity, we prove this result in five steps.
STEP 1: INTEGRAL SPLITTING
Define
5/M
IP(S > u))n-IIIy (2) du (1(6) + I2(6),
T(6)
0
where
[,(6) = J. P(S > u)ua LT
6 du.
6
12(6)
P(S u)ua-11 LT
du.
STEP 2: ANALYSIS OF (0) (BOUNDED DOMAIN)
For (0, 1], we have:
lim (8) lim [ IPS > u)ua-1 (6(uu) du
6 Lr(5) 6
(+)
/. P(S > ul)a=idde du,
by the Dominated Convergence Theorem (DCT):
Pointwise convergence: for fixed u >> 0, limg- (r(5/u) (.((8)
1.
Dominating function: by Potter's theorem, for any 5 > there exists Cs 0 such that
LT Lō(u/)
< Csu for all large
(r)
Choose < a such that - is integrable on (0, 1], then
TPS > u)ua a- (6/u)) (6)
C-ua-1-6 (since P & 1),
and the dominating function C5ua-1-6 is integrable over (0, 1] for a > 8 0.
STEP 3: ANALYSIS OF Iz( (GrOWING DOMAIN)
For 8LM8 we have
lim T2(6) lim
6/M
P(S > u) (ōT(//u) (r
du
8 (g)
5-+00
P(S > u)u a- du by the DCT.
Lr(6/u) Pointwise convergence: for fixed > 1, lims-+00
Lr(5)
16

---

Dominating function: by Potter's theorem, for 8 > 0:
Lr(ő/u) (T(
& Csu
for all large 8, 2 1.
Choose such that k = a 1 + 5 > 0, and
P(S u)u du < c0,
since the Gumbel decay dominates. Then
P(S u)ua Lr(ō/u) & C&P(S> u)u" LT(()
and the dominating function C,P(S > u/u* is integrable over oo).
Tail control: as ō + xo, the upper limit 5/M + oo and
f 6/M
C&P(S > u)u* du - 0.
STEP 4: NEGLIGIBILITY OF OMITTED TAIL
The tail beyond 8/M is negligible:
R(6) f IPS u)u LT ( du.
For > 8/M, have 5/u & M, which is bounded compact sets: (5(b/u) & CM-
By the Gumbel tail properties, there exist a S.t. P(S u) & se-"" for large Thus
I c- a- du o(1) 8 + 00.
[R(0)] & CM
6/M
Since LT( + oo or is slowly varying, R(8) = o(L)()))
STEP 5: FINAL COMBINATION Combining all results, we have
T(6) L1 (8)
[1()) + T2(0) + R())
(r)
LT(6) + [2)
(8)
LT()) + o(1)
xcc
P(S > u)u² a- du +
P(S u)u du
P(S u)u-- du.
17

---

A4
AUXILIARY MATERIAL
N
Distribution
Mean
Std.Dev.
Q25
Median
Q75
Normal-Normal
0.1385
0.2541
0.0206
0.1293
0.2902
f(5)-Norma Normal-t(5). (5))
0.2417
0.2865
0.0863
0.2329
0.3994
20
0.2529
0.2479
0.0910
0.2505
0,4089
((5-1(5)
0.3271
0.2719
0.1601
0.3144
0.4922
Normal-Normal
0,1021
0.1803
0.0079
0.1101
0.2291
(55-Noolal
0.1989
0.1845
0.0917
0.1994
0.3067
30
Normal-t(0 (55)
0.2023
0.1846
0.0817
0.2101
0.3225
(5)
0.2670
0.1912
0.1453
0.2668
0.3961
Normal-Normal
0.0668
0.1255
0.0153
0.0631
0.1466
(55-NNorall
0.1708
0.1304
0.0890
0.1701
0.2532
50
Normal-t(5) ((5 0,2307
0,1843
0.1286
0.1015
0.1797
0.2704
0.1363
0.1420
0.2305
0.3203
Normal-Norma
0.0474
0.0960
-0.0163
0.0485
0,1176
t(5)-Normal
0.1500
0.1006
0.0821
0.1469
0.2173
75
Normal-t(5) t(5)-t(5)
0.1735
0.1057
0.1037
0.1749
0.2434
0.2214
0.1046
0.1561
0.2213
0.2885
Normal-Normal
0.0422
0.0788
0.0115
0.0429
0.0959
((5)-Norm)
0.1389
0.0846
0.0790
0.1369
0.1948
100
Normal-u(5) ((5)-((5)
0.1721
0.0865
0.1183
0.1731
0.2291
0.2083
0.0876
0.1492
0.2056
0.2639
Normal-Normal
0.0253
0.0506
0.0075
0.0253
0.0582
t(5)-Normal
0,1418
0.0587
0.1020
0.1403
0,1833
200
Normal-t(5) t(5)-t(5)
0.1655
0.0576
0.1272
0.1632
0.2013
0.2072
0.0595
0.1691
0.2054
0.2450
-ormallNormal
0.0169
0.0406
0.0096
0.0180
0.0455
f(5)-Normal
0.1423
0.0461
0.1122
0.1446
0.1731
300
Normal-((5) (5)
0.1705
0.0463
0.1377
0.1695
0.2006
0.2082
0.0511
0.1757
0.2082
0.2406
Normal-Norma
0.0157
0.0319
0.0049
0.0154
0.0381
Normalll
0.1488
0.0378
0.1245
0.1499
0.1739
500
Normal-t(5) (55)
0.1667
0.0354
0.1419
0.1622
0.1903
0.2087
0.0385
0.1833
0.2085
0.2323
Normal-Normal
0.0103
0.0220
-0.0042
0.0109
0.0243
t(5) Normal
0.1591
0.0287
0.1408
0.1589
0.1771
1000
Normal-t(5)
0.1734
0.0252
0.1565
0.1743
0.1905
t(5)
0.2105
0.0262
0.1938
0.2101
0.2273
Table Al: Inverse shape estimates for different distributions and sample sizes. They indicate fast convergence to asymptotic predictions (0.0 for the Normal-Normal, and 0.2 for other cases) - for all settings, the predicted value is contained between the t5 and t5 quantile. Estimates based on 1,000 repetitions.
18

---

(a) Average Gumbel Densities
(b) Location Parameter
(c) Scale Parameter
2000
BM::
True
BM (ndj.)
150
True: 0.091
Trues 2e-04
Froquency
100
Derely
100
50
0
0.000
0.001
0.002
3.003
0.0010
0.0014 0.0018
0.00010 0.00020
0.00030
TM
Figure Al: Simulation exercise for the performance of the simple MLE based on block maxima, correcting for block size. While the corrected location parameter à close to unbiased, the scale parameter i suffers some downward bias using simple block maxima, which is line with Dombry & Ferreira (2019). However, for practical purposes the block maxima are expected to be fitting reasonably well, as visible in panel (a).
Set Composition
Set Size
(S)
b
p-value
Full set
0.0214
0.0076
0.0029
0.4914
1st partial
2
0.0456
0.0050
0.0021
7.62e
2nd after excl. 1st
2
0.0241
0.0051
0.0022
0.0141
Table A2:
Influence of % Black Population on Violent Crimes. The table summarizes the results for testing the preselected set and its subsets for significant influence of the percent of black population on the violent crimes committed per population.
19


Reviews:
[{"review_id": "cgZ7CWPzuo", "reviewer": "Reviewer_mjVQ", "summary": "This paper investigates the problem of assess the statistical significance of the most influential sets in OLS and proposes the first testing framework by deriving extreme-value limits for the maximum subset influence and turning them into calibrated p-values.\n\nThe proposed method is applied to several existing datasets in economics, biology, and machine learning benchmarks, demonstrating that some datasets indeed contain outliers with excessive influence while the influence of most influential subsets in some other datasets are not significantly excessive than natural sampling variation.", "questions": "How does a failure to identify the most influential subset impact the estimated p-value?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem of testing most influential subsets is novel and well-motivated.\n- The proposed framework is theoretically sound.\n- The proposed approach may find broad applications in scientific domains that relies on OLS for data analysis.", "weaknesses": "- The proposed framework is limited to OLS.\n- The block-maxima MLE used in the proposed approach suffers from a bias.\n- The theory is only applicable to the low dim regime.\n\nMinor: this work may be a better fit to statistics or economic venues than machine learning ones.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SluICvMiNT", "reviewer": "Reviewer_7xpC", "summary": "This paper examines the most influential sets problem for linear regression model from the hypothesis testing point of view, deriving the limiting distribution of the influence of most influential sets.", "questions": "1. Typo: Line 118 states that $\\epsilon=0$ recovers $\\hat{\\theta}$, which is think is not, as the left-hand side is summing over $n\\notin \\mathbb{S}$. Similarly, the claim for $\\epsilon=-N^{-1}$ does not recover $\\hat{\\theta}_{-\\mathbb{S}}$, but rather $\\epsilon=0$.\n2. Typo: Equation (1): It is *fine* to define influence as what it is, but it is far from correct to write $\\hat{\\theta}(\\epsilon; \\mathbb{S}) \\approx \\mathcal{I}(\\mathbb{S})$. The correct interpretation is that, $\\\\hat{\\\\theta}\\_{-\\\\emptyset} - \\\\hat{\\\\theta}\\_{- \\\\mathbb{S}} \\\\approx \\\\epsilon \\\\mathcal{I} (\\\\mathbb{S})$.\n3. Line 113 should reference the influence function paper by Koh & Liang, 2017.\n4. To justify the contribution, I think it'll be beneficial to provide some critical (real-world/conceptual) examples that require hypothesis testing for the most influential sets problem.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Novel theoretical results: I'm not aware of related literatures that address the limiting distribution of the maximum influence for linear regression model.\n2. Conceptual clarity and motivation: The identified gap is indeed an important research question to address.", "weaknesses": "1. Practical guidance: Since the theory focuses on asymptotic behavior, it is unclear how many samples are needed to render the theory applicable.\n2. Presentation: The clarity of the paper can be improved by carefully restructuring the sections. For instance, it seems like the presentation before Section 3.2 largely follows [1]. However, some parts are not necessarily used, for instance, the influence function. While I won't say this is to the extent of \"plagiarism\", however, I do think some careful adaptation to the current paper will be more appropriate.\n3. Limited scope: This is my main concern. While I do think this paper studies an important research question, however, the scope and the result are limited. With the fact that the empirical experiments are also focusing on simple datasets and without significant empirical findings, I do not think I can suggest acceptance to the conference. This is only my opinion, and I'll leave the final judgment to the AC.\n\n\n[1] Yuzheng Hu, Pingbang Hu, Han Zhao, and Jiaqi W. Ma. Most influential subset selection: Challenges, promises, and beyond.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hCmNhcJ17C", "reviewer": "Reviewer_AdTk", "summary": "This paper studies maximally influential data subsets in linear regression and proposes a statistical testing framework to determine when identified influential sets reflect excessive rather than natural sampling variability. The authors derive asymptotic extreme value distributions (Fréchet vs. Gumbel) for the maximum influence statistic under different scaling regimes of subset size, provide computation formulas for exact influence, and apply the approach to simulated and real datasets across multiple domains.", "questions": "1. See weaknesses 1 and 2\n2. Would the hypothesis test disagree meaningfully with simpler leverage or leave-one-out metrics or Broderick et al. (2021)?\n3. What are the real-world applications that this approach can benefit? I understand that people can use it to find the influential sets, but what should be the next steps?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper tackles an important question on understanding the influential data subsets on the statistical estimator.\n2. The presentation is easy to understand, with theoretical contributions.", "weaknesses": "1. The motivation for p-values in influential subset testing is not clearly justified. The main distinction from Broderick et al., (2021) is that this paper added a statistical significance test, but the paper does not convincingly demonstrate why this hypothesis-testing perspective materially improves decision-making in practice. For example, all the real data applications in this paper can be similarly done wth Broderick et al. (2020). I would suggest that the authors provide why the proposed approach yields better conclusions by using these p-values compared to existing influence quantification approaches.\n\n2. The proposed approach is limited only to linear regression, while Broderick et al. (2021) can be applied to all M-estimation problems. Extensions to GLMs or modern models are only speculated about, yet the paper argues broadly about interpretability and fairness across ML. Can this approach be applied to all M-estimation problems?\n\nReference: \nBroderick, T., Giordano, R., & Meager, R. (2020). An automatic finite-sample robustness metric: when can dropping a little data make a big difference?. arXiv preprint arXiv:2011.14999.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UTc6yRRcQv", "reviewer": "Reviewer_mbGm", "summary": "This work introduces a rigorous hypothesis testing framework for influential sets on the coefficient in a univariate regression. It identifies the asymptotic distribution for the maximum influence of any such set for both fixed size and varying sized sets. These insights are then used to test whether the influence of a set is significant, meaning its influence is beyond that expected from natural variations in data. This framework is validated and applied in six case studies.", "questions": "- In the equation before (1), should the first sum be over n \\in [N] rather than not in S?\n- It is stated that \"we assume a univariate model with a positive coefficient\". However, the machine learning case studies include cases with negative coefficients. Does this violate the theory, or am I missing something?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is generally well written, with clean and easy to follow figures.\n- This work is original, presenting the first hypothesis testing framework for influential observations (to my knowledge).\n- This work is well motivated. The problem of rigorous testing for influential observations is challenging, but has ramifications for a wide range of practical applications. This range is highlighted well by the variety of case studies in this paper. In particular, the question of how terrain ruggedness influences GDP is interesting. Broadly, the domains chosen for these case studies are well selected.\n- The presented statistical theory is rigorous and validated through simulation studies.", "weaknesses": "- The presented framework only applies to univariate regression with positive coefficients. This restriction excludes many practical use cases, including simple extensions like multivariate regression. As such, the machine learning case studies presented in this work seem somewhat contrived, as machine learning practitioners tend to approach these datasets using multivariate regression.\n- It seems like there is a risk of extreme multiple hypothesis testing problem here. For example, on the Boston housing dataset, six observations were selected as the most influential set. If these were selected after considering, say, the 1, 2, 3, 4, and 5 most suspicious observations, six hypothesis tests have already been performed (or at the very least, six effect sizes measured). In practice, it seems very tempting (and easy) to consider many potential influential sets, ultimately invalidating the conclusion or making the p-values large if a correction is applied.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ml1JpB7V8E", "reviewer": "Reviewer_PJQ8", "summary": "This paper develops a statistical framework to distinguish natural sampling variation from excessive influence in most influential subsets. The authors derive exact expressions for subset influence in univariate linear regression and establish that the maximal influence follows asymptotic extreme-value distributions (Fréchet or Gumbel). Building on this, they propose a hypothesis testing procedure based on MLE (to estimate the EVD parameters) and an adaptive greedy algorithm from prior work (to estimate maximum influence). Empirical studies on economics, biology, and classic ML datasets illustrate the method’s interpretability and robustness benefits.", "questions": "I don't have further questions at this point, but would be willing to reconsider my rating if the authors can provide a clear and technically plausible pathway for extending their theoretical framework beyond the 1D OLS setting.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Neat theoretical framework, which elegantly links subset influence to extreme-value asymptotics\n\n- Comprehensive empirical evaluations across diverse domains", "weaknesses": "- The theory focuses exclusively on 1D OLS, which is too restrictive. Based on my understanding, it is very difficult (if not impossible) to generalize Proposition 1 to high-dimensional OLS. All empirical studies also concentrate on tabular data and linear models, further limiting the paper’s applicability to modern ML settings.\n\n- ML practitioners typically treat the training dataset as fixed rather than as a random sample from an underlying distribution. The authors should better motivate why testing statistical significance rather than simply detecting influence is meaningful in this context.\n\n- The paper would benefit from clearer exposition of extreme-value theory and stronger intuition for the Gumbel vs. Fréchet distinction, which are currently introduced with limited background or explanation.\n\nIn summary, while the work’s statistical rigor is admirable, its theoretical contribution does not quite meet the bar for a top-tier ML conference, and its relevance to representation learning is limited. I encourage the authors to further develop and generalize the theory and consider submitting to a statistics or econometrics venue, where inferential rigor is the primary focus.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)