Title: On the Adversarial Robustness of Benjamini Hochberg
Venue: NeurIPS 2024
Accepted: yes
Keywords: multiple testing

p-values

false discovery rate

adversarial robust
Abstract: The Benjamini-Hochberg (BH) procedure is widely used to control the false detection rate (FDR) in multiple testing. Applications of this control abound in drug discovery, forensics, anomaly detection, and, in particular, machine learning, ranging from nonparametric outlier detection to out-of-distribution detection and one-class classification methods. Considering this control could be relied upon in critical safety/security contexts, we investigate its adversarial robustness. More precisely, we study under what conditions BH does and does not exhibit adversarial robustness, we present a class of simple and easily implementable adversarial test-perturbation algorithms, and we perform computational experiments. With our algorithms, we demonstrate that there are conditions under which BH's control can be significantly broken with relatively few (even just one) test score perturbation(s), and provide non-asymptotic guarantees on the expected adversarial-adjustment to FDR. Our technical analysis involves a combinatorial reframing of the BH procedure as a ``balls into bins'' process, and drawing a connection to generalized ballot problems to facilitate an information-theoretic approach for deriving non-asymptotic lower bounds.

On the Adversarial Robustness of Benjamini Hochberg

Abstract

The Benjamini-Hochberg (BH) procedure is widely used to control the false detection rate (FDR) in multiple testing. Applications of this control abound in drug discovery, forensics, anomaly detection, and, in particular, machine learning, ranging from nonparametric outlier detection to out-of-distribution detection and one-class classification methods. Considering this control could be relied upon in critical safety/security contexts, we investigate its adversarial robustness. More precisely, we study under what conditions BH does and does not exhibit adversarial robustness, we present a class of simple and easily implementable adversarial test-perturbation algorithms, and we perform computational experiments. With our algorithms, we demonstrate that there are conditions under which BH's control can be significantly broken with relatively few (even just one) test score perturbation(s), and provide non-asymptotic guarantees on the expected adversarial-adjustment to FDR. Our technical analysis involves a combinatorial reframing of the BH procedure as a 'balls into bins' process, and drawing a connection to generalized ballot problems to facilitate an information-theoretic approach for deriving non-asymptotic lower bounds.

1 Introduction

Multiple testing has broad applications in drug discovery, forensics, candidate screening, anomaly detection, and in particular, machine learning. Indeed, recent works [5, 18, 28, 32], in nonparametric outlier detection, out-of-distribution detection (OOD), and one-class classification have all adopted multiple testing methodology in developing principled decision rules with statistical guarantees. In fact, the Benjamini-Hochberg (BH) multiple testing procedure, widely used to control the false detection rate (FDR), is either used or modified in all these recent methods. Considering this FDR control could be relied upon in some critical (safety/security) contexts, for which false positives incur costs, we investigate its adversarial robustness.

Adversarial corruption presents a challenge to statistical methodology, and is a modern-day concern due to not only the ease with which high volumes of data can now be accessed/processed but also the increasingly widespread use of statistical procedures. This threat poses vulnerabilities to machine learning tasks like OOD, which would aim to fortify security systems like fraud detection [5].

Manipulation of data and experimental results are common means by which incorrect conclusions can be reached. Worse, strategic perturbation can dramatically decrease the fidelity of the models and methods used. A burgeoning field of adversarial corruption has gained traction in recent years to meet this concern, most notably in the area of (deep) machine learning; see, for example, [27, 16, 20]. In this work we address adversarial corruption in hypothesis testing, specifically in the large-scale context in which the primary focus is on the aggregate metric: FDR.

BH [6] is one of the most widely used multiple testing procedures, which upon input of a collection of p-values, outputs a rejection region ensuring that the FDR is no greater than a user-defined threshold q ∈ ( 0 , 1 ) . This control of FDR holds under independently generated p-values - as well as some restricted forms of dependency like positive regression dependent on a subset (PRDS) [7] - but it generally holds without strong assumptions on the alternative distributions. This degree of distributional robustness, however, could be said to come at the cost of adversarial robustness, as we show in this work.

1.1 Literature Review

Although OOD methods [23, 12, 21, 22] are often complex and not always supported by statistical guarantees, conformal inference has made possible the use of one-class classifiers to generate conformal p-values for which OOD can now be conducted via multiple testing. This has led to the adoption of the BH procedure in OOD. Indeed, [5] leverages the FDR control afforded by BH over conformal p-values (shown to be PRDS) to test for outliers. More precisely, given a test set of observations for which we wish to identify as inliers or outliers (out of distribution), a conformal p-value is generated for each observation, which is then processed by BH to decide which are likely outliers. We refer the reader to [18, 28] for other recent works along this vein.

In recent years, concerns have risen over the possibility of adversarial manipulation of statistical methodologies. This manipulation commonly occurs at the level of data collection and training, often invalidating the assumptions made regarding how data is drawn, but it can also occur at test time. There is a growing literature on adversarial robustness , which is concerned with securing statistical methods like (deep) machine learning [27, 16, 20], linear regression [8], M-estimation [9], and online learning [26, 17, 1]. In particular, [13] considers contamination models that incorporate (adaptive) adversarial perturbation of up to an ϵ -fraction of drawn data. Indeed, we adopt this modeling in our own study - see (c-Perturb). As well, a similar concept to the notion of adversarial robustness that we adopt in this paper is one the literature refers to as perturbation resilience . Generally speaking, a problem instance is called α -perturbation resilient when despite a degree (parameterized by α ) of perturbation to the instance, the optimal solution does not change. First introduced in [10] for combinatorial optimization (in particular, MAX-CUT), the concept has since also inspired research into devising resilient unsupervised learning, particularly in clustering [4, 3, 2].

With respect to the hypothesis testing literature, there are recent adversarial robust studies focused on simple [19] and sequential hypothesis testing [11] from a game theoretic perspective, in which protection of statistical power, risk, or sample size from corruption is of chief concern. Complementing the adversarial robust perspective are several distributionally robust studies, in which the data-generating distribution is known only to lie in a parametric family. Recent works include [11, 24, 15] which focus on test risk in single and sequential hypothesis testing settings employing uncertainty sets of distributions of fixed distance (e.g. Wasserstein, phi-divergence) for the null and alternative hypotheses. In contrast to these works, this paper is focused on FDR, not individual test risk. Furthermore, distributional robustness is not equivalent to the perturbation-robustness that this paper and other adversarial robust studies seek in general. Indeed, [30] shows that the BH procedure's FDR control exhibits a distributional robustness to possible dependence between null and non-null hypotheses. On the other hand, our work would illustrate that, distributional robustness aside, BH can lack adversarial robustness.

1.2 Preliminaries

Let N ∶= { 1 , . . . , N } , where N ∈ ❩ + , be a finite set for which each member i ∈ N denotes a binary hypothesis test deciding between a null and alternative hypothesis. Further, there exists a partitioning, N = H 0 ⊍ H 1 , such that the correct decision for test i ∈ N is either null if i ∈ H 0 , or alternative if i ∈ H 1 . Here, H 0 and H 1 are the (unknown) sets of null and alternative test indices, respectively;

consequently, for any test i , the correct decision (i.e. set membership) is unknown to the decision maker. In fact, while the number of tests N is known (and large, on the order of thousands), neither N 0 ∶= ∣H 0 ∣ nor π 0 ∶= N 0 N is known to the decision maker, although π 0 ≥ 0 . 90 'is reasonable in most large-scale testing situations" - ([14], p. 285).

For each test i ∈ N , p-value p i ∈ [ 0 , 1 ] is randomly generated (independent of all other p j , j ≠ i ), which we model as a draw from either U ( 0 , 1 ) when i ∈ H 0 ( p i then referred to as a null p-value ) or some alternative distribution /a80 1 i on [ 0 , 1 ] when i ∈ H 1 ( p i then referred to as an alternative p-value ). Amultiple-testing algorithm A takes as input a randomly generated collection of p-values p = { p i } i ∈N and outputs for each test i a determination A( i ) ∈ { 0 , 1 } with A( i ) = 1 iff the determination is to reject the null hypothesis (i.e., claim i ∈ H 1 ), or sometimes referred to as 'make a discovery" for the i -th test. With a p ∶= ∑ i ∈H 0 A( i ) denoting the number of false discoveries, and R p ∶= ∣A -1 ( 1 )∣ denoting the number of rejections/discoveries made, we refer to FDP [A ; p ] ∶= a p R p ∨ 1 as the false detection proportion summarizing A 's decisions on p-values p , where x ∨ y is shorthand for max ( x, y ) for any x, y ∈ ❘ . We refer to its expectation with respect to the random generation of p as the false detection rate , FDR (A) ∶= ❊ p FDP [A ; p ] .

In this work, we focus on the Benjamini Hochberg procedure, a widely-used multiple-testing algorithm.

1.3 The Benjamini Hochberg (BH) Procedure

Given a collection of p-values p = { p i } i ∈N and desired control level q ∈ ( 0 , 1 ) to bound FDR, the BH procedure BH q operates as follows:

The p-values are sorted in increasing order, p ( 1 ) ≤ p ( 2 ) ≤ . . . ≤ p ( N ) .

The index i max ∶= max { i ∈ [ 0 , N ] ∶ p ( i ) ≤ i q N } is identified, with p ( 0 ) ∶= 0 .

BH q provides provable FDR control at the level of q without any assumptions on the alternative distributions { /a80 1 i } i ∈H 1 .

❩ 3. Reject the tests corresponding to the smallest i max p-values: p ( 1 ) , p ( 2 ) , . . . , p ( i max ) .

Lemma 1.1 (Theorem 4.1 from [14]) . If every null p-value is super-uniform, equiv., p i ∼ /a80 0 i ≽ U ( 0 , 1 ) for all i ∈ H 0 , and the collection is jointly independent, then regardless of the collection of alternative distributions { /a80 1 i } i ∈H 1 ,

Remark 1.2 . In fact, the assumption of joint independence of the null p-values can be relaxed to a form of dependency known as PRDS - see [7] for this generalization of Lemma 1.1.

1.4 The Adversary and the c-Perturbation Problem

We model an ( omniscient ) adversary with knowledge of H 0 , H 1 , and that knows the decision maker's choice of control level q . The adversary receives the p-values p = ( p i ) N i = 1 after they are generated but before they are received by the decision maker, or before test time. Given the ability to perturb c ≥ 1 p-values, the adversary solves

where p ′ denotes the p-values derived from an adjusted collection of p-values p ′ = ( p ′ i ) N i = 1 . In words, the adversary finds the perturbation of at most c -many p-values before the execution of BH q so as to maximize the adversarially-adjusted false detection proportion FDP [ BH q , p ′ ] . Perturbation of p-values is implicitly the result of data perturbation, and we refer to both Remark 4.6 and Section 5.2 for examples and experiments involving direct perturbation of data.

While we assume omniscience for the adversary throughout, we will briefly address modifications in analysis for an oblivious adversary that has no knowledge of H 0 , nor H 1 . Indeed, our algorithms to be presented can be modified naturally for implementation by an oblivious adversary (see comments in Section 3); further, there is nearly equivalent performance when π 0 is large, as is typical. Hence, it is for the sake of brevity that we omit explicit analysis of the oblivious adversary.

1.5 Main Results

Intuitively, the effect of a small number of p-value perturbations becomes insignificant in settings where a large number of tests are rejected (see Theorem 3.2). This happens, for instance, when either the number of tests N , or the control level q , or the distance (e.g. KL-divergence) between the null and alternative distributions, is large. For this reason, we focus on results that are non-asymptotic in the number of tests N .

In Section 3, we present the algorithm INCREASE-c that uses strategic increases to c null p-values to induce expansion of the BH rejection region. We also present an efficient, optimal algorithm MOVE-1 (Appendix Section 7.2) for the adversary's maximization of FDR with at most one (i.e. c = 1 ) p-value perturbation.

In Section 4 we discuss the adversarial robustness of BH through study of the adversarial adjustments to FDR by the INCREASE-c algorithms, revealing where its control is and isn't adversarially robust.

In Section 5 we provide accompanying numerical experiments on i.i.d. as well as PRDS p-values.

2 BH as Balls into Bins

In this section we will establish important notation for the discussions to follow. We reduce the real line to a collection of N + 1 'bins". N 0 and N 1 balls will each be assigned to one of these bins independently of each other, from discrete distributions that are specified in the next subsection. The main motivation for this reduction is to facilitate discussion of effective perturbations in Section 3, and for the technical analysis in Section 4.

2.1 The Balls into Bins System

We partition the segment [ 0 , 1 ] into N equiprobable segments that will be referred to as bins . We define the i -th bin B i ∶= { p ∈ ❘ ∶ ( i -1 ) q N ≤ p < i q N } , for i = 1 , . . . , N. What remains forms bin N + 1 , i.e., B N + 1 ∶= { p ∈ ❘ ∶ 1 ≥ p ≥ 1 -q } . For shorthand, we write B 1 ∶ i ∶= ∪ i l = 1 B l = { p ∈ ❘ ∶ 0 ≤ p < i q N } , Finally, we write B 0 i ∶= ∣ B i ∩{ p j } j ∈H 0 ∣ and B N i ∶= ∣ B i ∩{ p j } j ∈N ∣ for bin i 's, respectively, null-load and total load . The alternative- load B 1 i is defined analogously, as are B 0 1 ∶ i B N 1 ∶ i , and B 1 1 ∶ i .

Rejection Count: Borrowing terminology from the classic balls into bins problem of probability theory [29], this framework facilitates a re-interpretation of the random drawing of p-values as balls being randomly placed into an ordered collection of bins, enumerated 1 up to N + 1 . Framed in this way, we see that BH q operates by identifying the rejection count

❩ which corresponds to the largest collection of consecutive bins 1 , . . . , i that collectively contain precisely i balls, so that BH q rejects all tests with p-values lying in the first ˜ k bins. The case ˜ k = 0 corresponds to rejecting no tests. In fact, ˜ k is a stopping time under a filtration F that we define next.

Filtration F = {F i } N i = 0 : Let Ω ∶= [ 0 , 1 ] N be a sample space with the classical Borel σ - algebra B and (with slight abuse of notation) probability measure /a80 ∶= (⊗ i ∈H 0 U ( 0 , 1 )) ⊗ (⊗ i ∈H 1 /a80 1 i ) . We define a filtration beginning with F N ∶= σ ( B N N + 1 , B 0 N + 1 ) , and continuing inductively ( N towards 0 ), let F i be the σ -algebra generated by { B N j } N + 1 j = i + 1 and { B 0 j } N + 1 j = i + 1 . In words, this filtration corresponds to what is cumulatively learned about the bin loads (null and total) upon examination of the bins in sequence starting with bin N + 1 and concluding with bin 1 , assuming each observed p value comes with correct identification of whether or not i ∈ H 0 .

B 0 1 ∶ N N , B 0 1 ∶ N -1 N -1 , . . . , B 0 1 ∶ 1 1 form a martingale sequence adapted to the filtration. This fact will prove useful when combined with the optional stopping theorem to facilitate several results in this work.

Under this lens, the adversary's (algorithmic) task reduces to reshuffling p-values among the bins, and in Section 3 we demonstrate there are indeed simple, tractable ways of performing this to manipulate

BH, with potentially great effect on FDR control. The key insight is that there exist alternative stopping times that present alternative rejection counts (/regions) that can break FDR control.

3 Adversarial Algorithm: INCREASE-c

Throughout this section, we don the role of the adversary and study the c-Perturb problem, in which we are given q ∈ ( 0 , 1 ) , a realized collection p = { p i } i ∈N (along with the labels of null or alternative for each p i ), and a budget c ≥ 1 , and our task is to produce a perturbed collection p ′ . Toward this, we focus on a procedure called INCREASE-c that despite its sub-optimality (see Appendix Section 7.2) is intuitive and simple to execute; further, as theoretical and empirical analysis in sections 4 and 5 respectively show, it has strong performance in expectation.

We begin by defining a random variable that is a stopping time adapted to F ; given an integer c ≥ 1 , let and we choose to write ˜ k + in place of ˜ k + 1 .

Increasing the Rejection Count The interest in ˜ k + c is that if we moved any selection of c null p-values from bin N + 1 into bin ˜ k + c (in fact any bin i ≤ ˜ k + c ), then BH q would output a new, increased rejection count ˜ k + c . We formally study this in Section 4. In the meantime, we comment on the increase ˜ k + c -˜ k , which is a difference between two stopping times

It is easy to see that ˜ k + c -˜ k ≥ c whenever B 0 N + 1 ≥ c ; hence, the increase in the rejection count is at least c , but possibly more. We provide a stronger lower bound on this increase by utilizing the ratio between the number B 0 ˜ k + 2 ∶ N of nulls not rejected by BH q and the number N -( ˜ k + 1 ) of bins left outside of the BH q rejection region in the case of no corruption. Computational experiments indicate comparable performance of this bound with those of simulations presented in Section 5's Table 1.

Theorem 3.1. If c ≥ 1 , then

for any collection of alternative hypothesis distributions { /a80 1 i } i ∈H 1 .

INCREASE-c runs as follows:

IF B 0 N + 1 ≥ c , then move the largest c (ties broken arbitrarily) in the (N+1)-th bin to bin ˜ k + c .

ELSE leave the p-values unperturbed.

It in fact suffices for the c -many p-values to be placed in any bin i ≤ ˜ k + c . We remark that since an oblivious adversary cannot discern null-drawn from alternative-drawn in the collection p , INCREASEc as written is unimplementable in such a case. Hence, for the oblivious adversary, we modify INCREASE-c's criterion to B N N + 1 ≥ c and have the oblivious adversary now take the c -many p-values uniformly at random from among the p-values in the ( N + 1 ) -th bin. Intuitively, this modification for the oblivious adversary should yield nearly c null p-values being moved (on average) just as in the non-oblivious case, assuming the proportion of nulls among the B N N + 1 - many p-values is high, as a typically large π 0 would entail.

We conclude this section with a characterization of the average increase in FDR, denoted ∆ c that INCREASE-c induces.

Theorem 3.2. Given c ≥ 1 , let p + c denote the perturbed form of p that INCREASE-c produces. Then the adversarially-adjusted FDR induced by INCREASE-c is

for any collection of alternative distributions { /a80 1 i } i ∈H 1 , where

In Section 4 to follow, we provide analytical lower bounds for ∆ c as part of a discussion on BH's adversarial robustness. Section 5 presents computational experiments (e.g. Table 1). We remark that INCREASE-c is not optimal for all instances of c-Perturb; indeed, for c = 1 , we present a provably optimal algorithm MOVE-1 in Appendix Section 7.2, which in contrast to INCREASE1 sometimes induces a reduced rejection count. However, INCREASE-c remains a formidable adversarial procedure, as the results of Section 5 demonstrate on not only i.i.d. p-values but also PRDS conformal p-values.

4 Theoretical Analysis: Performance Guarantees and Insights into Adversarial Robustness

BH's FDR control - Lemma 1.1 - is (distributionally) robust in the sense that it holds no matter the alternative distributions { /a80 1 i } i ∈H 1 . However, as Theorem 3.2 indicates, the degree to which this control can withstand data perturbations at test time, i.e., its adversarial robustness, very much depends on { /a80 1 i } i ∈H 1 .

Recalling Lemma 1.1, we may assume without loss of generality that no alternative distribution /a80 1 i stochastically dominates U ( 0 , 1 ) (equiv., /a80 1 i ≽ U ( 0 , 1 ) ). That being said, the 'degree" to which the alternative distributions { /a80 1 i } i ∈H 1 are (stoch.) dominated by the null distribution U ( 0 , 1 ) (equiv., /a80 1 i ≼ U ( 0 , 1 ) ) is critical. We briefly preview two regimes of special interest for which each of the next two subsections cover.

High sub-uniformity: When the alternatives are sub-uniform /a80 1 i ≼ U ( 0 , 1 ) for all i ∈ H 1 , and highly so, such that for all i ∈ H 1 it holds that /a80 1 i ( p i < ϵ ) ≈ 1 for some small ϵ > 0 , then it follows that ˜ k is large and B 1 ∶ ˜ k should contain most alternative p-values. Consequently, in order for INCREAES-c to induce any sizeable increase to the FDR, the adversary will need to expand the BH rejection region significantly so as to introduce a commensurate number of nulls. Table 1 indicates c may need to be quite large to make a dent in FDR control. This message is made more precise in Theorem 4.1.

Low sub-uniformity: As we will see, when the alternative p-values are barely dominated by U ( 0 , 1 ) , ∆ c can be rather large. In fact, in the special case that there is no dominance such that /a80 1 i = U ( 0 , 1 ) for all i ∈ H 1 , a strikingly vulnerable state occurs with high probability. Indeed, in this case where nulls and alternatives are virtually indistinguishable, BH q (in fact any A ) admits an FDR of π 0 whenever any rejections are made (i.e., ❊ [ FDP [ BH q ; p ]∣∣ ˜ k ≥ 1 ] = π 0 ) so that BH q accordingly compensates by making no rejections with high probability ( /a80 ( ˜ k = 0 ) = 1 -q ), which follows by the distributional robust control (1) from Lemma 1.1. But those times when ˜ k = 0 is precisely when INCREASE-c's simultaneous expansion of the rejection region and injection of nulls into this region is most damaging. That this event and other similarly vulnerable events occurs with high probability is the fault of the distributional robustness. This message is made rigorous in the forthcoming Theorem 4.5.

4.1 Case of High Sub-Uniformity in Alternatives { /a80 1 i } i ∈H 1

If /a80 1 i ≼ U ( 0 , 1 ) for all i ∈ H 1 , with /a80 1 i ( p i < ϵ ) ≈ 1 for some small ϵ > 0 , then it is clear that the number of alternatives rejected by BH q should be nearly the maximum number N 1 of correct rejections possible (i.e., B 1 1 ∶ ˜ k ≈ N 1 ) with high probability, limiting any potential impact of INCREASE-c.

The following bounds are formulated to elaborate on such dynamics in this case of large separation between alternatives and nulls, for which the event [ B 1 1 ∶ c = N 1 ] has probability close to 1.

Theorem 4.1. If c ≥ 1 , then

and

In words, for fixed c, as the alternative distributions concentrate more and more on 0 , it follows that /a80 ( B 1 1 ∶ c = N 1 ) ↑ 1 , so that the effect ∆ c of INCREASE-c on BH's FDR is dampened. And this occurs despite the fact that the increase ˜ k + c -˜ k in rejection count produced by INCREASE-c consists of mostly the introduction of nulls, and tends to a magnification of ( N 1 + c ) by at least a factor of the inverse of 1 -❊ [ B 0 1 ∶ N N ∥ B 0 1 ∶ N ≤ N 0 -c ] , which is straightforward to compute since B 0 1 ∶ N ∼ Binom ( N 0 , q ) . We refer the reader to Section 5's Table 1 for simulations illustrating the above.

4.2 Case of Low Sub-Uniformity in Alternatives { /a80 1 i } i ∈H 1

In the study of this regime, we aim to demonstrate that the adversarial increase ∆ c can be rather large. We provide a lower bound L c on ∆ c that will be a function of parameters q, N, N 0 , as well as the alternative distributions { /a80 1 i } i ∈H 1 .

4.2.1 Lower Bounding ∆ c

In view of 5 , the event [ ˜ k + c = c ] is clearly of significance in the computation of ∆ c . In words, this event describes the case that the BH q rejection region captures nothing, and yet when the adversary successfully executes INCREASE-c the rejection will now capture only nulls - generating a false detection proportion of 1. Hence, we lower bound the adjustment ∆ c of Theorem 3.2 by lower-bounding the probability of this event.

Our strategy: (1) first, characterize this probability under the special case that /a80 1 i = U ( 0 , 1 ) for all i ∈ H 1 ; (2) second, to handle when /a80 1 i ≼ U ( 0 , 1 ) for some i ∈ H 1 , we translate the KL divergence of the resulting discrete, bin-assignment distributions into a bound via Pinsker's Inequality.

The key to the first step will be to recognize that when /a80 1 i = U ( 0 , 1 ) for all i ∈ H 1 , the vector of total loads ( B N 1 , . . . , B N N ) is exchangeable (i.e., its law is invariant under permutations), given B N 1 ∶ N . Indeed, exchangeability, combined with the generalized Ballot Theorem of [31] yields the following result:

Armed with Corollary 4.2, we can begin to estimate the probability laws of ˜ k and ˜ k + c .

Next, we carry out the second step of our plan. Towards this, we make the following assumption. Assumption 4.4. Suppose the alternative p-values are independent and identically distributed, with common distribution /a80 1 , i.e., { p i } i ∈H 1 iid ∼ /a80 1 . We write δ ∶= ( 1 -q ) -/a80 1 ( p i ∈ B N + 1 ) , and δ j ∶= /a80 1 ( p i ∈ B j ) -q N , and we assume that /a80 1 ( p i ∈ B j ) > 0 for all j ∈ [ N ] , as well as /a80 1 ( p i ∈ B ℓ ∶ N ) > 0 for arbitrary ℓ < N .

This assumption not only reduces the notational burden (otherwise documenting N 1 many alternative distributions) but it also allows the leveraging of KL divergences between the different bin assignment distributions implied by the alternative /a80 1 versus the null U ( 0 , 1 ) .

Theorem 4.5. Suppose Assumption 4.4. Then

Remark 4.6 . For a more concrete application/example, consider when the p-values { p i } i ∈N are derived from z-scores { z i } i ∈N via p i = /a80 ( Z > z i ) , where Z ∼ N ( 0 , 1 ) , with null z-scores { z i } i ∈H 0 i.i.d. N ( 0 , 1 ) and alternative z-scores { z i ∼ N ( µ i 1 , 1 )} i ∈H 1 (with all µ i 1 ≥ 0 ). Indeed, under this framework, we may view the µ i 1 as the distance between /a80 1 i and U ( 0 , 1 ) .

Towards satisfying Assumption 4.4, we can assume throughout that there exists a nonnegative parameter µ 1 such that the alternative parameters ( µ i 1 , σ i ) = ( µ 1 , 1 ) for all i ∈ H 1 . Indeed, if every member of the collection { µ i 1 } i ∈H 1 is in reality only close to zero, then the forthcoming estimates (lower bounds) would provide close, conservative estimates when setting µ 1 = max { µ i 1 ∶ i ∈ H 1 } .

when the alternative distributions are 'far" (

µ

1

>>

when they are 'close" (

µ

1

=

0

0

) from the theoretical null, and the opposite holds

) - see Figures 2 and 3.

5 Simulations and Data Experiments

In this section, we provide computations (performed in R and Python on a Macbook Air-M2 chip, 8GB memory, with no experiment time exceeding 5 minutes) to demonstrate the performance of the adversarial algorithm INCREASE-c. We demonstrate its performance through simulation on synthetic data to make comparisons to the theoretical estimates provided in Section 4. We then demonstrate its performance on a real-data experiment in outlier detection.

5.1 INCREASE-c Simulations on Synthetic Data

5.1.1 INCREASE-c Simulations on i.i.d. p-values

Following the framework from Remark 4.6, we simulated 10 4 replications of the following experiment: (1) N = 1000 p-values are generated, with { p i } i ∈N 0 iid ∼ U ( 0 , 1 ) , and each p i among { p i } i ∈N 1 generated via p i = 1 -Φ ( z i -µ 1 1 ) , with { z i } i ∈N 0 iid ∼ N ( µ 1 , 1 ) ; (2) FDP [ BH q ; p ] and FDP [ BH q ; p + c ] are calculated. In Figure 1 each of the 10 4 - many ( FDP [ BH q ; p ] , FDP [ BH q ; p + c ] ) pairs are plotted. As can be seen, the vast majority of the pairs satisfy FDP [ BH q ; p + c ] > FDP [ BH q ; p ] , and, further, all pairs lie above the horizontal line situated at the level of the BH control level π 0 ⋅ q = . 09 , i.e, FDP [ BH q ; p + c ] > π 0 ⋅ q.

In Table 1, we present the effectiveness of INCREASE-c over ranges of corruption budget c and µ 1 from small to large. As can be seen, when µ 1 = 0 , any amount of corruption budget c yields large post-corruption FDR ❊ z FDP [ BH q ; z + c ] . When µ 1 > 0 grows, however, the budget c must correspondingly grow in order for there to be nontrivial post-corruption FDR. Finally, we note that for any fixed c , the increase in rejection count ˜ k + c -˜ k is on the average larger when µ 1 is larger. For experiments on non- i.i.d., PRDS p-values, we refer the interested reader to Section 5.2 or Appendix Section 7.4.

5.1.2 INCREASE-1 Simulations versus Theoretical Bounds Under Small µ 1

In Figures 2 and 3 we illustrate how well the insights discussed in Section 4.2 capture the sensitivity of BH to adversarial perturbations when µ 1 is near 0. Specifically, for each q in the grid { . 01 , . 02 , . . . , . 99 } , we computed the difference FDP [ BH q ; p + ] -FDP [ BH q ; p ] across 10 3 replications of the setup as in Section 5.1.1, with the average of this difference being an estimate of ∆ 1 . The plot of these ∆ 1 estimates with respect to q is then compared with the plot of our lower bound L 1 as a function of q.

Figures 2 and 3 illustrate that the stricter the control, i.e., the smaller q is, the more effective INCREASE-c becomes. In fact, the bound indicates high vulnerability for the typical use case

Simulation of INCREASE-10

N 0=900,  N 1=100, q = 0.10,

m

1=1,

s

1=1, # Simulations  = 1e4

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.00

0.25

0.50

0.75

1.00

BH FDP before Adversary

BH FDP after Adversary

Frequency

250

500

750

1000

versus L。 when μ= O, To = .90

1.00

0.75 -

FDR Adjustment

0.50 =

0.25 -

0.00 -

0.00

0.25

0.50

0.75

1.00

Lc

Control Level q

△。versus L。 when μ= 0.25,To = .95,c = 1

1.00

0.75 =

Proportion

False Detection P

0.50

二

c

Le

0.25 =

0.00 =

0.00

0.25

0.50

0.75

1.00

Control Level q

of q ∈ ( 0 , 0 . 10 ) . On the other hand, as q increases, INCREASE-c's effect weakens; however, for increased q, the decision maker is accepting a higher FDR already.

5.2 Real-Data Experiment: Credit Card Fraud Detection

The Credit Card 2 dataset D ∶= {( X i , Y i )} n i = 1 ⊆ ❘ 30 × { 0 , 1 } contains n = 284 , 807 credit card transactions in September 2013 by European cardholders over the course of two days 492 of which were frauds. Each X i ∈ ❘ 30 consists of numerical input variables that are the result of a PCA transformation. The 'Class" label Y i takes value 1 in case of fraud and 0 otherwise (/genuine), yielding the partition [ n ] = n 0 ⊍ n 1 , with ∣ n 0 ∣ = 284 , 315 and ∣ n 1 ∣ = 492 .

Fraud Detection Experiment

Given a set of unlabeled transactions { X i } i ∈ S , where S ⊆ [ n ] , the fraud detection task is to identify which members of S belong to n 1 , i.e., are fraudulent. We experiment with the BH-based, outlier detection method of [5] on this fraud detection task - specifically, we consider the false detection proportion of this method in the absence and presence of an adversary. The details of the experimental setup will now be discussed.

Training

We begin by training an unsupervised decision-tree-based algorithm on a training set. From the set n 0 of genuine transactions, we uniformly at random select a subset n train ⊆ n 0 of size 141 , 758 to form a training set D train ∶= { X i } i ∈ n train upon which we train an isolation forest [25] ˆ s ∶ ❘ 30 → ❘ + using the R library isotree 3 , where, in principle, ˆ s ( X i ) returns an isolation depth that is smaller if Y i = 1 (i.e. is an outlier) and larger if Y i = 0 (i.e. is an inlier)

Calibration and (Adversarially-Perturbed) Testing

Then for each of 10 2 simulations, we uniformly at random selected a subset ˜ n cal ⊆ n 0 ∖ n train of size 141 , 657 to form a calibration set ˜ D cal ∶= { X i } i ∈ ˜ n cal of strictly genuine transactions. As well, we uniformly at random selected a subset ˜ n test, 1 ⊆ n 1 of 100 fraudulent transactions to append to the 900 remaining genuine transactions comprising ˜ n test, 0 ∶= n 0 ∖ n train ∖ ˜ n cal to form a test set ˜ D test ∶= { X i } i ∈ ˜ n test , where ˜ n test ∶= ˜ n test, 0 ∪ ˜ n test, 1 . Finally, we transformed X i ↦ ˜ p i ∈ ( 0 , 1 ) for each i ∈ ˜ n test via ˜ p i = 1 +∣{ j ∈ ˜ n cal ∶ ˆ s ( X j )≤ ˆ s ( X i )}∣ ∣ ˜ n cal ∣ . The resulting collection of conformal p-values ˜ p ∶= ( ˜ p i ) i ∈ ˜ n test is PRDS, as proven in [5], and hence the FDR control of Lemma 1.1 holds (see [7]). In contrast, upon executing INCREASE-c to generate a corresponding adversarially-perturbed collection ˜ p + c ∶= ( ˜ p + c,i ) i ∈ ˜ n test , we obtain a collection for which BH's FDR control no longer holds.

Experimental Results

We executed BH 0 . 1 , on both ˜ p and ˜ p + c , with an execution providing the decision for each i ∈ ˜ n test whether to report it as genuine (null) or fraudulent (alternative). We report the average (over the 10 2 simulations) false detection proportion (FDP) produced by BH 0 . 1 , i.e., both E [ FDP [ BH 0 . 1 ; ˜ p ]] and E [ FDP [ BH 0 . 1 ; ˜ p + c ]] (for c = 1 , 5 , 10 ). As well, we report the average number of alleged frauds ❊ [ ˜ k ] and ❊ [ ˜ k + c ] . As Table 2 indicates, although the method of [5] can ordinarily control the

FDR below the explicit 0 . 10 level, INCREASE-c has the ability to push the FDR above this control level.

6 Conclusions

This is the first work to consider adversarial corruption of the popular Benjamini Hochberg multiple testing procedure to break its FDR control. While BH may exhibit robustness when the alternative distributions are 'far" from the null, it exhibits great sensitivity in practical cases when the alternatives are 'closer" to the null. In such cases, with the modification of few p-values (as few as one), the attacker can increase the expected FDR well past the guarantee stipulated by the BH procedure. This study suggests some caution may be necessary when using BH, especially in safety-security settings. Numerical experiments support the analytical results. Finally, BH is but one member of the family of step-up multiple testing procedures, which generally entail rejection regions decided via a stopping time, which since our paper shows can be manipulated in the case of BH, it means other step-up procedures can be similarly prone.

Acknowledgments and Disclosure of Funding

We wish to thank Wang Chi Cheung for a stimulating conversation. This work was supported by the Air Force Office of Scientific Research (Mathematical Optimization Program) under the grant: 'Optimal Decision Making under Tight Performance Requirements in Adversarial and Uncertain Environments: Insight from Rockafellian Functions'.

References

Idan Amir, Idan Attias, Tomer Koren, Yishay Mansour, and Roi Livni. Prediction with corrupted expert advice. Advances in Neural Information Processing Systems , 33:14315-14325, 2020.

Haris Angelidakis, Konstantin Makarychev, and Yury Makarychev. Algorithms for stable and perturbation-resilient problems. In Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing , pages 438-451, 2017.

Pranjal Awasthi, Avrim Blum, and Or Sheffet. Center-based clustering under perturbation stability. Information Processing Letters , 112(1-2):49-54, 2012.

Maria-Florina Balcan, Nika Haghtalab, and Colin White. K-center clustering under perturbation resilience. ACM Trans. Algorithms , 16(2), mar 2020.

Stephen Bates, Emmanuel Candès, Lihua Lei, Yaniv Romano, and Matteo Sesia. Testing for outliers with conformal p-values. The Annals of Statistics , 51(1):149-178, 2023.

Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) , 57(1):289-300, 1995.

Yoav Benjamini and Daniel Yekutieli. The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics , 29(4):1165-1188, 2001.

Daniel Berend, Aryeh Kontorovich, Lev Reyzin, and Thomas Robinson. On biased random walks, corrupted intervals, and learning under adversarial design. Annals of Mathematics and Artificial Intelligence , 88:887-905, 2020.

Sujay Bhatt, Guanhua Fang, Ping Li, and Gennady Samorodnitsky. Minimax m-estimation under adversarial corruption. In Proceedings of the 39th International Conference on Machine Learning (ICML), Baltimore, MD , 2022.

Yonatan Bilu and Nathan Linial. Are stable instances easy? Combinatorics, Probability and Computing , 21(5):643-660, 2012.

Shuchen Cao, Ruizhi Zhang, and Shaofeng Zou. Adversarially robust sequential hypothesis testing. Sequential Analysis , 41(1):81-103, 2022.

D.Hendrycks and K.Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In Proceedings of International Conference on Learning Representations , 2017.

Ilias Diakonikolas and Daniel M Kane. Algorithmic high-dimensional robust statistics . Cambridge university press, 2023.

Bradley Efron and Trevor Hastie. Computer age statistical inference, 2013.

Rui Gao, Liyan Xie, Yao Xie, and Huan Xu. Robust hypothesis testing using wasserstein uncertainty sets. Advances in Neural Information Processing Systems , 31, 2018.

Ian Goodfellow, Patrick McDaniel, and Nicolas Papernot. Making machine learning robust against adversarial inputs. Commun. ACM , 61(7):56-66, jun 2018.

Anupam Gupta, Tomer Koren, and Kunal Talwar. Better algorithms for stochastic bandits with adversarial corruptions. In Conference on Learning Theory , pages 1562-1578. PMLR, 2019.

Ying Jin and Emmanuel J Candès. Selection by prediction with conformal p-values. Journal of Machine Learning Research , 24(244):1-41, 2023.

Yulu Jin and Lifeng Lai. On the adversarial robustness of hypothesis testing. IEEE Transactions on Signal Processing , 69:515-530, 2021.

Akshay Krishnamurthy, Thodoris Lykouris, Chara Podimata, and Robert Schapire. Contextual search in the presence of adversarial corruptions. Operations Research , 71(4):1120-1135, 2023.

K. Lee, K. Lee, H. Lee, and J. Shin. A simple unified framework for detecting out-ofdistribution samples and adversarial attacks. In NeurIPS , 2018.

K. Lee, K. Lee, H. Lee, and J. Shin. Training confidence-calibrated classifiers for detecting out-of-distribution samples. In International Conference on Learning Representations , 2018.

Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems , 31, 2018.

Zishuo Li, Yilin Mo, and Fei Hao. Game theoretical approach to sequential hypothesis test with byzantine sensors. In 2019 IEEE 58th Conference on Decision and Control (CDC) , pages 2654-2659. IEEE, 2019.

Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In 2008 eighth ieee international conference on data mining , pages 413-422. IEEE, 2008.

Thodoris Lykouris, Vahab Mirrokni, and Renato Paes Leme. Stochastic bandits robust to adversarial corruptions. In Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing , pages 114-122, 2018.

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations , 2018.

Akshayaa Magesh, Venugopal V Veeravalli, Anirban Roy, and Susmit Jha. Principled out-ofdistribution detection via multiple testing. Journal of Machine Learning Research , 24(378):1-35, 2023.

Martin Raab and Angelika Steger. 'balls into bins'-a simple and tight analysis. In International Workshop on Randomization and Approximation Techniques in Computer Science , pages 159170. Springer, 1998.

Weijie J Su. The fdr-linking theorem. arXiv preprint arXiv:1812.08965 , 2018.

Lajos Takács. A generalization of the ballot problem and its application in the theory of queues. Journal of the American Statistical Association , 57(298):327-337, 1962.

Weiwei Liu Xinsong Ma, Xin Zou. A provable decision rule for out-of-distribution detection. Proceedings of the 41st International Conference on Machine Learning (ICML), Vienna, Austria , 2024.

7 Appendix / supplemental material

7.1 Adversarial Algorithm: INCREASE-c

Theorem 3.1. If c ≥ 1 , then

for any collection of alternative hypothesis distributions { /a80 1 i } i ∈H 1 .

Proof. Since the statement in the case of c = 1 is trivially true, we henceforth assume that c > 1 . Recall ˜ k + c = max { i ∈ [ c, N ] ❩ ∶ B 1 ∶ i = i -c } . Let us define ˜ k 0 + c ∶= max { i ∈ [ c, N ] ❩ ∶ B 0 1 ∶ i = i -( B 1 1 ∶ ˜ k + c )} . Then we begin by establishing that ˜ k + c ≥ ˜ k 0 + c > ˜ k + 1 .

To see why this holds, we note that B 0 1 ∶ ˜ k + 1 = ˜ k -B 1 1 ∶ ˜ k + 1 = ˜ k + 1 -( B 1 1 ∶ ˜ k + 1 + 1 ) > ˜ k + 1 -( B 1 1 ∶ ˜ k + 1 + c ) , and this implies ˜ k + 1 < ˜ k 0 + c . Further, B 1 ∶ ˜ k 0 + c = B 0 1 ∶ ˜ k 0 + c + B 1 1 ∶ ˜ k 0 + c = ˜ k 0 + c - ( B 1 1 ∶ ˜ k + c ) + B 1 1 ∶ ˜ k 0 + c = ˜ k 0 + c -c +( B 1 ˜ 0 -B 1 ˜ ) ≥ ˜ k 0 + c -c, which implies ˜ k + c ≥ ˜ k 0 + c .

Next, we justify the relation

It suffices to establish it for any fixed, joint realization of ˜ k, B 1 1 ∶ ˜ k that occurs consistent with the event [ B 0 N + 1 ≥ c ] with positive probability. With slight abuse of notation, we will continue to use ˜ k, B 1 1 ∶ ˜ k for such a fixed realization, and write ¯ ❊ [⋅] for ❊ [⋅∥ ˜ k, B 1 1 ∶ ˜ k , [ B 0 N + 1 ≥ c ]] . The plan is to apply the Optional Stopping Theorem on a martingale sequence. To do so, let us form a (backwards-running) filtration: let F N be the sigma-algebra generated by the event [ B 0 N + 1 ≥ c ] as well as the random variable B 0 N + 1 , and let F i be the sigma-algebra generated by the event [ B 0 N + 1 ≥ c ] as well as the random variables { B 0 j } N + 1 j = i + 1 . Then for any integers k, ℓ ∈ { N,N -1 , . . . 1 } such that ℓ > k ,

measurable function of the random variables in F ℓ . In other words, the collection { ˜ k + 2 ∶[ k ∨( ˜ k + 2 )] [ k -( ˜ k + 1 )]∨ 1 } forms a martingale under the filtration. As well, ˜ k 0 + c is a stopping time. Hence, by the Optional Stopping Theorem,

We note that the martingale property could be established even without the event [ B 0 N + 1 ≥ c ] in the filtration but for the sake of forthcoming computations we have included it.

Next we observe that

we derive

Hence,

It follows that

where we have used (conditional) Jensen's Inequality - valid because c > 1 and (*) ensures 1 -❊ [ B 0 ˜ k + 2 ∶ N N -( ˜ k + 1 ) ∣∣ ˜ k, B 1 1 ∶ ˜ k , [ B 0 N + 1 ≥ c ]] > 0 .

Theorem 3.2. Given c ≥ 1 , let p + c denote the perturbed form of p that INCREASE-c produces. Then the adversarially-adjusted FDR induced by INCREASE-c is

for any collection of alternative distributions { /a80 1 i } i ∈H 1 , where

Proof. We derive

as desired.

7.2 MOVE-1

MOVE-1 is an efficient, optimal algorithm for an (omniscient) adversary with budget c = 1 . We begin with several small insights en route to describing MOVE-1 in full.

Perturbing the Rejection Count

If p ′ = p , i.e., the sample statistics are left unperturbed, we obtain a false detection proportion of FDP [ BH q ; p ] = B 0 1 ∶ ˜ k ˜ k . If it is possible to improve on this, it can be easily shown that in the case of c = 1 we must induce an altered rejection count

❩ henceforth referred to as the (adversarially) adjusted rejection count resulting from the adversary's choice of p ′ . As it turns out, for the special case of c = 1 , ˜ k ′ ≠ ˜ k is necessary if a larger FDP is to be obtained, as the following lemma indicates.

Proof. We suppose the contrary for the sake of a contradiction, i.e., ˜ k ′ = ˜ k . It follows that

and then by FDP [ BH q ; p ′ ; ] ≠ FDP [ BH q ; p ] , it follows that

Since ∣∣ p -p ′ ∣∣ 0 ≤ 1 , these two conclusions are at odds, presenting a contradiction.

Considering Lemma 7.1, the adversary's p ′ ∈ [ 0 , 1 ] N decision simplifies to deciding on an adjusted rejection count ˜ k ′ from among a constrained set of integers consistent with the constraint ∥ p ′ -p ∥ 0 ≤ 1 . An optimal ˜ k ′ can indeed be larger or smaller than, or even equal to ˜ k. Hence, towards understanding this new search space, it suffices to characterize the set of feasible ˜ k ′ that are larger than ˜ k , and smaller.

Increasing the Rejection Count (c = 1)

Towards understanding how to increase the rejection count, we first highlight the following fact that follows immediately from (2).

Proof. Suppose there exists i ∈ { ˜ k + 1 , . . . , N } such that B N 1 ∶ i > i -1 . If B N 1 ∶ i = i , then i > ˜ k presents a contradiction of (2). However, proceeding with B N 1 ∶ i ≥ i + 1 , we see that there necessarily exists j ∈ { i + 1 , . . . , N } for which B N 1 ∶ j = j , for if this were not the case, then B N 1 ∶ j ≥ j + 1 for all j ∈ { i + 1 , . . . , N } , meaning B N 1 ∶ N ≥ N + 1 , yet another contradiction since there are only N p-values.

Consequently, for ˜ k ′ > ˜ k, we require a perturbed collection p ′ for which bin counts increase. This necessitates a decrease of a p-value - see Lemma 7.5 for details. Hence, we define

and note that these are precisely the positions i for which the addition of one (and only one) p-value into B 1 ∶ i through the decrease of a p-value brings i into candidacy for the new rejection count - see equation (2).

Proof. To prove the first statement, we recall that by Lemma 7.2, if i > ˜ k, then B N 1 ∶ i ≤ i -1 . So, if i ≥ ˜ k + 1 with B N 1 ∶ i < i -1 , then no decrease of a single p-value could increase B N 1 ∶ i to i (Lemma 7.5), precluding the possibility of ˜ k ′ = i. In other words, no i ∉ L is achievable for the new rejection count ˜ k ′ .

On the other hand, say we enumerate L with

For the case of i L , ˜ k ′ = i L is achieved if and only if a p-value is decreased from any bin B j with j > i ℓ to a bin B j -s where i L ≥ j -s ≥ 1 . For the case of i ℓ , ˜ k ′ = i ℓ is achieved if and only if a p-value is decreased from any bin B j with i ℓ + 1 ≥ j > i ℓ to a bin B j -s where i ℓ ≥ j -s ≥ 1 . Finally, all these movements of p-value just described are always possible for any given p = { p i } i ∈N .

Decreasing the Rejection Count (c = 1)

Analogously, it follows that the increase of a p-value is necessary for the decrease of the rejection count - see Lemma 7.6, and we define

where i ∗ ∶= 0 ∨ max { i ∈ { 1 , . . . , ˜ k -1 } ∶ B N 1 ∶ i = i + 1 } . We note that R is precisely the set of all the achievable new (decreased) rejection counts ˜ k ′ < ˜ k we could induce with the perturbation of a single p-value.

Proof. We begin by proving the first statement that ˜ k ′ < ˜ k implies ˜ k ′ ∈ R . To do so, we proceed in two steps. First we show that ˜ k ≥ i ∗ , so that i ∗ ≤ ˜ k ′ ≤ ˜ k -1 . Then we show that if ˜ k ′ = i for some i ∈ { i ∗ + 1 , . . . , ˜ k -1 } in which B N 1 ∶ i ≠ i , we arrive at a contradiction.

To see that ˜ k ′ ≥ i ∗ , if i ∗ > ˜ k ′ , then B N 1 ∶ i ∗ = i ∗ + 1 becomes B N 1 ∶ i ∗ ≤ i ∗ -1 by Lemma 7.2; in other words, the change in magnitude of B N 1 ∶ i ∗ is at least 2, which contradicts Lemma 7.6. Next, if i ∈ { i ∗ + 1 , . . . , ˜ k -1 } , then B N 1 ∶ i ≤ i by the definition of i ∗ . This means if B N 1 ∶ i ≠ i , then B N 1 ∶ i < i , so that Lemma 7.6 indicates ˜ k ′ could not be i .

As for the second statement, let R be enumerated

For the case of i R , ˜ k ′ = i R is achieved if and only if a p-value is moved from bin B j with ˜ k ≥ j > i R to a bin B j + s > ˜ k . For R -1 ≥ r ≥ 2 , by Lemma 7.6 it holds that ˜ k ′ = i r is achieved if and only if a p-value is moved from a bin B j with i r + 1 ≥ j > i r to a bin B j + s with j + s > ˜ k . Finally, for the case of i 1 , ˜ k ′ = i 1 = i ∗ is achieved if and only if a p-value is moved from B j with i ∗ ≥ j to a B j + s with j + s > ˜ k . Finally, all these movements of p-values just described are always possible for any given p = { p i } i ∈N .

It follows that ˜ k ′ ≠ ˜ k if and only if ˜ k ′ ∈ L ⊍ R , so an efficient, optimal search procedure becomes straightforward. Informally, we iterate over the bins in reverse order, beginning with N + 1 and terminating with i ∗ . At iteration (/bin number) i , if i ∈ L ⊍ R , then the trivial subproblem max p ′ ∶∣∣ p -p ′ ∣∣ 0 ≤ 1 , ˜ k ′ = i FDP [ BH q ; p ′ ] is solved; otherwise, nothing is done. Upon termination, the best FDP encountered is the answer. This is summarized in Theorem 7.7

Lemma 7.5 (Decreasing a p-value) . Let p ′ be such that ∣∣ p -p ′ ∣∣ 0 ≤ 1 . If p ′ is the decrease of a single p-value in p from B j to B j -s , where j ∈ { 2 , . . . , N + 1 } and 1 ≤ s ≤ j -1 , then

i ∈ { 1 , . . . , j -s -1 } /Leftrightline⇒ B N 1 ∶ i remains constant

i ∈ { j -s, . . . , j -1 } /Leftrightline⇒ B N 1 ∶ i increases by 1

i ∈ { j, . . . , N + 1 } /Leftrightline⇒ B N 1 ∶ i remains constant.

Lemma 7.6 (Increasing a p-value) . Let p ′ be such that ∣∣ p -p ′ ∣∣ 0 ≤ 1 . If p ′ is the increase of a single p-value in p from B j to B j + s , where j ∈ { 1 , . . . , N } and 1 ≤ s ≤ N + 1 -j , then

i ∈ { 1 , . . . , j -1 } /Leftrightline⇒ B N 1 ∶ i remains constant

i ∈ { j, . . . , j + s -1 } /Leftrightline⇒ B N 1 ∶ i decreases by 1

i ∈ { j + s, . . . , N + 1 } /Leftrightline⇒ B N 1 ∶ i remains constant.

Theorem 7.7 (MOVE-1) . Let q ∈ ( 0 , 1 ) , p-values { p i } i ∈N and sets H 0 , H 1 be given. For each i ∈ L ∪ R , let

Then

This result explains that with one pass of the p-values from the largest to the smallest in the collection, we can ascertain the optimal perturbation p ′ . As the execution based on this result is straightforward, we omit the pseudocode for the sake of brevity.

In Table 3, we compare the average performance of INCREASE-1 against that of the optimal MOVE-1 over 10 4 simulations. The experiments followed the setup described in Remark 4.6, in which N = 10 3 p-values are derived from independently generated z-scores, with N 0 (= 900 ) null z-scores i.i.d. N ( 0 , 1 ) and N 1 (= 100 ) alternative z-scores i.i.d. N ( µ 1 , 1 ) , for µ 1 = 1 , 2 . As Table 3 indicates, INCREASE-1 can provide nearly identical performance in adjustment to FDR; however, the perturbation distance ∥ z -z ′ ∥ is on the average much greater than in MOVE-1.

7.3 Theoretical Analysis: Performance Guarantees and Insights into Adversarial Robustness Theorem 4.1. If c ≥ 1 , then

and

Proof. When B 1 1 ∶ c = N 1 , it easily follows that ˜ k + c ≥ c + N 1 + B 0 1 ∶ N 1 + c . This combined with Theorem 3.2 easily yields the first inequality.

As for the second inequality, let

implies that

so that

since whenever B 1 1 ∶ c = N 1 , it follows that ˜ k + c = ˜ k 0 + c .

Consequently, we derive

yielding

Corollary 4.2. Let n ≥ 1 , p ∈ [ 0 , 1 ] , and x a non-negative integer such that 0 ≤ x ≤ n . If ˜ B = ( ˜ B 1 , . . . , ˜ B n ) ∼ Multinomial ( x, ( p, . . . , p )) , then /a80 ˜ B (∩ n r = 1 [∑ r i = 1 ˜ B i < r ]) = 1 -x n .

Proof. We will appeal to the following result of [31]:

Lemma 7.8 (Theorem 1 of [31]) . Let there be n ≥ 1 non-negative integers z 1 , . . . , z n summing to x . If ˜ π denotes a permutation drawn uniformly at random, then

Let ˜ π be a random permutation on { 1 , . . . , n } that is drawn uniformly at random, independently of ˜ B . Since ˜ B = ( ˜ B 1 , . . . , ˜ B n ) ∼ Multinomial ( x, ( p, . . . , p )) , then ˜ B ˜ π ∶= ( ˜ B ˜ π ( 1 ) , . . . , ˜ B ˜ π ( n ) ) is equivalent in distribution to ˜ B , or ˜ B ˜ π D = ˜ B . Therefore,

Proof. The event [ ˜ k = ℓ ] is equivalent to {∩ N j = ℓ + 1 [ B N ℓ + 1 ∶ j < j -ℓ ]} ∩ [ B N 1 ∶ ℓ = ℓ ] , an intersection of two events. Regarding the event [ B N 1 ∶ ℓ = ℓ ] , because µ 1 = 0 , it is clear that /a80 ( B N 1 ∶ ℓ = ℓ ) = ( N ℓ )( qℓ N ) ℓ ( 1 -qℓ N ) N -ℓ . To complete the characterization of /a80 ( ˜ k = ℓ ) it suffices to find /a80 (∩ N j = ℓ + 1 [ B N ℓ + 1 ∶ j < j -ℓ ] ∣∣ [ B N 1 ∶ ℓ = ℓ ]) . Towards doing so, we note that conditioned on [ B N 1 ∶ ℓ = ℓ ] , we have

and, upon further conditioning on B N ℓ + 1 ∶ N ,

Hence, we derive

by Corollary 4.2. To conclude, we integrate out B N ℓ + 1 ∶ N from this derivation; more precisely,

As for the second statement, in similar fashion to above, Corollary 4.2 yields

implying

Hence,

Theorem 4.5. Suppose Assumption 4.4. Then

Proof.

Observe that

By Theorem 4.3,

where we now proceed to lower bound (#). The following discussion outlines how we proceed in the case of c = 1 , but this is without loss of generality.

Let B 0 N + 1 and B 1 2 ∶ N be given, along with the event [ B N 1 = 0 ] . Then there are N 0 -B 0 N + 1 many null p-values and B 1 2 ∶ N many alternative p-values each of whose assignment to one of bin number 2 , . . . , N remains stochastic. Let there be an arbitrary enumeration of these null p-values, upon which we let the collection of their bin-assignment random variables be denoted ( α 0 i ) N 0 -B 0 N + 1 i = 1 . Let there also be an arbitrary enumeration of these alternative p-values, upon which we let the collection of their bin-assignment random variables be denoted ( α 1 i ) B 1 2 ∶ N i = 1 . More precisely, α 0 i ∼ Unif ({ 2 , . . . , N }) , and α 1 i = j with probability q / N + δ j q + δ -( q / N + δ 1 ) , for j = 2 , . . . , N . Finally, we will write /a80 bins 1 for the joint distribution derived from the independent coupling of all the random variables ( α 0 i ) N 0 -B 0 N + 1 i = 1 and ( α 1 i ) B 1 2 ∶ N i = 1 . For contrast, let ˜ α 1 i ∼ Unif ({ 2 , . . . , N }) , which expresses the random bin assignment of any alternative p-value to one of B 2 , . . . , B N given that B N 1 = 0 , were /a80 1 = U ( 0 , 1 ) . Then /a80 bins 0 , the joint derived from the independent coupling of ( α 0 i ) N 0 -B 0 N + 1 i = 1 and ( ˜ α 1 i ) B 1 2 ∶ N i = 1 , offers a useful counterpoint to /a80 bins 1 . By the chain rule for KL-divergence, we find

∆

c

=

❊

[

c

˜

k

+

;

B

0

N

c

/a80

(

˜

k

+

c

=

c, B

0

N

)

c

.

c

+

1

+

1

and by Pinsker's Inequality, for any event E,

Let a 0 i (respectively a 1 i ) denote the realization of the i -th null (respectively alternative) p-value's bin assignment. Then if we set E to be the collection of realizations ( a 0 i ) N 0 -B 0 N + 1 i = 1 and ( a 1 i ) B 1 2 ∶ N i = 1 that are consistent with ∩ N j = 2 [ B 2 ∶ j < j -1 ] , we find

Hence, for the case of general c,

It follows that

To conclude,

where

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0

0.1

0.2

0.3

FDP[BH

q

; p]

FDP[BH

q

; p

+

c

]

Frequency

20

40

60

N 0=900,  N 1=100, q = 0.10,

a=1.5, # Simulations  = 1e3

Simulation of INCREASE-5 on Marginal Conformal P-values

7.4 Simulations and Data Experiments

7.4.1 INCREASE-c Simulations on PRDS p-values

In Table 4 we illustrate the effectiveness of INCREASE-c in disrupting the nonparametric outlier detection method of [5] that is based on the application of BH on conformal p-values, and in doing so, demonstrate effectiveness of INCREASE-c on PRDS p-values. We follow the simulation setting of Section 5.2 in [5], using their publicly available source code to generate the conformal p-values. In short, a data set is generated in ❘ 50 , along with 10 3 training observations used to fit a one-class SVM classifier, as well as 10 3 observations forming a calibration set to be used with a test set to derive (marginal) conformal p-values. In each of 10 3 independent replications, INCREASE-c was applied to a new test set consisting of 10 3 conformal p-values, designed to discern inliers (signals drawn from a mixture of multivariate gaussians with identity covariance matrices) from outliers (signals drawn from a mixture of multivariate gaussians with identity covariance matrices scaled by a strength √ a ). This was performed for a ∈ { 1 , 1 . 5 , 2 , 2 . 5 , 3 } ; a signal strength a = 1 corresponds to identical null and alternative distributions, while larger values of a make it easier to detect outliers. We set the fraction of outliers in each test set to π 1 = . 1 , so that a fraction π 0 = . 90 of the observations are inliers in each data set.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our abstract and introduction precisely summarize the results of our paper's content.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Yes, we discuss to what extent our analysis could be extended and further generalized, but were not done due to space constraints.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Our appendix contains the full proofs of all theoretical results.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: All such information is disclosed in our Experimental Results section.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: Experiments conducted were either with simulations on synthetic data, or a Kaggle data set in which the URL is provided. They are simple to set up, and we described them precisely. The codes we implemented were either our own scripts, or code that we cited from [5].

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Training of an isolation forest was done in Section 5.2. Training of a one-class SVM was also done in Appendix Section 7.4 when we implemented the simulations of [5] as executed with their publicly available code at: https://github.com/msesia/conditionalconformal-pvalues.git

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We found it unnecessary to report error bars as the purpose behind our simulations was to showcase/illustrate how large the false detection proportion could be made as a result of small perturbation. Further, the simulation averages were consistent with our theory.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: See Section 5

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We affirm that the research conforms to the code of ethics.

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: In the Introduction and Conclusion we discuss potential concerns for safetysecurity settings

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer:[NA]

Justification: [NA]

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We give credit to [5] for the code we implemented in our own experiments.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: [NA]

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: [NA]

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer:[NA]

Justification: [NA]

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nBenjamini-Hochberg provides a means for controlling the false detection rate (FDR) in multiple hypothesis testing.  This paper explores how the FDR region can be perturbed by an adversary.  Specifically, the author(s) propose two adversarial attacks *Increase-C* and *MOVE-1* to efficiently perturb *c* scores and 1 score, respectively.  The author(s) prove analytical bounds on their attacks' guarantees.  The author(s) also provide simulated experiments demonstrating their attacks effectiveness on synthetic z-scores.\n\nSTRENGTHS:\nI was not familiar with Benjamini-Hochberg before reading this paper (I did not bid on the paper).  However, with significant effort and outside reading, the paper was still attainable.  That is a strength of the work in general.  The *Balls in Bins* warm-up was definitely helpful to a reader.  I provide comments on how this warm-up could be improved below.\n\n### Novel Area and Good Execution\n\nTo the extent of my knowledge, no existing work has studied the adversarial robustness of methods to control FDR.  I discuss why I believe that is below in the limitations section.  Nonetheless, the authors provide reasonable first-pass attack methods that are intuitive and performant.  They even provide an optimal method in the case of singleton perturbations.  The authors also provide thorough theoretical analysis.\n\nWEAKNESSES:\n### Very Little Analysis of Section 5's Empirical Results\n\nSections 5.1 and 5.2 are essentially just an overview of the two experimental setups.  There is no real analysis or discussion of what the experimental results show or why it's important.  While Section 5.3 has some basic analysis, it is again quite minimal.  I understand that the page limit forces difficult decisions over content but so little in the way of commentary on the results is a disservice to the reader and overall a poor choice.\n\nI would have preferred empirical results on real datasets as opposed to the toy datasets in the paper, but I understand the author(s)'s choices here, especially given related work.\n\n### Move-1 Relegated to the Appendix\n\nBeyond a very definition in Section 1.5, all discussion of MOVE-1 including how the method works and why its optimal are relegated to the appendix.  There should at least a basic overview of the key ideas of MOVE-1 including some basic intuitions how it differs from INCREASE-c in the main paper. \n\n### Making the Work More Intuitive\n\nSection 2 in my view is a warmup for readers to help build intuitions and understanding of BH, even if they are not familiar with the related background. Section 2 could be significantly improved if figures were used to better illustrate the key ideas.  These figures need not be large and any additional camera ready page could be used to visualize the ideas.  Even if the authors chose not to include such figures in the main paper, they could be in the appendix.\n\n### Limited Potential Impact\n\nThere is undeniable merit and utility in studying the robustness of a method that provide tools to control FDR, such as Benjamini-Hochberg.  Nonetheless, this work falls essentially is a niche (BH) within a niche (adversarial analysis).  Hence, I view it as unlikely that this work will have a significant impact.\n\nQUESTIONS:\nNone.\n\nLIMITATIONS:\nThis paper describes two adversarial attacks and identifies the brittleness of Benjamini-Hochberg.  While such work inherently creates a non-zero risk, there are no serious negative societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-10"}

comments: SUMMARY:
This paper studies the adversarial robustness of the Benjamini-Hochberg (BH) procedure, introducing simple adversarial test-perturbation algorithms. The experiments show that BH's control can be significantly compromised with minimal perturbations. The analysis uses a combinatorial perspective and generalized ballot problems to derive non-asymptotic lower bounds.

STRENGTHS:
This paper addresses a novel and significant aspect of the BH procedure—its adversarial robustness. This research probelm is relatively unexplored, providing new insights into the vulnerabilities of widely used statistical methods.

WEAKNESSES:
This paper would benefit from a discussion of potential mitigation strategies that could enhance the robustness of the Benjamini Hochberg procedure against attacks such as INCREASE-c.

QUESTIONS:
Please refer to the weaknesses.

LIMITATIONS:
The authors' justification is "Yes, we discuss to what extent our analysis could be extended and further generalized, but were not done due to space constraints.". However, even in Appendix, there is not any analysis of limitations.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors have tested the Benjamini-Hochberg (BH) 's adversarial robustness, as this procedure is deployed in critical applications such as drug discovery, forensics, and anomaly detection. Specifically, the authors develop a class of simple and easily implementable adversarial test-perturbation algorithms to analyze under what conditions BH does and does not exhibit adversarial robustness. Next, to support their findings, the authors provide non-asymptotic guarantees on the expected adjustments to the FDR due to these adversarial attacks. Their technical analysis involves a combinatorial reframing of the BH procedure as a \"balls into bins\" process. It connects this to generalized ballot problems to utilize information-theoretic approaches for deriving non-asymptotic lower bounds. Finally, the authors also conducted experiments to support their findings.\n\nSTRENGTHS:\n- In this paper, the authors provided a detailed theoretical analysis of the BH procedure's robustness against adversarial attacks. They also introduced the INCREASE-c algorithm, which gives methodical mathematical bounds and probabilities for different cases (large and small alternative means).\n\n- I liked how advanced probabilistic and statistical tools, such as KL divergence and Pinsker's inequality, are used to provide a solid mathematical foundation for the analysis. The proofs and the lemmas were thoroughly justified.\n\n- Finally, the paper presentation is well-structured. The paper's organization is clear and logical. In each section, a comprehensive analysis is done and well-presented.\n\nWEAKNESSES:\n- In practical scenarios, an adversary manipulates the input directly, not the z-scores; thus, the insight into how perturbations applied to z-scores translate back to the original data samples or vice-versa is missing in the paper. \n\n- My other concern is that since z-scores are computed only utilizing the means and the standard deviation of the sample, the manipulation at the data level can get lost at the z-scores level.\n\nQUESTIONS:\n- In practical scenarios, an adversary manipulates the input directly, not the z-scores; thus, could the authors provide more insight into how perturbations applied to z-scores translate back to the original data samples or vice-versa? Or, when the inputs are perturbed, what will impact the proposed analysis of the BH algorithm (given the z-scores computed from the input samples)?\n\n- A statement on Page 4: \"A benefit of this approach is that perturbation at the level of z-scores places the corruption more directly at the point of data collection than does perturbation of the p-values.\" Could the authors answer: Won't the perturbation at the data collection directly place the corruption at the data level? Also, why was it not assumed to be corruption at the data level, then the computation of either the z-scores or the p-values? How this mapping between the data and the z-scores is computed?\n\n- The evaluation results are a bit difficult to navigate in the paper. For example, Figure 1 and Figure 4 are not understandable. Could the authors make the evaluation section comprehensible?\n\n- Could the authors add more evaluations to test how the proposed method performs with real-world data?\n\n\nPost-rebuttal: I have changed my rating from borderline reject to borderline accept, confidence from 3 to 4, and soundness of the paper from 2 to 3.\n\nLIMITATIONS:\nSame as Weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper explores the adversarial robustness of the Benjamini-Hochberg procedure. In particular, the authors theoretically show that it is possible to perturb test scores to cause the BH procedure to not be robust to adversarial attacks. BH is reframed as a \"balls into bins\" problem, and the authors propose an algorithm (INCREASE-c) to increase the rejection count. The authors also provide experiments to support their theoretical findings.\n\nSTRENGTHS:\n- The technical contribution seems solid and sound.\n- There is a clear novelty aspect, as the adversarial robustness of BH has not been considered before.\n- The authors show an interesting gap between distributional robustness (which was shown to hold (ref [29] in the paper)) and adversarial robustness for BH procedure.\n- The (synthetic) statistical experiment supports the theoretical claims.\n\nWEAKNESSES:\n- In the introduction, the paper (correctly) highlights the importance of hypothesis testing to various safety applications such as OOD detection. However, the experimental section does not, in any capacity, consider an end-to-end application such as OOD detection. In fact, it reads purely as a statistical simulation (which is fine as a synthetic experiment). But a minimal experiment with a MNIST/FashionMNIST OOD-classifier, to show that indeed the Z-scores can be affected by INCREASE-c could make the paper much stronger.\n- The attack is dependent on the algorithm (BH) used for FDR -- stronger results in adversarial robustness are often model/algorithm-independent.\n- There seems to be a certain lack of justification for choices in the problem setup (addressed in questions below).\n\nQUESTIONS:\n- Problem setup questions:\n\t- What is the rationale for the attacker having access to z-scores at that stage of the pipeline? Usual studies consider attackers having access to data, and it may take quite a lot of changes in data points to change a z-score.\n\t- Why is the attacker's knowledge of $q$ realistic? I assume that in some models, the deployment will come with a (public) guarantee for $q$ -- but this should be at least mentioned.\n\t- It would be good to have a comment justifying the choice of perturbation ( $||\\cdot||_0$ ), which I suppose is because BH is an algorithm for order statistics. What happens when considering a different type of budget (e.g., arbitrary $||\\cdot||_p$ norms)?\n- l.128-132: When are these assumptions met in practice?\n- What is the impact of attacking BH procedure on the whole OOD system? Can you break OOD detection in a meaningful way?\n\n\n****\n\nPost-rebuttal: I have increased my confidence score from 2 to 3 following the authors' response and other reviews.\n\nLIMITATIONS:\nI think some of the problem setup questions above would address some limitations of the work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-26"}