e-GAI: e-value-based Generalized $\alpha$-Investing for Online False Discovery Rate Control
Authors: Yifan Zhang
Zijian Wei
Haojie Ren
Changliang Zou
Venue: ICML 2025 poster (2025)
Keywords: online multiple testing
generalized $\alpha$-investing
e-value
false discovery rate
Primary area: probabilistic_methods->everything_else

Abstract: Online multiple hypothesis testing has attracted a lot of attention in many applications, e.g., anomaly status detection and stock market price monitoring. The state-of-the-art generalized $\alpha$-investing (GAI) algorithms can control online false discovery rate (FDR) on p-values only under specific dependence structures, a situation that rarely occurs in practice. The e-LOND algorithm (Xu & Ramdas, 2024) utilizes e-values to achieve online FDR control under arbitrary dependence but suffers from a significant loss in power as testing levels are derived from pre-specified descent sequences. To address these limitations, we propose a novel framework on valid e-values named e-GAI. The proposed e-GAI can ensure provable online FDR control under more general dependency conditions while improving the power by dynamically allocating the testing levels. These testing levels are updated not only by relying on both the number of previous rejections and the prior costs, but also, differing from the GAI framework, by assigning less $\alpha$-wealth for each rejection from a risk aversion perspective. Within the e-GAI framework, we introduce two new online FDR procedures, e-LORD and e-SAFFRON, and provide strategies for the long-term performance to address the issue of $\alpha$-death, a common phenomenon within the GAI framework. Furthermore, we demonstrate that e-GAI can be generalized to conditionally super-uniform p-values. Both simulated and real data experiments demonstrate the advantages of both e-LORD and e-SAFFRON in FDR control and power.

e-GAI: e e-value-based Generalized a-Investing for Online False Discovery Rate Control
Yifan Zhang' Zijian Wei
Haojie Ren
Changliang Zou?
Abstract
1. Introduction
Online multiple hypothesis testing has attracted lot of attention in many applications, e.g-. anomaly status detection and stock market price monitoring. The state-of-the-ar generalized investing (GAI) algorithms can control online false discovery rate (FDR) p-values only un- der specific dependence structures, a situation that rarely occurs in practice. The e-LOND al- gorithm (Xu Ramdas, 2024) utilizes e-values achieve online FDR control under arbitrary dependence but suffers from significant loss in power as testing levels are derived from pre- specified descent sequences. To address these limitations, we propose a novel framework on valid e-values named e-GAL. The proposed e-GAl can ensure provable online FDR control under more general dependency conditions while im- proving the power by dynamically allocating the testing levels. These testing levels are updated not only by relying on both the number of previous rejections and the prior costs, but also, differing from the GAI framework, by assigning less a- wealth for each rejection from a risk aversion per- spective. Within the e-GAI framework, we intro- duce new online FDR procedures, e-LORD and c-SAFFRON, and provide strategies for the long-term performance to address the issue of a- death, a common phenomenon within the GAI framework. Furthermore, we demonstrate that c-GAI can be generalized to conditionally super- uniform p-values. Both simulated and real data experiments demonstrate the advantages of both c-LORD and c-SAFFRON in FDR control and power.
The online multiple hypothesis testing problem arises from range of applications. For example, regulators record the number of NYC taxi passengers every 30 minutes, aim- ing to detect the anomalous intervals corresponding to spe- cial events (Lavin & Ahmad, 2015); economists build on- line monitoring procedures based on monthly stock market prices to identify bubbles in financial series (Genoni et al., 2023); industrial factories monitor machine operation sta- tus in real time for fault detection, thereby enabling early warnings for potential system issues (Ahmad et al., 2017). These different scenarios can all be formulated as the on- line multiple testing problem, which is concerned with the investigation of an online sequence of null hypotheses. At each time I, we must immediately make real-time decision on whether to reject the current hypothesis based on all the observed data far, without having knowledge of future data the total number of hypotheses.
Consider an online sequence of null hypotheses HI,..., Iil,,... Define 0g 0/1 if Iil, is true/false for each time and a class of online decision rules 6, = {ō,
... where 6, indicates that H, is rejected and 6, otherwise. is necessary to control the error rates of those decisions d, A natural quantity control is the false discovery rate (FDR) at target level as introduced by Benjamini & Hochberg (1995), that is, the ratio of falsely rejected nulls to the total number of rejections. The online FDR is defined as:
EjEno(e) Ög [E 5,) V1
FDR(() B[FDP
Sa,
where Ho(() = (j s t: 0:00 = 0} the true null set up to time
School of Mathematical Sciences, Shanghai Jiao Tong Univer- sity, Shanghai, China "School of Statisties and Data Sciences, LPMC, KLMDASR and LEBPS, Nankai University, Tianjin, China. Correspondence to: Haojic Ren <haojieren@ @ www.eeennsnuu..uu.......
Related works. Methods for online FDR control were pio- neered by Foster & Stine (2008), who proposed the so-called a-investing (AI) strategy. It was further extended by the generalized a-investing (GAI) procedure, which has served as the fundamental framework for online testing problem (Aharoni & Rosset, 2014; Javanmard Montanari, 2018). GAI deals with sequence of p-values P1, .......'e Pt.... and
Proceedings of the nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

e-GAT; e-value-based Generalized a- e-Investing for Online False Discovery Rate Control
encompasses wide class of algorithms assign the testing levels 01;---;, 04 in online fashion, effectively reject- ing the 6-th null hypothesis whenever Pt Ot (Ramdas et al., 2017; 2018: Tian & Ramdas, 2019). The GAI ensures provable online FDR control only for the p-values with in- dependence or positive regression dependence on a subset (PRDS; Benjamini & Yekutieli, 2001), which is rarely the case in practice; see definition of PRDS in Appendix B. In fact, many applications involve unknown complex de- pendence, such as time dependence in stock market prices. Hence, it is important to develop online multiple testing procedures under more general dependency conditions.
The e-GAl framework ensures online FDR control based conditional valid e-values with theoretical guarantees, which is achieved through new FDP esti- mator. In contrast to GAI, we propose a novel investing strategy named risk aversion investing (RAT) built on the new FDP estimator, enabling e-GAl to dynamically allocate testing levels based on both prior rejections and costs and assign less a-wealth for each rejection save budget.
Within the e-GAT framework, we propose two new algorithms called c-LORD and e-SAFFRON, Further- more, considering the long-term performance, we pro- pose corresponding algorithms, mem-e-LORD and mem---SAFFRRO, to address the issue of a-death, a common phenomenon in the GAI framework.
Xu & Ramdas (2022) proposed the SupLORD algorithm with the aim of false discovery exceedance control and dis- covered that, under a weaker baseline assumption, i.e., the null p-values are conditionally super-uniform as formalized in (1) in the subsequent text, it ensures valid FDR control at arbitrary stopping times. However, the SupLORD algorithm necessitates the selection of multiple parameters, which are intricately linked to its performance, and currently lacks well-established criteria for their optimal selection.
Moreover, the e-GAI framework can be generalized to conditionally super-uniform, p-values while preserving guaranteed FDR control. Numerical results demon- strate that the algorithms within the e-GAI framework are effective for online FDR control and achieve higher power compared to existing methods.
To deal with the problem introduced by dependence, another strategic direction is proposed by Wang & Ramdas (2022) to utilize e- e-values as potential alternatives to p-values as
We compare the e-GAI with several commonly used algo- rithms and summarize their characteristics in Table 1.
measures of uncertainty, significance, and evidence. The e- values have gained considerable attention, and many works have devoted significant effort to constructing valid e-values (Vovk & Wang, 2021; Ren & Barber, 2024; Li & Zhang, 2025) and applying e-values to ensure offline FDR control (Wang & Ramdas, 2022). In the online testing problem, related work is Xu & Ramdas (2024), which exploited e-values and proposed the e-LOND algorithm to provide online FDR control under arbitrary. possibly unknown, de- pendence. However, the e-LOND algorithm does not make full use of the entire error budget and assigns testing levels only by some pre-specified descending sequences related to the number of rejections. Hence, e-LOND yields conserva- tive FDR and sacrifices power, which hampers its practical use. Xu & Ramdas (2024) proposed to improve e-L.OND by incorporating independent randomization, though this operation only achieves a smaller improvement in power while introducing additional randomness.
2. Preliminaries
2.1. p-values & e-values
In this paper, the goal is to make real-time decision 8 while controlling online FDR at user-specific level a. The rejection decision 6, with p-values or e-values is defined as, respectively,
1 (pr S aah
if using p-values,
8.
1 1
e 2 1 if using e-values.
===0061.... 0661.... 61) the sigma-field time t, which is generated by historical decisions in the past. Here the testing level a, is required to be predictable at time t, that is ar is meeasurbll, i.e. Or E Ft
Denote F, =
In the studies of online testing, a valid p-value pt satisfies the conditionally super-uniform property under the null:
Therefore, a natural question is whether is possible to con- struct GAI-like framework based on e-values to achieve online FDR control under more general dependence, i.e., conditional validity in (2), while efficiently and effectively assigning testing levels to achieve high power.
P(p. S u F.-1) I u for all E [0, 1] if 0, = 0.
(1)
Meanwhile, a non-negative variable e, is a valid e-value if it satisfies the conditional validity:
Our contributions. To address this challenge, this pa- per proposes a novel framework based on e-values, named e- e-value- based generalized a-investing (c-GAI). Our contri- butions are summarized as follows:
Elex Fi-1] S 1 if = 0.
(2)
In contrast to the condition on the distribution of a p-value in (1), (2) only requires that the expectation of the e-value
2

---

e-GAI; e-value-based Generalized o- a-Investing for Online False Discovery Rate Control
Table 1. Online testing algorithms with their properties and performance of FDR control.
Framework
Algorithm
Statisties
Dependence conditions
O₂ relying prior costs
LORD++ (Ramdas et al., 2017)
p-value p-value
Independence PRDS
GAT
SAFFRON (Ramdas et al., 2018)
Independence PRDS
SupLORD (Xu & Ramdas, 2022)
p-value
(1)
e-LOND (Xu & Ramdas, 2024)
e-value
Arbitrary dependence
e-GAI
c-LORD
e-value
e-SAFFRON
e-value
exists and is bounded. This relaxed restriction provides greater flexibility in constructing valid e-values for various practical purposes (Vovk& Wang, 2021; Wang & Ramdas, 2022: Ren & Barber, 2024; Li & Zhang, 2025).
estimate by approximating the proportions of nulls:
(1) Ei=1 1(,2A)
FDP
SAFFRON
aj
1-A
(5)
E 61 V1
2.2. Recap: GAI
where (0,1) is a user-chosen parameter.
The GAI rules are capable of handling an infinite stream of hypotheses and incorporating informative domain knowl- edge into dynamic peeision-making process. Beginning with a pre-specified a-wealth, the key idea in GAI algo- rithms is that each rejection gains some extra a-wealth, which may be subsequently used to make more discoveries at later time points.
If the null p-values independent of each other and of the non-nulls, and {ar} J is chosen to be (coordinate-wise monotone function of 8,-1, then LORD++ and SAFFRON control the FDR at all times. Fisher (2024) considered the performance of LORD++ and SAFFRON and proved online FDR control when the popular PRDS condition (Ben- jamini Yekutieli, 2001) holds. However, the conditions of independence PRDS usually violated in practical applications.
Ramdas et al. (2017) provided a statistical perspective on online FDR procedures and proposed to design new algo- rithms by keeping an estimate of online FDP less than a. Specifically, Ramdas al. (2017) proposed an oracle ap- proximation of online FDP as:
3. e-GAI: e-value-based GAI
In this section, we first define the oracle estimate of FDP that tailored for e-values and demonstrate the theoretical results for FDR control (Section 3.1). We then design new investing strategy based on the proposed FDP estimator, and propose our testing algorithms, e-LORD (Section 3.2) and e- c-SAFFRON (Section 3.3), from the risk aversion per- spective to optimize the use of limited budget.
Ft) EjEHo(t) aj (E)=1 V1
(3)
This FDP*(t) overestimates the unknown FtP(t) and pro- vides guidance for online FDR procedures based inde- pendent p-values, including LORD++ (Ramdas et al., 2017), SAFFRON (Ramdas et al., 2018) and ADDIS (Tian & Ram- das, 2019) algorithms.
3.1. Online FDR Control with e-values
Suppose we observe valid e-values e ct... and make decision & = (et 2 1/ar} at each time t. Inspired by (3), we define an oracle e-value-based estimate of FDP and bound this overestimate to realize FDR control. Denote R² Σ 83 as the rejection size up t.
Specifically, LORD++ (Ramdas et al., 2017) realizes on- line FDR control by providing simple upper bound of FDP*(t):
LORD
Ej= ag
Theorem 3.1. Suppose the online e-values are valid in (2). Let the oracle e-value-base estimate of FDP be given
FDP
(t)
(4)
E 03 51
FDP
Σ
0i
(6)
If the proportion of alternatives is non-negligible, then - LORD
R--++1
SEHa(
LORD+ with FDP sults due to the overestimation of Ft)
(t) yields very conservative re-
E [FDP". S a, then FDR(() S foror all I.
Motivated by Storey-BEe (Storey, 2002), SAFFRON (Ram- das al., 2018) was derived from an adaptive upper bound
Theorem 3.1 provides a general theoretical result for FDR control with conditionally valid dependent e-values, guiding
3

---

e-GAT; e-value-based Generalized a- o-Investing for Online False Discovery Rate Control
and inspiring the construction of testing levels {ar}. which will be detailed in the following subsection. The proof of Theorem 3.1 and any other necessary proof' will be detailed in Appendix A.
time point and illustrates that the prior costs associated with each testing (investing) will affect the current test. The target FDR level is considered as the limited budget (a- wealth) of the entire testing procedure, and Proposition 3.2 reveals that it cannot be increased once the testing begins. Unlike GAI's updating strategy, we cannot compensate for a-wealth in the subsequent testing process based on the estimate of FDP in e-GAI, as the complex correlations make it diffieult to measure the future loss of one false discovery effectively. In contrast, we adopt risk aversion investing (RAI) strategy to update as follows.
Before pursuing further, we discuss the effect of the denomi- nator R + 1 of (t) in (6). To avoid the dependence inflating FDR, the denominator R--1+ + 1 of FDP; (t) plays an important role in "predicting" the number of possible future rejections at each rejection moment. As the true de- nominator R2 V 1 of FDP is unobservable at time - 1 and R + - (R, V 1) holds for each j E {e S t : 6e = 1}, we use R + as a substitute that serves as a (j- 1)- measurable lower bound for R, V 1. Since is placed in the denominator, this results in an overestimation of the oracle FDP, which can subsequently be leveraged achieve FDR control.
Intuitively, one may update testing levels by allocating a prescribed proportion of the remaining budget to satisfy the condition in Proposition 3.2. Therefore, we dynamically allocate testing levels as 01 ow and for t 2 2.
When complex dependence exists, the correlation between the number of false rejections and the total number of re- jections cannot be characterized, making it impossible to control their proportion. For instance, consider that the se- quential e-values are strongly positively correlated. When false rejection occurs, it indicates that the next e-value is likely to belong to the null but be falsely rejected as well, leading to an increased FDR. Hence, it implies that one can expect a quantity between R + 1 and Rg that provides a more efficient approximation for the denomina- tor of FtP (t) when knowing some specific dependence structure among e-values, which warrants further study for improving the efficiency of e- G-G Typically, the denom- inator of FDP in (6) can be directly chosen R2 for independent valuuus see details in Appendix B.5.
04 Wt Σ 1 +
aj
(Re (Be-1+
(8)
where wi....,wy We E (0, 1) control the proportion of the re- maining a-wealth allocated to the current testing. It's noted that a larger we indicates that more a-wealth is currently invested, which also implies a greater possibility of reject- ing the current hypothesis. However, since rejections are not able gain additional wealth and each test consumes proportion Wi of the remaining wealth, e-GAI views the entire testing process as risky investment. When failing to reject the current hypothesis, one may consider increasing the investment proportion W/ to encourage further testing. Upon hypothesis rejection (deciding to invest), this deci- sion carries both the risk of false discovery and induces a significant downward bias in the denominator of the FDP estimator earlier time points. As each rejection introduces new risks akin to an investment, we prioritize updating w, from the RAI perspective as
3.2. e-LORD
Inspired by the GAI framework, we design testing levels on by proposing upper bound for (6) to realize FDR control according to Theorem 3.1.
W(+1 = wi + WI 1p Ri Ri((1
(1- - 81) - wiih Re de
(9)
One natural overestimate of (t) is to define
t- R
R1
= =wi +
FDP.
LORD (t) Σ
03
R R1
ل 0, where w1 (0,1) is a user-defined initial allocation coefficient, and oo > 0, dh O are user-defined parameters that characterize the investment stimulation intensity post-acceptance and the risk regulation level post-rejection, respectively.
with convention > プー=
LORD
Since FDP. (t) 2 (t) any rejection rule al- gorithm assigning 0₁ in an online fashion such that FDPg (t) S a holds for all can control online FDR.
LORD
Remark 3.3. To ensure that wt E (0, 10, for each time t, we can select any W1 E (0, 0.5), & E [0, 0.5]. and se [0. 0.5). In fact, the conditions can be relaxed to ensure that wa is F, measuubae and Wt E (0. We suggest choosing W = O(1/T) with the total number of hypotheses to avoid spending too much wealth in the early stages while retaining sufficiently effective wealth for testing at each
Proposition 3.2. Suppose online e-values are valid in (2). LORD
For ai E F.-1 satisfying FDP.
(t) S a, we have
FDR(t) S for all
LORD
Note that FDP. (t) in (7) is constituted by the sum- mation of terms associated with both a, and R at each
4

---

e-GAT; e-value-based Generalized o- o-Investing for Online False Discovery Rate Control
Algorithm 1 e-LORD
2018), we further propose an adaptive estimate defined
I: Input: target FDR level a, initial allocation coefficient w1 (0, 1), parameters yp and v E (0, 1), sequence of e-values e <2, ...
1 1 {e < 1 x3
FDP. e
SAFFRON
Σ
dj R 1 X
aw1 and decide 81 = 1 { 2 a
2: Calculate
3: Update Ri = 51 and w2 by (9);
where {Ar is a predictable sequence of user-chosen parameters in the interval (0, 1). Here the term adaptive means that it is based on an estimate of the proportion of true nulls as in Storey (2002); Ramdas et al. (2018).
4: for = 2, 3.... do
5: Update testing level 0t by (8);
6: Make decision d,
2 1
De) f
7: Update R = R + 6e and Wi+1 by (9);
LORD In contrast to FDP.
8: end for
(t), the summation in (t) includes only those test levels associated SAFFRON
9: Output: decision set {ōi, J..
SAFFRON FDP.
with relatively small e-values. Although FDP. e is not necessarily always larger than FDP (t), we can SAFFRON
(t)
time point; more detailed discussions are provided in Ap- pendix B.2. Note that the choice of W; satisfying the above conditions does not affect the guarantee of the FDR control. This observation opens up greater flexibility, enhances the applicability of our algorithms,, and enables users to leverage domain knowledge for dynamically adjusting the allocation.
verify that E FDP.
(1) > E [FDP which is
sufficient for FDR control according to Theorem 3.1. The properties of the adaptive estimate are formalized below.
Proposition 3.4. Given a predictable sequence {>e if online e-values are valid in (2), then for Or E Fc-1 satisfy- ing FDP. c
The whole algorithm is referred to e-LORD and summa- rized in Algorithm 1. In e-LORD, the testing levels {ar) } are updated not only by relying on both the number of pre- vious rejections and the prior costs, but also by assigning less w-wthh for each rejection using the RAI strategy.
SAFFRON
(t) S a, we have:
SAFPRON
(a) E FDP. e
(1)
2 E[DP and
We find that the e-LOND algorithm proposed by Xu & Ramdas (2024) can be converted into the e-LORD al- gorithm. Let of aOLON and R RN-LOND denote the testing level and the number of rejections at time & in the e- LOND algorithm, respectively, Xu & Ramdas (2024) as- signed af L--LOND any (Rę_LOND + 1), where {7 is pre- specified non non-negative sequence summing to one. It can be LORD
(b) FDR(() S for tll
In the following, consider X À (0,1) for simplic- ity. Embracing the RAI principle, we propose an adaptive algorithm, called e-SAFFRON. The e-SAFFRON allocates testing levels as
a(1 - A)wi and for 2 2.
verified that FDP
(1) in (7) satisfies
a c
0q = We a(1 (al--a)
Σ
(Ro-1 + +1),
LORD FDP.
Rg-1 +1
Tt.
+1
j=1
(10)
where wi E (0,1) is updated by (9).
Furthermore, if choosing 7t = We 11 (1-w;) in e- LOND with }wi} in e-LODD, then we have a.-L.NTTT equals at in (8) for any /; refer to (18) in Appendix B.6 for more details. In this case, at each time f, the rejection set of e- LOND will be identical to the result of Algorithm 1. Thus, we can consider e-LOND as operating on a special type of e-LORD by designing Wi from given sequence }
We summarize e-SAFFRON in Algorithm 2. In particular, setting 0 in Algorithm 2 simplifies it to Algorithm 1, demonstrating that e-SAFFRON serves as the adaptive coun- terpart to e-LORD, similar to the relationship between SAF- FRON and LORD++ within the GAI framework.
Remark 3.5. Note that the choice of À will affect the total "wealth" a(1 - A). which will be no further increased in the subsequent period of C-FFFOOO.. We prefer a relatively small value À to preserve wealth, with À = 0.1 as the default choice in our numerical experiments, which differs from the value recommended in the SAFFRON procedure (Ramdas et al., 2018). The latter, SAFFRON with independent p- values, allows for additional rewards when a hypothesis is rejected and suggests À = 0.5.
3.3. e-SAFFRON
LORD To approximate (I). the estimate FDP. culated by summing all non non-negative terms over time. Thus, it serves as a crude and conservative overestimate if the proportion of alternatives is non non-neglinee negligibbe. Inspired by Storey-BH (Storey, 2002) and SAFFRON (Ramdas et al.,
(4) is cal-
5

---

e-GAT; e-valuc-based Generalized o- o-Investing for Online False Discovery Rate Control
Algorithm 2 e-SAFFRON
Theorem 4.1. Suppose the online e-values are valid in (2).
Input: target FDR level a, initial allocation coefficient
Let the oracle e-value-basee estimate of mem-FDP be
w1 (0, 1), parameters A, 4 and v (0, sequence of le-values e1, 62,-...
mem-FDP" (t)
E
jj
(11)
2: Calculate a1
a(1 A)wi and decide ò1
jEHo(E)
dR
J-1
+1
1 { 2 1
E [mem-FDP* (0) S a, then mem-FDR(() S a for all
3: Update R1 = ój and W2 by (9);
4: for 2.3..... ......do do
5: Update testing level 04 by (10);
Theorem 4.1 provides an oracle estimate of mem-FDP, and offers insights and guidance for designing algorithms that control mem-FDR. Note that (11) is facilitated by under- standing of the unknown denominator R, of the true mem- FDP. A natural choice that can serve as a - 1)-masuurable lower bound for predicting Ri is d' (dft)) + 1) since
6: Make decision 6,
2 1
7: Update R = R + óg and W+i1 by (9);
8: end for
9: Output: decision set {61. J.....}
this predicted value d' (dR) + I [R V 1) holds for each j with 5, =
4. Further Discussions on e-GAI
In this section, we further investigate the properties of the e-LORD and e-SAFFRÓN algorithms within the e-GAI framework.
mem-e-LORD & mem-e-SAFFRON. Adopting the core idea of the e-GAI framework, we can construct upper bounds for mem-FDP (t) in (11) and design the testing levels accordingly to achieve mem-FDR control.
4.1. Long-Term Performance
One natural overestimate of mem-FDP*(I)) is
We provide strategies for the long-term performance of our methods to address the issue of a-death, halting rejections once a-wealth tends to zero, a common phenomenon within the GAI framework (Ramdas et al., 2017).
(t) = Σ j=1
a2
mem- FDP
(12)
dR
and any algorithm is referred to as mem-e-L.ORD that allo-
cates testing levels {ax} satisfying mem- FDP LORD
a-death. In long-term testing process, there may be ex- tended periods during which no hypotheses are rejected, particularly when the true alternatives are rare, leading to continuous accumulation of the allocation proportion wi. As result, testing levels may become severely diminished in the later stages, making it difficult to achieve any fur- ther rejections. This phenomenon is referred to as a-death, which induces a loss of power and ultimately compromises the long-term efficacy of our online testing algorithm.
(t) <
a. As an example, adopting the RAI strategy, mem-e-LORD allocates testing levels as 01 awl and for 2 2.
(dR + 11), (13)
02 we
a
dR
+
where wi (0,1) is updated by (9).
When the proportion of alternatives is non-negligible, it is preferable to employ an adaptive overestimate defined as
mem-FDR control. To alleviate a-death over long pe-
riod (potentially infinite), Ramdas et al. (2017) defined de- caying memory FDR (mem-FDR) to allow more attention to recent rejections by introducing a user-defined decay param- eter d E (0, 1] and proposed mem-LORD++ that controls mem-FDR under independence. Specifically, mem-FDR is defined as
1 1fe, < 1
mem- FDP
SAFFRON
(t)
aj + 1 1-
J-1
where {Ae] satisfying A E (0,1) 1) is a predictable se- quence of user-chosen. We refer to an algorithm as mem- e-SAFFRON that allocates testing levels {ax) satisfying mem- FDP SAFFRON
Σ EjeHo(t) d' d'-18 E d' 18
(t) S a. For simplicity, we consider X = E E (0, 1) and employ nem-eeSAFFRRO from RAI perspective, allocating testing levels a1 a(1 A)w1 and for > 2,
meme FDR(F =
To address the issue, we adapt e-GAI and design mem-e- GAI to control mem-FDR in our setting. The technique used here is similar to the e-GAI framework to control FDR in Section Denote R. = Ej=1''' 15- for simplicity.
- af - < 1/1
a₁ w/ ((1-))-
(dRL + +1),
dRd カ-1 +
==1
6

---

e-GAT; e-value-based Generalized a- a-Investing for Online False Discovery Rate Control
and updating wi E (0, as in (9).
0.05
1.00
According to Theorem 4.1, both mem-e-LORD and mem-e- SAFFRON achieve mem-FDR control.
0.04
0.75
Power
Proposition 4.2. Suppose online e-values are valid in (2).
0.02
0.01
0.25
LORD
0.00
0.00
(a) For ar E Fi-1 satisfying mem- FDP we have mem-FDR(t) S for all t.
(t) S a
0.2
033
Tl:
e-LORD - e-SAFFRON + e-LOND
(b) Given a predictable sequence 1 for at E Fi-1
(a) e-valuc-based methods
SAFFRON
satisfying mem- FDP SAFFRON
(t) S a we have
E mem- FDP
(0)
> E [mem--DD"()]
0.09
and mem-FDR(t) S for all
FDR
0.06
0.03
0.75
4.2. Extension to p-values
0.00
0.00
01
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
0.5
While the e-GAI framework is initially developed based on the study of e-values, we demonstrate that it can also be generalized to p-values satisfying conditionally super- uniformity in (I), enriching the proposed framework and making the theory more comprehensive and complete.
T1
PL-RAI
P5-RAI
- LORD++ SAFFRON SUPLORD
(h) p-value-hased methods
Figure I. Empirical FDR and power with standard error versus proportion of alternative hypotheses for various online methods, with = 0.5, L = 30 and pte = 3.
Suppose we observe valid p-values p1..... .t.... and make decision &,- = [pr S ar} each time t. When using p-values for online testing, we demonstrate that FDP: (t) in (6) can still serve as an oracle estimate of FDP, in which the denominator involves the number of rejections RG relevant to p-values. By controlling this estimator to be bounded, we can achieve FDR control for p-values satisfying the conditionally super-uniform property in (1).
of mem-e-LORD and mem-e-SAFFRON through simu- lated numerical experiments in Appendix C.2. The code for all numerical experiments in this paper is available https:/ pttpss:/ggithn /ttp:://gtttbbbt// com/ zijianwei01/ GAI.
Theorem 4.3. Suppose the online p-values are condition- ally super-uniform in (I). If (I) in (6) satisfies E [[FDP,(t) S then FDR(t)= S for all t.
5.1. Simulation: Testing with Gaussian Observations
We use an experimental setup that tests the mean of a Gaus- sian distribution with the total number of data T = 500. The null hypothesis takes HI, f4 = O for each time f E [T]. The true labels 0g are generated from Bernoulli(T) (.1). The Gaus- sian variates vector fr E R² and covariance matrix Σ E R Tx The elements in μ = (H1 (J1....,, HTT) satisfy pr 0 6, and 14, = Me 0 0, 1, where fe is the signal param- eter. Additionally, the signals of the true alternatives correlated with the correlation coefficient p. The covariance matrix satisfies Σ > O, and Σij = plo- - fli- jl S L}. Note that the data at different time points will influence each other, which is in line with real-life online scenarios.
Theorem 4.3 provides general theoretical result for FDR control with conditionally super-uniform p-values. Building upon the analogous strategy outlined in Section 3 and Sec- tion 4.1, the e-GAI framework can be naturally extended to p-value-coppatible algorithms and corresponding versions for the long-term performance. For precise algorithmic differentiation, we denote the evalue-adapted variants of e-LORD and e-SAFTRON as pL-RAI and pS-RAL, respec- tively, emphasizing the adoption of p-vuuues as the test statis- ties and the dynamic updating mechanism of testing levels a through the RAI strategy. We provide more detailed discussions and technical explanations in Appendix B.4,
is from N(1 Σ) with mean
Under the normality assumption at each time point t, we compute the e-value as the corresponding likelihood-ratio statistic and the p-value by evaluating the conditional distri- bution. We take W/ 0.005, y = sb 0.5 in e-LORD and PL-RAI, and additionally À = 0.1 in e-SAFFRON and PS- RAI, while we use default parameters from the R package on1 DnlineFDR (Robertson et al., 2022) for other benchmarks. The target FDR level is set as a = 0.05.
5. Numerical Experiments
In this section, we evaluate the performance of our on- line testing framework on both synthetic and real data. We compare e-LORD, e-SAFFRON, PL-RA, and PS-RAI with e-LOND, LORD++, SAFFRON, and SupLORD in terms of FDR and power. We validate the performance
7

---

e-GAI; e-value-based Generalized a- o-Investing for Online False Discovery Rate Control
5.3. Real Data: Dating Financial Bubbles
Table 2. Proportion of points rejected out of anomalous regions. e-LORD, e-SAFFRON and e- E-LOND control the estimated FDP under a 0.1. e-SAFFRON loses some power due to a tiny proportion of true alternatives.
We follow Genoni et al. (2023), building a sequential test stock market prices for financial bubbles. Online testing procedures enable decision- making regarding bubble occur- rence based on current observation before the subsequent one is observed. Controlling FDR is proper guarantee to make decisions with a controllable proportion of mistakes.
Method
LOLOR
e-SAFFRON
e-LOND
FDP
0.085
0.087
0.061
Num Discovery
47
46
33
The analysis is performed on the monthly stock price of the Nasdaq series. The calibration is implemented by using the first 1/3 observations, assuming the related period to be freee of bubbles. Genoni et al. (2023) uses the standard p-values of the ADF test (R package urca), which are cal- culated point-wise and thus do not satisfy the conditionally supeeruniform property. Moreover, such a valid p-value is difficult to construct for time series. Building on Dickey & Fuller (1981). we reformulate the likelihood ratio for the unit root test as sequential e-values and further normalize them by the estimated conditional expectation under the null to guarantee conditional validity. The task is to identify the bubble beginning date (BBD) and bubble ending date (BED).
Figure show the results of various methods with p = 0.5 and Mee = 3. The empirical results show that SAFFRON will inflate the FDR heavily, while others realize the FDR control. However, there is no theoretical guarantee for con- trolling the FDR with complex dependent data in LORD++, which makes the rejection decisions not as safe as they seem. Owing to dynamically updating the testing levels, both e- LORD and e-SAFFRÓN lead to much higher power than e-LOND, and pL-RAI and pS-RAI lead to higher power than LORD++ and SuDLORD. Similar performance can be found for other settings, as presented in Appendix C.I.
5.2. Real Data: NYC Taxi Anomaly Detection
As shown in Figure 3, with significance level o = 0.005, e-LOND only detects the initial potential change associated with the gradual emergence of technology companies in the marketplace. The e-LORDD and e-SAFFRON exhibit similar behavior, providing a comprehensive characterization of the entire potential bubble influence region by marking a dense rejection region. The date 1990-01-20, corresponding to the onset of the bubble, has been identified, aligning with the classical view. The BED in Figure is determined based empirical experience due the long-term effects of the dot-com bubble burst. Specifically, while the burst of the dot-com bubble led to reset in valuations, NASDAQ's volatility remained elevated, exceeding pre-bubble levels and establishing the groundwork for subsequent financial cycles. As a result, this sustained volatility led to persistent rejections in online procedures even after BBD. The region detected by e-LORD also includes other historical events after BED.
We analyze the NYC taxi dataset from the Numenta Anomaly Benchmark (NAB) repository (Lavin & Ahmad, 2015). The dataset captures the number of NYC taxi pas- sengers every 30 minutes from July 1, 2014, to January 31. 2015. Five known anomalous intervals correspond to notable events such as the NYC marathon, Thanksgiving, Christmas, New Year's Day, and a snowstorm. We visualize the data with the known anomalous intervals highlighted using red rectangles in Figure 2. The anomaly detection problem is formulated an online multiple testing problem.
We employ the R package stlplus to perform STL de- composition (Cleveland et al., 1990) to remove the seasonal and trend components. We derive tests on the residuals, which are assumed to form an independent sequence. The first 2000 time points are taken as the initial sequence for model calibration. We focus on the comparisons among e- based methods and apply e-LORD, e-SAFFRON and e-I.OND to analyze this dataset. We use the estimated likelihood ratio as e-values (shown in (21)). We choose W1 = 0.0001 and À = 0.1 and set both &e and & as 0.5.
6. Summary
In this paper, we propose a novel framework named e-GAI that introduces an oracle estimate of online FDP and ensures online FDR control under conditional validity with theoreti- cal guarantees. The e-GAI dynamically allocates testing lev- els from the RAI perspective, which relies on both the num- ber of previous rejections and prior costs and assigns less a-wealth for each rejection. Within the e-GAI framework, we propose two new algorithms, e-LORD and e-SAFFRON, respectively. Both e-LORD and S-FFFOON are more pow- erful than e-LOND under complicated dependence. We also
We compare their performance in terms of the proportion of discoveries out of marked anomalous regions, denoted here as FDP, and the number of discovered anomalous regions in Table 2. Our e-GAl procedures effectively maintain FDP below the target level. As illustrated in Figure 2, e-LORD and C-SAFFRON demonstrate higher power than e-LOND, as both identify more points within the anomalous regions, shown in red squares. More comparisons for p p-value-base methods are shown in Appendix C.3.
8

---

e-GAI; e-value-based Generalized o- o-Investing for Online False Discovery Rate Control
Abnomalies marked L-LO procedure
40000
Remainder
20000
10000
Nox-01-2014
Nov-15-20 15-2014
Dec-01-2014
0ec-16-2014 Timestamp
Jon-01-2015
-80-15-2015
Feo-01-2015
Abnormalies marked by e -AFFFOO procedure
40000
Remainder
00000
20000
10000
0-
Nox- 21-2016
Nov- 10x-15-2014
010-011-0144
100-115-014 Timestamp
Jan Jen-01-2015 2015
2an-15-2015
Feo-01-2015
Abnomalies marked e-LOND procedure
40000
Remeinder
20000
10000
Nov-D1-2314
Nov-16-2014
Dec-D112014
Doc-16-2014
Jan-01-2015
-an- 2-2016
Foo-01-2016
Timestamp
Figure& 2. Anomaly points detected by e-LORD (above), e-SAFFRON (middle) and e-LOND (below). Rejection points of all procedures marked by dark blue points. Red regions refer known anomalies. The testing level is chosen 0.1. Red squares indicate additional discovery of c-LORD and e-SAFFRON compared to e-LOND.
c-SAFFRON provides an adaptive online FDR procedure, how to more accurately approximate the proportions of nulls like ADDIS (Tian & Ramdas, 2019) deserves further research. Secondly, as discussed in Section 3.1, more accurate approximation of FDP" (t) can be expected when knowing more specific dependence structure. It warrants future study on the relationship between the denominator of FDP: and the dependence structure, and how to design more efficient strategy to assign testing levels under such cases.
4000
Engins
Slook Pros
3000
0-
Date
Figure 3. Dating of BBD and BED in the Nasdaq series. Blue and red points refer to e-LORD and e-LOND rejection points under 0.005, respectively. Blue and red regions refer to bubble influence regions detected by e-LORD and e-LOND. Blue and red dashed lines refer to BBD and BED marked by the Nasdaq series. Significant historical events are marked.
A Acknowledgements
We sincerely thank the anonymous reviewers for their in- sightful comments and constructive suggestions, which have greatly improved the quality of this manuscript. Haojie Ren was supported by the National Key R&D Program of China the National Natural Sci-
(Grant No. 1024YFA1012200 ence Foundation of China (Grant No. 22471262) the Young Elite Scientists Sponsorship Program by CAST and Shang- hai Jiao Tong University 2030 Initiative. Changliang Zou was supported by the National Key R&D Program of China (Grant Nos. 2022YFA1003703 1022YFA1103800 and the National Natural Science Foundation of China (Grant No. 12231011).
propose mem-e-LORD and mem-e-SAFFRON correspond- ingly for the long-term performance to alleviate a-death. Moreover, we demonstrate that e-GAl can be generalized to conditionally super-uniform p-values, making e-GAI more versatile tool with reliable theoretical guarantees and increased practical value.
We conclude this work with two remarks. Firstly, although

---

e-GAT: e-value-based Generalized o- c-Investing for Online False Discovery Rate Control
Impact Statement
Li, G. and Zhang, X. A note on e-values and multiple testing. Biometrika, 112(1 :20aaes0000 2025.
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Ramdas, A., Yang, F., Wainwright, M. J., and Jordan, M. I. Online control of the false discovery rate with decaying memory. In Proceedings of the 31st International Con- ference on Neural Information Processing Systems, pp. 5655-5664, 2017.
References
Aharoni, E. and Rosset, S. Generalized a-investing: def- initions, optimality results and application to public databases. Journal of the Royal Statistical Society Se- ries B: Statistical Methodology, 76(4):771-794, 2014.
Ramdas, A., Zrnie, T., Wainwright, M., and Jordan, M. Saf- fron: an adaptive algorithm for online control of the false discovery rate. In International Conference on Machine Learning, pp. 9286-4294, PMLR, 2018.
Ahmad, S., Lavin, A., Purdy, S., and Agha, Z. Unsupervised real-time anomaly detection for streaming data. Neuro- computing, 262: 134-147, 2017.
Ren, Z. and Barber, R. Derandomised knockoffs: lever- aging e-values for false discovery rate control. Journal of the Royal Statistical Society Series Statistical Method- ology, 86(1):122-15 1544 2024.
Benjamini, Y. and Hochberg. Y. Controlling the false dis- covery rate: a practical and powerful approach to multiple testing. Journal of the Royal stutistical society: series B (Methodological), 57(1):289-300, 1995.
Robertson, D. S., Liou, L.,, Ramdas, A., and Karp, N. A. onlineFDR: Online error control, 2022. R package 2.5.1.
Shafer, G. Testing by betting: A strategy for statistical and scientific communication. Journal of the Royal Statistical Society Series A: Statistics Society, 184(2):407-433 2021.
Benjamini, Y. and Yekutieli, D. The control of the false discovery rate in multiple testing under dependency. The Annals of Statisties, 29(4): 165-1188, 2001.
Shafer, G., Shen, A., Vereshchagin, N., and Vovk, V. Test martingales, bayes factors and p-values. Statistical Sci- ence, 26 26(1):84-101, 2011.
Cleveland, R. B., Cleveland, W. S., McRae, J. E., Terpen- ning, I., al. Stl: A seasonal-trend decomposition. off. Star, 6(1):3-73, 1990.
Storey, J. D. A direct approach to false discovery rates. Jour- nal of the Royal Statistical Society Series B: Statistical Methodology, 64(3):479-498, 2002.
Dickey, D. A. and Fuller, W. A. Likelihood ratio statistics for autoregressive time series with a unit root. Econometrica, 19(4):1057-1072, 1981.
Tian, J. and Ramdas, A. Addis: an adaptive discarding algorithm for online fdr control with conservative nulls. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 9388-9396, 2019.
Fisher, A. Online false discovery rate control for lord++ and saffron under positive, local dependence. Biometrical Journal, 66(1):2300177, 2024.
Foster, D. P. and Stine, sequential control of expected false discoveries. Jour- nal of the Royal Statistical Society Series B: Statistical Methodology, 70(2):429-44) 444, 2008.
a-investing: a procedure for
Vovk, V. and Wang, R. E-values: Calibration, combination and applications. The Annals of Statistics, 49(3):1736 1754, 2021.
Genoni, G., Quatto, P., and Vacca, G. Dating financial bubbles via online multiple testing procedures. Finance Research Letters, 58: 28:104238, 2023.
Wang, R. and Ramdas, A. False discovery rate control with e-values. Journal of the Royal Statistical Society Series B: Statistical Methodology, (4(3):822-852 2022.
Javanmard, A. and Montanari, A. Online rules for control of false discovery rate and false discovery exceedance. The Annals of statistics, (6(2):52--554, 2018.
Xu, Z. and Ramdas, A. Dynamic algorithms for online multiple testing. In Mathematical and Scientific Machine Learning, pp. 955-986. PMLR, 2022.
Xu, and Ramdas, A. Online multiple testing with e-values. In International Conference on Artificial Intelligence and Statistics, pp. 3997-4005. PMLR, 2024.
I Lavin, A. and Ahmad, S. Evaluating real-time anomaly detection algorithms-tte numenta anomaly benchmark. In 2015 IEEE 14th international conference on machine learning and applications (ICMLA), pp. 38-44. IEEE, 2015.
10

---

e-GAT; e-value-based Generalized a- o-Investing for Online False Discovery Rate Control
A. Proofs
Here, we include all the proof' of the results throughout the paper.
A.1. Proof of Theorem 3.1
Proof. Suppose desired level a is given. For all time i, we have
FDR(() =E
EjEHo(t) 5j
(1)
E
Σ
6
E
Σ
1(e 1
(8))
S E
Σ
Ejaj
Rs V
R +1
R R
R
JEHo(()
(EHa(t)
(EHo(()
E leg F oj
(1i))
aj
(20) E [E(DP, Saa a,
E
S
E
R1
R
+1
(t)
jEHo(!)
where the inequality (i) holds since R +1 (R2 V1 for every (i St: dj 1] by definition, the inequality (ii)
holds since [ 2 1} S y for any y > 0, the inequality (iii) follows after taking iterated expectations by conditioning
on F1 and then applying the property of e-values, and inequality (iv) holds by condition. Thus, finish the whole proof.
A.2. Proof of Proposition 3.2
LORD FDP.
Proof. To prove the property, we only need to verify that E
(t)
2 E (tD) for all t, then we can obtain
LORD
the desired result by Theorem 3.1. This holds trivially since FDP.
(tP by construction.
A.3. Proof of Proposition 3.4
Proof. Given a desired level a and predictable sequence (Ae) for all 4
1 1(65
SAFFRON
t
Bg < x
E
FDP.
(t)
E
R 1 +1
1-
1=1
1 Lej= < }
Σ
E
0j
R5-1+1 +1
1 1-
(tHo((t)
E 1 [
< 1 }/ F5-1
Σ (tHo(tt)
ajj
R Rý-1+1 +1
1-
Σ
0jj
= ()))
R +
(EH)(t)
where inequality (i) holds because IE [e < +)
|
A by the property of e-values. This concludes the proof of part (a), and then (b) can be derived by Theorem 3.1, which completes the proof.

---

e-GAT: e-valuc-based Generalized a- o-Investing for Online False Discovery Rate Control
A.4. Proof of Theorem 4.1
Proof. Suppose desired level a is given. For all time we have
{)) d' 18
(
di
1f; 2 1 d +1
mcm-FDR(() =E
I
E
Σ
= E
R2 V1
d' (d + 1)
(iHHo(t)
(EHo((!
((0 S E
cjag
= E
Eie, 1 aj
(156)
E
Σ
aj
d.. +1
dR
-R +1
dR 1 +1
(EHo(())
(iEHo(t)
(EHo(t)
E [mem-FDP* (1)] (te)
where the inequality (i) holds since R = ==
d' d d' 10,0+>20 55,
d' 66 +E dd - and hence
(dRR 1) (R) V 1) for every j ft::: 1} by definition, the inequality (ii) holds since 1 2 1} y for any y > 0, the inequality (iii) follows after taking iterated expectations by conditioning on Fj-1 and then applying the property of e-values, and incquality (iv) holds by condition. Thus, we finish the whole proof.
A.5. Proof of Proposition 4.2
LORD
Proof. (a) To prove the property, we only need to verify that E mem- FDP
(1)
> E [mem- FDP (D) for all I, then
LORD
obtain the desired result by Theorem 4.1. This holds trivially since mem- FDP construction.
(t) mem- FDP* (f) by
(b) Given a desired level a and predictable sequence (A for all t.
160 < }
SAFFRON
E
mem-- FDP
(t)
E
dRd
1 1-1
1 [ < x5}
Σ
E
03
dRd
+1 1
1 1
jEHo(()
J-1
1 [ < } Fイ
E
Σ
jj
dR
+1
1- Aj
jEHo(t)
Σ
E 1immm-FDP*(h (D),
dRd
+1
jEHo(t)
[1 < x Fz
where inequality (i) holds because TF. have mem-FDR(t) < a for all
2 1 xg by the property of e-values. By Theorem 4.1, we
A.6. Proof of Theorem 4.3
Proof. Recall that the decision 5, (p, S ar) for each time t. Suppose a desired level is given. For all time t, we have
ESEROCO 5, Rí V1 2
(3)
83
1 (p S {a)) R +1 (t)
FDR(1) =E
E
E
Ry-1
(t)
E [Lipp S { FF
aj
(16) E EE(FDP
=E
S E
a,
R
Ry-1 +
(EH)(())
(EHa(()
12

---

e-GAT: e-value-based Generalized o- o-Investing for Online False Discovery Rate Control
where the inequality (i) holds since R5-1 + 1 (R2 1) for every j (j ::: 1] by definition, the inequality (ii) follows after taking iterated expectations by conditioning on F3-1 and then applying the conditionally super-uniform property of p-values, and inequality (iii) holds by condition. Thus, we finish the whole proof.
B. Deferred Discussions
B.1. Definition PRDS
Fisher (2024) introduced an online version of the well-known positive regression dependence on a subset (PRDS) condition proposed by Benjamini & Yekutieli (2001), considered the performance of LORD++ and SAFFRON, and proved online FDR control when the testing statistics, p-values, conditional PRDS.
Before formally defining PRDS, it is necessary to first introduce the concept of increasing sets. A set / E R is called increasing if x E I implies E I for all 2 x. Here 2 implies that each component of y is no smaller than the corresponding component of x.
Definition B.1. (Conditional PRDS between p-values; Fisher, 2024) The p-values are conditional PRDS if for each time t, any j I t satisfying j E Hu(t), and increasing set c R', the probability P ((Pi.....Pl E I Pj = u, F--) is non-decreasing in u.
Wang & Ramdas (2022) discussed the PRDS condition on the studies of e-values in the offline setting. We generalize it to the version applicable to the online scenario. A set D E RRK is called decreasing if implies y E / for all y S x.
Definition B.2. (Conditional PRDS between e-values) The e-values are conditional PRDS if for each time f, any j S satisfying E Ht)t), and decreasing set D C R', the probability P ((EI.....,.)) D eg u, F1-1) is non-increasing in
B.2. Choices for W1
In this section, we further elaborate on the motivation and advantage of updating w, in (9) from the RAI perspective. Building on this, we provide a theoretical justification for the recommended choice of wi O(1/T) and present supporting experimental results.
In the e-LORD and e-SAFFRON algorithms, Wi controls the proportion of the remaining a-wealth allocated to the current testing. A larger wt indicates that more a-wealth is currently invested, which also implies a greater possibility of rejecting the current hypothesis, and meanwhile, it will be more possible to exhaust the entire wealth. Therefore, we prioritize updating Wa from the RAI perspective, dynamically allocating the testing levels and enabling data-driven updates to achieve higher power. In contrast, the testing levels og in e-LOND are derived from pre-specified decay sequence that sums to I (Xu & Ramdas, 2024).
simplified version (9) is to set = 1b = and thus w. = w for all t. In this case, natural and reasonable way to choose W1 is to assign equal weight at each time point, i.e., W1 = 1/T, motivating the choice of initial value for dynamic updates.
Empirical results support this analysis. The power results for different choices of Wi across varying T under an AR(1) model, introduced in Appendix C.2, are shown in Table 3. From Table our algorithms with W1 = 1/TT achieve the highest power and have the latest time of the last rejection and the largest tail testing level, supporting potential subsequent long-term testing. Moreover, it can be seen that the updates in e-LORD and e-SAFFRON are data-driven: as 7 varies, the remaining wealth for these algorithms does not change significantly and shows robustness. In contrast, e-LOND uses a pre-specified allocation ratio, and as increases, the a-wealth at time T of c-LOND diminishes progressively.
B.3. Recursive Update Forms of e-LORD & e-SAFFRON
In this section, we provide recursive update forms of e-LORD and e-SAFFRON, respectively. The computation is highly efficient and memory- -eienryyfrienlly since the update of both Wt (expressed a recursive form in (9)) and a₁ can be expressed in a recursive form as follows.
13

---

e-GAI; e-value-based Generalized o- a-Investing for Online False Discovery Rate Control
Table Average results under an AR(1) model over 100 repetitions with Pec = 4, Ii 0.4, and 0.05.
Method
W1
Power (%)
Time of last rejection
@T/a(x100)
1/T
70.0
498
1031.0
e-LORD
1/VT
22.2
275
0.0
1/T4
8.6
483
0.7
500
1/T
70.5
498
1368.1
e-SAFFRON
1/VT
38.7
441
0.0
1/T2
8.0
483
0.6
e-LOND
30.9
491
1/T
70.1
998
1029.0
c-LORD
1/TT 1/T2
16.2
406
0.0
4.5
962
0.2
1000
1/T
70.9
998
1366.7
c-SAFFRON
1/VT
28.0
706
1/T"
4.2
958
0.2
c-LOND
23.9
983
1.0
To compute a of e-LORD in Algorithm 1, we define the remaining wealth as
Lw.-LORD = R1 +1
and update
a₁ We rw; LOORD CR + 1).
A similar recursive form of a, of e-SAFFRON in Algorithm 2 can be obtained by defining the remaining wealth as
r-1 rw, SAPPRON a(1 - -1) - Σ
ae </AA}
R--1+1
j=1
and update
a, We Iw,
-SFFFRON
(Ra-1 + 1).
Through these formulations, the update of testing levels a, is expressed as a recursive relationship based on information from the previous time step, allowing us compute it recursively and efficiently.
We evaluate the runtime of various algorithms in the experiments under an AR(1) model, introduced in Appendix C.2, and the results are included in Table 4. It shows that the e-LORRD and e-SAFFRON algorithms are computationally efficient.
Table 4. Average runtime of different algorithms under an AR(1) model over 100 repetitions.
e-LORD
e-SAFFRON
e-LOND
LORD+
SAFFRON
SupLORD
Runtime (x10- As)
9.7
18.8
18.0
9.9
8.1
72.1
B.4. Extension of the e-GAT Framework to p-values
In this section, we adapt the e-GAI framework to p-values and analyze the corresponding algorithms for the long-term performance following the same strategy in Section 3 and Section 4.1. Recall that the decision 6, = [p: S az} for each time t.
14

---

e-GAT: e-value-based Generalized a- a-Investing for Online False Discovery Rate Control
Extension of c-LORD & e-SAFFRON to p-values. Theorem 4.3 in the main text provides general theoretical result for FDR control with conditionally super-uniform p-values. Leveraging Theorem 4.3, both e-LORD and e-SAFFRON can be adapted to the corresponding version applicable to p-values by replacing (e 2 1/ar} with 1 [pr I a. Specifically, LORD
FDP- (1) in (7) and the update rule for at in (8) of the e-LORD algorithm can be directly adapted to p-values. The SAFFRON algorithm estimates the proportion of true nulls, requiring slight modification when converting e-values to p-values. Considering X = X, the modified FDP overestimate and testing levels of e-SAFFRON are respectively given by 0,
Σj- R,-1 1-x
21, and 0₁ = wi (a(1 - X) - E (,1(p, R1-1+1
(Re-1+ + 1).
To clearly distinguish the algorithms, we refer the versions of e-LORD and e-SAFFRON adapted p-values as pL--AII and pS-RAI, respectively, emphasizing that the testing statistics are p-values and testing levels at are updated by the RAI strategy. Both pL-RAI and PS-RAI can realize provable FDR control under conditional super-uniformity. To avoid notational LORD
confusion, define FDP FDPPL-RAI (t) = FDP.
PL-RAI
aj (t) - E 11--1,,-11 R3-1+1 and FDP TDPPS-RAA
PS-RAI
(t) Σj= R2-1
aj 10y2A)
Proposition B.3. Suppose the online p-values are conditionally super-uniform in (1).
(a) For at E Fi-1 satisfying FDP PL-RAI
(t) S a, we have FDR(t) S a for all
(b) Given predictable sequence { for at E Fi-1 satisfying FDP PS-RAI (t) S a. we have: (i)
FDP RA
E
(1) 2 T (FDP where (t) is defined in (6), and (ii) FDR(() a for all t.
The proof strategy for Proposition B.3 follows the same approach as Proposition 3.2 and Proposition 3.4, with the key distinction residing the application of the inequality E {pş > }3} Fリ-1] 2 1 >3. which is derived from the conditionally super-uniform property of p-values. Hence, we omit the detailed derivations.
Long-Term Performance of pL-RAI & PS-RAL To control mem-FDR using p-values, we demonstrate that mem-FDP" (t) in (11) can still serve oracle estimate of mem-FDP. By controlling this estimator to be bounded, can achieve mem-FDR control for p-values satisfying the conditionally super-uniform property in (1).
Theorem B.4. Suppose the online p-values are conditionally super-uniform in (I). If mem-FFD" (t) in (11) satisfies E [meem-FDP (0) I then mem-FDR(() S a for all t.
Leveraging Theorem B.4, both mem-c-LORD and mem-e-SAFFRON can be adapted to the corresponding version applicable
p-values. Specifically, mem- FDP LORD
(L) in (12) and the update rule for or in (13) of the mem-e-LORD algorithm can be directly adapted to p-values. Considering A = A, the modified mem-FDP overestimate and testing levels of mem-e-
SAFFRON are respectively given by Σ/=1 dmc a; 1p,>A) and a; = (a(1 A) Σ o, (p,>A) dRg ++1
(dR + 1).
1-A
To clearly distinguish the algorithms, we refer to the versions of mem-e-LORD and mem-e- mem-e-SAFFRRN adapted to p- values as mem-pL-RAI and mem-pS-RAL, respectively. Both mem-pL-RAI and mem-pS-RAL PS-RAI can realize provable Pl-RAI
PI-RA
mem-FDR control under conditional super-uniformity. To avoid notational confusion, we define mem- FDP RAT
(t) ==
mem- FDP LORD
(1) Ef dRL a and mem- FDP PS-RAT
(1) =
1Pp
+I
1-A
Proposition B.5. Suppose the online p-values are conditionally super-uniform in (1).
(a) For on E Fi-1 satisfying mem- FDP FDPPL-RA (t) S a, we have mem-FDR(() S for all
PL-RAI
(b) Given a predictable sequence {A for 0f E Fi-1 satisfying mem- FDP PS-RAI
(L) S a, we have: (i)
E mem- FDP PS-RAI
(t) E [mem-FDP" (f) where mem-FDP* (t) is defined in (11), and (ii) mem-FDR()) S a
for all t.
We omit the proofs of Theorem B.4 and Proposition B.5, as they follow analogous reasoning and use identical techniques to those previously demonstrated.
15

---

e-GAT: e-value-based Generalized o- o-Investing for Online False Discovery Rate Control
B.5. Connection Existing Methods in GAI under Independence
In this section, demonstrate the FDR control with independent e-values, and compare the e-GAI framework with the GAI methods, highlighting that e- e-GAI is unified framework.
When working with independent e-values, the correlation between the number of false rejections and the total number of rejections can be analyzed using the leave-one-out technique (Ramdas et al., 2018, Lemma 1), which ensures the control of FDR. This property is formalized as follows, paralleling Theorem 3.1 in the independent situation.
Theorem B.6. Suppose online e-values are valid in (2), and the null e-values are independent of each other and of the non-nulls, Under independence, let the oracle e- avalue-basel estimate of FDP be given as
FDP G-ind (1)
Σ
02
(14)
R4
jEHo(!)
If B FDDP (n)d S a and {ar} is monotone function of 8, 1 for all t, then FDR(f) S for all t.
Proof. To control online FDR, we have
EjeHo(t) oj
Σ EjEnoco) Ifes 2 1
S ag) V1
FDR(() =E
IE
=B
Σ
E
FJ-1
R₂ V1
R
(E)((())
(1) SE
Ejenom aj
Σ
E
= E
Sa.
RRVI
R.
(EE((()
where the inequality (i) uses the transformation from e-values into p-values and the leave-one-out technique as in the (Ramdas et al., 2017, Lemma 1) and (Ramdas al., 2018, Lemma 1) due to the independence. Thus we finish the whole proof.
Theorem B.6 builds bridge between framework and previous methods based p-values in prior works introduced in Section 2.2. Specifically, we established the connection between e-LORD and LORD, as well as between e-SAFFRÓN and SAFFRON, respectively.
LORD
e-LORD & LORD++. Within the e-GAI framework, overestimate FDP (t) in (14) by FDP
e-ind (t) :=
- 04 RVI
same FDP LORD
(t) in (4), the independent scenario. Then the process of generating testing levels {at} by
LORD
LORD++ (Ramdas et al., 2017) be regarded as the e-LORD algorithm, it satisfies the condition FDP o-ind (t) $ Therefore, LORD++ can be included within the e-GAl framework.
Furthermore, we can show the equivalence between e-LORD and LORD++ On the one hand, when the independent e-values { are available at each time, then 1/e, is valid p-value by (23) in Appendix B.7, and applying LORD++ to {1/ec} is equivalent to applying e-LORD to {et), wherein the testing levels {ar} } for c-LORD are derived following the LORD++ procedure (Ramdas et al., 2017), i.e..
TORD++ 7/e Wo + (a Wo) N- + a
Σ
(15)
where Wo > the initial o- wealth, 7i is the time of the 1-th rejection and (7) ) is pre-specified non-negative sequence summing one.
On the other hand, when the independent v-luuess {pr} are available instead, then L(p: S act/as is valid e-value. This is because
1 (pr S az)
E
Fi-1
<
due to the conditionally super-uniform property of Pt- Define e, = 1p S as)/a and applying LORD++ to {pt} is equivalent to applying e-LORD to (et) with the testing levels {ar} generated in the same manner as LORD++ (Ramdas
16

---

e-GAT: e-valuc-based Generalized a- o-Investing for Online False Discovery Rate Control
al., 2017). To prove it, denote of [pv S ) and o; eee aF) respectively easily distinguish. It can be readily verified that of 8f. Suppose for all j v t 1, we have 87 69. Then for time t, a a are both generated by the same LORD++ algorithm in (15) and hence we omit the superscript. If oz = 1, it follows that p.. S Os,, which implies that e, = 1/aa, and consequently of = 1. Conversely, if o% = 0, it also holds that of = The proof can then be concluded through recursive reasoning.
e-SAFFRON & SAFFRON. It is natural to derive the relationship between e-SAFFRON and SAFFRON. In the
SAFFRON
Σ. 2/ Oq 3 51/3
independent scenario, we can overestimate FDP in (14) by FDP e-ind
(1)
R.I same as
FDP SAFPRON
(t) in (5) when P: 1/ee. Therefore, the process of generating testing levels {ar} by SAFFRON (Ramdas SAFFRON
al., 2018) can be regarded as the e-SAFFRON algorithm it satisfies FDP e-ind
(t) S
However, the conclusion on the equivalence between e-SAFFRON and SAFFRON differs from that between e-LORD and LORD++. When the independent e-values } are available at each time, applying SAFFRON to valid p-values {p: = 1/e,} is equivalent to applying e-SAFFRON to {et] wherein the testing levels {az} for e-SAFFRON are derived following the SAFFRON procedure (Ramdas et al., 2018), i.e.,
SAFFRON min X Co + ((1 A)a Wo) - Σ(1 (1- -0st at
(16)
122
Here À E (0, 1) is a user-chosen parameter and Ci+ -1) -1) (p; A}
The situation changes when the independent p-values are available. In addition to the conditions of independence, if the p-values further satisfy the conditionally uniformly distributed in (1), then there no valid e-values in (2) such that applying SAFFRON to {pe} is equivalent to applying e-SAFFRON to those e-values, wherein the testing levels {ax) J for c-SAFFRON are derived following the SAFFRON procedure (Ramdas et al., 2018). We demonstrate it by contradiction. Suppose there exists valid e-values {er} with E (p:) such that applying SAFFRON to {pr} is equivalent to applying c-SAFFRON to {cr]. wherein the testing levels {az} for e-SAFFRON are derived following the SAFFRON procedure in (16). Since the p-values are conditionally uniformly distributed, the p-values are also continuously distributed. Note that the p-values play three different roles in the SAFFRON algorithm: rejection if p. S at, candidate if a < pt S X. and inclusion in the estimate of FDP" if P. X. Correspondingly, the e-values (P:) satisfy (p.) 2 1/a, if p. I a,, f((e)) 1/2 if
Pt S X, and (p:) 1/2 if p. > X. By the above assumptions, we have
Elf((p))] - / / (p) aiddp++ / M())llas <PSA)api / (p))l(p Aidp
/ - 1p s anddp+ / Aller alla<ps <ps xjdp+ / {())l(p>> A\de
21+ / >
L x far Apd
with a generated by SAFFRON, which contradicts the assumption that (P:) is a valid e-value. In summary, there no valid e-values transformed by p-values that also serve the three roles accordingly in e-SAFFRON. This conclusion implies that there may be a loss of power when testing by e-values compared to using p-values in the independent case.
B.6. Alternative Expressions of e-LORD & e-SAFFRON
In this section, we provide alternative expressions of e-LORD and e-SAFFRON, respectively. These expressions may aid in better understanding the dynamics of the allocation process and how the e-LOND algorithm can be considered as a special case within the e-GAl framework with design w. from a given sequence {7+)
e-LORD. Recall the e-LORD algorithm updates testing levels as 0n = aw' and for t 2:
a2 W/
aj
(Bi-1 + 1),
(17)
R - 1
17

---

e-GAT: e-value-based Generalized a-Investing for Online False Discovery Rate Control
where w1, ;i....;4p We Fe-1 are the allocation coefficients, updated in (9). By recursion and calculation, (17) can be simplified to
0₁ = a(B4-1 + 1)wt
I
(18)
j=1
e-SAFFRON. We follow the same routine as above. Recall the e-SAFFRON algorithm updates testing levels as 0r a(1 1) and for 2 2:
at = (a(1->)- Σ
[ a 20 < +
R R (Ri-1+1),
(19)
where w E Ft-1 are the allocation coefficients, updated as (9). By recursion and calculation, (19) can be simplified
[-1
a: = a(1 -)(R--1) + 1)ur II - w (1-wll fej <//2))
(20)
j=1
(18) and (20) provide an alternative perspective for understanding the allocation process. In addition to the item regarding the number of previous rejections, the remaining items focus on the process of wealth allocation, in which the total wealth a dynamically allocated by w/ and the prior allocation coefficients {w;}
B.7. Construction of Online e-values
In this section, we offer useful suggestions about the construction valid e-values that can be applied to the online setting. Recall that a non-negative variable et a valid online e- e-value if satisfies the conditional validity property:
E
Fi-1) s 0 0.
Likelihood ratio e-values. Assume that the null conditional distribution Fo.4 is known for all / and denote the corresponding density function fo At each time t, the conditional distribution F.r,,-, and the corresponding density f are unknown, and we can estimate fIJF.. using parametric or nonparametric methods. Then we can construct the likelihood ratio e-value as
fr ferr, (xxe)
(21)
Sop (111)
for each time t. The e-value in (21) is valid since
E fe Fe-1l= = / fAl (2e) for7, (x,) SOLF, (0;5,,(x;)d = / fepr =
p-value-based e-values. In case the p-values associated with each hypothesis are available, it is possible to convert them to e-values using 'p-to-e calibrator' (Shafer et al., 201 I), albeit with possible power loss (Vovk & Wang, 2021). A 'p-to-e calibrator' is a decreasing function f [0,1]++ + [0, oo). such that fo fs(dss 1. Then can construct the p-value-based e-value as
(p.)
(22)
for each time t, where Pt is the corresponding p-value. The e-value e, in (22) is valid long the p-value pt is conditionally super-uniformly distributed under the null. Note that the choices for fo vary. For example, we can simply take [*() = n- for some y E (0, 1) (Shafer, 2021; Vovk & Wang, 2021).
Note that an e-value can be naturally transformed into a p-value pr by minn11/e 1}. This is because
P(p <u Fi-1) & P(e. 1/u Fi-1) BB F
(23)
for all u E (0,1) 1) by Markov's inequality.
18

---

e-GAI; e-value-based Generalized a- o-Investing for Online False Discovery Rate Control
0.05
1.00
0.05
1.00
0.75
0.03
0.03
0103
0:23
0.01
0.23
0.2
0.3
0.0
0.2
0.3
0.4
0.5
02
0.3
0.4
0.0
0.2
033
6
4.
(a) W1 0.001
(b) W1 0.005
1.00
0.04
0.04
Power
Power
0.50
0.50
0.02
0.01
0.01
0.00
0.00
0.0
01
0.2
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.
0.5
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.4
0.5
4
W
4
(c) w1 0.01
(d) W1 = 0.05
Figure Empirical FDR and power with standard error versus 1 for c-LORD, with p 0.5, L 30, μt and X1 0.2. The
(o; ..,.... methods correspond to = 0, o.l,..., yielded most favorable performance.
.......,.... 0.5, respectively. The parameter configuration (wi = 0.005, 6 = 0.5, 4 = 0.5)
C. Additional Simulation Results
C.1. Simulation: FDR control
In this section, investigate the impact of different parameters the performance of proposed method and, based our findings, provide recommendations for appropriate parameter choices. Simulation results under other settings also present to show the performance of c-LORD, e- C-SAFFRON, PL-RAI, pS-RAI RAI and other algorithms.
Updating wi with different parameters wi, iP; 1/. To investigate the performance of e-GAI with varying parameters W1: ip, vi conduct series of experiments on e-LORD, c-SAFFRON the settings outlined in Section 5.1, with 0.05, p 0.5, L 30, f = 3, and . 0.2.
0.1), pL-RAI and pS-RAI 0.1), following
We vary W1, s., and v comparing the performance of the e-I LORD, e-SAFFRON, pl.-RAI, and pS-RAI under various parameter settings. Figures 4 to 7 show the results of c-LORD, e-SAFFRON, pL-RAI, and pS-RAI with different parameters, respectively. As shown in Figures 4 to 7, the e-LORD, e-SAFFRO, pL-RAI, and RS-RAl algorithms successfully achieve FDR control across all settings, which aligns with the theoretical guarantees. The power of the algorithm is affected by variations the parameters W1, (p, tb. When w1 = 0.005, yp = 0.05, all e-LODD, e-SAFFRON, pL.-RAI, and pS-RAI demonstrate high statistical power. Therefore, the parameter settings are chosen w1 0.005, o +f 0.05 in our experiments.
Using different constant parameter in e-SAFFRON & pS-RAI.
To evaluate the performance of e-SAFFRON and PS-RAI with different values of X, we adopt the same experimental settings as described in Section 5.1.
Figure show the results of e-SAFFRON and pS-RAI with different X. As illustrated in Figure 8, both e-SAFFRON and PS-RAI maintain FDR control across all different values of X, which is consistent with the theoretical guarantees. It is evident that when À 0.1, both the e-SAFFRON and pS-RA algorithms attain the highest statistical power. Therefore, recommend setting x = 0.1 as the default value when applying the e-SAFTRON and pS-RAI algorithms.
Setting different correlation parameter To assess the performance of e-LORD, C-FFFROON, pL-RAI and pS-RAI under varying levels of dependence, we adopt the same experimental settings as described in Section 5.1, and compare them
19

---

e-GAI; e-value-based Generalized a- o-Investing for Online False Discovery Rate Control
0.05
1.00
0.00
0.04
0.75
0.04
0.75
0.03
0.03
0.02
0.02
0.03
0:23
0.00
0.23
0.2
0.3
0.2
0.3
0.3
0.2
9.3
0.4
0.
0.3
6.
0.5
8 -8-
-p-
-
(a) W1 0.001
(b) w1 0.005
0.05
1.00
0.05
1.00-
0.64
0.04
Powerr
Power
0.50
0.50
0.02
0.01
0.01
0.00
0.00
0.0
01
0.2
0.3
0.4
05
0.0
0.1
0.2
0.
0.5
3.0
0.1
0.2
0.3
0.4
0.5
0.0
0.1
0.2
033
0.4
05
(c) W1 0.01
(d) w1
0.05
Figure 5. Empirical FDR and power with standard &i for e-SAFFRON, with À 0.1, p 0.5. L 30, μ4. and m 0.2.
The sP: ......., methods correspond to 0, 0.1...... 0.5, respectively. The parameter configuration (w1 = 0.005, Φ = 0.5, g 0.5) yielded the favorable performance.
0.05
1.00
0.00
1.00
0.04
0.75
0.03
0.03
0.02
0.00
0.25
0.02
0.25
0.00
0.00
+ 8 +
-
(a) W1 0.001
(b) W1 0.005
0.05
1.00
0.05
1.00
0.04
0.04
Power
POwer
0.50
FDR
0.03
0.50
0.02
0.01
0.01
0.00
0.00
0.0
0.
0.2
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.4
05
0.0
0.1
0.2
0.3
0.4
0.5
0.0
0.1
0.2
0.3
0.6
05
11
1
(c) W1 0.01
(d) W1 0.05
Figure 6. Empirical FDR and power with standard
for pL-RAI, with p 0.5. 30, μc and 0.2. The
..,..., methods correspond to x = 0, 0.1...., yiclded the most favorable performance.
.........,, 0.5, respectively. The parameter configuration (w)) = 0.005, 6 = 0.5, ş = 0.5)
20

---

e-GAI; e-value-based Generalized o- o-Investing for Online False Discovery Rate Control
0.04
0.75
3.04
0.73
FDR
0.03
8.07
0.25
0.23
0.01
0.00
0.00
0.3
-9-
8:
9:
(a) W/ 0.001
(b) W1 0.005
0.05
1.00
0.05
1.00
0.04
Power
FDR
Power
0.50
0.50
0.02
0.25
0.25
0.01
0.0
03
04
05
0.0
011
0.2
0.5
0.0
0.1
04
05
0.0
011
02
0.4
65
4
(c) W1 0.01
(d) W1 0.05
Figure 7. Empirical FDR and power with standard error versus 10 for PS-RAI, with À 0.1, 0.5, L 30, Mr = and Ti 0.2. The
So: 1 fS methods correspond to o = 0, 0.1,..., 0.5, respectively. The parameter configuration (wi = 0.005, Φ = 0.5, y = 0.5) yielded the most favorble performance.
0.05
1.00
0.05
1.00
0.04
Prwee
Power
0.50
0.03
0.50
0.02
0.00
0.01
0.00
01
0.2
0.3
0.6
0.1
0.3
0.5
01
0.2
0.3
0.6
0.5
0.1
0.3
11
T1
T1
TL
(a) e-SAFFRON
(b) pS-RAI
Figure Empirical FDR power with standard error versus proportion alternative hypotheses n. e-SAFFRON pS-RAI, with = 0.5 and μ.. 3. The Ar, A2,..., As methods correspond to À 0.1, 0.2. 0.5, respectively. The parameter = 0.1 yielded the most favorable performance.
21

---

We consider new experimental setup where the samples follow time-varying auto-regressive AR(1) model. For each time l.i.d. e-GAT; e-value-based Generalized a- a-Investing for Online False Discovery Rate Control
0.06
1.00
0.05
1.00
FDR
0.04
Power
0.75
0.03
Power
0.75
0.50
0.50
0.02
0.25
0.03
0.25
0.00
0.00
0.00
0.00
0.3
0.2
0.3
0.4
0.5
0.1
0.2
0.3
2.4
055
0.3
0.2
0.3
0.4
0.5
0.3
0.2
0.3
0.4
05
TG
T1
TG
e-LORD e-SAFFRON
e-LOND
PL-RAI
p5 PS-RAI
LORD+
SAFFRON
SupLORD
(a) p
p
0.00
1.00
0.05
1.00
0.75
0.04
0.75
FDR
0.04
Power
FDR
0.03
Power
0.50
0.50
0.02
0.02
0:25
0.03
0:25
0.00
0.00
0.00
0.2
0.
0.2
0.4
02
0.4
B-LORD
e-SAFFRON
e-LOND
PL-RAI - PS-RA
LORD++
SAFFRON
SupLORD
(c) = 0.1
(d) p 0.1
0.06
0.06
Power
0.75
0.04
Power
0.75
EDR
0.04
FDR
0.02
0.02
0.25
0.25
0.00
0.00
0.1
02
05
0.4
05
0.1
02
0.3
0.4
0.5
01
0.2
0.3
0.4
0.5
0.1
0.2
0.3
0.4
05
T1
T1.
T1:
Ti
e-LORD
-SSAFFRON - e-LOND
PL-RAI
PS-RAI
+ LORD++ + SAFFRON SupLORD
(c) 0.3
(f) = 0.3
Figure 9. Empirical FDR and power with standard eтror versus proportion of alternative hypotheses T1 for e- e-LORD, e-SAFFRON, e-I.OND, pl.-RAI. PS-RAI, LORD++, SAFFRON and SupL.ORD, with varying p and fixed ftc = 3. Our Rl-RAT and pS-RAI consistently outperform LORD++ and SupLORD, whereas SAFFRON fails to control FDR when is large and W. is small.
with e-LOND, LORD++. SAFFRON and SupLORD.
Figure 9 show the results of various methods with varying P when Me = 3. The performance of each method is similar to that in the main body. For e-value-based methods, irrespective of the correlation parameter p. e-LORD, e-SAFFRON, and e-LOND successfully achieve FDR control. Moreover, e-LORD and e-SAFFRON exhibit substantially higher statistical power than e-LOND across all settings by dynamically updating the testing levels, leading to more effective discoveries. For vaauue-based methods, shown in Figure 9, SAFFRON exhibits FDR inflation under such conditions. Besides, our PL-RAI and pS-R -AIR always gain higher power than LORD++ and SupLORD. In the independence case (p 0), each method successfully maintains FDR control empirically, as guaranteed by their theoretical results.
C.2. Simulation: mem-FDR control
f, Xx = P. + p + E, with ~ (0, 1), where the auto-regressive coefficient P: (+exp (o)(ttoo) 1 E (-1,
We set 0.01 and Ig T/2. We aim to test whether there is a positive drift and the null hypothesis takes H., μr 0 for each time < [Z']. The true labels 0, is generated from Bernoullimm) and the positive drift μ4 Me > 0 0, 1. Note that the correlationn coefficient Pe between adjacent samples varies over time, resulting in a complex correlation structure.
We compare the performance of mem-e-LORI,, mem-e-SAFFRON mem-pL--AA and mem-pS-RA. with mem-L-ORD++
22

---

e-GAI; e-value-based Generalized a- o-Investing for Online False Discovery Rate Control
0.94
0.73
0.04
0.73
107-4-4000 - F07-4-6471101
10h-000
7N0--44220 -
mat.i 4A
(a) T 10000, M4
(b) T 20000, M4c
1.34
15
2.34
0.2%
8.85
0.31
0.25
2.3
72
PET-0-547P90
(c) 10000, phe
(d) 20000, M4c
Figure 10. Empirical DDR-FDRRBR and mem-Power with standard error versus proportion of alternative hypotheses for mem-c-LORD, mem-e-SAFFRRN, mem-LORD++, Rem- and mem-pS-RAI with fe: = 3, 4, at T = 10000, 20000. All procedures maintain FDR control. Our e-GAl and RAI methods exhibit superior mem-power than mem-LORD++ when μ4, is small.
at T E {10000, 20000} with different proportions of alternative hypotheses л and strengths of signals Me. We utilize the AR(1) model and the normal distribution of noise to calculate e-values and p-values that satisfy (2) and (1), respectively.
As shown in Figure 10, all methods successfully control the mem-FDR. Our procedures perform well over long-term testing periods, especially with sparse alternatives. such cases, these algorithms achieve higher mem-Power than mem-LORD++ The mem-Power refers to the decaying memory power in (Ramdas al., 2017), defined as
Eje d' d' LjsHo(e) d'-5
mem-Power(t)
C.3. More results for Real Data: NYC Taxi Anomaly Detection
We adjust 0.2 and apply pL-RAL, pS- RS-RAL, SAFFRON, LORD++, and SupLORD here to analyze this dataset. We construct two-sided Gaussian p-values by estimating the mean and variance from the residuals.
We compare their performance in terms of the proportion of discoveries out of marked anomalous regions, denoted here as FDP and the number of discovered anomalous regions in Table 5. We observe that the FDP of LORD++ and SAFFRON far exceeds the testing level 0.2 and SupLORD slightly exceeds a. Meanwhile, pL-RAI and pS-RAI effectively maintain FDP below the target level. As illustrated in Figure 11, pL-RAI and pS-RAI identify many points within the anomalous regions.
Table 5. Proportion of points rejected out of anomalous regions.
Method
PL-RAI
LORD++ PS-RAI
SAFFRON
SupL.ORD
FDP
0.197
0.195
0.261
0.361
0.217
Num Discovery
201
259
257
595
406
23

---

e-GAI; e-value-based Generalized o- o-Investing for Online False Discovery Rate Control
pl-RRAl procedure
40000
Remeinder
50000
20000
20000
0-
Nox-01-2014
Nov-15-2014
Dec-01-2014
Dec-15-20L4 Timestamp
01-205
Jan-15-2015
Feb-01-2015
Abnomalies marked p5 -R procedure
40000
Rerreinder
20000
10000
0-
Nov-01-200
Now-15-7014
Dec-01-2014
Dec Dec-15-2014
Jan-01-2011
Jan-15-9015
Fab-01-2015
Timestamp
Abnomalies marked by LORD++ procedure
Remairder
30000
20000
10000
Jan 2015
Jan-
Timestamp
Abnomalles marked by SU procedure
40000
Remairder
30000
20000
30000
0-
21-01-2014
11-15-2014
12-01-2014
12-15-2014 Timestamp
01-01-2015
01-15-2015
02-01-2015
Figure Anomaly points detected by PL-RAL RS-RAI, LORD++, and SupLORD, Rejection points all procedures marked by dark blue points. Red regions refer to known anomalies. The testing level is chosen as 0.2.
24


Reviews:
[{"review_id": "xva41DDSCi", "reviewer": "Reviewer_caTX", "summary": "The paper extends the generalized $\\alpha$-investing (GAI) framework for online testing by allowing it to be based on e-values as well (hence the name e-GAI). This allows for online false discovery rate control under arbitrary dependencies of the hypotheses and can lead to improved power under good dynamic allocation of the test levels. More concretely, they do this by defining an oracle estimate of the false discovery proportion at each time that is based on e-values. Then, they employ existing methods of bounding it from the literature, leading to the corresponding methods e-LORD and e-SAFFRON. They supplement their proposal by numerical experiments (including a simulation, taxi anomaly detection, and financial bubble detection).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments on our paper. We would like to part-wisely respond to your comments. \n\n> One small note is that maybe Figure 2 could benefit from some improvements in its display as currently the squares and dots are quite small and subtle (the latter could be fixed by different coloring I think).\n\nThank you for your advice! We have improved Figure 2 by increasing the size of the squares and dots and adjusting the coloring for better contrast. These changes will present in the future revision.\n\n> A potential weakness is that the paper doesn't go into too much depth on devising new methods, and mainly instantiates LORD and SAFFRON in their own framework.\n\nThank you for your comments! Our work aims to address the challenge of online FDR control under arbitrary dependence. To overcome the limitations of p-values in this context, we propose a novel framework leveraging e-values for hypothesis testing, along with two concrete and implementable algorithms, i.e., e-LORD and e-SAFFRON. Furthermore, under the independence assumption, we establish connections between our framework and existing methods, such as LORD++ and SAFFRON from the GAI framework. We have named our approach _e-GAI_, which reflects the key method is to build a new GAI framework with the use of e-values under arbitrary dependence. \n\n> One general small personal suggestion would be to not center the A/B testing for tech companies applications so much in the abstract and introduction, and instead showcase more exciting (possibly for social good) applications.\n\nThank you for this suggestion! We will revise the abstract and introduction to emphasize additional use cases, such as real-time monitoring of machine operation status in industrial settings. Indeed, we have applied the e-GAI algorithm to one relevant real example in Appendix C.2. We appreciate that this will better showcase the potential benefits of our framework.\n\n> Typos: add “The” before “e-LOND” in line 19, left column; add “the\" before “GAI” in line 70 right column; “guarantee” should be “guarantees” in line 417 right hand side\n\nThank you for the careful reading! We will correct these typos in the final version.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "I skimmed through the proofs and they appear correct.", "experimental_designs_or_analyses": "I found the experimental analyses to be thorough and well-done. I appreciated the inclusion of different types of data (simulated and real). One small note is that maybe Figure 2 could benefit from some improvements in its display as currently the squares and dots are quite small and subtle (the latter could be fixed by different coloring I think).", "relation_to_broader_scientific_literature": "This paper is connected to the broader scientific literatures of sequential hypothesis testing via betting and, especially, the line of work in online testing with FDR control. The authors do a good job of discussing and comparing to prior methods.", "essential_references_not_discussed": "Not to my knowledge.", "other_strengths_and_weaknesses": "Beyond the explicit contributions, an additional strength is that the proposed framework unifies multiple existing approaches. A potential weakness is that the paper doesn't go into too much depth on devising new methods, and mainly instantiates LORD and SAFFRON in their own framework.", "other_comments_or_suggestions": "One general small personal suggestion would be to not center the A/B testing for tech companies applications so much in the abstract and introduction, and instead showcase more exciting (possibly for social good) applications.\n\nTypos:\n- add “The” before “e-LOND” in line 19, left column\n- add “the\" before “GAI” in line 70 right column\n- “guarantee” should be “guarantees” in line 417 right hand side", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iOi8OFlwWP", "reviewer": "Reviewer_PQaa", "summary": "This paper proposes a framework for generalized $\\alpha$-investing (GAI) with e-values, an approach for online multiple hypothesis testing.  While prior work had considered GAI with p-values, this paper contributes two things:  First (Section 3.1) they derive bounds on the false discovery proportion (FDP) for the setting of arbitrarily dependent e-values, and second (Section 3.2) they design a schedule for \"spending\" alpha that depends on previous costs.  While the theory speaks to the false positive rate control, experiments on simulated and real-world data demonstrate that the proposed schedule yields greater power than existing approaches.  There are several baselines considered:  SAFFRON and LORD++ are both based on an assumption of independent p-values (or p-values satisfying the PRDS condition), and so do not have guarantees in the setting of arbitrary dependence, while e-LOND does allow for arbitrary dependence, but uses a spending schedule that does not incorporate prior costs, and empirically has lower power as a result.\n\n**Post rebuttal update**:  See my comments in the chain below.  \n> After reading the response and the other reviews, I will maintain my score, though I have somewhat lower confidence in my assessment after reading the review of k3m9. Nonetheless, I'm still in favor of the paper being accepted, assuming that these clarifications (and those made to other reviewers) are incorporated.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your comments on our paper. We would like to part-wisely respond to your comments. \n\n> First, the empirical argument would have been stronger in Section 5.1 with a simulated scenario that caused LORD++ to exceed the desired FDR bounds.\n\nThank you for your valuable suggestion! We have to acknowledge that we explored various simulation settings and did not get the results that LORD++ exceeds the target FDR level for now. This is mainly because the LORD++ algorithm itself is inherently conservative. However, we found that LORD++ does not perform well with $\\widehat{\\text{FDP}}=0.189$ in the real data analysis in Section 5.2. As we know, LORD++ lacks reliable theoretical guarantees with the dependence of p-values, which may compromise its safety and reliability in practical applications.\n\n> Section 2.2, it is stated that the \"key idea in GAI...is that each rejection gains some extra $\\alpha$-wealth\", shouldn't this be the opposite, that accepting the null gains wealth, and rejections spend it?\n\nWe would like to make the following clarifications. \n+ The GAI algorithm is designed from an investment perspective: each hypothesis test is treated as an investment, where the significance level $\\alpha_t$ is drawn from a wealth pool $W_t$. Thus, each test consumes a certain amount of \"wealth\", and if the test results in a rejection, it is considered a successful investment, thereby gaining additional \"wealth\" and increasing the cumulative wealth. \n+ In contrast, our e-GAI framework adopts a different design philosophy: e-GAI views the entire testing process as a risky investment and the whole wealth does not increase; instead, each test consumes a fixed proportion of the remaining wealth (e.g., $\\omega_t$ in e-LORD). Therefore, each rejection introduces risk, and from a risk-averse perspective, it is necessary to reduce the investment proportion in subsequent tests to ensure long-term sustainability. \n+ The difference in design philosophy between e-GAI and GAI stems from the distinct estimators used for the FDP. For the GAI method, which allows for wealth accumulation, increased investment is advantageous. In contrast, for the e-GAI method, where total wealth cannot increase, each investment inherently entails risk.\n\n> I get the math, but had trouble following the conceptual explanation below Theorem 3.1, ... A little more explanation would have been helpful here.\n\nThank you for your suggestion! As you noted, the key role of $R_{j-1}+1$ here is that it serves as a $(j-1)$-measurable lower bound for $R_t\\vee1$. Since $R_{j-1}+1\\leq R_t\\vee 1$ and it is placed in the denominator, it results in an overestimation of the oracle FDP. As the true $R_t\\vee1$ is _unobservable_ at time $j-1$, we use $R_{j-1}+1$ as a substitute, effectively \"_predicting_\" the value at time $t$ from the perspective of time $j-1$. This is why we used the verb \"predict\" in this context. Per your comment, we will provide a more detailed explanation in the final version to make this point clear. \n\n> The PRDS condition is mentioned in several places in the introduction / Section 2 (see page 1, \"p-values with...positive regression dependence on a subset\") but is not formally defined, might have been nice to at least include in the appendix for completeness.\n\nWe appreciate your attention to detail and your valuable feedback. We will include a formal definition of PRDS and relevant references in the appendix of the final version. \n\n> It may be worth taking an editing pass for grammar, e.g., Line 092, left-hand column ...\n\nThank you for pointing this out! We will carefully review the grammar throughout the paper and make the necessary corrections, including the sentence you highlighted. \n\n> Could the authors clarify how their results in Theorem 3.1 differ from those in the e-LOND paper? I didn't see this exact result anywhere in that paper on a very brief skim, but I imagine they need to have used some similar result to also achieve FDR control with arbitrary dependence?\n\nTheorem 3.1 introduces an estimator for the FDP and proves that FDR control can be achieved when the expectation of this estimator does not exceed $\\alpha$. Based on this key estimator, we subsequently propose algorithms for designing testing levels (e-LORD and e-SAFFRON). In contrast, the e-LOND algorithm follows a different approach. Specifically, e-LOND does not introduce an FDP estimator; instead, inspired by the LOND algorithm [1], it defines the testing levels $\\alpha_t$ by directly leveraging a decay sequence that sums to 1 and the number of rejections. This approach allows for the cancellation of a common term in the numerator and denominator of the FDP, thereby achieving FDR control. \n\nRef:\n[1] Zrnic, T., Ramdas, A., and Jordan, M. I. Asynchronous online testing of multiple hypotheses. Journal of Machine Learning Research, 22(33), 1-39, 2021.", "claims_and_evidence": "Yes, the claims are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Yes, the evaluation criteria make sense to me.  The theoretical claims demonstrate control over the FDR, and experiments are used to assess power in real-world situations, both of which I would expect to see in a paper like this one.", "theoretical_claims": "I checked the proofs in Appendix A for the main results, which appear correct to me.", "experimental_designs_or_analyses": "Yes, I found both the synthetic (Section 5.1) and the real-data experiments (Sections 5.2, 5.3) to be sound and valid.  Both experiments demonstrate control of the real (in 5.1) and estimated (in 5.2,5.3) false discovery rates based on known anomalies.  I found both real-world experiments easy to follow and compelling.", "relation_to_broader_scientific_literature": "The key contributions appear a little incremental to me, but are summarized well in introduction (Table 1 and \"Related Works\").  Both prior approaches to GAI with p-values do not allow for arbitrary dependence between p-values, and the existing approach with e-values (e-LOND) does not consider scheduling based on prior costs, and empirically (as shown in experiments) has lower power as a result.", "essential_references_not_discussed": "I did not notice any essential references not discussed, though I am less familiar with this literature, so I may be missing something.", "other_strengths_and_weaknesses": "First, the empirical argument would have been stronger in Section 5.1 with a simulated scenario that caused LORD++ to exceed the desired FDR bounds.\n\nSecond, these are all very minor, but my main critiques of this paper would be related to clarity of presentation, with a few examples where more context would have been helpful:\n* Section 2.2, it is stated that the \"key idea in GAI...is that each rejection gains some extra $\\alpha$-wealth\", shouldn't this be the opposite, that accepting the null gains wealth, and rejections spend it?\n* I get the math, but had trouble following the conceptual explanation below Theorem 3.1, that $R_{j-1} + 1$ \"predicts\" the number of possible future rejections, since it obviously under-predicts the total number of rejections. As I understand it, the key is to observe that the \"right\" denominator is something like $R_t \\lor 1$, and that by under-predicting this quantity, we get an upper bound (since we're in the denominator).  A little more explanation would have been helpful here.\n* The PRDS condition is mentioned in several places in the introduction / Section 2 (see page 1, \"p-values with...positive regression dependence on a subset\") but is not formally defined, might have been nice to at least include in the appendix for completeness.", "other_comments_or_suggestions": "It may be worth taking an editing pass for grammar, e.g., Line 092, left-hand column \"though this operation only achieves a smaller improvement in power while introduces additional\" sounds off, should perhaps be something like \"though this operation only achieves...while **introducing** additional\" or \"though this operation only achieves a smaller improvement in power, **and introduces** additional\"", "questions_for_authors": "There is one area where I wasn't as clear on the contribution:  Could the authors clarify how their results in Theorem 3.1 differ from those in the e-LOND paper?  I didn't see this exact result anywhere in that paper on a very brief skim, but I imagine they need to have used some similar result to also achieve FDR control with arbitrary dependence?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DTguTxonMg", "reviewer": "Reviewer_k3m9", "summary": "The paper proposes a framework for online multiple testing with false discovery rate (FDR) control that utilizes e-values with generalized alpha-investing methods to improve power for e-values that satisfy conditional validity. This paper then establishes connections between these methods and existing generalized alpha-investing methods based on p-values, and provide numerical simulations and real data experiments demonstrating the performance of their new methods.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your careful reading and constructive suggestions. Per your comments, we would like to clarify key **contributions** of our work as follows:\n\n1. We **propose the e-GAI framework** for online testing with using e-values, which **achieves FDR control under arbitrary dependence among valid e-values** (i.e., satisfying condition (2)).\n2. We designed **a new e-value-based FDP estimator** and **provided theoretical guarantees for online FDR control** building on it under the relevant conditions.\n3. Within the e-GAI framework, we **developed a new updating approach for allocating testing levels from a risk aversion perspective,** aiming to save budget while achieving high power. Along this line, we proposed two new algorithms (**e-LORD and e-SAFFRON**) for implementation.\n\nThanks for your comments on our paper. We would like to part-wisely respond to your comments. \n\n> This claim (and variants) are central thrust of the paper and repeated throughout...\n\nWe sincerely apologize for any confusion. As you have pointed out, by \"arbitrary dependence\", we mean allowing any dependence structure among test statistics for each hypothesis, i.e., online conditionally valid p-values/e-values.  In the revision, we will refine descriptions to minimize confusion. Further details on the experimental design will be clarified in the following. Thank you!\n\n> I don't believe LORD++ and e-LORD have any equivalence (nor SAFFRON and e-SAFFRON). ...\n\nThank you for your insightful comments. We agree with you that e-GAI is a different framework from GAI. The \"equivalence\" in Section 4 was to explore the connection between LORD++ and e-LORD when _independent_ p-values are available. In such cases, we can design testing algorithms using a less conservative FDP estimator, i.e., $\\sum_{j = 1}^t \\alpha_j / (R_t \\vee 1)$ such as the one in Theorem 4.1, which will coincide LORD++ and e-LORD.  Previously, for notational simplicity, Theorem 4.1 did not explicitly highlight the change in the FDP estimator. The final version will revise the discussion in Section 4 for clarity and precision. Hope these interpretations can ease your doubts! \n\n> As referred to before, the design of the numerical simulation is unclear. ...\n\nThank you for pointing this out! We apologize for the unclear setup description. The covariance matrix assumes positive correlations between all elements, including nulls and alternatives. In simulations, we assessed our methods using oracle p-values/e-values, calculated from the known conditional distributions derived from the normal distribution. Furthermore, we conducted additional experiments by designing other dependence structures, and these results will be added into the future revision. For details, please see our response to **Reviewer kFoC**'s Q1 due to the space limitation.\n\n> The real data experiments are lacking critical details. ...\n\nWe appreciate your feedback and offer the following clarification. In Section 5.2, the data is assumed to be independent and normally distributed under $H_0$ as discussed in (Ahmad et al., 2017), using p-values derived from the normal distribution. We employed KDE to estimate $f_t(x)$ and $f_0(x)$, taking their ratio as the e-value. Due to the independence assumption, these e-values are valid.\n\nIn Section 5.3, following [1] modeling framework for financial bubbles, we characterized the problem as a right-tailed unit root test in an $\\text{AR}(1)$ model. We built valid e-values on normalized likelihood ratios by dividing its conditional expectation under $H_0$.  Both this likelihood ratio and p-values are proposed by Dickey & Fuller's series works. We will clarify these details in the future revision. \n\n> Two References Not Discussed\n\nThank you for your recommendation! We have introduced and cited Fisher (2024) in Section 2.2 (lines 126-129 of right column) in the main text. Additionally, we include a comparison with the SupLORD algorithm (Xu & Ramdas, 2022) in Table 2 of our response to **Reviewer kFoC**'s Q1 due to space constraints. In the new setting, SupLORD controls FDR but has low power, whereas our methods achieve FDR control with higher power. In future revisions, we will incorporate Xu & Ramdas (2022) and add comparative experiments.\n\n> Looking at the proof in Appendix A.1, is there anything in particular that is specific to e-values? ...\n\nThe proof in Appendix A.1 does indeed rely on a unique property of e-values: the larger the e-value, the more significant the evidence against the null. Note that $\\delta_t=\\mathbb{I}[e_t\\geq 1/\\alpha_t]\\leq e_t\\alpha_t$ deterministicly in the numerator of inequality (ii) of the proof. This, however, does _not hold_ for p-values. Hence, e-GAI cannot theoretically guarantee FDR control when directly applied to p-values satisfying condition (1).\n\nRef:\n\n[1] Phillips, P. C., & Yu, J. Dating the timeline of financial bubbles during the subprime crisis. Quantitative Economics, 2(3), 455-491, 2011.", "claims_and_evidence": "> \"The proposed e-GAI can ensure provable online FDR control under arbitrary dependence while improving the power by dynamically allocating the testing levels.\" (in abstract)\n\nThis claim (and variants) are the central thrust of the paper and repeated throughout, though it is incorrect. All FDR controlling results require condition (2), which enforce that the e-values are conditionally valid on past rejection decisions. This is a special case of the conditional superuniformity assumption for p-values that has been repeated throughout the existing literature, and indeed the authors themselves acknowledge it in Appendix B. Prior literature in both online and offline multiple testing have consistently referred to the term \"arbitrary dependence\" as allowing any kind of dependence structure among p-values/e-values (test statistics) for each hypothesis. Notably, positive dependence among all test statistics (e.g., jointly Gaussian with only positive correlations) *do not* satisfy conditional validity, but fall under the umbrella of arbitrary dependence. The inaccuracy of this claim becomes relevant in the setup choice in numerical simulations (see later point in \"Experimental Design and Analyses\")\n\n> Theorem 4.1 + \"applying LORD++ to $\\\\{1/e_t\\\\}$ is equivalent to applying e-LORD to $\\\\{e_t\\\\}$\" in Appendix B.\n\nI don't believe LORD++ and e-LORD have any equivalence (nor SAFFRON and e-SAFFRON). The $FDP_e^*(t)$ estimator introduced in this paper is much more conservative than the typical $\\widehat{FDP}^{\\text{LORD}}(t)$ from prior work, since the denominator is smaller in the former. This smaller denominator is a key part of the proof. This discrepancy is noted in the intro by the authors, so it seems that the intro and Appendix B are incoherent. Further, in Theorem 4.1 they require $\\mathbb{E}[FDP_e^*(t)] = \\mathbb{E}[\\sum_{j = 1}^t \\alpha_j / (R_t \\vee 1)]$, which also is incorrect, since $\\mathbb{E}[FDP_e^*(t)]$ is conservative (though this seems like a typo).", "methods_and_evaluation_criteria": "The methods and evaluation criteria (FDR + power) make sense.", "theoretical_claims": "The errors with the theoretical claims are discussed in the prior section.", "experimental_designs_or_analyses": "- As referred to before, the design of the numerical simulation is unclear --- is the covariance positive for all alternatives or all hypotheses? If it is for all hypotheses (as suggested by \"The $\\Sigma > 0$ is with all diagonal elements being 1 and off-diagonal elements being \\rho$\"), then the data simulation is does not provide e-values that satisfy condition (2) that is necessary for all proofs of FDR validity for all methods presented in the paper. This setting is then a bit odd for a paper which focuses on methods with provable control of the FDR. Could the authors confirm whether the covariance is positive for just the alternatives (and independent for the nulls) or across all hypotheses?\n\n- The real data experiments are lacking critical details --- in particular, what are the assumptions on the data generating process under the null in each setting, and how are the e-values/p-values explicitly constructed? This is particularly relevant for Section 5.3 since the choice of p-value and e-value seem to be quite different, and it's unclear whether the gap can be explained by that difference.", "relation_to_broader_scientific_literature": "Although the presentation of the paper has somewhat obfuscated what the contributions are, I believe there is an interesting contribution here where the authors have designed a new FDP estimator that allows for online multiple testing with FDR control under conditional validity, at the cost of some power compared to when independence is assumed. This then allows them to use generalized alpha-investing to design procedures valid in that specific setting.", "essential_references_not_discussed": "- Aaron Fisher. Online false discovery rate control for LORD++ and SAFFRON under positive, local dependence. *The Biometrical Journal*, 2024.\n\nThis paper shows that LORD++ and SAFFRON have valid FDR control under conditional superuniformity (a more general condition that is implied by the conditional validity in the paper), and an additional positive dependence assumption on the test statistics. This should at least be cited in the context of existing work with FDR control under conditional superuniformity.\n\n- Ziyu Xu and Aaditya Ramdas. Dynamic Algorithms for Online Multiple Testing. *Mathematical and Scientific Machine Learning*, 2022.\n\nThe SupLORD algorithm in this paper (to my knowledge) is the only algorithm that has valid FDR control under (just) conditional superuniformity (apart from algorithms valid under arbitrary dependence) and should be compared to in experiments.", "other_strengths_and_weaknesses": "The real data experiments in this paper are seem quite comprehensive, and the authors did a great job presenting visualizations for the analysis.", "other_comments_or_suggestions": "Looking at the proof in Appendix A.1, is there anything in particular that is specific to e-values? It seems like it would hold for p-values that satisfy conditional superuniformity --- that would be a solid result if it were the case.", "questions_for_authors": "No questions/concerns outside ones stated. The paper generally seems unclear about what its actual contributions are, and missing comparisons to key prior work/comparisons.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "KJc5SCX36V", "reviewer": "Reviewer_KFoC", "summary": "The paper proposed the e-GAI framework which can control the FDR under arbitrary dependence structures by defining a conservative e-value-based FDP estimator and adopting a risk-averse strategy.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your helpful comments and suggestions. We would like to part-wisely respond to your comments. \n\n> In financial bubble detection, ω1 is set to 0.0001 without justification.\n\nThis application can be seen as a long-term time series (i.e., $T=10000$). As response to your W1&W2, we suggest choosing $\\omega_1$ as $1/T$ approximately to avoid spending too much wealth in the early stages and resulting in $\\alpha$-death. \n\n> The paper does not evaluate runtime or memory consumption...\n\nWe would like to make the following clarification. \n\n+ The update of $\\omega_t$ and $\\alpha_t$ can be expressed in a recursive form, so computation is highly efficient and memory-friendly. Specifically, updating $\\omega_t$ is shown in (9) (line 195). To compute $\\alpha_{t}$ for e-LORD (lines 165-177), we define remaining wealth $r_t^{\\text{e-LORD}}=\\alpha-\\sum_{j=1}^{t}\\frac{\\alpha_j}{R_{j-1}+1}$ and update $\\alpha_t=\\omega_t r_t^{\\operatorname{e-LORD}}(R_{t-1}+1)$. Similar recursive $\\alpha_t$ for e-SAFFRON can be obtained. \n+ We also evaluated runtimes of various algorithms in Table 2 below. It shows that e-LORD and e-SAFFRON are computationally fast. We will add the recursive update form and present results in final version.\n\n> W1: The user is required to set the initial parameters (e.g., ω₁, λ)...\n\nThank you for your insightful comments! We would provide some suggestions about $\\omega_1$ and $\\lambda$. \n\n+ In general, a larger $\\omega_1$ means more wealth will be assigned to each hypothesis testing, making it easier to discover alternatives, and meanwhile, it will be more possible to exhaust entile wealth at an early stage. Hence, we recommend a relatively small $\\omega_1$ for a long period, e.g., $\\omega_1\\approx 1/T$ empirically. \n+ $\\lambda$ affects initial wealth and the proportion of hypotheses counted into FDP's estimator. We recommend $\\lambda=0.1$ to preserve wealth as discussed in _Remark 3.5_. \n+ We have simulation results with different ($\\omega_1$,$\\lambda$) in Appendix C.1. Additional results of powers of e-LORD and e-SAFFRON ($\\lambda>0$) under an $\\operatorname{AR}(1)$ model are shown in Table 1 below.  All FDRs are controlled and omitted due to the space limit. We can see the choices of  ($\\omega_1,\\lambda$) do not affect results too much. So, it enables users to integrate domain knowledge in real applications. \n\nTable 1: Power (%) with different ($\\omega_1,\\lambda$) and $\\alpha=5$%. \n\n|  | e-LORD | $\\lambda=0.1$ | $\\lambda=0.2$ | $\\lambda=0.3$ | $\\lambda=0.4$ | $\\lambda=0.5$ |\n| --- | --- | --- | --- | --- | --- | --- |\n| $\\omega_1=0.001$ | 69.92 | 70.53 | 69.36 | 68.04 | 66.46 | 64.56 |\n| $\\omega_1=0.005$ | 67.16 | 75.03 | 74.19 | 72.99 | 71.53 | 69.79 |\n| $\\omega_1=0.01$ | 49.41 | 66.47 | 65.69 | 64.54 | 63.19 | 61.48 |\n\n> W2: Long-term performance of algorithm has not been analyzed. ...\n\nAs noted, excessive early-stage wealth consumption leads to \"$\\alpha$-death\", halting rejections once $\\alpha$-wealth is zero. It is a common phenomenon within GAI framework. So, we recommend using a small $\\omega_1$ to preserve wealth as response to your W1. \n\nThis issue warrants further research, such as considering \"activation\" after reaching a certain condition or exploring scenarios where online sequence inherently exhibits block structure, among other possibilities. In addition, this work focuses on measuring the cumulative error by FDR. Exploring other metrics is also a valuable future direction.\n\n> Q1: ...what if there are some time-varying correlations...\n\nWe design a new time-varying  $\\text{AR}(1)$ model: $X_t=\\rho_t X_{t-1}+\\epsilon_t$ for $H_0$ and $X_t=4+\\rho_t X_{t-1}+\\epsilon_t$ for $H_1$ with $\\rho_t=\\frac{2}{1+\\operatorname{exp}(-0.01(t-T/2))}-1$. The table below shows that e-LORD and e-SAFFRON control FDR while leading to relatively high power. \n\nTable 2: FDR, power, and runtime with $\\alpha=1$%. SupLORD is referenced by **Reviewer k3m9**.\n\n| | e-LORD | e-SAFFRON | e-LOND | LORD | SAFFRON | SupLORD |\n| --- | --- | --- | --- | --- | --- | --- |\n| FDR (%) | 0.01 | 0.03 | 0.00 | 0.15 | 1.07 | 0.00 |\n| Power (%) | 54.29 | 58.03 | 15.42 | 78.05 | 91.06 | 10.22 |\n| Runtime ($\\times 10^{-4}$s) | 11.7 | 17.8 | 17.0 | 11.0 | 5.0 | 77.6 |\n\n> Q2: How to consider long-term performance?\n\nHere we focus on controlling online FDR, a common criterion in online multiple testing. To alleviate \"$\\alpha$-death\" over a long period (potentially infinite), Ramdas et al. (2017) discussed controlling decaying memory FDR under independence. But, how to control it under dependence remains challenging due to additional complexity introduced by these dependencies. As response to your W1, another potential way is to consider other strategies or specific structures. We will include a careful discussion of long-term issues in a future revision per your suggestions. Thank you!", "claims_and_evidence": "yes", "methods_and_evaluation_criteria": "yes", "theoretical_claims": "no", "experimental_designs_or_analyses": "1. In financial bubble detection, ω1 is set to 0.0001 without justification.\n2. The paper does not evaluate runtime or memory consumption for ultra-long sequences. I noticed that recursive updates in Eq 8 (where αt depends on all historical Rj ) may lead to linear computational complexity, which might be inefficient in real-world scenarios.", "relation_to_broader_scientific_literature": "The paper’s contributions build upon and extend prior work in online false discovery rate control and e-value theory.", "essential_references_not_discussed": "no", "other_strengths_and_weaknesses": "Strength: Unlike some traditional methods, the proposed e-GAI framework does not need prior knowledge of dependency patterns, which could be more flexible in real-world scenarios.\n\nWeaknesses: 1. The user is required to set the initial parameters (e.g., ω₁, λ), which may rely on experience or a large number of experiments in practical applications.\n2. The long-term performance of the algorithm has not been analyzed. For example, if the dynamic allocation strategy (Eq 9) consume too much α-wealth with high rejection rates in the early stages, will the late test sacrifice overall efficacy due to lack of budget, and does it have cumulative errors, etc.?", "other_comments_or_suggestions": "Please see above comments.", "questions_for_authors": "Q1. The covariance matrix assumes equal correlation coefficients (ρ) across all time points, what if there are some time-varying correlations (e.g., increasing ρ) or nonlinear dependencies?\nQ2: How to consider the long-term performance of the algorithm?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper studies a framework for online multiple testing with false discovery rate (FDR) control that utilizes e-values and generalized alpha-investing methods to improve power when the e-values satisfy conditional validity. It then establishes connections between these methods and existing generalized alpha-investing approaches based on p-values, and provides numerical simulations and real data experiments demonstrating the performance of the proposed methods.

Reviewer k3m9 points out a few issues in both the writing (e.g., conditional superuniformity) and the experiments (e.g., SupLORD). Upon digging into it, I tend to agree with Reviewer k3m9's comments. To ensure this work be rigorous and sound, I believe the paper needs to be revised before it can be published. The response and constructive feedback (e.g., discussion on parameter choices) from other reviewers should also be considered in the next version.
Decision: Accept (poster)