From Individual Experience to Collective Evidence: A Reporting-Based Framework for Identifying Systemic Harms
Authors: Jessica Dai
Paula Gradu
Inioluwa Deborah Raji
Benjamin Recht
Venue: ICML 2025 poster (2025)
Keywords: post-deployment auditing and evaluation
fairness
sequential hypothesis testing
public reporting
individual reporting
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: We propose individual reporting as a means to post-deployment (fairness) evaluation.

Abstract: When an individual reports a negative interaction with some system, how can their personal experience be contextualized within broader patterns of system behavior? We study the *reporting database* problem, where individual reports of adverse events arrive sequentially, and are aggregated over time. In this work, our goal is to identify whether there are subgroups—defined by any combination of relevant
features—that are disproportionately likely to experience harmful interactions with the system. We formalize this problem as a sequential hypothesis test, and identify conditions on reporting behavior that are sufficient for making inferences about disparities in true rates of harm across subgroups. We show that algorithms for sequential hypothesis tests can be applied to this problem with a standard multiple testing
correction. We then demonstrate our method on real-world datasets, including mortgage decisions and vaccine side effects; on each, our method (re-)identifies subgroups known to experience disproportionate harm using only a fraction of the data that was initially used to discover them.

From Individual Experience to Collective Evidence: A Aaporting-Basede Framework for Identifying Systemic Harms
Jessica Dai
Paula Gradu'
Inioluwa Deborah Raji' Benjamin Recht
Abstract
development time (e.g., surveyed in Pessach & Shmueli (2022))-o0 or in identifying unfair ones, motivating post-hoc audits that occur after the entire decision-making process has completed (c.g., Byun et al. (2024); Martinez Kirch- ner (2021)). However, in most applications where fairness is a concern, problems with the system may only emerge over time, and it is not necessarily obvious which subgroups might be important. Moreover, such approaches to fairness provide no mechanism for individuals to raise concerns.
When an individual reports a negative interac- tion with some system, how can their personal experience be contextualized within broader pat- terns of system behavior? We study the report- ing database problem, where individual reports of adverse events arrive sequentially, and are ag- gregated over time. In this work, our goal is to identify whether there are any combination of relevant teatures-that are disproportionately likely to experience harmful interactions with the system. We formalize this problem as sequential hypothesis test, and iden- tify conditions on reporting behavior that suf- ficient for making inferences about disparities in true rates of harm across subgroups. We show that algorithms for sequential hypothesis tests can be applied to this problem with a standard multi- ple testing correction. We then demonstrate our method on real-world datasets, including mort- gage decisions and vaccine side effects; on each, our method (re- tren)idensssies subgroups known experience disproportionate harm using only a fraction of the data that was initially used to dis- cover them.
cegroups-definee by
It is exactly this question of individual agency that drives our work. In addition to normative reasons, which suggest that individuals ought to have a voice in expressing concerns with their treatment (e.g.. the literature on contestability of algorithmic decisions (Vaccaro et al., 2019)), recent legis- lation has also highlighted individual reporting as a policy mandate for the governance of AI systems (e.g., the E.U. AI act (European Parliament, 2023)). While such legisla- tion has yet to see full implementation, mechanisms for individual incident reporting already exist in a variety of domains, including consumer finance, medical devices, and vaccines and pharmaceuticals. A key component of report- ing databases in the latter settings is that information from individual reports are aggregated to build collective knowl- edge about specific vaccines or pharmaceuticals-ann when applicable, this aggregated information can drive down- stream decisionmaking. such as updating vaccine guidelines or drug treatment protocols (e.g., Oster et (2022)).
1. Introduction
In this paper, we consider what realistic approach to as- sessing fairness claims from incident reporting database might look like in practice. We are primarily interested in designing a framework for the general public to report and contest large-scale harms by leveraging reports of individual experience to inform collective evidence of discrimination. To this end, we propose reporting databases, which allow individuals submit reports of negative interactions, new mechanism for post-deployment fairness auditing. In particular, we identify conditions on reporting behavior and show how they can be used to to make inferences about rates of true harm Section 3. Our formalization of the problem allows us to leverage known approaches to sequential hy- pothesis testing. In Section 4 we show how to instantiate two reasonable algorithms for our test and provide theoreti-
The impact of injustice is most acutely felt by the individual. But if an individual experiences harm, how can they know whether their experience is an isolated incident or part a larger pattern of discrimination?
Fairness work has historically focused on model develop- ers and third-parte party auditors as the main actors involved in creating fair mechanisms, motivating methods to construct models that are fair with respect to pre-defined subgroups at
Department of Electrical Engineering and Computer Science, U.C. Berkeley, Berkeley, CA, USA. Correspondence to: Jessica Dai <jessicadai @ berkeley.edu>>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

From Individual Experience to Collective Evidence: A Reporting- Eaporting-Based Framework for Identifying Systemic Harms
cal guarantees for each. Finally, in Section 5, we illustrate the usefulness of our approach using real-world datasets, for applications with known disparity in per-subgroup rates of harm. On both real vaccine incident reports and on mort- gage allocation decisions, our algorithm correctly identifies groups that disproportionately experience harm-and does so using a comparatively small number of reports.
describe systems for monitoring the adverse impact of algo- rithmic deployments, which often take the form of accident catalogs that focus on one-off, large-scale events (e.g., Feffer et al. (2023); Raji et al. (2022); McGregor (2021); Ojew- ale et al. (2024); Turri & Dzombak (2023)). However, in the context of our work, we are actively excluding these accident catalog databases. Instead, we focus on reporting databases that provide records of individual experiences of adverse events that are tied to specific systems.
1.1. Related work & application context
The reporting database problem is at the intersection of various challenges addressed in fairness and statistics. We discuss additional work, including application and policy context, in Appendix A.
2. Model, Notation, and Preliminaries
The goal of constructing a reporting database is to determine whether some system that individuals interact with-for ex- ample, an (algorithmic) loan decision system, or a medical treatment-results in disproportionate harm to some mean- ingful subgroups. For the reporting database associated with particular system, we will use Y E {0, 1) as an indicator variable that denotes the undesirable event corresponding to that system. For example, in loan decisions, this could correspond to the event that a highly-qualifee was denied a loan; in the medical setting, this may be an adverse physical side effect due to the treatment.
Algorithmic accountability via (individual) reports. Some recent work considers methods for learning about fairness problems via individual reports from both theoreti- cal (Globus-Hek) Harris et al., 2022) and practical (Agostini et al., 2024) perspectives. However, most discussion of individual experiences in machine learning fairness literature is limited to contexts where the objective is to assess, appeal, contest or seek recourse for that individual to change their individ- ual outcomes, rather than forming collective judgment about the system as whole (Sharif--Malvajerdi et al., 2019; Ustun et al., 2019; Karimi et al., 2022).
individual
Subgroup definitions. Individuals are characterized with feature vectors X E X, and we index individuals as X; ("features of individual i"") or X, ("features of the individual who reports at time f*)). Every individual X; "belongs to" at least one group G, and we will denote the event that X, belongs to G as (X; E G): we will use G to denote the set of all possible groups. This set of possible groups G can be defined arbitrarily long all groups can be determined function of covariates X. We allow for groups to be overlapping thatt is, we allow each individual X, to be in multiple groups so that HG' e G X; E G'H 2 1.
Work on identifying fairness-related issues via reporting data has typically focused on learning in batch contexts, e.g. via positive-unlabeled learning for handling disparate reporting rates across subgroups (e.g., Shanmugam et al. (2024); Wu & He (2022)). In other works, identifying dis- parate reporting rates is itself is the central challenge (e.g., Liu & Garg (2022); Liu et al. (2024)). On the other hand, an emerging body of literature from the interaction community develops the concept of contestubil- ity (c.g.. Almada (2019); Vaccaro al. (2019); Landau et al. (2024)); though contestability is still typically understood in terms of individual outcomes, see our work as one possible path to implementing this ideal.
Reference population. The system for which the database is constructed naturally has a corresponding reference pop- ulation of eligible individuals. For example, this could be everyone who has applied for a loan, or everyone who has been prescribed a certain medication. Thus, given a set of groups G. we assume that it is possible to compute the composition of the reference population.
Fairness auditing as hypothesis testing. Existing pro- posals to formalize fairness auditing via hypothesis test- ing mainly consider batch settings (i.e. post-hoc or pre- deployment) (Cen& & Alur, 2024; Cherian & Candès, 2023). Two more closely related works are that of Chugg et al. (2024) and Feng et al. (2024), who propose applying se- quential hypothesis tests with the explicit goal of identifying problems in deployed systems in real time. However, as neither of these works study a reporting model, we propose fundamentally different tests: they test equality of means across different groups, while we compare within groups.
Assumption 2.1 (Reference population). For every G E G. the quantity H := Pr[X E G] is known. Throughout this work, we refer to the set ( boe as base preponderances.
Probabilistic model of reporting. As the database admin- istrator, the high-level goal is to determine whether there exists some subgroup G E G where Pr[Y X E G is ab- normally high. Crucially, the database does not have access to information about every individual who interacts with the system; instead, individuals may report to the database they believe that they experienced bad event Y. We thus let
Key definitions & elarifications. Finally, we note that for Al systems, the term "incident database" has been used to
2

---

From Individual Experience to Collective Evidence: A Reporting-Base Based Framework for Identifying Systemic Harms
R2 be random variable representing whether individual decides to report (with R. = 0 indicating no report).
Pr[R= [| group-conditional analogue as pG := Pr(y= -
In Proposition 3.2, we show that if the group-conditional RIR of some group G at most some constant multiple of the population-wide RIR, then we can convert lower bound report preponderance into a lower bound on true relative risk (see Appendix C. for proof).
Each report X, is received sequentially, and assumed to be sampled i..d. from some underlying reporting distribution. Given a group G. we denote its corresponding mean among reports Pr[X, E R2 = I] as PaG. We will sometimes refer to Jegg as (reporting) preponderances, as they represent the proportion of reports that each G comprises. A central claim of this paper is that comparing jag to 1ig-i.., the extent to which group G' is (over)representte within the reporting database-caa can be a useful signal for Pr[Y G] in a wide class of applications.
Proposition 3.2. Define the relative risk of group G to be RRR: = Pr|Y-I|Ci -P[(Y=I] Suppose that for some group G we have Pa S b. Suppose that we determine that PG 2 Bpg for some 3 > 1. Then, the tree relative risk experienced by G is at least RRG 2 2//b
Suppose we take maxg Po/p - b 1.25, i.c., no group over-reports 25% more often than the population average. Then, if a test identifies group G for which PG 2 1.75-1&. this implies that the true relative risk for group G is at least RRg 2 tht is, G experiences harm 40% more frequently relative to the population average.
3. Identifying Discrimination by Modeling Preponderance
A major challenge of assessing potentially-differenttal rates of harm across subgroups using only reporting data to relate the event that someone submits a report to the event that they experienced harm. That is, if someone did ex- perience negative outcome, how likely is it for them to have reported it, and conversely, if someone submitted report, how likely is it to reflect "true" harm? Moreover, as is known from prior work, reporting rates themselves can vary across subgroups.
3.2. Preponderance as true incidence rate
We now discuss an alternate way to convert a lower bound on preponderance into a guarantee on real-world harm. In this case, we can infer the true incidence rate of harm (that is, no longer relative to the average) if we are able estimate- or willing to make assumptions ca-true and false reporting behavior in groups. Moreover, assumptions (or estimations) of these reporting rates need only be made in relation to the population average reporting rate Pr[M].
Our central proposal is to conduct a hypothesis test for each group to determine whether it is overrepresented by factor of A among reports. That is, for each G G, we test the following hypotheses:
Definition 3.3 (Reporting rates). Let r == Pr[R] be the average reporting rate over the full population. Let ne := +
Hb PG < BBC
H : PG > 3p6-
(1)
1 Pr(R, = Y, = X, E G]. JR == 1 Pr[R, Y; 0, X; E GJ. Finally, let IRg == Pr[Y G] represent the true incidence rate, i.e. the likelihood that an individual in G experiences Y.
In Section 4, we will discuss concrete algorithms for con- ducting this test sequentially and their corresponding the- oretical guarantees. Before doing so, we first argue that testing for preponderance among reports, i.e., tracking PG in this way, can be meaningful way to identify discrimi- nation, even when exact reporting behavior is unknown. In Sections 3.1 and 3.2, we describe two distinct ways that this particular test can be interpreted: in Appendix B, we discuss some practical considerations for the modeling task.
Note that r. Th rate at which individuals X, E G who experience Y actually report, while r 7e represents the rate that individuals X, E G who do not experience Y report. Thus, v0 and T represent how much more (or less) particular group G makes true or false reports relative how much the whole population reports on average (which includes both true and false reports). The following proposition makes the relationships between NG TG and our quantity of interest IRG, more precise.
TR represents the (possibly group-conditional)
3.1. Preponderance as relative risk
The first interpretation of our test allows us to make infer- ences about relative risk, the ratio between the rate of harm experienced by group G and on average over the popula- tion. In this interpretation, the key quantity is the report-to- incidence ratio.
FR
Proposition 3.4. Suppose that, for some G, it is determined
that HG 2 8Pr for some B > As long as T > -G for every G E G. IRG
3
Definition 3.1 (Renidc-to ratio). We define the report-to-ineen -ow--o--iiiidd-eatiiiii (RIR) as p := P and the
Pr(R-1]]
See Appendix C for the (short) proof. Proposition 3.4 shows that the exact computation of IRG depends on reporting rates 70th and 7 While these quantities are not directly
Because allow groups to overlap, we cannot enforce Ea 2 = or EoHe. = 1.
3

---

From Individual Experience to Collective Evidence: A Reporting- Baani Framework for Identifying Systemic Harms
estimable from reporting data-in fact, estimating reporting rates is itself distinct research challenge (see, e.g., Liu et al. (2024))-these results can nevertheless guide qualitative interpretation of how severe IRg is.
rections often seem onerous in non-sequential settings, we show that, for sequential problems, the Bonferroni correc- tion incurs only modest increase in stopping time.
In Section 4.1, we give simple sequential Z-test- E-test-inspired approach which leverages a finite-time Law of the Iterated Logarithm. Section 4.2 presents a more complicated algo- rithm that leverages recent developments in anytime-vall -alid inference. The main differences in each algorithm lie in how they implement Lines and 6 of Algorithm that is, how test statistics and thresholds are computed. For each instantiation of Algorithm 1, we show validity and power guarantees. Omitted proofs are given in Appendix C.
For example, suppose a test is run for B = 1.5. Suppose G overreports relative to the population average, with FR = 1, and NG = 2. Under these (generous) assumptions, we will have IRg- 0.5, an extremely high incidence rate for any application regardless of incidence rates for other groups.
Alternatively, suppose reporting rates did not vary by group (i.e., N =yTH and - = = for all G). Then, we can lower bound the disparities between true incidence rates across groups: if G is flagged at 3 > 1. there must be some other group Gi' with IRg IR 2 7t gon If is further
8-
Algorithm 1: General protocol for testing overrepresen-
assumed that 7 = 0, then IRG RR 2 3 1.
tation Input:
Set of groups G: base preponderances
{ul Iaeg; test level a; relative strength 3
4. Identifying Subgroups with High Reporting Overrepresentation
Initialize test statistic wy for every G E 9 and set
threshold lo(a);
Initialize set of rejected nulls (flagged groups)
How might the test proposed in Equation (1) be carried out in practice, with reports arriving over time, and what prop- erties might we want for such a test? In this section, we provide two ways to instantiate this sequential hypothesis test. For each, we provide two types of guarantees. The first is (sequential) a-validity, which, roughly speaking, guaran- tees correctness of groups identified in F More formally, we say that sequential test is valid for single group G at level a if Pr[at: : Hg rejected] S when HD is true. Be- cause we are testing for all groups in 9 simultaneously, we say that a sequential test is valid with respect to all groups 9 Pr/3t, 3G Hg crroncously rejected] S a.
g := 0; for t = 1, do
See report Xt; 4 s
for G E G do
Update test statistic w/ and compute threshold 0((a):
if uf 2 0,(a) then
7
Add G to gFla and take requisite action for
G, applicable.
4.1. Sequential Z-test
The second type of guarantee is power, which guarantees that the test will identify a harmed group. if one exists. In particular, we are interested in the stopping time T of the test, which is the number of samples required for the test to reject the first null, i.c. to raise an alarm for any group.
One simple observation that arises from the model presented in Section 2 is that if each report X, is drawn i.i.d from some underlying distribution, then one might expect to be able to use concentration as tool to conduct this test, since as time passes, the fraction of reports within the database from group G should converge to the true mean MaG. We refer to this style of approach as sequential Z-test, as it relies on measuring deviation from the mean.
At a high level, our algorithms for conducting this test follow the protocol outlined in Algorithm 1. For each group G, we maintain test statistic w/f that is updated reports X, are received over time. At each time t, each of these test statistics are compared to a threshold Oe (a). which depends on the test level o; the null hypothesis HC for group G is rejected if wi > (a)). For ease of exposition, Algorithm written so that groups corresponding to rejected nulls are collected in set Gflla; in practice, a database administrator may choose to stop the test entirely as soon as one harmed group has been found.
Updating the test statistic wf. Given this intuition, the test statistic is simple count of the number of times report from each group has been seen, i.e. (with wp = 0),
we t - + 1|x, E G].
Setting the threshold (a). Given the way that wof accu- mulates evidence, one natural way to construct the threshold at each t is to use the mean under the alternative, plus a correction term for both sample complexity and repeated testing over time. With C set to either (3pg((1 - (34) or
Correcting for multiple hypothesis testing across groups is handled by simple Bonferroni correction-thal is, given a particular test level a, we test each individual group G at level 2/10 rather than level a. Though Bonferroni cor-
4

---

From Individual Experience to Collective Evidence: A Reporting-Basee Framework for Identifying Systemic Harms
1/2, the threshold (including a Bonferroni correction) is
with X xf....., [0, 1]. Here, the algorithm accumulates nonlinear function, with an adaptive parameter XS that weights the influence of each new sample. Our setting of X is motivated by the goal of minimizing stopping time under the alternative, and thus to maximize w Drawing from the well-studied problem of portfolio optimization in the online learning literature (Cover, 1991; Zinkevich, 2003; Hazan al.. 2016). we use Online Newton Step (Hazan et al., 2007; Cutkosky & Orabona, 2018) to ensure that wf is not too far from the best achievable in hindsight. This results in the following update for {de)121
(2 log)(())² (3)
8.(a) = + i.
|
a
Theoretical guarantees. Our first guarantee a bound on the probability that any group incorrectly flagged.
Theorem 4.1 (Validity). Running Algorithm I with (a(a) as in Equation (3), setting C = 1/2, and W updated as in Equation (2), guarantees that the probability that F will ever contain a group G where Hy is true is at most a, i.e.
Pr [ EG E Gflus s.1. Hg holds] S a.
141 t -Pooj (X + 2-In(3) [0.1]
2
1+ 1+ 1+
) (5)
The choice of C affects the nature of the guarantee: the true, finite-sample -itiiiim--aiiityyy guarantee requires C = 1/2. If instead C / (1 3yg)) then, strictly speaking, the guarantee holds only asymptotically. However, higher value of C' affects stopping time unfavorably, so the asymp- totic approximation can be useful practically. In this case, care must be taken to ensure that the algorithm does not erro- neously reject too early due noise; one way to implement this is to mandate a minimum stopping time.
where 2 (+
- and Au 0.3
Setting the threshold (((a). Unlike the sequential Z-test, we use the same threshold for all timesteps. Including a Bonferroni correction, we use 0e(a) == In(()laa) for all I; the motivation for this setting will become clear in our discussion of Theorem 4.3.
Theoretical guarantees. We first give a validity guarantee that essentially identical to the Sequential Z-test
Finally, we give stopping time guarantee for this test.
Theorem 4.2 (Power). Let T be the stopping time of Al- gorithm I with (a)):) as in Equation (3), C= 1/22, and we as in Equation (2). Let Amax = maxGeg PG - B If Amax > 0, then Pr(TT- < oo) = 1. Furthermore, with proba- bility 1 - 2/191, we have T S Ö [In([]] (1((1())))(((11))))) ) and for any 6 E (0, 9/191), we have with probability at least 5
Theorem 4.3 (Validity). Running Algorithm I with (,(a) = In (9%/a) and we updated as per Equations (4) and (5) guarantees that the probability that F will ever contains group G where HC is true is most O, i.e.
Ainax
Pr B3 3G E gFla s.t. HG holds] Sa.
that T < Ō lm(1/8)
A
This result follows directly from the prior work referenced at the beginning of this section. At a high level, every se- quence (cxp(w))) ) is a non-negative super- super-martingale under HC: informally, this means that under the null hypoth- esis, the sequence (exp(w>)) ) should be non-increasing, in expectation. This allows us to apply Ville's inequal- ity, which guarantees that it is unlikely that exp(wf") ever becomes too large under Hg More specifically, for any a E (0, 1)) under the null, Pr[3t exp(w()) > 1/a] S a. Thus, maintaining threshold of (a(a) ln(19/a)) is suf- ficient to provide per-hypothesis o/1g/-validity guarantee, and thus o-validity overall.
4.2. Betting-style approach
We refer to our second algorithm betting-sryle approach, due to the way we construct our test statistics (Shafer, 2021;
Waudby-Smith & Ramdas, 2024; Vovk & Wang, 2021; Chugg et al., 2024); one way to interpret this approach is that the test "bets against" the null hypothesis HG be- ing true. We direct the reader to these references for more detailed technical exposition.
Updating the test statistic wy. As in the previous ap- proach, we let wf represent some accumulated amount of evidence against the null hypothesis HC by time f, with higher value of wf corresponding to greater level of evi- dence.? 2 We initialize wff = 0, and use the update rule
We also provide the following bound on stopping time; see Appendix C.3 for additional discussion of the w, notion of gap.
Theorem 4.4 (Power). Let T be the stopping time of Al- gorithm! 1 with (a( = In (19) (())/a) and wf updated as per Equations (4) and (5). If maxgeg HG - 3pp > 0, then, we (4)
+ + In (1 + (1x,eG - Bp()),
Thh quantity exp(w()) can also be referred to as an e-value (Vovk& & Wang. 2021), measure of evidence against null hy- pothesis similar to a p-value.
'The constant 2 ت is due to Cutkosky & Orabona (2018). who give a tighter version of ONS than in Hazan et al. (2007).
5

---

From Individual Experience to Collective Evidence: A Reporting Reporting-Based Framework for Identifying Systemic Harms
have that Pr[T < co] 1. Furthermore,
most drastically group came in later studies (e.g.. Witherg et al. (2021); Oster et al. (2022)).
1 +
In([91) + In(1/a)
E E < 000
In the U.S., these reports are collected inthe Vaccine Ad- verse Event Reporting System (VAERS). If we had been able to run the hypothesis tests proposed in the preceding sections on the reports collected in VAERS, would we have correctly identified this problem- and if so, how quickly? Concretely, we let Y, be the event that individual experi- ences myocarditis after receiving COVID-19 vaccine, and run the test with the end-goal of identifying elevated inci- dence rate Pr[x; X, E G] for group(s) G corresponding to adolescent men (ages 12-17 and 18-29).
W+
where w. == maXGEs.Nelo..) E El X(1x.EG 3p&) is the maximal expected one-step increase in w over all groups and choices of X.
We conclude this section with two further remarks on Theo- rems 4.2 and 4.4 in the context of our test. First, our model- ing in Section 3 measures severity of harm via a multiplica- tive factor of overrepresentation. However, both notions of gap in Theorems 4.2 and 4.4 also on the absolute size of the group PG. Thus, for two groups G and G' with identical multiplicative gaps, i.e. Pa/Na Por/ the test would stop faster in expectation for G if and only if ng= > r&-- That is, if two groups are "harmed" to the same extent, both algorithms will identify the larger one first.
Defining G. We consider (intersections of) sex and age buckets to be the subgroups of interest.* Age buckets are discretized into 0-4, 5-11, 12-17, 18-29, 30-39, 40-49, 50- 64, 65-74, and 75+; the sex categories represented in the data are (binary) male and female. After removing groups for which no vaccines were recorded, 9 contains 29 groups
Second, for both tests, the Bonferroni correction results in only an additive factor (In()))//2.. in Theorem 4.1, and In(191/w.. in Theorem 4.3) in stopping time over the scenario where had only been testing the one group with the largest gap. This means that, in terms of worst-case guar- antee on stopping time, the contribution of the Bonferroni correction is small relative to the contribution of the test level a and, especially, to the gap. In fact, the impact of Bonferroni on real-world data appears be much smaller even than this additive term.
Setting B. For this application, absolute incidence rate (that is, Pr[Y = G) is the quantity of interest to use for determining 3. As suggested by Proposition 3.4, setting 3 requires considering three quantities of inter- est: the threshold on an "unacceptable" incidence rate, the relative rates of true reporting 7 and the relative rates of false reporting TG Then, we can set B maxg ((>.² - ) IR + +G))
FR).
We will choose as the threshold on an "umacceptable" inci- dence rate. It is therefore sufficient to set , = max While this quantity cannot be determined from report data alone, conservative assumption could be that any group erroneously reports most twice the average reporting rate over the population, with > = 2.0. If the algorithm is first run with 8 = 2.0, stopping and flagging a group very quickly, the test may be re-run with increasing values of B, as a higher ,3 corresponds to a more severe true incidence rate; thus, we also show results for 3 2.5 and / = 3.5
5. RRel-World Examples
To demonstrate the applicability of our approach, we apply our framework to two real-world datasets. We begin by showing that our approach correctly and quickly identifies that young men experience myocarditis after the COVID-19 vaccine; then, mortgage allocation data, show that we identify known instances of discrimination under several reasonable reporting models. Code for all experiments, in- cluding instructions for data download and pre-processing, is available in the supplemental materials; additional experi- mental details can be found in Appendix D.
We begin by running our algorithms on the ac- tual sequence of reports in chronological order, as received in VAERS. In particular, we consider Algorithm 1 instanti- ated with wsf updated according to Equation (2) and (a(a) as in (3) and C 1/2 (Finite-sample Z-test); with w/ up- dated according to Equation (2) and (,(a) in (3) and C /3pg By6) (Asymptotic Z-test); and with wf updated according to Equations (4) and (5), and Oz(a) =
Results.
5.1. Myocarditis from COVID-19 vaccines
is by now well-known that COVID-19 vaccines induce an elevated risk of myocarditis among young men. While initial suspicions of elevated myocarditis risk relied on case studies (c.g., Mouch et al. (2021); Larson al. (2021); Marshall et al. (2021)), more systematic understanding ienediiniiiinnunnnnngggg the pattern of disproportionate impact-war was made possi- ble by post-hoc analysis of reports from incident databases. Barda et al. (2021) appears to be the first analysis based on a database of reports, but did not disaggregate by demo- graphic subgroups; the confirmation of young men as the
including
"While in principle it would have been interesting also con- sider racefethnicity, we are limited by the availability (and granu- larity) of the data given in VAERS, which does not include infor- mation on ethnicity/race in reports.
'Re-using this data is statistically valid due the equivalence between one-sided hypothesis testing and confidence sequences.
6

---

From Individual Experience to Collective Evidence: A Reporting-Basee Based Framework for Identifying Systemic Harms
Asymptotic Z-test
Finite-sample Z-test
Betting-style test
(M, 18-29)
(M, 12-17)
(M. 18-29)
(M, 12-17)
(M, 18-29)
(M. 12-17)
3 = 2.0
34 (Feb. 22)
256 (May 10)
69 (Mar. 28)
530 (May 30)
61 (Mar. 23)
241 (May 8)
8 = 2.5
49 (Mar. 10)
302 (May 15)
74 (Mar. 31)
546 (Jun. 1)
69 (Mar. 28)
259 (May 11)
8 = 3.0
70 (Mar. 30) 324 (May 18)
111 (Apr. 20)
612 (Jun. 6)
80 (Apr. 5) 302 (May 15)
Table 1: On real historical sequence of myocarditis reports, time to identification of harmed groups. In each cell, report the number of total reports to the rejection of the hypothesis corresponding to (M, 18-29) and the number of total reports corresponding to (M, 12-17). In all tests, the (M, 18-29) group is identified first vaccines were authorized for the 2-15 age group only in May.
In (aa)aa (Betting-style -tyle test). For the asympiotically-vaald Z-test, we require minimum stopping time of & 25, to prevent early rejections. We run all experiments for = 0.1.
Time to first alarm 2)
100
80-
In Table 1, we report the stopping that is, the number of reports it takes for the first null hypothesis to be rejected- of each algorithm for various values of 3. as well as the corresponding date by which an alarm would have been triggered. Note that, in all tests, the (M, 18-19) group is identified first. This is consistent with the timeline of regu- latory approvals: vaccines were authorized for ages 12-15 only by May 10 (Lovelace, 2021).
# of permutations
60-
40
20-
Algorithm
Betting-style Asymptotic Z Z-test
Finite-sample Z-test
50
150
200
250
300
350
To explore the robustness of these results, we also run syn- thetic experiments, permuting the ordering of reports to get sense of possible variance in the stopping time. We run 100 random permutations of the full set of reports. Figure tracks the number of reports it takes for each algorithm to reject the null hypothesis for any group-that is, a scenario when the test is stopped and an alarm is raised as soon as one harmed group is identified. Each point on these plots reflects the number of trials (out of 100) in which a rejection has occurred by time 4, when tests are run at [ 2.
14 of reports received) at which first alarm was triggered
Figure Stopping time (i.e. first identification harm) for each algorithm, over 100 random permutations of COVID- 19 vaccine reports, with / = 2. Each point on the plot reflects the number of trials (out of 100) in which rejection has occured by time t.
than was actually used for this finding.
5.2. Mortgage Allocations
In Figure 1, we compare the performance of the three al- gorithms. To interpret the figure, by time t = 100, the asymptotically-valic z-test had already identified harm in all 100 permutations; the betting-style test identified harm in around 80 permutations; and the finite-sample z-test had only identified harm in around 20 permutations. Figure shows clear ordering in terms of how quickly each al- gorithm tends to identify harm: the asymptoticallly-all sequential z-test (dashed, red) is faster than the betting-style algorithm (solid, purple), which is faster than the finite- sample z-test (dotted, yellow).
In 2021, Martinez & Kirchner (2021) found that, based on publicly-released data from the Home Mortgage Disclosure Act (HMDA), substantial racial disparities in 2019 loan ap- provals persisted even after controlling for financial status of applicants-most notably, healthy deht-to-income ratios (DTT). If loan applicants had been able to submit reports when they believed they had experienced unfavorable out- comes, could those reports have been used to identify this discrimination? If so, how accurately, and how quickly?
We are interested primarily in disparity among applicants with healthy DTI, even though all loan applicants would have been eligible to submit reports. Concretely, we let A, 0 be the event that loan is not made to applicant i, and 7, = 1 be the event that applicant has healthy debt- to-income ratio. Then, we let Y; = (A, = 0, Z, = 1} be the event that individual i has a healthy DTI and did not receive a loan, and run the test with the end-goal of identifying groups that have relatively high rates of loan denials for
Overall, our experimental results suggest that our proposed tests would in fact have been effective in determining that young men were disproportionately affected by myocarditis. Moreover, though it is difficult determine exact timelines and the nature of clinical practice during early phases of the vaccine rollout, it is possible that such test could have identified problems using less data- that is, more quickly-
7

---

From Individual Experience to Collective Evidence: A Reporting Bashd Framework for Identifying Systemic Harms
Asymptotic Z-test
Finite-sample Z-test
Betting-styy test
Reporting model
Stopping time
Rel. risk
Stopping time
Rel. risk
Stopping time
Rel. risk
Correlated
85
1.62
2002
1.67
638
1.70
1.2
All Denials
69
1.59
1546
1.60
519
1.65
Anti-Con
60
1.50
1065
1.53
403
1.65
Correlated
316
1.69
4306
1.73
1542
1.77
8 1.4
All Denials
163
1.62
3214
1.72
1073
1.72
Anti-Corr.
95
1,47
2215
1.66
718
1.68
Correlated
886
1.68
11755
1.73
4157
1.82
B = 1.6
All Denials
586
1.69
7410
1.72
2714
1.75
Anti-Corr.
271
1.05
4668
1.72
1688
1.71
Correlated
4959
1.74
164252
1.98
8 1.8
All Denials
2703
1.72
297513
1.73
9977
1.89
Anti-Corr.
935
1.58
14072
1.73
4629
1.76
Table Average stopping times (i.e. time to first alarm) and true relative risk (i.e., Pr|4,=0,2==1Ix,d; first-identifed group over 100 Pr|A,=0,
random permutations. for varying 3, across algorithms and reporting models. For 8 = 1.8, some combinations of algorithm/reporting model failed to stop within 40,000 steps for some trials: 'stopped in 0/100 trials, "stopped in 99/100 trials, "stopped in 76/100 trials.
applicants with healthy DTI, i.e. 1, [EG]]
ods under varying models of reporting apporting-thatto is, whether our algorithms identify groups that do in fact have high rates of healthy DTI denials, even if not the case that every report X, corresponds to Y actually occurring. Modeling the idea that reporting behavior may be related to finan- cial health, we simulate the following possible patterns of reporting.
Defining G. While Martinez & Kirchner (2021) analyzed disparities with respect to race, we define groups as all pos- sible intersections of demographic features across gender, race, and age. The available race categories include Native, Asian, Black, Pacific Islander, White, and Latino; sex cate- gories include female, male, and unknown/nonbinary: and age categories include <25, 25-34, 35-44, 45-54, 55-64, and 65+. In total, after removing groups which comprise less than 0.1% % of all loan applicants, G contains 115 groups.
(1) Correlated:
The likelihood of reporting increases with financial health. That is, "Healthy" denials report with probability 0.9, "Manageable" with probability 0.5, "Unmanageable" with probability 0.3, and "Struggling" with probability 0.1. Under this reporting model, the 95th-percentile (among all groups) pa/p is 1.2, and maxg pa/p 1.4.
Setting B. this application, the quantity of interest is relative risk, so we draw on Proposition 3.2 to inform our setting of B. We will our relative risk threshold be 1.2- that is, want our algorithm to raise alarm when any group experiences event Y 20% more frequently than aver- age over the population. Recall Definition 3.1 and Propo- sition 3.2: to flag relative risk at 1.2, & should be set to PriR=110 and 1.2 where b = maxg pa/p. with PG P+YY=IG
(2) All Denials: financial health. Under this reporting model, the 95th- percentile pa/p 1 1.5, and maxç pa/p = 2.3.
All denials submit reports regardless of
(3) Anti-Correlated The (unlikely) case where individu- als with worse financial health are more likely to re- port, i.e. "Healthy" denials report with probability 0.1, "Manageable" with probability 0.5, "Unmanageable" with probability 0.7, and "Struggling" with probability 0.9. Under this reporting model, the 95th "5th-percentile pa/p is 1.8, and maxg pa/p 2.7.
p Pr R=] that is, b is the extent which the group- conditional report-to-incidenc to-incidence ratio for any group deviates from the population average report-to-incnt toiinideence ratio.
i=
As before, can first test at B 1.2 then re-test for higher values of 3; in this case, we will also test 8 (1.4, 1.8]. Setting 3 = 1.2 corresponds to assuming b b 1, i.e., no variance in report-to-incidene ratios across groups; the additional values 3 suggest possible values of 7/6, 7/6,//3, 4/3, and 3/2, respectively.
Note that the ground-truth ratios Pa/e would have been unknown the time that a practitioner sets B3 we are able to determine these only because we have full infor- mation about the dataset and control over the reporting model. However, these computations suggest that the as- sumptions on reporting rates implied by the settings of
Reporting models. The existence of verifiable disparities in this dataset allows us evaluate the efficacy of our meth-
8

---

From Individual Experience to Collective Evidence: A Reporting Basti Framework for Identifying Systemic Harms
3 == (1.2, 1.4, 1.6, 1.8) are generally reasonable, especially after considering outliers-note the disparity between the 95th-percentile vs max ratios of pa/p, especially for the All Denials and Anti-Correlane models.
expected that stopping times (and the ground-truth relative risks) should increase with (9, the increase in stopping time is dramatic-by sometimes by orders of magnitude- what appear to be relatively small changes in 3. Moreover, the disparity in stopping time across reporting models also increases dramatically with 3. In fact, for B = 1.8, some combinations of reporting and algorithm do not stop within 40,000 steps in at least one trial.
for
Results. We run all three algorithms discussed in Section 4 at a = 0.1, for all four reporting models discussed above, and for B = (1.2, 1.4, 1.6, 1.8). For the asymptotically- valid Z-test, we (heuristically) choose a higher minimum stopping time of 50, to reflect the more challenging problem instance compared to the vaccine reporting problem. For each algorithm, reporting model, and B. we again run 100 random permutations.
6. Discussion
This work is an initial approach to using reporting databases for post-deployment auditing: we believe there is a rich range of future work that develops the ideas in this paper, both technically and conceptually.
One important question for this application is the extent to which our tests identify the type of harm we are interested in, across various reporting models: while the algorithms guarantee statistical validity in terms of overrepresentation (i.e., in terms of whether μa 2 Bpğ), they cannot intrin- sically guarantee that reports themselves reflect true harm. With the benefit of hindsight (and access to the full dataset), we are able to calculate "ground truth" relative risks; the hope for our algorithms is that they identify groups that actually do experience elevated relative risk.
On the statistical and algorithmic side, because our frame- work allows for plugging in any existing sequential test, new methods that control for multiple hypothesis testing both over time and over the number of distinet hypotheses would be directly beneficial for this application. On the other hand, one might hope for online methods that do not require pre-specifying hypotheses and instead develops them se- quentially in quasi-unsuperrvied fashion, or that improve guarantees by exploiting relationships across hypotheses, as has proven useful in multi-objectivg learning.
Our results suggest that this is indeed generally the case, although the actual behavior varies by algorithm and report- ing model. Table shows report the average stopping times and average true relative risks of the first-identified group for 100 permutations. Across all algorithms and values of 3, the stopping time under the Correlated reporting model is the longest, followed by the All Denials and Anti-Correlated reporting models. On the other hand, the relative risk of the group that is first identified in each of these settings follows the same ordering, with the Correlated model having the highest relative risk. That is, more "favorable" reporting behavior required a test to run longer, but the group identi- fied is more severely harmed, whereas more "adversarial" reporting behavior raised an alarm sooner, but identified a less severely-harmed group.
More conceptually, while the application examples in Sec- tion are somewhat stylized, they demonstrate that report- ing databases can be promising starting points for new types of post-deployment evaluation. For reporting databases to be practically useful, there are plethora of additional con- siderations to incorporate from a variety of disciplines. For instance, if a reporting system was available, how would individuals engage with them in theory, and in practice? To what extent do, and should, individual incentives affect the database, and how it is designed? How can the result of a test (a null hypothesis rejection) be contextualized by existing and emerging legal frameworks?
To the best of our knowledge, we are the first to propose individual incident reporting to identify patterns of dispro- portionate harm in interactions with a particular system; more generally, however, one might imagine that similar re- porting systems can be developed to provide insights about concerns beyond fairness. In fact, while the framework introduced in our work is not intrinsically about algorith- mic deployments, it is one way to operationalize recent regulatory movement in AI policy towards allowing for or requiring individual reports. Any way to make such reports actionable at large scale must, to some extent, aggregate of individual reports to develop more systematic evaluations of an underlying algorithm. We therefore see our work as one step towards giving voice to individual experiences-and towards having them make a difference.
Similar tradcoffs arise when comparing algorithms: the asymptoticallly-aali Z-test stops far more quickly, but ap- pears to identify less severely- groups. On the other hand, while the betting-style test and the finite-sample Z-test tend to identify samilarly-harmer groups, the latter stops much faster than the former, overall, it appears that the betting-style test is a reasonable approach to balancing fast identification with confidence in the severity of harm.
While overall trends across algorithms and reporting models consistent across values of B, seeing these results for different 33 highlights an additional insight. While it is to be
Since simulating reporting, there is no "true" historical sequence of reports to run an algorithm on, unlike in Table 1.
9

---

From Individual Experience to Collective Evidence: A Reporting-Basee Framework for Identifying Systemic Harms
Acknowledgements
Netzer, D., al. Safety of the bnt162b2 mrna covid-19 19 vaccine in nationwide setting. New England Journal of Medicine, (15(11):10781100 2021.
We are grateful to lan Waudby-Smith, Kevin Jamieson, and Robert Nowak for helpful discussions in developing this work.
Bartroff, J. and Song, J. Sequential tests of multiple hy- potheses controlling type and ii familywise error rates. Journal of statistical planning and inference, 153:100- 114, 2014.
JD is supported in part by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 2146752. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and not necessarily reflect the views of the National Science Foundation, JD also thanks the Al Policy Hub at UC Berkeley for funding support in the 2023-2024 academic year. BR is generously supported in part by NSF CIF award 2326498, NSF IIS Award 233188 881, and ONR Award N00014-244 1-0011-24-1-2531. IDR is supported by the Mozilla Foundation and MacArthur Foundation.
Bertsekas, D. and Tsitsiklis, N. Introduction to probability, volume 1. Athena Scientific, 2008
Biden, J. R. Executive order on the safe, secure, and trust- worthy development and use of artificial intelligence. 2023.
Byun, Y., Sam, D., Oberst, M., Lipton, Z. and Wilder, B. Auditing fairness under unobserved confounding. In International Conference on Artificial Intelligence and Statistics, pp. 4339-4347. PMLR, 2024.
Impact Statement
This work proposes individual reporting databases as promising approach to post-deployment evaluation for fair- ness and performance. While we hope that the work impacts the practice of post-deployment evaluation, we do not be- lieve its societal impacts merit discussion beyond what is already given in the paper and its appendices.
Cen, S. H. and Alur, R. Ai auditing and the access ques- tion: Exploring black-box auditing and its connection to hypothesis testing. 2024.
Chen, R. T., Rastogi, S. C., Mullen, J. R., Hayes, S. W., Cochi, S. L., Donlon, J. A., and Wassilak, S. G. The vaccine adverse event reporting system (vaers). Vaccine, 12(6):542-550, 1994.
References
Cherian, J. and Candès, E. J. Statistical inference for fairness auditing, arXiv preprint arXiv:2305.03712 2023.
Adam, H., Yin, F., Hu, H., Tenenholtz, N., Crawford, L., Mackey, L., and Koenecke, A. Should I Stop or Should I Go: Early Stopping with Heterogeneous Populations. Advances in Neural Information Processing Systems, 36, 2024.
Chi, Z., Ramdas, A., and Wang, R. Multiple testing under negative dependence. arXiv preprint arXiv:2212.09776, 2022.
Agostini, G., Pierson, E., and Garg. N. Bayesian Spatial Model to Correct Under-Reporting in Urban Crowdsoure- ing. In Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pp. 21888-21 11888821896,,,, 2024.
Cho, B., Gan, K., and Kallus, N. Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams, June 2024.
Chugg, B., Cortes-Gomez, S., Wilder, B., and Ramdas, A. Auditing fairness by betting. Advances in Neural Information Processing Systems, 36, 2024.
Almada, M. Human intervention in automated decision- making: Toward the construction of contestable systems. In Proceedings of the Seventeenth International Confer- ence on artificial intelligence and law, pp. 2-11, 2019.
Cover, T. M. Universal portfolios. Mathematical finance, 1 (1):1-29, 1991.
Assembly, U. G. Seizing the opportunities of safe, secure and trustworthy artificial intelligence sys- tems for sustainable development, 2024. URL https: http::///ocuunotsso/ un www//oo///oooooooooooooooooo/ooo undoc/ 1td/n24/065 00024//65//22 http://///0/////////////2 pdf.
Cutkosky, A. and Orabona, F. Black-box reductions for parameter-free online learning in banach spaces. In Con- ference On Learning Theory, pp. 1493-1529. PMLR, 2018.
Balsubramani, Sharp finite-time iterated-logariine mar- tingale concentration. arXiv preprint arXiv: 105.2239, 2014.
Dai, J., Haghtalab, N., and Zhao, E. Learning with multi- group guarantees for clusterable subpopulations. arXiv preprint arXiv:2410. 145888 2024.
Barda, N., Dagan, N., Gen-Shlomo, Y., Kepten, E., Waxman, J., Ohana, R., Hernán, M. A., Lipsitch, M., Kohane, I.,
European Parliament. EU AI Act: First Regulation on Artificial Intelligence, 2023.
10

---

From Individual Experience to Collective Evidence: A Reporting Based Framework for Identifying Systemic Harms
Feffer, M., Martelaro, N., and Heidari, H. The Al Incident Database Educational Tool Raise Awareness of AI Harms: A Classroom Exploration of Efficacy, Limi- tations, & Future Improvements. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pp. 1-11, 2023.
Larson, K. F., Ammirati, E., Adler, E. D., Cooper Jr, L. T.,
Hong, K. N., Saponara, G., Couri, D., Cereda, A., Proco- pio, A., Cavalotti, C., et al. Myocarditis after bnt162b2 and mmma-mmaa 1273 vaccination. Circulation, 144(6):506- 2021.
Lee, K. M., Rushovich, T., Gompers, A., Boulicault, M., Worthington, S., Lockhart, J. W. and Richardson, S. S. A gender hypothesis of sex disparities in adverse drug events. Social Science & Medlccnee 339:116385, 2023.
Feng. J., Gossmann, A., Pennello, G. A., Petrick, N., Sahiner, B., and Pirracchio, R. Monitoring machine learning-based risk prediction algorithms in the presence of performativity. In International Conference on Arti- ficial Intelligence and Statistics, pp. 919-927. PMLR, 2024.
Liu, Z. and Garg, N. Equity in resident crowdsourcing: Mcasuring under-reportng without ground truth data. In Proceedings of the 23rd ACM Conference on Economics and Computation, pp. 1016-1017, 2022.
Globus-Harre I., Kearns, M., and Roth, A. An algorith- mic framework for bias bounties. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 1106-1124, 2022.
Liu, Z., Bhandaram, U., and Garg, N. Quantifying spa- tial under-reporting disparities in resident crowdsoureing. Nature Computational Science, 4(1 ((1):57-665) 2024.
Hazan, E., Agarwal, A., and Kale, S. Logarithmic regret al- gorithms for online convex optimization. Machine Learn- ing, (9(2):16991)) 2007.
Lovelace, B.
FDA Permits Use of the Pfizer- BioNTech Covid Vaccine in Kids Ages 12 to 15. fecine-ffaaccleas--ss- in-kids-ages--12-to 1--0:----112tt115hhmm 2021.
http://wwww.abbbccooo222////////
Hazan, E. et al. Introduction to online convex optimization. Foundations and Trends@ in Oplimization, 2(3-4):157- 325, 2016.
Marshall, M., Ferguson, I. D., Lewis, P., Jaggi, P., Gagliardo, C., Collins, J. S., Shaughnessy, R., Caron, R., Fuss, C., Corbin, K. J. E., et al. Symptomatic acute myocarditis in 7 adolescents after pfizer-biontech covid-19 vaccination. Pediatrics, 148(3), 2021.
Hébert-Johnson U., Kim, M., Reingold, O., and Rothblum, G. Multicalibration: Calibration for the (computationally- identifiable) masses. In International Conference Ma- chine Learning, pp. 1939-1948. PMLR, 2018.
Martinez, E. and Kirchner, L. The secret bias hidden in mortgage-approval algorithms. The Markup, 2021.
Jamieson, K., Malloy, M., Nowak, R., and Bubeck, S. lil'ucb: An optimal exploration algorithm for multi- armed bandits. In Conference on Learning Theory, pp. 423 439. PMLR, 2014.
McGregor, S. Preventing repeated real world AI failures by cataloging incidents: The AI incident database. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 35, pp. 1 15458-15463 2021.
Karimi, A.- Barthe, G., Schölkopf, B., and Valera, I. A survey of algorithmic recourse: contrastive explanations and consequential recommendation recommendations. Surveys, 55(5):1-29 2022.
Mouch, S. A., Roguin, A., Hellou, E., Ishai, A., Shoshan, U., Mahamid, L., Zoabi, M., Aisman, M., Goldschmid, N., and Yanay, N. B. Myocarditis following COVID-19 mRNA vaccination. vaccine, 39(29):3790-377 2021.
ACM Computing
Koolen, W. M. time law of the iterated logarithm result. https:/blog..wuutrrrooeennn coouterkoolen.info/Omn Lttp/oDDDD.///tttmmmm https://blog. 2017.
quick and dirty finite
Ojewale, V., Steed, R., Vecchione, B., Birhane, A., and Raji, I. D. Towards ai accountability infrastructure: Gaps and opportunities in ai audit tooling. arXiv preprint arXiv:2402. 2024.
Kulldorff, M., Davis, R. I., Kolezak, M., Lewis, E., Lieu, T., and Platt, R. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Sequential analysis, 30(1):58-78, 2011.
Oster, M. E., Shay, D. K., Su, J. R., Gee, J., Creech, C. B., Broder, K. R., Edwards, K., Soslow, J. H., Dendy, M., Schlaudecker, E.,, al. Myocarditis cases reported after mrna-based covid-19 19 vaccination the from december 2020 august 2021. Jama, 327(4)3331-340, 2022.
Landau, S., Dempsey, J. X., Kamar, E., Bellovin, S. M., and
Pool, R. Challenging the Machine: Contestability in Gov- ernment AI Systems. arXiv preprint arXiv:2406.10430 2024.
Pessach, D. and Shmueli, E. A review on fairness in machine learning. ACM Computing Surveys (CSUR), 55(3):1 44, 2022.

---

From Individual Experience to Collective Evidence: A Reporting Bashd Framework for Identifying Systemic Harms
Raji, I. D., Xu, P., Honigsberg, C., and Ho, D. Outsider oversight: Designing third party audit ecosystem for ai governance. In Proceedings of the 2022 AAAI/ACM Con- ference on AI, Ethics, and Society, pp. 557-571, 2022.
Witberg, G., Barda, N., Hoss, S., Richter, Wiessman, M., Aviv, Y., Grinberg, T.,, Auster, O., Dagan, N., Balicer, R. D., et al. Myocarditis after Covid-19 vaccination in large health care organization. New England Journal of Medicine, 385(23) 2021.
Shafer, G. Testing by betting: A strategy for statistical and scientific communication. Journal of the Royal Statistical Society Series A: Statistics in Society, 184(2)4007431, 2021.
Wu, Z.. and He, Faimess-aware model-agnostic posi- tive and unlabeled learning. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Trans- parency, pp. 1698-1708, 2022.
Shanmugam, D., Hou, K., and Pierson, E. Quantifying disparities in intimate partner violence: a machine learn- ing method to correct for underreporting. npj Women's Health, 2(1):15, 2024.
Zinkevich, M. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning, pp. 928
936, 2003.
Sharif--Mavvjerddi, S., Kearns, M., and Roth, A. Average individual fairness: Algorithms, generalization and ex- periments. Advances in Neural information Processing Systems, 32, 2019.
Shimabukuro. T. T., Nguyen, M., Martin, D., and DeSte- fano, F. Safety monitoring in the vaccine adverse event reporting system (vaers). Vaccine, 33(86))4398 2015.
Turri, V. and Dzombak, R. Why we need to know more: Exploring the state of ai incident documentation practices. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pp. 576-583, 2023.
Ustun, B., Spangher, A., and Liu, Y. Actionable recourse in linear classification. Proceedings of the conference on Fairness, Accountability, and Transparency, pp. 10-19,
2019.
Vaccaro, K., Karahalios, K., Mulligan, D. K., Kluttz, D., and Hirsch, T. Contestability in algorithmic systems. Com- panion Publication of the 2019 Conference on Computer Supported Cooperative Work and Social Computing, pp. 523-527, 023-527,22 2019.
Ville, J. Erude critique de la notion de collectif. Gauthier- Villars Paris, 1939.
Vovk, V. and Wang, R. E-values: Calibration, combination and applications. The Annuls of Statistics, 1733): 1754, 2021.
Waudby-Smith, I. and Ramdas, A. Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodol- ogy, 86(1):1-27, 2024.
Whitley, H. P. and Lindsey, W. Sex-based differences in drug activity. American family physician, 80(1 12554 1258, 2009.
12

---

From Individual Experience to Collective Evidence: A Reporting Eeporting-Based Framework for Identifying Systemic Harms
A. Additional Related Work
Identifying and defining subgroups. One approach to subgroup definition, following the line of work in multicalibration (Hébert-Johnson al., 2018), is to simply enumerate over all possible combinations of covariates. For sequential problems, per-group guarantees can be provided for subgroups that are learned online (Dai et al., 2024), though these guarantees are in terms of prediction quality rather than statistical validity. For sequential experiments, Adam et al. (2024) propose an approach to early stopping that does not require the experimenter to pre-specify the group experiencing harm, but instead identifies those who appear to be harmed more frequently. Though this is in spirit similar to the idea of identifying groups that report more frequently, their algorithm is substantially different from ours, in addition to the distinct application context.
Sequential and multiple hypothesis testing with anytime guarantees. One of our proposed tests provides anytime- validity guarantees by adapting the analysis of (Jamieson et al., 2014) and (Balsubramani, 2014). Our second proposed test leverages the recent literature on e-values (e.g. (Waudby-Smith & Ramdas, 2024; Vovk & Wang, 2021)), which can be used to construct sequential tests that have validity guarantees in finite samples. While existing literature suggests methods for global null testing that can aggregate e-processes (e.g., Cho et al. (2024) or Chi et al. (2022)), such approaches are unable to provide per-hypothesis guarantees. More classical approaches include Wald's Sequential Probability Ratio Test (SPRT) and its extensions, such as Max-SPRT (Kulldorff' et al., 201 I). or a sequential generalization of the Holm procedure Bartroff & Song (2014).
Application & policy context. Sequential hypothesis tests have been used for real-world monitoring of adverse incidents in vaccines and medical devices (see, e.g., (Shimabukuro et al., 2015)). Descriptive studies have identified disparate adverse impacts in pharmaceutical (Lee et al., 2023; Whitley & Lindsey, 2009) and vaccine settings (Oster et al., 2022). More generally, post-market surveillance is standard across various industries, especially as it relates to product safety enforcement and monitoring.
In Al policy contexts, there have already been several calls to adopt a post-market surveillance regime for Al governance (e.g.. Raji et al. (2022)). The U.N. General Assembly's first AI Resolution (7 8/265 and 78/311) explicitly encourages "the incorporation of feedback mechanisms to allow evidence-based discovery and reporting by end-users and third parties of [...] misuses of artificial intelligence systems and artificial intelligence incidents" (Assembly, 2024). In the U.S., Biden's 's (now repealed) AI Executive order explicitly directs the Department of Health and Human Services (HHS) to "establish a [...] central tracking repository for associated incidents that cause harm, including through bias or discrimination" (Biden, 2023). In the E.U., Chapter IX of the 2024 EU AI Act focuses on post-market surveillance, with Articles 85 and 87 specifically highlighting individual reporting of harms.
B. Practical Considerations
Choosing G. In our experiments in Section 5, we choose to define subgroups as all possible combinations of available demographic characteristics. That said, a practitioner may seek to define G more carefully in accordance with their application. For instance, if the goal is to illustrate discrimination in a legal sense, 9 should be defined with respect to (protected) demographic features, rather than arbitrary combinations of covariates. On the other hand, groups need not be solely demographic, which allows our approach to test for safety rather than solely fairness. For example, G could include which batch of medication an individual received; our tests could then help identify whether some batches were improperly manufactured.
Baseline rates (ag). A natural question that arises from the modeling in this section is how {no can be determined, or if Assumption 2.1 is strictly necessary. Practically speaking, these base preponderances may be estimated, possibly with some amount of noise; however, the estimation problem can be addressed with standard techniques and not core to our contribution. Similarly, in practice these baseline preponderances may change over time (e.g. if some subgroups increased uptake of vaccine, or applied for loans more frequently, over time); however, such situations are relatively straightforward to handle under our algorithmic frameworks (see, e.g., the variants discussed in Chugg et al. (2024)). We therefore focus on the case where we have access to the true, underlying values of (n&)Geg for ease and clarity of exposition.
Note that testing against base preponderances of the reference population (i.e., to compare Pa to #4)) is a new test proposed by this work, and the analysis in Sections 3.1 and 3.2 is specific to this test. Existing approaches to monitoring in incident
13

---

From Individual Experience to Collective Evidence: A Reporting Bashe Framework for Identifying Systemic Harms
databases compare different baselines, most commonly the historical overall incidence rate for the specific symptom, sometimes by subgroup (Shimabukuro al., 2015; Kulldorff et al., 201 1; Oster al., 2022). These baselines could, in principle, be plugged into the algorithms in Section 4, but new analysis for (possibly group-vryynng) reporting rates would be necessary to draw inferences about analogous quantities of interest (e.g., RR or IR), as current approaches do not generally consider reporting behavior. In contrast, our modeling allows us to identify what quantities may affect the true incidence rate even if they may be unmeasurable.
Setting 3. Finally, to run the test proposed in Equation (1), it is necessary to determine how to set the value of B. As we will see in Section 4, when / set too high, then the test may never identify problematic groups, or identify them more slowly; on the other hand, as is clear from the previous subsections, rejecting the null hypothesis for smaller 3 requires more stringent assumptions on reporting behavior. Thus, we suggest procedure to set follows: (1) choose relative risk incidence rate threshold where it would be problematic for any group if RRg IRg surpassed that threshold; (2) make the corresponding assumptions about reporting behavior; (3) use these quantities to compute reasonable B. We give some example computations in Section 5. Due an equivalence between hypothesis testing and confidence intervals, it is statistically valid to rerun tests with different Is once data collection has begun. Thus, it may be prudent to begin by setting the lowest 3 that reporting assumptions would allow; then, the tests appear to be stopping very quickly, to re-run them at higher Bs, which would allow practitioner to get better sense of the severity of the harm.
C. Omitted Proofs
C.1. Omitted proofs from Section 2
We prove Proposition 3.2, restated below.
Proposition 3.2. Define the relative risk of group G to be RRg ==
Pr]Y==1[]] Pi(Y==]] Suppose that for some group G we have
PG S p. Suppose that we determine that PG 2 Bpla for some 3 > Then, the true relative risk experienced by G is at least RRG 8/b.
Proof of Proposition 3.2. First, note that by definition of p. Pc. and RRg, we have
Pr| = 1
Pr[R 1] -
Pr[R = G] RRG 2 Pr Pr[R=1] = 5
PG S..
Pr[Y =1 G <
By Bayes' rule, Pr|R=1 Pr R=1]
furthermore, by assumption, we have Pe 2 3. The result follows from PC
Pr[R=1]
Pr[G]
combining with the previous display.
We prove Proposition 3.4, restated below. Proposition 3.4. Suppose that, for some G. it is determined that pa 2 for for some 3 > 1. As long as NG
TR > 1G
FR for
every G E G, IRo 2 8-
Proof of Proposition 3.4. Recall that we have defined PG Pr(G R), and ng Pr[G] known by Assumption 2.1. By Bayes' rule, we have Mc = [r[G R] Pt
Pr[R] = 166 Pr|R|]] Now, let us decompose Pr[R G] by "true" reports
(Y = 1) and "false" reports (Y = 0). By the law of total probability, Pr[R G] = r I + FR (1 IRG)): more precisely,
1 Pr(R G Pr(R G.Y = 1] |
Pr|R| G,Y 0] - Pr|Y G))
T IR + FR IR&)
TR
FR = 1G
+ IR TR - -
--
FR.
combining this with the Bayes' rule computation, cancelling the r factor, gives us IRG the assumption that na/e8 2 3.
PC
The result follows from
-
14

---

From Individual Experience to Collective Evidence: A Reporting- Eearrddng-Basedd Framework for Identifying Systemic Harms
C.2. Omitted proofs for Sequential Z-test
We prove Theorem 4.1, restated below.
Theorem 4.1 (Validity). Running Algorithm 1 with (+(a) as in Equation (3), setting C = 1/2, and we updated as in Equation (2), guarantees that the probability that G will ever contain a group where H6 is true is at most a, i.e.
Pr [ : E gfllo s.1. H6 holds] Sa.
To prove this result, we will use a foundational result known as Ville's inequality (Ville, 1939).
Theorem C.1 (Ville's inequality). Let {Mr heNt be a non-negative supermartingale, i.e. for all t, M; 2 0, and E[ Fil I My where Fe is the filtration (history) of all realizations of randomness up to and including time t. Then. for any E (0, we have Pr[3t: Mt > E[Mol/s] S x.
The central thrust of our proof of Theorem 4.1 due to Koolen (2017) (which itself draws from Balsubramani (2014), and refinement of Jamieson al. (2014)); we reproduce the argument in the context of our work below, though we emphasize that we do not claim the proof technique ours.
Proof of Theorem 4.1. is sufficient show that for any group G where H, holds, we have Pr[at: G G S a/191; the statement of the theorem follows from the Bonferroni correction over all [9] hypotheses.
Ville's inequality (Theorem C.I) appears similar in form to the statement we hope to prove; we therefore seek to transform our test statistic uif = Exerq [xx, E G] into a quantity that can be interpreted as a (non-negative) supermartingale. Although {wr hent is by itself clearly not a non-negative supermartingale, each w is the sum of Bernoulli trials with mean PG. and Bernoulli random variables are sub-Gaussiae with variance parameter 1/4. Each w therefore satisfies the property that ([e E[ I exp(1)//3 /8).
This holds for any 7, we will construct distribution e 1j and use it to construct a martingale M,. In particular, note that under H6 [[[["] < t- Syg. Thus, we let S. = wf - =[w = wf - t33gg. We will let M, = f (T)) exp(nS, -tn /)) Then, for any distribution Φ, {Ma)ent is a neg-negative supermartingale with respect to the randomness in realizations of reports X₁. To see this, we have
[ /(l))xpp ([( + 1(x+11 E eG]-Bul) (++1²²) dr Fr
E[M++1 Fi. = E
/ ((n) exp (nss 1-5--177 11²)
exp (n[1[X++1& EG 3c333pp] -3p8) (c+1)²)) Fr dn
< <(n) exp (1st 8
1n22 dn
= Mcs
where the inequality is due to B S Brg and subgaussianity. It thus remains to use this martingale to compute an appropriate threshold (a) on wgf
Mt will satisfy the conditions of Theorem C.I for any choice of o, including which puts point mass of 1 on 77 7/' and 0 elsewhere, i.e. ((r') = 1 and Φ(n) 0 for any 17 A ry'. One path towards establishing the threshold (a)) is to simply pick one value of 77, however, such an cannot depend on and would thus result in a suboptimal threshold. Instead, we will construct & such that it a discrete distribution, indexed by E N+ so that 1j takes values ,,,..., with probability ......, Φ;; this allows each 7/ to depend on and therefore more finer-grained optimization of the threshold. Before committing to the exact distribution é. we first illustrate how di and 17s will be used in the threshold.
Note that M, Σieni di exp(()ss - tn//8) 2 max; di exp(n: 1n7 /8), for any 8, have
(M 2 1/8} 2 {max os exp(n:S tn 1/6} = { St min
tmp
)
+ In
8
2,6
and thus, picking 0,(a:) = =1,3pg + min;
tm (8
+ In क..ा.
would guarantee that Pr w 0r(a)) S a//9/.
15

---

From Individual Experience to Collective Evidence: A Reporting Eeporting-Based Framework for Identifying Systemic Harms
Finally, we must commit to 9r "h then optimize for Let d4 a(+1)) (note that Σ, db 1, so this is a valid distribution), 7 2
2 2l1(1/6/(0/111) and i = [[oog(()]. For logy(1) (t) (without rounding), this would have yielded = 2 (ln((log,())) ((ogg(t)(((/aa))) and 0r(a) = 1 26 26 In ()ogg + 1)/a/191). The statement follows from
2'
handling the numerical impact of rounding.
Remark C.2. A key constant in the proof of the version of the algorithm that is valid in finite samples is the subgaussian variance parameter, for which we used 1/a (and which propagates to a multiplicative factor of v1/4 = 1/2 on the threshold), This is because the variance any Bernoulli is at most 1/4; however, this also motivates the choice of constant for the asymppootically-valid version of the test, which instead uses the variance parameter 3/ Br&)
We now prove the power result.
Theorem 4.2 (Power). Let T be the stopping time of Algorithm 1 with 0c(a) as in Equation (3), C 1/2/2, and wf as in Equation (2). Let Amax maxGes PG PG- 3p& If Amax > 0, then Pr(T co] = 1. Furthermore, with probability %/10),
we have T < Ö In(19)))n(1/a)) and for any (0, a/19)), we have with probability least 1- that T O In(1//
Amax
Proof. Let G = arg maXGeg Nc By6 and let A = PG- BpG.. Without loss of generality, we can consider only the test corresponding to G* (while still testing at level o//G)). Recall that for this instantiation of Algorithm 1, the test statistic w = -.e(] 1[2 E G*] is simply the number of all reports belonging to G by time t, and that stopping time T is the
first time where wf" surpasses the threshold (a()), i..., T == inf wg tBpl. + b/y 2.06t In (19) (2+1og,00))) For
a
ease of notation, we will denote C1 == 1/2 V22006 0.718 within this proof.
For the first claim, it is sufficient to show lim inf Pr[T > 1 o.? Recall that, by our modeling, we can consider wf to be the sum i..dd Bernuulii trials with parameter pc... Applying Hoeffding's 's inequality to this sum yields for any that
Pr[T 1 = Pr wf <tBpg +cy
(19)
logz (t))
a
-2 A - 12--2Ac/// /
(2 + logg(1)))
I exp
Note that (1) O; can thus be seen that lim Pr[f's t lim exp(-t) 1.
For the second claim, we apply Hoeffding's inequality again to that for all
(2+ log, (t)) (2+1og,(t))
1 ④
Pr wi < [[w] Ci v fln
< Pr w/ sBww In
S
8.
8
Thus, with probability at least 1 - 8, for all / simultaneously, w > lua- - Cipy Inn (2+log.(4)2 at time t if and only if
The algorithm stops
(2+ (o))())))
( (1))
( (2+
tpc -1 thn
> t8p + +c Cipten
8
a/10/
Rearranging, we have
C
2 2
A
(1+) a/i0|
In (2+1og.(0))
In
+
Forr a simple proof of this fact, see the solution Problem 1.13 in Bertsekas & Tsitsiklis (2008).
16

---

From Individual Experience to Collective Evidence: A Reporting Bassd Framework for Identifying Systemic Harms
2
Note that we can can upper bound the denominator of the left hand side as
(vm
(2+log, 9/101
+
(2+log,(0))
S
d
4ln
(2+Hog,(t))
Setting
and rearranging gives A
min("/191.5
4
(B+1og2(01= (1011.))
t
4C1 In(max((//0011111 1/11)
(6)
1+ (n( logg (1))2)
A
Thus, with probability 1 5, the algorithm terminates at the smallest satisfying Equation (6). The statement of the theorem follows from separating the two for a/1g) and a/19), and noting that O notation suppresses the (negligible) log-log factor.
C.3. Omitted proofs for betting-style algorithm
We first prove Theorem 4.3, restated for the sake of presentation.
Theorem 4.3 (Validity). Running Algorithm 1 with (,(a) In (191/) and we updated as per Equations (4) and (5) guarantees that the probability that Ffln will ever contains group G where Hy is true is most o, i.e.
Pr Bt 3G E FlaR s.t. H6 holds] S a.
3G
Proof. First note that for any G for which Hg holds, the sequence (exp(w))) Pr20 is non-negative super-martingale. The non-negative property follows directly from the quantity being an exponential of real (albeit possibly negative) number, while the fact that it is super-martingale follows from the computations below:
Elexp(w;)) Elexp(wf_1 In(1 +
31()
exp(wf (1+ (L+ 19 -3pd))
exp(wx_1 .). (1 GHG B1())
S exp(wx_1 . (1 + (8 Bp(a))
= -xp(w=f1),
where the first equality follows by Eq. the second by re-arranging and noting that all quantities except x, completely determined by J,S, and the third by definition (see Section 2). Finally, the inequality follows because μG S 8pg under HC and X 2 O.
Next, for any group G such that HC holds, we can apply Ville's inequality (Theorem C.I), plugging in the super-martingale (exp(wf))=20 and taking x to be (a(a) = log (l91/a). This yields the following guarantee:
Pr(3t : w > log([9//o) Pr(3t exp(w:") [g]/o]
S [[(ep((w a/l9
a a/
where the final line follows because w initialized as and hence cxp(w6) is equal to Finally, by union bound we get the desired guarantee:
Pr[3t: 3G E Flog s.t. Hg holds] S
Σ
Pr(3t w > log (191/a))
G G. Ng holds
< IG s.t. H6 holds a//g
< a.
Before proving Theorem 4.4, we first state and prove some helper results.
'In particular, it imposed that X 'predictable' which precisely implics that is fixed given Ft.
17

---

From Individual Experience to Collective Evidence: A Reporting-Banee Based Framework for Identifying Systemic Harms
Claim C.3. For any T 2 4 and group G, have that the expected value over the randomness in the realizations of each X w defined as per Equations (4) and (5) can be lower bounded as
max WT wT(2))
21n T.
where we define (X) to be the quantity obtained by applying Equation (4) with Xe = Afor all E (T].
By the definition of regret we have that maxxe[o.11 (f()) -w S Rr. Rearranging and taking expectations, we have
Proof.
Blwfl
2 E
max x AE [0.1
E[Rr].
Next, it can be verified that Equation (5) implements the Online Newton Step algorithm for in(1 + X (1x.ec Bp6]) (see Appendix C of Cutkosky & Orabona (2018)). We therefore have that Ry 2-im(3) In(T in general, and Ry S 2 (T() for T 4. The statement of the claim plugging this into the expression above.
NG Bpg
Lemma C.4. For each group G, taking N/= = Proijo.1]
maximizes expected log-wealth (at every step t).
sp By&)
The resulting expected log-wealth at time T (had NG been used at every time t) is equal to
E [w = w
where we denote S == E[n(( + (1x,ea 3/&) the expected one-step wealth change under the bet N
Proof. For a fixed X, the log- wealwhhlth at time T given by
(( = .y In (1 X(1 - Bp&)) + (T N2) In A3p0),
where Nr = -t 1x,c. Taking expectations, we have that E[Nr] = T μg and therefore
(W)( T [PG In (1 + A(1 - 3p&)) - - (G) In (1 A3p&)]
(7)
To maximize (7), only need to find N/ [0, 1] that maximizes the expressions in the square brackets. Taking the derivative we see that the function is concave, and, therefore, we can solve for X2 by setting the derivative to 0 and then Plugging this back into (7) we get
projecting the resulting value to [0, 1]. This yields X/= Proj
per p
Bug
W [Xt)] = T [uo (1+ -
- 3y&)) + (1 - pc) (1 A23)88]
= =T-E[l + 1x(1x.e0 - B46)l
= T...
Remark C.5. Note that we can explicitly compute
w = PG In (1+ (pg(1-Ho) + In -
I-
where Ag la Bpg, but this quantity is difficult to analyze, and it is not clear that w can be explicitly lower bounded as O(A)).
We now prove Theorem 4.4, which restate below.
Theorem 4.4 (Power). Ler T be the stopping time of Algorithm with 0z(a) = In ((91/a) and wf updated as per Equations (4) and (5). If maxceg HG - 3p% > O, then, we have that Pr[T < oc] = 1. Furthermore,
1 In([g))
+ In(1/a)
E < O
W
W+
where w. := maxces.Ne(o.j E|n(t (0]nn1+A((11)) A(1x,e - Bub))] the maximal expected one-step increase in w over all groups and choices of X.
18

---

From Individual Experience to Collective Evidence: A Reporting Eeoordiig-Basedd Framework for Identifying Systemic Harms
Proof. Let G* arg maxg の and denote the corresponding one-step wealth change w. w Note that under the alternative this will correspond strictly positive quantity and is equivalent the definition in the theorem statement. We can analyze the likelihood that its null has not been rejected by time & as follows:
Pr w < <In(191/a Pr w - B|w < (n((0//a) - -B(s
Pr wi - BBw5 < In (191//)) - (1. - 2 21n
where the inequality follows by Claim C.3 and Lemma C.4, and the fact that E/maxxx(o, w/ (A)) 2 Blw (1.") t-
Whenever t is large enough such that in(r) I w. have
Pr[w <In((9%/a E
Pr Pr [up -EWW < In((91/a) - ]01 [.)...))..
(8)
Since vl 2 ln for all I N this is satisfied in particular by taking 2 2n (19%a) }, we have
2 o
Further, note that In(19/a) < tw whenever
t t 2 (19))) So, for t 2 max 24 we
Pr[wf In(99//a S Pr wus ] (t w.)
Now, note that since X; E [0, we have that each In(1 X, (1x,e 3y&)) lies in [ln (1 3p8), 3p&)] and is therefore sub-Gaussian with parameter 1 In 1 1-g
then, Hoeffding's inequality gives
Pr w G -E(w,"1 - [t. w.)] Pr
1 Σ In(1 X²" (1x,Ea- Bp(g)) Bius")] S
- W.
2
SE|(
(tt. w.)2
S exp
1t (1 + 1-Bp6.
w
exp
2n 2 (1 1 9p
(1 - 3pd- )2
S exp
w
where for the last inequality we used In (1 + x² I x. Now we are ready to analyze the stopping time T of Algorithm 1.
Test of power one. Let Fg be the event that we stop at time I, i.e. E, {3G such that w/ 2 191/a. We have that
Pr[T 20| = Pr
lim Ooge - Es 10000
lim Pr[n, E,}]
11000
< lim Pr(-E:]
1+000
lim Pr[VG, w//² < In((s/aa)
t1000
Pr [u[" < <In(191/a)
< lim
(- (1 32622
lim exp
w2
1+000
2
19

---

From Individual Experience to Collective Evidence: A Reporting Ceporting-Based Framework for Identifying Systemic Harms
Expected Stopping Time. Since T is a positive integer, express the expected stopping time
E
Pr[T
Pr[-E] E1 A A -Ed
Pr[-Ex]
Σ
Pr[yG, w/ < <In(10%/a]
G < <In((s/aa)
Pr
1=1
24
2 In(101/a)
(- (1-
8p )2
S max
exp
w
(9)
w
WA
2
1=1
21
2 In(1s1/a)
max
w?
w+
exp ((1 - Bpgg. /w2/22
24
2
In(191/a)
2
max
w
(1 Bp..
(10)
W+
(l919//a
0
w2
W.
where (9) follows from the upper bound on Pr [w. < In(191/a)] for t 2 max
{ we
{ 2' 22 22-1)(01/)) derived in (8), and (10)
follows from exp(x)= 2 1 ++x. x.
D. Supplemental Material for Experimental Results
D.1. Supplemental material for vaccine side effect experiments
Data sources. The Vaccine Adverse Event Reporting System (VAERS) is national adverse event incident database for U.S.- U.S.-licensed vaccines, co-managed by the Centers for Disease Control and Prevention (CDC) and the U.S. Food and Drug Administration (FDA) (Chen et al., 1994; Shimabukuro et al., 2015). The database is a combination of voluntary reports from patients that have received the vaccine, as well as mandatory reports from vaccine manufacturers and healtheare professionals. For this case study, we filter the set publicly-available reports from VAERS to reports about the COVID-19 vaceine with a complaint keyword including "myocarditis." As for how a database administrator would have known focus on myocarditis a priori, one might imagine, for example, that the series of case studies found in early 2021 raised the alarm that more systematic analysis was warranted for myocarditis in particular.
To determine perdeemographic base rates, i.c. to compute Ju&Jaeg we utilize Vax View, a govermment dataset tracking national vaccine coverage (publicly accessible here), managed by the CDC. Vax View does not track vaccination rates by granular subgroups, only providing coverage rates disaggregated by age, gender, and ethnicity separately. We thus impute the vaccination rates for intersections of subgroups (e.g.. "12-17, M") by multiplying the known marginal rates (i.e., (12211,,Mr) == P(12-17)* H(M))
Additional results: Impact of Bonferroni correction. In Figure 2, we show the same axes as in Figure 1 number of reports to first alarm on the x-axis, vs. number of permutations in which an alarm was triggered on the y-axis-for the three algorithms at B = 2. Here, we show the impact of Bonferroni correction for multiple hypothesis testing on stopping time. As expected, the invalid version of the test, which has a lower threshold for rejecting each null, stops more quickly for all three algorithms (dashed, lighter). The difference between the invalid version and the valid version (solid, darker) is relatively minor, though the impact varies across algorithms.
20

---

From Individual Experience to Collective Evidence: A Reporting- Based Framework for Identifying Systemic Harms
Time to finst alars IAsympeodic 2-test, B
Time to first alam (firte--amplee F=2)
Timee to first alarm (lattnietttinngggggg.. dhy test, P
81-
63-
ireaid Do denterant Volid (with mond
Invald Besterert Valid LAit Rosteren
Imoaled (no Sonts Suil (wth Serde monO)
43
200
100
300
200
250
90
1500
375
5/44 strppott recuivedl at frre alam WAST siggedddddd
1.0 reports recelvedd an wric FIEET SINTT Enggered
1000 reportt receivdd which VIDE Digganed
Figure 2: Impact of multiple hypothesis correction on stopping time across algorithms. As in Figure 1, each point on the plot reflects the number of trials (out of 100) in which rejection has occurred by time t. In all plots, the lighter, dashed line reflects stopping time of the invalid test that does not correct for multiple testing; the dark, solid line reflects stopping time of the valid test including Bonferroni correction.
D.2. Supplemental material for mortgage allocation experiments
Data sources. We use the data (and preprocessing code) of Martinez & Kirchner (2021), which uses 2019 data from the HMDA." " The analysis of Martinez & Kirchner (2021) used the full year of data from 2019; we reduce the dataset to applications for conventional loans at three of the largest lending institutions, from applicants who have positive income. We assume that reports will only come from applicants whose loans were denied; in all, there are 183k applicants which satisfy these criteria.
Reporting models. The dataset gives several levels of financial health measured by DTI-in ascending order, are Strrgggggg "Unmanageable," "Manageable," and "Healthy:"" Modeling the idea that reporting behavior may be related to financial health, we use these categories to simulate the following possible patterns of reporting.
(1) Correlated: The likelihood of reporting increases with financial health. That is, "Healthy" denials report with probability 0.9, "Manageable" with probability 0.5, "Unmanageable" with probability 0.3, and "Struggling" with probability 0.1. Under this reporting model, the 9 (among all groups) ra/e is 1.2, and maxe Pafp 1.4.
(2) All Denials: All denials submit reports regardless of financial health. Under this reporting model, the -ereepeee po/p is 1.5, and maxg pa/p 2.3.
(3) Anti-Corree Correlated: The (unlikely) case where individuals with worse financial health are more likely to report, i.e. "Healthy" denials report with probability 0.1, "Manageable" with probability 0.5, "Unmanageable"" with probability 0.7, and "Struggling" with probability 0.9. Under this reporting model, the 55th-percentile Pc/p is 1.8, and maxe pa/p- = 2.7.
Note that the ground-truth ratios po/p would have been unknown at the time that a practitioner sets 3; we are able to determine these only because we have full information about the dataset and control over the reporting model. However, these computations suggest that the assumptions on reporting rates implied by the settings of {1.2, 1.4, 1.6, 1.8] are generally reasonable, especially after considering outliers- note the disparity between the 9 max ratios of pe/p, especially for the All Denials and anti-Correlated models.
9The Consumer Financial Protection Bureau (CFPB) collects and publishes this data from financial institutions annually, with two-year lag; the report (and our work) uses 2019 data which finalized of Dec. 31, 2022. The most recent year for which data available 2022, though it available for edits through 2025.
21


Reviews:
[{"review_id": "Vs8qPi3hki", "reviewer": "Reviewer_c4AK", "summary": "This paper work studies the problem of identifying systemic harms through individual reporting mechanisms using incident databases where individuals can report negative interactions with a system (such as loan denials or vaccine side effects) to identify subgroups disproportionately experiencing harm. The authors frame this as a sequential hypothesis testing problem and for each subgroup, test whether that group is overrepresented in reports relative to their representation in the base population by a factor of β. Under their assumptions about reporting behavior, this overrepresentation serves as a proxy for actual disparities in harm. Two approaches are are looked at for operationalizing: a sequential Z-test and a betting-style approach. The authors present results on two real-world applications: identifying myocarditis risk from COVID-19 vaccines in young men, and detecting racial disparities in mortgage loan approvals. In both cases, the methods successfully identify known instances of disproportionate harm using only a fraction of the data that was originally used to discover these issues.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time writing this review and the thoughtful questions! \n\n(Q1) This is a good question --- differential rates of (access to) reporting is something that we’ve thought about a lot. In the current version of this work, this can be modeled with the group-specific reporting parameters discussed on section 3 — and, though we don’t focus on it in the main exposition, in principle a different $\\beta$ could be set for each group. That said, while it is natural to model known underreporting (e.g. that arises due to access reasons), the current version of our framework doesn’t help with identifying or estimating the _degree_ of underreporting (which, e.g., some of the related work on L80-86 addresses). We think better understanding this question is an important direction of future work. \n\n(Q2) Our theory shows that the stopping time increases by only a logarithmic factor in the number of groups, and only additively (rather than multiplicatively). Thus, for settings where the number of groups is combinatorially large (e.g. we have $2^d$ groups in the case of $d$ binary features), we would expect an additive impact on the stopping time of approximately $O(d)$. Our experiments with 29 and 115 groups suggest that the impact of Bonferroni in practice is even less pronounced than the $\\log(|\\mathcal{G}|)$ suggested by theory, and we suspect this to be true in general. \n\nAs for algorithmic improvements, it is not obvious that current mathematical tools allow for any improvement over the Bonferroni correction. Some recent developments in sequential testing with e-values can handle composite null testing (e.g. [1]) — however, their guarantee is subtly different, in that they are only able to confirm that a harm _has_ occurred to one of the groups, but not identify which one it was. This is an area of future work we are definitely interested in, though it seems that it will require developing more sophisticated theory.\n\n[1] Cho et.al. Peeking with PEAK: Sequential, Nonparametric Composite Hypothesis Tests for Means of Multiple Data Streams", "claims_and_evidence": "Yes, I think the technical claims made in this paper are well supported. The authors provide a nice formulation of the problem in terms of hypothesis testing, with Theorems 4.1-4.4 which establishing the control of false positives and power of the proposed approaches. It would seem that we may encounter difficult as the number of groups grows large, but I don't feel that is a fundamental flaw.", "methods_and_evaluation_criteria": "Yes, I think the evaluation is reasonable. I'm not a public health expert so I'm not sure whether the scenarios are exactly aligned with what would be used in practice, but to my reading this makes sense.", "theoretical_claims": "Yes, I read through all proofs, and they are all sound.", "experimental_designs_or_analyses": "See above re:my comment on not being a subject matter expert, but the design themselves are good and demonstrate the central claims of the paper.", "relation_to_broader_scientific_literature": "This work builds on pre-deployment auditing and batch post-hoc methods (Cen & Alur, 2024; Cherian & Candès, 2023) by creating a continuous post-deployment monitoring framework, dynamically discovering affected subgroups rather than relying solely on predefined protected categories, sharing philosophical goals with multicalibration (Hebert-Johnson et al., 2018). From a statistical perspective, the work uses sequential testing frameworks to fairness monitoring, connecting with recent applications by Chugg et al. (2024) and Feng et al. (2024) but with distinct test objectives. The betting-style algorithm leverages cutting-edge advances in sequential hypothesis testing (e.g., Waudby-Smith & Ramdas, 2024).", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "I found this paper to be well described and implemented.  The paper creatively combines individual reporting, sequential testing, and fairness auditing into a coherent framework. This integration addresses a real gap in current practice for assessing disparate impacts. I thought the authors do a nice job of looking at multiple statistical approaches and assessing their efficacy. The empirical evidence is also quite good. \n\nMy main concern is how scalable this method would be in practice, but I believe that to be second order.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. What guarantees or assumptions are needed regarding access to the reporting system across different demographic groups? Since the method relies on comparing the proportion of reports from a group to their base population proportion, would systemic barriers to report submission (e.g., technology access, language barriers) impact the validity of your conclusions?\n2. While you demonstrate effectiveness with 29 and 115 groups respectively, how does your approach scale to settings where the number of potential subgroups grows combinatorially with the number of features? Are there modifications to make this more efficient beyond the Bonferroni correction?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "7Y2T01wsvd", "reviewer": "Reviewer_UJh6", "summary": "This paper introduces methods for identifying subgroups disproportionately affected by AI-related harms. It does so by applying sequential hypothesis testing methods to a stream of incidents incoming into a database. Two methods are proposed: sequential Z testing and “betting-style” approach where the test essentially “bets against” the null hypothesis. The paper includes some theoretical results on validity and shows that the two algorithms are essentially equivalent from a validity perspective. The work also tests two real-world examples: myocarditis reports from COVID-19 vaccines and mortgage allocations. In both cases, they report empirical “times to first alarm” from their tests as well as relative risk metrics.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time writing the review and the thoughtful questions!\n\n(Q1) We are overall optimistic about how our methods might work for a real-world system, and in future work we hope to develop and/or highlight collaborations with practitioners with real incident reporting databases. The linked AI incident database is not directly compatible with our framework (see also our discussion in Appendix A). While incidentdatabase.ai collects one-off stories of problematic incidents, they are not necessarily linked to specific systems — any incident with any AI system is eligible for report — which makes it hard to make claims about patterns of problems with specific systems. While perhaps those incident reports could be separated by an associated AI system, an additional challenge is to formalize and identify appropriate “base rates” or “subgroups” for this setting.\n\n(Q2) This is definitely a question on our minds for future work. One related work we mention [1] identifies subgroups by clustering online, but the task of running a hypothesis test is much more stringent than making predictions. Dealing with the intricacies of a sequential hypothesis test is the main technical challenge in this setting — such an approach must address data reuse for both identifying subgroups and running a valid test (i.e., avoiding the sequential analogue of p-hacking). \n\n[1] Dai et.al. Learning With Multi-Group Guarantees For Clusterable Subpopulations", "claims_and_evidence": "The claims in the work do seem to be supported by clear and convincing evidence. Both theoretical proofs and empirical results on real datasets are provided. I do not find any problematic claims.", "methods_and_evaluation_criteria": "The methods and evaluation criteria are appropriate for the problem being solved. The framing of the problem as a sequential hypothesis test is both a novel formulation and clever method to elucidate impacted groups.", "theoretical_claims": "I did not verify the proofs in detail.", "experimental_designs_or_analyses": "As far as I could tell without running code or exploring data myself, the experimental designs and analyses seem sound.", "relation_to_broader_scientific_literature": "This work is an important contribution to the broader literature on AI harms. Previous work has established and described some of the incident databases of the type described in this paper. Others have taxonomized systemic risks of harm to people. None, to my knowledge, have proposed such a method for identifying groups that are more likely at risk of harm based on existing and incoming incident reports.", "essential_references_not_discussed": "I am not aware of any essential references that were not discussed.", "other_strengths_and_weaknesses": "The primary strength of this paper is in its clever formulation of the subgroup detection problem as a hypothesis testing problem. I think this gives the method considerable flexibility to detect a wide variety of vulnerable groups. The work is clearly organized and presented, and the experimental results are convincing. \n\nI don’t really see any major weaknesses with this work. I think it’s a well done contribution that is important and unique. It should be in ICML because it deals with the ever more important realm of AI harm detection in a novel and sound way.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "1. Given the times to first alarm that you observe, do you have a sense of how well these methods might perform in terms of runtime, etc. in practice in a real system? For example, could you see this being implemented on the existing AI incident database at https://incidentdatabase.ai/.\n\n2. Do you have thoughts on how to handle a situation where you do not have access to sensitive group variables or demographic data? Would it be possible to model the group membership as a latent variable with some uncertainty? Or try to extract relevant variables from the report itself?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "J0ccS1xRyT", "reviewer": "Reviewer_y9JF", "summary": "The authors propose a framework to identify subgroups that are more likely to experience adverse events in a incident database. Therefore, they construct two algorithms that can deal with the sequentially arriving events to perform hypothesis testing. They show that their algorithms work nicely in empirical practice.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time writing the review and reading our paper! It is a good catch on the 4.1 proof, and we agree it would be clearer to break it up as you suggested — we’ll do so in the revision!\n\nTo answer your question about handling variations in $\\mu_G^0$, [1] show in Section 3 how to extend their standard algorithm to handle a variety of settings where the problem varies over time. These properties come almost “for free” from the testing by betting setup, and doing something analogous for our betting-style algorithm is straightforward despite our tests themselves being different. We glossed over this point a bit in the version of the draft we submitted but will be more explicit in our revision.\n\n[1] Chugg et.al. Auditing Fairness by Betting", "claims_and_evidence": "The authors break the argument for their claim (identify subgroups with adverse events leveraging reports of negative interactions) into three parts:\n1) They relate reported incidence rates to true incidence rates, under assumptions on the reporting behavior of the group (proofs in appendix).\n2) They show the theoretical validity and power of their two algorithms (proofs in appendix).\n3) They further support it with convincing empirical evidence. They show how assumptions on the reporting behavior might be chosen and how to relax them bit by bit (while still performing valid tests).", "methods_and_evaluation_criteria": "Their approach address how to access fairness claims from individual reports, and how to do so on a regular basis. Hence, they show a way how to implement mechanisms to ensure fair treatment of AI systems in practice.", "theoretical_claims": "I've checked the proofs of Proposition 3.2, Proposition 3.4, Theorem 4.1 on validity of the sequential z test and Theorem 4.3 on validity of the betting style test. They are well-written and sound.\n\nThere is one tiny error in the proof of Theorem 4.1. When showing that $M$ is a supermartingale, they take the $exp(-t\\eta^2/8)$ out of the conditional expectation in the second equality. However, it's also inside $exp(-(t+1)\\eta^2/8)$ instead of $exp(-1\\eta^2/8)$.\nIn my opinion, it would be worth it to show the two steps for the next inequality in detail (upperbound - $\\beta\\mu$ by - the expectation & subgaussianity). Showing the supermartingale property is the crucial proof step afterall, and might be checked by others too.", "experimental_designs_or_analyses": "The authors show two experiments, one fully empirical (real-world data and reporting) and one semi-synthetic (real-world data and synthetic reporting). They are well-conducted and give insights into expected stopping times of their algorithm.", "relation_to_broader_scientific_literature": "They expand the literature on sequential testing for monitoring adverse incidents, with the focus on group fairness.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "The paper is well-written, and address an important question, especially in light of the recent political developments.", "other_comments_or_suggestions": "The authors could also stress in the main paper, that it is statistically valid to rerun tests with different $\\beta$. It is done so in the empirical results, but the argument that it is statistically valid is only mentioned in the appendix.", "questions_for_authors": "1. The authors mention in the practical considerations that varying baseline preponderance can be handled under their framework. How?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yVrs1zwczI", "reviewer": "Reviewer_zPe3", "summary": "This paper introduces a method for identifying systemic discrimination or harm by aggregating individual reports of adverse events. The authors formalize this as the incident database problem, where reports arrive sequentially and are analyzed to detect subgroups that experience disproportionate harm. \n\nThe authors propose a sequential hypothesis testing framework that determines whether specific subgroups are overrepresented in reports of harm.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your time writing the review! We have grouped responses to your comments below. If there are any further weaknesses in the work that are concerning for you, please don’t hesitate to let us know. \n\n**Experiments**\n\n> _“For the simulated experiment, it would be better to show the result from synthetic experiment with different parameters for analysis.”_\n\nTo clarify, the mortgage experiments in 5.2 are only partially simulated: they draw from real-world data, and the component that we simulate is reporting behavior, which is exactly the main parameter of interest for our problem setting. \n\nTo this end, we simulate three different patterns of reporting, as discussed on L396. In Appendix D.2, we give more details on these simulated reports, as well as how they relate to the parameters that affect modeling (i.e. $\\rho_G/\\rho$) that were discussed in Section 3.1. As our computations (in D.2) show, the three ways that we simulate reports do correspond to meaningfully different values of the parameter $\\rho_G/\\rho$. \n\nIf you meant something else by “parameter,” please let us know! \n\n> _“a explicit false positive/negative rate analysis could help reader understand better.”_\n\nIn our problem setting, we felt that false positives/negatives were not quite the right abstraction for understanding the performance of algorithms. Thus, while we covered measures that are conceptually similar, we did not use the language of false positives/negatives explicitly. We will outline how these ideas relate below. \n\nFor false positives: The notion of “false positive” in our setting is subtle. For the pure hypothesis testing setting, a false positive would be a group for which $\\mu_G \\leq \\beta\\mu_G^0$ but is returned by the algorithm; the likelihood of this error is what is provably controlled at level $\\alpha$. For both the vaccine and mortgage experiments, all tests identified groups where $\\mu_G \\geq \\mu_G^0$, i.e. “FPR” of 0. \n\nFor our application, we are also interested in a notion of “true” harm — i.e., we hope that the groups identified by the algorithms actually reflect groups that (post-hoc) we know to have been harmed. For the vaccine experiment, this was broadly the “young men” category; all our algorithms only identified the groups (M, 12-17) and (M, 18-24), also suggesting a “FPR” of 0. For the mortgage experiment, we wanted to identify groups with a high relative risk of denial in general, but there are not necessarily hard cutoffs for what would have counted as a “true/false positive.” Tables 2-3 show that our algorithms generally found groups with high true relative risk, suggesting a low “FPR” overall. \n\nFor false negatives: Because our tests are sequential and could run for arbitrarily many timesteps, it is impossible to ever fully conclude that a non-null hypothesis has not been rejected.  (In fact, our power results indicate that, in the limit of $t \\to \\infty$, both our proposed tests will, with probability 1, identify any group with $\\mu_G > \\beta\\mu_G^0$.)\nFurthermore, all of our algorithms stop fairly quickly (see Tables 1, 2, 3) in almost all the settings we test. As we discuss in Table 3, there are a handful of settings where a group has not been identified within 40k steps; heuristically, these could be considered “false negatives” and we report those rates in Table 3. \n\n\n**Writing (clarity & structure; problem definition)**\n\n\nWhile our work does involve many moving parts, we have done our best to keep the presentation modular. Section 2 is focused fully on notation and model; the beginning of Section 3 gives the problem statement explicitly; and the beginning of Section 4 outlines our general solution concept. \n\nWe would love to hear if there are specific aspects of the presentation that were unnecessarily confusing, or any concrete suggestions for revision that would improve clarity. \n\n**Knowledge of base rates**\n\n> _“Assumption that baseline rates (μ0G) are known, which is unclear in practice.”_\n\nWe believe that there is a strong case to be made that it is reasonable to expect $\\mu_G^0$ to be tracked by system owners. \nFor example, it is already mandatory, by the Home Mortgage Disclosure Act, for banks to record and publicize the demographic details of all home loan applicants, and the CDC tracks vaccine uptake rates). Generally, we expect that most organizations track some internal metrics for system usage even if this information is not released publicly. On the other hand, we hope that for future systems, our work is one motivation to actively to ask or mandate organizations to collect/share this data. \n\n**Other methods**\n\nWe considered the question of baselines for this problem carefully, but for our problem setting, the two algorithms proposed in section 4 are adaptations of the main approaches to sequential testing given in the literature. One notable alternative that we excluded was Wald’s SPRT, which requires making more parametric decisions and is thus not directly comparable.", "claims_and_evidence": "The paper proposed two statistical algorithms (Sequential Z-test and Betting-style test) that effectively identify subgroup disparities. The paper provide guarantee by derive bounds on the error probability stopping time.", "methods_and_evaluation_criteria": "The proposed method is evaluated with synthetic and real-world dataset. The paper measures how quickly the method can detect harm, which is critical for real-world applications.", "theoretical_claims": "I checked the proof of Prop 3.2, 3.4.", "experimental_designs_or_analyses": "The experimental design in the paper is generally sound and well-structured, with strong empirical validation. I have checked real-world and simulated experiment. For the simulated experiment, it would be better to show the result from synthetic experiment with different parameters for analysis.", "relation_to_broader_scientific_literature": "Earlier work formalize the fairness auditing as a batch hypothesis testing problem, this paper formulate the problem sequential hypothesis testing, and enables the use of existing method for sequential hypothesis testing.", "essential_references_not_discussed": "I'm not aware of essential references that is missed.", "other_strengths_and_weaknesses": "- The paper tackles a well-motivated real-world problem.\n\n- The paper is experimented with both synthetic and real-world dataset. \n\n- The clarity and structure of the paper could be improved, for example, explicitly stating the problem definition in a more direct and structured manner.\n\n- The evaluation of experiment seems to be limited. For example, a explicit false positive/negative rate analysis could help reader understand better.\n\n- Assumption that baseline rates $(\\mu_0 G) $ are known, which is unclear in practice.", "other_comments_or_suggestions": "NA", "questions_for_authors": "Are there other existing method that could be comparable to the proposed approach?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a framework for identifying systemic harms through sequential analysis of individual reports. It formalizes the "incident database" problem using tools from sequential hypothesis testing and applies these methods to both real and semi-synthetic datasets. The proposed approach is well-motivated and addresses the growing need for post-deployment auditing methods that are both flexible and statistically grounded.

The reviewers provided generally positive evaluations. One reviewer nicely summarized that "the paper creatively combines individual reporting, sequential testing, and fairness auditing into a coherent framework." I share this view. The theoretical analysis is sound, and the experiments help illustrate the practical applicability of the framework.

One reviewer raised concerns regarding clarity and the depth of empirical evaluation. However, the authors responded thoroughly and constructively, addressing the key points and offering specific revisions. I find their responses reasonable and satisfactory.
One additional aspect that may benefit from clarification is the definition of “harm.” While the paper adopts a statistical framing—focused on subgroup overrepresentation relative to baseline rates—harm is often understood as a causal notion. A short discussion in the final version comparing this statistical notion of harm with causal definitions, and how it aligns with or differs from related literature, would help position the work more clearly.

Overall, I believe this paper offers a thoughtful and well-executed approach to fairness monitoring in practice. I recommend acceptance.
Decision: Accept (poster)