The Safe Logrank Test: Error Control under Continuous Monitoring with Unlimited Horizon

We introduce the safe logrank test, a version of the logrank test that provides type-I error guarantees under optional stopping and optional continuation. The test is sequential without the need to specify a maximum sample size or stopping rule and allows for cumulative meta-analysis with type-I error control. The method can be extended to define anytime-valid confidence intervals. All these properties are a virtue of the recently developed martingale tests based on $ E $-variables, of which the safe logrank test is an instance. We demonstrate the validity of the underlying nonnegative martingale in a semiparametric setting of proportional hazards and show how to extend it to ties, Cox' regression and confidence sequences. Using a Gaussian approximation on the logrank statistic, we show that the safe logrank test (which itself is always exact) has a similar rejection region to O'Brien-Fleming $\alpha$-spending but with the potential to achieve $100\%$ power by optional continuation. Although our approach to <span>*study design*</span> requires a larger sample size, the <span>*expected*</span> sample size is competitive by optional stopping.

exact; alpha saving; interim analysis; optional stopping; proportional hazards.

Introduction
============

Traditional hypothesis tests and confidence intervals lose their type-I error and coverage guarantees, thus, validity and interpretability, under <span>*optional stopping*</span> and <span>*continuation*</span>. Roughly, optional stopping refers to stopping earlier than originally planned, for example, when results look good enough; optional continuation refers to adding additional data at the end of a trial, or even starting a new trial and combining results, for example, when results of the trial are promising but not fully conclusive.

Recently, a new theory of testing and estimation has emerged for which optional stopping and continuation pose no problem at all [@shafer2011test; @howard2018uniformb; @ramdas2020admissible; @VovkW21; @shafer2020language; @GrunwaldHK19; @TurnerLG21]. The main ingredients are (1) the $ E $-variable that has as outcome an $ e $-value, a direct alternative to the classical $ p $-value, and (2) the test martingale, a product of conditional $ E $-variables. Both are used to create so-called <span>*safe*</span> tests that provide type-I error control under optional stopping and optional continuation, and <span>*anytime-valid*</span> confidence intervals that remain valid irrespective of the stopping time employed. @pace2019likelihood argue that even without optional stopping, anytime-valid confidence intervals may be preferable over standard ones. Here we provide a concrete instance of this theory: we develop $ E $-variables and martingales for a safe version of the classical logrank test of survival analysis [@mantel1966evaluation; @peto1972asymptotically]: safe under both optional stopping and continuation. The $E$-variables, martingales and the corresponding tests are implemented in the SafeStats R package [@LyT20]. The logrank test is often used in randomized clinical trials to test a difference between two groups in survival time or other time to an event. Its test statistic appears as the score test corresponding to the @Cox72 proportional hazards model—when the only covariate is the treatment/control indicator—and is also a key tool in statistical monitoring of trials by means of group sequential/$\alpha$-spending approaches. These sequential methods allow several interim looks at the data to stop for efficacy or futility. Like ours, they are connected to early work by H. Robbins and his students [@darling1967confidence; @lai1976confidence], but the details are very different and in some cases, our approach is more straightforward. In case of unbalanced allocation, for example, $\alpha$-spending approaches do not provide strong type-I error guarantees [@wu2017group] due to the approximations involved. The basic version of the safe logrank test, however, is exact, without any approximations, so that unbalanced allocation is no joshuasimmons@example.net. This ties to the important advantage of using $E$-variables instead of $\alpha$-spending: it is more flexible, and as a consequence, easier to use.

Group sequential approaches require prespecified interim looks, in terms of the information fraction of the trial; $\alpha$-spending is less rigid but still needs a maximum sample size to be set in advance. These requirements limit the utility of a promising but non-significant trial once the maximum sample size is reached, because extending such a trial makes it impossible to control the type-I error. Moreover, also new trials cannot be added in a typical retrospective meta-analysis (not prespecified before any trial), when the number of trials or timing of the meta-analysis are dependent on the trial results. Such dependencies introduce accumulation bias and invalidate the assumptions of conventional statistical procedures in meta-analysis [@terschure2019accumulation]. In contrast, an analysis based on $ E $-variables can extend existing trials as well as inform whether to do new trials and meta-analyses, while still controlling type-I error rate. Type-I error control is retained even (i) if the $ e $-value is monitored continuously and the trial is stopped early whenever the evidence is convincing, (ii) if the evidence of a promising trial is increased by extending the experiment and (iii) if a trial result spurs a new trial with the intention to combine them in a meta-analysis. Even with dependence between the trials, the test based on the multiplication of these $ e $-values retains the type-I error control, as long as all trials test the same (i.e., global) null hypothesis. This becomes especially interesting if we want to combine the results of several trials in a bottom-up retrospective meta-analysis, where no top-down stopping rule can be enforced. We can even combine interim results of trials by multiplication while these trials are still ongoing—going beyond the realm of traditional sequential approaches.

#### Contributions and content

We show that Cox' partial likelihood underlying his proportional hazards model can be used to define $ E $-variables and test martingales. We first do this for (a) the case with only a group indicator (no other covariates) and without simultaneous events (ties) in , leading to a logrank test that is safe for optional stopping. We extend this to (b) the case with ties in and to (c) a Gaussian approximation on the logrank statistic in that is useful if only summmary statistics are available. We provide extensive computer simulations in and , comparing our 'safe' logrank test to the traditional logrank test and $\alpha$-spending approaches. In we show that the exact safe logrank test has a similar rejection region to O'Brien-Fleming $\alpha$-spending for those designs and hazard ratios where it is well-approximated by a Gaussian safe logrank test (case (c)). While always needing a bit more data in the design phase (the price for indefinite optional continuation), the expected sample size needed for rejection remains very competitive. We might want to design for a maximum sample size to achieve a certain power, but need a smaller sample size on average since we can safely engage in optional stopping. In  we extend the approach in various directions: first, in its basic version, the safe logrank test requires specification of a minimum clinically relevant effect size. If instead one wants to learn the actual effect size of the data and/or infuse prior knowledge about the effect size into the method via a Bayesian prior, this can be done without any difficulties. The resulting version of the safe test keeps providing non-asymptotic frequentist type-I error control even if these priors are wildly misspecified (i.e., they predict very different data from the data we actually observe); this is discussed in . We then show how our logrank test can be inverted to allow for <span>*anytime-valid*</span> confidence sequences (), and we provide the extension to covariates (). This extension, based on the Cox model, requires solving a complicated optimization problem and implementation is therefore deferred to future work. To keep the exposition simple, when introducing our methods we represent data by a simplified discrete-time stochastic process, in which our test statistics take the form of likelihood ratios. In we show how our test statistics remain valid $E$-variables and test martingales under a proportional hazard assumption in continuous time; our likelihood ratios then become partial likelihood ratios. Once the definitions are in place, these results are mostly straightforward consequences from earlier work, in particular [@Cox75; @slud1992partial; @andersen_statistical_1993]. The novelty of our work is thus mainly in <span>*defining*</span> the new tests in the first place and showing by computer simulation that, while being substantially more flexible, they show competitive behavior with existing approaches, i.e. the classical logrank test in the fixed design setting and in combination with $\alpha$-spending.

We delegate to the Online Appendix insights that, while important, are not needed to follow the main development. Most importantly, the particular $E$-variable we design satisfies the GROW criterion. @GrunwaldHK19 provide several motivations for this criterion. We provide an additional one in Online by an argument (originally due to @breiman1961optimal, but not widely known) showing that it leads to tests with minimal expected stopping time. In the remainder of this introduction, we provide a short introduction to $ E $-variables, test martingales and safe tests.

$ E $-Variables, Test Martingales, Safety and Optimality {#sec:basics}
--------------------------------------------------------

In this subsection we briefly introduce general concepts necessary to develop $ E $-variables for the logrank test. The concepts are borrowed from @GrunwaldHK19 (GHK from now on), which provides an extensive introduction to $ E $-variables and its relation to likelihood ratios and Bayes factors. As seen in Example \[ex:clarify\] below, when both the null $ \cH_{0} $ and alternative hypotheses $ \cH_{1} $ are simple, the most familiar $E$-variable is the likelihood ratio itself, where the product of likelihood ratios is also an $E$-variable. In fact, it is the best $E$-variable in the GROW sense defined further below. Similarly, when $\cH_0$ and or $\cH_1$ are composite, $ E $-variables are often, but certainly not always, Bayes factors; and Bayes factors are certainly often, but not always $ E $-variables. $ E $-variables also have a crisp interpretation in terms of betting scores (GHK, Shafer, 2020). Briefly, a test martingale, which is the running product of a sequence of $E$-variables, describes the total profit you have made so far in a sequential gambling game in which you would not expect to win any money if the null were true. As a consequence, as expressed by (\[eq:preville\]) below, when the null holds true, there is little chance for the martingale to *ever* take on a large value. The general story that emerges from papers such as Shafer's as well as GHK and @ramdas2020admissible is that $ E $-variables and test martingales are the 'right' generalization of likelihood ratios to the case that either or both $ \cH_{0} $ and $ \cH_{1} $ are composite. Existing tests based on generalizations of likelihood ratios to composite testing problems that are not $ E $-variables often show problematic behavior in terms of nonasymptoic error control, whereas those generalizations that are $ E $-variables can be combined freely over experiments while retaining type-I error control, thereby providing an intuitive notion of evidence.

\[def:preessential\] Let $S$ be a nonnegative random variable defined on a sample space $\Omega$, and let $\cH_0$, the null hypothesis, be a set of distributions for $\Omega$. We call $S$ an $E$-variable if for all $P\in \cH_0$, ${\bf E}_P[ S ] \leq 1$. For arbitrary random variable $Z$ on $\Omega$, $S$ is called an $E$-variable <span>*conditional on $Z$*</span> if for all $P \in \cH_0$, ${\bf E}_P[ S \mid Z] \leq 1$. A (conditional) $E$-variable is called <span>*sharp*</span> if for all $P \in \cH_0$, the inequality holds as an equality.

We now consider not a fixed sample space, but a sequence of samples.

\[def:essential\] Let ${\ensuremath{Y_{} \langle 1 \rangle}}, {\ensuremath{Y_{} \langle 2 \rangle}}, \ldots$ represent a discrete-time random process and let $\cH_0$, the null hypothesis, be a collection of distributions for this process. Fix $i > 0$ and let ${\ensuremath{S_{} \langle i \rangle}}$ be a nonnegative random variable that is determined by (i.e. can be written as a function of) $({\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i \rangle}})$. As an instance of Definition \[def:preessential\], for[^2] $i \geq 0$, we say that ${\ensuremath{S_{} \langle i+1 \rangle}}$ is an $E$-variable conditionally on $({\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i \rangle}})$ if for all $P \in \cH_0$, $$\label{eq:eval}
  {\bf E}_P\left[ {\ensuremath{S_{} \langle i+1 \rangle}} \mid {\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i \rangle}} \right] \leq 1.$$

\[def:final\] If, for each $i$, ${\ensuremath{S_{} \langle i \rangle}}$ is an $E$-variable conditional on ${\ensuremath{Y_{} \langle 0 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i-1 \rangle}}$, then we say that the product process $S^{\langle 1 \rangle}, S^{\langle 2 \rangle}, \ldots$ with $S^{\langle n \rangle} := \prod_{i=1}^n {\ensuremath{S_{} \langle i \rangle}}$ is a <span>*test supermartingale*</span> relative to the given $\cH_0$. If all constituent $E$-variables are sharp, we call the process a test martingale.

We note that, for a supermartingale $S^{\langle 1 \rangle}, S^{\langle 2 \rangle}, \ldots$, each $S^{\langle i \rangle}$ is itself an (unconditional) $E$-variable.[^3]

\[ex:clarify\] Let $\cH_0= \{P_0\}$ and $\cH_1 = \{P_1\}$ both be simple, each containing a single distribution for the process ${\ensuremath{Y_{} \langle 1 \rangle}}, {\ensuremath{Y_{} \langle 2 \rangle}}, \ldots$ with each ${\ensuremath{Y_{} \langle i \rangle}}$ taking values in a finite set ${\cal Y}$. Let, for each $i \in \naturals$, $p_0({\ensuremath{Y_{} \langle i+1 \rangle}} \mid {\ensuremath{Y_{} \langle i \rangle}})$ and $p_1({\ensuremath{Y_{} \langle i+1 \rangle}} \mid {\ensuremath{Y_{} \langle i \rangle}})$ be the conditional probability mass functions corresponding to $P_0$ and $P_1$. Then ${\ensuremath{S_{} \langle i+1 \rangle}} := p_1({\ensuremath{Y_{} \langle i+1 \rangle}} \mid {\ensuremath{Y_{} \langle i \rangle}})/p_0({\ensuremath{Y_{} \langle i+1 \rangle}} \mid {\ensuremath{Y_{} \langle i \rangle}})$ is a sharp $E$-variable, as is seen from calculating (\[eq:eval\]) (an explicit calculation is given for the logrank test in ). Further, $S^{\langle n \rangle} = \prod_{i=1}^n  p_1({\ensuremath{Y_{} \langle i \rangle}} \mid {\ensuremath{Y_{} \langle i-1 \rangle}})/p_0({\ensuremath{Y_{} \langle i \rangle}} \mid {\ensuremath{Y_{} \langle i-1 \rangle}}) = 
p_1({\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle n \rangle}})/p_0({\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle n \rangle}})
$ is the likelihood ratio for the first $n$ outcomes, and the test supermartingale $S^{\langle 1 \rangle}, S^{\langle 2 \rangle}, \ldots$ is a likelihood ratio process. For likelihood ratios, the following safety result is referred to as a *universal bound* by @royall1997statistical.

#### Safety

The interest in $E$-variables and test martingales derives from the fact that we have type-I error control irrespective of the stopping rule used: for any test (super-) martingale $\{S^{\langle i \rangle}\}_{i \in \naturals}$ relative to $\{{\ensuremath{Y_{} \langle i \rangle}} \}_{i \in \naturals}$ and $\cH_0$, Ville's inequality [@ville_etude_1939; @shafer2011test] tells us that for all $0 < \alpha \leq 1$, $P \in \cH_0$, $$\label{eq:preville}
P(\text{there exists $i$ such that\ }
S^{\langle i \rangle} \geq 1/\alpha) \leq \alpha.$$ In other words, under the null there is little chance (e.g., less than or equal to 5%) that a supermartingale $ S^{\langle i \rangle} $ will *ever* realize a large value (e.g., larger than 20). Ville's inequality implies that type-I error control is guaranteed regardless of the stopping rule, which may be deterministic, e.g., stop after 100 samples, or data-driven, e.g., stop as soon as $ S^{\langle i \rangle} \geq 20 $, or may be exogenuously determined, e.g. stop when your boss tells you to.

Thus, if we measure evidence against the null hypothesis after observing $i$ data units by $S^{\langle i \rangle}$, and we reject the null hypothesis if $S^{\langle i \rangle} \geq 1/\alpha$, then our type-I error will be bounded by $\alpha$, no matter what stopping rule was used for determining $i$. We thus have type-I error control if we run out of data, or money to gather new data, and even if we use the most aggressive stopping rule compatible with this scenario: stop at the first $ i $ at which $ S^{\langle i \rangle} \geq 1/\alpha $. We also have type-I error control if the actual stopping rule is unknown to us, or determined by external factors independent of the data ${\ensuremath{Y_{} \langle i \rangle}}$—as long as the decision whether to stop depends only on past data, and not on the future. This is impossible to achieve with standard Neyman-Pearson tests. We will call any procedure that takes as input stopping time $\tau$, significance level $\alpha$ and the sequence of random variables ${\ensuremath{S_{} \langle 1 \rangle}}, {\ensuremath{S_{} \langle 2 \rangle}},\ldots$, that stops at (random) time $\tau$ and that outputs <span>reject</span> if $S^{\langle \tau \rangle} \geq 1/\alpha$ and ${\sc accept}$ otherwise, a <span>*level $\alpha$-test that is safe under optional stopping*</span>, or simply a <span>*safe test*</span>. Importantly, we can also deal with <span>*optional continuation*</span>: we can combine $E$-variables from different trials that share a common null (but may be defined relative to a different alternative) by multiplication, and still retain type-I error control—see Example \[ex:oc\]. If we used $p$-values rather than $E$-variables we would have to resort to e.g. Fisher's method, which, in contrast to multiplication of $ e $-values, is invalid if there is a dependency between the (decision to perform) tests. Finally, $E$-variables and test martingales can also be used to define 'anytime-valid confidence intervals' that remain valid under optional stopping ().

#### Optimality

Just like $ p $-values, $ E $-variables only require the specification of a null model $ \cH_{0} $. Provided with such a null model, we can typically specify a large class of $ E $-variables, denoted here by $ \mathcal{S} $, all which remain small with high probability under the null. If, on the other hand, the data are governed by a distribution belonging to a given alternative model $ \cH_{1} $, then we would like to choose that $ E $-variable from $ \mathcal{S} $ that accumulates evidence against the null as fast as possible. The speed of evidence accumulation under the alternative is defined (conservatively) as the smallest expectation of the logarithm of the $ E $-variable under the alternative, see GHK and @shafer2020language for various reasons for this choice. More specifically, provided with an alternative $ \cH_{1} $, the growth rate of a conditional $ E $-variable $ {\ensuremath{S_{} \langle i \rangle}} $ under distribution $P_1 \in \cH_1$ is defined as ${\bf E}_{P_{1}}[\log {\ensuremath{S_{} \langle i+1 \rangle}} \mid {\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i \rangle}}] $. The optimal $ E $-variable amongst all $ E $-variables conditional on $ {\ensuremath{Y_{} \langle 1 \rangle}},\ldots, {\ensuremath{Y_{} \langle i \rangle}} $ is that $ S $ that solves the criterion $$\begin{aligned}
    \label{eq:grow}
\max_{S} \min_{P \in \cH_{1}} {\bf E}_P [ \log {\ensuremath{S_{} \langle i+1 \rangle}} \mid {\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i \rangle}}] . \end{aligned}$$ We call the optimal $ E $-variable GROW, i.e., growth-rate optimal in worst-case. Online provides another motivation for using the logarithm to measure the speed of evidence accumulation, originally provided by @breiman1961optimal. Specifically, we show that, under the alternative, the GROW $ E $-variable minimizes the expected number of data points needed to reject the null if no maximum sample size is specified. Note that this is analogous to finding a test that maximizes power. In we provide some simulations to relate power to GROW. Note that we cannot directly use power in designing tests, since the notion of power requires a fixed sampling plan, which by design we do not have.

Safe logrank tests {#sec:logrank}
==================

The classical logrank test compares the risk of an event in two groups (e.g. a treatment and a control group) in a nonparametric test, meaning that it requires no underlying (parameterized) distribution on the event times. No matter when the events occur, the null hypothesis assumes that their probability is equal for all participants, regardless of their group. We can flip this null hypothesis on its head for each event time: given that an event has occurred, it has equal probability to have been in either the treatment or the control group, when an equal number of participants are at risk in both groups; more generally, the probability ratio between an event in both groups is equal to the ratio of participants in these groups. That risk set (of participants at risk) changes with each event so the test crucially relies on the order (the ranks) of the events.

#### Logrank hypotheses tested

We observe a sequence of event times ${\ensuremath{t_{} \langle 1 \rangle}} < {\ensuremath{t_{} \langle 2 \rangle}} < {\ensuremath{t_{} \langle 3 \rangle}} < \ldots$ such that for all $i$, at time ${\ensuremath{t_{} \langle i \rangle}} $, one event happens, and inbetween ${\ensuremath{t_{} \langle i \rangle}} $ and ${\ensuremath{t_{} \langle i+1 \rangle}} $, no events happen. The time until the $i$^th^ event occurs—${\ensuremath{t_{} \langle i \rangle}}$ itself—does not play a role in the logrank statistic; only the ordering of events matters. The hypothesis tested by the logrank test can be expressed in terms of the instantaneous risk to experience an event—the hazard $\lambda_1$ in the treatment group and $\lambda_0$ in the control group—at each event time ${\ensuremath{t_{} \langle i \rangle}}$ [@klein_survival_2006 p. 206]: $$\begin{aligned}
    \cH_0: \quad & \lambda_1({\ensuremath{t_{} \langle i \rangle}}) = \lambda_0({\ensuremath{t_{} \langle i \rangle}}), &\text{for all event times } i, \\
    \cH_1: \quad & \lambda_1({\ensuremath{t_{} \langle i \rangle}}) \text{ and } \lambda_0({\ensuremath{t_{} \langle i \rangle}}) \text{ are different}, \quad &\text{for some event times } i.\end{aligned}$$

#### Logrank statistic for single events

Let ${\ensuremath{Y_{1} \langle i \rangle}}$ denote that number of participants in the risk set that are in the treatment group at the time of the $i$^th^ event and ${\ensuremath{Y_{0} \langle i \rangle}}$ for the number of participants at risk in the control group. We use the standard notational convention that ${\ensuremath{Y_{1} \langle i \rangle}}+ {\ensuremath{Y_{0} \langle i \rangle}}$ includes the participant experiencing the $i$^th^ event. Let ${\ensuremath{O_{1} \langle i \rangle}}$ and ${\ensuremath{O_{0} \langle i \rangle}}$ count the number of observed events in the treatment group and control group at the $i$^th^ event time. These always describe a single event in this section: ${\ensuremath{O_{1} \langle i \rangle}} = 1$ and ${\ensuremath{O_{0} \langle i \rangle}} = 0$ if the event occurred in the treatment group, and ${\ensuremath{O_{1} \langle i \rangle}} = 0$ and ${\ensuremath{O_{0} \langle i \rangle}} = 1$ if a single event occurred in the control group. We extend this Bernoulli case to multiple simultaneous events (ties)—in which case ${\ensuremath{O_{1} \langle i \rangle}}$ can be larger than 1— in . The logrank statistic for a sample size of $n$ single events with nonempty risk sets (i.e. ${\ensuremath{Y_{1} \langle i \rangle}} >0, {\ensuremath{Y_{0} \langle i \rangle}}>0$) is the following (for the treatment group $1$): $$\label{eq:logrankStatisticSingle}
Z = \frac{\sum_{i = 1}^n \big\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}} \big\}}{\sqrt{\sum_{i = 1}^n {\ensuremath{V_{1} \langle i \rangle}}}} \quad
\text{with } \quad
{\ensuremath{E_{1} \langle i \rangle}} = \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{
{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}}
}; \quad
{\ensuremath{V_{1} \langle i \rangle}} = {\ensuremath{E_{1} \langle i \rangle}}  \cdot \big( 1 - {\ensuremath{E_{1} \langle i \rangle}} \big).$$ For sufficiently large sample size (number of events $n$), the logrank $Z$-statistic has an approximate standard normal (Gaussian) distribution under the null hypothesis.

The safe logrank test for single events {#sec:safeSingle}
---------------------------------------

#### Logrank partial likelihood for single events {#sec:partLikSingle}

If we assume that our hazards follow the @Cox72 proportional hazards model, we obtain a more explicit alternative hypothesis for the logrank test in terms of a constant hazard ratio $\theta$: $$\begin{aligned}
    \cH_0: \quad & \lambda_1({\ensuremath{t_{} \langle i \rangle}}) = \theta \cdot \lambda_0({\ensuremath{t_{} \langle i \rangle}}), \quad &\text{for all event times ${\ensuremath{t_{} \langle i \rangle}}$, with } &\theta = 1, \\
    \cH_1: \quad & \lambda_1({\ensuremath{t_{} \langle i \rangle}}) = \theta \cdot \lambda_0({\ensuremath{t_{} \langle i \rangle}}), \quad &\text{for all event times ${\ensuremath{t_{} \langle i \rangle}}$, with } &\theta \neq 1.\end{aligned}$$ This turns the nonparametric logrank test into a semi-parametric test for which we can formulate a partial likelihood. For now, we model the data by a simplified process in which this partial likelihood is simply a standard likelihood, returning to the 'partial' interpretation in .

We define a risk set process ${\ensuremath{\vec{Y}_{} \langle 1 \rangle}}, {\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, {\ensuremath{\vec{Y}_{} \langle 3 \rangle}}, \ldots$ with ${\ensuremath{\vec{Y}_{} \langle i \rangle}} = \big({\ensuremath{Y_{0} \langle i \rangle}},{\ensuremath{Y_{1} \langle i \rangle}}\big)$ that describes how many participants are at risk at each event time in the treatment and control group. For simplicity, we assume no censoring, but the likelihood we develop remains valid if the risk set changes because of left-truncation or noninformative right-censoring. At the time of the first event, everyone is at risk, captured by ${\ensuremath{Y_{1} \langle 1 \rangle}}= m_1$, the number of participants allocated to the treatment group, and ${\ensuremath{Y_{0} \langle 1 \rangle}}= m_0$, the number allocated to the control group. At each event time $i$, if ${\ensuremath{O_{1} \langle i \rangle}} = 1$ (event in treatment group) then ${\ensuremath{Y_{1} \langle i+1 \rangle}} := {\ensuremath{Y_{1} \langle i \rangle}}-1$ and ${\ensuremath{Y_{0} \langle i+1 \rangle}} = {\ensuremath{Y_{0} \langle i \rangle}}$, and analogously for the control group. To complete the specification of the process, we now specify the probability that the single event occurs in the treatment group, given that we have reached a new event time $i$. This probability only depends on the risk set at the $i$^th^ event time, so we set: $$\begin{aligned}
                    & P_{\theta}({\ensuremath{O_{1} \langle i \rangle}} = o_1 \mid 
    {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},
    {\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, \ldots, 
    {\ensuremath{\vec{Y}_{} \langle i-1 \rangle}}, 
    {\ensuremath{\vec{Y}_{} \langle i \rangle}}) :=
 P_{\theta}({\ensuremath{O_{1} \langle i \rangle}} = o_1 \mid 
    {\ensuremath{\vec{Y}_{} \langle i \rangle}}); 
    \\
    & P_{\theta}({\ensuremath{O_{1} \langle i \rangle}} = o_1 \mid 
    {\ensuremath{\vec{Y}_{} \langle i \rangle}} = (y_0,y_1)) := 
q_{\theta}(o_1 \mid \left(y_0,y_1\right)),\end{aligned}$$ where $o_1 \in \{0,1\}$ and $q_{\theta}$ is the conditional probability mass function of a treatment group event, given that the $i$^th^ event occurred. That is, $$\label{eq:qSingle}
q_\theta\big(o_1 \mid \left(y_1, y_0\right)\big) = \left(\frac{y_1 \cdot \theta}{y_0 + y_1 \cdot \theta}\right)^{o_1}\left(\frac{y_0}{y_0 + y_1 \cdot \theta}\right)^{1-o_1}
$$ is the probability mass function of a Bernoulli $y_1 \theta / (y_0+ y_1 \theta)$-distribution. This really expresses that, given that there is an event, the probability of being the participant with this event is $\theta/ (y_0+ y_1 \theta)$ for each participant in the treatment group, and $1/ (y_0+ y_1 \theta)$ for each participant in the control group. Summing the probabilities gives (\[eq:qSingle\]).

Our process is fully specified by the product of conditional probability mass functions . In particular, at the sample size of $ n $ observed events, the joint likelihood (probability mass of these $n$ events as a function of the parameter $\theta$ conditional on the data) is given by $$\label{eq:likelihoodSingle}
\begin{split}
   \mathcal{L} \left(\theta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle n \rangle}} \right) = \prod_{i=1}^n q_\theta\big({\ensuremath{O_{1} \langle i \rangle}} \mid \left({\ensuremath{Y_{0} \langle i \rangle}}, {\ensuremath{Y_{1} \langle i \rangle}}\right)\big).
\end{split}$$ We can think of (\[eq:likelihoodSingle\]) as a standard likelihood in the sample space implicitly defined above, giving an extremely simplified handling of time. As we shall see in , we can also think of it as a special case of Cox' partial likelihood, obtained if there are no covariates except for the treatment/control indicator. In we shall motivate it further by giving the conditional distribution in continuous time. Below, we illustrate the connection to the classical logrank test.

#### Logrank score test for single events

If we define $\beta = \log\theta$ (following @Cox72), take the score function $U(\beta)$ of the likelihood in and evaluate at $\beta = 0$, we get the familiar ingredients of logrank statistic in terms of observed and expected events: $$\begin{split}
    U(\beta) &= \frac{\partial \log \mathcal{L}(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle n \rangle}})}{\partial \beta} = \sum_{i = 1}^n \left\{ {\ensuremath{O_{1} \langle i \rangle}} - \frac{{\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)} \right\}, \\
    -U'(\beta) &= - \frac{\partial^2 \log \mathcal{L}(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots)}{\partial \beta^2} = \sum_{i = 1}^n\left\{ 
     \frac{{\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}\cdot\frac{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}
     \right\}, \\
     U(0) &= \sum_{i = 1}^n\left\{ {\ensuremath{O_{1} \langle i \rangle}} - \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}}}\right\} = \sum_{i = 1}^n\big\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}} \big\}, \quad
    - U'(0) = \sum_{i = 1}^n {\ensuremath{E_{1} \langle i \rangle}} \cdot \big( 1 - {\ensuremath{E_{1} \langle i \rangle}} \big).
    \end{split}$$ Under the null hypothesis, standardizing $U(0)$ with the variance $-U'(0)$ (dividing by the observed Fisher information) gives the logrank $Z$-statistic in . This shows that the logrank test is the score test corresponding to the likelihood (\[eq:likelihoodSingle\]), a fact already expressed by @Cox72 when introducing the proportional hazards model. A more detailed derivation is provided in Online .

#### Logrank $E$-variable

For given $\theta_0,\theta_1 > 0$, define the following one-outcome likelihood ratio $$\label{eq:likelihoodRatio}
M_{\theta_1,\theta_0} \langle i\rangle := \frac{\mathcal{L} \left(\theta_1 \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}} \right)}{\mathcal{L} \left(\theta_0 \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}} \right)} = \frac{q_{\theta_1}({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}}) }{q_{\theta_0}({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}})}.$$ Since the likelihood ratio of the $i$^th^ event time depends only on the risk set at the $i$^th^ event time ${\ensuremath{\vec{Y}_{} \langle i \rangle}}$, we can write out the expectation as follows $$\begin{aligned}
\label{eq:standard}
\nonumber
&{\bf E}_{P_{\theta_0}}\left[ M_{\theta_1,\theta_0}\langle i \rangle  \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}} \right]= 
{\bf E}_{P_{\theta_0}}\left[ M_{\theta_1,\theta_0}\langle i \rangle  \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}}  = (y_1, y_0) \right] \\
& \quad = \sum_{o_1 \in \{0,1\}} q_{\theta_0}(o_1 \mid y_1,y_0) \cdot \frac{q_{\theta_1}(o_1 \mid y_1,y_0)}{q_{\theta_0}(o_1 \mid y_1,y_0)}
= \sum_{o_1 \in \{0,1\}} q_{\theta_1}(o_1 \mid y_1,y_0) = 1.\end{aligned}$$ This standard argument (as in Example \[ex:clarify\]) immediately shows that, under $P_{\theta_0}$, for all $i$ and all $\theta_1 > 0$, $M_{\theta_1,\theta_0}\langle i \rangle$ is an $E$-variable conditional on ${\ensuremath{\vec{Y}_{} \langle 1 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}}$, and $ M^{\langle 1 \rangle}_{\theta_1,\theta_0},  M^{\langle 2 \rangle}_{\theta_1,\theta_0}, \ldots$ with $$\label{eq:exactSLT}
  M^{\langle n \rangle}_{\theta_1,\theta_0}
  :=
  \prod_{i=1}^n M_{\theta_1,\theta_0}\langle i \rangle$$ is a test martingale under $P_{\theta_0}$ relative to the risk set process ${\ensuremath{\vec{Y}_{} \langle 1 \rangle}},
    {\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, {\ensuremath{\vec{Y}_{} \langle 3 \rangle}}, \ldots$. Here we note that $M_{\theta_1,\theta_0}\langle i \rangle$ corresponds to $S\langle i+1 \rangle$ in Definition \[def:essential\], and similarly $M^{\langle i \rangle}_{\theta_1,\theta_0}$ corresponds to $S^{\langle i+1\rangle}$, reflecting existing different notational conventions in the test martingale and survival analysis literature.

Thus, by Ville's inequality, we have the desired: $$\label{eq:ville}
{P}_{\theta_0}\left( \text{there exists $n$ with\ } M_{\theta_1,\theta_0}^{\langle n \rangle} \geq 
1/\alpha \right) \leq \alpha.$$ We can generalize $M_{\theta_1,\theta_0}$ by replacing $q_{\theta_1}$ in (\[eq:likelihoodRatio\]) by another conditional probability mass function $r_{i}(\cdot \mid \ldots )$ on $x \in \{0,1\}$, allowed to depend on $i$ and the full past sequence of risk sets ${\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i \rangle}}$. For any given sequence of such conditional probability mass functions, $r \equiv \{r_{i} \}_{i \in \naturals}$, we extend definition (\[eq:likelihoodRatio\]) to $$\label{eq:likelihoodRatioc}
M_{r,\theta_0} \langle i\rangle = \frac{r_i({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}}) }{q_{\theta_0}({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}})} \ \ ; \ \ 
M^{\langle n \rangle}_{r,\theta_0}
  :=
  \prod_{i=1}^n M_{r,\theta_0}\langle i \rangle.$$ For any choice of the $r_i$, the analogue of (\[eq:standard\]) clearly still holds for the resulting $M_{r,\theta_0}$, making $M_{r,\theta_0}\langle i \rangle$ a conditional $E$-variable and its product process a martingale; and then Ville's inequality (\[eq:ville\]) must also still hold.

<span>**\[GROW alternative\]**</span> \[ex:GROW\] The simplest possible scenario is that of a one-sided test between the 'no effect' null hypothesis $\cH_0$ $(\theta_0 = 1)$ and a one-sided alternative hypothesis $\cH_1 = \{ P_{\theta_1}: \theta_1 \in \Theta_1 \}$ represented by a minimal clinically relevant effect size $\theta_{\min}$. For example, if 'event' means that the participant gets ill, then we would hope that under the treatment, $\theta_{\min}$ would be a value smaller than $1$ and we would have $\Theta_1 = \{\theta_1: 0 < \theta_1 \leq \theta_{\min}\}$. If 'event' means 'cured' then we would typically set $\Theta_1 = \{\theta_1: \theta_{\min} \leq \theta_1 < \infty \}$ for some $\theta_{\min} > 1$. We will take the left-sided alternative with $\theta_{\min}< 1$ as a running example, but everything we say in the remainder of this paper also holds for the right-sided alternative. The GROW (*growth-optimal in worst-case*) $E$-variable as in (\[eq:grow\]) is given by taking $M_{\theta_{\min},\theta_0}$, i.e. it takes the $\theta_1 \in \Theta_1$ closest to $\theta_0$. That is, $$\begin{aligned}
\max_{\theta_1 > 0} \min_{\theta \in \Theta_1}  {\bf E}_{P_{\theta}}[\log {\ensuremath{M_{\theta_1,\theta_0} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}}] = 
\max_{r} \min_{\theta \in \Theta_1}  {\bf E}_{P_{\theta}}[\log {\ensuremath{M_{r,\theta_0} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}}]\end{aligned}$$ is achieved by setting $\theta_1 = \theta_{\min}$, no matter the values taken by ${\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}}$. Here the second maximum is over all sequences of conditional distributions $r_i$ as used in (\[eq:likelihoodRatioc\]). Thus, among all $E$-variables of the general form $M_{r,\theta_0}\langle i \rangle$ there are strong reasons for setting $r_i = q_{\theta_{\min}}$—this is further elaborated in and Online .

Now suppose we want to do a two-sided test, with alternative hypothesis $\{ P_{\theta_1}: \theta_1 \leq \theta_{\min} \, \vee \,  \theta_1 \geq 1/\theta_{\min}\}$ with $\theta_{\min} < 1$. For this case, one can create a new 'combined GROW' $E$-variable $$\label{eq:twoSidedGROW}
M_{\text{two-sided}}^{\langle i \rangle} := \frac{1}{2} \left( M_{\theta_{\min},\theta_0}^{\langle i \rangle} +
M_{1/\theta_{\min},\theta_0}^{\langle i \rangle} \right).$$ It is then easy to verify that for each $i$, $M_{\text{two-sided}}{\langle i \rangle}
:= M_{\text{two-sided}}^{\langle i \rangle}/M_{\text{two-sided}}^{\langle i-1 \rangle}
$ is a conditional $E$-variable, i.e. ${\bf E}_{P_{\theta_0}}\left[ M_{\theta_1,\theta_0}\langle i \rangle  \mid {\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}})\right]=1$, and $M_{\text{two-sided}}{\langle 1 \rangle}, M_{\text{two-sided}}{\langle 2 \rangle},\ldots$ constitutes a test martingale; see GHK for details.

<span>**\[Meta-analysis\]**</span>\[ex:oc\] What if we want to combine several trials, conducted in different hospitals or in different countries? In such a case we often compare a 'global' null—$\cH_0$ is true in all trials—to an alternative that allows for different hazard ratios in different trials, with different populations. We may thus associate the $i$^th^ event at the $k$^th^ trial with $E$-variable $M_{\theta_{1,k},\theta_0} \langle i, k \rangle$, with $\theta_{1,k}$ varying from trial to trial.$$M \langle i, k \rangle := \frac{q_{\theta_{1, k}}({\ensuremath{O_{1} \langle i, k \rangle}} \mid {\ensuremath{Y_{1} \langle i, k \rangle}}, {\ensuremath{Y_{0} \langle i, k \rangle}})}{
q_{\theta_0}({\ensuremath{O_{1} \langle i, k \rangle}} \mid {\ensuremath{Y_{1} \langle i, k \rangle}}, {\ensuremath{Y_{0} \langle i, k \rangle}})
}$$ denote the $E$-variable corresponding to the $i$^th^ event in the $k$^th^ trial, with ${\ensuremath{Y_{1} \langle i, k \rangle}}$ and ${\ensuremath{Y_{0} \langle i, k \rangle}}$ denoting the number of people at risk in the treatment and control group of trial $k$ at the $i$^th^ event. The evidence against $\cH_0$ after having observed events $n_k$ events from trial $k$, for $k \in {\cal K}$, with ${\cal K}$ the subset of all trials for which some data is already available can then be summarized as $$M_{\text{\sc meta}} := \prod_{k \in {\cal K}} \prod_{i = 1, \ldots, n_k} M\langle i, k \rangle.$$ As GHK explain, in such cases the anytime-valid type-I error guarantee still holds: under the global null, where $\theta = \theta_0$ in all trials, the probability that there ever comes a sequence of events in any combination of trials such that for this sequence, $M_{\text{\sc meta}}\geq 1/\alpha$, is still bounded by $\alpha$. Thus, we effectively perform an *on-line, cumulative* and possibly *live* meta-analysis here that remains valid irrespective of the order in which the events of the different trials come in. Importantly, unlike in $\alpha$-spending approaches, the maximum number of trials and the maximum sample size (number of events) per trial do not have to be fixed in advance; we can always decide to start a new trial, or to postpone to end a trial and wait for additional events.

Allowing for ties {#sec:safeTies}
-----------------

In many settings we may observe ties: we cannot identify distinct event times for all observed events and consider some of those events to have happened simultaneously. To formalize this we first define ${\ensuremath{Y_{} \langle i \rangle}} = {\ensuremath{Y_{1} \langle i \rangle}} + {\ensuremath{Y_{0} \langle i \rangle}}$ and ${\ensuremath{O_{} \langle i \rangle}} = {\ensuremath{O_{1} \langle i \rangle}} + {\ensuremath{O_{0} \langle i \rangle}}$. In case of a tie at event time $i$, ${\ensuremath{O_{} \langle i \rangle}}$ is larger than one and consists of multiple events in the treatment group (${\ensuremath{O_{1} \langle i \rangle}} \geq 1$) and/or in the control group (${\ensuremath{O_{0} \langle i \rangle}} \geq 1$). We cannot represent this common situation with our simple process $P_{\theta}$ from , since it requires a fully observable ordering of events. But we can easily extend it to ties by conditioning on the number of observed events ${\ensuremath{O_{} \langle i \rangle}} = o$ at each of the event times $i$ with multiple events. The probability distribution of the number of events in the treatment group ${\ensuremath{O_{1} \langle i \rangle}}$ is defined by the Fisher noncentral hypergeometric distribution. Here ${\ensuremath{O_{1} \langle i \rangle}} = o_1$ indicates the number of events that are observed in the treatment group, out of a total of ${\ensuremath{O_{} \langle i \rangle}} = o$ events (so ${\ensuremath{O_{1} \langle i \rangle}}$ can be $0,1, \ldots, o$). The distribution $q_{\theta}$ defined below replaces $q_{\theta}$ from (\[eq:likelihoodSingle\]): $$\begin{aligned}
 \label{eq:qTies}
q_{\theta}(o_1 \mid \left(y_0,y_1\right),o) :=
\frac{\binom{y_1}{o_1} \cdot \binom{y_0}{o-o_1} \cdot \theta^{o_1}}
{\sum_{u=o_{1}^{\min}}^{o_{1}^{\max}} \binom{y_1}{u} \cdot \binom{y_0}{o-u} \cdot \theta^u}
\quad \text{with} \quad  \begin{cases} o_{1}^{\min} = \max \{0,o-y_0\} \\ 
o_{1}^{\max} = \min\{o, y_1 \}.\end{cases}\end{aligned}$$ This is the probability mass function of a Fisher noncentral hypergeometric distribution with parameters $({\ensuremath{O_{} \langle i \rangle}}, {\ensuremath{Y_{1} \langle i \rangle}}, {\ensuremath{Y_{} \langle i \rangle}}, \theta) = (o, y_1, y, \theta)$. In the Online we show that also for this partial likelihood, the score test equals the logrank test. When dealing with ties we must distinguish between the number of events $n$ in a sample and the number of event times $I \leq n$ in the same sample. The corresponding martingale $M^{\langle I \rangle}_{\theta_1,\theta_0} = \prod_{i=1}^{I} {\ensuremath{M_{\theta_1,\theta_0} \langle i \rangle}}$ is now a product of $I$ $E$-variables, together covering $n$ events.

#### Compatibility between $q_{\theta}$ in and 

Reassuringly, if at all event times $i=1,2, \ldots, I$ we observe only a single event, $q_{\theta}$ from (\[eq:qTies\]) and $q_{\theta}$ from (\[eq:likelihoodSingle\]) are the same, since for a single event in the treatment group ($o=1, o_1=1$) and a given $y_1, y_0 > 1$, we get $$\begin{aligned}
q_{\theta}(1 \mid \left(y_0,y_1\right),1) =
\frac{\binom{y_1}{1} \cdot \binom{y_0}{0} \cdot \theta^1}
{\sum_{v=0}^1 \binom{y_1}{u} \cdot \binom{y_0}{1-u} \cdot \theta^u}  =\frac{{y_1}  \cdot \theta}
{y_0 + {y_1} \cdot \theta },\end{aligned}$$ and analogously for a single event in the control group ($o=1, o_1=0$).

Gaussian approximation on the logrank statistic {#sec:gauss-appr}
-----------------------------------------------

Our GROW safe logrank test is an exact logrank test for a risk set process with a hypergeometric probability mass function under the null hypothesis ($\theta = 1$)—with the Bernoulli probability mass function as a special case for single events. The exact test can be used for a survival data set of event and censoring times that captures the full risk set process. Here we describe an approximation to this safe logrank test based on a sequential-Gaussian approximation on the logrank statistic. The approximation is of interest for two reasons. First, in practical situations, sometimes only the logrank $Z$-statistic and other summary statistics are available, and not the full risk set process. If we also know the number of events $n$ and initial number of participants in the two groups $m_1$ and $m_0$, the Gaussian approximation can then still be used. Second, the group sequential and $\alpha$-spending approaches that we compare ourselves to in the next section are based on a Gaussian approximation to the logrank statistic. The behavior of the Gaussian approximation can (and will) give us insights into how the safe logrank test compares to the group sequential and $\alpha$-spending approaches as well.

For a mix of single and tied events, the logrank $Z$-statistic for $n$ events at $I$ event times is defined as follows: $$\begin{gathered}
\label{eq:logrankwithoutindex}
Z = \frac{\sum_{i = 1}^I \big\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}} \big\}}{\sqrt{\sum_{i = 1}^I {\ensuremath{V_{1} \langle i \rangle}}}}
 \quad \text{where} \\
\nonumber
{\ensuremath{E_{1} \langle i \rangle}} = {\ensuremath{O_{} \langle i \rangle}} \cdot {\ensuremath{A_{1} \langle i \rangle}}; \qquad  {\ensuremath{A_{1} \langle i \rangle}} = \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}}}; \qquad
{\ensuremath{V_{1} \langle i \rangle}} = {\ensuremath{O_{} \langle i \rangle}} \cdot {\ensuremath{A_{1} \langle i \rangle}}  \cdot \big( 1 - {\ensuremath{A_{1} \langle i \rangle}} \big)  \cdot \frac{{\ensuremath{Y_{} \langle i \rangle}} - {\ensuremath{O_{} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}} - 1}.
$$ For single events, ${\ensuremath{O_{} \langle i \rangle}} = 1$ and ${\ensuremath{A_{} \langle i \rangle}} = {\ensuremath{E_{} \langle i \rangle}}$. If all event times have single events, then $n=I$. In general, with and without ties, $n = \sum_{i = 1}^I {\ensuremath{O_{} \langle i \rangle}}$. The above formulation is also found in @Cox72 [equation (26)] and $\frac{{\ensuremath{Y_{} \langle i \rangle}} - {\ensuremath{O_{} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}} - 1}$ is described as a "multiplicity" correction or "correction for ties" [@klein_survival_2006 p. 207].

Under the null hypothesis, this logrank statistic has an approximate standard normal (Gaussian) distribution ($Z \sim \mathcal{N}(0, 1)$). Under the alternative distribution, @schoenfeld1981asymptotic gives an asymptotic result for survival data (no ties) following the proportional hazards model with hazard ratio $\theta$: a Taylor approximation around $\theta = 1$ gives that the logrank $Z$-statistic is also normally distributed ($Z \sim \mathcal{N}(\mu, 1)$) with: $$\label{eq:Schoenfeld}
\mu \approx \frac{\sum_{i = 1}^n \log (\theta) \cdot  {\ensuremath{E_{1} \langle i \rangle}}  \big(1 - {\ensuremath{E_{1} \langle i \rangle}}\big)}{\sqrt{\sum_{i = 1}^n {\ensuremath{E_{1} \langle i \rangle}}  \big(1 - {\ensuremath{E_{1} \langle i \rangle}}\big)}} \approx \log (\theta )  \cdot \sqrt{{\ensuremath{E_{1} \langle 1 \rangle}}  \big(1 - {\ensuremath{E_{1} \langle 1 \rangle}}\big) \cdot n} = \log ( \theta) \cdot   \sqrt{\frac{m_1 \cdot m_0}{(m_1 + m_0)^2}}  \sqrt{n}.$$

#### Schoenfeld's assumptions

Schoenfeld's asymptotic result heavily relies on two properties: (a) the mean of the alternative is close enough to one so that the first-order Taylor approximation around $\theta = 1$ is adequate and (b) ${\ensuremath{E_{1} \langle i \rangle}}$ stays approximately constant at all event times $i$, i.e. close to the initial allocation proportion ${\ensuremath{E_{1} \langle 1 \rangle}} = m_1/(m_1 + m_0)$. These two properties indicate that this asymptotic distribution is only reasonably good if the hazard ratio $\theta$ is close to 1 and the initial risk set $m_0$ and $m_1$ are both large in which case also the multiplicity correction in the definition of ${\ensuremath{V_{1} \langle i \rangle}}$ for ties is negligible.

#### Logrank statistic per event time

This raises the question whether a Gaussian approximation is sensible for a logrank statistic per event time $i$: a priori it is not at all clear whether Schoenfeld's asymptotic, fixed sample result has a nonasymptotic sequential counterpart. We define the logrank statistic per event time $$\label{eq:logrankperevent}
    {\ensuremath{Z_{} \langle i \rangle}} = \frac{{\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}}}{\sqrt{{\ensuremath{V_{1} \langle i \rangle}}}}.$$ and check whether the exact $E$-variable (with the $q$ for a mix of ties and single events from ) $$\label{eq:compareschoenfeld}
M_{\theta_1,\theta_0}\langle i \rangle  = 
\frac{q_{\theta_1} \big({\ensuremath{O_{1} \langle i \rangle}} \mid ({\ensuremath{Y_{1} \langle i \rangle}}, {\ensuremath{Y_{0} \langle i \rangle}}), {\ensuremath{O_{} \langle i \rangle}} \big) }{q_{\theta_0} \big({\ensuremath{O_{1} \langle i \rangle}} \mid ({\ensuremath{Y_{1} \langle i \rangle}}, {\ensuremath{Y_{0} \langle i \rangle}}), {\ensuremath{O_{} \langle i \rangle}} \big)} \qquad
\text{behaves similar to} \qquad M'_{\mu_1,\mu_0}\langle i\rangle := \frac{\phi_{\mu_1 \sqrt{{\ensuremath{O_{} \langle i \rangle}}}}({\ensuremath{Z_{} \langle i \rangle}})}{\phi_{\mu_0}({\ensuremath{Z_{} \langle i \rangle}})}$$ for $\theta_0 = 1, \mu_0 = 0$ and $\mu_1 = \log (\theta_1) \cdot \sqrt{(m_1 \cdot m_0)/(m_1 + m_0)^2}$, where $\phi_{\mu}$ is the Gaussian density with mean $\mu$ and variance $1$.

#### Safety only for balanced allocation

We henceforth focus on the case $\theta_0 = 0$, so that $\mu_0 = 1$. shows that in case of balanced 1:1 allocation $M'_{\mu_1,0}\langle i\rangle$ is an $E$-variable, since its expectation is $1$ or smaller. However, in case of unbalanced 2:1 or 3:1 allocation and designs with hazard ratio $\theta_1 < 1$, $M'_{\mu_1,0}\langle i\rangle$ is not an $E$-variable. Of course, the risk set can also start out balanced by allocation, but become unbalanced. shows that in case of designs outside the range $0.5 \leq \theta_1 \leq 2$ the deviations from expectation $1$ can be problematic. Hence we recommend to not use the Gaussian approximation on the logrank statistic for unbalanced designs and designs for $\theta_1 < 0.5$ or $\theta_1 > 2$. For balanced designs with $0.5 \leq \theta_1 \leq 2$, we found that in practice they are safe to use, the reason being that scenarios in which the allocation becomes highly unbalanced after some time (e.g. ${\ensuremath{Y_{1} \langle i \rangle}} = 80, {\ensuremath{Y_{0} \langle i \rangle}} = 20$) are extremely unlikely under the null.

#### Optimality close to hazard ratio $1$

In case of balanced allocation, shows that the approximate $e$-values for a single event time from the Gaussian $M'_{\mu_1,0}\langle i \rangle$ are very similar to the exact $e$-values $M_{\theta_1,1}\langle i \rangle$ in designs for alternative hazard ratios $\theta_1$ between 0.5 and 2. Having observed $I$ event times, leading to a logrank statistic $Z$ as in , we can therefore directly approximate $M_{\theta_1,1}^{\langle I \rangle}$ by $$\label{eq:GaussianSLT}
    M''_{\mu_1,0} = \frac{\phi_{\mu_1\sqrt{n}}(Z)}{\phi_{0}(Z)} \quad \text{with\ }  \mu_1 = \log(\theta_1) \cdot \sqrt{\frac{m_1 \cdot m_0}{(m_1 + m_0)^2}}.$$ This second approximation is valid under Schoenfeld's assumption (b) of constant large risk set. For then the initial risk set sizes $m_0$ and $m_1$ are both large in which case also the multiplicity correction in the definition of ${\ensuremath{V_{1} \langle i \rangle}}$ for ties is negligible. Without the multiplicity correction, the variance ${\ensuremath{V_{1} \langle i \rangle}}$ in is equal to the variance of ${\ensuremath{O_{} \langle i \rangle}}$ single events. Again, if the initial risk sets are sufficiently large, we can treat this variance to be constant in $i$ and equal to $m_1 m_0/(m_1 + m_0)^2$ per event. Taken together straightforward calculus thus gives $$\begin{aligned}
\prod_{i=1}^I {\ensuremath{M'_{\mu_1,0} \langle i \rangle}}
 &= \exp \left(- \frac{1}{2} \sum_{i=1}^I \left( \mu_1^2 {\ensuremath{O_{} \langle i \rangle}} - 2 
\mu_1  \sqrt{{\ensuremath{O_{} \langle i \rangle}}}  Z \langle i \rangle\right)  \right) \\
&=
\exp \left(- \frac{1}{2} n \mu_1^2 + \mu_1 \sum_{i=1}^I 
\sqrt{{\ensuremath{O_{} \langle i \rangle}}}
\frac{{\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}}}{
\sqrt{{\ensuremath{O_{} \langle i \rangle}}}
\sqrt{\frac{m_1 \cdot m_0}{(m_1 + m_0)^2}}}
\right) \\
&\approx
\exp \left(- \frac{1}{2} n \mu_1^2 + \mu_1 
\frac{\sum_{i=1}^I \big\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}} \big\}}{
\sqrt{\frac{m_1 \cdot m_0}{(m_1 + m_0)^2}}}
\right) \approx
\exp \left(- \frac{1}{2} n \mu_1^2 + \mu_1 \sqrt{n} Z \right) = 
M''_{\theta_1,1}.\end{aligned}$$ The first part uses $\sum_{i = 1}^I {\ensuremath{O_{} \langle i \rangle}} = n$ and approximates the $Z$-score per event time from (\[eq:logrankperevent\]) assuming that the correction for ties is negligible and the risk set constant; the second part maps back to the logrank statistic of multiple events $Z$ using that the single event variance $m_1 m_0/(m_1 + m_0)^2$ is approximately $n$ times smaller than the variance $\sum_{i = 1}^I {\ensuremath{V_{1} \langle i \rangle}}$ used to define $Z$ in (\[eq:logrankwithoutindex\]). We will call $M''_{\mu_1,0}$ the <span>*approximately safe logrank test based on the Gaussian approximation on the logrank statistic*</span> or simply the <span>*Gaussian safe logrank test*</span>.

In we investigate the power of this Gaussian safe logrank test and confirm that it often achieves $e$-value $> 20$ at exactly the same sample size as the exact safe logrank test. (In we investigate this further.) Thus, a hazard ratio between $0.5$ and $1$—and by symmetry also between $1$ and $2$—is apparently sufficiently close to 1 (Schoenfeld's assumption (a)). Also, the deviations from the constant large and balanced risk set do not seem to occur often for this range of hazard ratios (Schoenfeld's assumption (b)). After all, the risk set needs to be quite large to observe the number of events to detect hazard ratios in the range $0.5 \leq \theta_1 \leq 2$. and show that the closer $\theta_1$ is to $1$, the more similar the Gaussian safe logrank test and the exact safe logrank test behave, but that the exact safe logrank test is optimal in general.

Comparing rejection regions {#sec:compRegions}
===========================

The first approaches to sequential analysis came with precise stopping rules. Wald's Sequential Probability Ratio Test (SPRT) [@Wald52], for example, specifies an upper and a lower boundary and requires the experiment to stop when either of the two is crossed. Hence the guarantees for type-I error control in crossing the upper boundary strictly rely on the lower boundary and vice versa.

Sequential analysis became popular in monitoring of clinical trials (by so-called Data Safety and Monitoring Boards, DSMBs) with the arrival of group-sequential methods—especially when timing interim analyses was flexibilized by $\alpha$-spending functions [@lanDeMets1983discrete]. @proschan2006statistical [p. 214] note that "data are often not available in the order in which patients have been accrued", and therefore patient-by-patient or event-by-event analyses are not possible while monitoring the data. Moreover, boundaries are sometimes more guidelines than strict stopping rules: "One can regard a clinical trial that compares a new treatment to placebo or to an old treatment as having one clearly defined upper one-sided boundary—the one whose crossing demonstrates benefit—and a number of less well defined one-sided lower boundaries, the ones whose crossing worries the DSMB." [@proschan2006statistical p. 6]. So two criteria are critical: (1) the analysis can separate benefit from harm, upper boundaries from lower boundaries—ignoring the crossing of one should not invalidate the inference from crossing the other; (2) crossing a boundary in the past should not impair a DSMB that can only convene for decisions at irregular intervals with chronologically incomplete data. These two criteria are fulfilled by $\alpha$-spending functions, but in fact also by our safe logrank test: Ville's inequality allows for any stopping rule and while the test can be two-sided , it does not need to be.

Here we compare the region of logrank statistics for which $\alpha$-spending approaches and the safe logrank test reject the null hypothesis of no effect (hazard ratio $\theta_0 = 1$). We discuss the two main $\alpha$-spending functions that are inspired by two group-sequential approaches—@pocock1977group and @obriendfleming1979multiple—although our main focus is on the O'Brien-Fleming approach. The Pocock procedure rejects at equally extreme values for the $Z$-statistic for small sample sizes as for larger ones, while DSMB's usually do not want to stop a trial very early for efficacy. Pocock himself now believes that his boundary is unsuitable [@pocock2006current]. Moreover, in contrast to the Pocock approach, the O'Brien Fleming $\alpha$-spending approach can be used to monitor the data after each new event-time. Hence the fair assessment is to compare continuously monitoring the safe logrank test to continuously monitoring using the O'Brien-Fleming $\alpha$-spending function.

GROW safe logrank (Gaussian) vs $\alpha$-spending {#sec:compRegionsSafeOBrienFleming}
-------------------------------------------------

In the previous section, and show that, in case of balanced allocation, the Gaussian approximation to the logrank statistic behaves very similar to the exact logrank test for certain designs. This is the case if we design a trial with 10000 participants to detect an effect of minimum clinical relevance of $\theta_1 = 0.7$ using a design that is *growth rate optimal in the worst case* (GROW, see Example \[ex:GROW\]). We will take this trial design as an example to compare the safe logrank test to $\alpha$-spending. Since the O'Brien-Fleming rejection region can be uniquely defined in terms of the logrank $Z$-statistic, we compare it to the Gaussian safe logrank test defined on the same logrank statistic.

#### Gaussian safe logrank $Z$-rejection region

For the Gaussian approximation we can specify the region of values for the logrank $Z$-statistic that rejects the null hypothesis when the $e$-value is larger than $1/\alpha$ as follows: rejection takes place for values of $Z$ such that $$\begin{gathered}
\nonumber
M''_{\mu_1,\mu_0} = \frac{\phi_{\mu_1\sqrt{n}}(Z)}{\phi_{\mu_0}(Z)} = \frac{\exp[-\frac{1}{2}(Z - \mu_1\sqrt{n})^2]}{\exp[-\frac{1}{2}(Z)^2]} \geq \frac{1}{\alpha} \quad \text{with} \quad \mu_0 = 0; \, \mu_1 = \log (\theta_1) \cdot \sqrt{\frac{m_1 \cdot m_0}{(m_1 + m_0)^2}},\\
\text{such that for $m_1 = m_0$, we reject if:} \qquad 
\begin{cases} 
Z \geq \frac{1}{2} \log (\theta_1 )\sqrt{n/4} - \frac{\log(\alpha)}{\log (\theta_1) \sqrt{n/4}}
& \text{if } \theta_1 > 1\\
Z \leq \frac{1}{2} \log (\theta_1 )\sqrt{n/4} - \frac{\log(\alpha)}{ \log(\theta_1)\sqrt{n/4}}
& \text{if } \theta_1 < 1.
\end{cases}\end{gathered}$$

#### O'Brien-Fleming $\alpha$-spending $Z$-rejection region

The O'Brien-Fleming $\alpha$-spending function gives a boundary that is constant in $B(n/n_{\max}) = Z/\sqrt{n/n_{\max}}$, where $n_{\max}$ is a maximum number of events that has to be set in advance. Result 5.1 in @proschan2006statistical gives that if we take $B(n/n_{\max})$ to be continuous Brownian motion: $$\begin{gathered}
\nonumber
    P_{\mu_0}\Big[B(n/n_{\max}) > c \quad \text{for some } n \leq n_{\max}\Big] = 2P_{\mu_0}\Big[B(n_{\max}/n_{\max}) > c \Big] = 2P_{\mu_0}\Big[Z > c\Big] = \alpha\\
    \text{such that} \qquad 
    \begin{cases} 
    Z \geq \frac{\Phi^{-1}(1-\alpha/2)}{\sqrt{n/n_{\max}}} & \text{in case of a right-sided test design}\\
    Z \leq \frac{\Phi^{-1}(\alpha/2)}{\sqrt{n/n_{\max}}} & \text{in case of a left-sided test design}.
    \end{cases}\end{gathered}$$

plots both the Gaussian safe logrank and the O'Brien-Fleming $\alpha$-spending rejection regions. The two regions of $Z$-statistic values share an important feature: they are more conservative to reject the null hypothesis at small sample sizes than at larger ones, requiring more extreme values for the $Z$-statistic. This sets them apart from the Pocock spending function that requires equally extreme values for the $Z$-statistic at small and large sample size. shows the boundary of the Pocock spending function for 10 interims. Note that the definition of the safe logrank test rejection region requires a very explicit value for the effect size $\theta_1 = \theta_{\min}$ of minimum clinical relevance, while that value is implicit in the definition of the $\alpha$-spending rejection region. To specify an maximum sample size $n_{\max}$ to achieve a certain power, you also assume an effect size of minimal interest. A fixed sample size analysis designed to detect a minimum hazard ratio of $0.7$ would need a number of events $n_{\max} = n = 195$ to achieve $80\%$ power if the true hazard ratio is $0.7$. A sequential analysis using $\alpha$-spending needs a bit more, a maximum sample size of $n_{\max} = 205$ events for an O'Brien-Fleming spending function and $n_{\max} = 245$ for a Pocock spending function, when we design for 10 looks. We investigate the number of events needed by the safe logrank test in the next section (note that the test allows for unlimited monitoring, so there is no real equivalent to $n_{\max}$).

The benefit of a sequential approach is that if the data looks better than hazard ratio $0.7$ we can detect that with a number of events that is smaller than this maximum sample size. illustrates that we benefit because the true hazard ratio could be more extreme than we designed for (e.g. $0.5$ instead of $0.7$; a larger risk reduction in the treatment group) and the data reflects that. We also benefit from a sequential analysis if the true hazard ratio is $0.7$ but by chance the values of our $Z$-statistics are more extreme than expected. The major difference between $\alpha$-spending approaches and the safe logrank test is that the safe test does not require to set a maximum sample size. It in fact allows to indefinitely increase the sample size without ever spending all $\alpha$. While an $\alpha$-spending approach designed to have $80\%$ power will miss out on rejecting the null hypothesis in $20\%$ of cases (the type-II error)—shown to stay green in —the safe logrank test can potentially reject all. In the sequences of $500$ events in , all but one sequence of $Z$-statistics could be rejected at a larger sample size by the safe logrank test. By increasing the sample size, the safe logrank test can have $100\%$ power if the true hazard ratio is at least as small as the hazard ratio set for minimum clinical relevance in the design of the test. Still, type-I errors are controlled. shows two null sequences of $Z$-statistics with a true hazard ratio of $1$ that are rejected by the O'Brien-Fleming $\alpha$-spending region, but not by the safe logrank test. Here, the safe logrank test is a bit more conservative, since it is saving $\alpha$ for the future.

Unbalanced allocation
---------------------

$\alpha$-spending methods are known to behave poorly in case of unbalanced allocation [@wu2017group]. In we showed that our Gaussian approximation to the logrank test is also not an $E$-variable in case of unbalanced allocation. Our exact safe logrank test, however, is an $E$-variable under any allocation since it is defined directly on the risk set process , that takes into account the allocation. This suggests that in case of unbalanced allocation, the exact logrank test should be preferred over $\alpha$-spending methods.

Comparing sample size {#sec:sampleSize}
=====================

Figure \[fig:simulation\_expected\_vs\_worst\] shows simulation results establishing three types of sample sizes. The leftmost panels ('Maximum') give the sample size required to design a trial. It expresses the maximum number of events $n_{\max}$ that needs to be observed under the alternative to achieve 80% power.

In case of the classical logrank test and $\alpha$-spending designs any further number of events beyond $n_{\max}$ cannot be analyzed. The rightmost panels ('Mean') show the sample size that captures the expected duration of the trial. It expresses the mean number of events, under the alternative, that will be observed before the trial can be stopped (here, for the safe logrank tests, we use the aggressive stopping rule that stops as soon as $M^{\langle n \rangle} \geq 1/\alpha = 20$ or $n = n_{\max}$). In case of $\alpha$-spending approaches and the safe logrank test this number of events is always smaller than the maximum needed in the design stage. Finally, the middle panel ('Conditional Mean') shows an even smaller number for those tests that have a flexible sample size: the expected stopping time <span>*given*</span> that the trial is stopped before the maximum $n_{\max}$ was reached — which will only happen if the null is rejected. For comparison purposes, all sample sizes are shown relative to (i.e. divided by) the fixed sample size needed by the classical logrank test to obtain $80\%$ power. Note that for small sample size (for small hazard ratios), both the classical logrank test and O'Brien-Fleming $\alpha$-spending are not recommended due to lack of type-I error control.

#### GROW safe logrank vs classical logrank and O'Brien-Fleming $\alpha$-spending

shows that the classical logrank test and O'Brien-Fleming $\alpha$-spending require a smaller maximum number of events to obtain 80% power. This is the benefit of specifying a strict $n_{\max}$ in the design and spending all $\alpha$ such that you cannot analyze any further events. The additional number of events required for the maximum of the safe logrank test is the price to pay for the unlimited horizon, or for combining with the data from future trials, after some tial maximum has been reached (optional continuation). The Gaussian and exact safe tests are shown to be similar for $\theta_1 \geq 0.5$, but the Gaussian performs poorly for smaller hazard ratios. In terms of the mean number of events before the trial can be stopped, both the safe logrank test and O'Brien-Fleming outperform the classical standard logrank test at almost all hazard ratios for which the classical logrank test is recommended. The exact safe logrank test always needs more events than O'Brien-Fleming $\alpha$-spending, but is the only approach that has exact type-I error control with small sample size at the smallest hazard ratios. The conditional mean number of events for the exact logrank test outperforms O'Brien-Fleming for designs with hazard ratio $\theta_1 = 0.1$ and small risk set $m_1/m_0$ and behaves very similar under all other scenarios. While the mean of the stopping times (conditional mean) are similar, the rejection regions in illustrate, however, that the stopping times themselves are not the same. The safe logrank test will sometimes reject the null hypothesis at an earlier number of events than O'Brien-Fleming $\alpha$-spending can, but also stop later than the $n_{\max}$ that was used to design the O'Brien-Fleming $\alpha$-spending boundary.

Variations and extensions {#sec:VariationsExtensions}
=========================

Prequential Plug-In and Bayes predictive distributions for the alternative {#sec:bayes}
--------------------------------------------------------------------------

Now suppose we do not have a very clear idea of which parameter $ \theta_{1} \in \Theta_{1} $ to pick. One way to handle this case is to use rather than and , replacing the fixed conditional probabilities $q_{\theta_1}({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}},{\ensuremath{Y_{0} \langle i \rangle}})$ by a conditional probability mass function $r_i({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}})$ that depends on the past and enables implicitly learning $\theta_1$ from the data. For example, we may take $$\label{eq:spannend}
r_i({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}})
:= q_{\hat\theta_i }({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}}, {\ensuremath{{Y}_{0} \langle i \rangle}})$$ with $q$ as in and $\hat\theta_i := \hat\theta ({\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}} )$ the maximum likelihood estimate based on all past data, smoothed by adding two 'virtual' data points to the data, one for both groups: $$\begin{gathered}
    \hat\theta({\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}} )
:= \\ \arg \max_{\theta \in (0,\infty)}
\left( \prod_{k=1}^{i} q_{\theta}({\ensuremath{O_{1} \langle k \rangle}} \mid {\ensuremath{Y_{1} \langle k \rangle}}, {\ensuremath{Y_{0} \langle k \rangle}} ) \right) \cdot q_{\theta}(1 \mid {\ensuremath{Y_{1} \langle 1 \rangle}}+1,{\ensuremath{Y_{0} \langle 1 \rangle}}) \cdot  q_{\theta}( 0 \mid {\ensuremath{Y_{1} \langle 1 \rangle}}, {\ensuremath{Y_{0} \langle 1 \rangle}}+1). \nonumber\end{gathered}$$ Suppose the data are actually sampled from the process defined by $q_{\theta}$, for some arbitrary but fixed $\theta$. For sufficiently large initial risk sets ($m_0$, $m_1$ are not too small), by the law of large numbers, $\hat\theta_i$ will converge with high probability to $\theta$, and $q_{\hat\theta_i}$ will behave more and more like the real $q_\theta$ from which data are sampled. Thus, the process instantiated with (\[eq:spannend\]) will behave more and more similarly to the 'correct' likelihood ratio . The process $r_1, r_2, \ldots$ is a typical instance of Dawid's ([-@Dawid84]) <span>*prequential plug-in*</span> likelihood, that is often based on suitable smoothed likelihood-based estimators [@GrunwaldR20]. Instead of $r_i$ based on a plug-in estimate of $\theta$ based on ${\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}}$, one may just as well use a Bayes predictive distribution based on the same data and some prior $W_1$ on $\theta$. That is, we set $$\label{eq:rwithprior}
    r_i({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}})
:= q_{W_i}({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}}, {\ensuremath{{Y}_{0} \langle i \rangle}})$$ where $W_i := W \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}}$ is the Bayes posterior on $\theta$ based on prior $W$ and data ${\ensuremath{\vec{Y}_{} \langle 1 \rangle}},\ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}}$, and $q_W({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}}, {\ensuremath{{Y}_{0} \langle i \rangle}}) := \int q_{\theta} 
({\ensuremath{O_{1} \langle i \rangle}} \mid {\ensuremath{Y_{1} \langle i \rangle}}, {\ensuremath{{Y}_{0} \langle i \rangle}})
d W(\theta)$ is the Bayes predictive. By multiplying out the conditional probability mass functions $r_i$, we then get that $M^{\langle n \rangle}_{r,\theta_0} = \prod_{i=1}^n {\ensuremath{M_{r,\theta_0} \langle i \rangle}}$ is a Bayes factor between the Bayes marginal based on $W$ and $\theta_0$ (GHK explain how such a correspondence between Bayes factors and test martingales holds more generally for simple null hypotheses). We do not know of a prior for which this Bayes factor or the constituent products have an analytic expression, but it can certainly be implemented using e.g. Gibbs sampling.

As explained underneath (\[eq:likelihoodRatioc\]), the use of the $r_{i}$ instead of $q_{\theta_1}$ does not compromise on safety: type-I errors and confidence sequences based on $M_{r,\theta_0}$ remain valid, whether the $r_{i}$ are plug-in estimators or Bayes predictive distributions, no matter what prior $W$ was chosen. Thus, our set-up is actually more intimately related to the concept of <span>*luckiness*</span> in the machine learning theory literature [@GrunwaldM19] than to 'pure' Bayesian statistics. The type-I error guarantee always holds, also when the prior is 'misspecified', putting most of its mass in a region of the parameter space far from the actual $\theta$ from which the data were sampled. Given a minimal clinically relevant effect size $ \theta_{\min} $, the worst case logarithmic growth rate of $ M_{r,\theta_{0}} $ will in general be less than that of the GROW $ M_{\theta_{\min}, \theta_{0}} $. Nevertheless, $ M_{r, \theta_{0}} $ can come quite close to the optimal for a whole range of potentially data-generating $ \theta $ and may thus sometimes be preferable over choosing $ M_{\theta_{\min},\theta_{0}} $. More precisely, the use of a prior allows us to exploit favourable situations in which $\theta$ is even smaller (more extreme) than $ \theta_{\min} $. In such situations, the GROW $ M_{\theta_{\min}, \theta_{0}} $ is effectively misspecified. By using $r_i$ that learn from the data, we may actually get an $E$-variable that grows faster than the GROW $ M_{\theta_{\min}, \theta_{0}} $ which is fully committed to detecting the worst case $ \theta_{\min} $.

In we illustrate such a situation where we start with 1000 participants in both groups. We generated data using different hazard ratios, and used a 'misspecified' $M_{\theta_1, 1}$ that always used $\theta_1 = 0.8$. Note that while this is still the GROW (minimax optimal) martingale test under $\cH_1 = \{P_{\theta}: \theta_1 \leq 0.8\}$, if we knew the true $\theta$, we could use the faster-growing test martingale $M_{\theta,1}$. We will call the test based on this latter martingale the <span>*oracle*</span> exact safe logrank test, since it is based on inaccessible (oracle) knowledge. We estimated the number of events that allows for 80% power for the tests based on $M_{0.8, 1}$ and the oracle $M_{\theta,1}$ and the prequential plug-in $M_{r,1}$ with $r$ as in (\[eq:spannend\]). In all cases we used the aggressive stopping rule that stops as soon as $M_{\cdot,1} > 1/\alpha = 20$. We see that, as the true $\theta$ gets smaller than $0.8$, we need less events using the GROW test $M_{0.8, 1}$ (the data are favorable to us), but using the oracle exact safe logrank test we get a considerable additional reduction. The prequential plug-in $M_{r,1}$ 'tracks' the oracle $M_{\theta,1}$ by learning the true $\theta$ from the data: for $\theta$ near $0.8$, it behaves worse (more data are needed) than $M_{0.8,1}$ (which knows the right $\theta$ from the start), but for $\theta < 0.6$ it starts to behave better. For comparison we also added the methods of . Notably, the O'Brien-Fleming procedure, even though unsuitable for optional continuation, needs even more events than the misspecified safe logrank test $M_{0.8,1}$ as soon as $\theta$ goes below $0.8$ (the simulations were performed using exactly the same algorithms as for so the $y$-axis at $\theta = 0.8$ coincides with that of , but now with absolute rather than relative numbers); details are described in Online ).

Anytime-valid confidence sequences {#sec:confidence}
----------------------------------

Standard tests give rise to confidence intervals by varying the null and 'inverting' the corresponding tests. In analogous fashion, test martingales can be used to derive <span>*anytime-valid (AV) confidence sequences*</span> [@darling1967confidence; @lai1976confidence; @howard_exponential_2018; @howard_uniform_2018]. In our setting, a $(1-\alpha)$-AV confidence sequence is a sequence of confidence intervals $\{\text{\sc ci}_i\} _{i \in \naturals}$, one for each consecutive event, such that $$\label{eq:ci}
P_{\theta}\left(
\text{there is an $i \in \naturals$ with\ } \theta \not \in \text{\sc ci}_{i} \right) \leq \alpha.$$ A standard way to design $(1-\alpha)$- AV confidence sequences, translated to our logrank setting, is to use a prequential plug-in or Bayesian-based $r$ as described in the previous subsection. After observing $n$ events, one reports $\text{\sc ci}_{n,\alpha} = [\theta_{n,L}, \theta_{n,U}]$ where $\theta_{i,L}$ is the largest $\theta_0$ such that for all $\theta' \leq \theta_0$, $M_{r,\theta'}^{\langle n \rangle} \geq 1/\alpha$; similarly $\theta_{n,U}$ is the smallest $\theta_0$ such that for all $\theta' \geq \theta_0$, $M_{r,\theta'}^{ \langle n \rangle} \geq 1/\alpha$. That is, we check where we vary $\theta_0$ and we report the smallest interval such that $M_{r,\theta_0} > 1/\alpha$ outside this interval. Ville's inequality immediately shows that this gives an AV confidence sequence for arbitrary instantiations of $r$.

Covariates: the full Cox Proportional Hazards $E$-Variable {#sec:cox}
----------------------------------------------------------

We extend the process of (for now without ties) as follows. We identify each of the $m = m_0 + m_1$ participants by an index $j \in \{1, \ldots, m\}$. The vector $\vec{g} \in \{0,1\}^m$ encodes group assignment, $\vec{g}_j$ denoting the group assignment of participant $j$, so that the number of $0$-components of $\vec{g}$ is $m_0$. We assume that the group assignment is fixed in advance.

We fix a set of $d$ covariates and let ${\ensuremath{\mathbf Z}}\langle i \rangle  = (\vec{z}_1 \ldots \vec{z}_m)\langle i \rangle$ be the matrix consisting of the covariate vectors for each participant at the time of the $i$^th^ event: $\vec{z}_j\langle i \rangle = ({z}_{j,1}\langle i \rangle,\ldots, {z}_{j,d}\langle i \rangle)$. We let random variable ${\ensuremath{J_{} \langle i \rangle}}$ denote the index of the patient to which the $i$^th^ event happens, and consider the extended process ${\ensuremath{J_{} \langle 1 \rangle}},{\ensuremath{J_{} \langle 2 \rangle}}, \ldots$ where the information that is available at time $i$ is $\vec{g}$, ${\ensuremath{J_{} \langle 1 \rangle}},\ldots, {\ensuremath{J_{} \langle i \rangle}}$, ${\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle 1 \rangle}}, \ldots, {\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}}$. In this section we re-define ${\ensuremath{\vec{Y}_{} \langle i \rangle}} = (
{\ensuremath{Y_{1} \langle i \rangle}},\ldots, {\ensuremath{Y_{m} \langle i \rangle}}
) \in \{0,1\}^m$ to be a vector of $m$ components, the $j$ component indicating whether the $j$^th^ participant is still without an event just before the $i$^th^ event. Accordingly we set ${\ensuremath{\vec{Y}_{} \langle 1 \rangle}} = (1, \ldots, 1) $ and we set ${\ensuremath{\vec{Y}_{j} \langle i+1 \rangle}} = {\ensuremath{\vec{Y}_{j} \langle i \rangle}}$ for $j \neq {\ensuremath{J_{} \langle i \rangle}},{\ensuremath{\vec{Y}_{j} \langle i+1 \rangle}} = 0$ for $j = {\ensuremath{J_{} \langle i \rangle}}$. The conditional distribution underlying the process is now denoted $P_{\beta,\theta}$ with $\theta > 0$ and $\beta \in \reals^d$, defined as follows: $P_{\beta,\theta}$ is given by, for $j \in \{1, \ldots, m\} $ and $\vec{y} \in \{0,1\}^m$ with $\vec{y}_j = 1$: $$\begin{aligned}
\label{eq:coxmodel}
& P_{\beta,\theta}({\ensuremath{J_{} \langle i \rangle}} = j \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle 1 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle i \rangle}},{\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}}) := 
P_{\beta,\theta}({\ensuremath{J_{} \langle i \rangle}} = j \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}},{\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}}) 
\nonumber \\
& P_{\beta,\theta}({\ensuremath{J_{} \langle i \rangle}} = j \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}} = \vec{y},{\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}} = {\ensuremath{\mathbf z}}) :=
q_{\beta,\theta}(j \mid \vec{y},{\ensuremath{\mathbf z}}) :=     \frac{\exp(\beta^T \vec{z}_j + \theta' g_j)}{\sum_{j': \vec{y}_{j'} = 1} \exp(\beta^T \vec{z}_{j'} + \theta'g_{j'})},\end{aligned}$$ with $\theta'= \log \theta$ and ${\ensuremath{\mathbf z}} = (\vec{z}_1, \ldots, \vec{z}_m)$. This is consistent with Cox' ([-@Cox72]) proportional hazards regression model: the probability that the $j$^th^ participant has an event, assuming he/she is still at risk, is proportional to the exponentiated weighted covariates, with group membership being one of the covariates. In case $\beta =0$, this is easily seen to coincide with the definition of $P_{\theta}$ via (\[eq:qSingle\]).

#### $E$-Variables and Martingales

Let $W$ be a prior distribution on $\beta \in \reals^d$ for some $d > 0$. ($W$ may be degenerate, i.e. put mass one in a specific parameter vector $\beta_1$). We let $$\begin{aligned}
  q_{W,\theta}( j \mid \vec{y}, {\ensuremath{\mathbf z}}) =
    \int q_{\beta,\theta}(j \mid \vec{y}, {\ensuremath{\mathbf z}}) d W(\beta).\end{aligned}$$ Consider a measure $\rho$ on $\reals^d$ (e.g. Lebesgue or some counting measure) and we let $\cW$ be the set of all distributions on $\reals^d$ which have a density relative to $\rho$, and $\cW^{\circ} \subset \cW$ be any convex subset of $\cW$ (we may take $\cW^{\circ} = \cW$, for example). We define $\tilde{q}_{\leftarrow W,\theta_0}(\cdot \mid \vec{y}, {\ensuremath{\mathbf z}})$ to be the <span>*reverse information projection*</span> [@li_estimation_1999] (RIPr) of $q_{W,\theta}( j \mid \vec{y},{\ensuremath{\mathbf z}})$ on $\{q_{W,\theta_0}: W \in \cW^{\circ} \}$ such that $$\begin{aligned}
D( q_{W,\theta_1}( \cdot \mid \vec{y},{\ensuremath{\mathbf z}})  \|
\tilde{q}_{\leftarrow W,\theta_0} ( \cdot \mid \vec{y} ,{\ensuremath{\mathbf z}}) ) =
    \inf_{W' \in \cW^{\circ}} \ D( q_{W,\theta_1}( \cdot \mid \vec{y}, {\ensuremath{\mathbf z}})  \|
q_{W',\theta_0} ( \cdot \mid \vec{y},{\ensuremath{\mathbf z}}) ).\end{aligned}$$ We know from @li_estimation_1999 and GHK that $\tilde{q}_{\leftarrow W,0} ( \cdot \mid \vec{y})$ exists. As explained by GHK in the context of $E$-variables for $2 \times 2$ contingency tables, the fact that the random variables ${\ensuremath{Y_{} \langle i \rangle}}$ constituting our random process have finite range implies that, for each $W$, the infimum is in fact achieved by some distribution $W'$ with finite support on $\reals^d$. For given $\theta_0,\theta_1 > 0$, let $$\label{eq:likelihoodRatiob}
M_{W,\theta_1,\theta_0} \langle i\rangle = \frac{q_{W,\theta_1}({\ensuremath{J_{} \langle i \rangle}} \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}},{\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}}) }{q_{\leftarrow W,\theta_0}({\ensuremath{J_{} \langle i \rangle}} \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}}, {\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}})}$$ be our analogue of $M_{\theta_1,\theta_0} \langle i \rangle$ as in (\[eq:likelihoodRatio\]).

\[thm:infproj\]<span>**\[Corollary of Theorem 1 from GHK19\]**</span> For every prior $W$ on $\reals^d$, for all $\tilde\beta \in \reals^d$, $${\bf E}_{P_{\tilde\beta,\theta_0}}[M_{W,\theta_1,\theta_0} \langle i\rangle \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}}, {\ensuremath{{\bf Z}_{} \langle 1 \rangle}}, \ldots, 
{\ensuremath{\vec{Y}_{} \langle i \rangle}},{\ensuremath{{\bf Z}_{} \langle i \rangle}} ]
= \sum_{j \in [m]} q_{\beta,\theta_0}(j \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}}, 
{\ensuremath{{\bf Z}_{} \langle i \rangle}}) \cdot 
\frac{q_{W,\theta_1}(j \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}},{\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}}) }{q_{\leftarrow W,\theta_0}(j \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}}, {\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}})} \leq 1$$ so that $M_{W,\theta_1,\theta_0} \langle i\rangle$ is an $E$-variable conditional on ${\ensuremath{\vec{Y}_{} \langle 1 \rangle}}, {\ensuremath{{\bf Z}_{} \langle 1 \rangle}}, \ldots, 
{\ensuremath{\vec{Y}_{} \langle i \rangle}},{\ensuremath{{\bf Z}_{} \langle i \rangle}}$.

Note that the result does not require the prior $W$ to be well-specified in any-way: under any $(\tilde\beta,\theta_0)$ in the null distribution, even if $\tilde\beta$ is completely disconnected to $W$, ${\ensuremath{M_{W,\theta_1,\theta_0} \langle i \rangle}}$ is an $E$-variable conditional on past data.

In particular, since the result holds for arbitrary priors, it holds, at the $n$^th^ event time, for the Bayesian posterior $
W_{n+1} = W_1 \mid 
{\ensuremath{\vec{Y}_{} \langle 1 \rangle}}, {\ensuremath{{\bf Z}_{} \langle 1 \rangle}}, \ldots, 
{\ensuremath{\vec{Y}_{} \langle n \rangle}},{\ensuremath{{\bf Z}_{} \langle n \rangle}}$, based on arbitrary prior $W_1$ with density $w_1$, i.e. the density of $W_{n+1}$ is given by $$w_{n+1}(\beta) \propto \prod_{i=1}^n 
q_{\beta,\theta} ({\ensuremath{J_{} \langle i \rangle}} \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}},{\ensuremath{{\ensuremath{\mathbf Z}}_{} \langle i \rangle}})
w_1(\beta).$$ Using Definition , we can therefore, for each prior $W_1$, construct a test martingale $M^{\langle n \rangle}
:= \prod_{i=1}^n M_{W_{i},\theta_1,\theta_0}$ that 'learns' $\beta$ from the data, analogously to (\[eq:rwithprior\]), and computes a new RIPr at each event time $i$.

#### How to find the RIPr

While in general, it is not clear how to calculate the RIPr $q_{\leftarrow W,\theta_0}$, @li_estimation_1999 [@LiB00] have designed an efficient algorithm for approximating it, which is feasible as long as we restrict $\cW^{\circ}$ to be the set of all priors $W$ for which, for all $j \in [n]$, $Q_{W,\theta_0}({\ensuremath{J_{} \langle i \rangle}} = j \mid {\ensuremath{\vec{Y}_{j} \langle i-1 \rangle}} = 1) \geq \delta$, for $\ell = 1,\ldots,k$, for some $\delta > 0$. The algorithm achieves an approximation error of $O( (\log (1/\delta))/M)$ if run for $M$ steps, where each step takes time linear in $d$. Since the factor is logarithmic in $1/\delta$, we can take a very small value of $\delta$ and then the requirement does not seem overly restrictive. Exploring whether the Li-Barron algorithm really allows us to compute the RIPr for the Cox model, and hence $M_{W,\theta_1,\theta_0}$ in practice, is a major goal for future work.

#### Ties

While in the case without covariates, our $E$-variables allowing for ties () correspond to a likelihood ratio of noncentral hypergeometrics, the situation is not so simple if there are covariates—although deriving the appropriate extension of the noncentral hypergeometric partial likelihood is possible, one ends up with a hard-to-calculate formula [@Peto72]. Various approximations have been proposed in the literature [@Cox72; @Efron74]. In case these preserve the $E$-variable and martingale properties, they would retain type-I error probabilities under optional stopping and we could use them without problems. We do not know whether this is the case however; for the time being, we recommend handling ties by putting the events in a worst-case order, leading to the smallest values of the $E$-variable of interest, as this is bound to preserve the type-I error guarantees.

Discussion, Conclusion and Future Work {#sec:conclusion}
======================================

We introduced the safe logrank test, a version of the logrank test that can retain type-I error guarantees under optional stopping and continuation. Extensive simulations revealed that, if we do engage in optional stopping, it is competitive with the classical logrank test (which neither allows in-trial optional stopping nor optional continuation) and $\alpha$-spending (which allows forms of optional stopping but not optional continuation). We provided an approximate test for applications in which only summary statistics are available and also showed how the safe logrank test can be used in combination with (informative) prior and prequential learning approaches, when no effect size of minimal clinical relevance can be specified. Two of our extensions invite further research: We introduced anytime-valid confidence sequences for the hazard ratio, and will study these more in future work into their performance in comparison to other approaches. We also introduced an extension to Cox' proportional hazards regression, which promises type-I error guarantees even if the alternative model is equipped with arbitrary priors. In future work, we plan to implement this extension—which requires the use of sophisticated methods for estimating mixture models. The GROW safe logrank tests (exact and Gaussian) are already available in our SafeStats R package [@LyT20]. We end with two final points of discussion: <span>*staggered entries*</span> and <span>*doomed trials*</span>.

#### Staggered entry

Earlier approaches to sequential time-to-event analysis were also studied under scenarios of staggered entry, where each patient has its own event time (e.g. time to death since surgery), but patients do not enter the follow-up simultaneously (such that the risk set of e.g. a two-day-after-surgery event changes when new participants enter and survive two days). @sellke_sequential_1983 and @slud_sequential_1984 show that, in general, martingale properties cannot be preserved under such staggered entry, but that asymptotic results are hopeful [@sellke_sequential_1983] as long as certain scenarios are excluded [@slud_sequential_1984]. When all participants' risk is on the same (calendar) time scale (e.g. infection risk in a pandemic; staggered entry now amounts to left-truncation, which we can deal with), or new patients enter in large groups (allowing us to stratify), staggered entry poses no problem for our methods. But research is still ongoing into those scenarios in which our inference is fully safe for patient time under staggered entry, and those that need extra care.

#### Your trial is not doomed

In their summary of conditional power approaches in sequential analysis @proschan2006statistical write that low conditional power makes a trial futile. Continuing a trial in such case could only be worth the effort to rule out an effect of clinical relevance, when the effect can be estimated with enough precision. However, if "both conditional and revised unconditional power are low, the trial is doomed because a null result is both likely and uninformative" [@proschan2006statistical p. 63]. While this is the case for all existing sequential approaches that set a maximum sample size, this is not the case for safe tests. Any trial can be extended and possibly achieve 100% power or in an anytime-valid confidence sequence show that the effect is too small to be of interest. This is especially useful for time-to-event data when sample size can increase by extending the follow-up of the trial, without recruiting more participants. Moreover, new participants can always be enrolled either within the same trial or by spurring new trials that can be combined indefinitely in a cumulative meta-analysis.

Acknowledgement {#acknowledgement .unnumbered}
===============

This work is part of the research program with project number 617.001.651, which is financed by the Dutch Research Council (NWO). We thank Henri van Werkhoven, Richard Gill, Wouter Koolen, Aaditya Ramdas and Rosanne Turner for useful conversations.

Appendix\

connects the simple risk set processes in this paper to a continuous time survival process. In the online supplementary material, gives an additional argument for the GROW criterion, \[app:score\] gives a more detailed derivation of the logrank test as a score test for single events and ties, \[app:theory\] gives details of the proof for Theorem \[thm:martingalemoreevents\] and \[app:moreSampleSize\] gives a step-by-step account of the sample size comparison simulations. In the online supplementary material we also provide the R code that produces all simulations and figures in this paper.

Towards Continuous Time {#app:continuous}
=======================

In and , the expression (\[eq:coxmodel\]) and its simplified form (\[eq:likelihoodSingle\]) defining the safe logrank test appeared as a factor in a standard likelihood, defined relative to a basic stochastic process in which the time between events was not formalized. @Cox75 simply claimed it to be also a partial likelihood for the underlying continuous-time process with proportional hazards determined by $(\beta,\theta')$. @slud1992partial proved that it can indeed be seen as such, in the same general setting with time-varying continuous covariates that we consider here (see also @andersen_statistical_1993 for closely related results). This already shows that the test martingales of (no covariates, no ties) and (covariates, no ties) also remain test martingales in the continuous time setting. The only part that is not covered by such existing results is the case of ties, : if we plug the conditional distributions (\[eq:qTies\]) that allow for ties into the test martingale , do we still get a test martingale in continuous time? Since we have not been able to find a complete proof in the literature of this result or a result that would directly imply it, we derive a simple version of such a result below. The result continues to hold under noninformative right censoring, but for simplicity, we do not explicitly deal with it in our analysis. We first extend our formalism so as to be able to represent the required notions.

#### Three Processes: Event Time, Unit-Time, $\epsilon$-discretized Continuous Time

As in , we identify each of the $m= m_0+m_1$ participants by an index $j \in  [m] := \{1, \ldots, m\}$, with $\vec{g} \in \{0,1\}^m$, and $\vec{g}_j \in \{0,1\}$ denoting the group assignment of participant $j$. However, for simplicity we shall not make use of any covariates. For $t \in \reals^+_0$ (continuous time), we let $T_j = t$ denote that the $j$^th^ participant had an event at time $j$, and we let $Y_{j}(t):= {\ensuremath{{\mathbf 1}_{T_{j} \geq t}}}$ be the 'at risk' process for participant $j$. For $g \in \{0,1\}$, we let $Y_{g}(t) = \sum_{j: \vec{g}_j = g} Y_{j}(t)$ be the number of participants at risk in the group $g$ at time $t$. We set ${\ensuremath{O_{g}[t', t]}} = Y_g(t')- Y_g(t)$ to be the number of events that happened in group $g$ inbetween time $t'$ and time $t$ and $O[t',t] = O_0[t',t] + O_1[t',t]$. Analogous to (i.e. different from ), but now at the continuous-time level, $\vec{Y}(t) = (Y_{1}(t), \ldots, Y_{m}(t))$ is the $m$-dimensional indicator vector that indicates for each participant $j$ whether the participant is still at risk at time $t$.

We assume that a time increment of $1$ represents a natural 'unit time' for example an hour, a day, or a week—these are the times at which we check if $0, 1$ or more additional events took place.

We will look at the data at three levels: "almost continuous" time ($\epsilon$-discretized in the limit for $\epsilon \downarrow 0$: $T= 0, \epsilon, 2 \epsilon, \ldots$), unit times ($T=0,1,2,\ldots$) and event times ${\ensuremath{t_{} \langle 1 \rangle}}, {\ensuremath{t_{} \langle 2 \rangle}}, \ldots$. These are the times, measured at the unit scale (each ${\ensuremath{t_{} \langle i \rangle}}$ is required to be a natural number), at which we actually measured one or more events. We will show two things within this setting: first, Theorem \[thm:martingalemoreevents\], if the data really come from a random process satisfying proportional hazards, then in the limit for small $\epsilon$, the conditional distribution as given by (\[eq:qTies\]) (and its special case, if there can be no ties, (\[eq:likelihoodSingle\])), is correct. Second, this implies that (\[eq:likelihoodSingle\]) gives a partial likelihood in the 'almost-continuous time' (limit for $\epsilon \downarrow 0$) setting, and this allows us to define a test martingale, (\[eq:martingalemoreevents\]) below, at the unit time level. This will imply that our test martingale $M^{\langle i \rangle}_{\theta_1,\theta_0}$ which was defined at the coarsest, 'event-time process level', is also a test martingale at the almost-continuous time level.

#### The $\epsilon$-level Process

We start with the process at the finest, $\epsilon$-level, and only re-connect this to the event-time process after (\[eq:martingalemoreevents\]) below. We denote by $P_{[\epsilon],.}$ a distribution for the $\epsilon$-increment discrete-time random process $\{\vec{Y}(k \cdot \epsilon)\}_{k=0,1,2, \ldots}$. All random elements occurring inside the argument of $P_{[\epsilon],\theta}$ are defined relative to this $\epsilon$-increment process. Thus, while we write $P_{[\epsilon],\theta}({\ensuremath{O_{g} \langle i \rangle}} = 1 \mid {\ensuremath{\vec{Y}_{} \langle i \rangle}} = \vec{y})$ below it would be more precise to write $P_{[\epsilon],\theta}({\ensuremath{O_{g,[\epsilon]} \langle i \rangle}} = 1 \mid {\ensuremath{\vec{Y}_{[\epsilon]} \langle i \rangle}} = \vec{y})$ since the definition of random variables ${\ensuremath{O_{g,[\epsilon]} \langle i \rangle}}$ and ${\ensuremath{\vec{Y}_{[\epsilon]} \langle i \rangle}}$ depends on $\epsilon$. We omit this dependency in the notation.

We will generally assume that the hazard functions are Lipschitz continuous and bounded away from $0$ and $\infty$ in $t$, i.e. there exists some (unknown) constants $0 < c < C< \infty$ such that for all $t \geq 0, g \in \{0,1\}$, $c < \lambda_{g}(t) < C$.

We say that a distribution $P_{[\epsilon]}$ for random process $\{\vec{Y}(k \cdot \epsilon)\}_{k=0,1,2, \ldots}$ as above is <span>*compatible*</span> with hazard functions $\lambda_0,\lambda_1$ as above, if for all $j \in [m]$, we have $P_{[\epsilon]}(Y_{j}(0)=1)=1$ and, under $P_{[\epsilon]}$ the event times ${\ensuremath{T}}_1,{\ensuremath{T}}_2, \ldots, {\ensuremath{T}}_n$ are independent random variables with support $\{0,\epsilon, 2\epsilon,\ldots \}$ and marginal distribution satisfying, for $k\in \naturals$, $$\label{eq:compatiblehazard} \min_{(k-1) \epsilon \leq t < k \epsilon}
\lambda_{\vec{g}_j}(t) \epsilon \leq P_{[\epsilon]}( {\ensuremath{T}}_{j} < k \epsilon \mid (k-1) \epsilon \leq {\ensuremath{T}}_{j} ) \leq
\max_{(k-1) \epsilon \leq t < k \epsilon}
\lambda_{\vec{g}_j}(t) \epsilon.$$ We say that $P_{[\epsilon]}$ is <span>*compatible*</span> with <span>*proportional hazards ratio $\theta$*</span>, if it is compatible with two hazard functions $\lambda_{0}, \lambda_{1}$, again assumed Lipschitz continuous and bounded away from $0$ and infinity as above, with, for all $t$, $\lambda_{0}(t)/\lambda_{1}(t) = \theta$.

At time $t=1, 2, \ldots$ we observe that, between time $t-1$ and $t$, there have been $O_g[t-1,t]$ events in group $g$ where $O_g[t-1,t]$ can be $0$, $1$, but also more than $1$. In the latter case, we usually do not observe the order in which the specific events took place.

For simplicity we further assume that $\epsilon= 1/b$, where $b$ is some large integer. This ensures that our unit time is an integer multiple of an $\epsilon$-time interval. Theorem \[thm:martingalemoreevents\] below shows how the hypergeometric probability mass function $q$ of naturally arises in this context: let $
q_{\theta}(v \mid \left(y_0,y_1\right),s) 
$ be as given by (\[eq:qTies\]).

Now consider a tuple $(s,\vec{y},y_0,y_1)$, where $s$ denotes the number of events that happens within some time interval, $\vec{y}$ indicates the $m$-dimensional vector where $\vec{y}_j$ indicates whether the $j$^th^ participant is still at risk at the beginning of the time interval, and $y_g$ denotes the number of participants in group $j$ at the beginning of the interval. We call such a tuple <span>*internally compatible*</span> if it can actually arise in the process of unfolding events; that is, if $y_0$ is the number of 0s in $\vec{y} \in \{0,1\}^m$, $y_1$ is the number of 1s in $\vec{y}$, and if $s \leq y_0 + y_1= m$. The following result is proved in Online :

\[thm:martingalemoreevents\] Let, for each $b \in \naturals$, $P_{[1/b],\theta}$ be compatible with proportional hazards as defined above. We have for all $b \in \naturals$, $\epsilon = 1/b$, all $K \in \naturals$, all internally compatible tuples $(s, \vec{y},y_0,y_1)$ with $s \geq 1$, all $v \in \naturals_0$ with $0 \leq v \leq s$, that $$\begin{gathered}
P_{[\epsilon],\theta}({\ensuremath{O_{1}[K-1, K]}}  \mid
{\ensuremath{O_{}[K-1, K]}}, \vec{Y}(K-1)) =\\ 
P_{[\epsilon],\theta}({\ensuremath{O_{1}[K-1, K]}}  \mid
{\ensuremath{O_{}[K-1, K]}},
\vec{Y}(0), \vec{Y}(\epsilon), \vec{Y}(2 \epsilon), \ldots, \vec{Y}(K-1)) \end{gathered}$$ and $$\begin{gathered}
\lim_{b \rightarrow \infty} P_{[1/b],\theta}({\ensuremath{O_{1}[K-1, K]}} = v \mid
{\ensuremath{O_{}[K-1, K]}} = s,
\vec{Y}(K-1) = \vec{y}
) = q_{\theta}(v \mid (y_0,y_1),s).
 \end{gathered}$$ The same statements hold if we condition on $(Y_0,Y_1)$ instead of the full vector $\vec{Y}$, in particular we replace the condition $\vec{Y}(K-1) = \vec{y}$ by $Y_0(K-1) = y_0;Y_1(K-1)  = y_1$.

We have thus shown that the distribution (\[eq:qTies\]) and its simplified form (\[eq:likelihoodSingle\]) in our original process that only considered event times arises as a conditional distribution in almost-continuous time.

#### Test Martingale at the Unit Time Level

We next define, for given $\theta_0, \theta_1 >0$ and integer times $k=1,2, \ldots$: $$\begin{aligned}
\label{eq:martingalemoreevents}
U_{\theta_1,\theta_0}^{(K)} := \prod_{k=1}^{K} U(k)\ ,\ U_{\theta_1,\theta_0}(k) := \begin{cases} 1 & \text{\ if \ } O[k-1,k] = 0  \\
\frac{q_{\theta_1}(O_1[k-1,k] \mid Y_0(k-1),Y_1(k-1),O[k-1,k])}
{q_{\theta_0}(O_1[k-1,k] \mid Y_0(k-1),Y_1(k-1),O[k-1,k])}
& \text{\ if \ } O[k-1,k] > 0.
\end{cases}\end{aligned}$$ Theorem \[thm:martingalemoreevents\] implies that, with $\epsilon = 1/b$, $$\lim_{b\rightarrow \infty} {\bf E}_{P_{[1/b],\theta_0}}[U_{\theta_1,\theta_0}(K) \mid 
{\ensuremath{O_{}[K-1, K]}},
\vec{Y}(0), \vec{Y}(\epsilon), \vec{Y}(2 \epsilon), \ldots, \vec{Y}(K-1)] = 1.$$ Thus, in the limit towards continuous time, $U_{\theta_1\theta_0}(k)$ becomes an $E$-variable conditional on $\vec{Y}(0)$, $\vec{Y}(1)$, $\ldots$, $\vec{Y}(k-1)$, and $U^{(k)}_{\theta_1,\theta_0}$ becomes a test martingale. We may thus think of $U^{(k)}_{\theta_1,\theta_0}$ as being 'essentially' a test martingale in continuous time, so that, if the true process satisfies proportional hazards with rate $\theta_0$, the type-I error of testing based on $U^{(k)}_{\theta_1,\theta_0}$ is preserved under optional stopping.

#### Compatibility between Unit and Event Time Martingales

We next show that the test martingale $M_{\theta_1,\theta_0}^{\langle I \rangle}$ used in , with the tie-allowing $q$ of plugged in, is compatible with $U_{\theta_1,\theta_0}^{(K)}$ in the sense that both processes coincide at all event times. Since $U_{\theta_1,\theta_0}^{(K)}$ is compatible with assuming ('almost, limit for $\epsilon \downarrow 0$) continuous time, it will then follow that $M^{\langle I \rangle}_{\theta_1,\theta_0}$ is also compatible with continuous time, which is what we wanted to show.

As said, we assume that ${\ensuremath{t_{} \langle 1 \rangle}} < {\ensuremath{t_{} \langle 2 \rangle}}< \ldots$ are all integers (multiples of unit time), and, like in , that for each $i$, at event time ${\ensuremath{t_{} \langle i \rangle}}$, any number of events $\geq 1$ may happen, i.e. for $i \in \naturals$, $${\ensuremath{O_{g} \langle i \rangle}} := 
O_g[t\langle i \rangle, t \langle i -1 \rangle] \text{\ for $g \in \{0,1\}$ and\ }
{\ensuremath{O_{} \langle i \rangle}} := 
{\ensuremath{O_{0} \langle i \rangle}}+{\ensuremath{O_{1} \langle i \rangle}}$$ is at least, but not necessarily equal to, $1$. Here, for convenience, we defined $t \langle 0 \rangle =0$. The definitions of $U_{\theta_1,\theta_0}^{(K)}$ and $M_{\theta_1,\theta_0}^{\langle I \rangle}$ immediately give that for all $I \in \naturals$ we have $U_{\theta_1,\theta_0}^{(t \langle I \rangle)}= M_{\theta_1,\theta_0}^{\langle I  \rangle}$, where for convenience we defined $M_{\theta_1,\theta_0}^{\langle 0 \rangle} = 1$. Thus $U_{\theta_1,\theta_0}^{(K)}$ really represents the same process as $M_{\theta_1,\theta_0}^{\langle I \rangle}$, but at the more detailed unit time level, which is what we wanted to show.

Unlike $M_{\theta_1,\theta_0}$, $U_{\theta_1,\theta_0}$ cannot be thought of as a standard likelihood under any fully defined distribution for the process $\{ \vec{Y}(k \epsilon)\}_{k = 0,1,2 \ldots}$; it is simply set to $1$ at times at which no event happens, whereas any ratio of 'real' underlying likelihoods would presumably not be $1$ at all those times. But this is not a problem: it is really a <span>*partial*</span> likelihood, in the sense of @Cox75, which is sufficient for the corresponding ratios to constitute a test martingale.

#### Remark: Odds vs Hazard Ratios

The standard interpretation of the parameter $\theta$ in a hypergeometric distribution is as an <span>*odds ratio*</span>, i.e. a quantity of the form $(p_1/p_0) \cdot (1- p_0)/(1- p_1)$. At first sight it might seem strange that in this theorem, it takes the form $\theta = \lambda_{1}/\lambda_{0}$. What happened to the $(1-\lambda_{g})$s? This is readily explained by checking the proof: at discretization level $\epsilon$, $P_{[\epsilon],\theta}$ is really approximated by a hypergeometric with parameter $(\lambda_{1} \epsilon / \lambda_{0} \epsilon ) \cdot (1- \lambda_{0}
\epsilon) / (1- \lambda_{1} \epsilon)$. Taking the limit $\epsilon \downarrow 0$ the $1- \lambda_{g}$ factors disappear.

\[3\]<span>\#3</span>

Online Appendix B {#online-appendix-b .unnumbered}
=================

Expected Stopping Time, GROW and Wald's Identity  {#app:wald}
=================================================

Let $P_{\theta_0}$ represent our null model, and let, as before, the alternative model be given as $\cH_1 = \{P_{\theta_1}: \theta_1\in \Theta_1 \}$ with $\Theta_1 = \{\theta_1: 0 < \theta_1 \leq \theta_{\min} \}$ for some $\theta_{\min} < 1$. Suppose we perform a level $\alpha$ test based on a test martingale $M_{\theta_1,\theta_0}$ using the aggressive stopping rule: stop as soon as $M_{\theta_1,\theta_0} \geq 1/\alpha$. The GROW criterion (, Example \[ex:GROW\]) tells us to use $\theta_1=\theta_{\min}$. Here we motivate this GROW criterion by showing that it minimizes, in a worst-case sense, the expected number of events needed before there is sufficient evidence to stop. The calculation below ignores the practical need to prepare for a bounded maximum number of events. For such more complicated considerations, we need to resort to simulations as in the main text.

We will further make the simplifying assumption that the initial risk sets (i.e. $m_0$ and $m_1$) are large enough so that for all sample sizes we will ever encounter, $ {\ensuremath{Y_{0} \langle i \rangle}}/ {\ensuremath{Y_{1} \langle i \rangle}} \approx m_0/m_1$. This allow us to act as if the random variables ${\ensuremath{O_{1} \langle i \rangle}}$ are i.i.d. Bernoulli: the $i$th event is sampled from $q_{\theta_1}\big({\ensuremath{O_{1} \langle i \rangle}} \mid \left({\ensuremath{Y_{0} \langle i \rangle}}, {\ensuremath{Y_{1} \langle i \rangle}}\right)$ for some $\theta_1 \in \Theta_1$, with $q_{\theta_1}$ as given by (\[eq:qSingle\]), which becomes independent of $i$ if $y_0/y_1$ is replaced by $ m_0/m_1$. Assuming i.i.d. data enables a standard argument based on Wald's ([-@Wald52]) identity, originally due to @breiman1961optimal. As said, we stop as soon as $M:= M_{\theta_1,\theta_0} \geq 1/\alpha$ or when we run out of data, leading to a stopping time $\tau_{\theta_1}$ (which we will denote by $\tau$ when $\theta_1$ is clear from the context). Suppose first that we happen to know that the data comes from a specific $\theta \in \Theta_1$. Wald's identity now gives: $${\bf E}_{P'_{\theta}}[\tau]
=\frac{{\bf E}_{P'_{\theta}}[\log M^{\langle \tau \rangle}_{\theta_1,\theta_0}]}{{\bf E}_{P'_{\theta}}[\log {\ensuremath{M_{\theta_1,\theta_0} \langle 1 \rangle}}]}.$$ For simplicity we will further assume that the number of people at risk is large enough so that the probability that we run out of data before we can reject is negligible. The right-hand side can then be further rewritten as $$\begin{aligned}
\label{eq:bacon}
\frac{{\bf E}_{P'_{\theta}}[\log {M}^{\langle \tau \rangle}_ {\theta_1,\theta_0}]}
     {{\bf E}_{P'_{\theta}}\left[\log \frac{p'_{\theta_1}({\ensuremath{O_{1} \langle 1 \rangle}})
         }{p'_{\theta_0}({\ensuremath{O_{1} \langle 1 \rangle}})} \right]} =
     \frac{\log \frac{1}{\alpha} +\text{\sc very small}}{
     {\bf E}_{P'_{\theta}}\left[\log \frac{p'_{\theta_1}({\ensuremath{O_{1} \langle 1 \rangle}})
         }{p'_{\theta_0}({\ensuremath{O_{1} \langle 1 \rangle}})} \right]}\end{aligned}$$ with <span>very small</span> between $0$ and $\log |\theta_1/\theta_0|$, and $p'_{\theta_1}({\ensuremath{O_{1} \langle 1 \rangle}}) = q_{\theta_1}({\ensuremath{O_{1} \langle 1 \rangle}} \mid {\ensuremath{Y_{1} \langle 1 \rangle}},{\ensuremath{Y_{0} \langle 1 \rangle}})$. The first equality is just definition, the second follows because we reject <span>*as soon as*</span> $M^{\langle \tau \rangle}_{\theta_1,\theta_0} \geq 1/\alpha$, so ${M}_{\theta_1,\theta_0}^{\langle \tau \rangle}$ can't be smaller than $1/\alpha$, and it can't be larger by more than a factor equal to the maximum likelihood ratio at a single outcome (if we would not ignore the probability of stopping because we run out of data, there would be an additional small term in the numerator).

If we try to find the $\theta_1$ which minimizes this, and—as is customary in sequential analysis—we approximate the minimum by ignoring the <span>very small</span> part, we see that the expression is minimized by maximizing ${\bf
       E}_{P'_{\theta}}[\log \frac{ p_{\theta_1}({\ensuremath{Y_{} \langle 1 \rangle}}) }{p_{\theta_0}({\ensuremath{Y_{} \langle 1 \rangle}})} ]$ over $\theta_1$. The maximum is clearly achieved by $\theta_1= \theta$; the expression in the denominator then becomes the KL divergence between two Bernoulli distributions. It follows that under $\theta$, the expected number of outcomes until rejection is minimized if we set $\theta_1 = \theta$. Thus, we use the GROW $E$-variable relative to $\{\theta \}$ as our actual $E$-variable. We still need to consider the case that, since the real $\cH_1$ is 'composite', as statisticians, we do not know the actual $\theta$; we only know $0 < \theta \leq \theta_{\min}$. So we might want to take a worst-case approach and use the $\theta_1$ achieving $$\max_{\theta_1} \min_{\theta: 0 < \theta \leq \theta_{\min}} {\bf E}_{P'_{\theta}} \left[ \log \frac{p'_{\theta_1}({\ensuremath{O_{1} \langle 1 \rangle}})}{p'_{\theta_0}({\ensuremath{O_{1} \langle 1 \rangle}})}
\right],$$ since, repeating the reasoning leading to (\[eq:bacon\]), this $\theta_1$ should be close to achieving $$\min_{\theta_1} \max_{\theta: 0 < \theta \leq \theta_{\min}} {\bf E}_{P' _{\theta}} [\tau_{\theta_1}]$$ But this just tells us to use the GROW $E$-variable relative to $\cH_1$, which is what we were arguing for.

Logrank test as a score test {#app:score}
============================

Logrank test statistic for single events and ties
-------------------------------------------------

Let ${\ensuremath{Y_{1} \langle i \rangle}}$ denote that number of participants in the risk set that are in the treatment group at the time of the $i$^th^ event, and analogously ${\ensuremath{Y_{0} \langle i \rangle}}$ for the number of participants at risk in the control group. Let ${\ensuremath{O_{1} \langle i \rangle}}$ and ${\ensuremath{O_{0} \langle i \rangle}}$ count the number of observed events in the treatment group and control group at the $i$^th^ event time. For single events ${\ensuremath{O_{1} \langle i \rangle}} = 1$ if the event occurred in the treatment group, and ${\ensuremath{O_{1} \langle i \rangle}} = 0$ if a single event occurred in the control group. We can extend this Bernoulli case to multiple simultaneous events (ties) in which case ${\ensuremath{O_{1} \langle i \rangle}}$ can be larger than 1. Here we discuss both cases (single event and ties) together, but we will discuss them separately later on. We define ${\ensuremath{Y_{} \langle i \rangle}} = {\ensuremath{Y_{1} \langle i \rangle}} + {\ensuremath{Y_{0} \langle i \rangle}}$ and ${\ensuremath{O_{} \langle i \rangle}} = {\ensuremath{O_{1} \langle i \rangle}} + {\ensuremath{O_{0} \langle i \rangle}}$.

Under the null hypothesis, at each event time $i$, the number of observed events in the treatment group <span>$O_{1} \langle i \rangle$</span> follows a hypergeometric distribution with an expected number of events <span>$E_{1} \langle i \rangle$</span> and a variance <span>$V_{1} \langle i \rangle$</span> that depends on the risk set as follows: $${\ensuremath{A_{1} \langle i \rangle}} = \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}}}, \qquad 
{\ensuremath{E_{1} \langle i \rangle}} = {\ensuremath{O_{} \langle i \rangle}} \cdot {\ensuremath{A_{1} \langle i \rangle}}, \qquad
{\ensuremath{V_{1} \langle i \rangle}} = {\ensuremath{O_{} \langle i \rangle}} \cdot {\ensuremath{A_{1} \langle i \rangle}}  \cdot \big( 1 - {\ensuremath{A_{1} \langle i \rangle}} \big)  \cdot \frac{{\ensuremath{Y_{} \langle i \rangle}} - {\ensuremath{O_{} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}} - 1}.
$$ This is the formulation found in @Cox72 [equation (26)] with $\frac{{\ensuremath{Y_{} \langle i \rangle}} - {\ensuremath{O_{} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}} - 1}$ a "multiplicity" correction or "correction for ties" [@klein_survival_2006 p. 207]. In case of single events with ${\ensuremath{O_{} \langle i \rangle}} = 1$ the variance reduces to the Bernoulli variance: $${\ensuremath{V_{1} \langle i \rangle}} = {\ensuremath{A_{1} \langle i \rangle}} \cdot \big( 1 - {\ensuremath{A_{1} \langle i \rangle}} \big) = {\ensuremath{E_{1} \langle i \rangle}} \cdot \big( 1 - {\ensuremath{E_{1} \langle i \rangle}} \big).$$ As a test statistic, a logrank $Z$-statistic is constructed that is calculated either for the treatment or for the control group and has an approximate standard normal (Gaussian) distribution under the null hypothesis. This $Z$-statistic for the treatment group $1$ is: $$Z = \frac{\sum_{i}\big\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}} \big\}}{\sqrt{\sum_{i}{\ensuremath{V_{1} \langle i \rangle}}}}.$$ This test statistic is used to reject the null hypothesis at level $\alpha$ in favor of a one-sided alternative (e.g. $\cH_0 : \lambda_1({\ensuremath{t_{} \langle i \rangle}}) < \lambda_0({\ensuremath{t_{} \langle i \rangle}})$ for some $i$) in case the value of the $Z$-statistic is smaller than $z_\alpha$. You expect a negative value for the $Z$-statistic for a lower risk in the treatment group, so you need the $\alpha$^th^ lower percentage point of the standard normal (Gaussian) distribution. A two-sided test can be constructed by comparing $|Z| \geq z_{\alpha/2}$.

Score test for the Bernoulli partial likelihood (single events)
---------------------------------------------------------------

In we constructed a partial likelihood based on the probability mass function of a Bernoulli $y_1\theta/(y_0 + y_1\theta)$-distribution. Given an observed data set of participants at risk (<span>$\vec{Y}_{} \langle i \rangle$</span>) at all event times $i$ we can define a product of Bernoulli likelihoods. Following @Cox72, we define these in terms of the logarithm of the hazard ratio, i.e. $\beta = \log\theta$ with $\theta = \lambda_1({\ensuremath{t_{} \langle i \rangle}})/\lambda_1({\ensuremath{t_{} \langle i \rangle}})$ for all event times ${\ensuremath{t_{} \langle i \rangle}}$: $$\mathcal{L} \left(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle n \rangle}} \right) = \prod_{i = 1}^n q_\beta\big({\ensuremath{O_{1} \langle i \rangle}} \mid \left({\ensuremath{Y_{0} \langle i \rangle}}, {\ensuremath{Y_{1} \langle i \rangle}}\right)\big),$$ where $$q_\beta({\ensuremath{O_{1} \langle i \rangle}} \mid \left({\ensuremath{Y_{0} \langle i \rangle}}, {\ensuremath{Y_{1} \langle i \rangle}}\right)) = \left(\frac{{\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}\right)^{{\ensuremath{O_{1} \langle i \rangle}}}\left(\frac{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}\right)^{1-{\ensuremath{O_{1} \langle i \rangle}}}.$$ This is the likelihood formulated for single events by @Cox72 for his proportional hazards model in the two-sample case (a single categorical covariate indicating two groups).

Following the likelihood above, our loglikelihood is: $$\begin{split}
    L(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle n \rangle}}) &= \sum_{i = 1}^n
    \Big\{ {\ensuremath{O_{1} \langle i \rangle}} \cdot \Big( \log[{\ensuremath{Y_{1} \langle i \rangle}}] + \beta - \log\big[{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)\big]\Big) \\
    & \qquad \quad + \big(1-{\ensuremath{O_{1} \langle i \rangle}}\big) \cdot \Big(\log[{\ensuremath{Y_{0} \langle i \rangle}}] - \log[{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)] \Big) \Big\} \\
    &= \sum_{i = 1}^n \Big\{ {\ensuremath{O_{1} \langle i \rangle}}\beta + {\ensuremath{O_{1} \langle i \rangle}} \cdot \Big( \log[{\ensuremath{Y_{1} \langle i \rangle}}] - \log[{\ensuremath{Y_{0} \langle i \rangle}}] \Big) + \log[{\ensuremath{Y_{0} \langle i \rangle}}] \\
    & \qquad \quad - \log \big[{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)\big]\Big\}.
\end{split}$$ If we omit the parts that do not depend on $\beta$, we get the expression in @Cox72: $$L(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle n \rangle}}) \propto \sum_{i = 1}^n \Big\{ {\ensuremath{O_{1} \langle i \rangle}}\beta  - \log \big[{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)\big]\Big\}.$$ So if we take the score: $$U(\beta) = \frac{\partial L(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle n \rangle}})}{\partial \beta} = \sum_{i = 1}^n\left\{ {\ensuremath{O_{1} \langle i \rangle}} - \frac{{\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)} \right\},$$ with variance (by the observed Fisher information) $$\begin{split}
    -U'(\beta) =& - \frac{\partial^2 L(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}}, \ldots, {\ensuremath{\vec{Y}_{} \langle n \rangle}})}{\partial \beta^2} \\
    =& 
    \sum_{i = 1}^n\left\{ 
    \frac{\big({\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)\big)  \cdot {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta) - {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta) \cdot {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}{\big({\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)\big)^2}
     \right\} \\
     =& \sum_{i = 1}^n\left\{ 
     \frac{{\ensuremath{Y_{0} \langle i \rangle}}  \cdot {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}{\left({\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)\right)^2}
     \right\}
     = \sum_{i = 1}^n\left\{ 
     \frac{{\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}\cdot\frac{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}} \cdot \exp(\beta)}
     \right\}.
\end{split}$$ If we evaluate the score function at a hazard ratio $\theta$ of 1, so with $\beta = \exp(\theta) = \exp(1) = 0$, we get: $$\begin{split}
    U(0) &= \sum_{i = 1}^n \left\{ {\ensuremath{O_{1} \langle i \rangle}} - \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}}} \right\} = \sum_{i = 1}^n\left\{ {\ensuremath{O_{1} \langle i \rangle}} - \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}}} \right\} = \sum_{i = 1}^n\left\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{A_{1} \langle i \rangle}} \right\}, \\
     - U'(0) &= \sum_{i = 1}^n 
     \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}}}\cdot\frac{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{Y_{0} \langle i \rangle}} + {\ensuremath{Y_{1} \langle i \rangle}}} = \sum_{i = 1}^n{\ensuremath{A_{1} \langle i \rangle}} \cdot \left(1-{\ensuremath{A_{1} \langle i \rangle}}\right).
\end{split}$$ ${\ensuremath{A_{1} \langle i \rangle}} = {\ensuremath{E_{1} \langle i \rangle}}/{\ensuremath{O_{} \langle i \rangle}} = {\ensuremath{E_{1} \langle i \rangle}}$ in case of single events. Hence by standardizing the score function under the null hypothesis, we get the logrank statistic for single events: $$Z = \frac{\sum_{i = 1}^n\left\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{A_{1} \langle i \rangle}} \right\}}{\sqrt{\sum_{i = 1}^n{\ensuremath{A_{1} \langle i \rangle}} \cdot \left(1-{\ensuremath{A_{1} \langle i \rangle}}\right)}} = \frac{\sum_{i = 1}^n\left\{ {\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}} \right\}}{\sqrt{\sum_{i} {\ensuremath{V_{1} \langle i \rangle}}}}.$$

Score test for the Fisher hypergeometric partial likelihood (tied events) {#app:scoreTies}
-------------------------------------------------------------------------

The Peto one-step estimator for odds ratios is build from a general score test for 2x2 tables. The appendix of @yusuf_beta_1985 gives description that we can use for the logrank score test in the case of ties.

If at the $i$^th^ event time ${\ensuremath{O_{} \langle i \rangle}}$ events are observed, the expression in describes the probability that ${\ensuremath{O_{1} \langle i \rangle}}$ of these events happen in the treatment group. The general description of the likelihood for the $i$^th^ event time that follows from this is: $$\begin{split}
    s_{\beta}({\ensuremath{O_{1} \langle i \rangle}} \mid \left( {\ensuremath{Y_{0} \langle i \rangle}},{\ensuremath{Y_{1} \langle i \rangle}} \right),{\ensuremath{O_{} \langle i \rangle}}) := \frac{p_{0}({\ensuremath{O_{1} \langle i \rangle}}) \cdot \exp({\ensuremath{O_{1} \langle i \rangle}} \cdot \beta)}{\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot \exp(k \cdot \beta)} \\
\text{\ with\ } {\ensuremath{O^{\min}_{1} \langle i \rangle}} = \max \{0,{\ensuremath{O_{} \langle i \rangle}}-{\ensuremath{Y_{0} \langle i \rangle}}\} ; \quad {\ensuremath{O^{\max}_{1} \langle i \rangle}} = \min\{{\ensuremath{O_{} \langle i \rangle}}, {\ensuremath{Y_{1} \langle i \rangle}} \},
\end{split}$$ with $p_0({\ensuremath{O_{1} \langle i \rangle}})$ the null hypothesis probability of ${\ensuremath{O_{1} \langle i \rangle}}$ observed events in the treatment group. Our null hypothesis is a hypergeometric distribution: $$p_0({\ensuremath{O_{1} \langle i \rangle}}) = P_0\left(({\ensuremath{O_{1} \langle i \rangle}} \mid \left( {\ensuremath{Y_{0} \langle i \rangle}},{\ensuremath{Y_{1} \langle i \rangle}} \right),{\ensuremath{O_{} \langle i \rangle}}\right) = \frac{\binom{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{O_{1} \langle i \rangle}}} \cdot \binom{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}} - {\ensuremath{O_{1} \langle i \rangle}}}}{\binom{{\ensuremath{Y_{} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}}}},$$ such that $s_{\beta} = q_{\beta}$, our Fisher hypergeometric likelihood in : $$\begin{split}
    s_{\beta}({\ensuremath{O_{1} \langle i \rangle}} \mid \left( {\ensuremath{Y_{0} \langle i \rangle}},{\ensuremath{Y_{1} \langle i \rangle}} \right),{\ensuremath{O_{} \langle i \rangle}}) &= \frac{p_{0}({\ensuremath{O_{1} \langle i \rangle}}) \cdot \exp({\ensuremath{O_{1} \langle i \rangle}} \cdot \beta)}{\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot \exp(k \cdot \beta)}
    = \frac{\frac{\binom{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{O_{1} \langle i \rangle}}} \cdot \binom{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}}-{\ensuremath{O_{1} \langle i \rangle}}}}{\binom{{\ensuremath{Y_{} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}}}} \cdot \exp({\ensuremath{O_{1} \langle i \rangle}} \cdot \beta)} {\sum_{k={\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} \frac{\binom{{\ensuremath{Y_{1} \langle i \rangle}}}{k} \cdot \binom{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}}-k}}{\binom{{\ensuremath{Y_{} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}}}} \cdot \exp(k \cdot \beta)} \\
    &= \frac{\binom{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{O_{1} \langle i \rangle}}} \cdot \binom{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}}-{\ensuremath{O_{1} \langle i \rangle}}} \cdot \exp({\ensuremath{O_{1} \langle i \rangle}} \cdot \beta)} {\sum_{k={\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} \binom{{\ensuremath{Y_{1} \langle i \rangle}}}{k} \cdot \binom{{\ensuremath{Y_{0} \langle i \rangle}}}{{\ensuremath{O_{} \langle i \rangle}}-k} \cdot \exp(k \cdot \beta)}
    = q_{\beta}({\ensuremath{O_{1} \langle i \rangle}} \mid \left( {\ensuremath{Y_{0} \langle i \rangle}},{\ensuremath{Y_{1} \langle i \rangle}} \right),{\ensuremath{O_{} \langle i \rangle}}).
\end{split}$$ So our likelihood is: $$\mathcal{L} \left(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}},{\ensuremath{\vec{Y}_{} \langle 3 \rangle}},\ldots \right) = \prod_{i} s_\beta({\ensuremath{O_{1} \langle i \rangle}} \mid \left({\ensuremath{Y_{0} \langle i \rangle}}, {\ensuremath{Y_{1} \langle i \rangle}}\right), {\ensuremath{O_{} \langle i \rangle}})$$ And the loglikelihood is: $$\begin{split}
    L(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}},{\ensuremath{\vec{Y}_{} \langle 3 \rangle}},\ldots) = \sum_{i} \left\{ \log\left[ p_0({\ensuremath{O_{1} \langle i \rangle}}) \right] + {\ensuremath{O_{1} \langle i \rangle}} \cdot \beta - \log \left[ \sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot \exp(k \cdot \beta)) \right] \right\}.
\end{split}$$ And its score: $$U(\beta) = \frac{\partial L(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}},{\ensuremath{\vec{Y}_{} \langle 3 \rangle}},\ldots)}{\partial \beta} = \sum_{i} \left\{ {\ensuremath{O_{1} \langle i \rangle}} - \frac{\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot k \cdot \exp(k \cdot \beta)}{\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot \exp(k \cdot \beta)} \right\},$$ with variance (by the observed Fisher information) $$\begin{split}
    -U'(\beta) &= - \frac{\partial^2 L(\beta \mid {\ensuremath{\vec{Y}_{} \langle 1 \rangle}},{\ensuremath{\vec{Y}_{} \langle 2 \rangle}},{\ensuremath{\vec{Y}_{} \langle 3 \rangle}},\ldots)}{\partial \beta^2} \\
    &= 
    \sum_{i} \left\{ \frac{\sum_{k} p_{0}(k) \cdot \exp(k \cdot \beta) \cdot \sum_{k} p_{0}(k) \cdot k^2 \cdot \exp(k \cdot \beta) - \left(\sum_{k} p_{0}(k) \cdot k \cdot \exp(k \cdot \beta)\right)^2}{\left(\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot \exp(k \cdot \beta)\right)^2} \right\}.
\end{split}$$ If we evaluate the score function at a hazard ratio $\theta$ of 1, so with $\beta = \exp(\theta) = \exp(1) = 0$, we get: $$\begin{split}
    U(0) &= \sum_{i} \left\{ {\ensuremath{O_{1} \langle i \rangle}} - \frac{\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot k}{\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k)} \right\}
    = \sum_{i} \left\{ {\ensuremath{O_{1} \langle i \rangle}} - \sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k) \cdot k \right\} \\
    &= \sum_{i} \left\{{\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{O_{1} \langle i \rangle}} \cdot \frac{{\ensuremath{Y_{1} \langle i \rangle}}}{{\ensuremath{Y_{} \langle i \rangle}}} \right\} = \sum_{i} \left\{{\ensuremath{O_{1} \langle i \rangle}} - {\ensuremath{E_{1} \langle i \rangle}} \right\} \\
    U'(O) &= \sum_{i} \left\{ \frac{\sum_{k} p_{0}(k) \cdot \sum_{k} p_{0}(k) \cdot k^2 - \left(\sum_{k} p_{0}(k) \cdot k\right)^2}{\left(\sum_{k = {\ensuremath{O^{\min}_{1} \langle i \rangle}}}^{{\ensuremath{O^{\max}_{1} \langle i \rangle}}} p_{0}(k)\right)^2} \right\} \\
    &= \sum_{i} \left\{ \sum_{k} p_{0}(k) \cdot k^2 - \left(\sum_{k} p_{0}(k) \cdot k\right)^2 \right\}
    = \sum_{i}{\ensuremath{V_{1} \langle i \rangle}}.
\end{split}$$

Proof of Theorem \[thm:martingalemoreevents\] {#app:theory}
=============================================

The first part is an immediate consequence of the independence of the $T_j$. The real work is in the second part: we define the $b$-component binary vector $$\vec{O}_{K} := (O{[K,K+\epsilon]}, O{[K+ \epsilon, K+ 2 \epsilon]}, \ldots, O{[K + (b-1)  \epsilon, K+1]})$$ which indicates at what $\epsilon = 1/b$-width time intervals between time $K$ and $K+1$ events happened in either group; the $k$th entry of this vector is the number of events that happened between time $K+ (k-1)\epsilon$ and $K+ k \epsilon$. Similarly, $\vec{O}_{K,g}$ is a defined as a vector whose $k$th entry indicates the number of events in group $g$ inbetween time $K + (k-1) \epsilon$ and $K+ k \epsilon$.

We now first determine, for binary $b$-component vectors $\vec{s}$ with $s\geq v$ ones, the probability $$\begin{gathered}
\label{eq:vratio}
P_{[\epsilon],\theta}(O_1[K-1,K] = {v} \mid
\vec{O}_{K-1} = \vec{s}, \vec{Y}(K-1) = \vec{y}) = \\
\frac{\sum_{\vec{w} \in \{0,1\}^b: |\vec{w}| = v} P_{[\epsilon],\theta}(\vec{O}_{K-1,1} = \vec{w},
\vec{O}_{K-1} = \vec{s} \mid \vec{Y}(K-1) = \vec{y})}{
\sum_{v'= 0,1, \ldots, s} \sum_{\vec{w} \in \{0,1\}^b: |\vec{w}| = v'} P_{[\epsilon],\theta}(\vec{O}_{K-1,1} = \vec{w},
\vec{O}_{K-1} = \vec{s} \mid \vec{Y}(K-1) = \vec{y})}.\end{gathered}$$ After having found an expression for this conditional probability, we will argue that it coincides, up to a $(1+ o(\epsilon))$ factor, with the probability that we are after. We can write the probabilities occurring in the sums in (\[eq:vratio\]) as $$\begin{aligned}
\label{eq:tuin}
P_{[\epsilon],\theta}(\vec{O}_{K-1,1} = \vec{w},
\vec{O}_{K-1} = \vec{s} \mid \vec{Y}(K-1) = \vec{y}) =
a_{\vec{w}} \cdot a'_{\vec{w}},\end{aligned}$$ where $a_{\vec{w}}$, satisfying (\[eq:aw\]), collects all factors corresponding to intervals of length $\epsilon$ during which nothing happens (i.e. the components of $\vec{s}$ with $0$ entry), and $a'_{\vec{w}}$, given by (\[eq:aws\]), collects the $s$ other factors: $$\label{eq:aw}
\left(
(1- \bar{\lambda}_1 \epsilon)^{y_1}
(1- \bar{\lambda}_0 \epsilon)^{y_0}
\right)^{b-s} \leq a_{\vec{w}} \leq \left(
(1- \underline{\lambda}_1 \epsilon)^{y_1 - s}
(1- \underline{\lambda}_0 \epsilon)^{y_0 - s}
\right)^{b-s}.$$ where $\bar{\lambda}_g = \sup_{K \leq t < K+1} \lambda_{g}(t)$, $\underline{\lambda}_g = \inf_{K \leq t < K+1} \lambda_{g}(t)$, so that $a_{\vec{w}}= 1+ o(\epsilon)$.

To get an expression for $a'_{\vec{w}}$, first let, for $u \in [s]$, $\ell_u$ be the index of the $u$^th^ component in $\vec{s}$ that is $1$. That is, if $\vec{s} = (0,0,0,1,0,1, \ldots)$ then $\ell_1 =4$, $\ell_2= 6$, and so on. $\vec{O}_{K-1} = \vec{s}$ thus means that a single event happened in intervals $[(K-1) + (\ell_1 -1) \epsilon, (K-1) + \ell_1  \epsilon]$ and $[(K-1) + (\ell_2-1) \epsilon, (K-1) + \ell_2 \epsilon]$, and so on. Let $k_1(\vec{v}), k_2(\vec{v}), \ldots, k_s(\vec{v})$ be defined such that $k_u(\vec{v}) = 1$ if ${v}_{\ell_u} = 1$ and $k_u(\vec{v})= 0$ otherwise. $k_u(\vec{v})$ represents whether the $u$^th^ of the $s$ events inbetween time $K-1$ and $K$ was in group $1$ or $0$. Below we abbreviate $\lambda_{g}((K-1) +  \ell_u \epsilon)$ to $\lambda_{g,u}$. We can then write: $$\begin{gathered}
 \label{eq:aws}
 a'_{\vec{w}}=   \prod_{u=1}^s
(y_1- \sum_{u' <  u} k_{u'}(\vec{w}) ) (1- \lambda_{1,u} \epsilon)^{y_1- \sum_{ u'\leq u} k_{u'}(\vec{w})}
\cdot ({\lambda}_{1,u} \epsilon) ^{k_u(\vec{w})}
\cdot  \\
(y_0- \sum_{ u' <  u} (1- k_{u'}(\vec{w})) ) (1- \lambda_{0,u} \epsilon)^{y_0- \sum_{ u'\leq u} (1-k_{u'}(\vec{w}))}
\cdot ( {\lambda}_{0,u} \epsilon)^{1-k_u(\vec{w})}
= (1+ o(\epsilon)) p(\vec{w}),\end{gathered}$$ with, with $w = |\vec{w}|$, and using that, by our proportional hazards assumption, $\lambda_{1,u} = \theta \lambda_{0,u}$: $$\begin{aligned}
    p(\vec{w}) & = \prod_{u=1}^s
\left(y_1- \sum_{0 < u' <  u} k_{u'}(\vec{w} )\right)
\cdot \lambda_{1,u}^{k_u(\vec{w})}
\cdot
\left(y_0- \sum_{0 < u' <  u} (1- k_{u'}(\vec{w}) )\right)
\cdot {\lambda}_{0,u}^{1-k_u(\vec{w})}
\\ & = \prod_{u=0}^{w-1} (y_1 - u) \cdot \theta^w \cdot \prod_{u=0}^{s-w -1 } (y_0 -u)  \prod_{u=1}^s \lambda_{0,u}
= w! (s-w)! \binom{y_1}{w} \binom{y_0}{s-w}
\theta^w \prod_{u=1}^s \lambda_{0,u}.\end{aligned}$$ This gives: $$\sum_{\vec{w} \in \{0,1\}^b: |\vec{w}| = w}
p(\vec{w}) = \binom{s}{w} p(\vec{w}) = s! \cdot
\binom{y_1}{w} \binom{y_0}{s-w}
\theta^{w}\prod_{u=1}^s \lambda_{0,u}$$ Plugging this into (\[eq:vratio\]), using (\[eq:tuin\]) and the already established fact that $a_{\vec{w}}= 1+ o(\epsilon)$, we get $$\begin{aligned}
& P_{[\epsilon],\theta}(O_1[K-1,K] = {v} \mid
\vec{O}_{K-1} = \vec{s}, \vec{Y}(K-1) = \vec{y}) =  (1 +o(\epsilon)) \cdot \frac{\sum_{\vec{w} \in \{0,1\}^b: |\vec{w}| = v} p(\vec{w})}{
\sum_{v'= 0,1, \ldots, s}
\sum_{\vec{w} \in \{0,1\}^b: |\vec{w}| = v'}
p(\vec{w})
} \\ = &
(1 +o(\epsilon)) \cdot \frac{ s! \cdot
\binom{y_1}{v} \binom{y_0}{s-v}
\theta^{v}\prod_{u=1}^s \lambda_{0,u}}{
\sum_{v'= 0,1, \ldots, s}  s! \cdot
\binom{y_1}{v'} \binom{y_0}{s-v'}
\theta^{v'}\prod_{u=1}^s \lambda_{0,u}} =
(1+ o(\epsilon)) \cdot
\frac{\binom{y_1}{v} \binom{y_0}{s-v}
\theta^v }{\sum_{v'=0..s} \binom{y_1}{v'} \binom{y_0}{s-v'}
\theta^w }\end{aligned}$$ which only depends on the number of events $s$ inbetween $K=1$ and $K$ and not on their specific location in the vector $\vec{s}$. We have thus shown that $$\begin{aligned}
&  P_{[\epsilon],\theta}(O_1[K-1,K] = {v} \mid
O[K-1,K]=s, \vec{Y}(K-1) = \vec{y}, \cE_{K}) =\\
&  P_{[\epsilon],\theta}(O_1[K-1,K] = {v} \mid
\vec{O}_{K-1} = \vec{s}, \vec{Y}(K-1) = \vec{y})=  (1+ o(\epsilon)) \cdot
\frac{\binom{y_1}{v} \binom{y_0}{s-v}
\theta^v }{\sum_{v'=0..s} \binom{y_1}{v'} \binom{y_0}{s-v'}
\theta^w }
   \end{aligned}$$ where $\cE_K$ is the event that not more than one event took place in any $\epsilon$-length interval between $K-1$ and $K$. The first version of the desired result (conditioning on $\vec{Y}(K-1) = \vec{y}$) then follows since the probability of interest satisfies $$\begin{gathered}
  P_{[\epsilon],\theta}(O_1[K-1,K] = {v} \mid
O[K-1,K]=s, \vec{Y}(K-1) = \vec{y})= \\ P_{[\epsilon],\theta}(O_1[K-1,K] = {v} \mid
O[K-1,K]=s, \vec{Y}(K-1) = \vec{y}, \cE_{K}) \cdot (1 + o(\epsilon))\end{gathered}$$ which in turn follows because $\Pr(\cE_K \mid \vec{Y}(K-1) = \vec{y}) = 1 - o(\epsilon)$, which in turn is easily shown (details omitted). The final theorem statement follows by replacing the conditioning event $\vec{Y}(K-1) = \vec{y}$ by $Y_0(K-1) = y_0;Y_1(K-1)  = y_1$ and following exactly the same steps as above.

Details of sample size comparison simulations {#app:moreSampleSize}
=============================================

In this section we lay out the procedure that we used to estimate the expected and maximum number of events required to achieve a predefined power as shown in and in and . First we describe how we sampled the survival processes under a specific hazard ratio. We then describe how we estimated the maximum and expected sample size required to achieve a predefined power (80% in our case) for any of the test martingales that we considered (that of the exact safe logrank, its Gaussian approximation, and its prequential-plugin variation). Finally, we explain how the numbers for the classical logrank test and the O'Brien-Fleming procedure were obtained.

In order to simulate the order in which the events in a survival processes happens, we used the risk set process from . Indeed, if we are testing some fixed $\theta_1$ with $\theta_1 \leq 1$ against $\theta_0=1$, the odds of next event at the $i$^th^ event time happening in group $1$ are $\theta_1{\ensuremath{Y_{1} \langle i \rangle}}:{\ensuremath{Y_{0} \langle i \rangle}}$ under the alternative hypothesis, which is the hypothesis we sample from. Thus, simulating in which group the next event happens only takes a (biased) coin flip. We consider the one-sided testing scenario $\theta_1$ (for some $\theta_1\in (0,1)$) vs. $\theta_0 = 1$, and we fix our desired level to $\alpha = 0.05$. For each test martingale $M_{\theta_1,1}^{\langle n \rangle}$ of interest we first consider the stopping rule $\tau = \inf\{n: M_{\theta_1,1}^{\langle n \rangle}\geq 1/\alpha\}$, that is, we stop as soon as $M_{\theta_1,1}^{\langle n \rangle}$ crosses the threshold $1/\alpha$. In order to estimate the maximum number of events needed to achieve a predefined power with a given test martingale, we turned our attention to a modified stopping rule $\tau'$. Under $\tau'$ we stop at the first of two moments: either when our test martingale $ M_{\theta_1,1}^{\langle n \rangle}$ crosses the threshold $1/\alpha$ (i.e. at $\tau$) or once we have witnessed a predefined maximum number of events $n_{\max}$. More compactly, this means using the stopping rule $\tau'$ given by $\tau' = \min (\tau, n_{\max})$. In those cases in which the test based on the stopping rule $\tau$ achieves a power higher than $1-\beta$ (for a type-II error rate $\beta$), a maximum number of events $n_{\max}$ smaller than the initial size of the combined risk groups can be selected to achieve approximate power $1-\beta$ using the rule $\tau'$. A quick computation shows that $n_{\max}$ has the following property: it is the smallest number of events $n$ such that stopping after $n$ events has probability smaller than $1-\beta$ under the alternative hypothesis, that is, $$P_{\theta_1}(\tau \geq n)\leq 1-\beta.$$ More succinctly, $n_{\mathrm{max}}$ is the (approximate) $(1-\beta)$-quantile of the stopping time $\tau$ and can in consequence be estimated experimentally in a straightforward manner.

In order to estimate $n_{\max}$ for a given risk-set sizes $m_1,m_0$ and alternative hypothesis hazard ratio $\theta_1$, we sampled $10^4$ realizations of the survival process (under $\theta_1$) using the method described at the beginning of this section. This allowed us to obtain the same number of realizations of the stopping time $\tau$. We then computed the $(1-\beta)$-quantile of the observed empirical distribution of $\tau$, and reported it as an estimate of the number of events $n_{\max}$ in the 'maximum' column in .

We assessed the uncertainty in the estimation $n_{\mathrm{max}}$ using the bootstrap. We performed 1000 bootstrap rounds on the sampled empirical distribution of $\tau$, and found that the number of realizations that we sampled ($10^4$) was high enough so that plotting the uncertainty estimates was not meaningful relative to the scale of our plots. For this reason we omitted the error bars in and .

In the 'mean' column of and we plotted an estimate of the expected number of events $\tau' = \min (\tau, n_{\max})$. For this, we used the empirical mean of the stopping times that were smaller than $n_{\max}$ on the sample that we obtained by simulation, with 20% of the stopping times being $n_{\max}$ itself. In the 'conditional mean' column, we plotted an estimate of $\tau' \mid \tau'< n_{\max}$, i.e. the stopping time given that we stop early (and hence reject the null). For comparison, we also show the number of events that one would need under the Gaussian non-sequential approximation of @schoenfeld1981asymptotic, and under the continuous monitoring version of the O'Brien-Fleming procedure. In order to judge @schoenfeld1981asymptotic's approximation, we report the number of events required to achieve 80% power. This is equivalent to treating the logrank statistic as if it were normally distributed, and rejecting the null hypothesis using a $z$-test for a fixed number of events. The power analysis of this procedure is classical, and the number of events required is $n_{\max}^{S} = 4(z_\alpha + z_\beta)^2 / \log^2\theta_1$, where $z_\alpha$, and $z_\beta$ are the $\alpha$, and $\beta$-quantiles of the standard normal distribution. In the case of the continuous monitoring version of O'Brien-Fleming's procedure, we estimated the number of events $n_{\max}^{OF}$ needed to achieve 80% as follows. For each experimental setting $(m_0,m_1, \theta)$, we generated $10^4$ realizations of the survival process under the alternative hypothesis under consideration and computed the corresponding trajectories of the logrank statistic. For each possible value $n$ of $n_{\max}^{OF}$, we computed the fraction of trajectories for which O'Brien-Fleming's procedure correctly stopped when used with the maximum number of events set to $n$. We report as an estimate of the true $n_{\max}^{OF}$ the first value of $n$ for which this fraction is higher than 80%, our predefined power.

[^1]: Corresponding author: joshuasimmons@example.net

[^2]: For the case $i=0$, (\[eq:eval\]) should be read as $ {\bf E}_P\left[ {\ensuremath{S_{} \langle 1 \rangle}} \right] \leq 1$ .

[^3]: Readers familiar with measure-theoretic terminology may recognize a test (super-) martingale as a nonnegative (super-) martingale relative to the filtration $\sigma({\ensuremath{Y_{} \langle 1 \rangle}}), \sigma({\ensuremath{Y_{} \langle 1 \rangle}}, {\ensuremath{Y_{} \langle 2 \rangle}}),
    \ldots$. The requirement that '${\ensuremath{S_{} \langle i \rangle}}$ is determined by $({\ensuremath{Y_{} \langle 1 \rangle}}, \ldots, {\ensuremath{Y_{} \langle i \rangle}})$' then means that the process $\{{\ensuremath{S_{} \langle i \rangle}}\}_{i \in \naturals}$ is adapted to this filtration.

{'source': 'arXiv_out/2011'}