Survival analysis in clinical trials — Log-rank test | by Tereza Burgetova | Towards Data Science

In Part 1 of the series, I explained the reasoning behind the Kaplan-Meier estimator of the survival curve. While it is a ubiquitous tool for visualizing the results of a clinical trial that involves a time-to-event analysis, it is only descriptive. To be able to draw conclusions and show the benefits of a new treatment, we need a formal testing procedure. One such procedure is the log-rank test.
I will take a look at how it’s derived and why it’s still considered one of the most important indicators of drug efficacy.
We start with a ggplot of a survfit object, using the same data set as in Part 1:
library(survival)library(survminer)data(ovarian)fit <- survfit(Surv(ovarian$futime, ovarian$fustat) ~ rx,                data = ovarian, conf.type="log")p_fit <- ggsurvplot(  fit, conf.int = TRUE, pval = TRUE)p_fit
How the survival probability estimates and confidence intervals are obtained is described in Part 1. If pval is set to TRUE, the function also prints the p-value 0.3, which is a result of a log-rank test. So how was it calculated?
The log-rank test is basically multiple separate Fisher’s exact tests pooled together. I will show exactly what I mean by that later. But it is essential to first understand how the Fisher’s exact test works. So let’s start with a fictitious example.
Suppose there is an Introduction to statistics class for second-year university students. They can attend tutorials during the semester, which some do and some don’t. After the final exam, we want to see if failing the exam is associated with tutorial attendance. And so for every student, we record whether they were present at the tutorials at least 75% of the time in addition to whether they passed or not. And we get the following picture:
Now we want to test whether tutorial attendance and passing the exam are independent. The probabilities table looks like this:
Meaning, for example, that
and the hypothesis we want to test is H0: π11/π1+ = π21/π2+. Now suppose this hypothesis is true. This means that we would expect the proportion of failed students in the group with high tutorial attendance to be similar to the proportion of failed students in the group with low attendance. In other words,
with A being the random variable whose value is the number of students who both attended 75%+ of tutorials and passed the exam. What is the distribution of A?
Note that n++ as well as the marginals (fail/pass & attendance - yes/no counts) are fixed. Under the null hypothesis, the probability that A=a (probability that the number of students who passed and attended 75%+ of tutorials is equal to a) is the probability that if out of n++=343 students, of which 217 attended 75%+ tutorials, we pick 281 at random, a would have attended 75%+ of tutorials.
This is because we assume that those 281 students who passed are no different to the whole sample of 343 of students in terms of how much they went to tutorial classes. The distribution of A is called the hypergeometric distribution, and one can both derive or just copy from Wikipedia its probability mass function and variance:
Calculating exact p-values for samples this big is problematic, but we can use the chi-squared approximation of the following statistic instead:
Suppose the same experiment was done at 12 different universities. We don’t want to combine these into one table, as there might be differences between the universities. We want to test the following hypothesis:
H0: π11 = π12, π21 = π22, ..., πk1 = πk2, where k is the number of sites (in this case 12).
This can be done using the Mantel–Haenszel statistic:
i is the group index, O is the sum of observed deaths, E the sum of expected deaths, and V is the sum of variances of the deaths in population 1, over all sites. We plug in the sums into MH and note its asymptotic distribution:
This is used to test the null hypothesis the usual way. Now we can use this same statistic and apply it on our censored data.
How does this relate to the log-rank test? Essentially, we look at each time point when death was observed as the “university” in the above example and create a contingency table for each.
Even though the samples (“universities”) are not independent, the asymptotic distribution of Menzel-Haenszel statistic is still chi-squared with one degree of freedom.
Effectively, we test the hypothesis H0: F1 = F2, or the equality of survival curves for the combined treatment and standard treatment group.
Now let’s check how they obtained the p-value on the graph from the beginning of the article. We basically create a contingency table for each time point t of an event.
From this table, we can calculate the number of deaths a and the expectation and variance of A for each ti under the null hypothesis. All that is left is to sum these and plug in into the expression for our test statistic, MH. In this case, MH2 = (7–5.23)2 / 2.94 = 1.066.
Note that the event times do not enter the statistic calculations, only their ranks do.
In R, the log-rank test can be performed with the function survdiff() from the package survival.
logr <- survdiff(Surv(ovarian$futime, ovarian$fustat) ~ rx,                  data = ovarian)logr## Call:##   survdiff(formula = Surv(ovarian$futime, ovarian$fustat) ~ rx,##            data = ovarian)## ##       N Observed Expected (O-E)^2/E (O-E)^2/V## rx=1 13        7     5.23     0.596      1.06## rx=2 13        5     6.77     0.461      1.06## ## Chisq= 1.1  on 1 degrees of freedom, p= 0.3
Apart from some differences due to rounding, the test gives the same answer as was calculated manually above. In particular (O-E)^2/V is 1.06, which is the MH2 statistic. The probability of a chi-squared distributed variable with 1 degree of freedom being larger than this is 0.3, which can be checked with the following code:
1 - pchisq(1.06, df = 1)## [1] 0.3032152
Hence, we cannot reject the null hypothesis that the combined treatment is no better (or worse) than the standard one.
To account for covariates, we can stratify the sample. In this case, there is a variable resid.ds in the data set, indicating whether there was minimal residual disease present (resid.ds=1), or the disease was advanced (resid.ds=2) at the beginning of the study.
What stratification does is it performs the analyses of the series of contingency tables for each strata separately and then pools the results. This ensures that if there are differences between the women with minimal vs. with extensive residual disease (which is to be expected), this will not skew the estimate of the treatment effect.
There are more patients with extensive residual disease than minimal residual disease in the control group (8 vs. 5), while the ratio is more balanced in the treatment group (7 vs. 6).
table(ovarian$resid.ds, ovarian$rx, dnn=c("resid.ds", "rx"))##         rx## resid.ds 1 2##        1 5 6##        2 8 7
To ensure that we test for difference in the treatment and not for the difference between the patients with less advanced vs. advanced disease, we calculate the observed and expected counts and their variance separately in each stratum (extensive vs. minimal residual disease). The event times (censored times are followed by a plus sign) are listed for each treatment group at the top right:
Pooling the results together (taking the sum of a, E(A) and Var(A) over both groups), we obtain MH2 = (7–5.09)2/(0.743+2.093) = 1.286. Now we check the output of the survdiff() function when the strata(resid.ds) argument in the formula is added:
logr_strat <- survdiff(Surv(ovarian$futime, ovarian$fustat) ~ rx +                          strata(resid.ds), data = ovarian)logr_strat## Call:##   survdiff(formula = Surv(ovarian$futime, ovarian$fustat) ~ rx + ##              strata(resid.ds), data = ovarian)## ##       N Observed Expected (O-E)^2/E (O-E)^2/V## rx=1 13        7      5.1     0.712      1.28## rx=2 13        5      6.9     0.525      1.28## ## Chisq= 1.3  on 1 degrees of freedom, p= 0.3
Again, except for a rounding error, we obtain the same result for the MH2, i.e. (O-E)^2/V is 1.28.
In this case, the p-value is still 0.3 and hence does not change much compared to the value before stratification. Note that for such important covariates that are expected to have a significant effect on the survival, it is crucial to perform stratification at the randomization stage, i.e. when designing the experiment. This means that we must ensure that the proportion of patients from each strata is approximately the same in each treatment group, so that the test is powerful enough for the potential treatment effect to be detected.
The log-rank test is used when we want to establish whether survival distributions of two groups differ. It is one of the most prevalent and accepted ways of providing evidence that a drug has a positive treatment effect. It arises as an adaptation of Fisher’s exact test, or more precisely the Mantel Haenszel test for censored data. A contingency table is created for each time of event and the results are then pooled to compute the desired statistic.
If we want to avoid accidentally testing for another covariate that is imbalanced between the groups, or prevent the treatment effect from being masked by a covariate, we can use stratification. In that case, the contingency tables are created separately for each level of the covariate, and the results are pooled together, as shown above.
The log-rank test is non-parametric: we did not assume any distribution of the event times.
Nevertheless, it arises as score test in the semi-parametric Cox proportional hazards model (more on that in the next article).
Note that ‘testing for a significant treatment effect’ is an arbitrary task in case of comparing two survival curves. Here is a nice illustration of the different scenarios that all have a positive treatment effect. The log-rank is oriented towards one type of those scenarios.