The usual development cycles are too slow for the development of vaccines, diagnostics and treatments in pandemics such as the ongoing SARS-CoV-2 pandemic. Given the pressure in such a situation, there is a risk that findings of early clinical trials are overinterpreted despite their limitations in terms of size and design. Motivated by a non-randomized open-label study investigating the efficacy of hydroxychloroquine in patients with COVID-19, we describe in a unified fashion various alternative approaches to the analysis of non-randomized studies and apply them to the example study exploring the question whether different methods might have led to different conclusions . A widely used tool to reduce the impact of treatment-selection bias are so-called propensity score (PS) methods. Conditioning on the propensity score allows one to replicate the design of a randomized controlled trial, conditional on observed covariates. Extensions include the doubly robust g-computation  , which is less frequently applied, in particular in clinical studies.  Here, we investigate the properties of propensity score based methods including g-computation in small sample settings, typical for early trials, in a simulation study. We conclude that the doubly robust g-computation has some desirable properties and should be more frequently applied in clinical research. In the hydroxychloroquine study, g-computation resulted in a very wide confidence interval indicating much uncertainty. We speculate that application of the method might have prevented some of the hype surrounding hydroxychloroquine in the early stages of the SARS-CoV-2 pandemic. R code for the g-computation is provided.
The usual development cycles are too slow for the development of vaccines, diagnostics and treatments in pandemics such as the ongoing SARS-CoV-2 pandemic. Given the pressure in such a situation, there is a risk that findings of early clinical trials are overinterpreted despite their limitations in terms of size and design. Motivated by a non-randomized open-label study investigating the efficacy of hydroxychloroquine in patients with COVID-19, we describe in a unified fashion various alternative approaches to the analysis of non-randomized studies . A widely used tool to reduce the impact of treatment-selection bias are so-called propensity score (PS) methods. Conditioning on the propensity score allows one to replicate the design of a randomized controlled trial, conditional on observed covariates. Extensions include the g-computation approach , which is less frequently applied, in particular in clinical studies. Moreover, doubly robust estimators provide additional advantages. Here, we investigate the properties of propensity score based methods including three variations of doubly robust estimators in small sample settings, typical for early trials, in a simulation study. R code for the simulations is provided.
D
and apply them to the example study exploring the question whether different methods might have led to different conclusions
D
doubly robust
A
approach
A
Moreover, doubly robust estimators provide additional advantages.
R
g-computation
three variations of doubly robust estimators
D
We conclude that the doubly robust g-computation has some desirable properties and should be more frequently applied in clinical research. In the hydroxychloroquine study, g-computation resulted in a very wide confidence interval indicating much uncertainty. We speculate that application of the method might have prevented some of the hype surrounding hydroxychloroquine in the early stages of the SARS-CoV-2 pandemic.
R
g-computation
simulations