Letter to the Editor: Comparison of statistical approaches dealing with time-dependent confounding in drug effectiveness studies (SMMR, Vol. 27, Issue 6, 2018)

To the Editor, In the recent paper by Karim et al., on the comparison of approaches dealing with time-dependent confounding in drug effectiveness studies, we are happy to see that one of the methods being considered is the sequential Cox approach, as suggested in Gran et al. The sequential Cox approach allows for the use of conventional Cox proportional hazard regression to deal with the rather complex problem of time-dependent confounding in a survival setting. This is a useful supplement to the novel, but more complex, methods that have been developed over the last decades, such as the inverse probability of treatment weighted marginal structural Cox model, which also is part of Karim et al.’s comparison. However, the conclusions made in the Karim et al. paper leave us with some concerns. Our first concern is that the suggested modification of the sequential Cox approach, as made by authors, is not properly justified. A related, second, concern is that the comparisons made between the methods are not fully reasonable, and that more work is needed to both understand the causal estimands in question and to improve current simulation studies. In the modified version of the sequential Cox approach by Karim et al. (p. 1712), they suggest, in each minitrial, to adjust for post-treatment measurements of time-dependent variables. The dangers of adjusting for such internal time-dependent variables are however well known and discussed extensively in the survival analysis literature (see for example Kalbfleisch and Prentice (p. 198) and Aalen et al. (p. 132)). The problem with adjusting for such variables is also discussed widely in the causal inference literature on time-dependent confounding. Here, it is being argued that adjusting for a variable acting as a mediator between exposure and outcome would remove parts of the total effect of treatment and – at best – estimate a direct effect of treatment (see for example Petersen et al.). The fact that these issues are not mentioned, or at all discussed the paper by Karim et al., leaves us worried that the proposed method does not estimate a causal parameter. We also notice that there are no formal justification made for why this modified approach should properly adjust for time-dependent confounding, other than the fact that it gives results that are closer to what the authors expect in rather arbitrary simulations studies. If the authors have a formal or heuristic reason for why adjusting for posttreatment variables is a good idea, it is not given in this paper. A fair comparison of the performance of two different estimating procedures would typically rely on (1) a good understanding of the parameter being estimated and (2) a flexible simulation framework for simulating data for a range of realistic scenarios satisfying the underlying models of both procedures. In the case of comparing the sequential Cox approach with other methods for time-dependent confounding, we feel that more development is needed on both ends. The sequential Cox approach intuitively estimate a conditional average treatment effect over time, but we acknowledge that the original article does not in depth discuss the causal estimand in question and that this is an area where more work is needed. For relevant references in that regard, we would like to point to the 2017 paper