Trial Emulation and Real-World Evidence.

Increasingly, articles are being published that advocate for the use of observational data to estimate the effects of medical treatments in daily practice. In contrast to evidence from randomized clinical trials (RCTs), observational studies provide evidence that applies to the real world—or so it is claimed. There are indeed various reasons why the results of RCTs may not apply directly to daily clinical practice. Traditionally, ideal conditions are created in RCTs to demonstrate treatment efficacy: strict inclusion and exclusion criteria, masking of participants and researchers, and close monitoring of the safety of participants and their adherence to the treatment protocol. Moreover, participants and their treating physicians in RCTs are explicitly asked to participate (ie, they provide informed consent), which can lead to a further selection. Observational studies, in contrast, are based on what is called real-world data, such as those from electronic health records, and often better represent daily practice. However, due to the absence of randomization and masking (of patients and of physicians), it is always questionable whether the observed results are unbiased. Incomparability of treatment groups (confounding) and (selective) dropout are serious limitations of observational studies.1 Also, the quality of electronic health records data tends to be inferior compared with those collected in RCTs, although there are examples in which they appear to be on par.2 Another difference between RCTs and observational studies is that the former usually estimate intention-to-treat effects, whereas the latter focus on perprotocol effects. Observational studies and RCTs of different adjuvant chemotherapy strategies in patients with stage III colon cancer have found different results. Boyne and colleagues3 investigated whether those differences could be because of methodological issues. They used observational data to mimic an RCT as much as possible, a technique called target trial emulation.4 Target trial emulation is not as simple as it might sound. Less than 20% of participants in the initial cohort in the study by Boyne et al3 were included for analysis. The studied treatment strategies did not exactly match those of the RCT, and the sample size of the observational study (485) was much smaller than that of the RCT (12 834), limiting the power to detect a difference between the effect estimates from the different studies, should it exist. Boyne and colleagues3 tried to separate methodological explanations for differences between observed results, notably those due to immortal time bias. In a naive observational analysis, exposure levels were based on the actual duration of treatment. Because this information becomes available during the study follow-up and therefore is not yet available at study entry, this introduces the risk of immortal time bias; participants have to survive until a certain time to classify as having received a certain exposure level. As a result, those who received the shorter adjuvant chemotherapy treatment regime had a worse prognosis. Immortal time bias is one of many possible sources of bias that become apparent by articulating what a target trial looks like (ie, target trial emulation). Results of the target trial–emulated observational study by Boyne and colleagues3 were indeed in line with those of the RCT, more so than the results of a naive analysis of the observational data. Does this show that target trial emulation yields observational studies that are as credible as RCTs? If results from observational studies concur with those of RCTs, this may suggest that the design and analysis of the former are valid. Unfortunately, this is not true. First, there may be alternative explanations for comparability of treatment effect sizes across different studies (with different designs). These include chance, cancellation of biases, and choices made when analyzing the data (eg, which effect is estimated), to name a few. + Related article