Difference Between Efficacy and Effectiveness Trials

I n a previous article, 1 I introduced you to a serious but sadly neglected condition affecting many health care professionals, called photonumerophobia (PNP), the fear that one’s fear of numbers will come to light. It was proposed that taking a course that used a specific textbook (mine would, of course, be my preference) would prevent this condition from developing, but not everyone can be crammed into a classroom, meaning that there will always be a number of unfortunate people who experience PNP. Consequently, we must come up with some other form of treatment for this problem, and we have: an experimental drug called Antiphobe. However, to get this medication approved by the powers that be, we must do some studies demonstrating that it actually works, and that means designing a randomized controlled trial (RCT). On paper, at least, an RCT is the essence of simplicity: we recruit a number of people who have this condition and then, as the name of the procedure implies, randomly divide them into 2 groups: one of which receives Antiphobe (the treatment arm) and the other of which gets a placebo (the controls). The major advantage of an RCT is that it tends to minimize any baseline differences between the groups. For example, if we suspect that the response to the drug may be affected by a person’s age or sex, we can always stratify the allocation; that is, we can modify who gets into each group by ensuring that the proportions of males to females and older to younger people are the same in both groups. However, stratification cannot take into account variables we do not know about or that we have not measured, such as genetic differences or family history. Randomization will minimize group differences due to these unknown factors. (At least, that is the theory. It works in practice only if the sample size in each group is infinite, which is clearly difficult to achieve. Even when the sample size is “only” 1000 or so, there may still be baseline imbalances between the groups because of chance.) However, as they say, the devil is in the details. The major one is what question do we want to answer: can the drug work or does the drug work? The technical terms for these 2 types of studies are efficacy trials (can it work under ideal circumstances) and effectiveness trials (does it work in the real world). Efficacy studies are sometimes referred to as explanatory trials; effectiveness studies, as pragmatic trials. The latter term makes sense, as we will see, because the findings of effectiveness studies yield results that can be used pragmatically. However, “explanatory,” in my opinion, makes less sense because the focus is less on explaining how the intervention works and more on simply can it work. Despite which terms are used, the point is that we cannot answer both questions in the same study and, aswe shall see, deciding which question we want to address influences all aspects of the trial: recruitment, how the intervention is delivered, who is counted at the end, what statistics are used, and what the results mean. It is important to bear in mind, though, that efficacy versus effectiveness are the ends of a continuum. In reality, most trials fall somewhere in between these extremes. There is a very useful tool, called PRECIS (which stands for PRagmatic-Explanatory Continuum Indicator Summary), consisting of 10 domains, that can help you determine where on the continuum any given trial sits.