Benefits, harms, and the design of clinical trials.

N one would challenge the notion that benefits and harms must be considered in tandem when physicians discuss treatment options with their patients. Should the concept of harm-benefit tradeoffs be extended to the design of clinical trials, as suggested by Barrett and associates in the current issue of Medical Decision Making? In designing a clinical trial to investigate a treatment that produces severe unpleasant side effects, for example, would there be any point in powering the trial to detect a degree of benefit that most patients would judge to be insufficient—given the severity of the side effects—for them to consider undergoing the treatment? The authors argue that the answer is no, and they present their case so persuasively that I had to ask myself why no one had previously proposed their concept of the “sufficiently important difference,” or SID. They define the SID to be the smallest amount of benefit a treatment would have to offer before a patient would judge the treatment to be worthwhile in view of its risks and costs—the “harm” side of the harm-benefit tradeoff. Although their definition of a harm and their proposed method of deriving SID values may be challenged, I agree with their position that the SID is an inherently sound concept that should not stand or fall on the strength of these particulars. That said, a few caveats are in order. First, using the SID to power a clinical trial would inevitably lead to a smaller sample size requirement than using conventional methods that do not consider harms—that is, methods based on the “minimal important difference,” or MID. The reason is that, unless a treatment incurs no risks or costs, the SID always exceeds the MID, so sample sizes based on the SID will be smaller than those based on the MID. Whether this is an advantage or disadvantage depends on one’s perspective. Sponsors of research might welcome the lower costs associated with smaller trials. Drug manufacturers might balk at the idea that their products would have to satisfy a more stringent standard: including harms in the equation raises the efficacy bar. That treatments in all arms of a clinical trial would face the same efficacy standard, however, ought to alleviate this concern. The research community may have a mixed reaction to the SID concept. On one hand, researchers might welcome smaller sample sizes, given the challenges of recruiting patients for many clinical trials. On the other hand, smaller samples could limit an investigator’s options in the interpretation and use of the results. For example, the authors propose setting the effect size for a clinical trial at or near SID medians derived from patient surveys. In that case, 50% of patients would consider a treatment exceeding the SID to be worthwhile in light of the risks. At the same time, however, the other 50% of patients would consider a smaller amount of benefit to be worthwhile, and the trial might not have sufficient power to detect these smaller effects. Leaving harms out of the picture—that is, using the MID instead of the SID—provides a power cushion capable of accommodating both groups. Second, the authors regard cost as an economic harm that should be considered along with treatment side effects and adverse events when establishing SID values. Although it is true that patients frequently consider out-of-pocket expenses along with other harms in deciding whether a given treatment is “worth it,” attempting to include cost when setting minimal effect sizes for clinical trials may do more harm than good. In the scenarios used by the authors to illustrate their proposed method of eliciting SID values, cost, like the other harms, is presented as a given. The subject is