Counterpoint: were industry-sponsored roflumilast trials appropriate? No.

An appropriate study design is always contextual. Small, uncontrolled studies are useful in ascertaining the toxicity of a novel agent (phase 1) or even whether a drug has clinical activity (phase 2). Randomized trials are needed to confirm the efficacy of therapies, and mega-trials are required to determine mortality effects for blockbuster medications, whose efficacy data only concern subjective end points.1 When it comes to the clinical care of patients, however, an appropriate study must answer only one question: Does this drug improve quality or quantity of life beyond the best available therapy? Industry-sponsored studies of roflumilast do not meet this mark. Roflumilast, a novel phosphodiesterase-4 inhibitor, gained US Food and Drug Administration (FDA) approval for severe COPD in 2012.2 After just 4 months of promotional activity, roflumilast (Daliresp) had been prescribed approximately 70,000 times, with primary care doctors and other health-care providers writing 78% of scripts (specialists wrote 22%).3 The surge in the popularity of roflumilast, however, cannot be attributed to its evidence base. Three papers from The Lancet (containing five randomized controlled trials) are the best studies to date on roflumilast, but all, unfortunately, fail to answer meaningful clinical questions. The earliest study, by Rabe and colleagues,4 published in 2005, assigned 1,411 patients with a diagnosis of COPD for at least 1 year to roflumilast 250 μg, roflumilast 500 μg, or placebo. Besides short-acting β-agonists and short-acting anticholinergics, all other respiratory medicines were stopped, including long-acting β-agonists (LABAs) and inhaled corticosteroids (ICSs). The trial’s authors conclude that the study showed improvements in FEV1 and the number of patients with exacerbations, but no change in quality-of-life scores. Others, however, have questioned the basic statistical analysis5 used by the authors and show that simple (χ2 test) comparisons of the number of patients with exacerbations do not yield significant results. Additionally, new COPD medications should show benefit over current and maximal therapy,6 and the 2005 study did not attempt to do this. Finally, nearly 20% of patients discontinued the study medication, almost double that of the control group, with diarrhea being the most common side effect.7 Thus, given real side effects, a questionable benefit, and a starting point that was unfair (withholding existing therapy), the case for roflumilast was not made in 2005. Accordingly, at this time, the FDA failed to approve the drug.8 By the time the next two pivotal papers were published on roflumilast, there were significant advances in COPD research. In 2007, the Towards a Revolution in COPD Health (TORCH9) study compared salmeterol plus an ICS (fluticasone) against each component alone and placebo in a trial whose primary end point was mortality. In the TORCH trial, patients were allowed to use all other drugs besides the classes of drug being intervened upon (ICSs and LABAs). The TORCH trial found that the combination of salmeterol and fluticasone decreased exacerbations, improved spirometric function, and exhibited a strong trend toward overall mortality benefit (17.5% relative risk reduction) compared with patients taking placebo. In 2008, the Understanding Potential Long-Term Impacts on Function with Tiotropium10 (UPLIFT) trial tested tiotropium against placebo in patients with COPD who were permitted to use all respiratory medications except inhaled anticholinergics. The majority (60%) of these patients used LABAs at baseline. The UPLIFT trial found improvements in lung function, quality of life, exacerbations, and a trend toward improved survival. Against this backdrop, the next two roflumilast papers emerge. The Calverley et al11 study contains two trials and is listed as pivotal in the FDA application of the drug.12 Trial sponsors randomized > 3,000 patients with COPD, bronchitis symptoms, a history of exacerbations, and severe airflow limitation to roflumilast or placebo. Although β-agonists and short-acting anticholinergics were allowed, long-acting muscarinic antagonists (such as tiotropium) and inhaled corticosteroids were not. The trial showed a 17% decrease in the rate of moderate to severe COPD exacerbations (from 1.37 to 1.14 per patient per year), but that was largely driven by a decrease in moderate exacerbations, defined as requiring oral or IV steroids. Severe exacerbations, those leading to admission or death, were not statistically different. In summary, the pivotal roflumilast studies prevented patients from using one class of drugs that is of benefit (long-acting muscarinic antagonists) and halted another that is standard of care (ICSs). Regardless of the results, the trial does not inform real-world practice. If the authors wanted to enroll patients who could not tolerate tiotropium, that would be a separate hypothesis, but requiring patients to stop this drug and ICSs raises serious doubts about the generalizability of the trial. The third high-impact paper on roflumilast13 again contained two randomized trials, which examined > 1,500 patients with moderate to severe COPD based on spirometry. In one study, roflumilast was added to salmeterol. In the other, it was added to tiotropium. The authors note, “no inhaled corticosteroids, short acting anticholinergic drugs, other long acting bronchodilator drugs, theophylline, or other respiratory drugs were allowed after study enrollment.”13 Out of the many end points examined, there were some improvements, but with rules like this, do the results even matter? Whose clinical practice does this trial inform? Thus far, we have ignored the implications of withholding ICSs in the roflumilast trials. Although the data for the use of ICSs have been disputed,14 it is worth highlighting that regardless of its purported efficacy, analysis of randomized controlled trial data suggests that the abrupt discontinuation of ICSs worsens exacerbation.15 Thus, in the roflumilast studies that show some marginal improvement in exacerbations, we might ask whether the drug offers benefit or merely temporizes harm. Altogether, the roflumilast studies enrolled 9,394 adult patients.16 Together, nearly 10,000 patients gave their time and energy to take part in inconclusive medical research—and not just because the results were ambiguous, but because the trials were flawed from the start. COPD has been called a “neglected epidemic,”17 but, if the trend we have examined continues, it may be transformed into a manipulated one. The impact of industry-sponsored studies has been investigated in countless publications. Two systematic reviews18,19 find industry-sponsored studies are more likely to reach pro-industry conclusions than those by nonconflicted bodies. Industry-sponsored studies are more likely to use inactive controls.18 Thus, such analyses use straw man opponents. However, regarding study quality, industry-sponsored studies have not been seriously faulted.18 At least four analyses have found industry-sponsored studies to be of comparable quality to non-industry-sponsored ones.20-23 The roflumilast trials we have discussed may make sense of previous negative findings. Industry-sponsored studies have typically been compared against nonprofit studies20 using validated rating systems. However, such scales are insensitive tests of study quality. The Jadad scale,24 for instance, consists of three questions: Was the study described as randomized? Was the study described as double-blind? Was there a description of withdrawals and dropouts? All of the roflumilast studies we examined achieve top marks by this metric. The studies are nevertheless of limited usefulness, not because they defy basic rules governing randomized controlled trials but by virtue of asking questions of little value and restricting the use of alternative standards of care. The FDA’s approval letter16 for roflumilast reminds the manufacturers of their post-marketing commitment to “conduct a controlled clinical trial to evaluate the efficacy of roflumilast as an add-on therapy to a LABA and inhaled corticosteroid…in the population of COPD patients for which Roflumilast is indicated…[and] appropriate to demonstrate a clinically relevant beneficial effect…”. In a way, this statement is a commentary on the poor evidence base for roflumilast. Although randomized trials examined > 9,000 patients, a basic request to show efficacy is relegated to a post-marketing commitment. The roflumilast studies can be thought of as seeding trials, studies that “appear as if they answer a scientific question but primarily fulfill marketing objectives.”25 We have yet to see a single appropriate trial of this novel agent.