Detecting meaningful changes in trials of headache treatments: which outcome measure is best?

A major premise of clinical drug trials in headache is that the outcome measures they use are valid and reliable; in other words that they closely and reproducibly gauge the burden of headache and associated symptoms. A third desirable psychometric characteristic is ‘responsiveness’, the ability of a measure to detect clinically meaningful changes in headache symptoms that occur as a result of treatment. These attributes, along with clinical examples and explanations, are listed in Table 1. In this issue of Cephalalgia, Aicher et al. retrospectively analyze data from a large German study to examine the responsiveness of a variety of efficacy measures commonly used in headache research (1). This study was well conducted, and the analyses they present meet the stringent criteria proposed in the COSMIN checklist to evaluate the methodological quality of studies on measurement properties of health status measurement instruments (2). In an open pre-phase portion of the trial, 1734 subjects with tension-type or migraine headaches treated a single headache with their usual non-prescription medication. They were then randomized to one of six medication groups for blinded treatment of a subsequent headache. The groups received either the fixed combination of acetylsalicylic acid, paracetamol and caffeine or the combination without caffeine, the single drugs, or placebo. The outcome measures listed in Table 2 were examined in both trial phases and compared to the subject’s global assessment of drug efficacy, which was selected by the researchers as a ‘gold standard’ measure. This global assessment was obtained within 12 hours of taking study medication by asking subjects ‘How do you assess the efficacy of your tablets?’ Possible answers were 1 (very good); 2 (good); 3 (less good) or 4 (poor). For the purposes of the responsiveness analysis, subjects rating the efficacy of their treatment as 1 or 2 were considered to be ‘satisfied’ and those rating the efficacy of their treatment as 3 or 4 were considered ‘non-satisfied’. This dichotomization allowed the researchers to evaluate responsiveness by generating receiver operator characteristic curves, a method commonly used in studies of diagnostic accuracy. The cut-off points for each measure were chosen to balance sensitivity and specificity. The area under the curve (AUC) for each continuous measure quantifies its ability to discriminate between satisfied and non-satisfied subjects. Put another way, the AUC is an estimate of the probability that a subject who is classified as ‘satisfied’ or ‘non-satisfied’ scored above or below the cut-point, respectively, for that measure. For the noncontinuous measure of ‘pain-free’, only a single measure of sensitivity and specificity can be determined. Results showed that all of the evaluated outcomes were correlated with the subject’s global assessment of efficacy. Subjects who were non-satisfied tended to be those who had little pain relief, while those who were satisfied were more likely to report substantial and more rapid pain relief. Furthermore, when subjects were disaggregated into four groups based on their original efficacy ratings of poor, less good, good and very good, there were step-wise increases in satisfaction that correlated with the magnitude and speed of relief. The AUCs for the different outcomes, however, were remarkably similar within and between the two phases of the trial, ranging from 0.77–0.86 in the prephase and 0.84–0.89 in the treatment phase. In both the pre-phase and treatment phases of the trial, the weighted sum of pain intensity difference (%SPID) performed slightly better than the other end-points. This advantage was slim, though, and the authors wisely do