Another Look at Outcomes and Outcome Measures in Psychiatry: Cui bono?
Authors: Cynthia L. Arfken, Richard Balon
Journal: Psychotherapy and Psychosomatics (2013)

On the Cochrane Collaboration webpage [1], visitors are met with the question: How do you know if one treatment will work better than another, or if it will do more harm than good? This question is at the core of clinical care and patient-oriented research in all fields. The question applies equally to psychiatry as it does to other fields of medicine. The approach to answering this question was supplied by Yusuf et al. [2] almost 30 years ago through the advice of 'ask an important question' and 'answer it reliably'. To their advice we add asking Cicero's famous question, 'Cui bono?' - Who really benefits from answers to this question currently and who should benefit (as well)? In this commentary we advocate using simpler measures and more ecologically valid designs to answer this central question both reliably and understandably for patients and their families.Contrary to the many contemporary articles focused on 'reliability', Yusuf et al. [2] did not write on the psychometric properties of scales or even different ways to measure their validity or reliability. They were addressing the fundamental need to answer what treatment will work better than another. Having valid and reliable scales is important, and we are not arguing against their use. However, the scales commonly used in psychiatry are difficult to understand by patients, families and policymakers. Most measures used in psychiatry (probably anything beyond the Clinical Global Impression scale) do not provide simple and understandable answers for the public. We clearly need to develop outcomes that are understandable to patients, families and policymakers. In their article, Yusuf et al. [2] were advocating a new approach, large simple randomized trials, as a way to answer the question 'reliably' with the emphasis on simple trials. Their rationale for advocating this approach was based upon several lines of reasoning, worth repeating here. The first line of reasoning was that common diseases are important from a societal and health care perspective. Psychiatry encompasses common diseases impacting patients in both highly developed and less-developed countries, of all cultures and all socioeconomic classes. Therefore, as a field we need to have large, more ecologically valid and more informative randomized trials to determine which treatment will work better than another and if it will do more harm than good.The second line of reasoning was that treatments which are efficacious and important will be widely practiced (i.e. not restricted to specialized centers or specialists). This criterion may at first only appear to apply to primary care. However, the process of identification and treatment or referrals of psychiatric disorders by primary care practitioners can, and by some recommendations [3] should be, widely practiced. Problematic alcohol use and, to a lesser extent, drug misuse are examples of these approaches. Identification and treatment or referral for depression and posttraumatic stress disorder are other examples. For trials examining alcohol and drug referrals, the outcomes of use, misuse or abuse need to be included. Nevertheless, other measures such as patient functioning (e.g. work), quality of life, and health care utilization should be measured. However, even the assessment of a seemingly simple issue such as functioning in schizophrenia is complex, with a plethora of scales available and a lack of information regarding the correlation between tools [4].Primary care practitioners are also dispensing medications to people with psychiatric disorders, making medications even more widely used. As pointed out by Ioannidis [5], 5 of the top 24 medications ranked by annual global sales are used to treat mental-health illnesses. Unfortunately, trials examining if the medication works and its relative benefit are usually limited by the low number of participants and the short duration of treatment. The short-duration treatment trial model based on the acute-disease model has recently been criticized by Fava et al. [6] as not reflecting clinical reality. In addition, the treatments are evaluated using changes on valid and reliable symptom rating scales. As stated above, results obtained by using these scales can be difficult for patients and their families to interpret; they can also be difficult for community-based psychiatrists to interpret. The information needed by patients, families and policymakers on long-term use and comparative benefit is not available. A decrease in rating on a suicidal ideation rating scale during an antidepressant trial does not provide very meaningful information to patients and their families. Showing them that antidepressants decrease the suicide rate, and thus save lives, is much more meaningful. Another example of using outcome measures not very meaningful for patients and their families is the use of some rating scales in efficacy trials of antipsychotics in schizophrenia. Some studies used a 20% decrease on psychopathology rating scales as a measure of efficacy. How meaningful and useful it is to the patient, or a family member, or even to most treating physicians?The third line of reasoning was that the important outcome is mortality. In psychiatry, mortality is uncommon in medication trials, but levels are elevated among our patients, especially those with chronic mental illnesses. Unfortunately, we lack information on the impact of our treatments on all-cause mortality, as well as on cause-specific mortality such as suicides and metabolic syndrome-related mortality. Our clinical trials are usually fairly short, and thus mortality due to a negative impact of medication such as metabolic syndrome is not going to be observed. More ecologically valid trials, defined as a simple protocol mimicking routine clinical practice and eligibility criteria, and trials of longer duration with mortality as the outcome would provide this knowledge. Ioannidis [5] suggested that the so-called 'blockbuster' drugs should be examined in megatrials of longer duration. He pointed out that one outcome that should routinely be collected is death. He also suggested that the megatrials 'would provide sufficient power to address the effects of drug interventions on important outcomes such as suicide attempts, hospitalizations, and job loss instead of relying solely on subjective scales and also would help define the spectrum of disease severity at which these treatments are effective'. Such calls support efforts to define effective staging of mental illnesses as proposed by Cosci and Fava [7] and to determine which treatment should be used at which stage of illness. Ioannidis [5] also suggested that these megatrials would help to resolve debates about various risks and mortality that are still unsettled after many conflicting meta-analyses. One of the barriers to conducting such research is cost. Ioannidis [5] identified possible funding and a mechanism for conducting these megatrials. We suggest that using simple measures will help decrease the cost of these trials (as compared to small trials using multiple complicated measures).These large, more ecologically valid and informative trials for medications already approved for use with known safety profiles are important and should be conducted. Such trials are overdue and would provide much-needed information to guide practice and policy in psychiatry and other fields of medicine. We owe a debt of gratitude to Yusuf et al. [2] and Ioannidis [5] for advocating these trials. But we do not need to wait until medications are widely used to advocate for trials to include practical and easily understandable outcome measures. In other clinical fields, biomarkers are used as outcomes of clinical trials. Psychiatry does not, as of this writing, have biomarkers. However, even if psychiatry had biomarkers, that would not make them understandable to patients, families and policymakers. In both cases, biomarkers and symptom scales are used as reliable intermediate outcomes to provide power to smaller and shorter-duration clinical trials. There is an underlying assumption that intermediate outcomes of symptom reduction (e.g. blood pressure reduction in cardiology) in time will lead to higher function, better quality of life and lower mortality rates for patients. Thus, the measures are correlated, yet they provide different information to practitioners and patients. Having smaller and shorter-duration clinical trials is important to 'quickly' obtain some measure of treatment efficacy and relative rates of common adverse events. It can furthermore be argued that symptom reduction - compared to biomarkers - measures a domain important to patients. However, symptom reduction by itself should not be the only outcome of interest.The outcomes should capture the patients', the families' and societal perspectives. Knowing on whom to use a treatment and when requires measures for the practitioner - as well as when in the staging of the disease it should be used. For the patients, families and society, we want to know the impact on mortality, functioning, health care utilization and quality of life. Practitioners need this information too. Additionally, practitioners need clinical measures that capture clinical signs and symptoms used to measure the progression and severity of diseases. Some trials have started to use outcome measures such as change of weight, glucose levels and lipid metabolism or the use of medications for side effects of antipsychotics. Tomba and Bech [8], in their discussion of clinimetrics and clinical psychometrics, suggested using prognostic markers (when possible - these could include stage of illness), assessment of comorbidity, longitudinal assessment (e.g. existence of residual symptoms, also related to prognosis), definition of recovery, assessment of stressor and allostatic load (e.g. using the Psychiatric Epidemiology Research Interview for Life Events), assessment of lifestyle, assessment of patient's well-being and illness behavior. All these possibly used outcome measures seem to have more meaning to patients, their providers, families and society, and they may be useful in planning further treatment.In trials of shorter duration, there may not be any impact on mortality, functioning, health care utilization or quality detected. In these cases, systematic reviews aggregating the different studies are needed to evaluate the relative benefit and harm of a treatment. To conduct these systematic reviews, common outcome measures must be collected and reported. One summary in the Cochrane Library states that 'despite the fact that 7,110 people participated in fifteen randomised aripiprazole studies, we were unable to extract any usable data on death, service outcomes, general functioning, behaviour, engagement with services, satisfaction with treatment; economic outcomes or cognitive functioning' [9]. This statement illustrates the problems outlined of present day studies, their designs and the outcome measures used. A few key outcome measures could provide that needed information, at least for the short duration of most medication trials. Currently, we lack large simple trials for widely used medications and relevant data on other medications!Large, more ecologically valid and informative trials provide lessons for smaller trials. The large simple trials emphasize limited data collection both during the active component of the trial and during the follow-up period. The limited data collection minimizes the demands on the participants, which in turn minimizes missing data and attrition. Other sources can be used for follow-up. For example, mortality follow-up can be conducted through vital statistics without the need to contact each participant. Health care utilization, such as inpatient admission and 30-day readmission, can be ascertained from administrative databases. One example of using meaningful outcomes was the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study [10]. For this trial, the investigators chose changing treatment and duration to discontinuation to compare the effectiveness of antipsychotics. This measure integrates patient preference, adverse events and lack of efficacy without requiring extra visits to measure the outcome. The selected outcome also has the advantage of few missing data. When needed, patient-specific data can be collected from participants at visits spaced to mimic clinical practice and/or via cell phones. The Sequenced Treatment Alternatives to Relieve Depression (STAR*D) used phone interviews to simplify data collection and lower demands on patients [11]. A minimalist design of few outcome measures can be supplemented with measures needed to address the big clinical questions through systematic reviews. These measures may even change over time as we learn more [12].In conclusion, the complexity of outcome measures used in psychiatry has allowed smaller trials to flourish, but at the expense of addressing the bigger question of whether the treatment works, and if so, for whom. Their complexity coupled with a lack of consensus on their use would appear to answer the 'cui bono' question fairly simply: that it is the researchers who benefit primarily. It is time to bring mortality, health care utilization, functioning, quality of life and other simple and meaningful measures back into treatment trials of mental illness. It seems obvious that large trials with simple or simpler measures are the way to go.External funding support for Dr. Arfken during the past three years includes that provided by the National Institutes of Health (NIH), local government agencies and foundations (Robert Wood Johnson Foundation and Institute for Social Policy and Understanding). There was no external funding support for Dr. Balon during the past three years.No conflicts of interest for the authors exist.