Order of operations in using expanded measurement to promote treatment quality improvement
Authors: Ronald C. Kessler
Journal: World Psychiatry (2018)

Kilbourne et al1 make a compelling case that measurement is required for mental health treatment quality improvement. While I agree that a “balanced portfolio” of structure, process and outcome measures will ultimately be needed for this purpose, I believe that the formidable barriers to quality improvement detailed by the authors make it critical to focus initially on outcome measurement. To support this suggestion, I note that Kilbourne et al's assertion that low mental health treatment quality “is due in part to lack of systematic methods for measuring quality” is not entirely true. Such methods exist for measuring outcomes, but administrative burden is often cited as the reason for not using them. This is not justified. Primary care doctors routinely monitor hypertension and diabetes by measuring blood pressure and blood glucose. It would be less burdensome to assess patient self-reported anxiety, depression and other symptom constellations with existing validated self-report symptom scales. As noted by Kilbourne et al, patient engagement, quality of life and clinical outcomes all increase significantly when this is done. And, importantly, there would be no need for complex infrastructure, as this kind of ongoing outcome monitoring can be done using paper and pencil charting. Why, then, do mental health clinicians not engage in systematic outcome monitoring? The overview of barriers presented by Kilbourne et al mostly apply to process measures (which make up most measures in mandated quality reporting programs). Process measures are difficult to use for quality assurance, especially for non-pharmacologic treatments, because the subtleties of treatment quality cannot be captured in structured measures or administrative data. Clinicians consequently lack enthusiasm for such measures. The situation is somewhat different in the case of pharmacologic treatments, as it is generally possible to abstract from claims data information on prescribing patterns (although often not on whether prescriptions are filled) and frequency of visits. But even here it is of great value also to track self-reported patient symptoms, as such data can help evaluate clinician decisions about medication titration, switching and augmentation. The only barriers discussed by Kilbourne et al that apply to outcome assessment are “administrative burden”, the possibility that “consumers of mental health services might feel burdened by the data gathering”, and the fact that few mental health treatment providers have received training in quality measurement. These are all barriers that can be overcome. With regard to administrative burden, I noted above that primary care physicians routinely engage in monitoring activities that are more administratively burdensome. The notion that patients might feel burdened is similarly unconvincing, given that primary care patients routinely put up with much more onerous tests. In addition, perceived burden would presumably be reduced if clinicians informed patients that systematic symptom tracking leads to improved treatment outcomes, and if checklist responses were used as starting points for more nuanced discussions with patients about recent symptoms. Nor is the fact that many mental health treatment providers have never received training in quality measurement a major barrier. In the case of cognitive behavioural therapy, ongoing patient outcome monitoring is a standard part of the clinical process2. It should not be difficult to convince other clinicians to engage in similar outcome monitoring if relatively simple technological approaches were available to facilitate patient reporting and generation of benchmarked symptom tracking curves. Easy-to-use products to do this are beginning to become available (e.g., http://www.mobiletherapy.com/; http://www.selfecho.com/). I believe a more important barrier to outcome monitoring is one not mentioned by Kilbourne et al: that outcome assessments could be aggregated by payers and used to impose financial constraints (e.g., pay-for-performance) on treatment providers. Although risk adjustment methods exist to address the problem of adverse selection3, uncertainties inherent in this process create reluctance to be held responsible for outcomes on the part of both clinicians and health plans. Yet we know from case studies, most notably the New York State Coronary Artery Bypass Surgery Report-Card System4, that mandated risk-adjusted outcome assessment can be highly effective in improving patient outcomes even in the absence of publically available measures of structure or process. This occurs because unrestricted public reporting of risk-adjusted comparative outcome scores leads to market pressures that encourage quality improvements. Importantly, measures of structure and process are centrally involved in guiding these quality improvements, but these measures tend to be part of something referred to by management consultants as “research to know” (i.e., for internal use to help treatment organizations detect problems they need to fix to improve patient outcomes), whereas risk-adjusted outcome measures are “research to show” (i.e., for public disclosure as ultimate measures of treatment quality). It is critical to appreciate the necessary order of operations and distinct roles of these different kinds of measurement. Publically reported risk-adjusted outcome measures need to come first to create market pressures for quality improvement. Internal structure and process measures come next and are implemented by treatment organizations to improve quality of care in an effort to change scores on publically-available outcome measures. Based on these considerations, I believe that it is critical to focus initially on risk-adjusted outcome measures. The obvious way to do this is for governments or other organized payers to mandate outcome assessments, carry out centralized risk adjustment, and report results publically at as low a level of aggregation as possible. Kilbourne et al mention initiatives along these lines in Australia and the Netherlands. The Access to Psychological Therapies program in England is another example5. The International Consortium of Health Outcomes Measurement is attempting to develop standard sets of outcome measures for this purpose across many different medical conditions, and to create an implementation network to help facilitate data collection and risk adjustment6. When governments and other organized payers are reluctant to embrace outcome assessments, I would suggest that organized consumer groups might fill the gap by creating an electronic system that allows patients to provide information about their conditions and relevant background at the beginning of treatment on a patient-report website, and to complete ongoing symptom tracking surveys on the same site. The baseline data would be used by the system developers for risk adjustment, and the tracking data would be used to generate patient-specific treatment response curves that could be made available to clinicians for monitoring treatment response. Patients would have an incentive to participate to provide information known to promote treatment quality improvement. Once such a system is in place, risk-adjusted treatment quality profiles could be generated to create the market pressures needed to encourage providers to engage in quality improvement initiatives. Once such a system becomes the norm, the “balanced portfolio” of structure, process and outcome measures called for by Kilbourne et al will evolve naturally, with payers using outcome assessments for pay-for-performance, and provider groups using structure and process measures for continuing quality improvement. But order of operations is important. The process needs to begin with risk-adjusted outcome assessments to create market pressures for quality improvement and to provide objective standards for quality assurance, with structure and process measures used primarily by provider organizations for internal purposes to improve patient outcomes. Ronald C. Kessler Department of Health Care Policy, Harvard Medical School, Boston, MA, USA