Defense Health Care: Expanded Use of Quality Measures Could Enhance Oversight of Provider Performance
Released: 2018-09-17T00:00:00

Letter

Background

Health Care Quality Measures

As we have previously reported in reviews of health care quality outside of the MHS, health care quality measures are standard, evidence-based metrics designed to assess the extent to which patients receive health care that increases the likelihood of desired health outcomes and are consistent with current professional knowledge. These measures may be used to assess the quality of care in various settings, including hospitals and physician offices. Health care quality measures are intended to (1) inform providers about opportunities for potential improvements in their delivery of care, (2) encourage or incentivize providers to consistently provide high quality care, and (3) inform consumers about which providers are most likely to deliver high quality care. There are broad categories of clinical quality measures that address various aspects of quality of care. See table 1 for a description of these broad categories of quality measures.

The data used to calculate the results of health care quality measures can come from a number of different sources. Some measures often require detailed clinical information obtained from patient medical records, such as process measures that indicate whether timely and effective care was provided in a specific situation, for example, or whether stroke patients received clot-dissolving medication appropriately. Other measures are designed to use information on patient demographics and diagnoses that can be obtained from more readily accessible sources, such as claims data or other administrative data that have already been collected for other purposes such as billing. In addition, patients can be asked directly, usually through surveys, to report on their experiences receiving care.

The MHS Structure and Administration of Direct and Purchased Care

The MHS is a complex organization in which responsibility for the delivery of health care is primarily shared among the military services—Army, Navy, and Air Force—and the Defense Health Agency (DHA). The Army, Navy and Air Force medical commands report through their service chiefs to their respective military department Secretaries and then to the Secretary of Defense. DHA reports through the Office of the Assistant Secretary of Defense for Health Affairs and the Under Secretary of Defense for Personnel and Readiness to the Secretary of Defense. The Office of the Assistant Secretary of Defense for Health Affairs manages the Defense Health Program appropriation, which funds the medical and health care programs at the medical commands of the military services. As of fiscal year 2018, most of the MTFs, including military hospitals and clinics, were under the direction and control of the military services, which are responsible for staffing, training, and equipping those MTFs to meet mission requirements. DHA has responsibility for the managed care support contracts through which the MHS administers its purchased care, and DHA also administers several MTFs in the vicinity of Washington, DC. Figure 1 depicts the MHS organizational structure.

Recently enacted changes will affect the administration of the MTFs in future years. Most notably, DOD will alter administration of the MTFs, shifting responsibility from the military services to DHA. Section 702 of the National Defense Authorization Act for Fiscal Year 2017 (NDAA 2017) directed DOD to give DHA responsibility for the administration of all MTFs, including budgetary matters, information technology, and health care administration and management. In the conference report for NDAA 2017, Congress stated its intention that the creation of a single agency responsible for all MTFs would improve and sustain readiness, reduce costs, and increase efficiency. DOD has since prepared a series of implementation plans as it works to develop the specific policies and procedures to enable this change to take effect starting October 1, 2018. The most recent plan issued by DOD in June 2018 envisions a 3-year transition to be completed October 1, 2021.

For purchased care, DOD contracts with civilian health care contractors to manage its civilian providers on a regional basis. The primary responsibilities of these managed care support contractors include the following: developing civilian provider networks, which include hospitals and ensuring adequate access to health care; referring and authorizing beneficiaries to receive health care; processing health care claims; educating providers and beneficiaries; and conducting utilization management and quality management programs.

There have been several generations of multi-year contracts since 1996. In July 2016, DOD awarded its fourth generation of managed care support contracts to two regional contractors, and on January 1, 2018, the MHS began health care delivery under these contracts.

Selection of Measures to Assess Quality of Direct and Purchased Care

According to our review of DOD documents, the MHS uses a structured process to select the measures on its dashboards that are used to assess the quality of direct and purchased care. Specifically, DOD documents state that the core direct care measures that are on the Core Dashboard are selected through the MHS’s performance management system called Partnership for Improvement (P4I), which began in 2015. The documents show that proposals for potential quality measures are developed by work groups that focus on different specialized areas, such as maternity care or mental health. These proposals are reviewed and approved by the Steering Committee for P4I, which develops the list of core quality measures for direct care. The Steering Committee then presents the list of core quality measures to a succession of governance bodies—each of which incorporates representation from the three military services plus DHA—for review and approval. DOD documents indicate that the MHS repeats this process annually as it decides which quality measures to add, drop, or modify for the coming fiscal year from the Core Dashboard.

The DOD documents we reviewed lay out a parallel process that the MHS follows to select which purchased care quality measures will be tracked in the Purchased Care Dashboard. A work group that specializes on purchased care issues with representation of DHA and the three military services develops the proposed list of quality measures for the Purchased Care Dashboard. This list is then reviewed and approved by the same succession of governance bodies that decide on the Core Dashboard measures.

Officials told us and DOD documents confirmed that the MHS and the purchased care contractors also track additional quality measures that are not included in the Core and Purchased Care Dashboards. For example, MHS clinicians who provide maternity care track a set of measures developed by the National Perinatal Information Center. Similarly, a number of military hospitals report on surgical quality measures to the National Surgical Quality Improvement Program. The MHS also conducts surveys of MHS beneficiaries from which it obtains data for patient experience measures for both direct and purchased care. Additionally, the MHS requires the managed care support contractors that administer the MHS’s networks of civilian providers for purchased care to monitor several different sets of quality measures or indicators, many of which focus on patient safety. These include patient safety indicators, hospital acquired conditions, and serious reportable events. They also analyze measures selected from Hospital Compare and the Healthcare Effectiveness Data and Information Set (HEDIS), some of which correspond to measures included in the Core and Purchased Care Dashboards.

The Hospital Compare and Core Quality Measures Collaborative Measure Sets Adopted by Medicare and Private Health Insurers

While health care systems in the United States can use a variety of measures to assess the quality of care, two of the most widely adopted sets of quality measures include the Hospital Compare measure set developed by Centers for Medicare & Medicaid Services (CMS) for inpatient care and the CQMC measure sets jointly developed by CMS and major private health insurers for outpatient care. Since 2005, CMS has collected results for individual hospitals on a specific list of health care quality measures that are posted on a website known as Hospital Compare. CMS does this to make comparable information on the quality of care provided by different hospitals publicly available. Hospital Compare currently covers more than 4,000 hospitals that participate in the Medicare program. These hospitals supply data to CMS for quality measures of inpatient and emergency department care. These data reflect the care provided to all patients treated at these hospitals, not just those covered by Medicare. Each year CMS goes through a formal process, including receiving input from experts and stakeholders, to review and revise the mix of quality measures that these hospitals are expected to report. The purpose of this review, according to CMS, is to ensure that the set of measures reported on Hospital Compare provides meaningful information for quality improvement while reducing unnecessary administrative burden.

Initiated in 2014, the CQMC is a multi-stakeholder voluntary effort focused on quality measure alignment that has developed eight sets of measures for outpatient primary and specialty care, known as the CQMC measure sets. In developing the measure sets, CMS and private health insurers negotiate sets of core measures on which they agree to focus on measuring care quality for certain conditions. Physician specialty societies, employer groups, consumer groups, and regional collaboratives also participate in the negotiations. The CQMC measure sets have been adopted by CMS for Medicare and by 15 major private health insurers for commercial health plans. Additionally, section 728 of the NDAA 2017 directs the MHS to use, to the extent appropriate, these quality measures to assess the quality of direct and purchased care. CQMC documents show that the members of the CQMC intend to continually update these core measure sets as more meaningful measures are developed over time. CMS and the private health insurers plan to expand their application of these measures incrementally, as CMS conducts its annual reviews of Medicare’s quality measures and the insurers update or renew their contracts with different providers.

The MHS Does Not Use a Common Set of Quality Measures for Direct and Purchased Care, and the Measures It Uses Assess a Limited Range of Quality Areas

The MHS does not use a common set of measures on its Core and Purchased Care Dashboards to assess the quality of care provided through direct and purchased care. In addition, for both direct and purchased care, the MHS uses measures on its dashboards that assess a more limited range of quality care areas and medical conditions as compared to the Hospital Compare and CQMC measures adopted by Medicare and private health insurers.

The MHS Does Not Use a Common Set of Quality Measures for Direct and Purchased Care

Although the NDAA 2016 directed the MHS to align its quality measures for direct and purchased care, we found that as of March 31, 2018, the MHS used separate sets of measures on the Core and Purchased Care Dashboards to assess the quality of care delivered in direct and purchased care, respectively. To assess the quality of direct care, the MHS tracks 43 measures on its Core Dashboard, and to assess the quality of purchased care, the MHS tracks 18 measures on its Purchased Care Dashboard. The MHS tracks 8 measures that are the same for both dashboards, leaving 35 measures tracked only on the Core Dashboard for direct care and 10 measures tracked only on the Purchased Care Dashboard for purchased care. (See fig. 2.)

According to MHS officials, since launching the P4I performance management system in 2015, the MHS has focused on making systematic improvements in the quality of care across the MTFs in direct care. As a result, the 43 measures they have chosen for the Core Dashboard reflect their priorities for quality improvement within direct care only. In the case of purchased care, MHS officials stated that requiring civilian providers to report on the same 43 measures that are used on the Core Dashboard for direct care would add burden, and the MHS had concerns that this would make civilian providers less likely to participate in purchased care. Instead, the MHS tracks 18 measures on the Purchased Care Dashboard that rely on information sources other than provider reporting, such as claims that the providers submit in the normal course of receiving payment for their services and surveys that the MHS conducts of its beneficiaries. MHS officials explained that they try to minimize the reporting burden for purchased care providers because for most of these civilian providers, eligible MHS beneficiaries represent only a small proportion of their patient population.

We also found that for direct care, the MHS uses its quality measures on the Core Dashboard to assess the quality of care delivered to beneficiaries served by individual MTFs, such as hospitals or clinics. However, for purchased care, the MHS uses its quality measures on the Purchased Care Dashboard to assess the quality of care delivered to the beneficiary population served by each contractor’s network as a whole– not the quality of care delivered by individual civilian hospitals, clinicians, or other providers in the network. Specifically: In direct care, the MHS uses the 43 measures on the Core Dashboard to track the quality of care delivered by individual MTFs. For example, on a measure of central line-associated bloodstream infections, the MHS tracks the incidence of such infections by individual MTF and by military service (i.e., the incidence of such infections in Army, Navy and Air Force MTFs).

In contrast, in purchased care, the MHS assesses information on the 18 measures on the Purchased Care Dashboard for all beneficiaries in each of the networks administered by the two managed care support contractors. For example, on a measure of the percentage of beneficiaries with diabetes who have their hemoglobin level tested annually, the MHS calculates an overall rate of hemoglobin testing across all the diabetic patients that receive care in each contractor’s network.

The beneficiary population-level reporting on quality measures on the Purchased Care Dashboard reflects the nature of the MHS’s relationship with its managed care support contractors for purchased care. Under the terms of the contracts that the MHS has negotiated with the contractors that administer the networks of civilian providers to care for eligible beneficiaries, the contractors bear responsibility for ensuring the quality of care delivered by those providers. While the MHS requires the managed care support contractors to monitor different sets of quality measures or indicators, such as patient safety indicators, hospital acquired conditions, and serious reportable events to identify possible cases of individual patient harm and determine appropriate interventions, the contractors report this information in annual reports to the MHS for their network as a whole, as opposed to reporting on individual providers.

Because the MHS largely uses separate measures for direct and purchased care on its dashboards and tracks the quality of care delivered by civilian providers in purchased care in the aggregate rather than individually, the MHS lacks the information it needs to make comparable assessments of the quality of care delivered across the MHS as a whole. This, in turn, limits the MHS’s ability to ensure it has the information needed to determine whether it is achieving the department’s overall strategic goals of providing high quality care across the MHS as a whole and ensuring that beneficiaries receive a consistent level of high quality care regardless of whether that care is delivered in direct or purchased care. Moreover, using a different set of quality of measures on the dashboards for direct and purchased care is inconsistent with section 730 of the NDAA 2016, which directs the MHS to align its measures for direct and purchased care so it can reduce performance variation across the MHS. MHS officials acknowledge in principle the value of using aligned measures to assess quality of care in direct and purchased care, but the officials cited a range of factors that pose challenges to achieving this objective, such as the large number of civilian providers and the lack of common health information technology systems.

Based on our review, we found that one way the MHS could have a common set of quality measures for both direct and purchased care, without increasing the reporting burden on civilian providers, would be to use, as appropriate, Hospital Compare and CQMC quality measures. Notably, the MHS states on its website that almost all of the civilian hospitals that are in the contractors’ networks for purchased care already report information on the measures posted on the Hospital Compare website. As a result, there potentially would be no additional burden for these purchased care providers to report information on the Hospital Compare quality measures. Similarly, major health plans report that they have begun implementing the CQMC measure sets in their contracts with physicians, meaning that physicians participating in those plans already report information on CQMC outpatient quality measures. To the extent that those physicians are also in the MHS contractors’ networks for purchased care, the information the physicians report on the CQMC measures could be used by the MHS.

We found the MHS is already using some Hospital Compare and CQMC measures for inpatient and outpatient care. There are a total of 76 measures that Medicare and private health insurers report to Hospital Compare and a total of 60 CQMC outpatient measures. Besides the measures used in the Core and Purchased Care Dashboards, MHS also collects 24 of 76 Hospital Compare measures and 10 of the 60 CQMC outpatient measures. For the most part, these measures are not part of the direct and purchased care dashboards that MHS leadership uses to assess the performance of direct and purchased care. Furthermore, MHS officials told us that they have no specific plans to increase the number of measures that the MHS uses from Hospital Compare for inpatient care delivered in its hospitals. In the case of outpatient care, our review of DOD documents shows that the MHS plans on expanding reporting to only 5 more CQMC quality measures, in large part to minimize its reporting burden.

The MHS Uses Quality Measures that Assess a Limited Range of Quality Care Areas and Medical Conditions Compared to Measures Adopted by Medicare and Private Health Insurers

We found that the measures the MHS uses on its Core and Purchased Care Dashboards to assess the quality of direct and purchased care address only a limited range of quality areas and medical conditions when compared with the Hospital Compare and CQMC measure sets that are adopted by Medicare and private health insurers. According to the National Quality Forum, which plays a central role in developing and annually reassessing the Hospital Compare measure set and also was consulted in the development of the CQMC measure sets, the measures used to assess quality of care should comprise an appropriate mix of recognized measure types, including outcome measures, process measures, experience of care measures, and cost and structure measures. These measures should cover a broad enough range of measure types and medical conditions so that they provide an accurate overall assessment of the quality of care patients receive.

Based on our analysis, Table 2 below shows the limited range of measures on the Core and Purchased Care dashboards used by the MHS to assess inpatient care, as compared to the range of inpatient measures that Medicare hospitals report for Hospital Compare. In general, each of the five types of measures shown in the table below addresses different aspects of health care quality in hospital settings. For direct care, the MHS uses no more than one measure on its Core Dashboard for all of these five measure types except for “Outcome” measures; for purchased care, the MHS does not use any inpatient care measures on its Purchased Care Dashboard.

Similarly, based on our analysis, Table 3 below shows the limited range of measures on the Core and Purchased Care Dashboards used by the MHS to assess outpatient care, as compared to the range of outpatient measures that are part of the CQMC measure sets adopted by Medicare and private health insurers. The MHS uses measures on its dashboards that assess fewer clinical focus areas and medical conditions as compared with those measures included in the CQMC measure sets. As with hospital care, the difference is greatest with respect to purchased care.

The limitations we found in the quality measures used by the MHS—the relatively narrow range of measures as well as the relatively few measures used across direct and purchased care—reflect the MHS’s priorities in selecting quality measures. In short, the MHS focuses on the value and impact of implementing individual measures, but does not prioritize aligning the measures used across direct and purchased care or expanding the range of medical conditions and quality areas covered in the aggregate by the measures. The MHS’s annual assessment of quality measures focuses only the Core Dashboard measures. For each Core Dashboard measure for which a change is under consideration—such as dropping, modifying, or adding another quality measure to the Core Dashboard—MHS officials apply a standard set of criteria involving both the feasibility of collecting the data needed for that measure and the utility of that measure for addressing a strategic priority or promoting performance improvement. When asked about the potential value of increasing the number of Hospital Compare measures, MHS officials said they need to make a value-based determination of whether the benefits of obtaining results for any given Hospital Compare measure justified the costs of collecting and transmitting the data required for that measure. In discussions about potential measures for the Purchased Care Dashboard, MHS officials also focused on the characteristics of specific measures being considered for inclusion in the dashboard.

Because the MHS does not prioritize expanding the range of medical conditions and quality areas covered by common measures across direct and purchased care, the measures the MHS uses provide DOD’s senior health care leadership with an incomplete picture of the quality of care across the MHS. As we have noted, the MHS has reported to the Congress that its DOD health care leaders rely on the Core and Purchased Care Dashboard measures to establish accountability throughout the MHS and identify areas where quality improvement is needed. However, the current approach may not lead to the selection of quality measures for the two dashboards that would enable MHS officials to identify the most critical quality of care issues in the MHS. The lack of that information, in turn, limits the ability of DOD’s senior health care leadership to target their performance improvement efforts most effectively in support of DOD’s overall strategic goals of providing high quality care across the MHS as a whole.

The MHS Has Established Performance Standards and Related Corrective Action Requirements for Individual Providers in Direct Care but Not in Purchased Care

The MHS has established performance standards in direct care related to the Core Dashboard measures and has corrective action requirements for MTFs that do not meet the standards. However, the MHS has not established performance standards related to the Purchased Care Dashboard measures for individual civilian providers in purchased care and therefore does not have related corrective action requirements for these providers.

The MHS Has Established Direct Care Performance Standards and Related Corrective Action Requirements for Military Treatment Facilities

As part of its P4I performance management system for direct care, the MHS has established specific performance standards that each MTF must meet in delivering quality care to MHS beneficiaries. These standards—some of which are under development—specify a minimum level of performance that each MTF should achieve related to the Core Dashboard quality measures tracked in direct care. For example, in the case of the HEDIS All Cause Readmission measure on the Core Dashboard, the MHS’s performance standard is that MTFs should have a rate of unplanned acute readmissions within 30 days of an initial hospital admission that is as good as or better than the national 75th percentile. This performance standard is based on the readmission rates that the National Committee for Quality Assurance, the lead entity for that measure, has observed across U.S. hospitals.

During regularly recurring governance meetings throughout the year, MHS governance bodies review how MTFs have performed relative to the performance standards for the Core Dashboard measures. Our review found that during these meetings, the governance bodies generally do not examine the circumstances of MTFs that do not perform well on the performance standards related to the Core Dashboard measures. Consequently, DOD’s senior health care leadership within the governance bodies may receive limited information on the challenges faced by low-performing individual MTFs. However, during these meetings, officials from the military services and DHA highlight MTFs that are performing well on the established performance standards, and the officials share best practices and specific strategies used to achieve high performance.

We also found that in direct care, the MHS requires MTFs that do not meet the MHS’s performance standards related to its Core Dashboard measures to take corrective actions to improve the quality of care they deliver. The military services—Army, Navy and Air Force—and DHA have been responsible for implementing this requirement. For example, Navy officials explained that they periodically review information collected on the MHS’s Core Dashboard quality measures to analyze areas where MTFs do not meet established performance standards tied to these measures and to oversee MTFs’ efforts to correct these deficiencies. Officials told us that each of the services exercises its discretion to independently develop and implement the corrective actions that the service determines best address the performance issues identified through the use of the MHS’s quality measures. For example, to help reduce the number of Central Line-Associated Bloodstream Infections (CLABSI), the Army began financially awarding MTFs that performed well on the CLABSI measure, whereas the Air Force developed a toolkit to help providers prevent CLABSI.

As the MHS moves to transfer administration of the MTFs from the individual military services to DHA as directed by section 702 of the NDAA 2017, the approach for assessing performance and implementing corrective actions is likely to change. The MHS’s recently issued implementation plan as of June 2018 outlines some alterations to the current performance assessment process. Specifically, MTFs will create and submit a performance plan that will be reviewed and approved by DHA. DHA will host monthly review sessions with MTFs to track performance on the plan. MTFs will be evaluated using a set of measures aligned to the Quadruple Aim that will include many but not all of the Core Dashboard measures.

The MHS Has Not Established Performance Standards and Related Corrective Action Requirements for Individual Civilian Providers in Purchased Care

The MHS has not established performance standards related to the 18 Purchased Care Dashboard measures for individual civilian hospitals, clinicians, or other providers in purchased care. Instead, the MHS has established performance standards related to the 18 Purchased Care Dashboard measures that MHS officials use to track the performance of each of the two managed care support contractors. According to MHS officials, the MHS does not require the contractors to ensure that each individual hospital, physician, or other provider in these networks meets the performance standards related to the Purchased Care Dashboard measures. For example, in the case of a measure on the use of imaging for low back pain, the MHS has set a performance standard for each managed care support contractor, one that aims at avoiding excessive imaging across the beneficiary population in the contractor’s network. However, officials told us that the information that the MHS collects on the measure—the number of beneficiaries in each of the contractors’ networks who receive imaging services for low back pain—does not indicate the extent to which each individual civilian provider in the contractor networks meets or fails to meet the performance standard. Thus, the information the MHS obtains on the quality measure and its related performance standard does not identify which hospitals, clinicians, or other providers need to improve their performance in order for all beneficiaries to receive the expected level of care quality that the performance standard represents.

Because the MHS has not established performance standards related to the Purchased Care dashboard measures for individual civilian hospitals, clinicians, or other providers in purchased care, there are no related requirements for corrective action. Instead, the MHS requires its managed care support contractors to undertake other activities to promote improved quality of care across civilian providers in their networks. These include investigations of quality issues, focused reviews, analyses of Hospital Compare data, and value-based purchasing pilots, as discussed further below. However, our review found that these efforts are not applied comprehensively across all individual purchased care providers.

Investigations of Quality Issues. One approach the MHS uses to promote improved quality of care across purchased care providers is to direct its managed care support contractors to investigate whether individual beneficiaries have experienced what the MHS refers to as a quality issue. Potential quality issues are defined by the MHS as any instance when there are indications that a purchased care provider has deviated from what the managed care support contractors deem acceptable standards of professional practice. The contractors can identify these potential quality issues through beneficiary complaints; analyses of patient safety indicators, hospital acquired conditions, and serious reportable events; or by the MHS or contractor staff. Once potential quality issues are identified, they are investigated by a clinician, who reviews the patient’s complete medical record. Based on the clinician’s review of the patient’s medical records, the clinician verifies whether or not a quality issue has occurred and, if so, assigns the quality issue a severity level. To address the quality issue, the managed care support contractors may take a range of steps, including educating the provider, monitoring the provider, notifying the appropriate state or federal bodies, and removing the provider from the MHS’s purchased care provider network. In practice, however, MHS officials said and documents we reviewed show that providers are rarely removed from the network. For example, MHS officials reported that one contractor estimated that one provider was removed from its network over quality issues every 1 to 2 years.

Focused Reviews. Another way the MHS uses its managed care support contractors to promote improved quality of care across purchased care providers is through focused reviews. During these reviews, the managed care support contractors review the medical records for a selected patient population to determine the extent to which a specified quality concern is a widespread problem. For example, in 2015 one contractor reviewed the medical records of 96 beneficiaries to determine the frequency of obstetric trauma, an injury related to vaginal deliveries. If a focused review determines that there is a widespread quality problem, the contractor may implement a quality improvement initiative designed to prompt all of its network providers to address that concern, as opposed to targeting specific providers.

Analyses of Hospital Compare Data. The MHS also requires the contractors to conduct an annual examination of the performance of hospitals in their networks on the different quality measures reported on Medicare’s Hospital Compare. However, the managed care support contractors have considerable flexibility in deciding how to structure these analyses and how to follow-up on results. Consequently, the two managed care support contractors have adopted different analytical approaches to define and identify hospitals with relatively low performance. For example, the managed care support contractors chose to examine different quality measures and use different criteria to identify hospitals with relatively low performance. In their most recent annual reports issued during 2017, both managed care support contractors indicated that they were considering contacting the lower performing hospitals to prompt remedial action, but because no action had yet occurred, the reports leave open what steps were ultimately taken and how these hospitals responded. Nonetheless, these activities suggest that the managed care support contractors have the ability to use Hospital Compare to analyze and address individual provider performance on a standard set of quality measures. However, the MHS has not specified how this process should proceed, leaving it to the managed care support contractors to decide what and how much they will do in conducting these analyses of individual hospitals.

Value-Based Purchasing Pilots. The MHS has recently begun to test different approaches to incentivize purchased care providers to deliver high quality care through several value-based purchasing pilots. For example, in February 2018 the MHS launched a maternity care pilot that pays providers more for better performance on specified quality measures. The pilot also implements a ‘steerage model’ approach that identifies higher performing providers in directories provided to patients by indicating providers as “Gold Stork” or “Silver Stork.” These pilots may provide the MHS another way to influence the quality of care provided by certain subsets of its purchased care providers. MHS officials stated that although DOD has not arrived at specific goals, it plans to expand these pilots to cover around 20 to 25 percent of its purchased care services by 2020.

The use of performance standards and corrective action requirements for individual hospitals, clinicians, or other providers who serve MHS beneficiaries is consistent with federal internal control standards for monitoring, which state that management should establish monitoring activities, evaluate the results, and remediate any deficiencies. While the MHS has established performance standards related to its Core Dashboard measures in direct care and has corrective action requirements for MTFs that do not meet those standards, it has not done so for individual civilian hospitals, clinicians, or other providers in purchased care related to its Purchased Care Dashboard measures. Additionally, if the MHS aligned quality measures on the Core and Purchased Care Dashboards at the provider level, the MHS could require its managed care support contractors to monitor the performance of individual civilian providers relative to set performance standards comparable to the ones that the MHS has established for MTFs. This approach would allow the MHS to determine the extent of performance variability, both among individual civilian providers and across MTFs and individual civilian providers. By not establishing consistent performance standards at the provider-level for direct and purchased care and requiring corrective action requirements to ensure that these standards are met by providers in both direct and purchased care, the MHS is limited in its ability to address variation in the quality of care delivered. This further limits the MHS’s ability to ensure that it is achieving the department’s overall strategic goals of providing high quality care across the MHS as whole and ensuring that beneficiaries receive a consistent level of high quality care regardless of whether that care is delivered in direct or purchased care.

Conclusions

Congress directed DOD to reduce variation in the quality of care beneficiaries receive through the MHS. DOD has taken important steps towards this goal by identifying a set of core measures that DOD senior health care leadership use to assess quality of care in direct care and another set of measures that they use to assess quality in purchased care. DOD health care leaders rely on these measures on their Core and Purchased Care Dashboards to establish accountability throughout the MHS and identify areas where quality improvement is needed. However, with few exceptions, the MHS uses different measures on its Core and Purchased Care Dashboards to assess the quality of direct and purchased care, making it difficult to determine the extent to which it is ensuring consistent quality across the MHS as a whole. Furthermore, for both direct and purchased care, the MHS uses measures on its dashboards that assess a limited range of quality areas and medical conditions when compared to the widely used quality measure sets adopted by Medicare and private insurers. Without using a broader range of available quality measures available—measures that many purchased care providers already report to CMS and private health insurers—DOD is missing an opportunity to better target the most critical quality of care issues in the MHS. The limitations we identified in the MHS’s Core and Purchased Care Dashboard quality measures reflect the fact that in its annual measure selection process, the MHS does not prioritize aligning the quality measures across direct and purchased care and expanding the range of measures it uses across the two systems of care.

Finally, our review shows that while DOD has established performance standards for the core measures in direct care and corrective action requirements for MTFs that do not meet these standards, DOD has not done so for individual purchased care providers. Notably, DOD does not set clear expectations that individual purchased care providers should meet the performance standards related to the quality measures on the Purchased Care Dashboard. Performance standards and related corrective action requirements are critical for holding both MTFs and individual civilian providers accountable for providing quality care. Without consistent standards and related corrective action requirements across the MHS, DOD is limited in its ability to ensure that beneficiaries consistently receive high quality care, regardless of whether they receive that care in the direct or purchased care systems.

Recommendations for Executive Action

We are making two recommendations to the Assistant Secretary of Defense for Health Affairs.

As MHS governing bodies conduct their recurring reviews of quality measures selected for MHS’s Core Dashboard and Purchased Care Dashboards, the Assistant Secretary of Defense for Health Affairs should direct those bodies to prioritize, as appropriate, the selection of measures that apply to both direct and purchased care at the provider level and that expand the range of quality measure types and medical conditions that are assessed. (Recommendation 1)

The Assistant Secretary of Defense for Health Affairs should establish, as appropriate, performance standards related to the Purchased Care Dashboard measures that are consistent with the MHS’s performance standards for direct care; ensure they are applied to individual purchased care providers; and take steps, such as amending its managed care support contracts, if necessary, to require corrective actions to be taken when providers do not meet those standards. (Recommendation 2)

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for review, and DOD provided written comments, which are reprinted in appendix I. In its written comments, DOD concurred with both of our recommendations. With regards to the first recommendation, DOD stated that it plans to enhance the process for selecting quality measures that apply to both direct and purchased care; optimize use of data on Hospital Compare to expand the types and medical conditions evaluated; augment their governance reporting structure so that senior leadership can review quality measures included on the Core Dashboard and Purchased Care Dashboard; and implement the CQMC measure sets for outpatient care. Additionally, DOD stated that it has efforts underway to create a library of all quality measures used across direct and purchased care.

With regards to the second recommendation, DOD acknowledged the need to strengthen accountability for meeting performance standards that apply to both direct and purchased care providers. It also agreed that measures of individual provider performance in purchased care should be augmented and consistent with measures in direct care, where possible. DOD noted, however, that because it works through managed care support contractors for purchased care, it can hold the contractors accountable for meeting performance standards but cannot currently take action against individual providers based solely on performance. Instead, DOD stated that rather than taking a corrective action approach, it plans to expand its value-based purchasing efforts and incentivize providers that meet and exceed certain quality standards. This raises concerns, as DOD’s current plans to expand its value-based purchasing efforts would only be applicable for between 20 and 25 percent of the services MHS beneficiaries receive from purchased care providers by 2020, as we noted in our report. Without having all providers managed consistently and subject to prompt remediation of deficiencies, DOD is missing an opportunity to improve the quality of purchased care, and it increases the risk that not all beneficiaries will receive a consistent level of high quality care across the MHS. Acknowledging DOD’s comment that it cannot currently take action against individual providers based solely on performance, we have modified our recommendation to clarify that DOD should take the steps it determines are necessary, such as amending its managed care support contracts, to institute corrective action requirements for purchased care providers.

We are sending copies of this report to the Secretary of Defense and appropriate congressional committees. The report is also available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff has any questions regarding this report, please contact me at (202) 512-7114 or silass@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II.

Appendix I: Comments from the Department of Defense

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Rashmi Agarwal, Assistant Director; Eric Peterson Analyst-in-Charge; Muriel Brown; Shaunessye Curry; Michael Erb; Krister Friday; Jacquelyn Hamilton; and Colbie Holderness made key contributions to this report.

Related GAO Products

Defense Health Reform: Steps Taken to Plan the Transfer of the Administration of the Military Treatment Facilities to the Defense Health Agency, but Work Remains to Finalize the Plan. GAO-17-791R. Washington, D.C.: Sep 29, 2017.

Health Care Quality: HHS Should Set Priorities and Comprehensively Plan Its Efforts to Better Align Health Quality Measures. GAO-17-5. Washington, D.C.: Oct 13, 2016.

VA Health Care Quality: VA Should Improve the Information It Publicly Reports on the Quality of Care at Its Medical Facilities. GAO-17-741. Washington, D.C.: Sep 29, 2017.