Defense Health Care Reform: Actions Needed to Help Realize Potential Cost Savings from Medical Education and Training
Published: 2014-07-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: To help address DOD's escalating health care costs, in 2013 DOD established the DHA to, among other things, combine common medical services such as medical education and training. DOD trains its servicemembers for a wide variety of medical positions, such as physicians, nurses, therapists, and pharmacists. DHA's Education and Training Directorate is to oversee many aspects of DOD's medical education and training and is now expected to begin operations in August 2014. GAO was mandated to review DOD's efforts to consolidate medical education and training.
GAO examined the extent to which DOD has (1) conducted analysis to reform medical education and training to achieve cost savings and (2) determined whether the consolidation of training at METC has resulted in cost savings and designed processes to assess its effectiveness. GAO compared DHA implementation plans and METC budget information from fiscal years 2010 through 2012 with best practices and interviewed officials from the DHA, METC, and military services' Surgeons General offices.
section_title: What GAO Found
paragraphs: In its 2013 plans for the implementation of the Defense Health Agency (DHA), the Department of Defense (DOD) outlined the responsibilities of a new Education and Training Directorate, but has not demonstrated how its proposed reforms will result in cost savings. The National Defense Authorization Act for Fiscal Year 2013 required DOD to develop business case analyses for its shared service proposals as part of its submissions on its plans for the implementation of DHA, including, among other things, the purpose of the shared service and the anticipated cost savings. Although DOD has stated that the Directorate is a shared service that combines common services and that it will result in cost savings, DOD has not fully developed the required business case analysis for the medical education and training reforms. This is because DOD has not yet completed the first step of the process, which includes identifying the specific problems that the reform is intended to address and thereby achieve cost savings. Unlike the medical education and training reforms, other DOD shared service projects present a clear linkage between (1) a stated problem, (2) proposed process changes, and (3) an estimate of benefits, costs, and risks. For the Directorate, DOD has identified the new processes it will employ, but has not identified the concerns the proposed new processes are intended to address and how they will achieve cost savings. In addition, some officials are unconvinced that the potential cost savings will be achieved, and stated that the creation of the Directorate serves more as a functional realignment than a cost savings endeavor. Without a fully developed business case analysis, it is unclear how DOD will measure any accomplishments and hold the Directorate accountable for achieving cost savings.
DOD is unable to determine whether the consolidation of training at the Medical Education and Training Campus (METC) resulted in cost savings; however, DOD is taking action to improve some of the processes for evaluating the effectiveness of training at METC. DOD co-located medical training for enlisted medical servicemembers at METC as part of the 2005 Base Realignment and Closure Commission (BRAC) process to achieve cost savings, and subsequently, the services decided to consolidate their training. However, some officials stated they were unsure whether all funds were transferred to METC. Furthermore, due to a shortage of military service funds, the Office of the Assistant Secretary of Defense for Health Affairs provided funding for METC in addition to the services' transfers. DOD is unable to determine whether the consolidation of training at METC resulted in cost savings because it did not develop baseline cost information as part of its metrics to assess METC's success. Baseline cost information is a key characteristic of performance metrics critical to ensuring that processes achieve the desired results. Without baseline cost information prior to future course consolidation of training at METC and within the Education and Training Directorate, DOD will be unable to assess potential cost savings. DOD has designed processes to evaluate the quality of training at METC—including processes related to certification rates, accreditation, and surveys. Further, DOD has taken action to improve some processes. For example, to improve the level of feedback received from METC surveys, METC officials have begun a pilot process to conduct their own post-graduation surveys.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD conduct a fully developed business case analysis for the Education and Training Directorate and develop baseline cost information as part of its metrics to assess cost savings for future consolidation efforts. In comments to a draft of this report, DOD concurred with each of GAO's recommendations.

section_title: Letter
paragraphs: The Department of Defense (DOD) operates a complex medical training  network that educates both officers and enlisted servicemembers. This  network consists of institutions located across the United States, with  some providing training to participants from all three military services and  others providing training within a single service setting. DOD trains its  officers and enlisted servicemembers for a wide variety of medical  positions (e.g., physicians, dentists, nurses, therapists, and laboratory  technicians) and expended approximately $655 million on such training in  fiscal year 2013. This includes funds appropriated to the Defense Health  Program, which provides support for education and training opportunities  for medical personnel and includes resources for specialized skill training,  formal educational programs outside DOD, and professional development  education programs at various military sponsored schools.
Over the past decade, decisionmakers have recognized the need to  transform the Military Health System (MHS) to better control rapidly  escalating health care costs and take advantage of opportunities to  realize savings. For example, the 2005 Base Realignment and Closure  (BRAC) Commission and DOD in its 2006 Quadrennial Defense Review  called for transforming medical education and training. Specifically, the  2005 BRAC recommended co-locating all (except Aerospace Medicine)  medical basic and specialty training for enlisted servicemembers from  several locations to a single site, known as the Medical Education and  Training Campus (METC), in San Antonio, Texas. In implementing this  BRAC recommendation, the services decided to consolidate courses  where feasible to achieve cost savings and facilitate interservice training,  and METC began training students in 2010 as a tri-service institution. In  April 2012, we reported that DOD had not documented savings resulting  from the creation of METC, but agency officials believed that combining  several training sites into METC had saved money and that other  efficiencies had been achieved. We recommended, and DOD concurred,  that DOD should employ key management practices to show both  financial and nonfinancial achievements from its governance initiatives.  Further, in October 2013, DOD established the Defense Health Agency  (DHA) to, among other things, combine common (“shared”) medical  services with the intent of achieving cost savings. DHA’s Education and  Training Directorate is intended to oversee and manage many aspects of  medical education and training and is scheduled to begin operations in  August 2014. Our recent report on DOD’s efforts to reform the MHS  focused on the establishment of the DHA. Specifically, in November  2013, we found that DOD lacked a thorough explanation of the potential  sources of cost savings in its shared service projects, a plan to monitor  potential implementation costs increases for these projects, and a current  baseline assessment of MHS personnel and final staffing requirements  for the DHA. As a result, we made a number of recommendations to  provide decision makers with more complete information on the planned  implementation, management, and oversight of DOD’s newly created  DHA, and DOD concurred with our recommendations, as discussed later  in this report.
The House Armed Services Committee Report accompanying the  National Defense Authorization Act for Fiscal Year 2014 (H.R. Rep. No.  113-102) mandated us to review DOD’s efforts to consolidate its medical  training and report the results of our review to the congressional defense  committees. This report examines the extent to which DOD has (1)  developed plans and conducted analyses to reform medical education  and training to achieve cost savings and (2) determined whether the  consolidation of training at the Medical Education and Training Campus  has resulted in cost savings and has designed processes to assess its  effectiveness.
To determine the extent to which DOD has developed plans and  conducted analyses to reform medical education and training to achieve  cost savings, we obtained and reviewed documents concerning DOD’s  plans for the implementation of the DHA, including the Education and  Training Directorate’s Concept of Operations, and briefings to internal  stakeholders. We compared DOD’s implementation plans with best  practices from GAO’s Business Process Reengineering and Assessment  Guide and DOD’s shared service analysis process. In addition, we  reviewed the requirement in the National Defense Authorization Act for  Fiscal Year 2013 for DOD to develop business case analyses for its  shared service proposals as part of its submissions on its plans for the  implementation of the DHA. We interviewed each of the service Surgeons  General, and officials from the DHA and METC, concerning their  understanding of the role and objectives of the Education and Training  Directorate.
To determine the extent to which DOD examined whether the  consolidation of training at METC resulted in cost savings, we reviewed  METC’s resource data from fiscal years 2010 through 2012 and best  practices from GAO’s Business Process Reengineering and Assessment   We interviewed officials responsible for the data concerning their  Guide.
GAO/AIMD-10.1.15. quality control processes, and determined that the data were sufficiently  reliable for our purposes. To determine the processes that METC has  designed to assess effectiveness of its training, we obtained  documentation from METC about those processes, including information  on certification rates, internal metrics, accreditation standards, and survey  procedures. We also reviewed the cost-savings and quality-related goals  of BRAC 2005 for creating METC. To obtain context, we interviewed the  Commandant and other officials from METC; each of the service  Surgeons General; officials from the Navy Medicine Education and  Training Command, the U.S. Army Medical Department Center and  School, the Air Force Air Education and Training Command, and the  Health Care Inter-service Training Office concerning issues related to the  establishment of METC.
We conducted this performance audit from November 2013 to July 2014  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Defense Health Agency
paragraphs: The National Defense Authorization Act for Fiscal Year 2013 required that  DOD develop a detailed implementation plan for carrying out its health  care system reform of creating the DHA, and provide the plan to the  congressional defense committees in three separate submissions in fiscal  year 2013. In October 2013, DOD established the DHA to assume  management responsibility for numerous functions of its medical health  care system, support the services in carrying out their medical missions,  manage the military’s health plan, oversee the medical operations within  the National Capital Region, and provide 10 shared services, including  oversight of medical education and training. According to DOD, a “shared  services concept” is a combination of common services performed across  the medical community with the goal of achieving cost savings. The  DHA’s Education and Training Directorate, a shared service, is scheduled  to begin operations in August 2014 and, according to DOD officials, when  operational, will constitute the first instance of oversight of medical  education and training at the Office of the Secretary of Defense level.  While the services establish training requirements, operate their own  service-specific training institutions, and provide manpower to conduct the  training at tri-service institutions, such as METC, the Directorate plans to  provide administrative support; academic review and policy oversight; and  professional development, sustainment, and program management to the  military departments’ medical services, the combatant commands, and  the Joint Staff. See figure 1 below for the organizational chart of the DHA.
section_title: DOD’s Medical Training Network
paragraphs: Medical personnel receive training throughout their careers to develop  and enhance their skills. Examples of the types of medical training they  can receive include  1.  initial training for enlisted servicemembers, which results in a new  2.  sustainment training for enlisted servicemembers, which does not  result in a new occupational classification but refreshes or augments  initial training;  3.  operational or readiness skills training, which provides training to  perform in operational situations throughout the world and includes  such training as burn and trauma care as well as emergency and  Chemical, Biological, Radiological, Nuclear, and Explosive  preparedness; and  4.  executive skills training for enlisted servicemembers, officers, and  civilians, which provides military health care leaders with executive  management and professional administrative skills.
These training courses can be presented in shared or service-specific  settings that involve varying degrees of a consolidated approach to  course curricula, faculty instruction, equipment, and facilities. Figure 2  depicts the locations of this training and whether it is shared (“tri-service”)  or service-specific training.
Four DOD institutions offer medical training to servicemembers from all  three services. These institutions vary in size and subject matter, and  include the following:
Uniformed Services University of the Health Sciences (USUHS):  DOD-funded medical school in Bethesda, Maryland, with a fiscal year  2015 budget estimate of about $146 million. This university provides  medical training to health professionals dedicated to a career as a  physician, dentist, or nurse in DOD or the U.S. Public Health Service.
Medical Education and Training Campus (METC): Provides initial  skills training to most medical enlisted servicemembers in about 50  areas such as pharmacy, laboratory, and dental technology; combat  medics, basic hospital corpsmen, basic medical technicians; and a  number of advanced medical training courses. METC resulted from a  2005 BRAC recommendation to establish a medical education and  training complex that collocated medical enlisted training being  conducted at five different locations by each of the military services  into one location at Fort Sam Houston, Texas. (See fig. 3.)
Since first becoming operational in 2010, METC has created 14 new  consolidated courses while 22 of its courses were consolidated prior  to METC’s creation. METC trains, on average, about 20,000 students  annually and is estimated to cost almost $27 million in fiscal year  2015. See appendix I for a list of courses taught at METC and course  participants.
Defense Medical Readiness Training Institute (DMRTI): Tri-service  organization that is staffed by servicemembers from the Army, the  Navy, and the Air Force as well as Department of the Army civilians  and according to officials, had a $1.4 million budget in fiscal year  2013. This organization offers resident and nonresident joint medical  readiness training courses as well as professional medical programs  that enable military medical personnel, both active duty and reserve,  to better perform a wide range of medical and health support missions  they face throughout the world. Courses include trauma care, burn  care, public health emergency preparedness, humanitarian  assistance, and emergency response to chemical, biological, nuclear,  and other events. During fiscal year 2013, approximately 3,600  students participated in 122 course iterations in 51 different locations.  According to officials, besides providing medical readiness training to  U.S. servicemembers, DMRTI has provided this training to officials in  38 countries at the request of a combatant command.
Joint Medical Executive Skills Institute (JMESI): Tri-service  organization that provides military health care leaders with executive  management skill programs, products, and services that are designed  to enhance their performance as managers and leaders in the military  healthcare environment. The training JMESI provides centers on the  Core Curriculum which is a collection of 35 executive administrative  competencies required of a military hospital commander that tri- service senior leaders are responsible for reviewing and updating  every 3 years. Each year approximately 200 managers graduate from  JMESI’s Healthcare Management Seminar and MHS Capstone  Symposium, and nearly 20,000 students participate in its online,  distance learning program.
subsections: section_title: Service-Specific Medical Training
paragraphs: In addition to tri-service training, each of the services operates its own  education and training entities that provide additional training to their  medical servicemembers. The Army and Navy education and training  entities are constituent commands of the Army Medical Command and  the Bureau of Medicine and Surgery, respectively, which are headed by  Surgeons General. The Air Force education and training entities conduct  a wide variety of training, including nonmedical training, and do not report  directly to the Air Force Surgeon General. These organizations include  the following:
Army Medical Department Center and School (AMEDD C&S): Army  training headquarters located at Fort Sam Houston, Texas. The center  formulates the Army Medical Department’s medical organization,  tactics, doctrine, and equipment. The school educates and trains  Army medical personnel. More specifically, the Academy of Health  Sciences is the “school” and is part vocational institution, part  community college, and part major university. The Academy of Health  Sciences includes 361 programs of instructions, with 41 of them  taught at METC; 2 levels of officer leader development programs; 6  Masters Degree programs; 7 Doctoral Degree programs; 94  professional postgraduate programs; as well as pre-deployment  training within three main centers and a graduate school. First, the  Center for Health Education and Training consists of 10 departments  whose primary mission is to instruct advanced or specialty courses  enhancing and building upon the initial training that enlisted soldiers  receive from METC and officers receive after finishing their basic  courses. Second, the Center for Pre-Deployment Medicine analyzes,  designs, and develops individual pre-deployment training courses and  products and provides professional expertise and pre-deployment  training to increase the technical and tactical abilities of physicians,  nurses, and other healthcare professionals. Third, the Leader Training  Center provides professional education, doctrinal, and individual  leadership training to execute Army missions across a full spectrum of  military operations. Additionally, aviation medicine classes are taught  at the US Army School of Aviation Medicine, in Fort Rucker, Alabama,  and forward surgical teams preparing for overseas deployment go  through training at the Army Trauma Training Center in Miami,  Florida.
Navy Medicine Education and Training Command (NMETC): Consists  of four centers that provide education, training, and support for Navy  medical personnel. The first center is the Navy Medicine Professional  Development Center headquartered in Bethesda, Maryland, which  offers educational programs such as the Naval Postgraduate Dental  School as well as leadership and specialty courses that focus on the  practice and business of military medicine in both the operational and  hospital settings delivered via in-person classes and online. The  second center is the Navy Medicine Training Support Center  headquartered in San Antonio, Texas. It serves as the Navy’s  component command for METC students and instructors to provide  administrative and operational control of Navy personnel assigned to  METC. The third center is the Navy Medicine Operational Training  Center, which is headquartered in Pensacola, Florida, and consists of  six detachments and nine training centers at 14 locations throughout  the country that teach such areas of Navy medicine as undersea,  aviation, expeditionary, special operations, and survival training.  Fourth, another section of the NMETC provides medical education  and training to the reserve components.
Air Force: There is no specific Air Force organization focused  exclusively on medical training. The Air Force Surgeon General  assists Air Force leadership in developing policies, plans, and  programs, establishing requirements, and providing resources to the  Air Force Medical Service, while the Air Force’s Air Education and  Training Command (AETC) and the Air Force Material Command  (AFMC) provide medical training. AETC, which is headquartered at  Joint Base San Antonio—Randolph, Texas, oversees a wide variety of  medical and nonmedical training. AETC is responsible for 114  medical-related courses: 35 initial skills courses conducted mostly at  METC; 73 sustainment or skills progression courses conducted at  METC and other various locations; and 6 medical readiness courses  taught at a military training site near San Antonio, Texas. AFMC,  which is headquartered at Wright-Patterson Air Force Base, Ohio,  includes the Air Force School of Aerospace Medicine (USAFSAM).  USAFSAM is a center for aerospace medical education and training,  and offers a series of courses comprising the initial qualification  training for flight surgeons, including hyperbaric medicine,  occupational medicine, aviation mishap prevention, and other unique  aeromedical issues pertinent to the flight environment. The school  trains 6,000 students annually.
section_title: DOD Has Outlined the Responsibilities of the New Directorate, but Has Not Conducted a Business Case Analysis to Demonstrate How It Will Achieve Cost Savings
paragraphs: DOD has outlined the areas of responsibility for its Education and  Training Directorate, including consolidation and management of a  number of activities currently performed by the services. However, in its  plans, DOD has not demonstrated through a fully developed business  case analysis how creating a shared service for education and training  will result in cost savings.
According to DOD’s third submission to Congress on its plans for the  implementation of the DHA in October 2013, DOD proposed a number of  projects or “product lines” for its shared service Education and Training  Directorate. Specifically, DOD identified three product lines for the  directorate, which involve (1) management of professional development,  sustainment, and related programs, including the METC, the Defense  Medical Readiness and Training Institute, and the Joint Medical Executive  Skills Institute; (2) academic review and policy oversight functions,  including management of online courses and modeling and simulation  programs; and (3) management of academic and administrative support  functions, such as training and conference approval processes. According  to DOD’s second submission to Congress, the overall purpose and core  measure of success for all shared services is the achievement of cost  savings. This focus differentiates the objective of establishing shared  services from the six other objectives outlined in DOD’s plans for the  implementation of the DHA. However, in its plans, DOD has not  demonstrated how its Education and Training Directorate projects will  result in cost savings through a fully developed business case analysis,  including an analysis of benefits, costs, and risks. In its third submission  to Congress on its implementation plans for DHA, DOD presented  estimates of costs and cost savings for two “sub-product lines”  concerning modeling and simulation and online learning. However, these  projects do not represent the core of the directorate’s mission, but rather  a portion of the academic review and policy oversight project. Further,  these projects overlap with DHA’s contracting and information technology  shared services. Specifically, while cost savings for modeling and  simulation are allocated to the Education and Training Directorate,  implementation costs are to be incurred by the DHA contracting shared  service. In addition, the savings for the online learning project are found  within the DHA information technology shared service portfolio. Aside  from these projects, DOD did not present information concerning the cost  savings of its other shared service projects within the Education and  Training Directorate.
GAO’s Business Process Reengineering Assessment Guide states that a  business case begins with (1) measuring performance and identifying  problems in meeting mission goals, which is then addressed through (2)  the development and selection of a new process. As noted above, the  primary stated purpose of the DHA’s shared service projects is to achieve  cost savings. The Guide further states that as a project matures, the  business case should be enlarged and updated to present a full picture of  the benefits, costs, and risks involved in moving to a new process. Such  analysis is to provide a sound basis to proceed with the reengineering  process. DOD’s own process for developing its shared services, outlined  in its second submission on implementation of the DHA, states that after  an assessment of the current state of performance and measures of  effectiveness have been identified, performance improvement and cost  reduction opportunities should be identified. It also states that new  processes and initiatives are to be developed to address these  challenges, along with associated implementation costs. Further, the  National Defense Authorization Act for Fiscal Year 2013 required DOD to  develop business case analyses for its shared service proposals as part  of its submissions on its plans for the implementation of the DHA,  including, among other things, the purpose of the shared service and the  anticipated cost savings.
DOD does not have a fully developed business case analysis for medical  education and training because it has not yet completed the first step of  that analysis, which is to identify specific problems, which, given the  stated purpose of shared services, should be directed toward the  achievement of cost savings. Several of DOD’s other shared service  projects present a clear linkage between (1) a stated problem, (2)  proposed process changes, and (3) an estimate of benefits, costs, and  risks. For example, DOD’s third submission on the implementation of  DHA, states that the pharmacy shared service will address rising costs  due to variation in drug purchasing, staffing, and formulary management  (the problem) through the introduction of MHS-wide standards and  business rules (the new processes), which will result in cost savings.  Similarly, the plan states that the contracting shared services will address  rising costs due to fragmentation in its acquisition strategy (the problem)  through a common approach to acquisition planning, program  management, contract execution, management, and administration (the  new processes). In contrast, DOD listed the new processes the  Directorate will employ, but it did not explain the problem its proposed  new processes will address, and how they will achieve cost savings.
DOD officials stated that they believe that a central problem for the  Directorate to address is unnecessary variation of practice between the  services, and they believe that efficiencies could be generated through  the consolidation of training. However, in its official plans for the  Directorate, DOD has not identified this issue or any other challenge  related to cost savings as the problem its shared service will address.  DOD also lacks the information to assess its current performance to then  identify a problem. Specifically, DOD officials stated that they lack data on  the cost of DOD’s education programs and potential redundancy within its  portfolio of courses, which would allow them to identify a problem and  develop processes to address these challenges. In fact, officials stated  they have identified the need for developing a baseline of current medical  education and training courses and associated spending as a goal for the  Directorate, and therefore have acknowledged the lack of such  information. In addition, some officials cast doubt on the potential cost  savings that could be achieved. Several DOD officials told us that the  creation of the Directorate represents a logical step in the course of  further cooperation among the services in the area of medical training.  However, senior service officials stated that the Directorate was unlikely  to achieve significant savings and that its creation serves more as a  functional realignment than a cost savings endeavor. For example,  officials stated that the Directorate provides an opportunity to assign a  parent agency to METC, JMESI, and DMRTI, which they described as  “orphan” agencies that lack a parent organization. Officials made similar  comments during our 2012 review, in which we found that DOD was not  able to demonstrate potential financial savings from the creation of  METC, but agency officials stated at the time that they believed  combining several training sites into the formation of METC had saved  money and that other efficiencies had been achieved.
GAO-14-49. particular, given that DOD continues to lack an understanding of how the  establishment of the DHA will affect staff levels, its challenges in  identifying cost savings and a clear mission for its education reforms  could result in increases in staff levels without any savings. As we noted  in our reviews of DOD’s plans for the implementation of the DHA,  DOD’s submissions did not include critical information necessary to help  ensure that DOD achieves the goals of its reform of the MHS.  Accordingly, in a recent report, the House Committee on Armed Services  has expressed concern regarding DHA’s staffing requirements, cost  estimates, performance metrics, and medical education and training  shared service.
Without a business case analysis that links (1) a stated problem, (2)  proposed process changes, and (3) an estimate of benefits, costs, and  risks, the role of the Directorate remains ambiguous, and it is unclear how  DOD will measure its accomplishments and hold the Directorate  accountable for achieving cost savings by sharing training and education  services. Without such information, the Directorate also potentially risks  increasing staff levels without achieving any cost savings.
section_title: DOD Cannot Determine Whether the Consolidation of Training at METC Has Resulted in Cost Savings but Is Taking Action to Improve Some Processes for Evaluating Training Effectiveness
paragraphs: DOD established METC as part of the 2005 BRAC process to provide  interservice training for enlisted service members and to achieve cost  savings. However, DOD is unable to determine whether the consolidation  of medical education and training for enlisted personnel at METC has  resulted in cost savings because it did not establish a baseline for  spending on education and training prior to METC’s establishment. METC  has designed processes to assess the effectiveness of its training and is  taking action to improve them.
subsections: section_title: DOD Officials Cannot Demonstrate Whether the Consolidation of Training at METC Has Resulted in Cost Savings
paragraphs: DOD cannot demonstrate whether the consolidation of training at METC  has resulted in cost savings. However, officials stated that while they  could not document cost savings, they believe that the consolidation of  training at METC has led to cost savings because of (1) increased  equipment sharing; (2) personnel reductions; and (3) cost avoidances,  such as those associated with the closure of medical education facilities  that were service-specific. In contrast, officials also identified areas where  the consolidation of training at METC may have resulted in cost increases  because of, for example, (1) the construction of new facilities; (2)  relocation of students to METC; and (3) replacement of personnel within  their organizations who had been transferred to METC.
To fund training at METC, the services transferred funding to a single  METC budget managed by the Air Force over 3 years from fiscal year  2010 through fiscal year 2012. The services continue to fund  compensation for military instructors at METC. Civilian funding was  transferred to the Air Force, and officials told us that this funding is likely  to be transferred to the DHA.
When METC was established, the services transferred funding for their  enlisted medical programs being consolidated at METC into a single  METC budget. However, some officials stated they are unsure whether  the services’ transfers were representative of their true costs for the  transferred programs prior to the creation of METC. Additionally, the  funding transfers from the services were not sufficient to fund training at  METC, and the Office of the Assistant Secretary of Defense for Health  Affairs provided additional funding to cover this shortfall. For instance, of  the total METC budget of $26.6 million in fiscal year 2012, Health Affairs  provided 28 percent; the Air Force, 22 percent; the Army, 36 percent; and  the Navy, 14 percent. Table 1 shows the funding amounts transferred by  each service to fund METC, from fiscal year 2010, the first year in which  the services transferred funds, until fiscal year 2012, when the services  completed a permanent transfer of their funds to METC.
GAO’s Business Process Reengineering Assessment Guide states that  performance measures are a critical part of a comprehensive  implementation process to ensure that a new process is achieving the  desired results. Additionally, through our prior work on performance  metrics, we have identified several important attributes of these  assessment tools, including the need to develop a baseline and trend  data to identify, monitor, and report changes in performance and to help  ensure that performance is viewed in context. By tracking and  developing a performance baseline for all measures, agencies can better  evaluate progress made and whether goals are being achieved, such as  cost savings targets.
DOD did not establish and monitor baseline cost information as part of its  metrics to assess performance to ensure that the establishment of METC  provided costs savings. Officials told us that their focus in establishing  METC was to ensure that DOD met the BRAC recommendation to co- locate enlisted medical training, not to ensure that this consolidation led to  cost savings. However, the METC business plan, developed in  response to the BRAC recommendation, noted that the intent of  establishing METC was to reduce costs while leveraging best practice  training programs of the three services. We found in April 2012 that  DOD was unable to provide documented savings associated with the  establishment of METC. We recommended that DOD employ key  management practices in order to show the financial and nonfinancial  outcomes of its reform efforts, and DOD concurred with our  recommendation. DOD noted that it would employ key management  practices in order to identify those outcomes; however, as of June 2014,  DOD officials have not documented the financial outcome of the  establishment of METC.
DOD justified its request for the 2005 BRAC round in part based on  anticipated savings. For example, DOD submitted to the 2005 BRAC  Commission a recommendation for the consolidation of 26 military  installations operated by individual military services into 12 joint bases to  take advantage of opportunities for efficiencies arising from such  consolidation and elimination of similar support services on bases located  close to one another. However, we found in 2012 that DOD did not have  a plan for achieving cost savings. For example, during our review of  DOD’s effort to implement this BRAC recommendation, joint base officials  provided us with anecdotal examples of efficiencies that had been  achieved at joint bases, but it was unclear whether DOD had achieved  any significant cost savings to date, due in part to weaknesses in such  areas as DOD’s approach to tracking costs and estimated savings.  Specifically, it did not establish quantifiable and measurable  implementation goals for how to achieve cost savings or efficiencies  through joint basing. We recommended that DOD develop and implement  a plan that provides measurable goals linked to achieving savings and  efficiencies at the joint bases and provide guidance to the joint bases that  directs them to identify opportunities for cost savings and efficiencies.
DOD did not concur with our recommendation, and we noted that this  position contradicts DOD’s position that joint basing would realize cost  savings. Similarly, the co-location and consolidation of training at METC  was, in part, premised on the achievement of cost savings, but DOD did  not establish baseline costs as part of its metrics for assessing  performance.
It is now likely not possible to develop baseline cost information for fiscal  year 2009 to determine the extent to which the establishment of METC  resulted in cost savings. However, without developing baseline cost  information before undergoing future course consolidation of training at  METC and within the Education and Training Directorate, DOD will be  unable to accurately assess cost savings in the future.
section_title: DOD Has Designed Processes for Evaluating the Effectiveness of Training and Is Taking Action to Address Accreditation and Survey Response Issues
paragraphs: METC has designed quality assurance processes to provide continuous,  evaluative feedback related to improvements in education and training  support, and is taking action to address issues regarding course  accreditation and the post-graduation survey process.
subsections: section_title: METC Has Established Quality Assurance Processes
paragraphs: Certification Rates: METC monitors the national certification exam  pass rates of its students, both to meet national requirements and to  make comparisons with national averages. According to METC  officials, certification rates are generally higher since the consolidation  of training at METC. Currently, certification rates for seven programs  exceed the national average.
Internal Metrics: According to METC officials, METC regularly  monitors a number of internal metrics, such as attrition, course  repetition, and graduation rates. To manage performance information  for all of their courses, officials produce a monthly snapshot of these  data to track trends in performance over time. Additionally, all of  METC’s courses are to be reviewed through a comprehensive  program review process conducted by the Health Care Interservice  Training Office.ensure, for example, that all service and accreditation requirements  are met; that faculty meet all required qualifications; and that internal  and external surveys are conducted, analyzed, reported, and acted on  according to policy.
This office is to review 30 specific standards to help
Accreditation Standards: METC is institutionally accredited by the  Council on Occupational Education and is officially an affiliated school  within the Community College of the Air Force (CCAF). Most METC  courses are accredited by a relevant external accrediting body, such  as the American Council on Education (ACE) or the CCAF.
Surveys: The METC Memorandum of Agreement states that METC  and the services will conduct external evaluations to document  program efficacy and to facilitate curriculum review, by gathering  feedback to measure whether the training received was relevant and  to determine whether the graduates are proficient in their job duties.  METC solicits this feedback through surveys sent by the services to  the supervisors of METC graduates at the gaining commands to  gauge satisfaction with the training they received at METC. These  surveys ask such questions as whether the graduates have the  cognitive skills necessary to do their jobs, whether they have met the  entry-level practice requirements of their organizations, and whether  any job tasks should be added to the METC curriculum for their  programs of study.
section_title: METC Is Taking Action to Address Issues Related to Course Accreditation and the Post-Graduate Survey Process
paragraphs: METC officials told us that some training courses were awarded fewer  recommended credits by the ACE than similar service-run courses had  received prior to METC’s consolidation. Officials also stated that the  consolidation of service-run curricula into single programs at METC was  conducted by a contractor, and that these consolidated curricula could be  improved. METC officials further noted that the ACE review of METC’s  consolidated curricula occurred after a change to that body’s process for  recommending credits, and that they are unaware whether the decrease  in the number of recommended credits was due to the consolidated  curricula or changes to ACE’s process. METC officials told us that they  are attempting to improve their programs through their regular process of  curriculum review ahead of future ACE reviews of recommended credits  for their courses.
METC officials also told us that the post-graduate survey process has  been ongoing since before METC was established; however, these  surveys have historically exhibited low response rates. For instance, one  sample survey provided by METC officials had a 14 percent student  response rate and a 0 percent supervisor response rate. To improve the  level of feedback received from these surveys, METC officials have  begun a pilot process to conduct their own post-graduation surveys, using  an online survey program that can be sent directly to the students’ and  supervisors’ personal email addresses. Depending on the success of the  pilot, METC officials plan to extend the process throughout all of METC.
section_title: Conclusions
paragraphs: DHA’s Education and Training Directorate is scheduled to begin  operations in August 2014 to oversee medical education and training  reform, but DOD does not have key information necessary to assess its  progress in realizing the reform effort’s goal of achieving cost savings.  When DOD responded to the 2005 BRAC recommendation to relocate  some medical education and training programs for enlisted  servicemembers at METC, DOD similarly did not have key information  necessary to determine whether the consolidation of training there had  resulted in cost savings. Although DOD’s plans for the implementation of  the DHA acknowledge the benefits of conducting business case analyses,  it has not done so for its medical education and training reforms. DOD’s  inability to demonstrate that cost savings had resulted from the  consolidation of training at METC risks being repeated on a larger scale  in the reform effort of the DHA’s Education and Training Directorate.  Specifically, absent analysis demonstrating how the Directorate’s efforts  will result in cost savings, the creation of the Directorate could increase  costs by increasing staff levels without achieving any cost savings. In  addition, without baseline cost information prior to future course  consolidation of training at METC and within the Education and Training  Directorate, DOD will be unable to assess potential cost savings. The risk  of cost growth also exists for any future consolidations of training at  METC, which could require significant investment of time and resources  without any long-term efficiencies.
section_title: Recommendations for Executive Action
paragraphs: To help realize the reform effort’s goal of achieving cost savings, we  recommend that the Assistant Secretary of Defense for Health Affairs  direct the Director of the DHA to conduct a fully developed business case  analysis for the Education and Training Directorate’s reform effort. In this  analysis the Director should  identify the cost-related problem that it seeks to address by  establishing the Education and Training Directorate, explain how the processes it has identified will address the cost- related problem, and  conduct and document an analysis of benefits, costs, and risks.
To help ensure that DOD has the necessary information to determine the  extent to which cost savings result from any future consolidation of  training within METC or the Education and Training Directorate, we  recommend that Assistant Secretary of Defense for Health Affairs direct  the Director of the DHA to develop baseline cost information as part of its  metrics to assess achievement of cost savings.
section_title: Agency Comments
paragraphs: We provided a draft of this product to DOD for comment. The Acting MHS  Chief Human Capital Officer provided DOD’s comments in an email dated  July 21, 2014. In that email, DOD concurred with the draft report's  findings, conclusions, and recommendations. Additionally, noted in the  email was that Medical Education and Training is the only shared service  that has never had any type of oversight by the Office of the Assistant  Secretary of Defense for Health Affairs or the pre-DHA TRICARE  Management Activity. Further, in that email, DOD noted that that much  credit goes to the sub-working group which has worked numerous hours  over the past 2 years to put this shared service together so the MHS can  realize efficiencies and garner maximum value, exploit best practices  from the services, and achieve standardization where it makes sense.
We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Assistant Secretary of Defense  for Health Affairs; the Director, DHA; and the Surgeons General of the  Army, the Navy, and the Air Force. In addition, the report is available at  no charge on GAO’s website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made major contributions to this report  are included in appendix II.
section_title: Appendix I: Medical Education and Training Campus Courses and the Course Participants
paragraphs: The Medical Education and Training Campus (METC) is the result of the  2005 Base Realignment and Closure (BRAC) Commission legislation that  required the bulk of enlisted medical training in the Army, Air Force, and  the Navy to be co-located at Fort Sam Houston, Texas. As a result, four  major learning institutions for Navy and Air Force relocated to Fort Sam  Houston, where the Army was already training its enlisted medical force  under the Army Medical Department Center & School’s (AMEDD C&S)  Academy of Health Sciences. The Naval School of Health Sciences in  San Diego, California; Naval School of Health Sciences in Portsmouth,  Virginia; Navy Hospital Corps School in Great Lakes, Illinois; and the  882nd Training Group (now the 937th Training Group) at Sheppard Air  Force Base moved to Fort Sam Houston, Texas. METC is now the largest  military medical education and training facility in the world.
METC started operating on June 30, 2010. Its initial training course was  radiography specialist. Other courses were phased in throughout the rest  of the year and into 2011. METC became fully operational on September  15, 2011. The longest program offered is cytology, which is the study of  cells, at 52 weeks; and the shortest, at 4 weeks, is patient administration.  METC offers about 50 medical training programs, which are listed in table  2 along with the course participants.
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Lori Atkinson, Assistant Director;  Rebecca Beale; Jeffrey Heit; Mae Jones; Carol Petersen; Michael Silver;  Adam Smith; and Sabrina Streagle made key contributions to this report.
section_title: Related GAO Products
paragraphs: Military Health System: Sustained Senior Leadership Needed to Fully  Develop Plans for Achieving Cost Savings. GAO-14-396T. Washington,  D.C.: February 26, 2014.
Defense Health Care Reform: Additional Implementation Details Would  Increase Transparency of DOD’s Plans and Enhance Accountability.  GAO-14-49. Washington, D.C.: November 6, 2013.
Defense Health Care: Additional Analysis of Costs and Benefits of  Potential Governance Structures Is Needed. GAO-12-911. Washington,  D.C.: September 26, 2012.
Defense Health Care: Applying Key Management Practices Should Help  Achieve Efficiencies within the Military Health System. GAO-12-224.  Washington, D.C.: April 12, 2012.  2012 Annual Report: Opportunities to Reduce Duplication, Overlap and  Fragmentation, Achieve Savings, and Enhance Revenue.  GAO-12-342SP. Washington, D.C.: February 28, 2012.
Follow-up on 2011 Report, Status of Actions Taken to Reduce  Duplication, Overlap, and Fragmentation, Save Tax Dollars, and Enhance  Revenue. GAO-12-453SP. Washington, D.C.: February 28, 2012.
Opportunities to Reduce Potential Duplication in Government Programs,  Save Tax Dollars, and Enhance Revenue. GAO-11-318SP. Washington,  D.C.: March 1, 2011.
Military Personnel: Enhanced Collaboration and Process Improvements  Needed for Determining Military Treatment Facility Medical Personnel  Requirements. GAO-10-696. Washington, D.C.: July 29, 2010.
Defense Health Care: DOD Needs to Address the Expected Benefits,  Costs, and Risks for Its Newly Approved Medical Command Structure.  GAO-08-122. Washington, D.C.: October 12, 2007.