Military Personnel: DOD Needs to Improve Funding Process for Morale, Welfare, and Recreation Programs
Released: 2018-08-08T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: DOD's MWR programs provide servicemembers and their families with three categories of programs: Category A (e.g., fitness and libraries), Category B (e.g., camping and performing arts), and Category C (e.g., golf). DOD oversees the percentage of appropriated funding allocated to MWR programs by category and measures the military services' compliance with established funding targets. DOD set the targets at 85 percent for Category A and 65 percent for Category B. DOD did not set a target for Category C since this category has the ability to generate revenue from user fees.

House Report 115-200 accompanying a bill for the National Defense Authorization Act for Fiscal Year 2018 includes a provision for GAO to review DOD' s MWR programs. GAO assessed the extent to which (1) the services have met DOD's established funding targets for each category of MWR programs and DOD has comprehensively evaluated the relevance of its targets, and (2) DOD has oversight structures and performance measures that include measurable goals, including those for cost-effectiveness, by which to review MWR programs. GAO analyzed MWR program information for fiscal years 2012-2017 and compared DOD's MWR policy with guidance for using measures and evaluating goals.

section_title: What GAO Found
paragraphs: The Department of Defense (DOD) established funding targets for two categories of Morale, Welfare, and Recreation (MWR) programs—Category A, which promotes the physical and mental well-being of servicemembers, and Category B, which funds community support systems for servicemembers and their families. These targets are intended to ensure that the military services adequately fund these programs with appropriated funds instead of requiring servicemembers and their families to pay fees out of pocket to cover program costs. However, GAO found the following:

In fiscal years 2012-2017, the military services generally met the DOD-set target to provide 85 percent of appropriated funding for Category A programs but not the 65-percent target for Category B programs. Service officials said they are taking steps to meet the Category B target, such as by restoring targeted levels of appropriated funding support in future budget planning. Data GAO reviewed indicate that these steps are helping the services get closer to meeting the target for Category B.

DOD has not comprehensively evaluated the targets, established more than 20 years ago, to ensure that they are appropriate. DOD officials said they agree that it is time to evaluate the relevancy of the targets as the current operating environment is fundamentally different than when the targets were established 2 decades ago. Further, DOD officials said that they are unsure of the process or methodology used to originally develop the targets because they have no documentation supporting these decisions. Until DOD comprehensively evaluates the appropriateness of the targets and, based on its evaluation, documents any changes made, it cannot be certain that the targets reflect the current operating environment and do not pose undue financial burden on servicemembers.

DOD established oversight structures and performance measures for MWR programs, but has not established measurable goals to assess the cost-effectiveness of the 55 activities that make up MWR programs. DOD's MWR policy identifies six broad performance measure categories for the program. DOD officials responsible for developing MWR program goals acknowledged that DOD's MWR policy does not include measurable goals for assessing the cost-effectiveness of program activities, and do not currently have plans to make any changes to the goals. Service officials told GAO that they collect and use various types of information within the categories to assess specific activities. While both the categories established by DOD and the service-specific efforts provide useful context about the status of individual MWR activities, they do not replace the need for measurable goals that can be used to assess whether the programs are operating cost-effectively. The services are in the early stages of developing more specific performance measures, but it is too early to determine whether these efforts will result in measurable goals that can be used to assess cost-effectiveness. Until DOD develops performance measures that include measurable goals, it cannot ensure that MWR programs meet servicemember needs in a cost-effective manner.

section_title: What GAO Recommends
paragraphs: GAO recommends that DOD evaluate the funding targets and document any changes needed and develop measurable goals for MWR programs' performance measures. DOD concurred with the recommendations.

section_title: Letter
paragraphs: The Department of Defense’s (DOD) Morale, Welfare, and Recreation  (MWR) programs are a multibillion dollar effort to provide  servicemembers and their families with a wide range of benefits designed  to support military missions and readiness, both in times of war and  peace. In fiscal year 2017, DOD spent approximately $3.9 billion on MWR  programs administered by the Army, the Navy, the Marine Corps, and the  Air Force. DOD’s three categories of MWR programs are mission- sustaining programs promoting the physical and mental well-being of  servicemembers (Category A), community support system programs for  servicemembers and their families (Category B), and recreational  activities for servicemembers and their families that are revenue- generating (Category C). The Office of the Under Secretary of Defense  for Personnel and Readiness (USD(P&R)) oversees the percentage of  appropriated funding provided to MWR programs by category and  measures the services’ compliance with established funding “targets.”  These targets set the minimum level of appropriated funding by category  as a percentage of total expenses compared with funding received from  nonappropriated funding sources, such as fees collected from revenue- generating activities.

House Report 115-200 accompanying a bill for the National Defense  Authorization Act for Fiscal Year 2018 includes a provision for us to  review DOD’s MWR programs. Specifically, the committee expressed  concern that the services have not complied with established DOD policy  to ensure appropriate financial support of family, welfare, and recreational  programs for the past several years, particularly for programs within  Category B, which relies on both appropriated and nonappropriated  funding. For this report, we assessed the extent to which (1) the services  have met DOD’s established funding targets for each category of MWR  programs and DOD has comprehensively evaluated the relevance of its  targets, and (2) DOD has oversight structures and performance measures  that include measurable goals, including those for cost-effectiveness, by  which to review MWR programs.

For our first objective, we assessed the extent to which the Army, the  Navy, the Marine Corps, and the Air Force met DOD’s established  funding targets in fiscal year 2012 through fiscal year 2017. We met with  officials from USD(P&R) and the services to discuss MWR program  policies and procedures and any challenges they have meeting funding  targets as well as any actions they plan to take to meet the targets. We  also reviewed memorandums the services developed showing their plans  to meet the funding targets when they missed them. Based on responses  to data reliability questionnaires from USD(P&R) and the services, we  determined that the data we obtained on DOD’s MWR programs are  sufficiently reliable for the purpose of reporting program information— including costs from fiscal years 2012 through 2017, which are the most  current data available.

For our second objective, we reviewed DOD Instruction 1015.10, Military  Morale, Welfare, and Recreation (MWR) Programs, which specifies  DOD’s oversight roles and responsibilities for the MWR programs, the  goals for those programs, and the information the services should collect  on the programs. We compared this instruction with the Standards for  Internal Control in the Federal Government that states, among other  things, that managers should establish activities to monitor performance  measures. We also compared DOD’s instruction with DOD’s Financial  Management Regulation and with guidance we have identified in our  prior work on performance measurement. This work provides guidance  for using performance measures and evaluating whether goals were met  and for assessing cost-effectiveness. We met with officials from  USD(P&R) and the services to discuss oversight of the program and  processes they have in place for meeting program goals and measuring  cost-effectiveness.

We conducted this performance audit from August 2017 to August 2018  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: DOD’s MWR Program Categories and Funding Sources
paragraphs: DOD Instruction 1015.10, Military Morale, Welfare, and Recreation  (MWR) Programs, establishes policy, assigns responsibilities, and  prescribes procedures for operating and managing programs for military  MWR programs. Specifically, the policy states that the services are to  establish MWR programs in order to maintain individual, family, and  mission readiness and that these programs are an integral part of the  military and its benefits package. The Office of USD(P&R) oversees  DOD’s MWR programs, develops policy, and oversees MWR programs’  funding. DOD’s instruction specifies the purpose of, the funding sources  for, and the activities within each of MWR’s three designated program  categories—all of which are summarized below in table 1. For a complete  listing of the activities by program category, see appendix I.

Each service supports MWR programs with a mix of appropriated and  nonappropriated funding. According to officials, the services allocate  appropriated funding amounts for MWR purposes, which primarily  supports Category A and B programs. Nonappropriated funding is  government money from sources other than amounts appropriated by  Congress and may be generated in a number of ways to support MWR  programs. For example, bowling programs, marinas, and golf programs  generate nonappropriated funding revenue through participation fees for  recreational activities paid by servicemembers and their families. Services  must use any nonappropriated funding generated from or associated with  MWR programs within their MWR programs.

section_title: DOD’s MWR Program Funding Targets
paragraphs: According to DOD Instruction 1015.10, the MWR programs are divided  into three distinct categories, two of which also have specific funding  targets. According to DOD’s 2016 report to Congress on appropriated  funding support for MWR programs, the funding targets are intended to  ensure that the services adequately fund MWR programs instead of  requiring the servicemembers and their families to pay out of their own  pockets for costs that should be borne by appropriated funding. While  DOD Instruction 1015.10 establishes minimum funding targets for MWR  Category A and B programs, it directs that the basic funding target,  regardless of program category, is to use appropriated funding for 100  percent of costs for which they were authorized. While DOD’s Instruction  allows the services to use appropriated funding for 100 percent of  authorized costs, according to service officials this is generally not  possible given budget constraints. Therefore, for MWR Category A  mission sustaining programs, the DOD instruction establishes the funding  target—stating that DOD is to use appropriated funding amounts for a  minimum of 85 percent of total expenditures. For the MWR Category B  community support system programs, the DOD instruction establishes the  funding target as DOD’s use of appropriated funding amounts for a  minimum of 65 percent of total expenditures. For the MWR Category C  recreational activities for servicemembers and their families, appropriated  funding support should generally be limited because this category has the  highest capability of generating nonappropriated funding revenues.

section_title: Budget, Funding, and Accounting Processes for MWR Programs
subsections: section_title: Budget Processes
paragraphs: The services have annual budget processes for MWR programs that vary  based on whether appropriated or nonappropriated funding is being used.  For MWR programs supported by appropriated funding, according to  officials, the services submit and validate program requirements through  DOD’s Planning, Program, Budgeting, and Execution process. DOD and  service guidelines for certain MWR programs as well as annual service- issued budget guidance provide input for determining MWR programs’  requirements. Service officials from the Army, the Marine Corps, and the  Air Force also stated that they determine program requirements using  input from installations and service components, while service officials  from the Navy stated that they use a budget model along with  performance measures and budget guidance to determine program  requirements. The requirements are then submitted to higher level  components within the services for review, adjustment, and approval.  Once the services validate the requirements, they are provided to the  Office of the Secretary of Defense for inclusion in the President’s Budget.  Figure 1 provides an overview of the general process the services use to  budget for appropriated funding support of MWR activities.

Budget processes and authorities for nonappropriated funding, or  program-generated revenue, vary by service. Specifically, the services  maintain nonappropriated funding budgets and budget approvals at  different levels within the service organization. For example, officials  stated that Marine Corps and Air Force installations maintain and manage  nonappropriated funding generated at their locations while Army and  Navy installations submit nonappropriated funding and budgets to a  higher level of command, Installation Directorates for the Army and  Regions for the Navy, as well as the service headquarters component.  The services plan for and manage their nonappropriated funding budgets  based on a number of factors, including revenue generated; projected  revenues; and the amount, if any, of appropriated funding available.  Figure 2 provides an overview of the general process the services use to  approve and manage nonappropriated funding generated within the  service.

Each service uses processes to provide funds for the implementation of  its MWR programs. Service officials stated that during program execution  the services execute their programs and make adjustments to their  budgets based on funding authorized from appropriated funding and  nonappropriated funding sources. Commanders have authority over  budget implementation and the guidelines and parameters for  commanders vary by service. For example, according to Army officials,  during the fiscal year Army commanders can change MWR program  budgets and have some flexibility to move funding to other non-MWR  command priorities. Installations report to the services actual  expenditures and income generated, which are included in the services’  annual reports. Figure 3 provides an overview of the general process the  services use to provide funding for MWR programs.

Each service uses accounting processes for its MWR programs.  According to service officials, accounting is handled differently at each  service depending on the service’s organizational structure. According to  service officials, the Navy and the Marine Corps centrally manage their  MWR accounting processes at their service headquarters; the Army  manages its accounting process at its headquarters and at the Defense  Financial and Accounting Services Nonappropriated Financial Services;  and the Air Force manages its accounting process at its Secretariat and  at the service components. According to service officials, program  managers at the service headquarters and activity level are able to review  financial data, such as expenditures and revenues, for MWR programs on  a recurring basis. DOD’s Instruction 1015.10 states that the services  should identify appropriated and nonappropriated funding accounts in  annual budgets, and the services have designated codes to categorize  expenditures. Service officials stated they use the codes to report  annually to USD(P&R) on MWR programs’ expenditures for both  appropriated and nonappropriated funding.

section_title: The Services Did Not Consistently Meet One of the Two Appropriated Funding Targets and Are Taking Steps to Address This, but DOD Has Not Comprehensively Evaluated the Targets to Ensure They Are Appropriate
paragraphs: The services generally met the funding target for fiscal years 2012  through 2017 for MWR Category A mission-sustaining programs, but did  not consistently meet the target for Category B programs that provide  community support systems to servicemembers and their families during  the same time period. Service officials said they are taking steps to meet  the Category B target, such as restoring targeted levels of appropriated  funding support in future budget planning. Data indicate that the services  are getting closer to meeting the target. However, DOD has not  comprehensively evaluated the funding targets, which were established  more than 20 years ago, to ensure they currently are appropriate.
subsections: section_title: The Services Generally Met the Funding Target for MWR Category A Mission- Sustaining Programs
paragraphs: For MWR Category A mission-sustaining programs, the services  generally met the 85-percent target for appropriated funding support.  Specifically, the Navy and the Air Force consistently met or exceeded the  85-percent funding target in fiscal years 2012 through 2017, and the Army  met or exceeded the target every year except for fiscal year 2012 when it  reported that 84 percent of its Category A programs were supported with  appropriated funds. The Marine Corps exceeded the minimum funding  target for Category A programs in fiscal years 2012 through 2017, but  consistently fell below the target with appropriated funding support  ranging from 77 percent to 84 percent from fiscal years 2013 through  2016. Table 2 provides additional detail on the extent to which each  service met the 85-percent funding target for MWR Category A mission- sustaining programs in fiscal years 2012 through 2017.

section_title: The Services Did Not Consistently Meet the Funding Target for MWR Category B Community Support Programs, but Are Taking Steps to Meet the Target in the Future
paragraphs: For MWR Category B community support programs, the services missed  the 65-percent target for appropriated funding support with increasing  frequency from fiscal years 2012 through 2017. Service officials stated  that constrained budgets and competing priorities have made it difficult to  allocate the appropriated funding needed to support their programs.  However, service officials said they are taking steps to meet the Category  B funding target in the future. Specifically, we found that the services  collectively missed the funding target over 60 percent of the time from  fiscal years 2012 through 2017. All four services missed the funding  target in fiscal years 2015 and 2016 with appropriated fund support  ranging from 55 to 63 percent. Most recently, in fiscal year 2017 the Army  met the 65-percent funding target, but the Navy, the Marine Corps, and  the Air Force fell below the 65-percent funding target with appropriated  funding support ranging from 60 percent to 62 percent. Although the Air  Force did not meet the 65-percent target for fiscal years 2012–2017 citing  resource issues, Air Force leadership has increased appropriated funding  for the MWR programs each year to help get closer to meeting the  Category B funding target. Air Force officials said they plan to continue to  increase funding each year so they can meet the target in the future.  Table 3 provides additional detail on the extent to which each service met  the 65-percent funding target for MWR Category B community support  programs in fiscal years 2012 through 2017.

The USD(P&R) monitors the services’ compliance in meeting the targets.  When a funding target is missed, USD(P&R) officials said a memorandum  is sent to the services that asks for a detailed plan on how they will  achieve the required level of appropriated funding support for the missed  target in the future, and these officials said that each service has provided  such a plan when they fell below the 65-percent funding target. In  instances when a service does not respond to the initial request for a  remediation plan, USD(P&R) officials said a second memorandum is sent  notifying the service that they missed the funding target and that they  need to submit a plan detailing how they intend to come into compliance.  For example, in fiscal year 2015 the Army did not meet the 65-percent  funding target for Category B programs. In June 2016, the Assistant  Secretary of Defense for Manpower and Reserve Affairs sent the Army a  memorandum asking it to submit a plan on how it would meet the target.  After not receiving a response, the Assistant Secretary of Defense for  Manpower and Reserve Affairs sent the Army a second memorandum in  September 2016 that noted the missed target and reiterated the need to  submit a plan for achieving compliance with designated funding targets.  Following the second memorandum, the Army issued a memorandum in  December 2016 stating it would fully fund Category A and B programs to  the required targets in fiscal year 2017. Following these communications,  in February 2018, the Army sent USD(P&R) its fiscal year 2017 program  and metric report showing that it had successfully met the Category A and  B funding targets as planned.

Service officials said they are taking steps to meet the Category B target,  and data from fiscal years 2015 through 2017 indicate that the services  are getting closer to meeting it. However, in the prior years when the  services have not met appropriated funding targets for Category B  programs, officials said that the services have relied on nonappropriated  funding as supplemental support to help ensure that such programs  continue to operate. Specifically, according to USD(P&R) officials, the  services have used nonappropriated funding—that is, revenue generated  largely through user fees incurred by servicemembers and their families— to cover MWR program costs for which appropriated funding was  authorized. However, the use of nonappropriated funds to cover shortfalls  in appropriated funding support for MWR programs has been a long- standing issue about which Congress has previously expressed concern.  Specifically, in House Report 104-563, which accompanied H.R. 3230, a  bill for the National Defense Authorization Act for Fiscal Year 1997, the  House Committee on National Security established the annual DOD  Category A and B MWR programs reporting requirement to Congress,  after receiving testimony from the services’ MWR managers and noting a  disparity in the degree of appropriated funding support afforded these  programs particularly in the area of Category A and B programs. While  the committee recognized that shortfalls in appropriated funding support  for MWR programs requires the use of nonappropriated funding to meet  requirements, it also stated that the use of nonappropriated funding  resources—soldier, sailor, airman, and Marine money—to subsidize  appropriated funding activities should be minimized.

While the Army met the Category B funding target for fiscal year 2017,  the Navy, the Marine Corps, and the Air Force have each submitted plans  and briefed USD(P&R) on how they plan to meet the target in the future.  Navy officials said that they acknowledged the Navy’s challenges with  meeting the Category B funding target and, as a result, began assessing  their Category B programs to eliminate those that had limited use,  consolidate some where possible, and implement operational efficiencies.  Marine Corps officials indicated that the Marine Corps is committed to  preserving valuable MWR programs and restoring appropriate levels of  appropriated funding support in future budget planning. Specifically, the  Marine Corps plans to readdress appropriated funding levels in the  budget planning process in 2019. However, Marine Corps officials noted  they may continue to have challenges meeting the 65-percent funding  target in fiscal year 2018. Air Force officials said they will continue to  advocate for retaining established MWR program funding in the budget  process. Air Force officials said that for fiscal years 2014 through 2017,  Air Force leadership has increased appropriated funding for the MWR  programs each year to help get the Air Force closer to meeting the  Category B funding target.

section_title: DOD Has Not Comprehensively Evaluated the Funding Targets to Ensure They Are Appropriate
paragraphs: DOD has not comprehensively evaluated the funding targets for Category  A and B programs, which were instituted more than 20 years ago, to  ensure they are appropriate. Standards for Internal Control in the Federal  Government recommends that management periodically review policies  and procedures for continued relevance and effectiveness in achieving an  entity’s objectives. According to USD(P&R)officials, a limited evaluation  took place prior to 1995 that resulted in the Category A funding target in  DOD’s instruction being changed from 100 percent to 85 percent.  USD(P&R) officials said that the Category A appropriated funding target  was changed because some of the activities within the category have  expenses, such as for the food and beverage elements, that are able to  generate revenue and thus not authorized to use appropriated funds.

USD(P&R) officials stated that since that time there have been no further  evaluations of the Category A or Category B targets and agree that it is  time to evaluate the current relevance of the targets. Specifically they  noted the considerable changes to the budgeting and funding  environment that have taken place in the more than 20 years since the  Category A funding target was modified. In addition, officials told us they  also agree that it is time to evaluate the relevance of the Category B  funding target, which has never been modified. Specifically, officials said  that the services’ extended engagement in overseas conflicts and  constrained budgets have resulted in an operating environment that is  substantially different from the peacetime setting in which the targets  were first established.

Moreover, Standards for Internal Control in the Federal Government  requires management to document internal controls to meet operational  needs. Documentation of controls, including changes to controls, is  evidence that controls are identified, capable of being communicated to  those responsible for their performance, and capable of being monitored  and evaluated by an entity. Documentation also provides a means to  retain organizational knowledge and mitigate the risk of having that  knowledge limited to a few personnel, as well as a means to  communicate that knowledge as needed to external parties, such as  external auditors. As previously stated, officials stated that the Category A  funding target was updated sometime prior to 1995; however, officials did not have any specific documentation related to this change. Furthermore,  USD(P&R) officials said the targets were developed so long ago that  there is a general lack of information on the funding targets’ origins and  that they are not sure of the process or methodology that was used to  develop them.

The amount of time that has passed since Category A’s target was  modified, recent challenges in meeting the Category B target, and the  general lack of information on the funding targets’ origins raise concerns  about the appropriateness and continued relevance and effectiveness of  the targets in achieving MWR programs objectives. Until DOD  comprehensively evaluates the appropriateness of current targets for  Category A and B programs and, based on its evaluation, documents any  changes it makes to its funding targets, DOD cannot be certain that the  targets reflect the current operating environment and do not pose undue  financial burden on the servicemembers.

section_title: DOD Has Established an Oversight Structure and Performance Measures for MWR Programs but Has Not Developed Measurable Goals for Determining Whether MWR Programs Are Cost-Effective
subsections: section_title: DOD Has Established a Structure to Provide Oversight of MWR Programs
paragraphs: DOD has established a structure that specifies roles, responsibilities, and  procedures for overseeing MWR programs. Specifically, DOD Instruction  1015.10 assigns roles and responsibilities for oversight of MWR programs  to the USD(P&R), the Secretaries of the military departments, and the  Chiefs of the military services (i.e., the Chiefs of Staff for the Army and  the Air Force, the Chief of Naval Operations, and the Commandant of the  Marine Corps). In addition, the services’ respective policies assign roles  and responsibilities for MWR program oversight to the commander level.  Table 4 summarizes the general oversight roles and responsibilities for  DOD’s MWR programs.

The first level of oversight responsibility for MWR programs is assigned to  the USD(P&R). Specifically, responsibilities include the development of  department-level policies, program goals, performance measures, funding  targets, and the oversight of appropriated and nonappropriated funding  and expenditures for all MWR programs. To help ensure consistent  quality, USD(P&R) monitors the services’ compliance in meeting  minimum MWR funding targets and performance measures. As  previously discussed, if a service misses a funding target, USD(P&R)  officials said they ask that service to submit a remediation plan that  summarizes its intent to meet the target in the future, as USD(P&R) did in  fiscal year 2015 when several services missed appropriated funding  targets for Category A and B activities.

The second level of oversight is assigned to the Secretaries of the military  departments who are responsible for designating a central point of  contact within their respective service to facilitate MWR programs policy  compliance, coordinating with USD(P&R), and establishing funding  priorities and strategy for MWR programs. For example, service officials  we met with from the military departments said they have designated their  respective Assistant Secretary Offices for Manpower and Reserve Affairs  as the central point of contact for the services’ MWR programs.

The third level of oversight is assigned to the Chiefs of the military  services who are responsible for the development of overall goals and  uniform quality measures, which could include performance measures, for  MWR programs consistent with the performance measures set by DOD in  its instruction. For example, the Commander, Navy Installations  Command has developed uniform quality measures for the Navy MWR  Fitness program based on items such as customer satisfaction, usage  rates, and equipment maintenance, among other things. According to  officials, these quality measures provide a common tool to measure  customer satisfaction and the quality of each installation’s MWR Physical  Fitness program. Additionally, these Chiefs are also responsible for  helping to ensure MWR programs are resourced with appropriated and  nonappropriated funding according to financial categories and for  identifying their respective appropriated and nonappropriated accounts in  annual budgets to meet DOD funding goals. Service Chiefs are also  responsible for ensuring that military installations operate customer-driven  MWR programs that are determined locally by market analysis.

Lastly, the services’ respective policies assign roles and responsibilities  for MWR program oversight to the commander level. Additionally,  according to service officials, commanders assist with preparing an  annual briefing for USD(P&R) on their MWR programs, which includes  initiatives, challenges, program trends, and financial information. For  example, in fiscal year 2017, each of the services reported on new  initiatives to support MWR programs for servicemembers and their  families, some of which are highlighted in table 5.

section_title: DOD and the Services Have Performance Measures to Assess MWR Programs but These Measures Lack Measurable Goals for Determining Cost- Effectiveness
paragraphs: DOD Instruction 1015.10 identifies six broad categories of performance  measures that the services use to assess their respective MWR  programs. However, these measures do not include measurable goals,  which are needed to assess the cost-effectiveness of the 55 activities that  currently make up the MWR programs. Specifically, DOD identifies six  broad performance measure categories in its instruction and, according to  service officials, the services collect and use various types of information  within these categories to periodically assess and adjust these activities,  as appropriate. Table 6 summarizes the types of information that DOD  requires the services to collect across the six categories established in its  instruction.

In addition to the information that is to be collected across these six broad  categories, DOD established separate, more specific performance  measures for 2 of the 55 activities—namely, for Physical Fitness and for  Library Programs and Information Services. For the Physical Fitness  activity, the services are required to submit annual reports to DOD on  their compliance with meeting more specific performance measures in a  variety of areas such as administrative operations, staff qualifications,  facility equipment, and child play areas. Similarly, DOD requires the  services to report on a variety of areas related to the Library Programs  and Information Services activity, such as library operation plans,  customer programs and service, and technology infrastructure. Unlike the  broad measures contained in DOD’s Instruction, the specific performance  measures DOD established for the Physical Fitness and Library  Programs and Information Services activities tell the services exactly  what information to collect and report in each performance measure  category instead of the services having to develop specific measures on  their own.

In an effort to better evaluate MWR programs, the services also have  efforts underway that include the following to develop specific  performance measures for their programs beyond the broad performance  measures contained in DOD Instruction 1015.10.

Army. Army officials told us that they partnered with the Army Public  Health Center to build evidence-based MWR programs. Based on this  review, the Army found that Army MWR Community Recreation and  Fitness programs have not been formally evaluated as directed by  DOD Instruction 1015.10 requirements to measure and assess  programs. Additionally, the Army found that, while the Army Office of  the Assistant Chief of Staff for Installation Management provides  program oversight, it does not possess the capability to conduct  program evaluations. According to the results of the Army Public  Health Center report issued in June 2017, the Army initiated a three- phase approach for evaluating its MWR programs. The report showed  that assessing the evaluability of the Army MWR programs is phase  one. According to the Army, these evaluations will enable the Army to  validate program outcomes and better position itself to compete for  scarce resources. The report also showed that many of the 13 Army  MWR programs selected for review do not have direct links between  activities and the priority outcomes with behavioral, social, and  physical health, and that they do not have sufficient outcomes data  that have been consistently collected. Army officials said that phase  two will include the development of formal evaluation plans for  selected evaluable MWR programs. Lastly, Army officials said that  phase three will be the execution of the evaluation for two selected  MWR programs, which is on target to be completed by December  2018. While Army officials are learning how to evaluate programs  through this partnership with the Army Public Health Center, they said  that they have also learned that these endeavors are costly. Officials  said that a very modest program evaluation requires approximately  $300,000 to $500,000. Army officials also stated that program  evaluation requires support and participation by those organizations  and people that deliver the programs. Furthermore, according to Army  officials, resource reductions at the operational level (garrisons) are  increasingly restrictive, preventing them from collecting critical  information to support this multiphase effort.

Navy. Navy officials said that they use the MWR Enterprise Modeling  System, which is based on performance measures that have been  developed and routinely reviewed and updated by headquarters,  regional, and installation program managers. The MWR Enterprise  Modeling System is used as the baseline for the annual MWR  performance data call that measures actual program performance  against performance standards. Navy officials said that the  performance measures provide the business strategy and guidance to  ensure efficient, effective and market-driven delivery of programs and  services.

Marine Corps. Marine Corps officials said they collaborated with the  RAND Corporation to provide an analytically rigorous assessment  framework to evaluate program performance. The RAND Corporation  provided draft measures of performance. Marine Corps officials said  that the RAND Corporation also provided a user guide that outlines an  evaluation methodology and ensures consistent and standard  application. Marine Corps officials said that they are reviewing the  draft measures to determine appropriate data collection and have  drafted an implementation plan. Specifically, Marine Corps officials  said that they plan to brief Marine Corps installations in June 2018 on  the performance measures they plan to collect data from, which will  begin in fall 2018.

Air Force. Air Force officials said that they are building off the work  that the RAND Corporation undertook for the Marine Corps and have  also started collaborating with the RAND Corporation. The objective of  the Air Force study is to develop an evidence-based evaluation  framework for MWR programs that identifies immediate and mid-term  outcomes that contribute to airman and family readiness and  resilience. Specifically, the goal is to provide the Air Force with logic  models and performance measures that are tied to each of the  programs and services in the MWR portfolio. Air Force officials said  they expect to finish this study by June 2018. However, the officials  noted that implementing the performance measures will be a  challenge since these types of MWR programs are difficult to measure  and hard to capture data for.

While both the broad and specific measures established by DOD and the  services can provide useful context about the status of individual MWR  activities, they do not contain measurable goals that service officials could  use to compare program results with costs to determine whether an  individual activity is cost-effectively operating. Because the services’  efforts to develop specific performance measures are in early stages of  development it is too early to determine whether these efforts will result in  measurable goals that can be used to assess the cost-effectiveness of  the MWR programs.

DOD’s Financial Management Regulation specifies that performance  measurement should include program accomplishments in terms of  outputs and how those outputs effectively meet intended agency mission  goals. Further, cost itself can be a performance metric, but should also be  combined with an effectiveness measure, such as the percentage of a  goal achieved at a level of expected performance, to ensure that the  resulting output is cost effective. Additionally, through our prior work on  performance measurement, we have reported that performance goals  and measures should align with an agency’s goals and mission. However,  in reviewing DOD Instruction 1015.10, we found no mention of any goals,  mission, objectives, or purpose for the MWR programs. There is one  section entitled “policy” in the instruction that included items that resemble  goals. Specifically, the instruction stated that MWR programs:  1.  are an integral part of the military and benefits package;  2.  build healthy families and communities and provide consistently high- quality support services that are commonly furnished by other  employers or by state and local governments to their employees and  citizens;  3.  encourage positive individual values and aid in recruitment and  retention of personnel; and  4.  promote esprit de corps and provide for the physical, cultural, and  social needs; general well-being; quality of life; and hometown  community support of servicemembers and their families.

USD(P&R) officials who have responsibility for developing MWR program  goals acknowledged that these policy items function as strategic goals  but were not clearly identified as such in the instruction and also  acknowledged that the instruction does not include measurable goals for  assessing cost-effectiveness. In addition, USD(P&R) officials said that  they are starting a review of DOD Instruction 1015.10 and did not know  yet whether they would make any changes to the goals or expand the  reporting requirement to include all 55 activities. Until DOD develops  performance measures that include measurable goals, DOD officials and  other decision makers, such as Members of Congress, may find it difficult  to determine whether the MWR programs and the activities that make up  the MWR programs are meeting servicemember needs in a cost-effective  manner.

section_title: Conclusions
paragraphs: DOD’s multibillion dollar MWR programs provide a wide range of benefits  for servicemembers and their families that ultimately help support military  missions and readiness, both in times of war and peace. DOD has  established funding targets for providing appropriated funding support for  Category A and B MWR programs. However, the funding targets have not  been comprehensively evaluated in the last 20 years to determine their  current relevance. Until DOD comprehensively evaluates the  appropriateness of current funding targets and documents any changes  made to the targets, DOD’s funding targets may not reflect the current  operating environment, and may be posing an undue burden on the  servicemembers. DOD has also not developed performance measures  with measureable goals that would allow it to assess the cost- effectiveness of its MWR programs. Without performance measures that  include such measurable goals, it will be difficult for DOD and Congress  to determine whether the individual activities and overall MWR programs  are meeting desired outcomes in a cost-effective manner.

section_title: Recommendations for Executive Action
paragraphs: We are making the following two recommendations to DOD.

We recommend that the Secretary of Defense ensure that the USD(P&R),  in consultation with the Secretaries of the military departments,  comprehensively evaluate the funding targets for Category A and B MWR  programs and document any changes made to the targets and the  methodology used. (Recommendation 1)

We recommend that the Secretary of Defense ensure that the USD(P&R),  in consultation with the Secretaries of the military departments, develop  measurable goals for its MWR programs’ performance measures to  determine the programs’ cost-effectiveness. (Recommendation 2)

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD for review and comment. In its  comments, DOD concurred with our recommendations and noted actions  that it is taking. DOD’s comments are reprinted in their entirety in  appendix II. DOD also provided technical comments, which we  incorporated into the report as appropriate.

We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Secretaries of the Army, the  Navy, and the Air Force; the Commandant of the Marine Corps; and the  Under Secretary of Defense for Personnel and Readiness. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3604 or farrellb@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff members who made key contributions to  this report are listed in appendix III.

section_title: Appendix I: Department of Defense’s Morale, Welfare, and Recreation Program Categories

section_title: Appendix II: Comments from the Department of Defense

section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Kimberly A. Mayo, Assistant  Director; Rebekah Boone; Mae Frances Jones; Felicia Lopez; Stephanie  Moriarty; Cynthia Saunders; John W. Van Schaik; Paul Seely; Carter  Stevens; and Roger Stoltz made key contributions to this report.


Question-summary annotations: questions: question: What categories received funding targets by the DOD?
answer: The Department of Defense (DOD) established funding targets for two categories of Morale, Welfare, and Recreation (MWR) programs—Category A, which promotes the physical and mental well-being of servicemembers, and Category B, which funds community support systems for servicemembers and their families.

question: Why were these targets created?
answer: These targets are intended to ensure that the military services adequately fund these programs with appropriated funds instead of requiring servicemembers and their families to pay fees out of pocket to cover program costs.

questions: question: How much of the category fundings have been met in the past years?
answer: In fiscal years 2012-2017, the military services generally met the DOD-set target to provide 85 percent of appropriated funding for Category A programs but not the 65-percent target for Category B programs.

question: What is the projected plan to meet the target for Category B programs?
answer: Service officials said they are taking steps to meet the Category B target, such as by restoring targeted levels of appropriated funding support in future budget planning.
child_questions: question: How helpful are these steps?
answer: Data GAO reviewed indicate that these steps are helping the services get closer to meeting the target for Category B.

questions: question: What is the current status of the evaluation of the targets?
answer: DOD has not comprehensively evaluated the targets, established more than 20 years ago, to ensure that they are appropriate.
child_questions: question: How has the lack of evaluation affected DOD's future plans?
answer: DOD officials said they agree that it is time to evaluate the relevancy of the targets as the current operating environment is fundamentally different than when the targets were established 2 decades ago.

question: What hesitations does the DOD possess?
answer: Further, DOD officials said that they are unsure of the process or methodology used to originally develop the targets because they have no documentation supporting these decisions.

question: What is being held back until the DOD makes its evaluation?
answer: Until DOD comprehensively evaluates the appropriateness of the targets and, based on its evaluation, documents any changes made, it cannot be certain that the targets reflect the current operating environment and do not pose undue financial burden on servicemembers.

questions: question: What has and has not been established by the DOD?
answer: DOD established oversight structures and performance measures for MWR programs, but has not established measurable goals to assess the cost-effectiveness of the 55 activities that make up MWR programs.

question: What has the DOD officials stated about the DOD's MWR policy?
answer: DOD officials responsible for developing MWR program goals acknowledged that DOD's MWR policy does not include measurable goals for assessing the cost-effectiveness of program activities, and do not currently have plans to make any changes to the goals.

question: How are activities within the categories assessed?
answer: Service officials told GAO that they collect and use various types of information within the categories to assess specific activities.
child_questions: question: What have the assessments revealed about the categories?
answer: While both the categories established by DOD and the service-specific efforts provide useful context about the status of individual MWR activities, they do not replace the need for measurable goals that can be used to assess whether the programs are operating cost-effectively.

question: What is known about the status of the services?
answer: The services are in the early stages of developing more specific performance measures, but it is too early to determine whether these efforts will result in measurable goals that can be used to assess cost-effectiveness.
child_questions: question: Why is it unclear whether or not the MWR programs are effective?
answer: Until DOD develops performance measures that include measurable goals, it cannot ensure that MWR programs meet servicemember needs in a cost-effective manner.

questions: question: How does the MWR programs help servicemembers and their families?
answer: DOD's MWR programs provide servicemembers and their families with three categories of programs: Category A (e.g., fitness and libraries), Category B (e.g., camping and performing arts), and Category C (e.g., golf).

question: How are the fundings checked and overseen?
answer: DOD oversees the percentage of appropriated funding allocated to MWR programs by category and measures the military services' compliance with established funding targets.

question: What targets have been set by the DOD?
answer: DOD set the targets at 85 percent for Category A and 65 percent for Category B.
child_questions: question: Why were there no targets set for Category C?
answer: DOD did not set a target for Category C since this category has the ability to generate revenue from user fees.

questions: question: What is included in the House Report 115-200?
answer: House Report 115-200 accompanying a bill for the National Defense Authorization Act for Fiscal Year 2018 includes a provision for GAO to review DOD' s MWR programs.

question: What has been assessed by the GAO?
answer: GAO assessed the extent to which (1) the services have met DOD's established funding targets for each category of MWR programs and DOD has comprehensively evaluated the relevance of its targets, and (2) DOD has oversight structures and performance measures that include measurable goals, including those for cost-effectiveness, by which to review MWR programs.

question: What was analyzed by the GAO in order to create this assessment?
answer: GAO analyzed MWR program information for fiscal years 2012-2017 and compared DOD's MWR policy with guidance for using measures and evaluating goals.