prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.



Report 2: Federal and state Medicaid spending on long-term care continues to increase; for example it increased from $146 billion in 2013 to $158 billion in 2015  Individuals seeking long-term care generally need care that is, by definition, longer term in nature and more costly than other types of care  Spending on long-term care services provided in home and community settings, including assisted living facilities, exceeds the amount spent on institutional settings such as nursing homes  State Medicaid programs may cover certain medical and non-medical services that assisted living facilities provide; however, the Medicaid statute does not provide for coverage of room and board charges of an assisted living facility  In their federal-state partnership, both CMS and states play important roles in the oversight of Medicaid  CMS is responsible for oversight of state Medicaid programs  To conduct this oversight, CMS issues program requirements in the form of regulations and guidance, approves changes states make to their programs, provides technical assistance to states, collects and reviews required information and data from states and, in some cases, reviews individual state programs  States are responsible for the day-to-day administration of their Medicaid programs, including monitoring and oversight of the different HCBS programs through which they cover assisted living services, within broad federal rules and requirements  Each state is required to identify and designate a single state agency to administer or supervise the administration of its Medicaid program  The state Medicaid agency may partially or fully delegate the administration and oversight of the state’s HCBS programs to another state agency or other entity, such as a state unit on aging, a mental health department, or other state departments or agencies with jurisdiction over a specific population or service  However, the state Medicaid agency is ultimately accountable to the federal government for compliance with the HCBS requirements  Under different authorizing provisions of federal law, states have considerable flexibility to establish multiple HCBS programs including those covering assisted living services  A state Medicaid program can have multiple HCBS programs operating under different federal authorities  CMS is responsible for ensuring that states meet the requirements associated with their HCBS programs under these different authorities  Key to states’ monitoring of the health and welfare of Medicaid beneficiaries is their tracking of, and response to, incidents that may cause harm to a beneficiary’s health or welfare, such as abuse, neglect, or exploitation—commonly referred to as critical incidents  Such monitoring is required for most HCBS programs; however, we previously found that requirements for states related to oversight of the health and welfare of beneficiaries in different types of HCBS programs varied, and recommended that CMS take steps to harmonize those requirements across programs  The most common HCBS programs with the most stringent federal requirements are HCBS waiver programs  These programs serve beneficiaries who are eligible for an institutional level of care; that is, beneficiaries must have needs that rise to the level of care usually provided in a nursing facility, hospital, or other institution  CMS oversees states’ HCBS waiver programs specifically by reviewing and approving applications and reviewing HCBS program reports that states submit  HCBS waiver program applications include specific requirements implementing various statutory and regulatory provisions  (See text box below ) One requirement is that states have the necessary safeguards in place to protect the health and welfare of beneficiaries receiving services covered by HCBS waiver programs  For each of their HCBS waiver programs, states must demonstrate to CMS that they are meeting various requirements CMS has established regarding beneficiary health and welfare  The Six Requirements States Must Demonstrate for Home- and Community-Based Services Waiver Programs 1  Administrative authority: The Medicaid agency retains ultimate administrative authority and responsibility for the operation of the waiver program by exercising oversight of the performance of waiver functions by other state and local/regional non-state agencies (if appropriate) and contracted entities  2  Level of care: The state demonstrates that it implements the processes and instrument(s) specified in its approved waiver for evaluating/re-evaluating an applicant’s/waiver participant’s level of care consistent with care provided in a hospital, nursing facility, or intermediate care facility  3  Qualified providers: The state demonstrates that it has designed and implemented an adequate system for assuring that all waiver services are provided by qualified providers  4  Service plan: The state demonstrates it has designed and implemented an effective system for reviewing the adequacy of service plans for the waiver participants  5  Health and welfare: The state demonstrates it has designed and implemented an effective system for assuring waiver participant health and welfare  6  Financial accountability: The state must demonstrate that it has designed and implemented an adequate system for insuring financial accountability of the waiver program  CMS also provides ongoing oversight of state HCBS programs through annual reports that states must submit for each of their HCBS waiver programs as well as renewal reports submitted about two years before an HCBS waiver is scheduled to end  The state reports are intended to provide CMS with information on the operation of state HCBS waiver programs  In contrast to long-term care services provided in nursing facilities, less is known at the federal level about the oversight and quality of care in assisted living facilities  Generally, states establish their own licensing and oversight requirements for assisted living facilities  As a result, the requirements for assisted living facilities and the type and frequency of oversight can vary across states  In contrast, nursing homes must meet a comprehensive set of federal requirements in order to receive payment for long-term care services for Medicaid and Medicare beneficiaries in addition to state requirements  CMS contracts with state entities to regularly inspect nursing facilities and investigate complaints to assess whether nursing homes meet these federal quality requirements  Annually CMS publishes a comprehensive report on nursing homes that serve Medicaid and Medicare beneficiaries, including the extent that beneficiaries are at risk for harm, based on these investigations and inspections  In addition, CMS publicly reports a summary of each nursing home’s quality data using a five-star quality rating based on health inspection results, staffing data, and quality measure data  The goal of this rating system is to help consumers make meaningful distinctions among high- and low-performing nursing homes  This type of standardized framework for oversight, investigation and inspections, and reporting on quality of care concerns does not exist for assisted living facilities and other types of HCBS providers  Forty-eight state Medicaid agencies reported collectively spending about $10 billion in state and federal Medicaid funds for assisted living services in 2014, according to our survey  The other 3 states reported that they did not pay for assisted living services  We estimate that this spending for services provided by assisted living facilities represents 12 4 percent of the $80 6 billion Medicaid spent on HCBS in all settings that year  More than 330,000 Medicaid beneficiaries received assisted living services, based on data reported to us by the 48 states  Nationally, the average spending per beneficiary on assisted living services in the 48 states in 2014 was about $30,000; states provided these HCBS services through fee-for-service and managed care delivery models  Fee-for-service spending comprised 81 percent of total spending on assisted living services and managed care spending was about 19 percent of the total  The cost per beneficiary reported by surveyed states also varied based on payment type; average per beneficiary cost was $31,000 for fee-for-service and $27,000 for managed care  About 21 percent of Medicaid assisted living enrollment was for beneficiaries receiving these services under a managed care delivery model  (See table 1 ) Average per-beneficiary spending varied significantly across the states  For example, for the nine states with the lowest spending per beneficiary, average Medicaid spending ranged from about $1,700 to about $9,500 per beneficiary  In contrast, in the nine states with the highest per- beneficiary spending, the average spending ranged from about $43,000 to $108,000 per beneficiary  (See Figure 1 ) For more information on each state’s enrollment, total spending, and average per beneficiary spending on assisted living services, see appendix I  The 48 states that reported covering assisted living services in 2014 said they did so through 132 different programs  The majority of the states, 31 of the 48, reported administering more than one program that covered assisted living services  As illustrated in table 2 below, of the different types of HCBS programs under which states can provide coverage for assisted living services, HCBS waivers were the most common type of program they used  Specifically, 39 states and 69 percent of the programs that provided assisted living services, were operated under the HCBS waiver program  (See appendix II for additional details on each state’s number of programs by program type and total number of HCBS programs that covered assisted living facility services in 2014 ) Almost all of the 48 states that covered assisted living services did so for two groups of Medicaid beneficiaries eligible through their programs  In 45 of 48 states, aged beneficiaries received services provided by assisted living facilities  Similarly, in 43 of 48 states, physically disabled beneficiaries received services  (See Figure 2 ) In 38 or more of the 48 states that covered assisted living services, six types of services were provided  For example, 45 states covered assistance with activities of daily living, such as bathing and dressing; 44 states covered medication administration; and 41 states covered coordination of meals  (See Figure 3 ) State Medicaid agency approaches for oversight of assisted living services varied widely in terms of who provided the oversight for their largest programs, according to their responses to our survey  Thirteen of the 48 state Medicaid agencies reported delegating administrative responsibilities, including oversight of beneficiary health and welfare, to other state or local agencies  State Medicaid agencies may delegate the administration of programs to government or other agencies through a written agreement; however, state Medicaid agencies retain the ultimate oversight responsibility for those delegated functions  For example, among the 13 states that delegated HCBS program administration, the administering agencies were those that provided services to the aged, disabled, or both of these populations, such as the states’ Departments of Aging  (See text box, below, for examples of states’ delegation ) Examples of State Medicaid Agencies’ Delegation of Authority for Administration of Home- and Community-based Services’ Programs Covering Assisted Living Services Georgia’s Elderly & Disabled Waiver Program was operated in 2014 by the Georgia Department of Human Services Division of Aging Services, a separate agency of the state that was not a division/unit of the Medicaid agency  The Georgia Medicaid Agency maintained a formal interagency agreement with the Division of Aging Services which describes by function the required deliverables to support compliance and a schedule for delivery of reports  Nebraska’s Waiver for Aged and Adults and Children with Disabilities is operated by the state Medicaid agency Division of Medicaid and Long Term Care  The majority of services are provided by independent contractors in order to allow service delivery in the rural and frontier areas of the state  The state Medicaid agency contracts with the Area Agencies on Aging, Independent Living Centers, and Early Development Network agencies to perform a variety of operational and administrative functions including authorizing services and monitoring the delivery of services  States also varied in the types of information they reported reviewing as part of the oversight of assisted living services, and the extent to which state Medicaid agencies review the information when another agency is responsible for administration  For example, other entities outside the state Medicaid agency—such as the agency delegated to administer an HCBS program, or a contractor that manages provider enrollment—may check to ensure a provider is allowed to deliver services to Medicaid beneficiaries; in such cases, however, the state Medicaid agency might not be aware of the results of such checks  As illustrated in table 3, in all 48 states the types of information generally reviewed by either the state Medicaid agency, the agency delegated administrative responsibilities, or other agencies were: critical incident reports, the HHS Office of Inspector General’s list of excluded providers, patient service plans, and information on concerns about care received directly from patients, relatives, caregivers or the assisted living facility itself  In many cases, the state Medicaid agency did not review all information sources reviewed by other agencies  For example, although all critical incident reports were reviewed in the 48 states by either the state Medicaid agency, the agency delegated administrative responsibilities, or another agency; in 16 of those states, the state Medicaid agency was not involved in those reviews, according to responses to our survey  Instead, the critical incident reports were reviewed by another entity designated responsible for the HCBS program in the state or another state entity with regulatory responsibility over the assisted living facility  Such reviews, including any critical incidents found, may not have been communicated back to the state Medicaid agency, according to responses to our survey  State Medicaid agencies also varied in reporting the extent to which they were made aware or notified when enforcement actions were taken as a result of concerns with beneficiary care identified by other entities  Various oversight actions may be taken by the state Medicaid agency, the agency delegated to administer an HCBS program, or a state regulatory agency, such as a state agency responsible for licensing and inspecting various types of HCBS providers  When delegated agencies or other licensing agencies take corrective action, the state Medicaid agency may not be aware unless notified by the agencies taking that action  For example, in 23 states, the investigation of potential incidents related to beneficiary health and welfare was delegated to another agency but in only 6 of these states was the state Medicaid agency always notified of such an investigation based on our survey  (See table 4 and text box below ) Example of a Collaborative Approach to Monitoring and Ensuring Quality Care Specifically for Assisted Living Facilities In 2009, the Wisconsin Coalition for Collaborative Excellence in Assisted Living was formed to redesign the way quality is ensured and improved for individuals residing in assisted living communities  This public/private coalition utilizes a collective impact model approach that brings together the state, the industry, the consumer, and academia to identify and implement agreed upon approaches designed to improve the outcomes of individuals living in Wisconsin assisted living communities  The core of the coalition is the implementation of an association developed, department approved, comprehensive quality assurance, quality improvement program  For their largest HCBS programs that covered assisted living services, the 48 states varied in how they monitored “critical incidents” that caused actual or potential harm to Medicaid beneficiaries in assisted living facilities  Specifically, the 48 states varied in their ability to report the number of critical incidents; how they defined incidents, and the extent to which they made information on such incidents readily available to the public  These states varied in whether they could provide us the number of critical incidents involving beneficiaries for their largest programs covering assisted living services, and for those that could report, the number of incidents they reported varied widely  In 26 of the 48 states the Medicaid agencies were unable to report, for their largest program covering assisted living services, the number of critical incidents that had occurred in assisted living facilities in 2014  The remaining 22 states reported a total of 22,921 critical incidents involving Medicaid beneficiaries in their largest programs covering assisted living services  The number of critical incidents reported in these states ranged from 1 to 8,900  For six of these states the number of critical incidents reported was more than 1,000, (See text box, below, for examples of selected state processes managing critical incidents ) Selected States’ Processes for Managing Beneficiary Harm or Potential Harm in Assisted Living Facilities Georgia: According to state officials in 2014 there was no centralized or comprehensive system for capturing and tracking the data on actual and potential violations  State officials acknowledged the lack of a centralized system prevents the Division of Community Health from tracking the status of each problem  Nebraska: According to state officials, Nebraska’s Adult Protective Services operates an electronic system that coordinates across state social service programs  When Adult Protective Services initiates an investigation of reported harm to an assisted living resident, the state Medicaid agency is automatically notified  Reasons state Medicaid agencies reported for being unable to provide us with the number of critical incidents included limitations in the data or data systems for tracking them  Nine states reported an inability to track incidents by provider type, and thus distinguish critical incidents in assisted living facilities from other providers of home and community based services  States also cited lacking a system to collect critical incidents (9 states), and that the system for reporting could not identify whether a resident was a Medicaid beneficiary (5 states)
Report 2:  Research Center Infrastructure: As with personnel matters, research centers have discretion to manage infrastructure in accordance with the policies and procedures of their parent institutions  While one center, Lincoln Laboratory, is located on government property, others primarily reside on property owned or leased by their parent institutions  According to agency officials, DOD contributes funding for the use and repair of these facilities through their contracts with research centers  Officials noted that Lincoln Laboratory uses military construction funding to pay for new buildings as it is located on government property  Trusted Advisor Role: FFRDCs and UARCs function as trusted advisors for the government and operate in the public interest with objectivity and independence  FFRDCs are independent, private-sector, non-profit organization units required to be free from personal or organizational conflicts of interest, as the FFRDCs answer to the government customer  As a result, DOD’s lab FFRDCs perform tasks that are closely associated with the performance of inherently governmental functions and have access to sensitive and proprietary data  Research center officials noted challenges limiting their work providing scientific expertise to DOD  FFRDCs are also limited in executing infrastructure investments  Limitation on Available Work Hours: DOD FFRDCs are limited by an annual ceiling set by Congress on the amount of staff years of technical effort (STE) that may be funded for defense FFRDCs  We previously found in October 2008 these limits were imposed in response to concerns that DOD was inefficiently using its FFRDCs  We found that the STE workload limitation aimed to ensure that FFRDC work was appropriate and limited resources were being used for DOD’s highest priorities  As a result, Software Engineering Institute officials said they decline many DOD programs’ requests for assistance due to the annual work hour limitation  Further, officials at the Office of the Secretary of Defense’s Studies and FFRDC Management Office reported that this limit significantly constrains the use of DOD’s FFRDCs and that DOD customer demand for their services is significantly greater than the annual STE limit  OSD officials indicated that FFRDC related work must be deferred to later years when these limits are reached, since there are no other legally compliant alternatives capable of fulfilling these requirements  Infrastructure: FFRDC officials we interviewed identified infrastructure challenges—including aging facilities and equipment—as hindering their research and development efforts  For example, many buildings at the Massachusetts Institute of Technology (MIT) Lincoln Laboratory are over 60 years old; MIT considers over half of them to be in substandard condition  According to an MIT official, these facilities, located on government property, were not structurally designed for modern research and have relatively poor vibration isolation, resulting in inefficient workarounds or work that could not be performed  Officials from the Defense Laboratories Office noted that the MIT Lincoln Laboratory is unique among DOD’s FFRDCs in that it is operated on government- owned property  A 2013 study, conducted on behalf of the White House Office of Science and Technology Policy, found that lab infrastructure project funding proposals must compete with hospitals, barracks, runways, and roads and, therefore, tend to be lower on the priority list for military construction funding  A 2017 Defense Science Board report and DOD officials we spoke with indicated this continues to be true  While contract research centers have significant flexibility to execute infrastructure work, they are still affected by limited availability of military construction funding  Officials at another center noted that in some instances, DOD sponsors have been unable or slow to provide required secure facilities and equipment within needed time frames  Delays of this nature can affect the research centers’ ability to deliver the technologies or related services needed by DOD  The Department of Energy (Energy) primarily relies on contractor- operated FFRDCs to operate its labs, while the majority of NASA labs and centers are government-operated  Energy’s national labs form the core of the agency’s scientific work and mission  This is in contrast to DOD-funded labs, which constitute a relatively small aspect of DOD’s overall mission  We have previously found that Energy’s labs can use funding for minor infrastructure improvements  NASA centers can also approve and fund certain facility projects, in accordance with NASA policies, and they have encountered significant challenges with aging infrastructure  Also, in some cases, energy and space research centers have significant challenges with hiring replacement staff and competing with private sector employers for staff  Energy’s labs can hire scientific personnel with the flexibility of private companies, while NASA centers were previously provided hiring flexibilities by Congress in 2004 to facilitate staff hiring  While Energy and NASA’s research entities follow their specific governance models, there are broad characteristics common across these agencies as well as DOD  Table 2 illustrates that while research centers are largely government-owned, the government is not always the operator  As we have reported, Department of Energy national labs are primarily operated by for-profit, non-profit and university FFRDC contractors using management and operating contracts, which are competed on a limited basis  Energy’s funding sponsors and headquarters officials are required to reevaluate FFRDC performance in increments not to exceed 5 years by federal acquisition regulations, which inform future decisions to renew the agreement  In 1990, we designated Energy’s contract management—including both contract administration and project management—a high-risk area because of Energy’s inadequate management and oversight of contractors, leaving the department vulnerable to fraud, waste, abuse, and mismanagement  In 2009, we subsequently narrowed the focus of Energy’s high-risk designation to the National Nuclear Security Administration and Office of Environmental Management, which together oversee four national labs  Further, in our 2017 High Risk report, we found that these two agencies had made progress in addressing our contract management concerns, but we identified continued problems with the agencies having sufficient capacity to mitigate contract and project management risks  Also, we found that they had demonstrated little progress in addressing contract management challenges, particularly in the area of financial management  The Department of Energy uses performance-based management and operating contracts, which have been subject to limited competition, with universities, non-profit companies and for-profit companies to operate the national labs on government-owned property  These contractor-operated FFRDCs provide the vast majority of Energy’s science and technology capacity, rather than supplementing the work of government-operated labs like DOD’s FFRDCs  Energy has depended on the expertise of private organizations to execute its science and technology work since the Manhattan Project produced the first atomic bomb during World War II  The Spallation Neutron Source is an experimental research facility at Oak Ridge National Laboratory—a government-owned contractor-operated laboratory  The Spallation Neutron Source includes the world’s most powerful pulsed-neutron sources and provides information about the structure and properties of materials that cannot be obtained by other means  The Spallation Neutron Source is a user facility whereby researchers from universities, national laboratories, and industry submit proposals, which are peer- reviewed and must compete for time at the user facility  The primary focus of each lab varies based on its expertise and facilities  Energy largely oversees its lab contractors through its headquarters program offices, which include the National Nuclear Security Administration, Office of Science, the Office of Fossil Energy, as well as co-located government field offices  Office of Science-sponsored labs primarily support scientific research for energy and physical sciences, while the National Nuclear Security Administration-sponsored (NNSA) labs primarily focus on nuclear weapons and related science and technologies  Energy also oversees its lab contractors’ activities through on-site Energy oversight offices that work alongside lab management at each FFRDC  Some labs specialize in earlier-phase science, while other labs work on later-phase nuclear weapons technologies in addition to earlier-phase science  As Figure 10 shows, these labs are spread across the United States  Energy has only one government-operated and government-owned lab, the National Energy Technology Laboratory  Key differences between Energy’s contractor-operated and government-operated governance models are described in table 3  Energy’s FFRDCs use their own personnel systems, which Energy officials stated provide more flexibility for hiring and retaining qualified staff  Management within these FFRDCs can move staff in or out of scientific areas more quickly than government labs can, thereby providing greater agility to meet Energy’s needs in emerging science areas  For example, Energy’s lab oversight staff at Oak Ridge National Laboratory told us that use of lab contractors’ human resources management systems allows for workforce flexibilities to meet Energy’s needs  While these contractors have leeway in managing their human resources systems, Energy’s headquarters maintains oversight—through its contracting officers—over employee compensation  Energy’s FFRDC contractors manage and operate nearly all of the department’s government-owned national lab facilities—including day-to- day management of government-controlled facilities and real property  Lab operators used funding to complete minor construction projects, which cost $10 million or less  This funding comes from a percentage of science and technology projects’ funding, requires local Energy oversight office approval, and has streamlined project management requirements  In contrast, major infrastructure upgrades are funded through relatively long and complex line-item funding processes, and projects over $50 million are subject to more rigorous project management requirements  Energy’s labs use a small portion of their funding to initiate discretionary projects for science and technologies that will benefit sponsors in the long-term by maintaining the scientific and technical vitality of the laboratories  To maintain and enhance lab expertise, the National Defense Authorization Act for Fiscal Year 1991 authorized Energy’s contractor-operated labs receiving funding for national security programs to use a percentage of lab funds to perform lab-directed R&D of a creative and innovative nature  The actual percentages allowed to be used for lab-directed R&D are subject to Energy’s approval  Energy’s entities sponsor most national lab projects based on their needs and lab expertise  Typically, earlier foundational science projects are funded through a process whereby funding sponsors issue calls for proposals to Energy’s national labs  Interested scientific teams at labs provide proposals to conduct these projects for sponsor consideration  Sponsors then assess proposals for scientific merit and decide which teams receive funding to execute their projects  NNSA provides funding for later-phase nuclear technology development projects to its labs after agreement is made regarding objectives and deliverables for specific projects, according to Lawrence Livermore National Laboratory officials  Despite their flexibilities with regard to hiring and infrastructure decisions compared to government operated labs, Energy’s lab leadership and government oversight officials noted human resource and facilities related challenges, such as: Sufficiently compensating staff located in high-cost of living areas  For example, the labor market of the San Francisco area, where several Department of Energy national labs are located, is highly competitive for employers  Commercial firms offer salaries and compensation that typically exceed those of government-funded, contractor-operated labs, although Energy’s contractors have more pay flexibility than is allowed for Energy’s government employees  Obtaining government clearances in a timely manner  Energy’s NNSA oversight officials and lab management staff, in particular, cited this challenge, which they stated has led to a backlog of people needing clearances  Government hiring freeze constraining overall hiring  Officials at Energy’s government-operated National Energy Technology Laboratory reported that as a result of a government hiring freeze, the lab has increasingly hired private contractor staff to the point that more than half of the total lab staff is now comprised of contractor employees  Major infrastructure challenges at Energy labs  Energy reported in July 2018 that over half of all national lab buildings are in either substandard or inadequate condition  The Energy Inspector General also identified infrastructure modernization as one of Energy’s top management challenges  This finding followed a mandated commission’s report in 2015 that facilities and infrastructure across Energy’s national lab network were hampered by high levels of deferred maintenance and excess facilities  The majority of NASA’s science and technology facilities are operated within the governance framework of government-operated research centers, similarly to most DOD labs  While government-operated, they have been granted additional legislative flexibilities for hiring employees beyond those normally available to government entities  NASA locates its science and technology staff at four government- operated research centers, one contractor-operated FFRDC, and at five NASA centers assisting space and space flight development  These centers and the Jet Propulsion Laboratory—NASA’s sole sponsored FFRDC—execute NASA’s research missions including technology development in exploration and aeronautics  The differences between these two governance approaches are described in Table 4  NASA also works with Johns Hopkins University Applied Physics Lab, a UARC, to develop major space flight missions  The NASA Glenn Research Center—a government- operated laboratory—is currently developing solar electric propulsion technologies intended to allow manned and unmanned spacecraft to be propelled far beyond earth orbit using solar power  This project is developing large, flexible, radiation-resistant solar arrays that can be unfurled to capture solar energy powering fuel-efficient electrostatic thrusters  Scientists expect a system-level flight test within the next decade to demonstrate key technologies supporting NASA’s Lunar Orbital Platform-Gateway project, a platform to mature necessary short- and long-duration deep space exploration capabilities  Headquarters, including funding sponsors providing oversight for their individual projects  Title 51, Chapters 201 and 203 of U S  Code and technology efforts with DOD and other organizations, including use of NASA lab and test facilities  Operated by university contractor having sole source contract  Not permitted to compete against industry, except for operation of an FFRDC  property (originally part of DOD)  5-year contract renewable to 10 years total  Headquarters, including funding sponsors providing oversight for their individual projects  Title 51, Chapters 201 and 203 of U S  Code; 10 U S C  § 2304 (c) (3)(B) ; Federal Acquisition Regulation § 35 017 based NASA oversight staff  sponsors seeking FFRDC assistance with NASA approval  “Lab” as used in this context refers to science and technology organizations equivalent to NASA Research Centers, DOD UARCs and FFRDCs  Mission leadership officials at NASA Headquarters—including the Associate Administrators for Aeronautics Research, Human Exploration, Science and Space Technology—oversee NASA’s research centers as well as the Jet Propulsion Laboratory  These officials are responsible for technology programs providing funds to research centers and the Jet Propulsion Laboratory to support their specific mission areas  NASA’s science and technology project portfolios are based on the requirements and priorities established by NASA’s leaders in collaboration with key stakeholders in academia and industry among others  In planning their science and technology work, NASA’s Glenn Research Center officials noted that NASA research center directors consider the capabilities and resources—including staff and facilities—of other research centers to minimize redundant work  NASA depends on a highly skilled civil servant and contractor workforce to plan and execute its missions  Congress provided NASA with additional human resource authorities beyond those otherwise allowed for federal government personnel through the NASA Flexibility Act of 2004  We found in September 2008 that NASA sought this flexibility to ensure that it could hire and retain the workforce it desired  This law consisted of multiple provisions to address a range of human capital challenges and to strengthen all levels of the workforce  The provisions included incentives—including compensation—to allow NASA to compete successfully in the labor market with the private sector and reshape its workforce more effectively to support the Agency’s mission  NASA also employs a significant contractor workforce across its different centers  Glenn Research Center officials we interviewed stated their portfolio of science and technology projects—and funding—mostly aligns with NASA’s top requirements and priorities  They, along with NASA sponsors, create technology roadmaps and investment plans to determine their future projects  NASA policy requires that NASA’s scientific teams offer proposals for potential research and science and technology projects  This is similar in some ways to how many DOD and Energy centers must find sponsors willing to fund specific technology development projects, rather than receiving technology development funding for a given year  These proposals are reviewed by peer review teams, who identify for selecting officials those proposals they believe have the most scientific merit  Ames Research Center officials said they believe this process can foster innovation, encourage employees to keep skills honed, and mitigate complacency  Glenn Research Center officials said that while most of the work they conduct is for sponsored applied research or advanced technology development, about 2 percent of their science and technology budget is spent on early-stage scientific innovation  Recommended projects of this nature proposed by the research center are typically approved by headquarters officials, according to these Glenn officials  NASA provides technical grants for basic research and applied science to university scientists nationwide on a competitive basis, and also funds similar research done internally at research centers  As with DOD and Energy’s research centers, NASA officials have identified some key operating challenges, including: Aging infrastructure and facilities  The NASA Inspector General listed infrastructure area as one of the top five management and performance challenges facing NASA  Further, the Inspector General identified deficiencies with facilities planning and reported that about 80 percent of facilities at three of four NASA research centers are over 50 years old, while about half of the facilities at the Jet Propulsion Laboratory and the fourth research center are that old  Infrastructure projects and upgrades of $1 million or less are undertaken by research center management instead of at the NASA headquarters level  Construction above this threshold has significantly more requirements and is approved by NASA headquarters  Glenn Research Center officials indicated it is difficult to obtain funding for projects that exceed the minor infrastructure threshold, in part, due competition with major construction of facilities proposals from across the agency for limited funds  As a result, they put most of their efforts into sustaining existing infrastructure  Workforce shortages in key technical areas  As we found in May 2018, NASA has experienced workforce challenges on several major projects such as the Mars 2020 and Europa Clipper projects  Also, over 40 percent of NASA’s workforce is either eligible to retire now or will be eligible in the next 5 years  NASA headquarters officials noted that NASA’s workforce is aging because NASA has a low attrition rate—about 4 percent annually—and high numbers of staff stay several years beyond retirement  Further, in 2017, the NASA Inspector General found gaps in NASA’s workforce planning for specific capability areas and how workforce plans would meet future needs, and recommended that NASA establish standardized guidance defining the data and analyses for these planning efforts  NASA concurred with and identified its plan to implement this recommendation  However, NASA has not implemented this recommendation, according to the NASA Inspector General’s latest semiannual report to Congress  Congress provided DOD lab directors with key authorities to foster targeted, timely investments in the most pressing technology areas  Lab directors have used these authorities—such as laboratory initiated research and direct hire authorities—to varying degrees, but more needs to be done to facilitate innovation and efficiency  Specifically, service specific obstacles in the Air Force, Navy, and Army impede lab directors from capitalizing on laboratory initiated research authority to a greater extent  Service leadership can take actions to better understand and potentially remove barriers to more fully use laboratory initiated research tools  We are making the following three recommendations to DOD: The Secretary of the Air Force should assess the potential costs and benefits of implementing accounting system improvements that would allow the Air Force Research Laboratory to charge customers a fixed percentage fee on provided science and technology activities to the extent allowed under the laboratory initiated research authority  (Recommendation 1) The Secretary of the Navy should clarify whether and how to use the laboratory initiated research authority within the Capital Investment Program  (Recommendation 2) The Secretary of the Army should assess existing Army policy for laboratory initiated research authority and determine whether to implement changes to eliminate disincentives for lab usage of the authority  (Recommendation 3) We provided a draft of this report to DOD, Energy, and NASA for review and comment  Energy and NASA did not provide any comments on the draft report  In DOD’s written comments, reproduced in appendix III, DOD concurred with our three recommendations  Further, in its response to our third recommendation, DOD stated that the Army plans to initiate a study by January 2, 2019, regarding its use of the laboratory initiated research authority  According to DOD, the Army’s study will identify potential opportunities for policy improvements  We are sending copies of this report to the appropriate congressional committees and offices; the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; the Secretary of Energy; and the NASA Administrator  In addition, the report will be made available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions concerning this report, please contact me at (202) 512-4841  Contact points for our offices of Congressional Relations and Public Affairs may be found on the last page of this report  Staff members making key contributions to the report are listed in appendix IV  This report examines (1) how the Department of Defense (DOD) labs have used selected legislative authorities to foster innovation and efficiency and identify what barriers impede their use; (2) identifies and describes governance models used by selected DOD-sponsored federally funded research centers and university affiliated research centers; and (3) identifies and describes governance models used non-defense labs, specifically at the Department of Energy (Energy) and National Aeronautics and Space Administration (NASA)  To address the first objective, we selected four specific authorities for our review based on previous work identifying science and technology best practices and expedited lab hiring: Laboratory Initiated Research Authority  The authority provided in Section 219 of the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009, as implemented, provides lab directors with flexibility to fund projects in four allowable categories: basic and applied research; technology transition; workforce development; and revitalization, recapitalization, or repair or minor military construction of lab infrastructure  Laboratory Enhancement Pilot Program  Section 233 of the National Defense Authorization Act for Fiscal Year 2017 established a pilot program for lab directors to propose alternative and innovative methods that might lead to more effectively managing and operating labs and authorized lab directors to waive any regulation, restriction, requirement, guidance, policy, procedure, or departmental instruction that would affect implementation of these methods unless such implementation would be prohibited by a provision of an existing statute or common law  Direct Hire Authority  Four types of direct hire authorities authorized by Congress since 2008 are intended to provide a streamlined and accelerated hiring process to allow the labs to successfully compete with private industry and academia for high-quality scientific, engineering, and technician talent  Micro-purchase Authority  The Federal Acquisition Regulation states a preference for government agencies, to purchase and pay for micro-purchases of supplies or services using the government-wide commercial purchase card up to and at the micro-purchase threshold, but micro-purchases may be conducted using any of the simplified acquisition methods  While the FAR micro-purchase threshold was generally $3,500 at the time of our review, Congress increased this threshold to $10,000 for activities of DOD science and technology reinvention laboratories in Section 217 the National Defense Authorization Act for Fiscal Year 2017  Although Congress has provided additional legislative authorities to defense lab directors to address hiring, infrastructure, and technology transition challenges, the authorities that we covered in our review are the ones that our prior and current work have shown are currently, or have the potential to be, the most critical for supporting science and technology reinvention laboratories’ innovation mission within DOD labs  DOD lab leaders use these authorities to flexibly fund projects intended to facilitate research and development; propose alternative and innovative methods that might lead to more effective lab management; directly hire personnel at DOD labs including students currently enrolled in science, technology, engineering, and mathematics (STEM) programs; and expand critical science and technology purchases using simplified acquisition methods  To identify the extent to which DOD laboratories have used these authorities as well as to identify what potential barriers existed to using these authorities, we administered a survey to 44 STRL directors (or their equivalent) to collect information on the use of these specific authorities, their perceptions about the effectiveness of those authorities, and their perceptions about any barriers to using these authorities  The members of the population surveyed were the 44 defense laboratories defined as science and technology reinvention laboratories  For the purposes of our review, we defined laboratories as inclusive of Air Force technical directorates (10), Army warfare centers (17), and Navy warfare centers (17)  We emailed questionnaires to the laboratories beginning in late March 2018, and survey data collection ended in early May 2018, with 31 labs returning completed questionnaires, for an overall response rate of 71 percent at the laboratory level  We took steps to minimize the potential errors that the practical difficulties of conducting any survey may introduce  Nonresponse error can result when a survey fails to capture information from all population members selected into a survey sample  Of the 13 questionnaires not returned, 4 were Army warfare centers, and 9 were Air Force research directorates  Throughout the data collection period, we made multiple follow-up attempts by email and phone to those labs not yet responding  The Air Force Research Laboratory (AFRL) provided a single survey response for the entire laboratory enterprise  Not all returned questionnaires may have answers to every question applicable to a respondent  However, this question-level nonresponse did not exceed one for any of the questions applicable to all 31 labs  Because we selected the entire population of laboratories for our survey, our estimates are not subject to sampling error  We developed our list of the 44 labs in our population in consultation with DOD, and are confident that none were left out, so our or survey has no known sources of coverage error  We conducted pretests of the draft questionnaire with 3 laboratories in the population and made revisions to reduce the possibility of measurement error from differences in how questions were interpreted and the sources of information available to respondents  After reviewing the answers received, we also followed up as necessary with respondents to clarify apparent inconsistencies or other possible misreports, and made changes to responses where corrections were needed  A second, independent analyst checked the accuracy of all computer analyses to minimize the likelihood of errors in data processing  To obtain additional information on this objective, we reviewed relevant legislation which established or amended these authorities and reviewed applicable DOD and service policy documentation  Further, we collected military service related information on the usage of two authorities, such as: Spending data on the use of the laboratory initiated research authority  We gathered this information from DOD-mandated reports to Congress on the use of this authority and military service officials  We determined these data to be reliable based on reviews of agency documentation collected and interviews with agency officials  Data on the usage of direct hire authorities by the service laboratories  We collected direct hire data from each of the military services including the number of direct hire authority candidates hired as well as the number of direct hire positions the laboratories were authorized to hire  We determined these data to be reliable based on reviews of agency documentation collected and interviews with agency officials  We also used select findings from our May 2018 report where we evaluated DOD’s use of hiring authorities, including direct hire authority  More information about the scope and methodology of our prior work can be found in that report  In addition, we also collected information on military service proposals to utilize the laboratory enhancement pilot program authority  To obtain further information on department- and service-level involvement in and perspectives of defense laboratory authorities and challenges, we interviewed officials responsible for the management, execution, and oversight of DOD’s science and technology enterprise, including military service labs  At the Office of the Secretary of Defense and military department headquarters level, those responsible for the management and oversight of science and technology activities, we met with officials from the: Office of the Assistant Secretary of Defense for Research and DOD Defense Laboratories Office; Office of the Deputy Assistant Secretary of the Army for Research and Office of the Deputy Assistant Secretary of the Air Force for Science, Technology, and Engineering; Office of the Assistant Secretary of the Air Force for Financial Office of the Deputy Assistant Secretary of the Navy for Research, Development, Test, and Evaluation; and Office of the Budget, within the Office of the Assistant Secretary of the Navy for Financial Management and Comptroller We also met with military department lab officials responsible for the management and execution of science and technology activities from the: Army Research, Development and Engineering Command; Army Research Laboratory; Army Aviation and Missile Research, Development, and Engineering Air Force Research Laboratory; Naval Research Laboratory; Naval Surface Warfare Center, Headquarters; and Naval Surface Warfare Center, Carderock Division To identify and describe governance models used by selected DOD- sponsored federally funded research centers (FFRDCs) and university affiliated research centers (UARCs), we focused our review on the 3 FFRDCs designated as research and development labs as well as all 13 UARCS sponsored by DOD entities  We reviewed appropriate sections of the FAR language related to FFRDCs and UARCs, DOD guidance for working with FFRDCs and UARCs, relevant contracts, and performance assessments  Further, we met with officials from the office of the Deputy Director, OSD Studies and Federally Funded Research & Development Centers Management and Office to discuss overall FFRDC and UARC management, policies, and challenges facing FFRDCs and UARCs  We interviewed officials at selected research and development FFRDCs and UARCS to discuss their experience conducting DOD research and interactions with their customers, such as defense program executive offices  We met with officials at the two major research and development lab FFRDCs—The Lincoln Laboratory at the Massachusetts Institute of Technology (MIT) and the Software Engineering Institute at Carnegie Mellon University  We also selected a university affiliated research center sponsored by the Army and Navy: The Applied Physics Laboratory at Johns Hopkins University and the Institute for Soldier Nanotechnologies also at the MIT  To identify and describe governance models by non-defense labs, we selected Energy and NASA to focus our efforts  We identified 17 Energy national labs and 4 NASA research centers conducting basic and applied research similar to DOD labs  These agencies, along with DOD, represent 3 of the top 4 agencies in terms of average federal research and development spending from fiscal years 2015 to 2017  In our August 2016 GAO Technology Readiness Assessment Guide, we drew heavily from DOD, NASA, and Energy for best practices, terminology, and examples  This contributed to our decision to focus on Energy and NASA’s research entities in this laboratory governance review  We did not include the fourth agency—the National Institutes of Health—in our review because it is not as similar to DOD  We also reviewed relevant Energy and NASA guidance as well as relevant FAR sections  At Energy, we met with officials from the National Nuclear Security Administration which is semi-autonomous entity within Energy responsible for managing the nation’s nuclear weapons and nuclear security  We also met with Officials from the Office of Science, a program office responsible for supporting energy related fundamental science and research  To gain further insights on operating structures, funding arrangements, and their overall experience we met with lab leadership at selected Energy labs which were chosen based on initial discussions with agency officials and our review of past GAO work: Oak Ridge National Laboratory, Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, and National Energy Technology Laboratory (the sole Energy government owned and operated laboratory) We also met with leadership from Battelle Memorial Institute, which is the sole or joint contract manager for five Energy national labs including Oak Ridge National Laboratory  In addition, Battelle is an integrated subcontractor at Lawrence Livermore National Laboratory  At NASA, we met with officials with NASA’s Science Mission Directorate and Mission Support Directorate to discuss overall research center management and operations  We also leveraged ongoing and recently completed work at GAO to gain additional insight on NASA’s operations such as human capital management  Almost all of NASA’s research, space, and space flight centers conduct research and development activities  However, we focused our review on four research centers where NASA primarily conducts its aeronautics research, which has substantial overlap with DOD activities  To gain additional insight into the experience of lab leaders at NASA research centers, we met with officials at NASA’s Glenn Research Center and Ames Research Center  In addition, we also met with officials at the NASA Jet Propulsion Center, which is the only NASA-sponsored FFRDC  We conducted this performance audit from July 2017 to December 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  Office of the Secretary of Defense (OSD) In addition to the contact named above, Christopher R  Durbin (Assistant Director); Charlie Shivers, III (Analyst-in-Charge); Emily Bond; Lorraine Ettaro; Carl Ramirez; Sylvia Schatz; Sean Seales; Brian Smith; and Robin Wilson made significant contributions to this report 

ground_truth: Military families with special medical and educational needs face unique challenges because of their frequent moves. To help assist these families, DOD provides services plans, which document the support a family member requires. The National Defense Authorization Act for Fiscal Year 2017 included a provision for GAO to review the Services' EFMPs, including DOD's oversight of these programs. This report examines the extent to which (1) each Service provides family support and (2) the Services monitor and DOD evaluates assignment coordination and family support. GAO analyzed DOD and Service-specific EFMP guidance and documents; analyzed fiscal year 2016 EFMP data (the most recent available); visited seven military installations, selected for their large numbers of military-connected students; and interviewed officials responsible for implementing each Service's EFMP, as well as officials in OSN that administer DOD's EFM policy. The support provided to families with special needs through the Department of Defense's (DOD) Exceptional Family Member Program (EFMP) varies widely for each branch of Military Service. Federal law requires DOD's Office of Special Needs (OSN) to develop a uniform policy that includes requirements for (1) developing and updating a services plan for each family with special needs and (2) resources, such as staffing, to ensure an appropriate number of family support providers. OSN has developed such a policy, but DOD relies on each Service to determine its compliance with the policy. However, Army and Navy officials said they have not received feedback from OSN about the extent to which their Service-specific guidance complies. Federal internal control standards call for developing clear policies to achieve agency goals. In addition, DOD's most recent annual reports to Congress do not indicate the extent to which each Service provides services plans or allocates sufficient resources for family support providers. According to GAO's analysis, the Military Services have developed relatively few services plans, and there is wide variation in the number of family support providers employed, which raises questions about potential gaps in services for families with special needs (see table). Each Service uses various mechanisms to monitor how servicemembers are assigned to installations (assignment coordination) and obtain family support, but DOD has not established common performance measures to assess these activities. DOD has taken steps to better support families with special needs, according to the DOD officials GAO interviewed. For example, DOD established a working group to identify gaps in services. However, OSN officials said that DOD lacks common performance measures for assignment coordination and family support because the Services have not reached consensus on what those measures should be. In addition, OSN does not have a process to systematically evaluate the results of the Services' monitoring activities. Federal internal control standards call for assessing performance over time and evaluating the results of monitoring activities. Without establishing common performance measures and assessing monitoring activities, DOD will be unable to fully determine the effect of its efforts to better support families with special needs and the adequacy of the Services' EFMPs as required by federal law. GAO makes a total of three recommendations to DOD. DOD should assess and report to Congress on the extent to which each Service provides sufficient family support personnel and services plans, develop common performance metrics for assignment coordination and family support, and evaluate the results of the Services' monitoring activities. DOD agreed with these recommendations and plans to develop performance metrics for assignment coordination and develop plans to evaluate the Services' monitoring activities.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.

Report 1: The EFMP provides support to families with special needs at their current and proposed locations  Servicemembers relocate frequently, generally moving every 3 years if in the Army, Marine Corps, and Navy, and every 4 years if in the Air Force  In fiscal year 2016, the Military Services relocated approximately 39,000 servicemembers enrolled in the EFMP to CONUS installations  To implement DOD’s policy on support for families with special needs, DOD requires each Service to establish its own EFMP for active duty servicemembers  EFMPs are to have three components—identification and enrollment, assignment coordination, and family support  Identification and enrollment: Medical and educational personnel at each installation are responsible for identifying eligible family members with special medical or educational needs to enroll in the EFMP  Once identified by a qualified medical provider, active duty servicemembers are required to enroll in their service’s EFMP  Servicemembers are also required to self-identify when they learn a family member has a qualifying condition  Assignment coordination: Before finalizing a servicemember’s assignment to a new location, DOD requires each Military Service to consider any family member’s special needs during this process, including the availability of required medical and special educational services at a new location  Family support: DOD requires each Military Service’s EFMP to include a family support component through which it helps families with special needs identify and gain access to programs and services at their current, as well as proposed, locations  Servicemembers assigned to a joint base would receive family support from the Service that is responsible for leading that installation  For example, an Airman assigned to a joint base where the Army is the lead would receive family support from the Army installation’s EFMP  As required by the NDAA for Fiscal Year 2010, DOD established the Office of Community Support for Military Families with Special Needs (Office of Special Needs or OSN) to develop, implement, and oversee a policy to support these families  Among other things, this policy must (1) address assignment coordination and family support services for families with special needs; (2) incorporate requirements for resources and staffing to ensure appropriate numbers of case managers are available to develop and maintain services plans that support these families; and (3) include requirements regarding the development and continuous updating of a services plan for each military family with special needs  OSN is also responsible for collaborating with the Services to standardize EFMP components as appropriate and for monitoring the Services’ EFMPs  OSN has been delegated the responsibility of implementing DOD’s policy for families with special needs by the Undersecretary of Defense for Personnel and Readiness through the Assistant Secretary for Manpower and Reserve Affairs according to DOD officials  Currently, OSN is administered under the direction of the Deputy Assistant Secretary of Defense for Military Community and Family Policy through the Office of Military Family Readiness Policy  In addition, each Military Service has designated a program manager for its EFMP who is also responsible for working with OSN to implement its EFMP (see fig  1)  DOD’s guidance for the EFMP (1) identifies procedures for assignment coordination and family support services; (2) designates the Assistant Secretary of Defense for Manpower and Reserve Affairs as being responsible for monitoring overall EFMP effectiveness; (3) assigns the OSN oversight responsibility for the EFMP, including data review and monitoring; and (4) directs each Service to develop guidance for overseeing compliance with DOD requirements for their EFMP  Table 1 provides an overview of the procedures each Service must establish for the assignment coordination and family support components of the EFMP  As a part of its guidance for monitoring military family readiness programs, DOD also requires each Military Service to certify or accredit its family readiness services, including family support services provided through the EFMP  In addition, DOD states that each Service must balance the need for overarching consistency across EFMPs with the need for each Service to provide family support that is consistent with their specific mission  To accomplish this, each Service is required to jointly work with DOD to develop a performance strategy, which is a plan that assesses the elements of cost, quality, effectiveness, utilization, accessibility, and customer satisfaction for family readiness services  In addition, each Military Service is required to evaluate their family readiness services using performance goals that are linked to valid and reliable measures such as customer satisfaction and cost  DOD also requires each Service to use the results of these evaluations to inform their assessments of the effectiveness of their family readiness services for families with special needs  According to DOD officials, each Military Service provides family support services in accordance with DOD guidance as well as Service-specific guidance  However, we found wide variation in each Service’s requirements for family support personnel as well as the practices and expectations of EFMP staff  As a result the type, amount, and frequency of assistance enrolled families receive varies from Service to Service and when a servicemember from one Service is assigned to a joint base led by another Service (see table 2)  For example, in terms of a minimum level of contact for families with special needs enrolled in the EFMP, the Services vary in the frequency with which they require family support providers to contact families with special needs: The Marine Corps specifies a frequency (quarterly) with which families with special needs should be contacted by their family support providers  The Air Force has each installation obtain a roster of families with special needs enrolled in the EFMP on a monthly basis, but it does not require family support providers to, for example, use this information to regularly contact these families  The Navy assigns one of three service levels to each family member enrolled in the EFMP  These service levels are based on the needs of each family with special needs; family support providers are responsible for assigning a “service level” that directs the frequency with which the family must be contacted  The Army has no requirements for how often families with special needs should be contacted  The Services also vary as to whether they offer legal assistance to families with special needs as follows: The Marine Corps employs two attorneys who can represent families with special needs who fail to receive special education services from local school districts, as specified in their children’s individualized education programs (IEP)  They can also advise EFMP-enrolled families on their rights and options if a family believes their child needs special education services from a local school district (e g , an IEP)  The Air Force, Army, and Navy choose not to employ special education attorneys  Officials with whom we spoke said families with special needs in these Services can receive other types of assistance that may help them resolve special education legal issues  For example, Air Force officials said servicemembers and their families can receive support from attorneys that provide general legal assistance on an installation, Army officials said installation EFMP managers can refer families with special needs to other organizations that provide legal support, and Navy officials said families can find support through working with their installation’s School Liaison Officers  The NDAA for Fiscal Year 2010 requires DOD’s policy to include requirements regarding the development and continuous updating of a services plan (SP) for each family with special needs, and DOD has specifically required these plans as part of the provision of family support services  These plans describe the necessary services and support for a family with special needs and document and track progress toward meeting related goals  According to DOD guidance, these plans should also document the support provided to the family, including case notes  In addition, the DOD reference guide for family support providers emphasizes that timely, up-to-date documentation is especially important each time a family relocates, as military families regularly do  Therefore, SPs are an important part of providing family support during the relocation process, and provide a record for the gaining installation  Requiring timely and up-to-date documentation is consistent with federal internal control standards, which state that agencies should periodically review policies, procedures, and related control activities for continued relevance and effectiveness in achieving their objectives  SPs follow families with special needs each time they relocate and without timely and up-to-date documentation, DOD cannot ensure that all families continue to receive required medical and/or special educational services once they relocate to another installation  For every Service the number of SPs was relatively few when compared to the number of servicemembers (known as sponsors) or the number of family members enrolled in the EFMP (see table 3)  The Services and OSN provided a range of reasons as to why the Services do not develop and maintain a SP for each family with special needs  For example, Air Force officials said their family support providers consider the needs of each family with special needs before determining whether a SP will help them receive the required services  In addition, Army and Marine Corps officials said they may not develop these plans if families do not request them  Further, according to a Navy official, some families lack the required SPs because installations may not have the staff needed to develop them—even though DOD requires the Services to maintain sufficient staff and certify their EFMPs  OSN officials with whom we spoke also said that the Services may not have developed many SPs during fiscal year 2016 because DOD had not yet approved a standardized form that could be used to meet this requirement  Finally, OSN officials also said that each family with special needs enrolled in the EFMP may not need a SP because their condition does not require this type of family support  To meet requirements of the NDAA for Fiscal Year 2010, in April 2017, DOD issued to the Services guidance that directed them to “rogram, budget, and allocate sufficient funds and other resources, including staffing,” to meet DOD’s policy objectives for the EFMP  According to OSN officials, DOD relies on each Service to determine what level of funds and resources is sufficient and what constitutes an appropriate number of family support personnel  To determine family support providers and related personnel staffing levels, the Service officials with whom we spoke said they consider a number of factors, including the number of families with special needs enrolled in the EFMP at any given installation (see app  II for more information about the EFMP data by installation)  See Table 4 for a summary of EFMP family support providers and other key personnel at CONUS installations  As required by DOD, all of the Services employ family support providers to assist families with special needs  In addition, some Services employ additional personnel to support implementation of the EFMP (see sidebar)  For example, the Air Force employs family support coordinators to administer its EFMP and no other personnel are dedicated to assisting these coordinators or enrolled families  The Army employs “system navigators” who provide individualized support to families with special needs at selected installations through its EFMP, as well as other personnel to administer the EFMP  workers at most of its CONUS installations to administer individualized support to families with special needs  In addition, the Marine Corps employs program managers, administrative assistants, as well as training and education outreach specialists  The Navy contracts regional case liaisons and case liaisons at selected CONUS installations to administer individualized support to families with special needs  In addition, the Navy employs collateral duty case liaisons who assist with the delivery of family support services at all other CONUS installations  Senior OSN officials said they rely on each Service to determine the extent to which its EFMP complies with DOD’s policy for families with special needs because they consider OSN to be a policy-making organization that is not primarily responsible for assessing compliance  In addition, these officials said the Services need flexibility to implement DOD’s policy for families with special needs because they each have unique needs and the number of enrolled families in the EFMP is constantly changing  However, DOD has not developed a standard for determining the sufficiency of funding and resources each Service allocates for family support  Air Force officials at one of the installations we visited said the Air Force identified the lack of staff and funding to provide individualized support to most families with special needs as an issue  In addition, officials from the Army and Navy said they have not received any guidance from OSN officials about their Service-specific guidance, including requirements for resources and services plans  Further, the Services may not know the extent to which their Service- specific guidance complies with DOD’s policy for families with special needs  The NDAA for Fiscal Year 2010 requires DOD to identify and report annually to the congressional defense committees on gaps in services for military families with special needs and to develop plans to address these gaps  However, DOD’s most recent reports to the congressional defense committees did not address the relatively few SPs being created for families with special needs, or whether the Services are providing sufficient resources to ensure an appropriate number of family support providers  Federal internal control standards require that agencies establish control activities, such as developing clear policies, in order to accomplish agency objectives such as those of the Services’ EFMPs  Without fully identifying and addressing potential gaps in family support across these programs, some families with special needs may not get the assistance they require, particularly when they relocate  Each Service monitors EFMP assignment coordination and family support using a variety of mechanisms, such as regularly produced internal data reports  However, DOD has not yet established common performance measures to track the Services’ progress in implementing its standard procedures over time or developed a process to evaluate the overall effectiveness of each Service’s assignment coordination and family support procedures  DOD requires each Service to monitor implementation of their EFMP, including their procedures for assignment coordination and family support  To comply with this requirement, each Service has developed guidance that establishes monitoring protocols and assigns oversight responsibilities  Officials from each Service told us they use internal data reports from each installation to monitor assignment coordination and family support  To monitor assignment coordination, officials from each Service told us their headquarters reviews proposed assignment locations for families with special needs enrolled in the EFMP  These officials said monitoring proposed assignment locations helps ensure that enrolled families will be able to access required services at their new installations  In addition, Army officials said each Army unit commander is responsible for tracking the number of families with special needs that have expired enrollment paperwork because it affects assignment coordination worldwide  Several years ago, the Army determined that 25 percent of soldiers (over 13,000) enrolled in the EFMP had expired enrollment paperwork, complicating the task of considering each enrolled family’s special medical or educational needs as part of proposed relocations  In response, in August 2011, the Army revised its policies and procedures for updating enrollment paperwork which would help ensure a family member’s special needs are considered during the assignment coordination process  To monitor family support provided by installations worldwide, each Military Service told us they use a variety of mechanisms (see table 5)  The Marine Corps pays particular attention to customer satisfaction  Marine Corps officials told us that every three years Marine Corps headquarters administers a survey of family members enrolled in the EFMP  We previously reported that organizations may be able to increase customer satisfaction by better understanding customer needs and organizing services around those needs  This survey is one of the primary ways Marine Corps headquarters measures customer satisfaction with family support services at installations worldwide  Marine Corps officials also said this survey helps ensure its EFMP is based on the current needs of families with special needs  To improve its oversight of the EFMP and implement its policy for families with special needs, DOD, through OSN, has several efforts under way to standardize the Services’ procedures for assignment coordination and family support  However, DOD has not developed common performance measures to monitor its progress toward these efforts and has not developed a process for assessing the Services’ related monitoring activities  Federal internal control standards emphasize the importance of assessing performance over time and evaluating the results of monitoring activities  To help improve family member satisfaction by addressing gaps in support that may exist between Services, OSN has begun to standardize procedures for assignment coordination and family support  To date, OSN’s efforts have focused on ensuring each Service’s EFMP considers the needs of family members during the assignment process and helps family members identify and gain access to community resources  According to OSN’s April 2017 Report to Congress, the long-term goal of these efforts is to help ensure that all families with special needs enrolled in the EFMP receive the same level of service regardless of their Military Service affiliation or geographic location  In addition, OSN officials told us its standardized procedures will also help DOD perform required oversight by improving its access to Service-level data and its ability to validate each Service’s monitoring activities  To date, efforts to standardize assignment coordination and family support have included efforts such as developing new family member travel screening forms which will be the official documents used during the assignment coordination process and completing a DOD-wide customer service satisfaction survey on EFMP family support (see table 6)  Despite its efforts to begin standardizing assignment coordination and family support services, DOD is unable to measure its progress in standardizing assignment coordination and family support procedures for families with special needs and assessing the Services’ performance of these processes because it has not yet developed common metrics for doing so  Federal internal control standards emphasize the importance of agencies assessing performance over time  We have also reported on the importance of federal agencies engaging in large projects using performance metrics to determine how well they are achieving their goals and to identify any areas for improvement  By using performance metrics, decision makers can obtain feedback for improving both policy and operational effectiveness  Additionally, by tracking and developing a baseline for all measures, agencies can better evaluate progress made and whether or not goals are being achieved—thus providing valuable information for oversight by identifying areas of program risk and causes of risks or deficiencies to decision makers  Through our body of work on leading performance management practices, we have identified several attributes of effective performance metrics relevant to OSN’s work (see table 7)  OSN officials said each Service is currently responsible for assessing the performance of its own EFMP, including the development of Service- specific goals and performance measures  OSN officials said that they recognize the need to continually measure the department’s progress overall in implementing its policy for families with special needs, and are considering ways to do so  They also said they have encountered challenges to developing common performance measures  In addition, OSN officials said its efforts to reach consensus among the Services about performance measures for the overall EFMP are still ongoing because each Service wants to maintain its own measures, and DOD has not required them to reach a consensus  Absent common performance measures, DOD is unlikely to fully determine whether its long-standing efforts to improve support for families with special needs are being implemented as intended  DOD requires each Service to monitor its own family readiness programs, including procedures for assignment coordination and family support through the EFMP, but lacks a systematic process to evaluate the results of these monitoring activities  To monitor family readiness services, as required by DOD, each Service must accredit or certify its family support services, including the EFMP, using standards developed by a national accrediting body not less than once every 4 years  In addition, personnel from each Service’s headquarters are required to periodically visit installations as a part of their monitoring activities for assignment coordination, among other things  The Services initially had the Council on Accreditation accredit family support services provided through their installations’ EFMPs using national standards developed for military and family readiness programs, according to the officials with whom we spoke  However, by 2016, each Service was certifying installations’ family support services using standards that meet those of a national accrediting body, Service-specific standards, and best practices  According to officials from each Service with whom we spoke, this occurred due to changes in the funding levels allocated to this activity  Table 8 provides an overview of the certification process currently being used by each Service  OSN officials said they do not have an ongoing process to systematically review the results of the Services’ activities, including the certification of EFMPs because they choose to rely on the Services to develop their own monitoring activities and ensure they provide the desired outcomes  In doing so, DOD allows each Service to develop its own processes for certifying installations’ family support services, including the selection of standards  In addition, OSN officials told us that efforts to standardize certification of EFMPs are ongoing because the Military Services have not been able to reach consensus on a set of standards that can be used across DOD for installations’ family support services  Further, OSN has not established a process to assess the results of the Services’ processes for certifying installations’ family support services  Federal standards for internal control state that management should evaluate the results of monitoring efforts—such as those the Services are conducting on their own—to help ensure they meet their strategic goals  The lack of such a process hampers OSN’s ability to monitor the Services’ EFMPs and determine the adequacy of such programs as required by the NDAA for Fiscal Year 2010  OSN’s job of developing a policy for families with special needs that will work across DOD’s four Services is challenging given the size, complexity, and mission of the U S  military  It has had to consider, among other things, the Services’ mission requirements, resource constraints, and the myriad demands on servicemembers and their families during their frequent relocations  Anything that further complicates a relocation—such as not receiving the required family support services for family members with special needs—potentially affects readiness or, at a minimum, makes an already stressful situation worse  By providing little direction on how the Services should provide family support or what the scope of family support services should be, some servicemembers get more—or less—from the EFMP each time they relocate, including when a servicemember from one Service is assigned to a joint base led by another Service  By largely deferring to the Services to design, implement, and monitor their EFMPs’ performance, DOD cannot, as required by the NDAA for Fiscal Year 2010, fully determine the adequacy of the Services’ EFMPs in serving families with special needs, including any gaps in services these families receive, because it has not built a systematic process to do so  Instead, it relies on the Services to self-monitor and address, within each Service, the results of monitoring activities  However, because servicemembers relocate frequently and often depend on the EFMP of a Service other than their own, a view of EFMP performance across all of the Services is essential to ensuring, for example, that relocating servicemembers get consistent EFMP service delivery no matter where they are stationed  Evaluating and developing program improvements based on the results of the Services’ monitoring would help DOD ensure the Services’ EFMPs achieve the desired outcomes and improve its ability to assess the overall effectiveness of the program  We are making the following three recommendations to DOD: We recommend the Secretary of Defense direct the Office of Special Needs (OSN) to assess the extent to which each Service is (1) providing sufficient resources for an appropriate number of family support providers, and (2) developing services plans for each family with special needs, and to include these results as part of OSN’s analysis of any gaps in services for military families with special needs in each annual report issued by the Department to the congressional defense committees  (Recommendation 1) We recommend that the Secretary of Defense direct the Office of Special Needs (OSN) to develop common performance metrics for assignment coordination and family support, in accordance with leading practices for performance measurement  (Recommendation 2) We recommend that the Secretary of Defense implement a systematic process for evaluating the results of monitoring activities conducted by each Service’s EFMP  (Recommendation 3) We provided a draft of this report to the Department of Defense (DOD) for comment  DOD provided written comments, which are reproduced in appendix IV  DOD also provided technical comments, which we incorporated as appropriate  DOD agreed with all three of our recommendations  In its written comments, DOD stated that additional performance metrics need to be developed for assignment coordination and that it is in the process of measuring families’ satisfaction with family support provided through the EFMP  DOD also stated that it is developing plans for evaluating the results of each Service’s monitoring activities for the EFMP  We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and Education, and other interested parties  The report also is available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (617) 788-0580 or nowickij@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix V  The National Defense Authorization Act (NDAA) for Fiscal Year 2017 includes a provision for GAO to assess the effectiveness of the Department of Defense’s (DOD) Exceptional Family Member Programs (EFMP)  This report focuses on the assignment coordination and family support components of the EFMP for dependents with special needs and examines: (1) the extent to which each Service has provided family support as required by DOD, and (2) the extent to which the Services monitor and DOD evaluates assignment coordination and family support  To address these objectives, we used a variety of data collection methods  Key methods are described in greater detail below  For both objectives, we reviewed relevant federal laws, regulations, and DOD guidance and documentation that pertain to the EFMP, including the following: The NDAA for Fiscal Year 2010, which established the Office of Special Needs and defined program requirements for assisting families with special needs, including assignment coordination and family support  DOD’s guidance for administering the EFMP  We assessed how DOD implements the requirements in the NDAA for Fiscal Year 2010; how each Service implements assignment coordination and family support; and how the Services and DOD monitor assignment coordination and family support using performance measures  Specially, we reviewed DOD Instruction 1315 19 - Exceptional Family Member Program; Service-specific guidance and related documents from the Air Force, Army, Marine Corps, and Navy; and DOD Instruction 1342 22 - Military Family Readiness  Standards for internal control in the federal government related to the documentation of responsibilities through policies, performance measures, and evaluating the results of monitoring activities  We compared each Service’s procedures for monitoring assignment coordination and family support to these standards  To determine the extent of the Services’ EFMP family support, we obtained and analyzed fiscal year 2016 EFMP data (the most recent available) for each Service  We reviewed DOD policy to identify data variables that each Service maintains related to its EFMP  We used these data to summarize key characteristics of each Service’s EFMP  The selected variables provided Service-wide and installation-specific EFMP information on, the number of continental United States (CONUS) and outside the continental United States (OCONUS) installations; the number of servicemembers (sponsors) enrolled in the EFMP; the number of family members with special needs enrolled in the EFMP; the number of EFMP family support personnel; and the number of services plans created for families with special needs enrolled in the EFMP  We determined that the selected data variables from each Service are sufficiently reliable for the purposes of providing summary results about family support for fiscal year 2016  To learn more about how the Services implement their EFMPs, we visited seven installations in five states  We selected the seven installations based on their location in states with the largest number of military- connected students in school year 2012-2013 (the most recent available and reliable data) or in states with the largest percentage of students enrolled in U S  DOD schools as of May 2017, as well as their status as a joint base  At each installation, we interviewed installation officials, EFMP managers, selected family support personnel, and family members and caregivers enrolled in the program  In states we visited that had the largest number of military-connected students, the EFMP personnel we interviewed collectively served 66 percent of students who attend local public schools and 42 percent of the students attending U S  DOD schools  To obtain illustrative examples about how the EFMP serves families with special needs, we conducted seven group interviews with EFMP-enrolled family members and caregivers (one at each of the seven installations we visited)  Using a prepared script, we asked participants to describe how they were identified and enrolled in the EFMP, how they were assigned to new installations, and the types of family support services they received  We also asked about how these services aligned with their family member’s EFMP-eligible condition, the benefits and challenges they experienced, as well as their overall satisfaction  A total of 38 self- selected volunteers participated in the seven group discussions  While the participants in these groups included a variety of family members and caregivers, the number of participants and groups were very small relative to the total number of family members enrolled in the EFMP  Their comments are not intended to represent all EFMP-enrolled family members or caregivers  Other EFMP-enrolled family members and caregivers may have had other experiences with the program during the same period  Finally, for both objectives, we conducted interviews with a variety of DOD, Service-level, and nonfederal officials  We spoke with DOD officials from the Office of the Assistant Secretary of Defense–Offices of Manpower and Reserve Affairs, Military Community and Family Policy, Military Family Readiness Policy, and Special Needs  We also spoke with EFMP Managers from Air Force, Army, Marine Corps, and Navy headquarters  We also met with officials from selected national military family advocacy organizations including the National Military Family Association; the Military Family Advisory Network; and the Military Officers Association of America to discuss the EFMP  We conducted this performance audit from February 2017 to May 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  Each Service has an Exceptional Family Member Program (EFMP) that provides support to military families with special needs  The tables below present the following information on selected EFMP and family support categories for each Service’s program at continental United States (CONUS) and outside the continental United States (OCONUS) installations in fiscal year 2016: City, state or country; Number of exceptional family members; Number of family support providers (by Full-Time Equivalent); Number of family support provider vacancies; Number of services plans; Number of indirect contacts; and Number of direct contacts  The information below is listed sequentially in alphabetical order by Service  We held small group discussions with Exceptional Family Member Program (EFMP) participants at the seven military installations we visited  Family members and caregivers who attended each session reported they had children or spouses with EFMP-eligible conditions  The discussion group participants were self-selected; and their comments are not intended to represent all EFMP -enrolled family members or caregivers in fiscal year 2016  In addition, other EFMP -enrolled family members and caregivers may have had different experiences with the program during the same period  There were a total of 38 participants representing all the Services  The following issues were discussed by one or more participants during the small group discussions at the installations we visited  The issues that emerged relate to the current and future overall effectiveness of the EFMP  Overall Satisfaction with EFMP (Discussed by 30 of 38 participants): Measure of participants’ approval of the family support services offered and experience with the EFMP  Many participants expressed overall satisfaction with the EFMP  Several participants expressed dissatisfaction with the EFMP  A participant expressed dissatisfaction with the lack of consistency in the provision of family support services (i e , special education advocacy) across installations  School Liaison Officers (Discussed by 20 of 38 participants): Serve as the primary point of contact for school-related matters as well as assist military families with school issues  Several participants noted that they received no response to their request for assistance from their School Liaison Officer or they only received general information  Several participants said School Liaison Officers were not helpful  Some participants found School Liaison Officers were helpful  Some participants were unaware of School Liaison Officers being available at their installation and the service(s) they provide  A few participants said School Liaison Officers did not follow up on requests for information  A participant noted there seems to be a disconnect between family support services provided through the EFMP and services provided by School Liaison Officers  Family Support Personnel (Discussed by 12 of 38 participants): Provide information and referral to military families with special needs  Some participants at one installation noted that the EFMP was understaffed  Some participants at one installation noted high turnover of family support personnel  Some participants noted family support personnel did not provide support for their family with special needs  Stigma (Discussed by 12 of 38 participants): A perception that participating in the EFMP may limit a soldier’s assignment opportunities and/or compromise career advancement  Several participants believe there is still stigma associated with participating in the EFMP  Some participants said participating in the EFMP has not affected career advancement  Assignment Coordination (Discussed by 10 of 38 participants): The assignment of military personnel in a manner consistent with the needs of armed forces that considers locations where care and support for family members with special needs are available  Some participants found the assignment coordination process challenging  Some participants described limitations with the assignment coordination process  A few participants noted there is a lack of information among families with special needs regarding how to express the need for stabilization and /or continuity of care  A few participants cited the challenges of assignment coordination as contributing to their decision to retire  One participant commented that the opinion of a medical professional was not reflected in the assignment coordination process  Special Education Services (Discussed by 10 of 38 participants): The provision of staff capable of assisting families with special needs with special education and disability law advice and/or assistance and attendance at individualized education program (IEP) meetings where appropriate  Several participants who had a family support provider assist them with preparing for or attending a school-based meeting, including IEP meetings, spoke positively of their experience(s)  Some participants at one installation agreed that assistance from family support providers during meetings with school officials regarding special education services is helpful  A few participants who were unable to get assistance with special education services from the EFMP sought the services of private attorneys at their own expense  Family Support Services (Discussed by 9 of 38 participants): The non-clinical case management delivery of information and referral for families with special needs, including the development and maintenance of a services plan  Some participants found that family support providers were helpful  Some participants could not identify needed resources or were unaware of the resources or services available to them  One participant noted that the family support provider had minimal contact  One participant said navigating the system can be challenging  Surveys (Discussed by 8 of 38 participants): The process of collecting data from a respondent using a structured instrument and survey method to ensure the accurate collection of data  Several participants noted that they had not or rarely had the opportunity to evaluate the family support services provided through the EFMP  One participant noted that comment cards used by each service are not effective for evaluating the EFMP  Warm hand-off (Discussed by 6 of 38 participants): Assistance to identify needed supports or services and facilitating the initial contact or meeting with the next program  Many participants at one installation agreed that the warm hand-off process worked well for them  Several participants said they found the warm hand-off process helpful when moving from one installation to the next  Outreach (Discussed by 5 of 38 participants): Developing partnerships with military and civilian agencies and offices (local, state, and national), improving program awareness, providing information updates to families, and hosting and participating in EFMP family events  Some participants found it difficult to obtain information regarding the types of family support services that are available  A few participants noted that communications regarding the EFMP were not targeted to address their needs  A few participants noted communications regarding the EFMP are untimely, (e g , newsletters not issued periodically)  Joint Base Family Support Services (Discussed by 1 of 38 participants): Family support services provided by the lead Service of the Joint Base that is different from that of the servicemember enrolled in the EFMP  One participant said that using family support services on joint bases may pose a challenge as each Service has different rules and procedures and as a result provides different types of family support services  In addition to the contact named above, Bill MacBlane (Assistant Director), Brian Egger (Analyst-in-Charge), Patricia Donahue, Holly Dye, Robin Marion, James Rebbe, Shelia Thorpe, and Walter Vance made significant contributions to this report  Also contributing to this report were Lucas Alvarez, Bonnie Anderson, Connor Kincaid, Brian Lepore, Daniel Meyer, and Mimi Nguyen 


Report 2:  Even in the 32 states where the state Medicaid agencies reported reviewing information about critical incidents, 20 states were unable to provide the actual number of critical incidents that occurred in assisted living facilities  State Medicaid agencies’ definitions of critical incidents also varied  As illustrated in Figure 4, all 48 states cited physical assault, emotional abuse, and sexual assault or abuse as a critical incident in their largest programs providing assisted living services in 2014  However, for other types of incidents, several states did not identify the incident as critical, including discharge and eviction from the facility (not a critical incident in 24 states), medication errors (not a critical incident in 7 states), and unauthorized use of seclusion, (not a critical incident in 6 states)  For other serious incidents, a relatively small number of states did not identify the incident as critical, such as unexplained death (not a critical incident in 3 states) and missing beneficiaries (not a critical incident in 2 states)  See appendix IV for a full list of the beneficiary-related incidents and the number of states that identify each as critical  Although half of the 48 states that cover assisted living services did not consider discharges or evictions to be critical incidents, according to state responses to our survey, 42 states offered certain protections related to involuntary discharge of Medicaid residents who live in assisted living facilities  The majority of protections consisted of a lease agreement requirement that applied to other housing contracts in the state, such as providing residents with eviction notices  Other protections included an appeals process (10 states) and a requirement for the facility to find an alternative location for the resident (10 states)  State Medicaid agencies also varied in whether they made information on critical incidents and other key information readily available to the public  (See table 5 ) Beneficiaries seeking care in an assisted living facility may want to know the number of critical incidents related to a particular facility  Through our survey we found that states differed in the availability of information related to health and welfare that was available to the public  For example, 34 of the 48 states reported that they made critical incident information available to the public by phone, website, or in person, and the remaining 14 states did not have such information available at all  Although all 48 states had information in some form on which assisted facilities accepted Medicaid beneficiaries, 8 states could not provide this information by phone and 22 states could not provide the information in person  In recent years, CMS has taken steps to improve oversight of beneficiary health and welfare in HCBS programs by adding new HCBS waiver application requirements for state monitoring of beneficiary health and welfare  CMS requires state waiver applications to include specific requirements that implement various statutory and regulatory provisions, including a provision that states assure that they will safeguard the health and welfare of Medicaid beneficiaries  In March 2014, CMS added unexplained death to the events that states must be able to identify and address on an ongoing basis, as part of their efforts to prevent instances of abuse, neglect, and exploitation, and added four new requirements for states to protect beneficiary health and welfare  (See table 6 ) In its guidance implementing the 2014 requirements, CMS noted that state associations and state representatives’ work groups had agreed that “health and welfare is one of the most important assurances to track, and requires more extensive tracking to benefit the individuals receiving services, for instance by using data to prevent future incidents ” As a condition for approval of their HCBS waiver applications for each of the requirements, states must identify and agree with CMS on the type of information they will collect to provide as evidence that they will meet the requirements  However, according to CMS officials, each state Medicaid agency has wide discretion over the information it will collect and report to demonstrate that it is meeting the health and welfare requirements and protecting beneficiaries  Although CMS added the additional requirements in 2014 for safeguarding beneficiary health and welfare, the agency generally did not change requirements for how it oversees state monitoring efforts once HCBS waivers are approved  We found a number of limitations in CMS’s oversight of approved HCBS waivers that undermine the agency’s ability to effectively monitor state oversight of HCBS waivers  These limitations include: unclear guidance on what states should identify and report annually related to any identified program deficiencies; lack of requirements on states to regularly provide CMS information on critical incidents; and CMS’s inconsistent enforcement of the requirement that states submit annual reports  Unclear guidance on what states should identify and report annually related to any identified program deficiencies  Federal law requires states to provide CMS with information annually on an HCBS waiver’s impact on (1) the type and amount, and cost of services provided and (2) the health and welfare of Medicaid beneficiaries receiving waiver services  CMS reporting requirements give states latitude to determine what to report as health and welfare deficiencies found through state monitoring of their HCBS programs  With respect to health and welfare, CMS’s State Medicaid Manual directs states when preparing their annual reports to “check the appropriate boxes regarding the impact of the waiver on the health and welfare” of beneficiaries and to describe relevant information  States are required to provide a brief description of the state process for monitoring beneficiary safeguards, use check boxes to indicate that beneficiary health and welfare safeguards have been met, and identify whether deficiencies were detected during the monitoring process  If states determine that deficiencies were identified through monitoring, states are required to “provide a summary of the significant areas where deficiencies were detected” and an explanation of the actions taken to address deficiencies and ensure the deficiencies do not recur  CMS’s written instructions for completing the HCBS annual report do not provide further guidance regarding reporting of deficiencies  For example, the reporting instructions do not describe or identify 1) what states are supposed to report as deficiencies, 2) how they are to identify which deficiencies are most significant, and 3) the extent to which states need to explain the steps taken to ensure that deficiencies do not recur  The lack of clarity is inconsistent with federal internal control standards, in particular, the need for federal agencies to have processes that identify information needed to achieve objectives and address risk  Without clear instructions as to what states must report, states’ annual reports may not identify deficiencies with states’ HCBS waiver programs that may affect the health and welfare of beneficiaries  States may determine that issues or problems they identified through monitoring do not represent reportable deficiencies and therefore may not report those deficiencies to CMS, increasing the risk that problems are not elevated to CMS’s attention  In the case of one of the selected states we reviewed, no problems were included on the annual reports submitted to CMS between 2011 and 2015  However, when CMS completed its review in the fourth year of the state’s waiver— for purpose of renewing the waiver—it determined the state was not assuring beneficiary health and welfare  CMS found that the information the state submitted for purpose of renewal suggested a “pervasive failure” by the state to assure the health and welfare of beneficiaries receiving services, including assisted living services  In particular, CMS noted the state provided insufficient information regarding the number of unexpected or suspicious beneficiary deaths  CMS concluded that the state failed to demonstrate that it has effective systems and processes for ensuring the health and welfare of beneficiaries  Lack of requirements on states to annually provide CMS information on critical incidents  Despite the importance of state critical incident management and reporting systems to protecting the health and welfare of beneficiaries, CMS lacks written requirements that states provide information needed for the agency oversight of state monitoring of critical incidents  According to CMS, a critical element of effective state oversight is the operation of data systems that support the identification of trends and patterns in the occurrence of critical incidents to identify needed improvements  Such a system is also consistent with federal internal controls standards which specify, in particular, the need for federal agencies to have processes that identify information needed to achieve objectives and address risk  CMS requires states to operate a critical incident reporting system  On their waiver applications states must check a box indicating they operate a system and also describe their system—including who must report and when, and what must be reported  Despite this requirement for states to have critical incident reporting systems, CMS does not require states to report to CMS any data from these systems on critical incidents as part of their required annual reports  Specifically, states are not required to include, in their annual reports, the number of critical incidents reported or substantiated that involve Medicaid beneficiaries  As a result, CMS does not have a method to confirm what states describe about critical incident management systems, which is a required component of states’ waiver applications or to assess the capabilities of states’ systems  For example, CMS cannot confirm whether the state systems can report incidents by location or type of residential provider, such as assisted living facilities; the type and severity of critical incidents that occurred; and the number of incidents that involved Medicaid beneficiaries  Without annual critical incident reporting, CMS may be at risk of (1) not having adequate evidence that states are meeting CMS requirements to have an effective critical incident management and reporting system and of (2) being unaware of problems with states’ abilities to identify, track, and address critical incidents involving Medicaid beneficiaries  Our prior work has shown that the lack of explicit reporting requirements on critical incidents not only impacts HCBS waiver programs but also impacts other types of Medicaid long-term services programs as well  Specifically, In a November 2016 report, we found that CMS requirements for states to report on their critical incident monitoring systems for the HCBS waiver program were more stringent than those for other types of HCBS programs, potentially leaving those other programs at even greater risk  We recommended that CMS take steps to harmonize requirements across different types of HCBS programs  HHS concurred with the recommendation stating it would seek input from states, stakeholders, and the public regarding harmonizing requirements across programs  In an August 2017 report we found similar issues in critical incident reporting requirements for other types of long term services programs, particularly those used to provide HCBS and other long term services under managed care  We found that CMS was not always requiring states that contracted with managed care organizations to provide long term services and supports to report to CMS sufficient information on critical incidents and other key areas needed to monitor beneficiary access and quality  We recommended that CMS take steps to identify and obtain key information needed to better oversee states’ efforts to monitor beneficiary access to quality services in their managed long-term services and supports programs  HHS concurred with this recommendation and stated that the agency would take this recommendation into account as part of an ongoing review of its 2016 Medicaid managed care rule  We continue to believe that the implementation of our prior recommendations is needed to help improve CMS oversight of states monitoring of beneficiary safety  CMS’s inconsistent enforcement of the requirement that states submit annual reports  States must prepare and submit an annual report for each HCBS waiver as a condition of waiver approval  According to CMS guidance, the agency’s review of the annual report is part of the ongoing oversight of HCBS waiver programs and not submitting an annual report jeopardizes the states renewal of HCBS waiver programs  However, some states have not been timely in submitting the required annual reports for their HCBS waivers  A review of 2013 HCBS annual reports by a CMS contractor, published in 2016, found that annual reports were missing for 29 HCBS waivers and multiple years’ of annual reports were missing for 8 waivers  In 2014, CMS adopted new strategies to ensure compliance with HCBS waiver requirements, including the requirement that states submit annual reports on a timely basis  These strategies include withholding federal funding, placing a moratorium on enrollment in the waiver, or other actions the agency determines necessary  CMS officials reported that the agency had not used these new strategies with states that were delinquent in submitting their annual reports  Officials said they were in the process of reviewing how to implement these new strategies in the case of one state; however, as of August 2017 officials had not finalized a decision  CMS’s ability to provide effective oversight of state programs and protect beneficiary health and welfare is undermined by the lack of enforcement and receipt of required annual waiver reports  Effective state and federal oversight is necessary to ensure that the health and welfare of Medicaid beneficiaries receiving assisted living services are protected, especially given the particular vulnerability of many of these beneficiaries to abuse, neglect, or exploitation  CMS has taken steps to strengthen beneficiary health and welfare protections in states’ HCBS waiver programs, the most common type of program that covers assisted living services and one that serves the most vulnerable beneficiaries  In particular, CMS now has multiple requirements for states to safeguard beneficiaries’ health and welfare, including requirements to operate an effective critical incident management and reporting system to identify, investigate, and address incidents of beneficiary abuse, neglect, exploitation, and unexplained death  However, CMS’s ability to effectively monitor how well states are assuring beneficiary health and welfare is limited by gaps in state reporting to CMS  CMS has not provided clear guidance to states on what information to include in annual reports on deficiencies they identify  As a result, CMS lacks assurance that it is receiving consistent, complete, and relevant information on deficiencies that is needed to oversee beneficiary health and welfare  Lacking clear guidance on the reporting of deficiencies may result in a delayed recognition of problems that may affect beneficiary health and welfare  Further, for years, states have been required to check a box attesting that they operate a critical incident management system, but have not always been required to report information on incidents of potential or actual harm to beneficiaries  Given the increasing prevalence of assisted living facilities as a provider of services to Medicaid beneficiaries, it is unclear why more than half of states responding to our survey could not provide us information on the number of critical incidents that occurred in these facilities in their states  Reporting data from their critical incident systems, such as the number of incidents, the type and severity of the incidents, or the location or type of facility in which the incident occurred would provide evidence that an effective system is in place, provide information on the extent beneficiaries are subject to actual or potential harm, and allow for tracking trends over time  Finally, CMS has not ensured that all states submit annual reports on their HCBS waiver programs as required  Without improvements to state reporting, CMS cannot ensure states are meeting their commitments to protect the health and welfare of Medicaid beneficiaries receiving assisted living services, potentially jeopardizing their care  We are making the following three recommendations to CMS: The Administrator of CMS should provide guidance and clarify requirements regarding the monitoring and reporting of deficiencies that states using HCBS waivers are required to report on their annual reports  (Recommendation 1) The Administrator of CMS should establish standard Medicaid reporting requirements for all states to annually report key information on critical incidents, considering, at a minimum, the type of critical incidents involving Medicaid beneficiaries, and the type of residential facilities, including assisted living facilities, where critical incidents occurred  (Recommendation 2) The Administrator of CMS should ensure that all states submit annual reports for HCBS waivers on time as required  (Recommendation 3) We provided a draft of this report to HHS for review and comment  HHS provided written comments, which are reproduced in Appendix V  The department also provided technical comments, which we incorporated as appropriate  In its written comments, the department concurred with two of our three recommendations, specifically, that CMS will clarify requirements for state reporting of program deficiencies and ensure that all states submit required annual reports on time  HHS did not explicitly agree or disagree with our third recommendation to require all states to report information on critical incidents to CMS annually  The department noted it has established a workgroup to learn more about states’ health and welfare systems and that it will use the results of this workgroup to determine which additional reporting requirements would be beneficial  The workgroup’s review will continue through calendar year 2018  In technical comments, HHS indicated that after the workgroup’s review is complete it will consider annual reporting of critical incidents  We believe establishing the workgroup is a positive first step towards improving oversight and state reporting and encourage HHS to require annual reporting on critical incidents when developing additional reporting requirements  As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date  At that time, we will send copies to the Secretary of Health and Human Services, the Administrator of CMS, the Administrator of the Administration for Community Living, appropriate congressional committees, and other interested parties  In addition, the report will be available at no charge on the GAO website at http://www gao gov  If you or your staff members have any questions about this report, please contact me at (202) 512-7114 or at iritanik@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff that made key contributions to this report are listed in appendix VI  Our survey of state Medicaid agencies regarding coverage, spending, enrollment, and oversight of assisted living services in 2014, obtained information on challenges for Medicaid beneficiaries to access assisted living services in their states  States provided information related to factors that create challenges for Medicaid beneficiaries’ ability to access and receive assisted living services and the extent states had policies to help beneficiaries with the cost of room and board  A number of states in our survey cited common factors as creating the greatest challenges to a beneficiary’s ability to access assisted living services, including the number of assisted living facilities willing to accept Medicaid beneficiaries (13 states or 27 percent of the 48 states) program enrollment caps (9 states or 19 percent of the 48 states) beneficiaries’ inability to pay for assisted living facility room and board (9 states or 19 percent of the 48 states), which Medicaid typically does not cover low rates the state Medicaid program paid assisted living facilities (8 states or 17 percent of the 48 states)  A number of states reported that they had policies to assist Medicaid beneficiaries with the costs of room and board charged by assisted living facilities, which Medicaid does not typically cover  Two common policies, cited by at least half of the states, were aimed at limiting how much assisted living facilities could charge Medicaid beneficiaries for room and board  For example, 30 of 48 states, limited the amount facilities could charge for room and board to the amount of income certain beneficiaries receive as Supplemental Security Income  The other commonly cited policies focused on providing financial assistance to the beneficiaries to defray the room and board costs  (See table 9 ) In addition to the contact named above, Tim Bushfield and Christine Brudevold (Assistant Directors), Jennie Apter, Shirin Hormozi, Anne Hopewell, Kelsey Kreider, Perry Parsons, Vikki Porter, and Jennifer Whitworth made key contributions to this report 

ground_truth: The number of individuals receiving long term care services from Medicaid in community residential settings is expected to grow. These settings, which include assisted living facilities, provide a range of services that allow aged and disabled beneficiaries, who might otherwise require nursing home care, to remain in the community. State Medicaid programs and CMS, the federal agency responsible for overseeing the state programs, share responsibility for ensuring that beneficiaries' health and welfare is protected. GAO was asked to examine state and federal oversight of assisted living services in Medicaid. This report (1) describes state spending on and coverage of these services, (2) describes how state Medicaid agencies oversee the health and welfare of beneficiaries in these settings, and (3) examines the extent that CMS oversees state Medicaid agency monitoring of assisted living services. GAO surveyed all state Medicaid agencies and interviewed officials in a nongeneralizeable sample of three states with varied oversight processes for their assisted living programs. GAO reviewed regulations and guidance, and interviewed CMS officials. State Medicaid agencies in 48 states that covered assisted living services reported spending more than $10 billion (federal and state) on assisted living services in 2014. These 48 states reported covering these services for more than 330,000 beneficiaries through more than 130 different programs. Most programs were operated under Medicaid waivers that allow states to target certain populations, limit enrollment, or restrict services to certain geographic areas. With respect to oversight of their largest assisted living programs, state Medicaid agencies reported varied approaches to overseeing beneficiary health and welfare, particularly in how they monitored critical incidents involving beneficiaries receiving assisted living services. State Medicaid agencies are required to protect beneficiary health and welfare and operate systems to monitor for critical incidents—cases of potential or actual harm to beneficiaries such as abuse, neglect, or exploitation. Twenty-six state Medicaid agencies could not report to GAO the number of critical incidents that occurred in assisted living facilities, citing reasons including the inability to track incidents by provider type (9 states), lack of a system to collect critical incidents (9 states), and lack of a system that could identify Medicaid beneficiaries (5 states). State Medicaid agencies varied in what types of critical incidents they monitored. All states identified physical, emotional, or sexual abuse as a critical incident. A number of states did not identify other incidents that may indicate potential harm or neglect such as medication errors (7 states) and unexplained death (3 states). State Medicaid agencies varied in whether they made information on critical incidents and other key information available to the public. Thirty-four states made critical incident information available to the public by phone, website, or in person, while another 14 states did not have such information available at all. Oversight of state monitoring of assisted living services by the Centers for Medicare & Medicaid Services (CMS), an agency within the Department of Health and Human Services (HHS), is limited by gaps in state reporting. States are required to annually report to CMS information on deficiencies affecting beneficiary health and welfare for the most common program used to provide assisted living services. However, states have latitude in what they consider a deficiency. States also must describe their systems for monitoring critical incidents, but CMS does not require states to annually report data from their systems. Under federal internal control standards, agencies should have processes to identify information needed to achieve objectives and address risk. Without clear guidance on reportable deficiencies and no requirement to report critical incidents, CMS may be unaware of problems. For example, CMS found, after an in-depth review in one selected state seeking to renew its program, that the state lacked an effective system for assuring beneficiary health and welfare, including reporting insufficient information on the number of unexpected or suspicious beneficiary deaths. The state had not reported any deficiencies in annual reports submitted to CMS in 5 prior years. GAO recommendations to CMS include clarifying state requirements for reporting program deficiencies and requiring annual reporting of critical incidents. HHS concurred with GAO's recommendations to clarify deficiency reporting and stated that it would consider annual reporting requirements for critical incidents after completing an ongoing review.