prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: FEMA’s mission is to help people before, during, and after disasters  It provides assistance to those affected by emergencies and disasters by supplying immediate needs (e g , ice, water, food, and temporary housing) and providing financial assistance grants for damage to personal or public property  FEMA also provides non-disaster assistance grants to improve the nation’s preparedness, readiness, and resilience to all hazards  FEMA accomplishes a large part of its mission through awarding grants to state, local, and tribal governments and nongovernmental entities to help communities prevent, prepare for, protect against, mitigate the effects of, respond to, and recover from disasters and terrorist attacks  As previously mentioned, for fiscal years 2005 through 2014, the agency obligated about $104 5 billion in disaster relief grants  In addition, as of April 2018, the four major disasters in 2017—hurricanes Harvey, Irma, and Maria; and the California wildfires—had resulted in over $22 billion in FEMA grants  The current FEMA grants management environment is highly complex with many stakeholders, IT systems, and users  Specifically, this environment is comprised of 45 active disaster and non-disaster grant programs, which are grouped into 12 distinct grant categories  For example, one program in the Preparedness: Fire category is the Assistance to Firefighters Grants (AFG) program, which provides grants to fire departments, nonaffiliated emergency medical service organizations, and state fire training academies to support firefighting and emergency response needs  As another example, the Housing Assistance grant program is in the Recovery Assistance for Individuals category and provides financial assistance to individuals and households in geographical areas that have been declared an emergency or major disaster by the President  Table 1 lists FEMA’s non-disaster and disaster-based grant categories  According to FEMA, the processes for managing these different types of grants vary because the grant programs were developed independently by at least 18 separate authorizing laws that were enacted over a 62-year period (from 1947 through 2009)  The various laws call for different administrative and reporting requirements  For example, the Robert T  Stafford Disaster Relief and Emergency Assistance Act, as amended, established the statutory authority for 11 of the grant programs, such as the administration of Public Assistance and Individual Assistance grant programs after a presidentially declared disaster  The act also requires the FEMA Administrator to submit an annual report to the President and Congress covering FEMA’s expenditures, contributions, work, and accomplishments, pursuant to the act  As another example, the National Dam Safety Program Act established one of the grant programs aimed at providing financial assistance to improve dam safety  Key stakeholders in modernizing the IT grants management environment include the internal FEMA officials that review, approve, and monitor the grants awarded, such as grant specialists, program analysts, and supervisors  FEMA has estimated that it will need to support about 5,000 simultaneous internal users of its grants management systems  Other users include the grant recipients that apply for, receive, and submit reports on their grant awards; these are considered the external system users  These grant recipients can include individuals, states, local governments, Indian tribes, institutions of higher education, and nonprofit organizations  FEMA has estimated that there are hundreds of thousands of external users of its grants systems  The administration of the many different grant programs is distributed across four divisions within FEMA’s organizational structure  Figure 1 provides an overview of FEMA’s organizational structure and the divisions that are responsible for administering grants  Within three of the four divisions—Resilience, United States Fire Administration, and Office of Response and Recovery—16 different grant program offices are collectively responsible for administering the 45 grant programs  The fourth division consists of 10 regional offices that help administer grants within their designated geographical regions  For example, the Office of Response and Recovery division oversees three different offices that administer 13 grant programs that are largely related to providing assistance in response to presidentially declared disasters  Figure 2 shows the number of grant programs administered by each of the four divisions’ grant program and regional offices  In addition, appendix II lists the names of the 45 grant programs  FEMA’s OCIO is responsible for developing, enhancing, and maintaining the agency’s IT systems, and for increasing efficiencies and cooperation across the entire organization  However, we and the DHS Office of Inspector General (OIG) have previously reported that the grant programs and regional offices develop information systems independent of the OCIO and that this has contributed to the agency’s disparate IT environment  We and the DHS OIG have reported that this disparate IT environment was due, in part, to FEMA’s decentralized IT budget and acquisition practices  For example, from fiscal years 2010 through 2015, the OCIO’s budget represented about one-third of the agency’s IT budget, with the grant program offices accounting for the remaining two-thirds of that budget  In February 2018, the OIG found that FEMA had shown limited progress in improving its IT management and that many of the issues reported in prior audits remained unchanged  As such, the OIG initiated a more comprehensive audit of the agency’s IT management that is ongoing  FEMA has identified 10 primary legacy IT systems that support its grants management activities  According to the agency, most of these systems were developed to support specific grant programs or grant categories  Table 2 summarizes the 10 primary legacy systems  According to FEMA officials, the 10 primary grant systems are all in operation (several have been for decades) and are not interoperable  As a result, individual grant programs and regional offices have independently developed work arounds intended to address existing capability gaps with the primary systems  FEMA officials stated that while these work arounds have helped the agency partially address capability gaps with its primary systems, they are often nonstandardized processes, and introduce the potential for information security risks and errors  This environment has contributed to labor-intensive manual processes and an increased burden for grant recipients  The disparate systems have also led to poor information sharing and reporting capabilities, as well as difficulty reconciling financial data  The DHS OIG and we have previously highlighted challenges with FEMA’s past attempts to modernize its grant management systems  For example, In December 2006, the DHS OIG reported that EMMIE, an effort to modernize its grants management systems and provide a single grants processing solution, was being developed without a clear understanding and definition of the future solution  The report also identified the need to ensure crosscutting participation from headquarters, regions, and states in developing and maintaining a complete, documented set of FEMA business and system requirements  In April 2016, we found weaknesses in FEMA’s development of the EMMIE system  For example, we noted that the system was implemented without sufficient documentation of system requirements, an acquisition strategy, up-to-date cost estimate and schedule, total amount spent to develop the system, or a systems integration plan  In response to our findings and related recommendations, FEMA took action to address these issues  For example, the agency implemented a requirements management process that, among other things, provided guidance to programs on analyzing requirements to ensure that they are complete and verifiable  We reported in November 2017 that EMMIE lacked the ability to collect information on all pre-award activities and, as a result, agency officials said that they and applicants used ad hoc reports and personal tracking documents to manage and monitor the progress of grant applications  FEMA officials added that applicants often struggled to access the system and that the system was not user friendly  Due to EMMIE’s shortfalls, the agency had to develop another system in 2017 to supplement EMMIE with additional grant tracking and case management capabilities  FEMA initiated GMM in 2015, in part, due to EMMIE’s failed attempt to modernize the agency’s grants management environment  The program is intended to modernize and streamline the agency’s grants management environment  To help streamline the agency’s grants management processes, the program established a standard framework intended to represent a common grants management lifecycle  The framework consists of five sequential phases—pre-award, award, post-award, closeout, and post- closeout—along with a sixth phase dedicated to continuous grant program management activities, such as analyzing data and producing reports on grant awards and managing IT systems  FEMA also established 43 distinct business functions associated with these six lifecycle phases  Figure 3 provides the general activities that may occur in each of the grant lifecycle phases, but specific activities would depend on the type of grant being administered (i e , disaster versus non-disaster)  GMM is expected to be implemented within the complex IT environment that currently exists at FEMA  For example, the program is intended to replace the 10 legacy grants management systems, and potentially many additional subsystems, with a single IT system  Each of the 10 legacy systems was developed with its own database(s) and with no standardization of the grants management data and, according to FEMA officials, this legacy data has grown significantly over time  Accordingly, FEMA will need to migrate, analyze, and standardize the grants management data before transitioning it to GMM  The agency awarded a contract in June 2016 to support the data migration efforts for GMM  The agency also implemented a data staging environment in October 2017 to migrate the legacy data and identify opportunities to improve the quality of the data  Further, the GMM system is expected to interface with a total of 38 other systems  These include 19 systems external to DHS (e g , those provided by commercial entities or other federal government agencies) and 19 systems internal to DHS or FEMA  Some of the internal FEMA systems are undergoing their own modernization efforts and will need to be coordinated with GMM, such as the agency’s financial management systems, national flood insurance systems, and enterprise data warehouses  For example, FEMA’s Financial Systems Modernization Program was originally expected to deliver a new financial system in time to interface with GMM  However, the financial modernization has been delayed until after GMM is to be fully implemented; thus, GMM will instead need to interface with the legacy financial system  As a result, GMM is in the process of removing one of its key performance parameters in the acquisition program baseline related to financial systems interoperability and timeliness of data exchanged  In May 2017, DHS approved the acquisition program baseline for GMM  The baseline estimated the total lifecycle costs to be about $251 million, initial operational capability to be achieved by September 2019, and full operational capability to be achieved by September 2020  FEMA intends to develop and deploy its own software applications for GMM using a combination of commercial-off-the-shelf software, open source software, and custom developed code  The agency plans to rely on an Agile software development approach  According to FEMA planning documentation, the agency plans to fully deliver GMM by September 2020 over eight Agile development increments  Agile development is a type of incremental development, which calls for the rapid delivery of software in small, short increments  Many organizations, especially in the federal government, are accustomed to using a waterfall software development model  This type of model typically consists of long, sequential phases, and differs significantly from the Agile development approach  We have previously reported that DHS has sought to establish Agile software development as the preferred method for acquiring and delivering IT capabilities  However, the department has not yet completed critical actions necessary to update its guidance, policies, and practices for Agile programs, in areas such as, developing lifecycle cost estimates, managing IT requirements, testing and evaluation, oversight at key decision points, and ensuring cybersecurity  (See appendix III for more details on the Agile software development approach ) FEMA’s acquisition approach includes using contract support to assist with the development and deployment efforts  The agency selected a public cloud environment to host the computing infrastructure  In addition, from March through July 2017, the agency used a short-term contract aimed at developing prototypes of GMM functionality for grant tracking and monitoring, case management of disaster survivors, grant reporting, and grant closeout  The agency planned to award a second development contract by December 2017 to complete the GMM system (beyond the prototypes) and to begin this work in September 2018  However, due to delays in awarding the second contract to develop the complete GMM system, in January 2018, the program extended the scope and time frames of the initial short-term prototype contract for an additional year to develop the first increment of the GMM system— referred to as the AFG pilot  On August 31, 2018, FEMA awarded the second development contract, which is intended to deliver the remaining functionality beyond the AFG pilot (i e , increments 2 through 8)  FEMA officials subsequently issued a 90-day planning task order for the Agile development contractor to define the work that needs to be done to deliver GMM and the level of effort needed to accomplish that work  However, the planning task order was paused after a bid protest was filed with GAO in September 2018  According to FEMA officials, they resumed work on the planning task order after the bid protest was withdrawn by the protester on November 20, 2018, and then the work was paused again during the partial government shutdown from December 22, 2018, through January 25, 2019  FEMA began working on the AFG pilot—GMM’s first increment—in January 2018  This increment was intended to pilot GMM’s use of Agile development methods to replace core functionality for the AFG system (i e , one of the 10 legacy systems) This system supports three preparedness/fire-related grant programs—Assistance to Firefighters Grants Program, Fire Prevention and Safety Grant Program, and Staffing for Adequate Fire and Emergency Response Grant Program  According to FEMA officials, the AFG system was selected as the first system to be replaced because it is costly to maintain and the DHS OIG had identified cybersecurity concerns with the system  Among the 43 GMM business functions discussed earlier in this report, FEMA officials specified 19 functions to be delivered in the AFG pilot  Figure 4 shows the planned time frames for delivering the AFG pilot in increment 1 (which consisted of four 3-month Agile development sub- increments), as of August 2018  As of August 2018, the program was working on sub-increment 1C of the pilot  In September 2018, GMM deployed its first set of functionality to a total of 19 AFG users—which included seven of 169 total internal AFG users, and 12 of more than 153,000 external AFG users  The functionality supported four of the 19 business functions that are related to the closeout of grants (i e , the process by which all applicable administrative actions and all required work to award a grant have been completed)  This functionality included tasks such as evaluation of final financial reports submitted by grant recipients and final reconciliation of finances (e g , final disbursement to recipients and return of unobligated federal funds)  According to FEMA officials, closeout functionality was selected first for deployment because it was the most costly component of the legacy AFG system to maintain, as it is an entirely manual and labor-intensive process  The remaining AFG functionality and remaining AFG users are to be deployed by the end of the AFG pilot  The GMM program is executed by a program management office, which is overseen by a program manager and program executive  This office is responsible for directing the day-to-day operations and ensuring completion of GMM program goals and objectives  The program office resides within the Office of Response and Recovery, which is headed by an Associate Administrator who reports to the FEMA Administrator  In addition, the GMM program executive (who is also the Regional Administrator for FEMA Region IX) reports directly to the FEMA Administrator  GMM is designated as a level 2 major acquisition, which means that it is subject to oversight by the DHS acquisition review board  The board is chaired by the DHS Undersecretary for Management and is made up of executive-level members, such as the DHS Chief Information Officer  The acquisition review board serves as the departmental executive board that decides whether to approve GMM through key acquisition milestones and reviews the program’s progress and its compliance with approved documentation every 6 months  The board approved the acquisition program baseline for GMM in May 2017 (i e , estimated costs to be about $251 million and full operational capability to be achieved by September 2020)  In addition, the program is reviewed on a monthly basis by FEMA’s Grants Management Executive Steering Group  This group is chaired by the Deputy Administrator of FEMA  Further, DHS’s Financial Systems Modernization Executive Steering Committee, chaired by the DHS Chief Financial Officer, meets monthly and is to provide guidance, oversight, and support to GMM  For government organizations, including FEMA, cybersecurity is a key element in maintaining the public trust  Inadequately protected systems may be vulnerable to insider threats  Such systems are also vulnerable to the risk of intrusion by individuals or groups with malicious intent who could unlawfully access the systems to obtain sensitive information, disrupt operations, or launch attacks against other computer systems and networks  Moreover, cyber-based threats to federal information systems are evolving and growing  Accordingly, we designated cybersecurity as a government-wide high risk area 22 years ago, in 1997, and it has since remained on our high-risk list  Federal law and guidance specify requirements for protecting federal information and information systems  The Federal Information Security Modernization Act (FISMA) of 2014 requires executive branch agencies to develop, document, and implement an agency-wide cybersecurity program to provide security for the information and information systems that support operations and assets of the agency  The act also tasks NIST with developing, for systems other than those for national security, standards and guidelines to be used by all agencies to establish minimum cybersecurity requirements for information and information systems based on their level of cybersecurity risk  Accordingly, NIST developed a risk management framework of standards and guidelines for agencies to follow in developing cybersecurity programs  The framework addresses broad cybersecurity and risk management activities, including categorizing the system’s impact level; selecting, implementing, and assessing security controls; authorizing the system to operate (based on progress in remediating control weaknesses and an assessment of residual risk); and monitoring the efficacy of controls on an ongoing basis  Figure 5 provides an overview of this framework  Prior DHS OIG assessments, such as the annual evaluation of DHS’s cybersecurity program, have identified issues with FEMA’s cybersecurity practices  For example, in 2016, the OIG reported that FEMA was operating 111 systems without an authorization to operate  In addition, the agency had not created any corrective action plans for 11 of the systems that were classified as “Secret” or “Top Secret,” thus limiting its ability to ensure that all identified cybersecurity weaknesses were mitigated in a timely manner  The OIG further reported that, for several years, FEMA was consistently below DHS’s 90 percent target for remediating corrective action plans, with scores ranging from 73 to 84 percent  Further, the OIG reported that FEMA had a significant number of open corrective action plans (18,654) and that most of these plans did not contain sufficient information to address identified weaknesses  In 2017, the OIG reported that FEMA had made progress in addressing security weaknesses  For example, it reported that the agency had reduced the number of systems it was operating without an authorization to operate from 111 to 15 systems  According to GAO’s Business Process Reengineering Assessment Guide and the Software Engineering Institute’s Capability Maturity Model Integration® for Development, successful business process reengineering can enable agencies to replace their inefficient and outmoded processes with streamlined processes that can more effectively serve the needs of the public and significantly reduce costs and improve performance  Many times, new IT systems are implemented to support these improved business processes  Thus, effective management of IT requirements is critical for ensuring the successful design, development, and delivery of such new systems  These leading practices state that effective business process reengineering and IT requirements management involve, among other things, (1) ensuring strong executive leadership support for process reengineering; (2) assessing the current and target business environment and business performance goals; (3) establishing plans for implementing new business processes; (4) establishing clear, prioritized, and traceable IT requirements; (5) tracking progress in delivering IT requirements; and (6) incorporating input from end user stakeholders  Among these six selected leading practices for reengineering business processes and managing IT requirements, FEMA fully implemented four and partially implemented two of them for its GMM program  For example, the agency ensured strong senior leadership commitment to changing the way it manages its grants, took steps to assess and document its business environment and performance goals, defined initial IT requirements for GMM, took recent actions to better track progress in delivering planned IT requirements, and incorporated input from end user stakeholders  In addition, FEMA had begun planning for business process reengineering; however, it had not finalized plans for transitioning users to the new business processes  Further, while GMM took steps to establish clearly defined and prioritized IT requirements, key requirements were not always traceable  Table 3 summarizes the extent to which FEMA implemented the selected leading practices  According to GAO’s Business Process Reengineering Assessment Guide, the most critical factor for engaging in a reengineering effort is having strong executive leadership support to establish credibility regarding the seriousness of the effort and to maintain the momentum as the agency faces potentially extensive changes to its organizational structure and values  Without such leadership, even the best process design may fail to be accepted and implemented  Agencies should also ensure that there is ongoing executive support (e g , executive steering committee meetings headed by the agency leader) to oversee the reengineering effort from start to finish  FEMA senior leadership consistently demonstrated its commitment and support for streamlining the agency’s grants management business processes and provided ongoing executive support  For example, one of the Administrator’s top priorities highlighted in FEMA’s 2014 through 2022 strategic plans was to strengthen grants management through innovative systems and business processes to rapidly and effectively deliver the agency’s mission  In accordance with this strategic priority, FEMA initiated GMM with the intent to streamline and modernize grants management across the agency  In addition, FEMA established the Grants Management Executive Steering Group in September 2015  This group is responsible for transforming the agency’s grants management capabilities through its evaluation, prioritization, and oversight of grants management modernization programs, such as GMM  The group’s membership consists of FEMA senior leaders from across the agency’s program and business support areas, such as FEMA regions, Individual Assistance, Public Assistance, Preparedness, Office of the Chief Financial Officer, Office of Chief Counsel, OCIO, and the Office of Policy and Program Analysis  In this group’s ongoing commitment to reengineering grants management processes, it meets monthly to review GMM’s updates, risks, and action items, as well as the program’s budget, schedule, and acquisition activities  For example, the group reviewed the status of key acquisition activities and program milestones, such as the follow-on award for the pilot contractor and the program’s initial operational capability date  The group also reviewed GMM’s program risks, such as data migration challenges (discussed later in this report) and delays in the Agile development contract award  With this continuous executive involvement, FEMA is better positioned to maintain momentum for reengineering the new grants management business processes that the GMM system is intended to support  GAO’s Business Process Reengineering Assessment Guide states that agencies undergoing business process reengineering should develop a common understanding of the current environment by documenting existing core business processes to show how the processes work and how they are interconnected  The agencies should then develop a deeper understanding of the target environment by modeling the workflow of each target business process in enough detail to provide a common understanding of exactly what will be changed and who will be affected by a future solution  Agencies should also assess the performance of their current major business processes to identify problem areas that need to be changed or eliminated and to set realistically achievable, customer- oriented, and measurable business performance improvement goals  FEMA has taken steps to document the current and target grants management business processes  Specifically, The agency took steps to develop a common understanding of its grants management processes by documenting each of the 12 grant categories  For example, in 2016 and 2017, the agency conducted several nationwide user outreach sessions with representatives from FEMA headquarters, the 10 regional offices, and state and local grant recipients to discuss the grant categories and the current grants management business environment  In addition, FEMA’s Office of Chief Counsel developed a Grants Management Manual in January 2018 that outlined the authorizing laws, regulations, and agency policies for all of its grant programs  According to the Grants Management Executive Steering Group, the manual is intended to promote standardized grants management procedures across the agency  Additionally, the group expects grant program and regional offices to assess the manual against their own practices, make updates as needed, and ensure that their staff are properly informed and trained  FEMA also documented target grants management business process workflows for 18 of the 19 business functions that were notionally planned to be developed and deployed in the AFG pilot by December 2018  However, the program experienced delays in developing the AFG pilot (discussed later in this report) and, thus, deferred defining the remaining business function until the program gets closer to developing that function, which is now planned for August 2019  In addition, FEMA established measurable business performance goals for GMM that are aimed at addressing problem areas and improving grants management processes  Specifically, the agency established 14 business performance goals and associated thresholds in an October 2017 acquisition program baseline addendum, as well as 126 performance metrics for all 43 of the target grants management business functions in its March 2017 test and evaluation master plan  According to FEMA, the 14 business performance goals are intended to represent essential outcomes that will indicate whether GMM has successfully met critical, business-focused mission needs  GMM performance goals include areas such as improvements in the satisfaction level of users with GMM compared to the legacy systems and improvements in the timeliness of grant award processing  For example, one of GMM’s goals is to get at least 40 percent of users surveyed to agree or strongly agree that their grants management business processes are easier to accomplish with GMM, compared to the legacy systems  Program officials stated that they plan to work with the Agile development contractor to refine their performance goals and target thresholds, develop a plan for collecting the data and calculating the metrics, and establish a performance baseline with the legacy systems  Program officials also stated that they plan to complete these steps by September 2019—GMM’s initial operational capability date—which is when they are required to begin reporting these metrics to the DHS acquisition review board  According to GAO’s Business Process Reengineering Assessment Guide, agencies undergoing business process reengineering should (1) establish an overall plan to guide the effort (commonly referred to as an organizational change management plan) and (2) provide a common understanding for stakeholders of what to expect and how to plan for process changes  Agencies should develop the plan at the beginning of the reengineering effort and provide specific details on upcoming process changes, such as critical milestones and deliverables for an orderly transition, roles and responsibilities for change management activities, reengineering goals, skills and resource needs, key barriers to change, communication expectations, training, and any staff redeployments or reductions-in-force  The agency should develop and begin implementing its change management plan ahead of introducing new processes to ensure sufficient support among stakeholders for the reengineered processes  While FEMA has begun planning its business process reengineering activities, it has not finalized its plans or established time frames for their completion  Specifically, as of September 2018, program officials were in the process of drafting an organizational change management plan that is intended to establish an approach for preparing grants management stakeholders for upcoming changes  According to FEMA, this document is intended to help avoid uncertainty and confusion among stakeholders as changes are made to the agency’s grant programs, and ensure successful adoption of new business processes, strategies, and technologies  As discussed previously in this report, the transition to GMM will involve changes to FEMA’s disparate grants management processes that are managed by many different stakeholders across the agency  Program officials acknowledged that change management is the biggest challenge they face in implementing GMM and said they had begun taking several actions intended to support the agency’s change management activities  For example, program officials reported in October 2018 that they had recently created an executive-level working group intended to address FEMA’s policy challenges related to the standardization of grants management processes  Additionally, program officials reported that they planned to: (1) hire additional support staff focused on coordinating grants change management activities; and (2) pursue regional office outreach to encourage broad support among GMM’s decentralized stakeholders, such as state, local, and tribal territories  However, despite these actions, the officials were unable to provide time frames for completing the organizational change management plan or the additional actions  Until the plan and actions are complete, the program lacks assurance that it will have sufficient support among stakeholders for the reengineered processes  In addition, GMM did not establish plans and time frames for the activities that needed to take place prior to, during, and after the transition from the legacy AFG to GMM  Instead, program officials stated that they had worked collaboratively with the legacy AFG program and planned these details informally by discussing them in various communications, such as emails and meetings  However, this informal planning approach is not a repeatable process, which is essential to this program as FEMA plans to transition many sets of functionality to many different users during the lifecycle of this program  Program officials acknowledged that for future transitions they will need more repeatable transition planning and stated that they intend to establish such plans, but did not provide a time frame for when such changes would be made  Until FEMA develops a repeatable process, with established time frames for communicating the transition details to its customers prior to each transition, the agency risks that the transition from the legacy systems to GMM will not occur as intended  It also increases its risk that stakeholders will not support the implementation of reengineered grants management processes  Leading practices for software development efforts state that IT requirements are to be clearly defined and prioritized  This includes, among other things, maintaining bidirectional traceability as the requirements evolve, to ensure there are no inconsistencies among program plans and requirements  In addition, programs using Agile software development are to maintain a product vision, or roadmap, to guide the planning of major program milestones and provide a high-level view of planned requirements  Programs should also maintain a prioritized list (referred to as a backlog) of narrowly defined requirements (referred to as lower-level requirements) that are to be delivered  Programs should maintain this backlog with the product owner to ensure the program is always working on the highest priority requirements that will deliver the most value to the users  The GMM program established clearly defined and prioritized requirements and maintained bidirectional traceability among the various levels of requirements: Grant lifecycle phases: In its Concept of Operations document, the program established six grants management lifecycle phases that represent the highest level of GMM’s requirements, through which it derives lower-level requirements  Business functions: The Concept of Operations document also identifies the next level of GMM requirements—the 43 business functions that describe how FEMA officials, grant recipients, and other stakeholders are to manage grants  According to program officials, the 43 business functions are to be refined, prioritized, and delivered to GMM customers iteratively  Further, for the AFG pilot, the GMM program office prioritized 19 business functions with the product owner and planned the development of these functions in a roadmap  Epics: GMM’s business functions are decomposed into epics, which represent smaller portions of functionality that can be developed over multiple increments  According to program officials, GMM intends to develop, refine, and prioritize the epics iteratively  As of August 2018, the program had developed 67 epics in the program backlog  An example of one of the epics for the AFG pilot is to prepare and submit grant closeout materials  User stories: The epics are decomposed into user stories, which convey the customers’ requirements at the smallest and most discrete unit of work that must be done within a single sprint to create working software  GMM develops, refines, and prioritizes the user stories iteratively  As of August 2018, the program had developed 1,118 user stories in the backlog  An example of a user story is “As an external user, I can log in with a username and password ” Figure 6 provides an example of how GMM’s different levels of requirements are decomposed  Nevertheless, while we found requirements to be traceable at the sprint- level (i e , epics and user stories), traceability of requirements at the increment-level (i e , business functions) were inconsistent among different requirements planning documents  Specifically, the capabilities and constraints document shows that five business functions are planned to be developed within sub-increment 1A, whereas the other key planning document—the roadmap for the AFG pilot—showed one of those five functions as being planned for the sub-increment 1B  In addition, the capabilities and constraints document shows that nine business functions are planned to be developed within sub-increment 1B, but the roadmap showed one of those nine functions as being planned for the sub- increment 1C  Program officials stated that they decided to defer these functions to later sub-increments due to unexpected technical difficulties encountered when developing functionality and reprioritizing functions with the product owners  While the officials updated the roadmap to reflect the deferred functionality, they did not update the capabilities and constraints document to maintain traceability between these two important requirements planning documents  Program officials stated that they learned during the AFG pilot that the use of a capabilities and constraints document for increment-level scope planning was not ideal and that they intended to change the process for how they documented planned requirements for future increments  However, program officials did not provide a time frame for when this change would be made  Until the program makes this change and then ensures it maintains traceability of increment-level requirements between requirements planning documents, it will continue to risk confusion among stakeholders about what is to be delivered  In addition, until recently, GMM’s planning documents were missing up- to-date information regarding when most of the legacy systems will be transitioned to GMM  Specifically, while the program’s planning documents (including the GMM roadmap) provided key milestones for the entire lifecycle of the program and high-level capabilities to be delivered in the AFG pilot, these documents lacked up-to-date time frames for when FEMA planned to transition the nine remaining legacy systems  For example, in May 2017, GMM drafted notional time frames for transitioning the legacy systems, including plans for AFG to be the seventh system replaced by GMM  However, in December 2017, the program decided to reprioritize the legacy systems so that AFG would be replaced first—yet this major change was not reflected in the program’s roadmap  Moreover, while AFG program officials were informed of the decision to transition the AFG program first, in June 2018 officials from other grant programs told us that they had not been informed on when their systems were to be replaced  As a result, these programs were uncertain about when they should start planning for their respective transitions  In August 2018, GMM program officials acknowledged that they were delayed in deciding the sequencing order for the legacy system transitions  Program officials stated that the delay was due to their need to factor the Agile development contractor’s perspective into these decisions; yet, at that time, the contract award had been delayed by approximately 8 months  Subsequently, in October 2018, program officials identified tentative time frames for transitioning the remaining legacy systems  Program officials stated that they determined the tentative time frames for transitioning the legacy systems based on key factors, such as mission need, cost, security vulnerabilities, and technical obsolescence, and that they had shared these new time frames with grant program officials  The officials also stated that, once the Agile contractor begins contract performance, they expect to be able to validate the contractor’s capacity and finalize these time frames by obtaining approval from the Grants Management Executive Steering Group  By taking steps to update and communicate these important time frames, FEMA should be better positioned to ensure that each of the grant programs are prepared for transitioning to GMM  According to leading practices, Agile programs should track their progress in delivering planned IT requirements within a sprint (i e , short iterations that produce working software)  Given that sprints are very short cycles of development (e g , 2 weeks), the efficiency of completing planned work within a sprint relies on a disciplined approach that includes using a fixed pace, referred to as the sprint cadence, that provides a consistent and predictable development routine  A disciplined approach also includes identifying by the start of a sprint which user stories will be developed, developing those stories to completion (e g , fully tested and demonstrated to, and accepted by, the product owner), and tracking completion progress of those stories  Progress should be communicated to relevant stakeholders and used by the development teams to better understand their capacity to develop stories, continuously improve on their processes, and forecast how long it will take to deliver all remaining capabilities  The GMM program did not effectively track progress in delivering IT requirements during the first nine sprints, which occurred from January to June 2018  These gaps in tracking the progress of requirements, in part, had an impact on the program’s progress in delivering the 19 AFG business functions that were originally planned by December 2018 and are now deferred to August 2019  However, beginning in July 2018, in response to our ongoing review, the program took steps to improve in these areas  Specifically, GMM did not communicate the status of its Agile development progress to program stakeholders, such as the grant programs, the regional offices, and the development teams, during most of the first nine sprints  Program officials acknowledged that they should use metrics to track development progress and, in July 2018, they began reporting metrics to program stakeholders  For example, they began collecting and providing data on the number of stories planned and delivered, estimated capacity for development teams, and the number of days spent working on the sprint, as part of the program’s weekly status reports to program stakeholders, such as product owners  Rather than using a fixed, predictable sprint cadence, GMM allowed a variable development cadence, meaning that sprint durations varied from 1 to 4 weeks throughout the first nine sprints  Program officials noted that they had experimented with the use of a variable cadence to allow more time to complete complex technical work  Program officials stated that they realized that varying the sprints was not effective and, in July 2018 for sprint 10, they reverted back to a fixed, 2 week cadence  GMM added a significant amount of scope during its first nine sprints, after the development work had already begun  For example, the program committed to 28 user stories at the beginning of sprint eight, and then nearly doubled the work by adding 25 additional stories in the middle of the sprint  Program officials cited multiple reasons for adding more stories, including that an insufficient number of stories had been defined in the backlog when the sprint began, the realization that planned stories were too large and needed to be decomposed into smaller stories, and the realization that other work would be needed in addition to what was originally planned  Program officials recognized that, by the start of a sprint, the requirements should be sufficiently defined, such that they are ready for development without requiring major changes during the sprint  The program made recent improvements in sprints 11 and 12, which had only five stories added after the start of a sprint  By taking these steps to establish consistency among sprints, the program has better positioned itself to more effectively monitor and manage the remaining IT development work  In addition, this improvement in consistency should help the program avoid future deferments of functionality  Leading practices state that programs should regularly collaborate with, and collect input from, relevant stakeholders; monitor the status of stakeholder involvement; incorporate stakeholder input; and measure how well stakeholders’ needs are being met  For Agile programs, it is especially important to track user satisfaction to determine how well the program has met stakeholders’ needs  Consistent stakeholder participation ensures that the program meets its stakeholders’ needs  FEMA implemented its responsibilities in this area through several means, such as stakeholder outreach activities; development of a strategic communications plan; and continuous monitoring, solicitation, and recording of stakeholder involvement and feedback  For example, the agency conducted nationwide outreach sessions from January 2016 through August 2017 and began conducting additional outreach sessions in April 2018  These outreach sessions involved hundreds of representatives from FEMA headquarters, the 10 regional offices, and state and local grant recipients to collect information on the current grants management environment and opportunities for streamlining grants management processes  FEMA also held oversight and stakeholder outreach activities and actively solicited and recorded feedback from its stakeholders on a regular basis  For example, GMM regularly verified with users that the new functionality met their IT requirements, as part of the Agile development cycle  Additionally, we observed several GMM biweekly requirements validation sessions where the program’s stakeholders were involved and provided feedback as part of the requirements development and refinement process  In addition, FEMA identified GMM stakeholders and tracked its engagement with these stakeholders using a stakeholder register  The agency also defined processes for how the GMM program is to collaborate with its stakeholders in a stakeholder communication plan and Agile development team agreement  Also, while several officials from the selected grant program and regional offices that we interviewed indicated that the program could improve in communicating its plans for GMM and incorporating stakeholder input, most of the representatives from these offices stated that GMM is doing well at interacting with its stakeholders  Finally, in October 2018, program officials reported that they had recently begun measuring user satisfaction by conducting surveys and interviews with users that have utilized the new functionality within GMM  The program’s outreach activities, collection of stakeholder input, and measurement of user satisfaction demonstrate that the program is taking the appropriate steps to incorporate stakeholder input  Reliable cost estimates are critical for successfully delivering IT programs  Such estimates provide the basis for informed decision making, realistic budget formulation, meaningful progress measurement, and accountability for results  GAO’s Cost Estimating and Assessment Guide defines leading practices related to the following four characteristics of a high-quality, reliable estimate  Comprehensive


Report 2:  The estimate accounts for all possible costs associated with a program, is structured in sufficient detail to ensure that costs are neither omitted nor double counted, and documents all cost-influencing assumptions  Well-documented  Supporting documentation explains the process, sources, and methods used to create the estimate; contains the underlying data used to develop the estimate; and is adequately reviewed and approved by management  Accurate  The estimate is not overly conservative or optimistic, is based on an assessment of the costs most likely to be incurred, and is regularly updated so that it always reflects the program’s current status  Credible  Discusses any limitations of the analysis because of uncertainty or sensitivity surrounding data or assumptions, the estimate’s results are cross-checked, and an independent cost estimate is conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results  In May 2017, DHS approved GMM’s lifecycle cost estimate of about $251 million for fiscal years 2015 through 2030  We found this initial estimate to be reliable because it fully or substantially addressed all the characteristics associated with a reliable cost estimate  For example, the estimate comprehensively included government and contractor costs, all elements of the program’s work breakdown structure, and all phases of the system lifecycle; and was aligned with the program’s technical documentation at the time the estimate was developed  GMM also fully documented the key assumptions, data sources, estimating methodology, and calculations for the estimate  Further, the program conducted a risk assessment and sensitivity analysis, and DHS conducted an independent assessment of the cost estimate to validate the accuracy and credibility of the cost estimate  However, key assumptions that FEMA made about the program changed soon after DHS approved the cost estimate in May 2017  Thus, the initial cost estimate no longer reflects the current approach for the program  For example, key assumptions about the program that changed include: Change in the technical approach: The initial cost estimate assumed that GMM would implement a software-as-a-service model, meaning that FEMA would rely on a service provider to deliver software applications and the underlying infrastructure to run them  However, in December 2017, the program instead decided to implement an infrastructure-as-a-service model, meaning that FEMA would develop and deploy its own software application and rely on a service provider to deliver and manage the computing infrastructure (e g , servers, software, storage, and network equipment)  According to program officials, this decision was made after learning from the Agile prototypes that the infrastructure-as-a-service model would allow GMM to develop the system in a more flexible environment  Increase in the number of system development personnel: A key factor with Agile development is the number of development teams (each consisting of experts in software development, testing, and cybersecurity) that are operating concurrently and producing separate portions of software functionality  Program officials initially assumed that they would need three to four concurrent Agile development teams, but subsequently realized that they would instead need to expend more resources to achieve GMM’s original completion date  Specifically, program officials now expect they will need to at least double, and potentially triple, the number of concurrent development teams to meet GMM’s original target dates  Significant delays and complexities with data migration: In 2016 and 2017, GMM experienced various technical challenges in its effort to transfer legacy system data to a data staging platform  This data transfer effort needed to be done to standardize the data before eventually migrating the data to GMM  These challenges resulted in significant delays and cost increases  Program officials reported that, by February 2018—at least 9 months later than planned—all legacy data had been transferred to a data staging platform so that FEMA officials could begin analyzing and standardizing the data prior to migrating it into GMM  FEMA officials reported that they anticipated the cost estimate to increase, and for this increase to be high enough to breach the $251 million threshold set in GMM’s May 2017 acquisition program baseline  Thus, consistent with DHS’s acquisition guidance, the program informed the DHS acquisition review board of this anticipated breach  The board declared that the program was in a cost breach status, as of September 12, 2018  As of October 2018, program officials stated that they were in the process of revising the cost estimate to reflect the changes in the program and to incorporate actual costs  In addition, the officials stated that the program was applying a new cost estimating methodology tailored for Agile programs that DHS’s Cost Analysis Division had been developing  In December 2018, program officials stated that they had completed the revised cost estimate but it was still undergoing departmental approval  Establishing an updated cost estimate should help FEMA better understand the expected costs to deliver GMM under the program’s current approach and time frames  The success of an IT program depends, in part, on having an integrated and reliable master schedule that defines when the program’s set of work activities and milestone events are to occur, how long they will take, and how they are related to one another  Among other things, a reliable schedule provides a roadmap for systematic execution of an IT program and the means by which to gauge progress, identify and address potential problems, and promote accountability  GAO’s Schedule Assessment Guide defines leading practices related to the following four characteristics that are vital to having a reliable integrated master schedule  Comprehensive  A comprehensive schedule reflects all activities for both the government and its contractors that are necessary to accomplish a program’s objectives, as defined in the program’s work breakdown structure  The schedule also includes the labor, materials, and overhead needed to do the work and depicts when those resources are needed and when they will be available  It realistically reflects how long each activity will take and allows for discrete progress measurement  Well-constructed  A schedule is well-constructed if all of its activities are logically sequenced with the most straightforward logic possible  Unusual or complicated logic techniques are used judiciously and justified in the schedule documentation  The schedule’s critical path represents a true model of the activities that drive the program’s earliest completion date and total float accurately depicts schedule flexibility  Credible  A schedule that is credible is horizontally traceable—that is, it reflects the order of events necessary to achieve aggregated products or outcomes  It is also vertically traceable—that is, activities in varying levels of the schedule map to one another and key dates presented to management in periodic briefings are consistent with the schedule  Data about risks are used to predict a level of confidence in meeting the program’s completion date  The level of necessary schedule contingency and high-priority risks are identified by conducting a robust schedule risk analysis  Controlled  A schedule is controlled if it is updated regularly by trained schedulers using actual progress and logic to realistically forecast dates for program activities  It is compared to a designated baseline schedule to measure, monitor, and report the program’s progress  The baseline schedule is accompanied by a baseline document that explains the overall approach to the program, defines ground rules and assumptions, and describes the unique features of the schedule  The baseline schedule and current schedule are subject to a configuration management control process  GMM’s schedule was unreliable because it minimally addressed three characteristics—comprehensive, credible, and controlled—and did not address the fourth characteristic of a reliable estimate—well-constructed  One of the most significant issues was that the program’s fast approaching, final delivery date of September 2020 was not informed by a realistic assessment of GMM development activities, and rather was determined by imposing an unsubstantiated delivery date  Table 4 summarizes our assessment of GMM’s schedule  In discussing the reasons for the shortfalls in these practices, program officials stated that they had been uncertain about the level of rigor that should be applied to the GMM schedule, given their use of Agile development  However, leading practices state that program schedules should meet all the scheduling practices, regardless of whether a program is using Agile development  As discussed earlier in this report, GMM has already experienced significant schedule delays  For example, the legacy data migration effort, the AFG pilot, and the Agile development contract have been delayed  Program officials also stated that the delay in awarding and starting the Agile contract has delayed other important activities, such as establishing time frames for transitioning legacy systems  A more robust schedule could have helped FEMA predict the impact of delays on remaining activities and identify which activities appeared most critical so that the program could ensure that any risks in delaying those activities were properly mitigated  In response to our review and findings, program officials recognized the need to continually enhance their schedule practices to improve the management and communication of program activities  As a result, in August 2018, the officials stated that they planned to add a master scheduler to the team to improve the program’s schedule practices and ensure that all of the areas of concern we identified are adequately addressed  In October 2018, the officials reported that they had recently added two master schedulers to GMM  According to the statement of objectives, the Agile contractor is expected to develop an integrated master schedule soon after it begins performance  However, program officials stated that GMM is schedule-driven—due to the Executive Steering Group’s expectation that the solution will be delivered by September 2020  The officials added that, if GMM encounters challenges in meeting this time frame, the program plans to seek additional resources to allow it to meet the 2020 target  GMM’s schedule-driven approach has already led to an increase in estimated costs and resources  For example, as previously mentioned, the program has determined that, to meet its original target dates, GMM needs to at least double, and possibly triple, the number of concurrent Agile development teams  In addition, we have previously reported that schedule pressure on federal IT programs can lead to omissions and skipping of key activities, especially system testing  In August 2018, program officials acknowledged that September 2020 may not be feasible and that the overall completion time frames established in the acquisition program baseline may eventually need to be rebaselined  Without a robust schedule to forecast whether FEMA’s aggressive delivery goal for GMM is realistic to achieve, leadership will be limited in its ability to make informed decisions on what additional increases in cost or reductions in scope might be needed to fully deliver the system  NIST’s risk management framework establishes standards and guidelines for agencies to follow in developing cybersecurity programs  Agencies are expected to use this framework to achieve more secure information and information systems through the implementation of appropriate risk mitigation strategies and by performing activities that ensure that necessary security controls are integrated into agencies’ processes  The framework addresses broad cybersecurity and risk management activities, which include the following: Categorize the system: Programs are to categorize systems by identifying the types of information used, selecting a potential impact level (e g , low, moderate, or high), and assigning a category based on the highest level of impact to the system’s confidentiality, integrity, and availability, if the system was compromised  Programs are also to document a description of the information system and its boundaries and should register the system with appropriate program management offices  System categorization is documented in a system security plan  Select and implement security controls: Programs are to determine protective measures, or security controls, to be implemented based on the system categorization results  These security controls are documented in a system security plan  For example, control areas include access controls, incident response, security assessment and authorization, identification and authentication, and configuration management  Once controls are identified, programs are to determine planned implementation actions for each of the designated controls  These implementation actions are also specified in the system security plan  Assess security controls: Programs are to develop, review, and approve a security assessment plan  The purpose of the security assessment plan approval is to establish the appropriate expectations for the security control assessment  Programs are to also perform a security control assessment by evaluating the security controls in accordance with the procedures defined in the security assessment plan, in order to determine the extent to which the controls were implemented correctly  The output of this process is intended to produce a security assessment report to document the issues, findings, and recommendations  Programs are to conduct initial remediation actions on security controls and reassess those security controls, as appropriate  Obtain an authorization to operate the system: Programs are to obtain security authorization approval in order to operate a system  Resolving weaknesses and vulnerabilities identified during testing is an important step leading up to achieving an authorization to operate  Programs are to establish corrective action plans to address any deficiencies in cybersecurity policies, procedures, and practices  DHS guidance also states that corrective action plans must be developed for every weakness identified during a security control assessment and within a security assessment report  Monitor security controls on an ongoing basis: Programs are to monitor their security controls on an ongoing basis after deployment, including determining the security impact of proposed or actual changes to the information system and assessing the security controls in accordance with a monitoring strategy that determines the frequency of monitoring the controls  For the GMM program’s engineering and test environment, which went live in February 2018, FEMA fully addressed three of the five key cybersecurity practices in NIST’s risk management framework and partially addressed two of the practices  Specifically, FEMA categorized GMM’s environment based on security risk, implemented select security controls, and monitored security controls on an ongoing basis  However, the agency partially addressed the areas of assessing security controls and obtaining an authorization to operate the system  Table 5 provides a summary of the extent to which FEMA addressed NIST’s key cybersecurity practices for GMM’s engineering and test environment  Consistent with NIST’s framework, GMM categorized the security risk of its engineering and test environment and identified it as a moderate- impact environment  A moderate-impact environment is one where the loss of confidentiality, integrity, or availability could be expected to have a serious or adverse effect on organizational operations, organizational assets, or individuals  GMM completed the following steps leading to this categorization: The program documented in its System Security Plan the various types of data and information that the environment will collect, process, and store, such as conducting technology research, building or enhancing technology, and maintaining IT networks  The program established three information types and assigned security levels of low, moderate, or high impact in the areas of confidentiality, availability, and integrity  A low-impact security level was assigned to two information types: (1) conducting technology research and (2) building or enhancing technology; and a moderate- impact security level was assigned to the third information type: maintaining IT networks  The engineering and test environment was categorized as an overall moderate-impact system, based on the highest security impact level assignment  GMM documented a description of the environment, including a diagram depicting the system’s boundaries, which illustrates, among other things, databases and firewalls  GMM properly registered its engineering and test environment with FEMA’s Chief Information Officer, Chief Financial Officer, and acting Chief Information Security Officer  By conducting the security categorization process, GMM has taken steps that should ensure that the appropriate security controls are selected for the program’s engineering and test environment  Consistent with NIST’s framework and the system categorization results, GMM appropriately determined which security controls to implement and planned actions for implementing those controls in its System Security Plan for the engineering and test environment  For example, the program utilized NIST guidance to select standard controls for a system categorized with a moderate-impact security level  These control areas include, for example, access controls, risk assessment, incident response, identification and authentication, and configuration management  Further, the program documented its planned actions to implement each control in its System Security Plan  For example, GMM documented that the program plans to implement its Incident Response Testing control by participating in an agency-wide exercise and unannounced vulnerability scans  As another example, GMM documented that the program plans to implement its Contingency Plan Testing control by testing the contingency plan annually, reviewing the test results, and preparing after action reports  By selecting and planning for the implementation of security controls, GMM has taken steps to mitigate its security risks and protect the confidentiality, integrity, and availability of the information system  Consistent with NIST’s framework, in January 2018, GMM program officials developed a security assessment plan for the engineering and test environment  According to GMM program officials, this plan was reviewed by the security assessment team  However, the security assessment plan lacked essential details  Specifically, while the plan included the general process for evaluating the environment’s security controls, the planned assessment procedures for all 964 security controls were not sufficiently defined  Specifically, GMM program officials copied example assessment procedures from NIST guidance and inserted them into its security assessment documentation for all of its 964 controls, without making further adjustments to explain the steps that should be taken specific to GMM  Table 6 shows an example of a security assessment procedure copied from the NIST guidance that should have been further adjusted for GMM  In addition, the actual assessment procedures that the GMM assessors used to evaluate the security controls were not documented  Instead, the program only documented whether each control passed or failed each test  GMM program officials stated that the planned assessment procedures are based on an agency template that was exported from a DHS compliance tool, and that FEMA security officials have been instructed by the DHS OCIO not to tailor or make any adjustments to the template language  However, the assessment procedures outlined in NIST’s guidance are to serve as a starting point for organizations preparing their program specific assessments  According to NIST, organizations are expected to select and tailor their assessment procedures for each security control from NIST’s list of suggested assessment options (e g , review, analyze, or inspect policies, procedures, and related documentation options)  DHS OCIO officials stated that, consistent with NIST’s guidance, they expect that components will ensure they are in compliance with the minimum standards and will also add details and additional rigor, as appropriate, to tailor the planned security assessment procedures to fit their unique missions or needs  In November 2018, in response to our audit, DHS OCIO officials stated that they were meeting with FEMA OCIO officials to understand why they did not document the planned and actual assessment procedures performed by the assessors for GMM  Until FEMA ensures that detailed planned evaluation methods and actual evaluation procedures specific to GMM are defined, the program risks assessing security controls incorrectly, having controls that do not work as intended, and producing undesirable outcomes with respect to meeting the security requirements  In addition, the security assessment plan was not approved by FEMA’s OCIO before proceeding with the security assessment  Program officials stated that approval was not required for the security assessment plan prior to the development of the security assessment report  However, NIST guidance states that the purpose of the security assessment plan approval is to establish the appropriate expectations for the security control assessment  By not getting the security assessment plan approved by FEMA’s OCIO before security assessment reviews were conducted, GMM risks inconsistencies with the plan and security objectives of the organization  Finally, consistent with NIST guidance, GMM performed a security assessment in December 2017 of the engineering and test environment’s controls, which identified 36 vulnerabilities (23 critical- and high-impact vulnerabilities and 13 medium- and low-impact vulnerabilities)  The program also documented these vulnerabilities and associated findings and recommendations in a security assessment report  GMM conducted initial remediation actions (i e , remediation of vulnerabilities that should be corrected immediately) for 12 of the critical- and high-impact vulnerabilities and a reassessment of those security controls confirmed that they were resolved by January 2018  Remediation of the remaining 11 critical- and high-impact vulnerabilities and 13 medium- and low- impact vulnerabilities were to be addressed by corrective action plans as part of the authorization to operate process, which is discussed in the next section  The authorization to operate GMM’s engineering and test environment was granted on February 5, 2018  Among other things, this decision was based on the important stipulation that the remaining 11 critical- and high- impact vulnerabilities associated with multifactor authentication would be addressed within 45 days, or by March 22, 2018  However, the program did not meet this deadline and, instead, approximately 2 months after this deadline passed, obtained a waiver to remediate these vulnerabilities by May 9, 2019  These vulnerabilities are related to a multifactor authentication capability  Program officials stated that they worked with FEMA OCIO officials to attempt to address these vulnerabilities by the initial deadline, but they were unsuccessful in finding a viable solution  Therefore, GMM program officials developed a waiver at the recommendation of the OCIO to provide additional time to develop a viable solution  However, a multifactor authentication capability is essential to ensuring that users are who they say they are, prior to granting users access to the GMM engineering and test environment, in order to reduce the risk of harmful actors accessing the system  In addition, as of September 2018, the program had not established corrective action plans for the 13 medium- and low-impact vulnerabilities  Program officials stated that they do not typically address low-impact vulnerabilities; however, this is in conflict with DHS guidance that specifies that corrective action plans must be developed for every weakness identified during a security control assessment and within a security assessment report  In response to our audit, in October 2018, GMM program officials developed these remaining corrective action plans  The plans indicated that these vulnerabilities were to be fully addressed by January 2019 and April 2019  While the program eventually took corrective actions in response to our audit by developing the missing plans, the GMM program initially failed to follow DHS’s guidance on preparing corrective actions plans for all security vulnerabilities  Until GMM consistently follows DHS’s guidance, it will be difficult for FEMA to determine the extent to which GMM’s security weaknesses identified during its security control assessments are remediated  Additionally, as we have reported at other agencies, vulnerabilities can be indicators of more significant underlying issues and, thus, without appropriate management attention or prompt remediation, GMM is at risk of unnecessarily exposing the program to potential exploits  Moreover, GMM was required to assess all untested controls by March 7, 2018, or no later than 30 days after the approval of the authorization to operate; however, it did not meet this deadline  Specifically, we found that, by October 2018, FEMA had not fully tested 190 security controls in the GMM engineering and test environment  These controls were related to areas such as security incident handling and allocation of resources required to protect an information system  In response to our findings, in October 2018, GMM program officials reported that they had since fully tested 27 controls and partially tested the remaining 163 controls  Program officials stated that testing of the 163 controls is a shared responsibility between GMM and other parties (e g , the cloud service provider)  They added that GMM had completed its portion of the testing but was in the process of verifying the completion of testing by other parties  Program officials stated that the untested controls were not addressed sooner, in part, because of errors resulting from configuration changes in the program’s compliance tool during a system upgrade, which have now been resolved  Until GMM ensures that all security controls have been tested, it remains at an increased risk of exposing programs to potential exploits  Consistent with the NIST framework, GMM established methods for assessing and monitoring security controls to be conducted after an authorization to operate has been approved  GMM has tailored its cybersecurity policies and practices for monitoring its controls to take into account the frequent and iterative pace with which system functionality is continuously being introduced into the GMM environment  Specifically, the GMM program established a process for assessing security impact changes to the system and conducting reauthorizations to operate within the rapid Agile delivery environment  As part of this process, GMM embedded cybersecurity experts on each Agile development team so that they are involved early and can impact security considerations from the beginning of requirements development through testing and deployment of system functionality  In addition, the process involves important steps for ensuring that the system moves from development to completion, while producing a secure and reliable system  For example, it includes procedures for creating, reviewing, and testing new system functionality  As the new system functionality is integrated with existing system functionality, it is to undergo automated testing and security scans in order to ensure that the integrity of the security of the system has not been compromised  Further, an automated process is to deploy the code if it passes all security scans, code tests, and code quality checks  GMM’s process for conducting a reauthorization to operate within the rapid delivery Agile development environment is to follow FEMA guidance that states that all high-level changes made to a FEMA IT system must receive approval from both a change advisory board and the FEMA Chief Information Officer  The board and FEMA Chief Information Officer are to focus their review and approval on scheduled releases and epics (i e , collections of user stories)  Additionally, the Information System Security Officer is to review each planned user story and, if it is determined that the proposed changes may impact the integrity of the authorization, the Information System Security Officer is to work with the development team to begin the process of updating the system authorization  Finally, GMM uses automated tools to track the frequency in which security controls are assessed and to ensure that required scanning data are received by FEMA for reporting purposes  Program officials stated that, in the absence of department-level and agency-level guidance, they have coordinated with DHS and FEMA OCIO officials to ensure that these officials are in agreement with GMM’s approach to continuous monitoring  By having monitoring control policies and procedures in place, FEMA management is positioned to more effectively prioritize and plan its risk response to current threats and vulnerabilities for the GMM program  Given FEMA’s highly complex grants management environment, with its many stakeholders, IT systems, and internal and external users, implementing leading practices for business process reengineering and IT requirements management is critical for success  FEMA has taken many positive steps, including ensuring executive leadership support for business process reengineering, documenting the agency’s grants management processes and performance improvement goals, defining initial IT requirements for the program, incorporating input from end user stakeholders into the development and implementation process, and taking recent actions to improve its delivery of planned IT requirements  Nevertheless, until the GMM program finalizes plans and time frames for implementing its organizational change management actions, plans and communicates system transition activities, and maintains clear traceability of IT requirements, FEMA will be limited in its ability to provide streamlined grants management processes and effectively deliver a modernized IT system to meet the needs of its large range of users  While GMM’s initial cost estimate was reliable, key assumptions about the program since the initial estimate had changed and, therefore, it no longer reflected the current approach for the program  The forthcoming updated cost schedule is expected to better reflect the current approach  However, the program’s unreliable schedule to fully deliver GMM by September 2020 is aggressive and unrealistic  The delays the program has experienced to date further compound GMM’s schedule issues  Without a robust schedule that has been informed by a realistic assessment of GMM’s development activities, leadership will be limited in its ability to make informed decisions on what additional increases in cost or reductions in scope might be needed to achieve their goals  Further, FEMA’s implementation of cybersecurity practices for GMM in the areas of system categorization, selection and implementation, and monitoring will help the program  However, GMM lacked essential details for evaluating security controls, did not approve the security assessment plan before proceeding with the security assessment, did not follow DHS’s guidance to develop corrective action plans for all security vulnerabilities, and did not fully test all security controls  As a result, the GMM engineering and test environment remains at an increased risk of exploitations  We are making eight recommendations to FEMA: The FEMA Administrator should ensure that the GMM program management office finalizes the organizational change management plan and time frames for implementing change management actions  (Recommendation 1) The FEMA Administrator should ensure that the GMM program management office plans and communicates its detailed transition activities to its affected customers before they transition to GMM and undergo significant changes to their processes  (Recommendation 2) The FEMA Administrator should ensure that the GMM program management office implements its planned changes to its processes for documenting requirements for future increments and ensures it maintains traceability among key IT requirements documents  (Recommendation 3) The FEMA Administrator should ensure that the GMM program management office updates the program schedule to address the leading practices for a reliable schedule identified in this report  (Recommendation 4) The FEMA Administrator should ensure that the FEMA OCIO defines sufficiently detailed planned evaluation methods and actual evaluation methods for assessing security controls  (Recommendation 5) The FEMA Administrator should ensure that the FEMA OCIO approves a security assessment plan before security assessment reviews are conducted  (Recommendation 6) The FEMA Administrator should ensure that the GMM program management office follows DHS guidance on preparing corrective action plans for all security vulnerabilities  (Recommendation 7) The FEMA Administrator should ensure that the GMM program management office fully tests all of its security controls for the system  (Recommendation 8) DHS provided written comments on a draft of this report, which are reprinted in appendix IV  In its comments, the department concurred with all eight of our recommendations and provided estimated completion dates for implementing each of them  For example, with regard to recommendation 4, the department stated that FEMA plans to update the GMM program schedule to address the leading practices for a reliable schedule by April 30, 2019  In addition, for recommendation 7, the department stated that FEMA plans to ensure that corrective action plans are prepared by July 31, 2019, to address all identified security vulnerabilities for GMM  If implemented effectively, the actions that FEMA plans to take in response to the recommendations should address the weaknesses we identified  We also received technical comments from DHS and FEMA officials, which we incorporated, as appropriate  We are sending copies of this report to the Secretary of Homeland Security and interested congressional committees  In addition, the report will be available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-4456 or harriscc@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix V  Our objectives were to (1) determine the extent to which the Federal Emergency Management Agency (FEMA) is implementing leading practices for reengineering its grants management business processes and incorporating business needs into Grants Management Modernization (GMM) information technology (IT) requirements; (2) assess the reliability of the program’s estimated costs and schedule; and (3) determine the extent to which FEMA is addressing key cybersecurity practices for GMM  To address the first objective, we reviewed GAO’s Business Process Reengineering Assessment Guide and Software Engineering Institute’s Capability Maturity Model for Integration for Development to identify practices associated with business process reengineering and IT requirements management  We then selected six areas that, in our professional judgment, represented foundational practices that were of particular importance to the successful implementation of an IT modernization effort that is using Agile development processes  We also selected the practices that were most relevant based on where GMM was in the system development lifecycle and we discussed the practice areas with FEMA officials  The practices are: Ensuring executive leadership support for process reengineering Assessing the current and target business environment and business Establishing plans for implementing new business processes Establishing clear, prioritized, and traceable IT requirements Tracking progress in delivering IT requirements Incorporating input from end user stakeholders We also reviewed selected chapters of GAO’s draft Agile Assessment Guide (Version 6A), which is intended to establish a consistent framework based on best practices that can be used across the federal government for developing, implementing, managing, and evaluating agencies’ IT investments that rely on Agile methods  To develop this guide, GAO worked closely with Agile experts in the public and private sector; some chapters of the guide are considered more mature because they have been reviewed by the expert panel  We reviewed these chapters to ensure that our expectations for how FEMA should apply the six practices for business process reengineering and IT requirements management are appropriate for an Agile program and are consistent with the draft guidance that is under development  Additionally, since Agile development programs may use different terminology to describe their software development processes, the Agile terms used in this report (e g , increment, sprint, epic, etc ) are specific to the GMM program  We obtained and analyzed FEMA grants management modernization documentation, such as current and target grants management business processes, acquisition program baseline, operational requirements document, concept of operations, requirements analyses workbooks, Grants Management Executive Steering Group artifacts, stakeholder outreach artifacts, Agile increment- and sprint-level planning and development artifacts, and the requirements backlog  We assessed the program documentation against the selected practices to determine the extent to which the agency had implemented them  We then assessed each practice area as: fully implemented—FEMA provided complete evidence that showed it fully implemented the practice area; partially implemented—FEMA provided evidence that showed it partially implemented the practice area; not implemented—FEMA did not provide evidence that showed it implemented any of the practice area  Additionally, we observed Agile increment and sprint development activities at GMM facilities in Washington, D C  We also observed a demonstration of how the program manages its lower level requirements (i e , user stories and epics) and maintains traceability of the requirements using an automated tool at GMM facilities in Washington, D C  We also interviewed FEMA officials, including the GMM Program Executive, GMM Program Manager, GMM Business Transformation Team Lead, and Product Owner regarding their efforts to streamline grants management business processes, collect and incorporate stakeholder input, and manage GMM’s requirements  In addition, we interviewed FEMA officials from four out of 16 grant program offices and two out of 10 regional offices to obtain contextual information and illustrative examples of FEMA’s efforts to reengineer grants management business processes and collect business requirements for GMM  Specifically, We selected the four grant program offices based on a range of grant programs managed, legacy systems used, and the amount of grant funding awarded  We also sought to select a cross section of different characteristics, such as selecting larger grant program offices, as well as smaller offices  In addition, we ensured that our selection included the Assistance to Firefighters Grants (AFG) program office because officials in this office represent the first GMM users and, therefore, are more actively involved with the program’s Agile development practices  Based on these factors, we selected: Public Assistance Division, Individual Assistance Division, AFG, and National Fire Academy  Additionally, the four selected grant program offices are responsible for 16 of the total 45 grant programs and are users of five of the nine primary legacy IT systems  The four selected grant program offices also represent about 68 percent of the total grant funding awarded by FEMA from fiscal years 2005 through 2016  We selected two regional offices based on (1) the largest amount of total FEMA grant funding for fiscal years 2005 through 2016—Region 6 located in Denton, Texas; and (2) the highest percentage of AFG funding compared to the office’s total grant funding awarded from fiscal years 2005 through 2016—Region 5 located in Chicago, Illinois  To assess the reliability of data from the program’s automated IT requirements management tool, we interviewed knowledgeable officials about the quality control procedures used by the program to assure accuracy and completeness of the data  We also compared the data to other relevant program documentation on GMM requirements  We determined that the data used were sufficiently reliable for the purpose of evaluating GMM’s practices for managing IT requirements  For our second objective, to assess the reliability of GMM’s estimated costs and schedule, we reviewed documentation on GMM’s May 2017 lifecycle cost estimate and on the program’s schedule, dated May 2018  To assess the reliability of the May 2017 lifecycle cost estimate, we evaluated documentation supporting the estimate, such as the cost estimating model, the report on GMM’s Cost Estimating Baseline Document and Life Cycle Cost Estimate, and briefings provided to the Department of Homeland Security (DHS) and FEMA management regarding the cost estimate  We assessed the cost estimating methodologies, assumptions, and results against leading practices for developing a comprehensive, accurate, well-documented, and credible cost estimate, identified in GAO’s Cost Estimating and Assessment Guide  We also interviewed program officials responsible for developing and reviewing the cost estimate to understand their methodology, data, and approach for developing the estimate  We found that the cost data were sufficiently reliable  To assess the reliability of the May 2018 GMM program schedule, we evaluated documentation supporting the schedule, such as the integrated master schedule, acquisition program baseline, and Agile artifacts  We assessed the schedule documentation against leading practices for developing a comprehensive, well-constructed, credible, and controlled schedule, identified in GAO’s Schedule Assessment Guide  We also interviewed GMM program officials responsible for developing and managing the program schedule to understand their practices for creating and maintaining the schedule  We noted in our report the instances where the quality of the schedule data impacted the reliability of the program’s schedule  For both the cost estimate and program schedule, we assessed each leading practice as: fully addressed—FEMA provided complete evidence that showed it implemented the entire practice area; substantially addressed—FEMA provided evidence that showed it implemented more than half of the practice area; partially addressed—FEMA provided evidence that showed it implemented about half of the practice area; minimally addressed—FEMA provided evidence that showed it implemented less than half of the practice area; not addressed—FEMA did not provide evidence that showed it implemented any of the practice area  Finally, we provided FEMA with draft versions of our detailed analyses of the GMM cost estimate and schedule  This was done to verify that the information on which we based our findings was complete, accurate, and up-to-date  Regarding our third objective, to determine the extent to which FEMA is addressing key cybersecurity practices for GMM, we reviewed documentation regarding DHS and FEMA cybersecurity policies and guidance, and FEMA’s authorization to operate for the program’s engineering and test environment  We evaluated the documentation against all six cybersecurity practices identified in the National Institute of Standards and Technology’s (NIST) Risk Management Framework  While NIST’s Risk Management Framework identifies six total practices, for reporting purposes, we combined two interrelated practices—selection of security controls and implementation of security controls—into a single practice  The resulting five practices were: categorizing the system based on security risk, selecting and implementing security controls, assessing security controls, obtaining an authorization to operate the system, and monitoring security controls on an ongoing basis  We obtained and analyzed key artifacts supporting the program’s efforts to address these risk management practices, including the program’s System Security Plan, the Security Assessment Plan and Report, Authorization to Operate documentation, and the program’s continuous monitoring documentation  We also interviewed officials from the GMM program office and FEMA’s Office of the Chief Information Officer, such as the GMM Security Engineering Lead, GMM Information System Security Officer, and FEMA’s Acting Chief Information Security Officer, regarding their efforts to assess, document, and review security controls for GMM  We assessed the evidence against the five practices to determine the extent to which the agency had addressed them  We then assessed each practice area as: fully addressed—FEMA provided complete evidence that showed it fully implemented the practice area; partially addressed—FEMA provided evidence that showed it partially implemented the practice area; not addressed—FEMA did not provide evidence that showed it implemented any of the practice area  To assess the reliability of data from the program’s automated security controls management tool, we interviewed knowledgeable officials about the quality control procedures used by the program to assure accuracy and completeness of the data  We also compared the data to other relevant program documentation on GMM security controls for the engineering and test environment  We found that some of the security controls data we examined were sufficiently reliable for the purpose of evaluating FEMA’s cybersecurity practices for GMM, and we noted in our report the instances where the accuracy of the data impacted the program’s ability to address key cybersecurity practices  We conducted this performance audit from December 2017 to April 2019 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  The Federal Emergency Management Agency (FEMA) awards many different types of grants to state, local, and tribal governments and nongovernmental entities  These grants are to help communities prevent, prepare for, protect against, mitigate the effects of, respond to, and recover from disasters and terrorist attacks  Agile software development is a type of incremental development that calls for the rapid delivery of software in small, short increments  The use of an incremental approach is consistent with the Office of Management and Budget’s guidance as specified in its information technology (IT) Reform Plan, as well as the legislation commonly referred to as the Federal Information Technology Acquisition Reform Act  Many organizations, especially in the federal government, are accustomed to using a waterfall software development model, which typically consists of long, sequential phases, and differs significantly from the Agile development approach  Agile practices integrate planning, design, development, and testing into an iterative lifecycle to deliver software early and often  Figure 7 provides a depiction of software development using the Agile approach, as compared to a waterfall approach  The frequent iterations of Agile development are intended to effectively measure progress, reduce technical and programmatic risk, and respond to feedback from stakeholders in changes to IT requirements more quickly than traditional methods  Despite these intended benefits, organizations adopting Agile must overcome challenges in making significant changes to how they are accustomed to developing software  The significant differences between Agile and waterfall development impact how IT programs are planned, implemented, and monitored in terms of cost, schedule, and scope  For example, in waterfall development, significant effort is devoted upfront to document detailed plans and all IT requirements for the entire scope of work at the beginning of the program, and cost and schedule can be varied to complete that work  However, for Agile programs the precise details are unknown upfront, so initial planning of cost, scope, and timing would be conducted at a high level, and then supplemented with more specific plans for each iteration  While cost and schedule are set for each iteration, requirements for each iteration (or increment) can be variable as they are learned over time and revised to reflect experiences from completed iterations and to accommodate changing priorities of the end users  The differences in these two software development approaches are shown in figure 8  Looking at figure 8, the benefit provided from using traditional program management practices such as establishing a cost estimate or a robust schedule, is not obvious  However, unlike a theoretical environment, many government programs may not have the autonomy to manage completely flexible scope, as they must deliver certain minimal specifications with the cost and schedule provided  In those cases, it is vital for the team to understand and differentiate the IT requirements that are “must haves” from the “nice to haves” early in the planning effort  This would help facilitate delivery of the “must-haves” requirements first, thereby providing users with the greatest benefits as soon as possible  In addition to the contact named above, the following staff made key contributions to this report: Shannin G  O’Neill (Assistant Director), Jeanne Sung (Analyst in Charge), Andrew Beggs, Rebecca Eyler, Kendrick Johnson, Thomas J  Johnson, Jason Lee, Jennifer Leotta, and Melissa Melvin 

ground_truth: FEMA, an agency of the Department of Homeland Security (DHS), has obligated more than $36 billion in PA grants to state, local, and tribal governments to help communities recover and rebuild after major disasters since 2009. Further, costs are rising with disasters, such as Hurricanes Harvey, Irma, and Maria in 2017. FEMA recently redesigned how the PA program delivers assistance to state and local grantees to improve operations and address past challenges identified by GAO and others. FEMA tested the new delivery model in selected disasters and announced implementation in September 2017. GAO was asked to assess the redesigned PA program. This report examines, among other things, the extent to which FEMA's new delivery model addresses (1) past workforce management challenges and assesses future workforce needs; and (2) past information sharing challenges and key IT management controls. GAO reviewed FEMA policy, strategy, and implementation documents; interviewed FEMA and state officials, PA program applicants, and other stakeholders; and observed implementation of the new model at one test location following Hurricane Matthew in 2016. The Federal Emergency Management Agency (FEMA) redesigned the Public Assistance (PA) grant program delivery model to address past challenges in workforce management, but has not fully assessed future workforce staffing needs. GAO and others have previously identified challenges related to shortages in experienced and trained FEMA PA staff and high turnover among these staff. These challenges often led to applicants receiving inconsistent guidance and to PA project delays. As part of its new model, FEMA is creating consolidated resource centers to standardize and centralize PA staff responsible for managing grant applications, and new specialized positions, such as hazard mitigation liaisons, program delivery managers, and site inspectors, to ensure more consistent guidance to applicants. However, FEMA has not assessed the workforce needed to fully implement the new model, such as the number of staff needed to fill certain new positions, or to achieve staffing goals for supporting hazard mitigation on PA projects. Fully assessing workforce needs will help to ensure that FEMA has the people and the skills needed to fully implement the new PA model and help to avoid the long-standing workforce challenges the program encountered in the past. FEMA designed a new PA information and case management system—called the FEMA Applicant Case Tracker (FAC-Trax)—to address past information sharing challenges, such as difficulties in sharing grant documentation among FEMA, state, and local officials and tracking the status of PA projects, but additional actions could better ensure effective implementation. Both FEMA and state officials involved in testing of the new model stated that the new information system allows them to better manage and track PA applications and documentation, which could lead to greater transparency and efficiencies in the program. Further, GAO found that this new system fully addresses two of four key information technology (IT) management controls—project planning and risk management—that are necessary to ensure systems work effectively and meet user needs. However, GAO found that FEMA has not fully addressed the other two controls—requirements development and systems testing and integration. By better analyzing progress on high-level user requirements, for example, FEMA will have greater assurance that FAC-Trax will meet user needs and achieve the goals of the new delivery model. GAO is making five recommendations, including that FEMA assess the workforce needed for the new delivery model and improve key IT management controls for its new information sharing and case management system, FAC-Trax. DHS concurred with all recommendations.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.

Report 1: Major disaster declarations can trigger a variety of federal response and recovery programs for government and nongovernmental entities, households, and individuals  FEMA’s Office of Response and Recovery manages the PA grant program, providing funds to states, territorial governments, local government agencies, Indian tribes, authorized tribal organizations, and certain private nonprofit organizations in response to presidentially declared disaster declarations to repair damaged public infrastructure such as roads, schools, and bridges  Figure 1 shows the total amount of PA funds obligated by county from January 2009 through February 2017 for federal disaster declarations  To implement the PA program, FEMA’s staff includes a mix of temporary, reservist, and permanent employees under two authorities, the Stafford Act and Title 5  Reservists make up the largest share of the PA workforce, which consisted of 1,852 employees––1,041 reservists, 634 full-time equivalents, and 177 temporary Cadre of On-Call Response/Recovery Employees––as of June 2017, according to PA officials  Figure 2 summarizes the key characteristics for each type of employee  After a disaster, FEMA sends PA program staff to the affected area to work with state and local officials to assess the damage prior to a disaster declaration  FEMA officials establish a temporary Joint Field Office (JFO) to house staff who will manage response and recovery functions after a declared disaster (including operations, emergency response and support teams, planning, administration, finance, and logistics)  Once the President has declared a disaster, PA staff work with grant applicants to help them document damages, identify eligible costs and work, and prepare requests for PA grant funds by developing project proposals  These proposals may include proposals for hazard mitigation if the hazard mitigation work is related to the repair of damaged facilities, referred to as permanent work projects  Immediate emergency measures, such as debris removal, are not eligible for hazard mitigation  Officials then review and obtain approval of the projects prior to FEMA obligating funds to state grantees  Figure 3 describes the process used to develop, review, and obligate PA projects  In addition to rebuilding and restoring infrastructure to its predisaster state, the PA program can be used to fund hazard mitigation measures that will reduce future risk to the infrastructure in conjunction with the repair of disaster-damaged facilities  There is no preset limit to the amount of PA funds a community may receive; however, PA hazard mitigation measures must be determined to be cost effective  Some examples of hazard mitigation measures that FEMA has predetermined to be cost effective, if they meet certain requirements, include installing shut-off valves on underground pipelines so that damaged sections can be isolated during or following a disaster; securing a roof using straps, clips, or other anchoring systems in locations subject to high winds; and installing shutters on windows or replacing glass with impact-resistant material  Applicants can also propose mitigation measures that are separate from the damaged portions of a facility, such as constructing floodwalls around damaged facilities to avoid future flooding  FEMA evaluates these proposals, considering how the proposed measure protects damaged portions of a facility and whether the measure is reasonable based on the extent of the damage, and determines eligibility on a case-by-case basis  FEMA’s Federal Insurance and Mitigation Administration (FIMA) deploys a cadre of mitigation staff to help coordinate and implement hazard mitigation activities during disaster recovery, including PA hazard mitigation  A primary task of these staff is to identify and assess opportunities to incorporate hazard mitigation into PA projects  Generally, if an applicant seeks to incorporate hazard mitigation measures into a PA project, FIMA’s hazard mitigation staff develop a hazard mitigation proposal  We, the DHS OIG, and others have reported past challenges with FEMA’s management of the PA program related to workforce management, information sharing, and hazard mitigation  For example, we reported in 2008 that the PA program had a shortage of experienced and knowledgeable staff, relied on temporary rotating staff, and provided limited training to their workforce, which impaired PA program delivery and delayed recovery efforts after Hurricanes Katrina and Rita  We found that staff turnover, coupled with information sharing challenges, delayed projects when applicants had to provide the same information each time FEMA assigned new staff and that poorly trained staff provided incomplete and inaccurate information during their initial meetings with applicants or made inaccurate eligibility determinations, which also caused processing delays  We recommended that FEMA strengthen continuity among staff involved in administering the PA program by developing protocols to improve information and document sharing among FEMA staff  In response, in 2013 FEMA instituted a PA Consistency Initiative, which included hiring new managers for FEMA regional offices, stakeholder training on PA program administration, and using a newly developed internal website to allow staff to post and share information to address continuity and knowledge sharing concerns during disaster operations  FEMA also developed the Public Assistance Program Delivery Transition Standard Operating Procedure to facilitate the transfer of responsibility for PA program activities during cases of staff turnover during recovery operations  Despite FEMA’s efforts to implement our recommendations, the DHS-OIG, in 2016, found continuing challenges after Hurricane Sandy with workforce levels, skills, and performance of reservists, who make up the majority of the PA workforce  Regarding information sharing, in 2008, we also identified difficulties sharing documents among federal, state, and local participants in the PA process and difficulties tracking the status of projects  We recommended that FEMA improve information sharing within the PA process by identifying and disseminating practices that facilitate more effective communication among federal, state, and local entities  In response, FEMA proceeded with the implementation of a grant tracking and management system, called EMMIE, which was used previously in 2007  However, in subsequent years we found weaknesses in how FEMA developed the system and the DHS-OIG found that information sharing problems similar to the ones identified in our 2008 report persisted  Regarding hazard mitigation, we reported in 2015 that state and local officials experienced challenges in using PA hazard mitigation during the Hurricane Sandy recovery efforts because PA officials did not consistently prioritize hazard mitigation, and in some cases discouraged mitigation projects during the PA grant application process, among other challenges  We recommended that FEMA assess the challenges state and local officials reported, including the extent to which they can be addressed, and implement corrective actions, as needed  In response to our recommendation, FEMA developed a corrective action plan that included actions and milestones for reviewing, updating, and implementing PA hazard mitigation policy  FEMA also identified the PA new delivery model as a solution for some of the challenges state and local officials reported  Previously, the OIG also reported that PA program officials did not consistently identify eligible PA hazard mitigation projects, and that PA officials did not prioritize the identification of PA hazard mitigation opportunities at the onset of recovery efforts after the 2005 Gulf Coast hurricanes  See appendix I for a summary of findings and the status of our past recommendations on challenges with workforce management, information sharing, and hazard mitigation related to the PA program since our last review in December 2008  FEMA’s own internal reviews and outreach efforts have also identified similar challenges  For example, at FEMA’s request the Homeland Security Studies and Analysis Institute assessed the effectiveness and efficiency of the PA program in 2011  The institute’s report outlined 3 key findings and 23 recommendations relating to the PA preaward process  For example, the report found that FEMA could enhance training programs to develop a skilled and experienced workforce; utilize technology and employ web-based tools to support centralized processing, transparency, and efficient decision making; and identify and address potential special considerations, such as hazard mitigation proposals, as early as possible in the preaward process to improve consistency  In 2014, PA program officials analyzed the PA grant process and used input from agency staff and officials involved in various aspects of the program to identify potential improvements  The resulting Public Assistance Program Realignment report found that challenges in workforce management, information sharing, and hazard mitigation continued, and included recommendations for improvement  For example, the report concluded that a shortage of qualified staff, high turnover, unclear organizational responsibilities, and inconsistent training were long-standing and continuing challenges that impaired the PA pre-award process  In addition, from January 2015 to April 2015, FEMA conducted extensive outreach with more than 260 stakeholders across FEMA headquarters, all 10 regions, 43 states, and 4 tribal nations to discuss challenges in the PA program and opportunities for improvement  For example, stakeholders identified challenges with ineffective information collection during the preaward process and suggested identifying special considerations, such as hazard mitigation, earlier in the PA process as an idea for improvement  In response, FEMA began redesigning the PA preaward process to operationalize the results of its 2014 report and address areas for improvement identified through its outreach efforts  FEMA awarded a contract for program support to help PA officials implement a redesigned PA program in 2015  This included a new process to develop and review grant applications, and obligate PA funds to states affected by disasters; new positions, such as a new program delivery manager who is the single point of contact throughout the grant application process; a new Consolidated Resource Center (CRC) to support field operations by supplementing project development, validation, and review of proposed PA project applications; and a new information system to maintain and share PA grant application documents  As part of the new process, PA program officials identified the need to ensure that staff emphasize special considerations, such as hazard mitigation, earlier in the process  Taken together, these efforts represent FEMA’s “new delivery model” for awarding PA program grants  Enhancements in the PA program under the new delivery model are presented in figure 4  Regarding the new delivery model process, FEMA introduced several changes to enhance outreach to applicants during the “exploratory call”— the first contact between FEMA and local officials—and during the first in- person meeting, called the “recovery scoping meeting ” FEMA also revised decision points during the process, when program officials can request more information from applicants, and applicants can review and approve the completion of project development steps  FEMA also incorporated special considerations, such as hazard mitigation, earlier in the new process during the exploratory calls and recovery scoping meetings  The changes and enhancements to the PA grant award process in the new delivery model are presented in figure 5  The new process divides proposed PA projects based on complexity and type of work into three categories—100 percent completed, standard, and specialized—that PA staff manage to expedite review or assign skilled staff to technical projects as needed  If the applicant has already completed work following a disaster, such as debris removal, it is considered “100 percent completed” and JFO staff collect the necessary documents and provide the information to the CRC staff who complete the development of project applications, validate the information, and complete all necessary reviews  Projects that require repairs and further assistance from PA program staff at the JFO include “standard” and “specialized” projects, which include a site inspection to document damages, before the JFO staff provide the information to the CRC  Further, PA program officials assign PA staff based on their skills and experience to standard projects, which are less technically complex to develop, and specialized projects, which are more technically complex and costly  We discuss the new workforce positions FEMA developed for JFOs and CRCs, the new information system FEMA developed to maintain and share PA grant documents with applicants, and FEMA’s efforts to incorporate hazard mitigation into PA projects later in this report  Since 2015, FEMA has invested almost $9 million to redesign the PA program through the reengineering and implementation of the new delivery model, including about $4 7 million for contract support for implementation, and $4 million for acquisition of the new information system  FEMA tested the new delivery model in a series of selected disasters, using a continuous process improvement approach to assess and improve the process, workforce changes, and information system requirements, prior to implementing the new model for all future disasters  For example, FEMA first tested the new process in Iowa in July 2015 and, in February 2016, PA program officials expanded their test to include all of the new staff positions  In October 2016, PA program officials added the new information system to achieve a comprehensive implementation of all of the elements of the new delivery model for the agency’s response to Hurricane Matthew in Georgia, two additional disasters in Georgia in January 2017, and in Missouri, North Dakota, Wyoming, Vermont, and two disasters in New Hampshire from June through August 2017  The timeline for PA’s implementation of the new delivery model is shown in figure 6  According to program officials, FEMA planned to implement the new model for all future disasters beginning in January 2018  However, historic disaster activity during the 2017 hurricane season accelerated full implementation  As a result, on September 12, 2017, FEMA officials announced that, unless officials determined it would be infeasible in an individual disaster, the program would use the new delivery model in all future disasters  According to FEMA’s 2014 PA Program Realignment report and other program documents, PA officials designed the new delivery model to respond to persistent workforce management challenges related to identifying the required number of staff and needed skills and training, among other things, to improve the efficiency and effectiveness of the PA preaward process  To address these challenges, PA program officials centralized much of the responsibility for processing PA projects in the CRCs, created additional new positions with specialized roles and responsibilities in JFOs, and established training and mentoring programs to help build the new staffs’ skills  In 2016, PA program officials centralized some of the project activities that otherwise were being carried out at individual JFOs at FEMA’s first new CRC in Denton, Texas  Officials did so by establishing 18 new positions, many of which directly correlated with positions that FEMA deployed to individual JFOs in the legacy PA delivery model  According to PA officials, centralizing positions will improve standardization in project processing, and result in a higher quality work product  As part of the new delivery model, PA program officials created a new hazard mitigation liaison position for PA program staff at the CRC that did not previously exist at individual JFOs  The other new positions that PA program officials either created or centralized at the CRC included two specialized positions responsible for costing and validating PA projects  Previously, the PA project specialist deployed to the JFO would complete these tasks and others; however, the consistency of project development varied across the regions and disasters  In contrast, CRC staff are full-time employees who receive training to specialize in completing standardized project development steps for PA projects from multiple disasters on an ongoing basis  Program officials anticipate that centralizing new specialized staff at the CRCs will also reduce PA administrative costs and staffing levels at the JFOs  For example, staff at the CRCs, such as the new hazard mitigation liaisons and insurance and costing specialists, could support project development for multiple disasters and regions simultaneously, whereas PA previously needed to deploy staff to each JFO to fulfill these roles  In addition, once JFOs operating under the new model send projects to the CRCs for processing and review, FEMA can more rapidly close its JFOs, reducing associated administrative costs  For example, following Hurricane Matthew, FEMA credited the new delivery model, in part, with its ability to close the JFO in Georgia sooner than several other JFOs in neighboring states not involved in the implementation of the new delivery model  PA program officials created new positions with more specialized roles and responsibilities to help PA staff at JFOs provide more consistency in the project development process and guidance to applicants  Program officials split the broad responsibilities previously managed at the JFOs by PA crew leaders and project specialists, into two new, specialized positions—the program delivery manager and site inspector  The program delivery manager serves as the applicant’s single point-of-contact throughout the preaward process, manages communication with the applicant, and oversees document collection  All three PA grant applicants we spoke to following Hurricane Matthew in Georgia greatly appreciated the knowledge and assistance provided by their program delivery managers  Site inspectors are responsible for conducting the site inspection to document all disaster-related damages; determining the applicant’s plans for recovery, coordinating with other specialists, and verifying the information collected with the applicant  Officials expect deployed staff at JFOs can complete the fieldwork faster and provide greater continuity of service to applicants  Further, program officials believe that specializing roles will enable them to provide more targeted training, and improve employee satisfaction  Site inspection, hazard mitigation, and environmental and historic preservation specialists, along with a new Public Assistance program mentor, conduct a site inspection with the applicant to document damages to a historic cemetery in Savannah, Georgia, following Hurricane Matthew in 2016  PA program officials designed new training and mentoring programs for the new positions at the CRCs and JFOs and used a continuous feedback process to update and improve the training, position guides, and task books throughout the implementation of the new delivery model, according to PA officials  According to a June 2017 update of the PA Cadre Training Plan, training for the new model has five major focuses: required training and skills for position qualification; on-site refresher training; mentor training; regional-based state, local, tribal, and territorial training; and training on the new information system  Specifically, officials developed six new training courses, and identified which are required for each position under the new delivery model  For example, a program delivery manager at the JFO is required to complete both the program delivery manager and site inspector specialist courses  As of June 2017, PA program officials had provided at least one new model training course to 93 percent of their cadre (including program delivery manager training to 366 individuals and site inspector training to 1,172 individuals) and planned to provide 28 additional courses through September 2017 to the PA cadre  According to regional and CRC officials, the training courses and mentoring from experienced staff helped maximize new staff’s capabilities in the new process  Throughout the third implementation of the new delivery model, JFO and CRC staff, as well as regional PA staff, stakeholders, and applicants, identified staff skills and training as a key area that needed more attention for full implementation of the new delivery model  Our work and FEMA’s after-action reports from the third test in Georgia identified problems with site inspector skills, which affected the timeliness and accuracy of projects  Specialists and managers at the CRC noted that poorly trained site inspectors did not consistently provide the necessary information from the field, which resulted in delays for the CRC staff to process projects, and after-action reports also identified challenges with site inspector skills  According to a PA applicant in Georgia, the inconsistency of skills and experience of their site inspector resulted in the need to conduct a “do-over” site inspection on one of the applicant’s projects, causing delays  PA staff and state officials attribute much of the site inspectors’ skill gaps to their lack of training and experience in the program  According to PA Region officials, providing timely training will be a resource-intensive challenge for implementing the new delivery model for all future disasters  For example, it can be difficult to train reservists before FEMA deploys them to disasters, and many of the program’s experienced reservists have retired or resigned, resulting in few mentors for the program and a high need to provide training to inexperienced and newly hired staff  PA officials and stakeholders also emphasized the need for FEMA to provide additional training for state and local officials to build capacity and support the goals of the new delivery model  For example, according to JFO officials at the third implementation, the new delivery model increases responsibilities for applicants, who will require more applicant training than FEMA currently provides  According to state officials, applicant capabilities vary, and FEMA should provide training to state and local officials on the new delivery model and the information system before a disaster  Skill gaps among applicants could result in inconsistent implementation of the new process, according to PA staff and stakeholders, and PA staff said that training was important to prevent applicants from reverting back to the legacy PA grant application process  To support full implementation of the new delivery model for all disasters, PA program officials have updated training courses for PA staff and applicants, and planned additional training to address these challenges and other lessons learned through the test implementation  For example, PA officials told us they updated the site inspector training program in May 2017 and scheduled a new site inspector training session in August 2017 to include more hands-on training to help address the skill gaps identified for site inspectors  PA officials created a new training course for FEMA’s regional offices, in part to enable regional PA staff to provide new delivery model training to state and local officials  PA officials also planned to develop a self-paced, online course for state and local officials by the end of 2017  PA officials have not fully assessed the workforce needed for JFO field operations, CRC staff, or FIMA’s hazard mitigation staff to support implementation of the new delivery model for all future disasters  PA program officials developed an initial assessment of the total number of staff needed in the field and the CRCs in 2016 to estimate cost savings associated with consolidating and specializing positions at the CRCs and deploying fewer staff to JFOs  However, the assessment did not identify the number of staff required to fill specific positions, including program delivery managers and hazard mitigation specialists, needed to support the new delivery model for full implementation  In reviewing the test implementations of the new delivery model, we found that inadequate staffing levels at the JFOs and CRCs, and with FIMA’s hazard mitigation staff, affected staffs’ ability to achieve the goals of the new delivery model  Staff levels at the JFO  We identified challenges with having the right number of program delivery managers and site inspection specialists to achieve program goals for customer satisfaction, efficiency, and quality in test implementations of the new delivery model  For example, in the second test implementation of the new delivery model in Oregon in 2016, PA did not deploy enough program delivery managers to the disaster, which resulted in unmanageable caseloads for program delivery managers, according to state and PA officials  PA program officials assigned program delivery managers an average caseload of 12 PA applicants, which was more than they could effectively manage, according to PA staff, and program officials aim for a caseload of 8 to 10 applicants  According to state officials, local officials reported they did not always receive the support they needed from program delivery managers who managed caseloads consisting of dozens of projects at multiple sites for each applicant during the Oregon implementation  As a result of overwhelmed program delivery managers, local officials faced challenges understanding their responsibilities, such as recognizing when they needed to provide information for the project development to proceed, according to state officials  PA staff involved with the third test implementation in Georgia in 2016 and 2017 said there were not enough site inspectors or program delivery managers to fully manage the workload at the JFO  Because of the specialization of roles, projects could not move forward when there were not enough staff to execute the next step in the process  For example, PA staff at the JFO said program delivery managers completed recovery scoping meetings rapidly, but faced a bottleneck in scheduling site inspections because there were more applicants awaiting site inspections than could be fulfilled by the number of site inspection specialists available  Staff levels at the CRC  Staff at the CRC reported challenges with staffing levels during the Oregon and Georgia test implementations, and expressed concerns about when PA officials will staff the CRCs to support full implementation of the new model for all disasters  During the Oregon test implementation, a CRC specialist said there were not enough technical specialists to manage the workload and, as a result, PA program officials had to redeploy site inspectors from their JFO field operations to the CRC to complete costing estimates  During the third test in Georgia, quality assurance specialists said that their workload resulted in added stress trying to complete the work in time while adhering to quality standards  According to CRC specialists in Denton, Texas, PA officials had not determined required staff levels for full implementation, but agreed that workload was too high and program officials needed to determine the appropriate staff levels for each CRC to support full implementation  PA officials were still evaluating CRC processing times and workload management from the Oregon and Georgia test implementations to determine staffing needs, according to PA officials  Further, PA program officials plan to establish a second CRC in Winchester, Virginia, before the end of 2017, but have not determined the number of additional permanent full-time staff needed to support the CRCs for full implementation of the new delivery model  Staff levels for the hazard mitigation specialists  PA officials have not identified the number of hazard mitigation specialists in FIMA’s hazard mitigation cadre needed for full implementation of the new delivery model  According to JFO staff, current hazard mitigation staff levels are insufficient to provide the desired in-person participation of hazard mitigation staff on all recovery scoping meetings to share information on hazard mitigation with applicants and help them identify potential mitigation opportunities  A PA program official said officials missed opportunities to pursue hazard mitigation during the test implementation after Hurricane Matthew in Georgia due to lack of hazard mitigation specialists  In addition, for the test implementation in Oregon, there were not enough hazard mitigation specialists to cover all site inspections and implement their new delivery model responsibilities, according to FEMA’s after-action reports  The absence of hazard mitigation specialists in the early stages of PA project development may cause delays in officials’ identifying hazard mitigation opportunities, according to a FIMA official  PA program officials said they did not work with FIMA to determine the appropriate levels of hazard mitigation staff under the new delivery model because they were refining the new process, but as of June 2017 were working with FIMA to do so  One of the key implementation activities in our Business Process Reengineering Assessment Guide includes addressing workforce management issues  Specifically, this includes identifying how many and which employees will be affected by the position changes and retraining  Further, our prior work has found that high-performing organizations identify their current and future workforce needs—including the appropriate number and deployment of staff across the organization— and address workforce gaps, to improve the contribution of critical skills and competencies needed for mission success  According to a PA program official, their initial workforce assessment was not comprehensive because they were still collecting data required to make informed decisions  PA officials agreed that updating their workforce assessments prior to full implementation could be helpful, and acknowledged that program officials needed to be more proactive applying the lessons learned as they pivot from testing to full implementation of the new delivery model in 2018  FEMA also conducts a standard agency wide workforce structure review every 2 to 3 years, which helps officials determine the appropriate disaster workforce levels  As of June 2017, PA officials were working with other offices within FEMA to expedite the agency-wide assessment of the PA and FIMA hazard mitigation cadres, but did not know when they would complete the assessment  PA officials also acknowledged that they faced an aggressive schedule to complete various planned activities for workforce management, training, and other efforts, in support of full implementation, and that they may not be able to complete all efforts as thoroughly as they would like in order to expedite the transition of the PA program to the new delivery model  The gaps in PA workforce assessment in the JFOs, CRCs, and for FIMA’s hazard mitigation cadre present a risk that PA program managers will not have a sufficient workforce to support the goals of the new delivery model  In addition, the timing and implementation of the hiring and training activities for new PA program staff could take multiple months, and program officials will need to know what staff levels are necessary for full implementation of the new delivery model to inform resource decisions for the program in coordination with other agency offices  According to PA program officials, workforce assessment efforts have been delayed as a result of disaster response and recovery efforts related to Hurricanes Harvey, Irma, and Maria  Completing a workforce assessment will help program officials identify gaps in their workforce and skills, which could help PA program officials minimize the effects of long- standing workforce staffing and training challenges on the PA program delivery and inform full implementation for all disasters  costs  For example, EMMIE does not collect information on all of the preaward activities that are part of the PA grant application process  As a result, PA program officials said they, and applicants, must use ad hoc reports and personal tracking documents to manage and monitor the progress of grant applications  PA officials added that EMMIE is not user- friendly and applicants often struggle to access the system  In response to these ongoing challenges, PA program officials developed FAC-Trax— a separate information system from EMMIE—with new capabilities designed to improve transparency, efficiency, and management of the PA program  Specifically, FAC-Trax allows FEMA staff (PA Grants Manager) and applicants (PA Grants Portal), to review, manage, and track current PA project status and documentation  For example, applicants can use FAC- Trax to submit requests for public assistance, upload required project documentation, approve grant application items, and send and receive notifications on grant progress and activities  In addition, the FAC-Trax system includes standardized forms, as well as required fields and tasks that PA program staff and applicants must complete before moving on to the next steps in the PA preaward process  According to PA officials, these capabilities increase transparency, encourage greater applicant involvement, and enhance collaboration and communication between FEMA and grant applicants, to improve efficiency in processing and awarding grant applications and enhance the quality of project development  Further, PA officials said that FAC-Trax could reduce challenges associated with staff turnover during the project development process because the system stores and maintains applicant information and project documentation, making it easier for transitioning staff to assist an applicant  They also said they use FAC-Trax to gather and analyze data that supports management of the PA process, including measuring the timeliness of the grant application process  For example, during the test implementation of the new delivery model in Georgia following Hurricane Matthew, officials were able to document that, on average, program delivery managers took 5 days to conduct the exploratory call and 14 days to hold the recovery scoping meeting with applicants, and CRC officials took 33 days to develop and review grant proposals  Managers use this data to assess staffing needs and identify bottlenecks in the PA process, according to PA officials  FAC-Trax is critical to the new PA delivery model and will be a primary means of sharing grant application documents, tracking ongoing PA projects, and ensuring that FEMA staff and applicants follow PA grant policies and procedures  Given the importance of developing and testing this new information sharing system, we evaluated its development against four key IT management controls—(1) project planning; (2) risk management; (3) requirements development; and (4) systems testing and integration  When implemented effectively, these controls provide assurance that IT systems will be delivered within cost and schedule and meet the capabilities needed by its users  We found that FEMA’s development of FAC-Trax fully satisfied best practices for project planning and risk management, but additional steps are needed to fully satisfy the areas of requirements development and systems testing and integration, as discussed below  See appendix II for the full assessment of each IT management control  PA program officials fully satisfied all five practices in the project planning control area, according to our assessment  Key project planning practices are (1) establishing and maintaining the program’s acquisition strategy, (2) developing and maintaining the overall project plan and obtaining commitment from relevant stakeholders, (3) developing and maintaining the program’s cost estimate, (4) establishing and maintaining the program’s schedule estimate, and (5) identifying the necessary knowledge and skills needed to carry out the program  To address the first and second practices, program officials established detailed plans that describe the acquisition strategy and objectives, the program’s scope, and its framework for using an Agile software development approach, among other key actions  Agile is a method of software development that utilizes an iterative process and constantly improves software based on user needs and feedback  Program officials also developed a plan detailing the program’s approach to deploy and maintain FAC-Trax and established stakeholder groups and an integrated product team to support and oversee the development of FAC-Trax  To address the third and fourth practices, they developed and maintained a master schedule of all implementation tasks and milestones through project completion, and developed a life-cycle cost estimate of over $19 million  Additionally, FAC-Trax’s acquisition performance baseline describes the system’s minimum acceptable and desired baselines for performance, schedule, and cost  Lastly, in regards to the fifth practice, program officials identified the knowledge and skills needed to carry out the program in the FAC-Trax Request for Proposal and FAC-Trax Capability Development Plan  PA program officials fully satisfied all four practices in the risk management control area, according to our assessment  Key risk management practices are (1) identifying risks, threats, and vulnerabilities that could negatively affect work efforts, (2) evaluating and categorizing each identified risk using defined risk categories and parameters, (3) developing risk mitigation plans for selected risks, and (4) monitoring the status of each risk periodically and implementing the risk mitigation plan as appropriate  To address the first and second practices, program officials identified key risks that could negatively affect FAC-Trax in a “risk register”—an online site used to track risks, issues, and mitigating actions  As of May 2017, officials had identified 13 risks in the risk register—four open and nine closed—and evaluated and categorized the identified risks based on the probability of occurrence and scope, schedule, and cost impacts  For example, program officials reported that two of its open risks have a “medium” risk rating—meaning the risk has the potential to slightly affect project cost, schedule, or performance  To address the third and fourth practices, program officials developed and documented risk mitigation plans for all identified risks  For example, program officials planned to mitigate the risk of limited engagement of subject matter experts by identifying and engaging with appropriate experts through workshops, and monitoring the capability development process to identify any issues that may cause project delays  In addition, PA program officials documented the responsible officials, reevaluation date, and risk status, among other things, for each risk in the register, and reviewed and updated risks during weekly and monthly program reviews with stakeholders throughout FEMA  PA program officials fully satisfied four out of five practices in the requirements development control area, according to our assessment  Key requirements development practices are (1) eliciting stakeholder needs, expectations, and constraints, and transforming them into prioritized customer requirements; (2) developing and reviewing operational concepts and scenarios to refine and discover requirements; (3) analyzing requirements to ensure that they are complete, feasible, and verifiable; (4) analyzing requirements to balance stakeholder needs and constraints; and (5) testing and validating the system as it is being developed  To address the first and second practices, program officials developed a requirements management plan outlining how officials capture, assess, and plan for FAC-Trax enhancements, and established a change control process to review, prioritize, and verify user requests for changes to the system and feedback  As of May 2017, the PA program office received 734 change requests related to FAC-Trax, of which program officials completed 420 changes and planned to address an additional 277 entries  Program officials also developed a functional requirements document outlining the high-level requirements for FAC- Trax and detailed operational concepts and scenarios for each phase of the preaward process in the system’s concept of operations  To address the fourth practice, program officials created a standard template to analyze and document the user needs and acceptance criteria for planned system capabilities in March 2017  In addition, PA program officials identified risks and dependencies for recommended changes to FAC-Trax, and balanced the cost and priority of system enhancements as part of the change control process  Lastly, regarding the fifth practice, program officials tested and evaluated FAC-Trax during development, which included validating system enhancements through user acceptance testing  However, program officials did not fully address the third practice— analyzing requirements to ensure they are complete, feasible, and verifiable—because they did not ensure detailed user requirements were necessary and sufficient by tracking them back to higher-level requirements  For example, although program officials reviewed change requests for completeness and followed up with users to verify requirements, officials did not track system enhancements, made in response to detailed user requirements (e g , allowing users to search PA projects by project number), back to the high-level requirements (e g , storing data and information provided by the applicant) identified in the FAC-Trax functional requirements document and performance work statement  Officials did not track system enhancements back to high-level requirements because they did not have a complete understanding of basic user needs and system requirements at the beginning of the FAC- Trax effort, according to the PA program manager  A PA official also said the change control process was a way to identify the basic capabilities FAC-Trax needed to have and that tracking enhancements back to high- level requirements could have made the change control process more difficult to manage, and reduced user participation if, for example, users needed to understand how their change requests related to high-level requirements  However, program officials could have tracked enhancements back to high-level requirements themselves using the change control process without putting any additional burden on users  Despite not having a complete understanding of user needs and system requirements at the beginning of the FAC-Trax effort, analyzing whether users’ change requests satisfy higher-level requirements identified in key design and planning documents would have provided officials with a basis for more detailed and precise requirements throughout project development and helped them better manage the project, according to IT management controls  Further, according to the PMBOK® Guide, tracking or measuring system capabilities against approved requirements is a key process for managing a project’s scope, measuring project completion, and ensuring the project meets user needs and expectations  Program officials acknowledged the importance of tracking system enhancements back to documented system requirements  Ensuring that FAC-Trax meets user needs and expectations is especially important because the information system is key to the success of the new delivery model, according to PA officials  By analyzing progress made on documented, high-level requirements, a step that reflects a key IT management control for requirements development, the PA program will have greater assurance that FAC-Trax will provide functionality that meets user needs and expectations  PA program officials did not fully satisfy either of the two practices in the systems testing and integration control area, according to our assessment  Key systems testing and integration practices are (1) developing test plans and test cases, which include a description of the overall approach for system testing, the set of tasks necessary to prepare for and perform testing, the roles and responsibilities for individuals or groups responsible for testing, and criteria to determine whether the system has passed or failed testing; and (2) developing a systems integration plan to identify all systems to be integrated, describe how integration problems are to be documented and resolved, define roles and responsibilities of all relevant participants, and establish a sequence and schedule for every integration step  In regards to the first practice, PA program officials and the FAC-Trax contractor established a test plan that identifies the method and strategy to perform testing, including the necessary tasks, such as responding to user feedback and testing errors, and incorporating necessary resolutions into future work, testing parameters, and the roles and responsibilities of the individuals responsible for testing  However, program officials have not developed system testing criteria to evaluate FAC-Trax, which would align with the practice described above of using criteria to determine whether the system has passed or failed testing  A key feature of Agile software development is the “definition of done”—a set of clear, comprehensive, and objective criteria, that the government should use to evaluate software after each iteration of development  PA program officials said they did not establish a definition of done because officials initially managing the FAC-Trax effort lacked familiarity with system development in the Agile environment  Officials acknowledged the importance of establishing a definition of done and said they are planning to develop one, but have not identified how or when to incorporate it into the development process  According to the TechFAR—the government’s handbook for procuring digital services using Agile processes—the government and vendor should establish this definition after contract award at the beginning of each cycle of software development  By establishing criteria, such as a definition of done, to evaluate the system—a step that reflects a key IT management control for system testing and is an effective practice for applying Agile to software development—the PA program will have greater assurance that FAC- Trax is usable and responsive to specified requirements  In regards to the second practice, PA program officials developed a systems integration plan in June 2017 that identified the potential for integration of FAC-Trax with four FEMA systems, including EMMIE  In addition, program officials included a description of how staff should document integration problems and the resolution of problems in FAC- Trax development and test plans  However, the systems integration plan does not define roles and responsibilities of all participants for system integration activities or establish a sequence and schedule for every integration step for the four FEMA systems  PA officials said that system integration planning for FAC-Trax is in the early stages, but acknowledged the importance of these elements of system integration planning  Officials plan to define roles and responsibilities of all participants for system integration activities and develop the sequence and schedule for every integration step as they add new systems to the FAC-Trax development plan and obtain funding needed for their integration  Nonetheless, FEMA has used FAC-Trax for selected PA disasters since October 2016 and plans to use FAC-Trax for all future disasters  According to IT management controls, agencies should establish the systems integration plan early in the project and revise it to reflect evolving and emerging user needs  By ensuring that the FAC- Trax systems integration plan defines the roles and responsibilities of relevant participants for all integration relationships and establishes a sequence and schedule for every integration step, the PA program will have greater assurance that FAC-Trax functions properly with other systems and meets user needs  FEMA’s new delivery model enhances participation of hazard mitigation staff with the goal of identifying opportunities for mitigation earlier in the PA preaward process, according to PA officials  Two key changes related to hazard mitigation under the new model include (1) an emphasis on engaging with hazard mitigation specialists at the JFO earlier in the PA process and involving them in specific PA preaward activities and (2) the establishment of the PA program’s hazard mitigation liaison at the CRC  For example, position guides direct program delivery managers to coordinate with FIMA’s hazard mitigation specialists prior to recovery scoping meetings, and site inspectors to coordinate with hazard mitigation specialists prior to site inspections to discuss a PA grant applicant’s damages and any potential mitigation opportunities  PA program officials also developed guidance for conducting the exploratory call and the recovery scoping meeting with applicants, which include questions for PA staff to ask on the applicant’s interest in or plans for incorporating hazard mitigation into potential projects  In addition, a new hazard mitigation liaison at the CRC is responsible for reviewing PA projects for hazard mitigation opportunities and serving as a mitigation subject matter expert for the PA program  According to data provided by FEMA, PA grant applicants incorporated hazard mitigation into approximately 18 percent of permanent work projects for all disasters nationwide from 2012 to 2015  During test implementation of the new delivery model, state, PA, and FIMA officials all reported an increase in the number of hazard mitigation activities on PA permanent work projects  For example, state officials who participated in the second new model test in Oregon said that effective communication and coordination between PA and hazard mitigation staff resulted in applicants incorporating hazard mitigation into over 60 percent of permanent work projects  Furthermore, PA officials reported an increase in hazard mitigation during the third test implementation of the new model in Georgia following Hurricane Matthew, where approximately 16 percent of permanent work projects included mitigation, as of June 2017  This represents an increase compared to the PA program’s estimate for the proportion of projects that incorporate hazard mitigation among previous PA hurricane disasters in Georgia, which was about 3 percent, according to PA officials  While PA officials are trying to increase hazard mitigation through the new delivery model, not all disasters present the same number of opportunities to incorporate hazard mitigation  First, the PA program only incorporates hazard mitigation measures for permanent work projects, such as repairs to roads, bridges, and buildings  For example, as of June 2017, approximately 60 percent of the projects FEMA funded in Georgia for the third test implementation after Hurricane Matthew were for emergency work, which is not eligible for hazard mitigation measures  Second, the PA program only funds mitigation measures that officials determine to be cost-effective  In addition, we have previously reported on other factors that affect whether applicants incorporate hazard mitigation into PA projects, such as their capacity to manage and ability to fund hazard mitigation projects  National Planning for Hazard Mitigation In our 2015 report on disaster resilience following Hurricane Sandy, we noted that disaster affected areas have different threats and vulnerabilities, and local stakeholders make the ultimate determination whether or not to incorporate hazard mitigation into a project  Further, without a strategic approach to making disaster resilience investments, the federal government and its nonfederal partners may be unable to fully capitalize on opportunities for mitigation on the greatest known threats and hazards  We recommended that the Mitigation Framework Leadership Group develop an investment strategy to help ensure that federal funds expended to enhance disaster resilience achieve the goal of reducing the nation’s fiscal exposure because of climate change and the rise in the number of federal major disaster declarations as effectively and efficiently as possible  In response, the Federal Emergency Management Agency (FEMA) plans to issue a final National Mitigation Investment Strategy in 2018  The goals of this strategy include increasing the effectiveness of investments in reducing disaster losses and increasing resilience, and improving coordination of disaster risk management among federal, state, local, tribal, territorial, and private entities  Although the new model establishes hazard mitigation activities for PA and FIMA staff in the preaward process, it does not standardize and prioritize hazard mitigation planning at JFOs in the way FEMA has done with prior PA program policy  Specifically, FEMA’s 2007 PA program policy standardized planning for hazard mitigation across PA recovery efforts by stating that agency and state officials should issue a memorandum of understanding (MOU) early in the disaster, outlining how PA hazard mitigation will be addressed for the disaster, including what mitigation measures will be emphasized, and identifying applicable codes and standards, and any potential integration with other mitigation grant programs  However, PA program officials did not include guidance that standardizes planning for hazard mitigation, such as encouraging the use of an MOU, in FEMA’s 2010 PA program policy, the most recent update to the Public Assistance Program and Policy Guide in April 2017, or the New Delivery Model Operations Manual  As a result, FIMA officials said FEMA and state officials do not consistently issue MOUs that outline how FEMA and the state plan to promote PA hazard mitigation during the recovery effort, explaining that the use of the MOU is based on the preferences and priorities of the FEMA officials involved  When not issuing an MOU, FIMA hazard mitigation staff and PA officials at the JFO meet to determine the extent which hazard mitigation staff interact directly with applicants regarding PA hazard mitigation during the recovery process, according to a FIMA official  Having a consistent approach to planning for and prioritizing hazard mitigation across all disasters is important for FEMA, given that FEMA experienced challenges consistently prioritizing and integrating hazard mitigation across PA recovery efforts, according to GAO and others  For example, in our 2015 report on resilience in the Hurricane Sandy recovery, we found that state and local officials experienced challenges maximizing disaster resilience in the recovery effort because PA officials did not consistently prioritize hazard mitigation during project development  According to FEMA’s National Mitigation Framework, planning is vital for mitigation efforts during disaster recovery, and federal, state, and local officials should establish procedures that emphasize a coordinated delivery of mitigation activities and capitalize on opportunities to reduce future disaster losses  Similarly, the Recovery Federal Interagency Operational Plan, which supports FEMA’s National Disaster Recovery Framework, identifies planning as a key task for identifying mitigation opportunities and integrating risk reduction considerations into decisions and investments during the recovery process  FIMA officials agreed that including the development of a formal plan, such as the historical 2007 PA program policy regarding the use of MOUs, for PA hazard mitigation in operations guidance would help program officials plan for and prioritize hazard mitigation  They noted that FIMA’s hazard mitigation field operations guide includes procedures for implementing proposed MOUs to achieve mitigation goals  PA program officials said that, in light of changes to the PA process under the new model and subsequent updates to program policies, the MOU policy from the 2007 PA program policy was outdated  But officials agreed that planning for and prioritizing hazard mitigation at the operational level is important and said they were examining additional ways to incorporate these activities early in the PA process  As FEMA continues to implement the new model, establishing procedures to standardize hazard mitigation planning for each disaster, as it did through prior policy, could improve the prioritization of hazard mitigation in PA recovery efforts and increase the effectiveness of mitigation for reducing disaster losses and increasing resilience  PA program officials developed performance objectives and measures for hazard mitigation in the new delivery model, but could add measures to better align performance assessment for the PA program with FEMA’s broader strategic goals for hazard mitigation  In its strategic plan for 2014–2018, FEMA established an agency-wide goal to increase the percentage of FEMA-funded disaster projects, such as those under the PA program, that provide mitigation above local, state, and federal building code requirements by 5 percentage points by the end of fiscal year 2018  For example, local building codes may require measures for new construction to mitigate against future damage  To align with FEMA’s strategic goal, PA officials would also need to measure the number of PA projects that included mitigation measures that bring any repaired infrastructure to a level above applicable building codes  However, under the new model, FEMA officials developed performance objectives (and associated measures) to increase the number of projects that include hazard mitigation by 5 percent, and increase the total dollars spent on hazard mitigation by 2 percent  While these measures could help to incentivize mitigation, they are not tied to building codes and do not include specific information that FEMA could use to continually assess the PA program’s contributions to meeting the agency’s strategic goal  According to Standards for Internal Control in the Federal Government, agency management should design control activities, such as establishing and reviewing performance measures, to achieve the agency’s objectives  In addition, our work on leading public sector organizations has found that such organizations assess the extent to which their programs and activities contribute to meeting their mission and desired outcomes, and strive to establish clear hierarchies of performance goals and measures  A clear connection between performance measures and program offices helps to both reinforce accountability and ensure that, in their day-to-day activities, managers keep in mind the outcomes their organization is striving to achieve  FEMA’s ability to evaluate and report on PA hazard mitigation data is constrained, but officials are addressing this challenge through the development of data reporting and analytics capabilities for the PA program’s new information system, according to PA officials  PA program officials developed measures they could use to evaluate the new model during test implementation and compare new model performance to the legacy PA process, and agreed that aligning PA program hazard mitigation goals with FEMA’s agency-wide strategic goals would be helpful  As FEMA continues to develop and implement the new model, developing performance measures and objectives to better inform and support the agency’s broader strategic goals could help to ensure that FEMA capitalizes on hazard mitigation opportunities in PA recovery efforts  FEMA’s Public Assistance grant program is a complicated, multi-billion dollar program that is critical to helping state and local communities rebuild and recover after a major disaster  In recent years, FEMA has undertaken a major reengineering effort to make the PA preaward process simpler and more efficient for applicants and to address challenges encountered during recovery from past disasters  FEMA’s new delivery model represents a significant opportunity to strengthen the PA program and address these past challenges, and growing pains are to be expected when implementing any large reengineering effort  Further, FEMA officials work to implement these changes while supporting response and recovery following disasters, including the catastrophic flooding from Hurricane Harvey in August 2017 and widespread damages from Hurricanes Irma and Maria in September 2017  As such, it is critical that feedback obtained and lessons learned while testing the new model inform decisions and actions as FEMA proceeds with full implementation for all disasters, including the complex recovery efforts in the states and territories affected by Hurricanes Harvey, Irma, and Maria  FEMA has redesigned the PA delivery model to address various challenges related to workforce management, information sharing with state and local grantees, and incorporating hazard mitigation into PA projects  FEMA has developed new workforce processes, training, and positions to address past challenges, but completing a workforce assessment that identifies the number of staff needed will inform workforce management and resource allocation decisions to help FEMA ensure a more successful implementation  This is particularly important as FEMA is using the new model for the long-term recovery from the 2017 hurricanes, and FEMA faces capacity challenges as its workforce is stretched thin  Further, FEMA’s new FAC-Trax information sharing system provides FEMA and state and local applicants and grantees with better capabilities to address past challenges in managing and tracking PA projects  In developing FAC-Trax, FEMA implemented many of the key IT management controls that can help ensure that new IT systems are implemented effectively  However, additional steps are needed to fully satisfy the areas of requirements development and systems testing and integration  Finally, FEMA has taken some actions to better promote hazard mitigation as part of its new PA model  However, more consistent planning for hazard mitigation following a PA disaster and developing specific performance measures and objectives that better align with and support the agency’s broader strategic goals related to hazard mitigation could help to ensure that mitigation is incorporated into recovery efforts, which presents an opportunity to encourage disaster resilience and reduce federal fiscal exposure from recurring catastrophic natural disasters  We are making the following five recommendations to FEMA’s Assistant Administrator for Recovery: The FEMA Assistant Administrator for Recovery should complete a workforce staffing assessment that identifies the appropriate number of staff needed at joint field offices, Consolidated Resource Centers, and in FIMA’s hazard mitigation cadre to implement the new delivery model nationwide  (Recommendation 1) The FEMA Assistant Administrator for Recovery should establish controls for tracking FAC-Trax capabilities to the system’s functional and operational requirements to more fully satisfy requirements development controls and ensure that the new information system provides capabilities that meets users’ needs and expectations  (Recommendation 2) The FEMA Assistant Administrator for Recovery should establish system testing criteria, such as a “definition of done,” to assess FAC- Trax as it is developed; define the roles and responsibilities of all participants; and develop the sequence and schedule for integration of other systems with FAC-Trax to more fully satisfy systems testing and integration controls  (Recommendation 3) The FEMA Assistant Administrator for Recovery, in coordination with the Associate Administrator of the Federal Insurance and Mitigation Administration, should implement procedures to standardize planning for addressing PA hazard mitigation at the joint field offices, for example, by requiring FEMA and state officials to develop a memorandum of understanding outlining how they will prioritize and address hazard mitigation following a disaster as it did through prior policy  (Recommendation 4) The FEMA Assistant Administrator for Recovery, in coordination with the Associate Administrator of the Federal Insurance and Mitigation Administration, should develop performance measures and associated objectives for the new delivery model to better align with FEMA’s strategic goal for hazard mitigation in the recovery process  (Recommendation 5) We provided a draft of this report to DHS and FEMA for review and comment  DHS provided written comments, which are reproduced in appendix III  In its comments, DHS concurred with our recommendations and described actions planned to address them  FEMA also provided technical comments, which we incorporated as appropriate  With regard to our first recommendation, that FEMA complete a workforce staffing assessment that identifies the number of staff needed at joint field offices, Consolidated Resource Centers, and FIMA’s hazard mitigation cadre, DHS stated that PA, in coordination with the Field Operations Directorate and FIMA, will continue to refine and evaluate staffing needs and update the cadre force structures under the new delivery model  DHS estimated that this effort would be completed by June 28, 2019  This action, if fully implemented, should address the intent of the recommendation  With regard to our second recommendation, that FEMA establish controls for tracking FAC-Trax capabilities to ensure the new information system meets users’ needs, DHS stated that Recovery program managers will update the FAC-Trax Requirements Management Plan and the FAC-Trax Release Plan to ensure the tracking and traceability of FAC-Trax functional and operational requirements  DHS estimated that this effort would be completed by January 31, 2018  This action, if fully implemented, should address the intent of the recommendation  With regard to our third recommendation, that FEMA establish systems testing criteria to assess the development of FAC-Trax; and define the roles and responsibilities, and sequence and schedule for system integration, DHS stated that Recovery program managers will update the FAC-Trax System Integration Plan to include integration with the Deployment Tracking System, Enterprise Data Warehouse, Preliminary Damage Assessment interface, and State Grants Management system interface  DHS estimated that this effort would be completed by June 29, 2018  This action, if fully implemented, should address the intent of the recommendation  With regard to our fourth recommendation, that FEMA implement procedures to standardize planning for addressing PA hazard mitigation at the JFO, DHS stated that PA will update current process documents or develop new documents to better incorporate mitigation into the operational planning phase of the new delivery model  DHS estimated that this effort would be completed by July 31, 2018  This action, if fully implemented, should address the intent of the recommendation  With regard to our fifth recommendation, that PA coordinate with FIMA to develop performance measures and associated objectives for the new delivery model that better align with FEMA’s strategic goals for hazard mitigation in the recovery process, DHS stated that PA will reconvene the PA-Mitigation working group to develop and refine PA related hazard mitigation performance measures  DHS estimated that this effort would be completed by June 29, 2018  This action, if fully implemented, should address the intent of the recommendation  We are sending copies of this report to the Secretary of Homeland Security and interested congressional committees  If you or your staff have any questions about this report, please contact me at (404) 679-1875 or CurrieC@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix IV  Appendix II: Assessment of Information Technology Management Controls for the FEMA Applicant Case Tracker (FAC-Trax) Table 2 shows details on the Federal Emergency Management Agency (FEMA) Public Assistance (PA) program office’s implementation of key practices across four information technology (IT) management control areas for its new information system, the FEMA Applicant Case Tracker (FAC-Trax)  PA developed FAC-Trax as a web-based project tracking and case management system to supplement the Emergency Management Mission Integrated Environment (EMMIE) and help resolve long-standing information sharing challenges  To determine the extent to which the FAC-Trax program office implemented IT management controls, we reviewed documentation from the FAC-Trax program and compared it to key management best practices, including the Software Engineering Institute’s Capability Maturity Model® Integration for Acquisition and Development, the Project Management Institute’s Guide to the Project Management Body of Knowledge (PMBOK® Guide), and the Institute of Electrical and Electronics Engineers’ Standard for Software and System Test Documentation  We assessed the program as having fully implemented a practice if the agency provided evidence that it fully addressed the practice; partially implemented if the agency provided evidence that it addressed some, but not all, portions of the practice; and not implemented if the agency did not provide any evidence that it addressed the practice  Table 2  Public Assistance (PA) Program Office’s Implementation of Key Information Technology Management Controls for FAC-Trax PA program officials developed an acquisition plan for FAC-Trax identifying the capabilities the system is intended to deliver, the acquisition approach, and acquisition objectives  Additionally, program officials developed a capability development plan outlining a strategy for the program to obtain approval to acquire FAC-Trax  Lastly, program officials developed a systems engineering plan describing the program’s scope and its framework for using an Agile development approach, as well as a deployment, support, and maintenance plan for FAC-Trax  PA program officials developed an acquisition program baseline detailing FAC-Trax’s cost parameters and a life-cycle cost estimate for the system  As of May 2017, the life- cycle cost estimate for FAC-Trax through fiscal year (FY) 2023 is approximately $19 3 million  PA program officials updated the life-cycle cost estimate for FYs 2016 and 2017 after price negotiations with the FAC-Trax contractor, and will continue to update the estimate as annual budgets are approved, according to the Integrated Logistic Support Plan  The contracting officer’s representative for FAC-Trax performs a cost review at the end of each month, according to program officials  Furthermore, the contractor’s weekly status report includes information on the number of hours worked and the percent of contract value spent  Program officials also review program costs with Office of Response and Recovery, PA, Office of the Chief Information Officer (OCIO), and other program office stakeholders during a weekly program review  PA program officials developed an acquisition program baseline detailing FAC-Trax’s schedule parameters, as well as an integrated master schedule for the system  The integrated master schedule identifies tasks, major milestones, and task dependencies  The PA program manager reviews and updates the integrated master schedule on a weekly basis  Program officials also review FAC-Trax’s schedule with Office of Response and Recovery, PA, OCIO, and other program office stakeholders during a weekly program review  PA program officials identified the knowledge and skills needed to carry out the program in FAC-Trax contract documentation and the capability development plan  Specifically, program officials included an attachment to the FAC-Trax contract listing the required labor categories and corresponding functional position descriptions  Program officials also described the role, position type, minimum grade, and minimum certification for required personnel resources for the acquisition, development, and implementation of FAC-Trax  PA program officials developed, reviewed, and maintained project planning documents and obtained commitment from relevant stakeholders  For example, program officials reviewed and updated the integrated master schedule and costs on a weekly and monthly basis, respectively  Further, program officials reviewed the status of project elements, such as the schedule, quality and technical issues, stakeholders, staffing, cost, and risks, with Office of Response and Recovery, PA, OCIO, and other program office stakeholders during a weekly program review  PA program officials also established tactical, functional, and stakeholder groups, as well as an Integrated Product Team to support and oversee the development of FAC-Trax  FEMA’s Recovery Technology Programs Division (RTPD) has a division-level risk management plan that serves as guidance for all Recovery systems, including FAC- Trax  Program officials identified key risks that could negatively affect FAC-Trax work efforts in RTPD’s “risk register”—an online site used to track risks, issues, and mitigating actions for the division and each program office  Program officials also identified five technical, cost, and schedule risks in the FAC-Trax acquisition plan  Program officials included one of these risks in the risk register, while the remaining four were managed outside of the register  As of May 2017, program officials had identified 13 risks in its risk register—four open and nine closed  The four open risks were (1) limited subject matter expert engagement during requirements development, (2) vacancies in program management office support positions, (3) unresolved service level agreement support and funding issues, and (4) the loss of the authority to operate due to a Trusted Internet Connection that is not compliant with Department of Homeland Security security policy  Program officials evaluated and categorized the identified risks based on the probability of occurrence and scope, schedule, and cost impacts  These four points of measurement are used to calculate an overall risk score  The risk score helps program officials determine a risk’s risk rating—low, medium, or high  For example, program officials reported that two of its open risks have a “medium” risk rating—meaning the risk has the potential to slightly impact project cost, schedule, or performance  In addition, program officials detailed the risk category, probability, and impact for the five risks identified in the FAC-Trax acquisition plan  Program officials developed risk mitigation and contingency plans for each risk in the risk register  For example, program officials planned to mitigate the open risk concerning subject matter expert engagement, by identifying and engaging with appropriate subject matter experts through requirements development workshops scheduled in advance of the sprint they are to support, and monitoring the development of user stories to identify any issues that may cause delays  In addition, program officials described the risk management plan and responsible officials for the five risks identified in the FAC-Trax acquisition plan  PA program officials review and update program risks during a monthly program meeting  Program officials also review program risks with Office of Response and Recovery, PA, OCIO, and other program office stakeholders during a weekly program review  Furthermore, the FAC-Trax contractor provides a weekly status update which includes a section on identified risks  Program officials established re-evaluation dates and recorded updates, including any actions taken, for each risk in the risk register  In addition, program officials were able to provide updates on the four risks identified in the FAC-Trax acquisition plan and managed outside of the register  According to PA officials, these risks were addressed and closed by the approval of program planning documents, such as the mission needs statement, concept of operations, and operational requirements document, following the solutions engineering review, which demonstrates the readiness of the program to proceed with the procurement, in September 2016  Program officials established a requirements management plan outlining how it captures, assesses, and plans for FAC-Trax enhancements, and established a change control process to review, prioritize, and verify user requests for changes to the system and feedback  As of May 2017, the PA program office received 734 change requests related to FAC-Trax, of which program officials completed 420 changes and planned to address an additional 277 entries  PA program officials also facilitated workshops to gather requirements for specific user groups and obtained additional requirements for FAC-Trax through customer feedback on a temporary technology tool— an Access database referred to as the Public Assistance Recovery Information System—used to support an early stage of the new model implementation  Further, program officials developed a functional requirements document outlining the high-level functional and operational requirements for FAC-Trax  PA program officials developed a concept of operations for FAC-Trax detailing operating concepts and scenarios for each phase of the PA preaward process  Program officials also detailed the workflow, phases, business functions, and data inputs and outputs for the re-engineered PA process in FAC-Trax’s functional requirements document  In March 2017, program officials developed a standard template to describe the process, tasks, and data inputs and outputs for specific system capabilities  As part of the change control process, PA program officials meet three times a week to discuss and prioritize change requests  Specifically, program officials review submissions to the change control form to ensure completeness, validate impacts and root cause, and research details for incoming requests  PA program officials also follow up with users to understand and verify requirements  In March 2017, program officials developed a standard template to capture acceptance criteria for specific requirements  However, PA program officials do not track system enhancements back to the high-level requirements identified in FAC-Trax’s operational and functional requirements documentation and performance work statement  PA program officials identified system requirements and constraints in the FAC-Trax concept of operations and functional and operational requirements documents  Further, through its change control process, program officials collect suggestions, issues, and feedback on FAC-Trax and system enhancements from stakeholders, identify risks for change requests, and balance prioritized requirements and estimated level of efforts with projected costs prior to each sprint  In March 2017, program officials developed a standard template to analyze and document the urgency and need for specific requirements  PA program officials and the FAC-Trax contractor established a testing and evaluation plan for the system, developed acceptance criteria for user stories, and obtained feedback from users during and after testing  The testing process concludes with user acceptance testing (UAT)  If a change request fails during UAT or a new requirement is discovered during development, the PA program will capture the failed request or new requirement in the product backlog for implementation in a future product release  Key practices Systems testing and integration Developing test plans and test cases PA program officials and the FAC-Trax contractor tested and evaluated the system during development  The FAC-Trax test plan identifies the method and strategy to perform the testing, including the necessary tasks, testing parameters, and the roles and responsibilities of the individuals responsible for testing  However, program officials did not develop system testing criteria to evaluate FAC-Trax  A key feature of Agile software development is the “definition of done”—a set of clear, comprehensive, and objective criteria, that the government should use to evaluate software after each iteration of development  PA program officials developed a systems integration plan in June 2017 that identifies potential integration of FAC-Trax and four FEMA systems, including the Emergency Management Mission Integrated Environment  Specifically, the plan includes data requirements and standards; descriptions of the four systems FEMA plans to integrate with FAC-Trax and the proposed relationship for each connection; and security and access management requirements  In addition, program officials included a description of how integration problems are to be documented and resolved in FAC-Trax development and test plans  However, the systems integration plan does not define roles and responsibilities of all participants for system integration activities or establish a sequence and schedule for every integration step for the four FEMA systems  ● Fully implemented: The agency provided evidence that it fully addressed this practice  ◐ Partially implemented: The agency provided evidence that it addressed some, but not all, portions of this practice  ◌ Not implemented: The agency did not provide any evidence that it addressed this practice  In addition to the contact named above, Chris Keisling (Assistant Director), Amanda R  Parker (Analyst-in-Charge), Mathew Bader, Allison Bawden, Anthony Bova, Eric Hauswirth, Susan Hsu, Rianna Jansen, Justin Jaynes, Tracey King, Matthew T  Lowney, Heidi Nielson, Claire Peachey, Brenda Rabinowitz, Ryan Siegel, Martin Skorczynski, Niti Tandon, Walter K  Vance, James T  Williams, and Eric Winter made key contributions to this report 


Report 2: The federal government owns and leases hundreds of thousands of buildings across the country that cost billions of dollars annually to operate and maintain  In recent years, the federal government has taken steps to improve the management of federal real property and address long-standing issues by undertaking several government-wide initiatives and issuing memorandums to the CFO Act agencies  Within the executive branch, OMB and GSA provide leadership in managing federal real property  As the chief management office for the executive branch, OMB oversees how federal agencies devise, implement, manage, and evaluate programs and policies  OMB provides direction to federal agencies by, among other things, issuing policies and memorandums on real property management  In 2012, OMB issued a memorandum that required agencies to move aggressively to dispose of excess properties held by the federal government and more efficiently use real estate assets  This memorandum initially laid out the requirement to “freeze the footprint ” In 2013, OMB issued a memorandum clarifying the Freeze the Footprint policy  This memorandum required agencies going forward to maintain no more than their fiscal year 2012 total square footage of domestic office and warehouse space  The policy required agencies to specifically identify existing properties to be disposed of to offset any new property acquisitions  In March 2015, OMB transitioned from freezing the federal government’s real property footprint to reducing it  Specifically, OMB issued the National Strategy for the Efficient Use of Real Property (National Strategy) to provide a framework to guide agencies’ real property management, increase efficient real property use, control costs, and reduce federal real property  The National Strategy outlined three key steps to improve real property management: (1) freeze growth in the inventory; (2) measure performance and use data to identify opportunities to improve the efficiency of the real property portfolio; and (3) reduce the size of the inventory by consolidating, co-locating, and disposing of properties  OMB also issued the RTF policy which clarified existing policy to dispose of excess properties and promote more efficient use of real property assets  The RTF policy requires agencies to: (1) submit annual Real Property Efficiency Plans (Plan) to GSA and OMB; (2) issue a policy that specifies a design standard for maximum useable square feet by workstation for use in domestic office space; (3) set and specify in their Plans annual reduction targets for their domestic office and warehouse space for a 5-year period; (4) set and specify in their Plans annual reduction targets for domestic owned building properties reported in the Federal Real Property Profile; and (5) continue to not increase the square footage of their domestic inventory of office and warehouse space  Additionally, agencies must identify in their Plans potential projects related to office and warehouse consolidation, co-location, disposal, as well as construction and acquisition efforts  OMB is responsible for reporting the progress of agencies’ efforts in reducing the amount of federal real property space under the RTF policy  GSA has two key leadership responsibilities related to real property management  First, GSA’s Public Buildings Service functions as the federal government’s principal landlord  In this role, GSA acquires, manages, and disposes of federally owned real property for which it has custody and control on behalf of federal agencies that occupy the space  Additionally, GSA leases commercial buildings on behalf of agencies and manages the lease agreements  In these situations, GSA executes an occupancy agreement with a customer agency for each space assignment that is similar to a sublease between GSA and the agency  The occupancy agreement outlines both the financial specifics of the agreement and the responsibilities of GSA and the customer agency  There are certain unique advantages for customer agencies when GSA leases on their behalf  For example, GSA is able to enter into longer-term leases, and agencies can release space back to GSA with 4 months’ written notice if certain conditions are met, relieving the agencies of the cost for the returned space  Second, GSA’s Office of Government-wide Policy is responsible for, among other things, identifying, evaluating, and promoting best practices to improve the efficiency of management processes  In this policy role, GSA provides guidance for federal agencies and publishes performance measures  It also maintains the Federal Real Property Profile, a real property inventory database that contains information on federal real property government-wide  Based on our review of agencies’ 2016 and 2017 Plans, we found that all 24 CFO Act agencies described strategies for reducing office and warehouse space  As previously mentioned, these annual Plans must include all potential projects related to office and warehouse consolidation, co-location, disposal, as well as construction and acquisition efforts  The agencies’ Plans cited consolidation, co-location, and disposal as the primary means to reduce their office and warehouse space, activities mentioned in the National Strategy  Agencies also cited other methods, such as utilizing telework and decreasing the space they allocate per person to achieve space reductions  The space reduction strategies included most often in the Plans we reviewed include the following  Consolidation: All 24 agencies reported planned or ongoing efforts to reduce their space by consolidating their offices or operations  For example, we spoke with officials at HUD, which is in the process of consolidating staff from four offices in the National Capital Region into its 1 12-million square foot headquarters building in Washington, D C  HUD started by remodeling one floor to create a more open floor plan and intends to apply this design throughout the building  As part of the consolidation project, HUD has reduced the size of some office cubicles from 64 square feet to 56 square feet  (See fig  1 ) HUD leases its space through GSA and estimates that it will be able to return about 175,000 square feet of unneeded space back to GSA once all four offices are closed  At that point, GSA would then bear the cost of the space and work to lease it to another agency or otherwise dispose of it  Once the project is completed, HUD estimated that its headquarters building will accommodate about 500 more personnel (for a total of 3,200) and reduce its annual lease payments by about $11 million  Fifteen of the 24 agencies identified consolidation opportunities outside of their headquarters buildings  For example, the Department of Agriculture (USDA) discussed a consolidation project involving five component agencies in Albuquerque, New Mexico, in its fiscal year 2017 Plan  According to USDA officials, four component agencies occupying nearly 44,500 square feet in one building were to be consolidated into about 34,000 square feet of space in another building already occupied by a different USDA agency  In the prior location, the multiple components spaces’ square footage per person averaged 327, but the proposed consolidation would bring the utilization rate down to 255 square feet per person  USDA estimated that the consolidation project would result in about $238,000 in annual rent cost savings for the four components  Additionally, to enable this consolidation project, the component agency already occupying the building consolidated and vacated about 20,000 square feet, a move that resulted in an annual rental savings of about $500,000  In its fiscal year 2017 Plan, Interior’s Bureau of Reclamation anticipated eliminating 87,000 square feet of office space by consolidating operations from two buildings in Denver, Colorado  Interior estimated that the consolidation will result in a 40 percent reduction in its overall utilization rate to 165 square feet per person and an annual cost savings of about $2 1 million  Co-location: Thirteen of the 24 agencies’ Plans stated that they are exploring or implementing co-location projects to reduce space by merging staff from different components or agencies into another agency’s space  For example, the Social Security Administration (SSA) recently initiated a co-location pilot program with the Internal Revenue Service (IRS) within Treasury to combine SSA field offices with IRS Taxpayer Assistance Centers  Co-location of operations can reduce the overall space required by allowing agencies to share common space such as waiting rooms, an action that can reduce rent and operating costs for the co-located agencies  Since the inception of the 1-year program in January 2017, four IRS offices are participating and have moved into SSA field offices  According to SSA, IRS and SSA staff have adjusted to sharing space and the IRS presence in SSA space has not affected SSA wait times or created security or parking issues  According to an IRS official, IRS employees continue all normal operations from their co-located offices with SSA, including meeting with taxpayers in-person  The official also noted that IRS has extended the terms of its agreement with SSA for an additional year  However, SSA noted that the agencies are still working through customer access issues that could determine whether it would be possible to expand the pilot program and pursue additional co-location opportunities  In another example, according to Interior officials, the U S  Geological Survey is co-locating staff from Menlo Park, California, to a National Aeronautics and Space Administration facility in the nearby city of Mountain View, California  About 40 percent of the staff will move early in fiscal year 2019, and the U S  Geological Survey expects the remaining staff to be co- located by the end of 2021  Interior officials estimate that the co- location will result in an overall reduction of 165,000 square feet (about 50 percent of its space) and expects to save about $12 to $14 million in annual rent costs  To help agencies identify potential co-location opportunities and work with other agencies to meet their space requirements, GSA developed and provided agencies access to its Asset Consolidation Tool in fiscal year 2015  This database tool provides agencies with information about federal spaces in their area, including the buildings’ vacancy and utilization rates  Disposal of unneeded space: Thirteen of the 24 agencies reported that they plan to reduce their real property footprint by disposing of unneeded space, including selling or demolishing federal buildings or terminating leases, among other actions  For example, IRS has five tax submission-processing centers that receive all mailed income-tax returns and have warehouses that store the physical tax records  Each of these five processing centers, which include both office and warehouse spaces in multiple buildings, is approximately 500,000 square feet  According to IRS officials, 87 percent of all 2016 individual income-tax returns were filed electronically  As a result, the IRS plans to dispose of three of the five centers by 2024 to align with its reduced need for income-tax return processing and storage space  GSA has the statutory authority to dispose of property for all federal agencies and generally does so on their behalf  In addition, some federal agencies, such as Energy, or departmental components have statutory authority to dispose of buildings and other types of property and are not required to notify or use the services of GSA to complete the disposal  Better utilization of existing space: In their Plans, agencies also reported using tactical tools, such as incorporating space utilization rates into their capital-planning process, to identify opportunities to reduce space  For example, 22 of the 24 agencies reported incorporating office space design standards and agency utilization rates into their processes to identify space reduction opportunities  Agencies set their own space design standards and space utilization rates, which may vary based on agency mission requirements across their components  The RTF policy requires agencies to establish a design standard for the maximum workstation size, which should, at a minimum, be applied to all space renovations and new acquisitions  In addition, GSA has a recommended office space-utilization rate range of 150 to 200 square feet per person  Officials from our case study agencies noted several practices they said were helpful to identify opportunities to better utilize and ultimately reduce their space  For example, Commerce officials described developing a process for identifying and prioritizing space reduction opportunities using a two-factor matrix  Through this process, Commerce plans to target office space with a large number of employees and poor utilization rates (compared to its 170 square foot utilization rate)  According to Commerce officials, these situations may offer the most opportunity for space reductions and achieving significant rent and operating cost savings, particularly in high-cost real estate markets  Using this process, Commerce identified the potential for reducing as much as 1 6-million square feet (16 percent) of its total office space within 52 high priority facilities  According to IRS, retirements, hiring freezes, budget reductions, and increased telework have resulted in excess space throughout its portfolio  In fiscal year 2016, IRS started using a Strategic Facility Plan model to help identify space reduction projects  IRS’s objectives include consolidating multiple offices within a metropolitan area, closing outlying buildings, and leveraging telework, mobility, and its attrition rates  This model utilizes a template form to provide a consistent decision-making framework for assessing various options, articulating the rationale for selecting the preferred option, and documenting decisions and concurrence  According to IRS officials, this model has helped IRS to reduce a lot of its space  In 2014, GSA developed and provided agencies with access to the Real Property Management Tool, which can aid agencies that want to more effectively utilize their space  The database tool provides agencies with the capability to comprehensively view their real property portfolio by consolidating data from the assets that agencies directly manage with the assets that GSA manages on their behalf  As such, regardless of whether an agency initiated the action or GSA did so on its behalf, the tool gives an agency the ability to see all of its data, such as on expiring leases, in one place  The tool enables agencies to create individualized analytic reports allowing them to analyze the data in various ways  Teleworking and hoteling: Fifteen of the 24 agencies also described alternate workplace arrangements enabled by information technology, such as telework and hoteling, to help reduce office space  Telework is a work flexibility arrangement under which an employee performs their work responsibilities at an approved alternative worksite (e g , home)  Executive agencies are required to establish policies that authorize eligible employees to telework, determine the eligibility of all employees to participate in telework, and notify all employees of their eligibility  Federal law also requires that agencies consider whether space needs can be met using alternative workspace arrangements when deciding whether to acquire new space  As such, some agencies are eliminating designated offices for staff who primarily telework, a step that can improve space utilization  In a hoteling arrangement, employees use non-dedicated, non-permanent workspaces assigned for use by reservation and on an as needed basis  For example, the Office of Personnel Management implemented a workspace sharing initiative at one of its program offices  Staff who are not physically present in the office 4 or more days per week are required to share cubicles and offices  The Office of Personnel Management estimated that the initiative resulted in a 47 percent office space reduction for the program office  As part of their fiscal year 2016 and 2017 Plans, the 24 CFO Act agencies also described the major challenges they anticipated facing in their efforts to meet their space reduction targets  The agencies most frequently cited the following challenges: Space reduction costs: Twenty of 24 agencies stated that the costs of space reduction projects pose a challenge  Agencies are generally responsible for the up-front costs associated with relocations and tenant improvements, such as acquiring new furniture and renovating existing areas to reduce space or to accommodate more personnel in a smaller area  For example, the Department of Labor (Labor) reported in its fiscal year 2017 Plan that it did not have sufficient funding to implement a space reduction project that would have reduced commercially leased office space by 4,000 square feet  Similarly, the Department of Veterans Affairs’ fiscal year 2017 Plan noted that assuming a limited budget, large scale consolidations would be difficult to achieve  Some agencies have used or report that they intend to use funding from GSA’s Consolidation Activities program to help fund their space reduction projects  According to GSA, from fiscal years 2014 to 2017, GSA’s Consolidation Activities program funded projects that will eliminate 1 4-million rentable square feet from the GSA inventory and reduce agencies’ annual rent payments by $54 million  According to the IRS, GSA’s Consolidation funds have helped the agency reduce about 500,000 square feet of space  IRS officials noted that these funds helped the agency implement larger and more expensive space reduction projects than it would have been able to do otherwise  However, according to officials from several agencies, to use this program, agencies must also contribute funds to the projects  HUD officials stated that they considered applying for project funding through GSA but did not do so because HUD did not have sufficient funds for the agency’s share of project costs  Three of the 24 agencies specifically noted that the cost to clean up environmentally contaminated buildings is a challenge to disposing of excess office and warehouse space  Agencies are required to consider the environmental impact of property disposals  We have previously found that assessments and remediation of contaminated properties can be expensive and complicate the disposal process  Also, agencies are responsible for supervising decontamination of excess and surplus real property that has been contaminated with hazardous materials of any sort  In its fiscal year 2017 Plan, Energy estimated that over 60 percent of its excess buildings require extensive decontamination prior to disposal  Overall, Energy projected that its total liability for environmental clean-up could cost more than $280 billion  Mission delivery: Thirteen of the 24 agencies reported that mission delivery requirements can also affect their ability to reduce space  Agency missions may require office locations in certain areas or require additional space to accommodate activities such as customer interactions  These requirements may preclude disposals or limit opportunities to reduce space  For example, in its fiscal year 2017 Plan, SSA stated that its efforts to reduce space are affected by its mission, which requires offices widely dispersed throughout the country to administer and support its benefit programs, among other things  SSA has about 1,500 office spaces nationwide, most of which require space to accommodate the public  SSA had an overall office space utilization rate of 301 square feet per person, which exceeded GSA’s recommended office space utilization rate range of 150 to 200 square feet per person  USDA’s fiscal year 2017 Plan stated that its missions require office space in rural areas to, among other things, provide program assistance and leadership on food, agriculture, natural resources, rural development, nutrition, and related issues  In its fiscal year 2017 Plan, USDA also observed that the real estate market in rural areas is less competitive than in urban areas because there are fewer rental options, a situation that can also drive up rent costs  As such, USDA noted that these factors may contribute to difficulties identifying disposal opportunities and finding alternate spaces that could allow for more effective space utilization  Employee organization concerns: Ten of the 24 agencies reported that considering employee organizations’ concerns and addressing collective bargaining requirements when reconfiguring space can add time and affect the extent of their space reductions  For example, in its fiscal year 2017 Plan, SSA noted that the agency must meet with three employee unions when revising office space policies or design standards and collaborating with these organizations adds to the project’s implementation timeline  In July 2017, we reported that SSA officials met with employee union groups about the impact of potential changes to its space configuration or usage  Officials said that while the interactions with the union groups were positive—including gaining input on issues such as ergonomics, the security of field offices, and overall implementation—at times, these negotiations caused delays to individual projects and complicated reduction efforts by requiring union buy-in  In addition, Labor reported in its fiscal year 2017 Plan that its collective bargaining agreement and agency mission requirements for offices and work stations do not always enable it to take advantage of the previously discussed GSA Consolidation Funding program as well as GSA’s Total Workplace Furniture & Information Technology program  For example, the Total Workplace Furniture & Information Technology program requires that cubicles and offices must not exceed a specified square footage  However, according to Labor officials, Labor’s Departmental Space Management Regulation requires a certain utilization rate per person which may make it challenging to also stay within the program’s square footage requirements  Workload growth: Eight of the 24 agencies noted that increases in their workload limited their ability to achieve overall agency space reductions  For example, according to the Department of Justice’s fiscal year 2017 Plan, the agency anticipated having to provide additional court rooms to support an increased volume of immigration cases and accommodate the additional immigration judges needed to handle that volume  The Department of Justice estimated that the space needed to accommodate the new judges and additional public areas could add about 155,000 square feet to its portfolio  Also, according to the Department of Health and Human Services’ fiscal years 2016 and 2017 Plans, the Office of Medicare Hearings and Appeals experienced a 30 percent growth in cases and expected 1 2- million new cases annually after 2017  The Department of Health and Human Services projected that the growth in cases and additional staff needed to process the cases required additional field offices, which would increase its total office space square footage  As previously mentioned, agencies are required to set annual square foot reduction targets for domestic office and warehouse space in their annual Plans  According to an OMB official, to help ensure the targets are realistic, agencies are also required to identify the specific projects that will help them to achieve their space reduction targets  According to GSA and OMB officials, agencies submit their Plans, including their reduction targets, and their Plans are reviewed by both GSA and OMB  But each individual agency ultimately establishes its targets based on what it determines to be cost-effective and feasible  Through its Real Property Efficiency Plan template, GSA provides guidance to agencies on what is expected in their annual submissions  Each agency is required to document its internal controls, such as the process for identifying and prioritizing reductions to office and warehouse space and disposal of properties based on return on investment and mission requirements  The identified internal controls should help ensure that an agency’s proposed space reduction projects reflect an efficient use of space and are cost effective  A review of our five case study agencies illustrated some of the different approaches agencies used to determine their reduction targets  For example, several agencies’ targets were based on the total estimated feasible reductions identified by each agency component  In contrast, one agency centrally established a reduction target percentage and then asked its components to develop projects to meet that target  According to case-study agency officials, the agencies considered many factors, including their missions, priorities, component needs, and available budgets, when determining their targets  We found that the number and magnitude of the space reduction projects agencies identified in their fiscal year 2017 Plans varied greatly and were generally proportional to the size of the agency’s real property portfolio  The number of projects identified in agency Plans ranged from as few as 3 projects (the minimum required in the Plans) to nearly 400 projects  The estimated space reductions per project across agencies ranged from about 1,400 to over 94,000 square feet  For example, the Department of Veterans Affairs has a relatively large office and warehouse portfolio of over 28-million square feet  As part of its fiscal year 2017 Plan, the agency reported 320 planned or ongoing projects with an average space reduction of about 1,800 square feet per project  Conversely, the Office of Personnel Management has a relatively small office space portfolio of about 1-million square feet; its fiscal year 2017 Plan identified 4 ongoing or potential projects with an average space reduction of about 6,000 square feet  In fiscal year 2016—the first and only year RTF data were available at the time of our review—the majority (71 percent or 17 of the 24 agencies) reported they achieved reductions in their office and warehouse space even though the agencies had varying success in achieving the individual targets they set for themselves  For example, as shown in figure 2, of the 17 agencies that reduced space, 9 exceeded their targets (i e , reduced more space than planned); 7 reduced space but missed their target (by anywhere between 2 8 and 96 7 percent); and 1 agency expected to increase in square footage, but reduced space  Whether an agency met its target is not the only indicator of an agency’s success in reducing space  For example, although some agencies missed their targets, they reduced their office and warehouse space by a larger percentage than some agencies that exceeded their targets  Also, the fact that some agencies missed their targets can in part be attributed to setting more aggressive targets than other agencies  Agencies’ fiscal year 2016 targets ranged from a 0 8 percent increase to an 8 4 percent decrease in office and warehouse space  Of the 9 agencies that exceeded their reduction targets, 4 more than tripled their target  As mentioned, agency targets are set by the agency and are a reflection of their unique situation including mission needs and priorities and therefore cannot be generalized across agencies  For example, Energy exceeded its fiscal year 2016 reduction target and reduced 292,140 square feet of space (0 8 percent of its total square footage)  However, the Environmental Protection Agency missed its target, which was the second most aggressive target across all the agencies at 7 2 percent of its total square footage; but the agency reduced 174,003 square feet (3 24 percent of its total square footage)  Of the three agencies with the most aggressive target reductions—those that ranged between 6 7 and 8 4 percent of their total square footage—only one met its target  Figure 3 shows the extent to which each of the CFO Act agencies met its fiscal year 2016 targets  See appendix II for more detailed information on each agencies’ square footage of space, reduction targets and fiscal year 2016 reductions  Officials from our case study agencies cited a number of factors that influenced whether or not they met their fiscal year 2016 targets, and may also affect their target achievement in subsequent years  Of our five case study agencies, three exceeded their fiscal year 2016 reduction target and two missed their target  Timing and funding: Officials from two case study agencies cited timing as a factor, noting that there is fluidity to the project’s planning, implementation, and disposal process that may not always be within an agency’s control  As a result, space reductions anticipated in one fiscal year may not be realized until a subsequent fiscal year; conversely, some space reduction opportunities may present themselves unexpectedly  For example, according to officials at HUD, which missed its fiscal year 2016 reduction target, some projects take longer than anticipated to start or complete  HUD officials said that their fiscal year 2016 target may have been too ambitious and planned projects were delayed because they were unable to secure sufficient funding  As such, the officials said the agency must carefully select which projects to move forward with in a given fiscal year, but expected to move forward with their delayed, planned projects in the next fiscal year  Energy on the other hand, exceeded its fiscal year 2016 reduction target  Energy officials said that they tend to be conservative in listing potential RTF projects in their Plans  They noted that it takes a long time to dispose of a building and the timing was dependent on the building’s level of contamination, location, size, agency budget, and other factors  As a result, even though the agency may have planned to dispose of a building in a given fiscal year, there were numerous reasons why the project may get delayed  Further, RTF is a long-term effort and should not be judged based on agencies’ progress in their first year  According to an OMB official, it is understood that there may be circumstances in a given year that may hinder agencies from reaching their RTF targets, such as budget constraints or the timing of leases; however, the expectation is that agencies will continue to work toward accomplishing their target in the next year  Accordingly, under RTF, agencies set annual space reduction targets for a 5-year period  Officials from our case study agencies emphasized that the 5-year targets are not static, but rather are subject to annual updates  The RTF policy also acknowledged that changes to mission requirements and the availability of budgetary resources may require modifications to an agency’s targets, particularly in each of the subsequent years  Lastly, given that the RTF policy is still relatively recent, an OMB official noted that agencies are still in the process of learning how to set appropriate targets  Previous space reductions: Officials from three of our case study agencies noted that prior space reductions made during the Freeze the Footprint policy limited their ability to reduce space more aggressively  Though the thrust of Freeze the Footprint was to maintain the fiscal year 2012 size of an agency’s portfolio, agencies started to look more strategically for opportunities to dispose of excess space in their portfolios  The majority of agencies (18 of 24) have been decreasing the square footage of their domestic office and warehouse space since the Freeze the Footprint policy was implemented in 2013  OMB reported that under Freeze the Footprint, agencies achieved a 24 7-million square foot reduction between fiscal years 2012 and 2015  Officials from the IRS, which accounts for 70 percent of Treasury’s real property inventory, noted it has released 2 7-million square feet (approximately 10 percent) in the past 5 years, bringing its total square footage down to 25 3 million  According to officials from three of our case study agencies, a certain amount of space is required to effectively fulfill their missions  As such, the closer agencies get to attaining their optimum footprint, their ability to achieve further space reductions may be limited  In November 2016, GSA put into effect a new standard operating procedure to, among other things, standardize and streamline the process of receiving, reviewing, and documenting agencies’ space release actions  As previously mentioned, GSA’s occupancy agreements for space it leases on behalf of its customer agencies generally allow the agencies to release space back to GSA with as little as 4 months’ notice, if certain conditions are met  This can enable agencies to reduce their space and related rent costs relatively quickly without penalty  As a result of this new process, GSA established a centralized e-mail for agencies to submit their space release requests  The e-mail is maintained at GSA headquarters before it is forwarded to the respective GSA region  GSA also developed a centralized space release tracking spreadsheet to help ensure that all GSA regions were (1) notifying the customer agency of GSA’s determination on whether the space release request was within GSA’s policy, and (2) processing the space release and ceasing rent billings in a timely manner  According to GSA headquarters officials, this new process was implemented to rectify past concerns that space release requests were not centrally tracked, GSA regions may not have been making consistent determinations, and some requests either were missed or were not processed within the appropriate time frames  GSA officials noted that GSA similarly manages all vacant space in federally owned property under its custody and control and in commercial space it leases, and the agency seeks to utilize the space as quickly as possible  GSA has 11 regional offices throughout the country that generally conduct the day-to-day real property management activities for its customer agencies  These responsibilities include acquiring, managing, and disposing of real property, as well as executing, renewing, and terminating leases on behalf of its customer agencies in exchange for a monthly fee for GSA’s services  GSA headquarters officials told us that GSA regional offices track all the occupancy agreements and proactively work with customer agencies to help manage their space needs well before the agreements expire to understand ongoing space requirements  For example, according to GSA headquarters officials, this process includes working with agencies at a strategic level and helping them think about how they can accomplish their space needs and meet their targets 4 to 5 years in advance  GSA headquarters and regional officials noted that the advance planning helps the GSA regional officials integrate agencies’ potential space needs into the work they are already doing in the region as GSA manages the regional inventory as a whole, including managing the amount of vacant space  GSA regional officials told us that they work closely with the agencies in their space consolidation and reduction efforts to minimize the likelihood that GSA would be caught off guard by a release of space  This work enables GSA to develop options for either filling vacant space based on the known needs in the region or developing an alternative plan to effectively utilize the unneeded space  One of GSA’s strategic objectives is to improve the federal utilization of space in order to lower the government’s operational costs  To assess progress, GSA has an agency-wide vacant space performance goal of 3 2 percent for its federally-owned and leased inventory (with a 5 percent goal for federally owned and 1 5 percent goal for leased space)  Based on GSA data, the agency has steadily lowered its percentage of vacant space under its custody and control from 3 8 percent in fiscal year 2013 to 3 percent in fiscal year 2016, exceeding its performance goal of 3 2 percent for the first time in 4 years  The vacant space performance goal’s data help GSA evaluate its real property assets and plan for and make investment decisions while meeting its customer’s needs  According to GSA officials, the lower vacant space percentage is a reflection of the agency’s continued focus on working with its customer agencies to: (1) move into federally owned space, when possible; (2) decrease the size of commercially leased space to reduce agency rental costs and overall government reliance on leased space; and (3) dispose of unneeded federally owned assets  However, GSA officials noted that a certain level of vacant space is necessary to meet the space needs of new customers and customers with changing space requirements  According to GSA officials, GSA also tracks and reports annual cost avoidance data for all office and warehouse space reductions  These data include space covered under RTF in federally owned buildings under GSA’s custody and control and commercial space that GSA leases  Cost avoidance is defined as the results of an action taken in the immediate timeframe that will decrease future costs  The government-wide cost avoidance for fiscal year 2016 was $104 million based upon a net 10 7 million square foot reduction to all office and warehouse space  Of the government-wide figure, according to GSA, the total cost avoidance associated with office and warehouse space reductions in federally- owned space under GSA’s custody and control and commercial space GSA leased in fiscal year 2016 was over $75 8 million and 3 1 million square feet  In its cost avoidance calculation, GSA accounts for space returned to it by customer agencies only if there is a net square footage reduction in GSA’s total square footage across all the space that it manages  Similarly, the space returned to GSA does not reduce the federal government’s overall office and warehouse square footage unless GSA disposes of it  However, space that is returned to GSA is reflected as a square footage reduction for the customer agency and contributes toward that agency’s RTF target reduction  According to GSA regional officials, agencies’ requests to return space prior to the end of their occupancy agreements appear to have increased since the implementation of the RTF policy  Thus far, GSA has processes to manage agencies’ space release requests and keep its vacant space to a minimum  However, it is too early to determine how the recent increase in space release requests, in combination with agencies’ continued focus on occupying a smaller footprint and reducing their square footage, will affect: (1) the size of GSA’s inventory of vacant space in the long term, (2) GSA’s regional office workload to manage the requests, and (3) the cost savings for the federal government  We provided a draft of this report to GSA, OMB, Commerce, Energy, HUD, Interior, and Treasury for review and comment  We received technical comments from Energy, which we incorporated, where appropriate  GSA, OMB, Commerce, HUD, Interior, and Treasury did not have comments on our draft report  We are sending copies of this report to the appropriate congressional committees; the Administrator of GSA; the Director of the OMB; the Secretaries of the Departments of Commerce, Energy, HUD, the Interior, and the Treasury; and other interested parties  In addition, the report will be available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions concerning this report, please contact me at (202) 512-2834 or rectanusl@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix III  Our objectives were to determine: (1) the approaches and any challenges the 24 Chief Financial Officers (CFO) Act agencies identified to achieving their Reduce the Footprint (RTF) reduction targets for all their domestic office and warehouse space; (2) the extent to which these agencies reduced space and met their fiscal year 2016 RTF targets; and (3) how the General Services Administration (GSA) manages vacated space that it had leased to these agencies  To obtain background information for all three objectives, we reviewed relevant literature, including laws governing federal real-property management and agencies’ efforts to reduce their real property portfolios and Office of Management and Budget’s (OMB) and GSA’s memorandums and guidance governing the RTF policy  We also reviewed prior GAO and GSA inspector general reports describing agencies’ real-property management and efforts to more efficiently manage their real property portfolios  To determine the approaches used and any challenges faced by the CFO Act agencies in achieving their RTF reduction targets for all their domestic office and warehouse space, we conducted a content analysis of the agencies’ 5-year Real Property Efficiency Plans (Plans) for fiscal years 2016 and 2017  These Plans were obtained directly from each of the agencies  Each Plan describes an agency’s overall strategic and tactical approach in managing its real property, provides a rationale for and justifies its optimum portfolio, and directs the identification and execution of real property disposals, efficiency improvements, general usage, and cost-savings measures  The content analysis of the Plans helped us to understand the approaches agencies used to reduce space, how space- reduction targets were set, and any challenges they experienced in reducing their space  To identify agencies’ approaches to achieving their RTF targets, we reviewed all agencies’ Plans to determine the most frequently mentioned approaches agencies reported using or planned to use to reduce their real-property footprints  As part of their plans, each agency is required to include a section detailing approaches it plans to use to reduce space  While these sections were the primary focus of the analysis, we analyzed the Plans as a whole for any additional mention of agencies’ approaches to reduce space  Based on the frequently identified approaches, codes were developed  An analyst reviewed all the agencies’ Plans and coded the approaches and another analyst reviewed the coding  If there was a disagreement, the two analysts reviewed and discussed until they reached an agreement  As a result of the analysis, five approaches were identified that agencies most frequently reported using or were planning to use to achieve their RTF targets  These five approaches are described in more detail in the report: (1) consolidation; (2) co-location; (3) disposition of unneeded space; (4) better utilization of existing space; and (5) teleworking and hoteling  For the purposes of our report, telework and hoteling were combined because these approaches are often used in combination  For example, agencies can use telework strategically to reduce space needs and increase efficiency by making hoteling (i e , desk sharing) possible  To identify any challenges agencies faced in achieving their RTF targets, we similarly conducted a content analysis of agencies’ fiscal year 2016 and 2017 Plans  As part of their Plans, each agency included a section describing challenges it faced to reducing space  While these sections were the primary focus of the analysis, we analyzed the Plans as a whole for any additional mention of agencies’ challenges  Based on the frequently identified challenges, codes were developed  An analyst went through all the agencies’ Plans to code the challenges and another analyst reviewed the coding  If there was a disagreement, the two analysts reviewed and discussed until they reached an agreement  As a result of the analysis, we identified the four challenges that agencies most frequently described in their Plans: (1) space reduction costs; (2) mission delivery; (3) employee organization concerns; and (4) workload growth  In our report, we relied specifically on agencies’ fiscal year 2016 and 2017 Plans to provide examples and context for our description of the approaches agencies use and challenges they experience in achieving their RTF targets  However, after these Plans were submitted, agencies reported that the specific details as described in their Plans may in some instances, have changed due to a variety of factors  For our case study agencies, to the extent possible, we have provided updated information from agency officials as of December 2017  We selected five agencies as case studies to inform our first two objectives  We selected the agencies using a variety of considerations such as the diversity in the size of the agency’s domestic office and warehouse portfolio, the extent to which the agency met its fiscal year 2016 RTF targets, the types of real property authorities the agency has, as well as suggestions from GSA and OMB related to agencies’ experiences  Based on these factors, we selected the: (1) Department of Commerce (Commerce); (2) Department of Energy (Energy); (3) Department of Housing and Urban Development (HUD); (4) Department of the Interior (Interior); and (5) Department of the Treasury (Treasury)  While our case-study agencies and their experiences reducing their space are not generalizable to all CFO Act agencies, they provide a range of examples of how agencies are implementing the RTF policy  We interviewed officials at the selected agencies as well as GSA and OMB, and reviewed relevant agency real-property management and RTF guidance, to obtain more detailed information about agencies’ RTF approaches, challenges, specific RTF projects, RTF project funding and prioritization, and experiences in meeting their RTF targets  In addition, we visited three office buildings of our case study agencies in Washington, D C , with ongoing or recently completed RTF projects that illustrated approaches the agencies used to reduce space and met with officials to discuss the projects in more detail  The spaces we visited were the headquarters buildings for Commerce, HUD, and Interior  We selected the buildings based on recommendations from officials at our case study agencies  To determine to what extent agencies reduced their space and met their fiscal year 2016 RTF targets, we analyzed the 24 CFO Act agencies’ data as submitted to GSA on their RTF targets and reported reductions for fiscal year 2016  The office and warehouse square footage reductions are calculated annually using GSA occupancy agreement data and agencies’ self-reported data in GSA’s Federal Real Property Profile  For example, for fiscal year 2016, the space reduction calculations based on these data sources at the end of the fiscal year was compared to the square footage reported in fiscal year 2015  At the time of our review, this was the first and only year of RTF data available as the policy was implemented in March 2015  We conducted a data reliability assessment of the RTF data GSA provided by interviewing GSA officials and reviewing documentation, and concluded the data were reliable for our purposes  We also interviewed officials at GSA and OMB and reviewed relevant documentation to learn more about each agency’s role and the requirements of the RTF policy  We interviewed officials from our selected case-study agencies to obtain supporting documentation and to improve our understanding of how agencies set their RTF targets, agencies’ progress toward those targets, and the approaches used and challenges faced in meeting those targets  We also asked the agency officials for examples of successful practices used to reduce their office and warehouse space  To determine how GSA manages vacated federally owned and commercially leased space that it leases to agencies, we reviewed federal requirements and GSA policies and vacancy data  We conducted a data reliability assessment of GSA’s vacancy and cost avoidance data by interviewing GSA officials and reviewing documentation, and concluded the data were reliable for our purposes  We also interviewed GSA headquarters and regional officials and obtained documentation on how GSA manages space returned by agencies  We conducted this performance audit from April 2017 to March 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  FY 2016- FY 2020 target reduction (118,127) Social Security Administration Missed target and increased in space Department of Health and Human Services (170,147) (520,987) (15,466) (47,946) (56,062) In addition to the individual named above, Maria Edelstein (Assistant Director); Lacey Coppage; Edgar Garcia; Delwen Jones; Catherine Kim (Analyst-in-Charge); Michael Mgebroff; Malika Rice; Kelly Rubin; and David Wise made key contributions to this report 

ground_truth: The federal government continues to work to reduce its real property inventory and associated costs. GSA provides space for agencies in government-owned and commercially leased buildings. In 2015, the OMB issued a memorandum requiring the 24 agencies with chief financial officers to reduce their domestic office and warehouse space. These agencies are required to set annual reduction targets for a 5-year time period and update their real property plans annually. GAO was asked to review the implementation of this space reduction initiative. This report discusses: (1) the approaches and any challenges the 24 agencies identified to achieving their reduction targets for all their domestic office and warehouse space; (2) the extent these agencies reduced their space and met their fiscal year 2016 targets; and (3) how GSA manages vacated space it had leased to these agencies. GAO conducted a content analysis of the 24 agencies' real property plans for fiscal years 2016 and 2017 and analyzed agencies' data as submitted to GSA on their targets and reductions for fiscal year 2016, the only year for which data were available. GAO selected five agencies as case studies based on several factors, including size of the agencies' office and warehouse portfolio, agency reduction targets, and fiscal year 2016 reported reductions. GAO reviewed relevant documentation and interviewed officials from GSA, OMB, and GAO's case study agencies. GAO provided a draft of this product to GSA, OMB, and our case study agencies for comment. GAO incorporated technical comments, as appropriate. Most of the 24 agencies with chief financial officers reported to the Office of Management and Budget (OMB) and the General Services Administration (GSA) that they planned to consolidate their office and warehouse space and allocate fewer square feet per employee as the key ways to achieve their space reduction targets. For example, the Department of Agriculture reported it will consolidate staff from five component agencies in two office buildings. When complete, the space allocated per employee will average about 250 square feet down from a high of 420 square feet per employee. In taking these actions, the agencies most often identified the cost of space reduction projects as a challenge to achieving their targets. Agencies cited costs such as for space renovations to accommodate more staff and required environmental clean-up before disposing of property as challenges to completing projects. Some agencies required to maintain offices across the country reported that their mission requirements limit their ability to reduce their space. In fiscal year 2016, 17 of the 24 agencies reported they reduced their space, but had varying success achieving their first-year targets. Of the 17 agencies, 9 exceeded their target and reduced more space than planned, 7 missed their target (by anywhere between 2.8 and 96.7 percent), and 1 reduced space, despite a targeted increase. Agency officials said that it is not unusual for projects to shift to different years and that such shifts could lead to missing targets one year and exceeding them the next. GSA has processes to manage the space vacated by agencies that is leased through GSA. For example, starting in November 2016, GSA started tracking agencies' space release requests centrally to help standardize the process and established an e-mail address to which agencies can submit requests. GSA relies on regional offices to manage real property in their regions and to identify tenants for vacant space or to remove unused space from the inventory. GSA's regional officials said regular monitoring and coordinating with agencies minimizes the likelihood GSA is caught off guard by a return of space. These processes also help them to plan ahead. GSA met its 2016 performance goal to have an annual vacant space rate of no more than 3.2 percent in its federally owned and leased buildings. However, given the recent implementation of the space reduction initiative, it is too early to determine the extent to which agencies will return space to GSA prior to the end of their leases and the effect on GSA's inventory.