prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: WMATA was created in 1967 through an interstate compact—matching legislation passed by the District of Columbia, state of Maryland, and Commonwealth of Virginia, and then ratified by Congress—to plan, develop, finance, and operate a regional transportation system in the National Capital area  A board of eight voting directors and eight alternate directors governs WMATA  The directors are appointed by the District of Columbia, Virginia, Maryland, and the federal government, with each appointing two voting and two alternate directors  WMATA operates six rail lines—the Red, Orange, Blue, Green, Yellow, and Silver Lines—connecting various locations within the District of Columbia, Maryland, and Virginia  WMATA’s rail system has 118 linear miles of guideway: 51 miles of subway, 58 miles at ground level, and 9 miles on aerial structures  WMATA’s capital investments are funded through multiple sources  These include a combination of grants it receives from the federal government, along with matching funds and other contributions it receives from the states and local jurisdictions in which it operates (see fig  1)  From fiscal years 2011 through 2017, WMATA received about $5 8 billion in capital funding  Over half of this funding came from the federal government ($3 2 billion), and state and local jurisdictions provided 41 percent ($2 4 billion)  WMATA also took on about $230 million in long- term debt to finance its capital program during this time period  The federal funding included grant awards, in addition to annual appropriations authorized under PRIIA  In 2008, PRIIA authorized $1 5 billion to WMATA, available in increments over 10 years beginning in fiscal year 2009, or until expended, for capital improvements and preventive maintenance  PRIIA funding and certain federal grants require state or local jurisdictions to provide matching funds  Additionally, a large portion of funding from state and local jurisdictions is governed by capital-funding agreements, which are periodically negotiated between WMATA and the states and localities  From fiscal years 2011 through 2017, state and local jurisdictions contributed on average about $340 million annually to WMATA, generally for capital purposes  The annual capital contributions from the jurisdictions are expected to more than double as a result of the recent legislation enacted by the District of Columbia, Maryland, and Virginia in 2018  In addition, WMATA officials told us that it will have the ability to further leverage this dedicated funding and issue debt to finance its capital projects  WMATA has several steps in its capital planning process  These include developing the following: Capital Needs Inventory  WMATA periodically identifies its capital investment needs in this inventory  WMATA issued a Capital Needs Inventory in February 2010 and another in November 2016, each covering a 10-year period  According to WMATA, Capital Needs Inventories help inform the annual capital budget and capital improvement program  Annual Capital Budget  Each year, WMATA prepares an annual capital budget, which identifies projects WMATA plans to undertake in the next fiscal year  WMATA’s fiscal year 2019 annual capital budget was approved by the board of directors at $1 3 billion  Six-Year Capital Improvement Program  Within WMATA’s annual capital budget, WMATA includes a Six-Year Capital Improvement Program identifying capital projects WMATA plans to implement over a 6-year period  WMATA’s most recent Six-Year Capital Improvement Program (covering the fiscal year 2019—2024 period) was approved by the board of directors at $8 5 billion  According to WMATA officials, WMATA is currently implementing a new capital planning process through which it will develop its fiscal year 2020 Capital Budget and fiscal year 2020-2025 Six-Year Capital Improvement Program  WMATA adopts and implements the capital budget by June 30 for the new fiscal year, which begins on July 1  The fiscal year 2020 Capital Budget is scheduled to be adopted and implemented by June 30, 2019  Among other things, the goals and objectives of this new capital planning process are to construct an objective, data-driven, and risk-based approach to estimate major rehabilitation and capital asset replacement needs; build a capital investment prioritization methodology aligned with WMATA’s strategic goals and grounded in asset inventory and condition assessments; and develop a process that will support the construction and ongoing stewardship of its Transit Asset Management Plan  The latter is discussed in more detail below  WMATA has also recently undertaken efforts to address issues related to the condition and maintenance of its track  After SafeTrack concluded in June 2017, WMATA implemented what officials describe as its first track preventive maintenance program designed to incorporate industry-wide best practices related to track maintenance, in order to improve the rail system’s long-term safety and reliability  The new program commenced in June 2017, and WMATA’s board reduced late-night service to allow for longer maintenance work hours  To make the best use of the extra maintenance hours, WMATA focused its new program on six separate initiatives that together would address what WMATA viewed as its two most pressing track maintenance concerns—electrical fires caused by cable and insulator defects along the track wayside, and defects to the track itself, including unsecured rail fasteners and worn track switches (see table 1)  These initiatives are planned to cover the entire transit system and will take various amounts of time to complete  FTA also plays a role in WMATA activities by providing and directing the use of federal funds, overseeing safety, and requiring transit asset management  FTA provides grants that support capital investment in public transportation, consistent with locally developed transportation plans, and has provided such funding to WMATA as noted above  Additionally, though states play a role in safety oversight of rail transit systems through state safety oversight programs, FTA also has the authority to conduct various safety oversight activities such as inspections and investigations  Furthermore, FTA has the authority to assume temporary, direct safety oversight of a rail transit system if it finds the state safety oversight program is inadequate, among other things  After FTA conducted a safety management inspection and issued a safety directive with 91 required actions, it found WMATA’s state safety- oversight program to be inadequate and assumed direct safety-oversight of WMATA in October 2015  Finally, FTA is responsible for assisting public transportation systems to achieve and maintain their infrastructure, equipment, and vehicles in a state of good repair  Specifically, in July 2016, FTA issued regulations establishing a National Transit Asset Management System  Applicable transit agencies were required to have an initial transit asset management plan completed by October 1, 2018  For “tier I providers,” such as WMATA, this plan is to contain nine elements, including an inventory of the number and type of capital assets, and a condition assessment of those inventoried assets for which a provider has direct capital responsibility  WMATA completed its Transit Asset Management plan, dated October 1, 2018  This plan outlines WMATA’s policy, approach, and targeted actions to improve its asset management practices over the next 4 years  WMATA expends its capital funds on a variety of capital assets as part of its capital budget and Capital Improvement Program  From fiscal year 2011 through 2017, WMATA expended approximately $5 9 billion on capital investments  Of this amount, WMATA expended the largest portion on assets related to the replacement, rehabilitation, and maintenance of its revenue vehicles (railcars, buses, and vans) and lesser amounts on other categories of assets, as discussed below and shown in figure 2  Rail and Bus Vehicle Fleet: WMATA expended approximately $2 16 billion (36 percent) of the total $5 9 billion on projects related to its rail and bus fleet from fiscal years 2011 through 2017  The $2 16 billion included approximately $1 1 billion (51 percent) on replacing, expanding, and rehabilitating its rail fleet and approximately $956 million (44 percent) on its bus fleet  According to WMATA, it initiated its railcar replacement program in 2005 to increase capacity and reduce maintenance costs  In addition, a June 2009 Red Line collision of two trains near Fort Totten resulted in nine deaths and led the NTSB to recommend that WMATA retire and replace all 1000 series railcars  From fiscal year 2011 through 2017, WMATA expended almost $656 million on replacing these and other railcars and expanding its overall fleet  This effort includes WMATA’s planned purchase of a total of 748 new 7000-series railcars (see fig  3)  Approximately $530 million was expended on replacing vehicles from fiscal years 2015 through 2017  For example, in fiscal year 2017 WMATA accepted delivery of about 50 percent (364 railcars) of its planned purchase of 748, 7000-series railcars  WMATA expects to complete its current railcar replacement program by fiscal year 2024, with an estimated total program cost of about $1 7 billion  Fixed Rail Infrastructure: WMATA expended about $1 23 billion of the total $5 9 billion (21 percent) to maintain its fixed-rail infrastructure  Of this $1 23 billion, WMATA expended about $650 million (53 percent) on rail infrastructure and rehabilitation projects and $573 million (47 percent) on improvements to its track and structures (e g , bridges and tunnels)  According to WMATA, the rail infrastructure and rehabilitation projects began in 2009 and were the first comprehensive rehabilitation of WMATA’s rail infrastructure in its history  Typical projects included rehabilitating WMATA’s water drainage pumps and tunnel ventilation, fire, and communications systems, among other things  WMATA work related to track and structures involved the maintenance and rehabilitation of the steel rail that guides railcars, the cross ties and fasteners that hold the rail in place, the third rail that provides power to trains, and the bridges and tunnels the track runs on or through  The share of WMATA’s total capital expenditures going to track and structures increased from about $80 million in fiscal year 2016 to $158 million in fiscal year 2017  This expenditure was primarily to implement SafeTrack  Maintenance Facilities and Equipment: WMATA expended approximately $1 1 billion of the total $5 9 billion (19 percent) on assets related to maintenance facilities and equipment, which include rail yards, bus garages, and equipment used to rehabilitate and maintain WMATA’s track and vehicle fleet  For example, from fiscal years 2011 through 2017 WMATA expended approximately $75 million in constructing the Cinder Bed Road bus maintenance facility in Lorton, Virginia  Passenger and Other Facilities: WMATA expended about $814 million of the total $5 9 billion (14 percent) on passenger, business, and security support facilities  Such facilities include rail and bus stations, police facilities, and elevator and escalator rehabilitation  Business Systems and Project Management Support: WMATA also expended about $628 million of the total $5 9 billion (11 percent) on assets related to operations and business support software and equipment  From fiscal years 2011 through 2017, WMATA frequently over-estimated in its annual budgets the annual amount of capital investments it could implement (see fig 4)  Out of the approximately $7 5 billion that WMATA budgeted for capital investments over this period, it expended approximately $5 9 billion (80 percent)  WMATA’s ability to fully expend its capital budget has varied from year to year  Specifically, WMATA expended about 65 percent ($700 million) of its $1 1 billion capital budget in fiscal year 2015, compared with 85 percent ($1 1 billion) of its $1 2 billion capital budget in fiscal year 2016  In fiscal year 2017, WMATA expended nearly 100 percent of its $1 18 billion capital budget  WMATA attributed the increased expenditures to intensified efforts to address deferred maintenance, primarily through the SafeTrack initiative and an increased delivery and acceptance rate for the new 7000-series railcars, among other things  The total amount expended in fiscal year 2017 to replace the older railcars with new vehicles totaled about $335 million  According to WMATA, there are a number of reasons why it has not fully expended its capital budget in any given year: Contracting and Scheduling Issues: WMATA officials stated that there were contract and scheduling delays in the implementation of planned capital projects  For example, WMATA officials said contracts were sometimes not executed during the fiscal year in which funds were originally budgeted for the work, and in other instances contract work was not carried out according to schedule and expenditures were delayed  Changing Priorities: WMATA officials stated that in some instances, the reevaluation and reprioritization of contracted projects affected WMATA’s ability to expend its capital budget  In such cases, new capital needs were sometimes identified and prioritized over other needs, which caused delays in work schedules and potential financial claims by contractors due to delays  For example, WMATA stated that in fiscal year 2011 the initiation of the Red Line rehabilitation program was delayed as a result of the prioritization of the safety needs in response to the 2009 Fort Totten accident  Federal Reimbursement Restrictions: WMATA officials cited FTA restrictions on its reimbursement of federal funds between fiscal years 2014 and 2015 as a reason for its inability to expend budgeted capital funds in those years  In a financial management oversight review completed by FTA in 2014, FTA found material weaknesses and significant deficiencies in WMATA’s financial management controls, policies, and procedures regarding its receipt of federal grant funds  Based on these preliminary findings, FTA restricted WMATA’s ability to automatically access federal grant reimbursements until WMATA undertook corrective actions  During these years, WMATA reported its management slowed expenditures on targeted capital projects due to concerns over reimbursement of grants  By October 2017, after WMATA implemented an action plan to improve its financial controls, FTA reinstated WMATA’s ability to automatically receive all awarded federal funds on a regular schedule  Unpredictable Funding: WMATA officials stated that unpredictable funding affected the level of its capital expenditures from year to year  Since WMATA had multi-year capital projects with multi-year procurements, according to WMATA officials, uncertainty with regard to how much capital funding would be received on an annual basis affected the implementation of projects  Inadequate Capital Planning Process: WMATA attributed some of its inability to expend budgeted capital funds to the absence of a uniform and efficient capital planning process  According to WMATA, it lacked formal procedures to initiate projects and newer projects often experienced delays in implementation, which delayed expenditures on these projects  Later in this report, we discuss WMATA’s efforts to develop a new capital planning process  Although WMATA expended more of its capital budget in fiscal year 2017 than it had in prior years, it estimated that capital spending will need to increase even more to address state-of-good-repair needs  In 2016, WMATA projected that its state-of-good-repair needs amounted to about $17 4 billion from 2017 through 2026  This level is almost $10 billion more than WMATA estimated for its state-of-good-repair needs from 2011 through 2020 in its February 2010 Capital Needs Inventory  WMATA officials attributed the increase to a capital planning process insufficient to identify capital needs and an increase in cost of needs that were previously unmet  In addition, WMATA officials said the quality and quantity of asset data had improved over time  To address its state-of- good-repair needs, in November 2016 WMATA estimated that it will need to expend about $1 74 billion annually on capital expenditures from 2017 through 2026  This is more than twice the $845 million average annual capital expenditures from fiscal year 2011 through fiscal year 2017  WMATA’s new capital planning process could address some of the weaknesses it identified in the previous process, such as better distinguishing capital needs (investments in groups of related assets) from capital projects (investments in specific assets)  However, WMATA has not established documented policies and procedures to guide the developed performance measures to assess capital projects and the capital planning process; and developed a plan to obtain complete information about the inventory and condition of WMATA assets  These remaining weaknesses could hinder sound capital investment decisions  WMATA’s new capital planning process could facilitate better identification of capital investment needs  Leading practices for capital planning, among other things, call for an organization to conduct a comprehensive assessment of its needs to meet its mission  WMATA uses the Capital Needs Inventory to assess its capital needs over a 10- year period across its various assets and help identify specific projects to include on subsequent capital improvement programs  In November 2016, WMATA issued its most recent Capital Needs Inventory, covering calendar year 2017 through 2026, and reported there were weaknesses and limitations in the process used to prepare the previous Capital Needs Inventory, issued in 2010  Those weaknesses and the actions WMATA has taken to address them include the following: Distinguishing capital needs from capital projects  WMATA reported in 2016 that the 2010 Capital Needs Inventory was primarily a list of proposed projects and did not provide proper attention to evaluating WMATA’s overall asset needs and the readiness of projects for programming in the capital improvement program  WMATA has taken actions to potentially address this weakness  In April 2016, WMATA issued a policy/instruction document that established policies and procedures for preparing capital needs inventories  This document defined the process for capital needs identification and established a framework evaluating and prioritizing capital investment needs  Among other things, this framework requires that WMATA departments develop capital needs justification packages and that these packages be reviewed by the Capital Program Advisory Committee for completeness and accuracy before being forwarded for further review  The guidance also requires that WMATA’s strategic objectives be considered when identifying and prioritizing capital projects  Qualitative rather than quantitative prioritization of needs  In 2016, WMATA reported that the prioritization of capital needs in the 2010 Capital Needs Inventory was primarily based on qualitative assessments by management rather than being driven by quantitative information and condition assessments  According to WMATA, the 2010 Capital Needs Inventory was largely based on the professional judgment of staff in consideration of WMATA’s strategic goals but was not data-driven  WMATA has taken actions to address this weakness by issuing a policy that requires WMATA’s senior management serving on the Capital Program Advisory Committee to use a more quantitative-based capital prioritization formula in preparing the Capital Needs Inventory  For example, the November 2016 Capital Needs Inventory used a quantitative approach to rank and prioritize capital needs  This approach included the use of four criteria—asset condition, safety and security, service delivery, and ridership impact— to numerically score capital needs and WMATA then used a risk- based weighting approach to combine these criteria into a single overall prioritization score  While WMATA has addressed some weaknesses it identified in its prior planning, it has not established documented policies and procedures to guide the annual capital planning process, or developed measures to assess capital project and program performance and a plan to obtain complete information on its assets and their physical condition  Although WMATA established policies and procedures for prioritizing capital needs—that is, investments in groups of related assets—for the 2016 Capital Needs Inventory, it has not established documented policies and procedures for the new capital planning process, including how WMATA will rank and select individual projects to address those needs through its annual capital budgets and Six-Year Capital Improvement Program  For example, through its Capital Needs Inventory WMATA stated it needed to invest $17 4 billion over a 10-year period to address its state-of-good-repair needs, including replacing vehicles, rehabilitating stations, and investing in other types of assets  WMATA uses the annual capital budget and Six-Year Capital Improvement Program to identify the specific projects to be funded to meet the 10-year investment needs  However, because WMATA has not established documented policies and procedures for the new capital planning process, it has not yet identified the specific methodologies to rank and select projects for funding on an annual basis  According to WMATA officials, the legacy annual capital planning process was based on implementing the list of projects that resulted from its 2010 Capital Needs Inventory and WMATA did not have a documented capital planning process that it followed on an annual basis  WMATA officials told us that the legacy capital planning process was “ad hoc” in nature, in part because WMATA was reacting to emergencies  For example, because WMATA needed to address the NTSB recommendation to replace the 1000-series railcars and address FTA safety directives after the 2015 smoke incident at the L’Enfant Plaza Station, it did not adhere to a formal annual-planning process  The COSO internal control standards point out the importance of organizations documenting their processes to facilitate retention and sharing of organizational knowledge  Leading practices contained in the Executive Guide also recommend that organizations have defined processes for ranking and selecting projects for capital funding  In addition, the Executive Guide noted that organizations find it beneficial to rank projects because the number of requested projects often exceeds available funding  Officials from all five of the peer transit agencies we spoke with told us they had or planned to develop documented processes for making capital investment decisions  For example, officials from four of the five peer transit agencies we spoke with said they use a project scoring and ranking system in their capital planning process, and officials from the fifth agency told us it plans to develop such a system  Officials from one agency provided us with its project evaluation and scoring system that assigns scores using eight selection criteria that are tied to the agency’s strategic business plan and state priorities  The selection criteria include such things as system preservation, safety, and cost-effectiveness  Officials from another agency told us they use an analytical tool to score projects and that every project (new or existing) gets re-scored annually  As a result of WMATA not having documented policies and procedures for its capital planning process, it is unclear how important parts of the process will work and the basis for WMATA’s investment decisions  WMATA has outlined some high-level policies for the capital planning process and prepared limited guidance for certain parts of the process  For example, WMATA officials told us that its recently issued Transit Asset Management Plan contains asset management policies that address the ranking and selecting of capital projects  Although the Transit Asset Management Plan discusses the process for estimating and prioritizing capital needs and, which are precursors of projects, the plan does not specifically address how projects would be selected for annual capital budgets and the capital improvement program  In addition, WMATA developed limited guidance for staff to use in developing new capital projects  Under this guidance, capital funds could be provided to evaluate, plan, and develop projects  While this guidance may be useful for developing projects, it does not establish the policies and procedures WMATA will follow to decide which projects will be funded through the annual capital budget and the capital improvement program  Further, the documentation prepared by WMATA to date does not establish policies and procedures for the entire capital planning process and how decisions will be made throughout the process  WMATA reported in its fiscal year 2019 annual budget that it had created a capital program manual that identifies the roles, responsibilities, processes, and calendars of events to inform the fiscal year 2020 capital program  WMATA officials told us that the previous Director of the Capital Planning and Program Management Department had included this information in the draft budget proposal when these documents were being developed  However, WMATA officials told us that these documents were not completed, and that the information was mistakenly not removed from the budget before the previous director of the department left the agency  WMATA officials told us they plan to formalize policies, procedures, and manuals for the fiscal year 2021–2026 capital-investment program cycle  The current leadership of the Capital Planning and Program Management Department told us that given the time-constraints facing WMATA in the current fiscal year 2020 planning cycle, WMATA decided not to formally document the new capital planning process until after WMATA has had a chance to test it through the current planning cycle to see how it works  According to the official, the department’s leadership has instructed staff to document steps taken in implementing the new process so that WMATA will have the opportunity to learn from the new process and make necessary changes before developing formal, written procedures that will guide future planning cycles  Although delaying formal development of policies and procedures may provide an opportunity to learn from the process while implementing it, it does not provide the guidance necessary now as WMATA uses its new capital planning process to develop the fiscal year 2020 capital program  In particular, because WMATA has not established policies and procedures for ranking and selecting projects, WMATA does not have a framework or clear criteria for programming projects in the annual capital budget for fiscal year 2020  WMATA has proposed a fiscal year 2020 capital budget of $1 4 billion  In addition, WMATA’s plan to document steps taken in implementing the new process as it is occurring does not provide reasonable assurance that WMATA is making decisions using a consistent process to direct investments toward WMATA’s highest priority needs  A consistent process is all the more important to ensure that WMATA does not continue to use an ad-hoc process for capital investment decisions, as it did in its legacy process  WMATA’s annual capital spending is anticipated to increase substantially over the fiscal year 2020-2025 period, as WMATA expects to be programing the additional $500 million annually for capital purposes committed by the District of Columbia, Maryland, and Virginia  Without a documented planning process that includes procedures for ranking and selecting projects for funding in the fiscal year 2020 capital budget, WMATA’s stakeholders lack reasonable assurance that WMATA’s capital investment decisions will be made using a sound and transparent process  WMATA has also not developed performance measures to assess capital projects and the capital planning process  Leading practices from the Executive Guide suggest that one way to determine if a capital investment achieved the benefits that were intended when it was selected is to evaluate its performance using measures that reflect a variety of outcomes and perspectives  By looking at a mixture of measures, such as financial improvement and customer satisfaction, managers can assess performance based on a comprehensive view of the needs and objectives of the organization  Leading organizations we studied in preparing the Executive Guide, such as private sector companies, use financial and non-financial criteria for success that are tied to organizational goals and objectives  According to the Executive Guide, project-specific performance measures are then used to develop unit performance measures and goals, which are ultimately used to determine how well an organization is meeting its goals and objectives  WMATA officials told us they have not developed performance measures for assessing the performance of individual projects or the capital planning process as a whole  One WMATA official told us that WMATA would like to evaluate results of the new capital planning process to determine whether organizational goals have been met  The official suggested that WMATA would work with a consultant to demonstrate a linkage between capital planning goals and WMATA’s organizational goals  However, the official did not indicate when this step would occur or provide additional information  Moreover, it is unclear whether the official’s intentions for this effort would result in measures for assessing individual projects as well as the overall capital planning process  By developing measures, WMATA will be better positioned to assess whether specific capital investments met their intended outcomes or if the capital planning process itself is helping WMATA achieve its strategic goals and objectives and effectively using taxpayer funds  WMATA also does not have a complete inventory or physical condition assessments of its assets  Leading practices for good capital decision- making call for organizations to conduct a comprehensive assessment of their needs and identify the organization’s capabilities to meet these needs  This process includes taking an inventory of assets and their condition and assessing where there are gaps in meeting organizational needs  The Transit Cooperative Research Program has also identified asset inventory and condition assessments as the first step in determining what asset rehabilitations and replacements are needed as transit providers address their state-of-good-repair requirements  Asset inventories and condition assessments provide critical information for capital-investment decision making  WMATA has initiated various efforts to obtain better information about its assets and their condition  These efforts have included: Transit Asset Inventory and Condition Assessment Project  In 2016, WMATA began this project to provide a physical inventory of WMATA assets and their condition, in part to comply with FTA’s Transit Asset Management regulations  According to WMATA, this project was to be the cornerstone in ensuring a complete, consistent, accurate, and centralized repository of relevant asset-related data  However, WMATA officials said that the project primarily focused on obtaining an inventory and condition assessment of WMATA facilities and equipment  A February 2018 WMATA memo to senior management stated that even when the project was completed, WMATA would still lack a robust database of track, guideway, infrastructure (e g , tunnels and bridges), systems, and communication assets—elements that the November 2016 Capital Needs Inventory noted were the largest gaps in the asset information used to support capital needs forecasting  According to WMATA, this project produced inventory and condition assessments for about 30 percent of WMATA’s asset base  As of October 2018, WMATA considered the project complete since it provided information to help prepare WMATA’s completed Transit Asset Management Plan, dated October 1, 2018  WMATA officials noted that they will continue to develop their asset inventories and condition assessments through its new Enterprise Asset Management Program, described below  Enterprise Asset Management Program  In December 2017, WMATA began development of an Enterprise Asset Management Program  According to WMATA, this program is an effort to institutionalize asset management practices that are aligned with industry best practices to provide, among other things, high quality asset data for informed decision-making, including for capital planning  Expected program tasks include updating asset records and improving and consolidating asset inventories in WMATA’s asset system of record (called Maximo)  WMATA’s efforts to develop more complete asset inventory and condition assessments are not complete  Among other things, WMATA documentation on the Enterprise Asset Management Program cited “inattention, poor standardization, and organizational silos” as factors that have resulted in WMATA having multiple sets of asset records in various states of accuracy and usefulness  The Enterprise Asset Management Program, according to WMATA, is an effort to help address this situation and improve asset data quality, including inventory and condition assessments  Although WMATA is developing a new Enterprise Asset Management Program, it has yet to develop a plan for obtaining a complete inventory or physical condition assessments of its assets  The Project Management Institute’s Guide to the Project Management Body of Knowledge, PMBOK® Guide describes the elements of good project management and their importance in achieving organizational goals  Among these elements are: Having a project charter that formally authorizes a project, that commits resources to the activity, and that provides a direct link to organizational strategic objectives; Preparing a project plan to define the basis of the project’s work and how the work will be performed; and Establishing a monitoring and control process to track, review, and report overall progress in meeting the plan’s objectives  WMATA has prepared draft documents that describe how it will implement the Enterprise Asset Management Program and that contain some elements of good project management  For example, in January 2018 WMATA circulated a proposed charter that once approved would authorize the Enterprise Asset Management program, identify needed resources, and link to WMATA’s strategic goals  As of October 2018, this proposed charter had not yet been finalized  Draft program documents also indicate there would be a monitoring and control process that would establish regular reporting to internal stakeholders to assess program accomplishments and progress implementing the program  While WMATA has developed a proposed charter and a monitoring and control process for its Enterprise Asset Management Program, it has not established a plan for collecting asset inventory and condition assessment information  The draft program charter includes general tasks for updating asset records and improving and consolidating asset inventory data in Maximo  However, a plan would provide more specific details for how the work would be completed, such as the information to be collected on different assets, how and when this information would be consolidated into Maximo, milestones for completing the work, or how the effort would be funded  Without a plan to obtain asset inventory and condition assessment information WMATA will continue to lack critical information needed for good capital planning and sound investment decision-making  WMATA has reported significant progress toward its goals of reducing track defects and fire incidents, but still faces several challenges with implementing its track preventive maintenance program  WMATA defines an incident as any unplanned event that disrupts rail revenue service  According to WMATA officials, within the track preventive maintenance program WMATA seeks to reduce incidents specifically caused by electrical wayside fires and track defects each by 50 percent from fiscal year 2017 to fiscal year 2019  WMATA reported that in fiscal year 2018 it had met its goal for track defect incidents but not for electrical wayside fires  According to officials, track defect incidents—which include incidents caused by defective fasteners, switches, and “ballast”—were reduced by 50 percent from a total of 778 in fiscal year 2017 to 387 in fiscal year 2018  Electrical-wayside-fire incidents—including incidents caused by cable and insulator fires—went down 20 percent from a total of 55 in fiscal year 2017 to 44 in fiscal year 2018 (see fig 5)  Although WMATA has reduced both track defect incidents and electrical fires, the track preventive maintenance program is not intended to address the full range of all defects and track fires that may occur on the system  WMATA officials told us that the track preventive maintenance program specifically seeks to reduce electrical-wayside-fire incidents, which are a specific sub-set of overall track fires, and does not include non-electrical fires or smoke incidents, such as the ones caused by railcars or debris  WMATA captures and publicly reports the non-electrical fires as part of its quarterly Metro Performance Report, but according to WMATA officials, these fires are not specifically addressed through the track preventive maintenance program  While electrical fires decreased in fiscal year 2018, non-electrical fires did not change, as WMATA reported 23 non-electrical fires for both fiscal years 2017 and 2018  Additionally the track preventive maintenance program addresses a certain sub-set of track defect incidents such as those caused by loose fasteners and defective switches  According to WMATA, these track defect incidents can be addressed through its track geometry, torqueing, and switch maintenance initiatives  WMATA addresses other types of track defects, such as rail breaks and third-rail defects, through its capital program  However, according to WMATA, track defects attributable to the capital program are still included as part of the overall goal to reduce all track defect incidents by 50 percent by fiscal year 2019  WMATA established goals for completing each of the six track preventive maintenance initiatives within a certain time period and reported that in fiscal year 2018 it was on-track to meet or exceed those goals for four of the initiatives  For example, in implementing its “cable meggering” initiative, WMATA established a goal to inspect and replace electric cables across its entire rail system within 4 years  According to WMATA, it met its target for fiscal year 2018 by completing 25 percent of the entire system in that year  In addition to cable meggering, WMATA also met its annual targets for the switch maintenance, track bed cleaning, and stray current-testing initiatives  As for the two initiatives behind schedule, the torqueing initiative was 70 percent complete and the tamping initiative stood at 90 percent for the 2018 target (see table 2)  Officials told us they have developed various ways to improve efficiency with these initiatives  For instance, WMATA improved the productivity of its switch maintenance initiative by separating the work to inspect the switches from the follow-up repair work to grind and weld them  These activities had previously been conducted by the same team  However, WMATA faces challenges in implementing the track preventive maintenance program moving forward  WMATA officials described track preventive maintenance as a necessary operation that must be continuously performed and balanced in conjunction with regular train operations that provide service to their customers  According to WMATA officials, executing this new program requires regular refinements to ensure it continues to progress toward its desired outcomes  Among the implementation challenges identified by WMATA officials were the following: Securing Sufficient Track Time  WMATA officials told us that getting adequate time to perform track maintenance is difficult because it requires reducing the number of hours in which WMATA provides service to customers  Consequently, increased maintenance hours can result in lost revenue  Officials from the peer transit agencies we interviewed stated that the tension between conducting maintenance and providing service is common in the transit industry  According to WMATA officials, prior to SafeTrack, windows for performing track maintenance were not sufficient to complete all necessary work, partially because of this need to balance maintenance hours and service hours  To address this issue, WMATA increased its weekly overnight work hours from 33 hours to 39 hours during SafeTrack  After SafeTrack was complete, WMATA extended weekly overnight work hours again to a total of 41 hours  However, maintaining these extended overnight work hours past fiscal year 2019 requires approval from WMATA’s board of directors  As a result, the long-term viability of WMATA’s track preventive maintenance program is partially dependent on the board’s decision to balance the competing demands for service hours and maintenance time  Work Time Productivity  To maintain extended track-maintenance hours into succeeding years, it will be important for WMATA to demonstrate the new program’s productivity  According to WMATA officials, making the most productive use of the extended working hours is a challenge, but it will be necessary to justify the extended maintenance windows  WMATA officials told us that only a portion of overnight work hours yields productive maintenance time  For example, once a line ceases operations, it takes an additional hour for all trains to reach their final destination, and another hour after that to safely turn off all power running to the track and then establish a work zone  Once maintenance work is completed, additional time must be allotted for restoring power and allowing trains to move back into position  Because of these requirements, a five-hour work window may only yield two hours of productive work time (called “wrench time”)  For this reason, WMATA began tracking its wrench time at the beginning of fiscal year 2018  As of June 2018, WMATA reported that average wrench time had increased from about 2 0 hours per day in July 2017 to 2 37 hours  Resource Constraints  According to WMATA officials, having sufficient people with the necessary skills and experience to perform track maintenance work is a significant challenge  For instance, expanded maintenance windows have increased WMATA’s workforce requirements  As a result, WMATA has used contractors to assist with its stray-current testing and track bed cleaning initiatives  In another example, WMATA’s torqueing initiative is particularly resource intensive as the entire rail system contains 135 miles of “direct fixation” track, where the torqueing work is being done, and over 504,000 fasteners to check and tighten as necessary  According to WMATA officials, bolts and fasteners are torqued during their initial installment and then again 90 days afterward as part of the initial capital expenditure  After that, any subsequent torqueing is executed as part of the new track preventive maintenance program  WMATA stated that the torqueing initiative seeks to torque all 135 miles of direct fixation track annually  WMATA officials said the torqueing initiative is a mix of contractor and in-house staff, with contractors supplementing WMATA forces as needed  WMATA’s track preventive maintenance program has followed certain leading program management practices such as establishing key performance metrics and monitoring progress toward them  Leading practices recommend that organizations establish performance baselines for their programs and communicate performance metrics to key stakeholders  For instance, as previously noted, WMATA established a measureable program goal to reduce track-defect and electrical-wayside- fire incidents by 50 percent within 2 years, and WMATA also established time periods to complete its system-wide preventive maintenance initiatives  In addition, WMATA’s Rail Services Department—which manages the track preventive maintenance program—among other things, holds a monthly “RailSTAT” meeting in which the teams leading the preventive maintenance initiatives report their progress toward these goals to WMATA’s management  However, WMATA’s program does not fully align with other applicable internal-control standards or leading program-management practices  Specifically, COSO internal control standards and leading practices identified by the Project Management Institute’s The Standard for Program Management stresses the importance of identifying and assessing program risks and developing a program management plan  COSO recommends that organizations identify risks to the achievement of its objectives and analyze risks as a basis for determining how the risks should be managed  Furthermore, the risk identification is to be comprehensive  The Standard for Program Management also recommends that when identifying risks, the assessments be both qualitative and quantitative in nature  Regarding program management plans: The Standard for Program Management recommends that organizations develop program management plans that align with organizational goals and objectives  This includes aligning the program management plan with the organization’s overall strategic plan  Elements of the plan are to provide a roadmap that identifies such things as milestones and decision points to guide program activities  In developing the track preventive maintenance program, WMATA did not fully identify or quantitatively assess risks associated with the program  WMATA officials told us that in developing the track preventive maintenance program they used their professional judgment to identify track-defect and fire incidents as the most significant risks that they needed to address through the program  However, WMATA’s risk identification was not comprehensive in nature, as it only considered two technical aspects of track maintenance: electrical fires and track defects  As previously mentioned, non-electrical fires—which were not included in the scope of the program or risk assessment—did not change from fiscal year 2017 through 2018 and represent approximately 30 percent of all fires on the system over those years  Although WMATA officials told us in designing the program they reviewed track-related incident data from 2016, they did not quantitatively analyze the impact of these incidents on service or safety  In addition, WMATA did not consider broader strategic risks to its program, such as the availability of a program’s funding and stakeholders’ support for the continuation of the program  Specifically, while WMATA has identified several challenges with implementing the program—such as securing sufficient track time, demonstrating work time productivity, and overcoming resource constraints—none of these factors, or potential mitigations, were documented in a risk assessment in developing the program  WMATA has also not prepared a program management plan for the track preventive maintenance program  Although WMATA has identified program goals, officials told us that WMATA has not formally documented the overall structure of the program or how it would be implemented  Instead, the officials said the presentations they provide to WMATA’s board of directors, along with their ongoing staff and executive team meetings, regarding the track preventive maintenance program cover the relevant information needed for running the program  While providing such information to the WMATA board of directors provides some accountability for the program, these presentations do not represent a formal program management plan that links with WMATA’s strategic plan or that identifies milestones and decision points necessary to guide the program  As we previously reported, WMATA did not develop a project management plan before starting its SafeTrack work, and due to this omission and other issues, we found that WMATA lacked assurance that the approach taken with SafeTrack was the most effective way to identify and address safety issues  Furthermore, as this is the first time WMATA has implemented a track preventive maintenance program, a program management plan could help formally establish the program, provide strategic guidance for this new program by providing accountability for both internal and external stakeholders, and ensure that program goals are met  A program management plan could also provide practical benefits, such as helping ensure that WMATA’s extended overnight work hours are efficiently implemented and that sufficient resources are devoted to the program  Without the strategic direction provided by a comprehensive risk assessment and a formal program management plan, WMATA lacks a documented vision for how the track preventive maintenance program should be structured and implemented in order to meet the agency’s strategic goals and improve track safety  Specifically, without a risk assessment that uses quantitative and qualitative data to assess risks— such as data for all fires on the system and qualitative risks such as securing sufficient time for maintenance—WMATA lacks assurance that the program is comprehensively designed to address risks affecting the safety of the rail system or other risks that could hinder the program’s success  Moreover, a program management plan that draws on information from a comprehensive risk assessment would provide WMATA officials with the assurance that they are prepared to respond to current and future challenges that could threaten the long-term viability of the program  Finally, although WMATA developed the track preventive maintenance program to prevent the need for another emergency repair project like SafeTrack, without a formal program management plan, the WMATA employees charged with managing and implementing the program lack an important document to guide their decision-making to meet that objective and the agency’s overall strategic objectives  Developing a program management plan would outline the specific requirements to successfully implement the program, including necessary track time, expected productivity of program initiatives, and required resources  Furthermore, it would provide WMATA’s board of directors with confidence that the program has a clear roadmap with milestones and decision points as the board considers maintaining the extended overnight work hours necessary to implement the program  WMATA’s rail and bus systems provide nearly a million passenger trips each day, and those passengers rely on WMATA for safe and reliable public transportation in the nation’s capital and the surrounding areas  The federal, state, and local jurisdictions that fund WMATA expect WMATA to wisely use taxpayer funds to ensure the system is safe and reliable  WMATA can better meet these expectations by establishing documented policies and procedures that outline how the new capital planning process will work and the basis of investment decisions  In addition, developing measures to assess the performance of individual projects and the capital planning process would provide greater assurance to WMATA’s funding partners that its investment decisions result in a measurable improvement in operating performance, reliability, or other metrics  Furthermore, WMATA’s recent efforts to establish an Enterprise Asset Management Program, once finalized, could help WMATA develop a more complete inventory of its assets and collect critical information on their condition—both of which are consistent with sound capital planning  However, without a plan that provides specific details for obtaining this information, WMATA will continue to lack the critical asset information necessary to make lasting improvements in its capital planning process and make sound capital-investment decisions  Similarly, track preventive maintenance plays a critical role as WMATA works to reduce the track defects and fires that have endangered safety and service reliability  WMATA could better demonstrate the direction of the track preventive maintenance program and how it can improve track safety by more comprehensively assessing the technical and broader risks facing the program and by developing a formal plan that provides greater assurance WMATA is prepared to address challenges that could threaten the long-term viability of the program  Both actions would help WMATA better focus the program on critical maintenance needs and demonstrate its value to WMATA’s board of directors and other stakeholders as WMATA endeavors to provide safe, reliable, and quality service to its riders  We are making the following five recommendations to WMATA  The General Manager of WMATA should establish documented policies and procedures for the new capital planning process  These policies and procedures should include methodologies for ranking and selecting capital projects for funding in WMATA’s fiscal year 2020 capital budget and fiscal years 2020-2025 Capital Improvement Program and for future planning cycles  (Recommendation 1) The General Manager of WMATA should develop performance measures to be used for assessing capital investments and the capital planning process to determine if the investments and planning process have achieved their planned goals and objectives  (Recommendation 2) The General Manager of WMATA should develop a plan for obtaining complete information regarding WMATA’s asset inventory and physical condition assessments, including assets related to track and structures  (Recommendation 3) The General Manager of WMATA should conduct a comprehensive risk assessment of the track preventive maintenance program that includes both a quantitative and qualitative assessment of relevant program risks  In addition to considering technical program risks, WMATA should also consider broader program risks, such as the availability of funding for the program and stakeholders’ support  (Recommendation 4) The General Manager of WMATA should prepare a formal program management plan for the track preventive maintenance program that aligns with WMATA’s strategic plan, addresses how the program is linked to overall strategic goals and objectives, and includes program milestones and decision points  (Recommendation 5) We provided a draft of this report to WMATA and the Department of Transportation for review and comment  WMATA provided written comments, which are reprinted in appendix II, and technical comments, which we incorporated as appropriate in the report  The Department of Transportation provided technical comments, which we incorporated as appropriate  WMATA concurred in part, or with the intent of four of the recommendations, and disagreed with a fifth  Specifically, regarding the first recommendation, which is that WMATA establish documented policies and procedures for the new capital planning process, and that the policies and procedures include methodologies for ranking and selecting capital projects for the fiscal year 2020 capital budget and fiscal year 2020—2025 capital-improvement program  WMATA stated that it agreed with the recommendation, in part  WMATA said it will continue its efforts to finalize and document policies and procedures for the capital planning process for fiscal year 2021 and beyond  WMATA noted that it already has in place numerous planning tools, such as the 2016 Capital Needs Inventory assessment, which helped inform the fiscal year 2020 capital planning process  According to WMATA, it is currently reviewing policies, procedures, training materials, and other documents for the fiscal year 2020 planning process, and those documents will be updated and formalized through final documentation in fiscal year 2021  WMATA noted that it anticipates that many of the elements we recommend regarding the capital planning process will be part of the process documented in fiscal year 2021  For example, WMATA expects that additional automation, decision-making, governance, and reporting capabilities, will be part of the process that will be documented for fiscal year 2021  However, while WMATA has tools available to inform the capital planning process, it has not prepared documented policies and procedures for this process in fiscal year 2020  As we reported, without documented policies and procedures, including those for ranking and selecting projects for the fiscal year 2020 capital budget, WMATA’s stakeholders do not have reasonable assurance that capital investment decisions are made using a sound and transparent process  Taking action now to establish methodologies for ranking and selecting projects for the fiscal year 2020 capital budget would provide WMATA with an opportunity to improve upon those methodologies for the fiscal year 2021 capital planning process to better ensure investments are directed to WMATA’s highest priority needs  As such, we continue to believe this recommendation is valid and that WMATA should fully implement it  Regarding the second recommendation that WMATA develop performance measures for assessing capital investments and the capital planning process, WMATA stated that it agreed with the intent of the recommendation  WMATA also stated that it has developed such measures through compliance with federal requirements, including the FTA’s performance-based planning requirements and the requirement under MAP-21 that tier I transit providers, such as WMATA, establish state-of-good-repair targets that are linked to the capital program  WMATA noted these targets are set forth in its Transit Asset Management Plan  Although WMATA’s October 2018 Transit Asset Management plan includes some broad performance measures and targets for the state-of-good-repair for its various asset classes, as we reported, WMATA has not developed performance measures to assess individual capital projects or the capital planning process itself, as suggested by leading practices in the Executive Guide  As discussed in the report, such measures are important to determine if capital investments have achieved their expected benefits and if they have achieved organizational goals  Leading practices also indicate that by using a mixture of measures managers can assess performance based on a comprehensive view of the needs and objectives of an organization  These needs and objectives can go beyond just the state-of-good-repair to include such things as measures for assessing projects that would improve service reliability, expand capacity, or achieve financial objectives  We continue to believe that fully implementing this recommendation would help ensure that capital investments meet their intended outcomes and that the capital planning process helps WMATA achieve its strategic goals and objectives  Regarding the third recommendation that WMATA develop a plan for obtaining complete information about asset inventories and condition assessments, WMATA stated that it agreed with the intent of the recommendation and that its 2018 Transit Asset Management Plan outlines plans for continuing its asset inventory update  WMATA also said that it is working to ensure it has a complete asset inventory that addresses legacy information and that includes accurate, up-to-date condition assessments  As we reported, the Enterprise Asset Management Program—the program that WMATA told us it plans to use to continue development of asset inventories and condition assessments—includes some elements of good project management, but it also lacks an established plan for collecting asset inventory and condition assessment information  Without a plan to obtain asset inventory and condition assessment information WMATA will continue to lack critical information needed for good capital planning and sound investment decision-making  Thus, we continue to believe that this recommendation is valid and that WMATA should fully implement it  Regarding the fourth recommendation that WMATA conduct a comprehensive risk assessment of the track preventive maintenance program that includes both quantitative and qualitative assessment of relevant program risks, WMATA stated that it agreed with the intent of the recommendation and is putting in place a new process that will address it  Specifically, WMATA stated it is in the process of developing a new Reliability Centered Maintenance process that will include a comprehensive risk assessment of track infrastructure that includes consideration of broader risks such as costs, funding, and track access  According to WMATA, the new process is an engineering framework that will define the maintenance regimen, including preventive maintenance, and improve safety, reliability, and cost-effectiveness  During our review, WMATA officials did not discuss the Reliability Centered Maintenance process in detail or provide documentation that allowed us to evaluate how this process might interface with the current track preventive maintenance program  As a result, we were not able to evaluate how it might address identification and assessment of risks associated with track preventive maintenance  As we reported, going forward track preventive maintenance will play a critical role as WMATA works to reduce track defects and fires  We will review WMATA’s actions to conduct a comprehensive risk assessment as part of our routine recommendation follow-up process  Regarding the fifth recommendation that WMATA prepare a formal program management plan for the track preventive maintenance program, WMATA stated that it disagreed with the recommendation  WMATA noted that specific technical details of the track preventive maintenance program are evolving as it better understands the most effective maintenance regime through implementation of the Reliability Centered Maintenance process  WMATA stated that it believes the framework of Reliability Centered Maintenance is better suited to the ongoing mission of physical asset management than traditional project and program management tools  According to WMATA, the purpose of Reliability Centered Maintenance is to ensure that all efforts are focused on the safety, reliability, and cost-effectiveness of assets through their lifecycle, which is more relevant and applicable to WMATA’s strategic plan than any individual preventive maintenance program  As stated above, WMATA did not provide details about Reliability Centered Maintenance during our review so we are not able to evaluate this process in relation to the track preventive maintenance program  We will review WMATA’s actions related to implementation of the Reliability Centered Maintenance process as part of our routine recommendation follow-up process  We continue to believe this recommendation is valid and that WMATA should fully implement it  We will send copies of this report to appropriate congressional committees, the Secretary of Transportation, the Administrator of the Federal Transit Administration, and the General Manager of WMATA  In addition, we will make copies available to others upon request, and the report will be available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2834 or goldsteinm@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix III  This report examines: (1) How WMATA expended its capital funding from fiscal years 2011 through 2017; (2) How WMATA’s new capital planning process addresses weaknesses it identified in the previous process; and (3) WMATA’s progress toward its track preventive maintenance goals and how the program aligns with leading program management practices  For each of our objectives we reviewed pertinent federal statutes and regulations as well as WMATA and FTA policies and documents  We also selected a non-generalizable sample of five similar U S  transit agencies based on similarity to WMATA in transit route mileage, system use, capital spending, system age, and rail fleet age  We also factored geographical diversity into our selection process  We then interviewed the officials from these selected transit agencies using a standard set of questions to learn how they utilize their capital funds, conduct capital planning, and oversee maintenance; and then we compared their processes to WMATA  Transit route mileage, system use, capital spending, and rail fleet age were measured using data from FTA’s National Transit Database  We measured system age according to data available within the American Public Transportation Association’s 2017 Public Transportation Fact Book, and geographical diversity was determined through data available from the U S  Census Bureau  The transit agencies we selected were: (1) Bay Area Rapid Transit, Oakland, California; (2) Chicago Transit Authority, Chicago, Illinois; (3) Massachusetts Bay Transportation Authority, Boston, Massachusetts; (4) Metropolitan Atlanta Rapid Transit Authority, Atlanta, Georgia; and (5) Southeastern Pennsylvania Transportation Authority, Philadelphia, Pennsylvania  To assess WMATA’s capital spending from 2011 through 2017, we interviewed knowledgeable officials from WMATA and FTA and also reviewed WMATA annual budgets, fourth-quarter and year-end financial reports, budget reconciliation reports, comprehensive annual financial reports, and FTA grant awards  We selected fiscal year 2011 because it was the first year in which WMATA received federal funding authorized by the Passenger Rail Investment and Improvement Act of 2008 (PRIIA), and we selected fiscal year 2017 because it was the most recent year that capital expenditure data were available at the time of our review  By analyzing this information we determined that the following sources provided the most comprehensive and reliable available data on each of the following topics for our report (see table 3): We collected the aforementioned data, analyzed them to identify errors or other anomalies, and interviewed officials to determine how the data are compiled and checked for accuracy  We determined that these data had some limitations, as an external audit report of WMATA financial information for fiscal year 2016 noted a material weakness with WMATA’s process for accounting acquisition costs of capital assets  Specifically, there were inconsistencies between WMATA’s general ledger and sub- ledger, which are used to record acquisition costs, depreciation, and other financial information related to capital assets  As a result, additional steps were required to reconcile the differences between the two sources and could have resulted in a material error  However, after interviewing WMATA officials about the weakness and assessing the available financial information, we determined that the data we used were sufficiently reliable for our purpose of showing general trends of capital expenditures  Our analysis sought to depict how WMATA allocates and expends funds according to major asset categories within its capital-improvement plan  However, these asset categories only remained consistent from 2011 through 2015, and were revised during 2016 and 2017  However, we determined that each asset category consisted of Capital Improvement Projects that were each assigned a number  These projects and their corresponding numbers remained in existence from fiscal year 2011 through 2017, even though the asset categories were updated in fiscal year 2016  Tracking by Capital Improvement Project number provided a means to report consistently through that time period  Therefore, we used the asset categories from fiscal years 2011 through 2015 as our base reporting categories  These categories consisted of: (1) Vehicles/Vehicle Parts, (2) Rail System Infrastructure Rehabilitation, (3) Maintenance Facilities, (4) Systems and Technology, (5) Track and Structures, (6) Passenger Facilities, (7) Maintenance Equipment, (8) Other Facilities, and (9) Project Management and Support  We consolidated WMATA’s nine asset categories into five asset categories in order to represent broader categories of investment: Rail and Bus Vehicle Fleet (Vehicle/Vehicle Parts), Fixed Rail Infrastructure (Rail System Infrastructure and Track and Structures), Maintenance Facilities and Equipment (Maintenance Facilities and Maintenance Equipment), Passenger and Other Facilities (Passenger Facilities and Other Facilities), and Business Systems and Project Management Support (Systems and Technology and Project Management and Support)  We then reviewed WMATA’s fiscal year 2016 Fourth Quarter Report, fiscal year 2017 Fourth Quarter Report, and fiscal year 2017 Budget Reconciliation Report to match each project number from those two years to their corresponding category from fiscal year 2011 through 2015  To assess WMATA’s new capital planning process and how it addresses weaknesses WMATA identified in the previous process, we interviewed WMATA officials about their capital planning process and reviewed WMATA documentation related to the capital planning process  This included Capital Needs Inventories, WMATA’s policy for preparation of the 2010 and 2016 Capital Needs Inventories, annual capital budgets—to include capital improvement programs, and guidance documents issued by WMATA related to submitting projects for inclusion in the annual capital budget  We also reviewed the fiscal year 2018 business plan for WMATA’s Capital Planning and Program Management Department  We also interviewed officials from the Metropolitan Washington Council of Governments, the American Public Transportation Association, and FTA to discuss WMATA’s capital planning and budgeting processes  Furthermore, we compared WMATA’s capital planning practices to leading practices identified in GAO’s Executive Guide  The Executive Guide was used since it identifies leading practices for capital decision- making that are applicable to a wide variety of organizations, both public and private  For example, the Executive Guide developed leading capital planning practices by (1) identifying government and private sector organizations recognized for outstanding capital decision-making practices and (2) identifying and describing the leading capital decision- making practices implemented by these organizations  To identify leading practices for capital planning, we also reviewed Transit Cooperative Research Program Report 157  This report developed a framework for transit agencies to use when prioritizing the rehabilitation and replacement of capital assets and discusses leading practices in how to do this  We also identified project management principles from the Project Management Institute, Inc  Finally, we discussed capital planning with the peer transit agencies and prepared a summary of various aspects of capital planning in these agencies  To examine progress toward goals in WMATA’s track preventive maintenance program and how the program compares with leading program management practices, we reviewed WMATA documentation about the program, interviewed WMATA officials, and analyzed track- defect data and electrical-wayside-fire data provided by WMATA for fiscal years 2016 through 2018—which were the only years detailed track defect and electrical fire incident data were available  In order to determine whether the data provided were sufficiently reliable, we checked the data for errors, conducted interviews with knowledgeable officials to learn their procedures for collecting and analyzing the data, and performed independent tests that included verifying WMATA’s final tally of track defect and fire incidents and verifying there were no extended periods of time where data was missing  We also provided a set of data reliability questions to determine whether procedures were sufficient  After performing these steps we determined that the data were sufficiently reliable for the purposes of our report  In our interviews with WMATA, officials also described what goals they had created for the track preventive maintenance program, their progress in meeting those goals, and provided documentation to demonstrate their progress, which we reviewed  We also interviewed officials from the American Public Transportation Association and the American Railway Engineering and Maintenance-of-Way Association about best maintenance practices in the transit industry  We then compared WMATA’s track preventive maintenance program to leading program management practices identified by the Project Management Institute, Inc ’s The Standard for Program Management and internal control standards published by the Committee of Sponsoring Organizations of the Treadway Commission (COSO)  The Project Management Institute’s, standards are utilized worldwide and provide guidance on how to manage various aspects of projects, programs, and portfolios  In particular, The Standard for Program Management provides guidance that is generally recognized to support good program-management practices for most programs, most of the time  We conducted our work from November 2017 to January 2019 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the contact named above, Matt Barranca (Assistant Director), Richard Jorgenson (Analyst in Charge), Melissa Bodeau, Lacey Coppage, Cory Gerlach, Erin Guinn-Villareal, Kirsten Lauber, Joshua Ormond, and Patrick Tierney made significant contributions to this report 


Report 2: FEMA’s mission is to help people before, during, and after disasters  It provides assistance to those affected by emergencies and disasters by supplying immediate needs (e g , ice, water, food, and temporary housing) and providing financial assistance grants for damage to personal or public property  FEMA also provides non-disaster assistance grants to improve the nation’s preparedness, readiness, and resilience to all hazards  FEMA accomplishes a large part of its mission through awarding grants to state, local, and tribal governments and nongovernmental entities to help communities prevent, prepare for, protect against, mitigate the effects of, respond to, and recover from disasters and terrorist attacks  As previously mentioned, for fiscal years 2005 through 2014, the agency obligated about $104 5 billion in disaster relief grants  In addition, as of April 2018, the four major disasters in 2017—hurricanes Harvey, Irma, and Maria; and the California wildfires—had resulted in over $22 billion in FEMA grants  The current FEMA grants management environment is highly complex with many stakeholders, IT systems, and users  Specifically, this environment is comprised of 45 active disaster and non-disaster grant programs, which are grouped into 12 distinct grant categories  For example, one program in the Preparedness: Fire category is the Assistance to Firefighters Grants (AFG) program, which provides grants to fire departments, nonaffiliated emergency medical service organizations, and state fire training academies to support firefighting and emergency response needs  As another example, the Housing Assistance grant program is in the Recovery Assistance for Individuals category and provides financial assistance to individuals and households in geographical areas that have been declared an emergency or major disaster by the President  Table 1 lists FEMA’s non-disaster and disaster-based grant categories  According to FEMA, the processes for managing these different types of grants vary because the grant programs were developed independently by at least 18 separate authorizing laws that were enacted over a 62-year period (from 1947 through 2009)  The various laws call for different administrative and reporting requirements  For example, the Robert T  Stafford Disaster Relief and Emergency Assistance Act, as amended, established the statutory authority for 11 of the grant programs, such as the administration of Public Assistance and Individual Assistance grant programs after a presidentially declared disaster  The act also requires the FEMA Administrator to submit an annual report to the President and Congress covering FEMA’s expenditures, contributions, work, and accomplishments, pursuant to the act  As another example, the National Dam Safety Program Act established one of the grant programs aimed at providing financial assistance to improve dam safety  Key stakeholders in modernizing the IT grants management environment include the internal FEMA officials that review, approve, and monitor the grants awarded, such as grant specialists, program analysts, and supervisors  FEMA has estimated that it will need to support about 5,000 simultaneous internal users of its grants management systems  Other users include the grant recipients that apply for, receive, and submit reports on their grant awards; these are considered the external system users  These grant recipients can include individuals, states, local governments, Indian tribes, institutions of higher education, and nonprofit organizations  FEMA has estimated that there are hundreds of thousands of external users of its grants systems  The administration of the many different grant programs is distributed across four divisions within FEMA’s organizational structure  Figure 1 provides an overview of FEMA’s organizational structure and the divisions that are responsible for administering grants  Within three of the four divisions—Resilience, United States Fire Administration, and Office of Response and Recovery—16 different grant program offices are collectively responsible for administering the 45 grant programs  The fourth division consists of 10 regional offices that help administer grants within their designated geographical regions  For example, the Office of Response and Recovery division oversees three different offices that administer 13 grant programs that are largely related to providing assistance in response to presidentially declared disasters  Figure 2 shows the number of grant programs administered by each of the four divisions’ grant program and regional offices  In addition, appendix II lists the names of the 45 grant programs  FEMA’s OCIO is responsible for developing, enhancing, and maintaining the agency’s IT systems, and for increasing efficiencies and cooperation across the entire organization  However, we and the DHS Office of Inspector General (OIG) have previously reported that the grant programs and regional offices develop information systems independent of the OCIO and that this has contributed to the agency’s disparate IT environment  We and the DHS OIG have reported that this disparate IT environment was due, in part, to FEMA’s decentralized IT budget and acquisition practices  For example, from fiscal years 2010 through 2015, the OCIO’s budget represented about one-third of the agency’s IT budget, with the grant program offices accounting for the remaining two-thirds of that budget  In February 2018, the OIG found that FEMA had shown limited progress in improving its IT management and that many of the issues reported in prior audits remained unchanged  As such, the OIG initiated a more comprehensive audit of the agency’s IT management that is ongoing  FEMA has identified 10 primary legacy IT systems that support its grants management activities  According to the agency, most of these systems were developed to support specific grant programs or grant categories  Table 2 summarizes the 10 primary legacy systems  According to FEMA officials, the 10 primary grant systems are all in operation (several have been for decades) and are not interoperable  As a result, individual grant programs and regional offices have independently developed work arounds intended to address existing capability gaps with the primary systems  FEMA officials stated that while these work arounds have helped the agency partially address capability gaps with its primary systems, they are often nonstandardized processes, and introduce the potential for information security risks and errors  This environment has contributed to labor-intensive manual processes and an increased burden for grant recipients  The disparate systems have also led to poor information sharing and reporting capabilities, as well as difficulty reconciling financial data  The DHS OIG and we have previously highlighted challenges with FEMA’s past attempts to modernize its grant management systems  For example, In December 2006, the DHS OIG reported that EMMIE, an effort to modernize its grants management systems and provide a single grants processing solution, was being developed without a clear understanding and definition of the future solution  The report also identified the need to ensure crosscutting participation from headquarters, regions, and states in developing and maintaining a complete, documented set of FEMA business and system requirements  In April 2016, we found weaknesses in FEMA’s development of the EMMIE system  For example, we noted that the system was implemented without sufficient documentation of system requirements, an acquisition strategy, up-to-date cost estimate and schedule, total amount spent to develop the system, or a systems integration plan  In response to our findings and related recommendations, FEMA took action to address these issues  For example, the agency implemented a requirements management process that, among other things, provided guidance to programs on analyzing requirements to ensure that they are complete and verifiable  We reported in November 2017 that EMMIE lacked the ability to collect information on all pre-award activities and, as a result, agency officials said that they and applicants used ad hoc reports and personal tracking documents to manage and monitor the progress of grant applications  FEMA officials added that applicants often struggled to access the system and that the system was not user friendly  Due to EMMIE’s shortfalls, the agency had to develop another system in 2017 to supplement EMMIE with additional grant tracking and case management capabilities  FEMA initiated GMM in 2015, in part, due to EMMIE’s failed attempt to modernize the agency’s grants management environment  The program is intended to modernize and streamline the agency’s grants management environment  To help streamline the agency’s grants management processes, the program established a standard framework intended to represent a common grants management lifecycle  The framework consists of five sequential phases—pre-award, award, post-award, closeout, and post- closeout—along with a sixth phase dedicated to continuous grant program management activities, such as analyzing data and producing reports on grant awards and managing IT systems  FEMA also established 43 distinct business functions associated with these six lifecycle phases  Figure 3 provides the general activities that may occur in each of the grant lifecycle phases, but specific activities would depend on the type of grant being administered (i e , disaster versus non-disaster)  GMM is expected to be implemented within the complex IT environment that currently exists at FEMA  For example, the program is intended to replace the 10 legacy grants management systems, and potentially many additional subsystems, with a single IT system  Each of the 10 legacy systems was developed with its own database(s) and with no standardization of the grants management data and, according to FEMA officials, this legacy data has grown significantly over time  Accordingly, FEMA will need to migrate, analyze, and standardize the grants management data before transitioning it to GMM  The agency awarded a contract in June 2016 to support the data migration efforts for GMM  The agency also implemented a data staging environment in October 2017 to migrate the legacy data and identify opportunities to improve the quality of the data  Further, the GMM system is expected to interface with a total of 38 other systems  These include 19 systems external to DHS (e g , those provided by commercial entities or other federal government agencies) and 19 systems internal to DHS or FEMA  Some of the internal FEMA systems are undergoing their own modernization efforts and will need to be coordinated with GMM, such as the agency’s financial management systems, national flood insurance systems, and enterprise data warehouses  For example, FEMA’s Financial Systems Modernization Program was originally expected to deliver a new financial system in time to interface with GMM  However, the financial modernization has been delayed until after GMM is to be fully implemented; thus, GMM will instead need to interface with the legacy financial system  As a result, GMM is in the process of removing one of its key performance parameters in the acquisition program baseline related to financial systems interoperability and timeliness of data exchanged  In May 2017, DHS approved the acquisition program baseline for GMM  The baseline estimated the total lifecycle costs to be about $251 million, initial operational capability to be achieved by September 2019, and full operational capability to be achieved by September 2020  FEMA intends to develop and deploy its own software applications for GMM using a combination of commercial-off-the-shelf software, open source software, and custom developed code  The agency plans to rely on an Agile software development approach  According to FEMA planning documentation, the agency plans to fully deliver GMM by September 2020 over eight Agile development increments  Agile development is a type of incremental development, which calls for the rapid delivery of software in small, short increments  Many organizations, especially in the federal government, are accustomed to using a waterfall software development model  This type of model typically consists of long, sequential phases, and differs significantly from the Agile development approach  We have previously reported that DHS has sought to establish Agile software development as the preferred method for acquiring and delivering IT capabilities  However, the department has not yet completed critical actions necessary to update its guidance, policies, and practices for Agile programs, in areas such as, developing lifecycle cost estimates, managing IT requirements, testing and evaluation, oversight at key decision points, and ensuring cybersecurity  (See appendix III for more details on the Agile software development approach ) FEMA’s acquisition approach includes using contract support to assist with the development and deployment efforts  The agency selected a public cloud environment to host the computing infrastructure  In addition, from March through July 2017, the agency used a short-term contract aimed at developing prototypes of GMM functionality for grant tracking and monitoring, case management of disaster survivors, grant reporting, and grant closeout  The agency planned to award a second development contract by December 2017 to complete the GMM system (beyond the prototypes) and to begin this work in September 2018  However, due to delays in awarding the second contract to develop the complete GMM system, in January 2018, the program extended the scope and time frames of the initial short-term prototype contract for an additional year to develop the first increment of the GMM system— referred to as the AFG pilot  On August 31, 2018, FEMA awarded the second development contract, which is intended to deliver the remaining functionality beyond the AFG pilot (i e , increments 2 through 8)  FEMA officials subsequently issued a 90-day planning task order for the Agile development contractor to define the work that needs to be done to deliver GMM and the level of effort needed to accomplish that work  However, the planning task order was paused after a bid protest was filed with GAO in September 2018  According to FEMA officials, they resumed work on the planning task order after the bid protest was withdrawn by the protester on November 20, 2018, and then the work was paused again during the partial government shutdown from December 22, 2018, through January 25, 2019  FEMA began working on the AFG pilot—GMM’s first increment—in January 2018  This increment was intended to pilot GMM’s use of Agile development methods to replace core functionality for the AFG system (i e , one of the 10 legacy systems) This system supports three preparedness/fire-related grant programs—Assistance to Firefighters Grants Program, Fire Prevention and Safety Grant Program, and Staffing for Adequate Fire and Emergency Response Grant Program  According to FEMA officials, the AFG system was selected as the first system to be replaced because it is costly to maintain and the DHS OIG had identified cybersecurity concerns with the system  Among the 43 GMM business functions discussed earlier in this report, FEMA officials specified 19 functions to be delivered in the AFG pilot  Figure 4 shows the planned time frames for delivering the AFG pilot in increment 1 (which consisted of four 3-month Agile development sub- increments), as of August 2018  As of August 2018, the program was working on sub-increment 1C of the pilot  In September 2018, GMM deployed its first set of functionality to a total of 19 AFG users—which included seven of 169 total internal AFG users, and 12 of more than 153,000 external AFG users  The functionality supported four of the 19 business functions that are related to the closeout of grants (i e , the process by which all applicable administrative actions and all required work to award a grant have been completed)  This functionality included tasks such as evaluation of final financial reports submitted by grant recipients and final reconciliation of finances (e g , final disbursement to recipients and return of unobligated federal funds)  According to FEMA officials, closeout functionality was selected first for deployment because it was the most costly component of the legacy AFG system to maintain, as it is an entirely manual and labor-intensive process  The remaining AFG functionality and remaining AFG users are to be deployed by the end of the AFG pilot  The GMM program is executed by a program management office, which is overseen by a program manager and program executive  This office is responsible for directing the day-to-day operations and ensuring completion of GMM program goals and objectives  The program office resides within the Office of Response and Recovery, which is headed by an Associate Administrator who reports to the FEMA Administrator  In addition, the GMM program executive (who is also the Regional Administrator for FEMA Region IX) reports directly to the FEMA Administrator  GMM is designated as a level 2 major acquisition, which means that it is subject to oversight by the DHS acquisition review board  The board is chaired by the DHS Undersecretary for Management and is made up of executive-level members, such as the DHS Chief Information Officer  The acquisition review board serves as the departmental executive board that decides whether to approve GMM through key acquisition milestones and reviews the program’s progress and its compliance with approved documentation every 6 months  The board approved the acquisition program baseline for GMM in May 2017 (i e , estimated costs to be about $251 million and full operational capability to be achieved by September 2020)  In addition, the program is reviewed on a monthly basis by FEMA’s Grants Management Executive Steering Group  This group is chaired by the Deputy Administrator of FEMA  Further, DHS’s Financial Systems Modernization Executive Steering Committee, chaired by the DHS Chief Financial Officer, meets monthly and is to provide guidance, oversight, and support to GMM  For government organizations, including FEMA, cybersecurity is a key element in maintaining the public trust  Inadequately protected systems may be vulnerable to insider threats  Such systems are also vulnerable to the risk of intrusion by individuals or groups with malicious intent who could unlawfully access the systems to obtain sensitive information, disrupt operations, or launch attacks against other computer systems and networks  Moreover, cyber-based threats to federal information systems are evolving and growing  Accordingly, we designated cybersecurity as a government-wide high risk area 22 years ago, in 1997, and it has since remained on our high-risk list  Federal law and guidance specify requirements for protecting federal information and information systems  The Federal Information Security Modernization Act (FISMA) of 2014 requires executive branch agencies to develop, document, and implement an agency-wide cybersecurity program to provide security for the information and information systems that support operations and assets of the agency  The act also tasks NIST with developing, for systems other than those for national security, standards and guidelines to be used by all agencies to establish minimum cybersecurity requirements for information and information systems based on their level of cybersecurity risk  Accordingly, NIST developed a risk management framework of standards and guidelines for agencies to follow in developing cybersecurity programs  The framework addresses broad cybersecurity and risk management activities, including categorizing the system’s impact level; selecting, implementing, and assessing security controls; authorizing the system to operate (based on progress in remediating control weaknesses and an assessment of residual risk); and monitoring the efficacy of controls on an ongoing basis  Figure 5 provides an overview of this framework  Prior DHS OIG assessments, such as the annual evaluation of DHS’s cybersecurity program, have identified issues with FEMA’s cybersecurity practices  For example, in 2016, the OIG reported that FEMA was operating 111 systems without an authorization to operate  In addition, the agency had not created any corrective action plans for 11 of the systems that were classified as “Secret” or “Top Secret,” thus limiting its ability to ensure that all identified cybersecurity weaknesses were mitigated in a timely manner  The OIG further reported that, for several years, FEMA was consistently below DHS’s 90 percent target for remediating corrective action plans, with scores ranging from 73 to 84 percent  Further, the OIG reported that FEMA had a significant number of open corrective action plans (18,654) and that most of these plans did not contain sufficient information to address identified weaknesses  In 2017, the OIG reported that FEMA had made progress in addressing security weaknesses  For example, it reported that the agency had reduced the number of systems it was operating without an authorization to operate from 111 to 15 systems  According to GAO’s Business Process Reengineering Assessment Guide and the Software Engineering Institute’s Capability Maturity Model Integration® for Development, successful business process reengineering can enable agencies to replace their inefficient and outmoded processes with streamlined processes that can more effectively serve the needs of the public and significantly reduce costs and improve performance  Many times, new IT systems are implemented to support these improved business processes  Thus, effective management of IT requirements is critical for ensuring the successful design, development, and delivery of such new systems  These leading practices state that effective business process reengineering and IT requirements management involve, among other things, (1) ensuring strong executive leadership support for process reengineering; (2) assessing the current and target business environment and business performance goals; (3) establishing plans for implementing new business processes; (4) establishing clear, prioritized, and traceable IT requirements; (5) tracking progress in delivering IT requirements; and (6) incorporating input from end user stakeholders  Among these six selected leading practices for reengineering business processes and managing IT requirements, FEMA fully implemented four and partially implemented two of them for its GMM program  For example, the agency ensured strong senior leadership commitment to changing the way it manages its grants, took steps to assess and document its business environment and performance goals, defined initial IT requirements for GMM, took recent actions to better track progress in delivering planned IT requirements, and incorporated input from end user stakeholders  In addition, FEMA had begun planning for business process reengineering; however, it had not finalized plans for transitioning users to the new business processes  Further, while GMM took steps to establish clearly defined and prioritized IT requirements, key requirements were not always traceable  Table 3 summarizes the extent to which FEMA implemented the selected leading practices  According to GAO’s Business Process Reengineering Assessment Guide, the most critical factor for engaging in a reengineering effort is having strong executive leadership support to establish credibility regarding the seriousness of the effort and to maintain the momentum as the agency faces potentially extensive changes to its organizational structure and values  Without such leadership, even the best process design may fail to be accepted and implemented  Agencies should also ensure that there is ongoing executive support (e g , executive steering committee meetings headed by the agency leader) to oversee the reengineering effort from start to finish  FEMA senior leadership consistently demonstrated its commitment and support for streamlining the agency’s grants management business processes and provided ongoing executive support  For example, one of the Administrator’s top priorities highlighted in FEMA’s 2014 through 2022 strategic plans was to strengthen grants management through innovative systems and business processes to rapidly and effectively deliver the agency’s mission  In accordance with this strategic priority, FEMA initiated GMM with the intent to streamline and modernize grants management across the agency  In addition, FEMA established the Grants Management Executive Steering Group in September 2015  This group is responsible for transforming the agency’s grants management capabilities through its evaluation, prioritization, and oversight of grants management modernization programs, such as GMM  The group’s membership consists of FEMA senior leaders from across the agency’s program and business support areas, such as FEMA regions, Individual Assistance, Public Assistance, Preparedness, Office of the Chief Financial Officer, Office of Chief Counsel, OCIO, and the Office of Policy and Program Analysis  In this group’s ongoing commitment to reengineering grants management processes, it meets monthly to review GMM’s updates, risks, and action items, as well as the program’s budget, schedule, and acquisition activities  For example, the group reviewed the status of key acquisition activities and program milestones, such as the follow-on award for the pilot contractor and the program’s initial operational capability date  The group also reviewed GMM’s program risks, such as data migration challenges (discussed later in this report) and delays in the Agile development contract award  With this continuous executive involvement, FEMA is better positioned to maintain momentum for reengineering the new grants management business processes that the GMM system is intended to support  GAO’s Business Process Reengineering Assessment Guide states that agencies undergoing business process reengineering should develop a common understanding of the current environment by documenting existing core business processes to show how the processes work and how they are interconnected  The agencies should then develop a deeper understanding of the target environment by modeling the workflow of each target business process in enough detail to provide a common understanding of exactly what will be changed and who will be affected by a future solution  Agencies should also assess the performance of their current major business processes to identify problem areas that need to be changed or eliminated and to set realistically achievable, customer- oriented, and measurable business performance improvement goals  FEMA has taken steps to document the current and target grants management business processes  Specifically, The agency took steps to develop a common understanding of its grants management processes by documenting each of the 12 grant categories  For example, in 2016 and 2017, the agency conducted several nationwide user outreach sessions with representatives from FEMA headquarters, the 10 regional offices, and state and local grant recipients to discuss the grant categories and the current grants management business environment  In addition, FEMA’s Office of Chief Counsel developed a Grants Management Manual in January 2018 that outlined the authorizing laws, regulations, and agency policies for all of its grant programs  According to the Grants Management Executive Steering Group, the manual is intended to promote standardized grants management procedures across the agency  Additionally, the group expects grant program and regional offices to assess the manual against their own practices, make updates as needed, and ensure that their staff are properly informed and trained  FEMA also documented target grants management business process workflows for 18 of the 19 business functions that were notionally planned to be developed and deployed in the AFG pilot by December 2018  However, the program experienced delays in developing the AFG pilot (discussed later in this report) and, thus, deferred defining the remaining business function until the program gets closer to developing that function, which is now planned for August 2019  In addition, FEMA established measurable business performance goals for GMM that are aimed at addressing problem areas and improving grants management processes  Specifically, the agency established 14 business performance goals and associated thresholds in an October 2017 acquisition program baseline addendum, as well as 126 performance metrics for all 43 of the target grants management business functions in its March 2017 test and evaluation master plan  According to FEMA, the 14 business performance goals are intended to represent essential outcomes that will indicate whether GMM has successfully met critical, business-focused mission needs  GMM performance goals include areas such as improvements in the satisfaction level of users with GMM compared to the legacy systems and improvements in the timeliness of grant award processing  For example, one of GMM’s goals is to get at least 40 percent of users surveyed to agree or strongly agree that their grants management business processes are easier to accomplish with GMM, compared to the legacy systems  Program officials stated that they plan to work with the Agile development contractor to refine their performance goals and target thresholds, develop a plan for collecting the data and calculating the metrics, and establish a performance baseline with the legacy systems  Program officials also stated that they plan to complete these steps by September 2019—GMM’s initial operational capability date—which is when they are required to begin reporting these metrics to the DHS acquisition review board  According to GAO’s Business Process Reengineering Assessment Guide, agencies undergoing business process reengineering should (1) establish an overall plan to guide the effort (commonly referred to as an organizational change management plan) and (2) provide a common understanding for stakeholders of what to expect and how to plan for process changes  Agencies should develop the plan at the beginning of the reengineering effort and provide specific details on upcoming process changes, such as critical milestones and deliverables for an orderly transition, roles and responsibilities for change management activities, reengineering goals, skills and resource needs, key barriers to change, communication expectations, training, and any staff redeployments or reductions-in-force  The agency should develop and begin implementing its change management plan ahead of introducing new processes to ensure sufficient support among stakeholders for the reengineered processes  While FEMA has begun planning its business process reengineering activities, it has not finalized its plans or established time frames for their completion  Specifically, as of September 2018, program officials were in the process of drafting an organizational change management plan that is intended to establish an approach for preparing grants management stakeholders for upcoming changes  According to FEMA, this document is intended to help avoid uncertainty and confusion among stakeholders as changes are made to the agency’s grant programs, and ensure successful adoption of new business processes, strategies, and technologies  As discussed previously in this report, the transition to GMM will involve changes to FEMA’s disparate grants management processes that are managed by many different stakeholders across the agency  Program officials acknowledged that change management is the biggest challenge they face in implementing GMM and said they had begun taking several actions intended to support the agency’s change management activities  For example, program officials reported in October 2018 that they had recently created an executive-level working group intended to address FEMA’s policy challenges related to the standardization of grants management processes  Additionally, program officials reported that they planned to: (1) hire additional support staff focused on coordinating grants change management activities; and (2) pursue regional office outreach to encourage broad support among GMM’s decentralized stakeholders, such as state, local, and tribal territories  However, despite these actions, the officials were unable to provide time frames for completing the organizational change management plan or the additional actions  Until the plan and actions are complete, the program lacks assurance that it will have sufficient support among stakeholders for the reengineered processes  In addition, GMM did not establish plans and time frames for the activities that needed to take place prior to, during, and after the transition from the legacy AFG to GMM  Instead, program officials stated that they had worked collaboratively with the legacy AFG program and planned these details informally by discussing them in various communications, such as emails and meetings  However, this informal planning approach is not a repeatable process, which is essential to this program as FEMA plans to transition many sets of functionality to many different users during the lifecycle of this program  Program officials acknowledged that for future transitions they will need more repeatable transition planning and stated that they intend to establish such plans, but did not provide a time frame for when such changes would be made  Until FEMA develops a repeatable process, with established time frames for communicating the transition details to its customers prior to each transition, the agency risks that the transition from the legacy systems to GMM will not occur as intended  It also increases its risk that stakeholders will not support the implementation of reengineered grants management processes  Leading practices for software development efforts state that IT requirements are to be clearly defined and prioritized  This includes, among other things, maintaining bidirectional traceability as the requirements evolve, to ensure there are no inconsistencies among program plans and requirements  In addition, programs using Agile software development are to maintain a product vision, or roadmap, to guide the planning of major program milestones and provide a high-level view of planned requirements  Programs should also maintain a prioritized list (referred to as a backlog) of narrowly defined requirements (referred to as lower-level requirements) that are to be delivered  Programs should maintain this backlog with the product owner to ensure the program is always working on the highest priority requirements that will deliver the most value to the users  The GMM program established clearly defined and prioritized requirements and maintained bidirectional traceability among the various levels of requirements: Grant lifecycle phases: In its Concept of Operations document, the program established six grants management lifecycle phases that represent the highest level of GMM’s requirements, through which it derives lower-level requirements  Business functions: The Concept of Operations document also identifies the next level of GMM requirements—the 43 business functions that describe how FEMA officials, grant recipients, and other stakeholders are to manage grants  According to program officials, the 43 business functions are to be refined, prioritized, and delivered to GMM customers iteratively  Further, for the AFG pilot, the GMM program office prioritized 19 business functions with the product owner and planned the development of these functions in a roadmap  Epics: GMM’s business functions are decomposed into epics, which represent smaller portions of functionality that can be developed over multiple increments  According to program officials, GMM intends to develop, refine, and prioritize the epics iteratively  As of August 2018, the program had developed 67 epics in the program backlog  An example of one of the epics for the AFG pilot is to prepare and submit grant closeout materials  User stories: The epics are decomposed into user stories, which convey the customers’ requirements at the smallest and most discrete unit of work that must be done within a single sprint to create working software  GMM develops, refines, and prioritizes the user stories iteratively  As of August 2018, the program had developed 1,118 user stories in the backlog  An example of a user story is “As an external user, I can log in with a username and password ” Figure 6 provides an example of how GMM’s different levels of requirements are decomposed  Nevertheless, while we found requirements to be traceable at the sprint- level (i e , epics and user stories), traceability of requirements at the increment-level (i e , business functions) were inconsistent among different requirements planning documents  Specifically, the capabilities and constraints document shows that five business functions are planned to be developed within sub-increment 1A, whereas the other key planning document—the roadmap for the AFG pilot—showed one of those five functions as being planned for the sub-increment 1B  In addition, the capabilities and constraints document shows that nine business functions are planned to be developed within sub-increment 1B, but the roadmap showed one of those nine functions as being planned for the sub- increment 1C  Program officials stated that they decided to defer these functions to later sub-increments due to unexpected technical difficulties encountered when developing functionality and reprioritizing functions with the product owners  While the officials updated the roadmap to reflect the deferred functionality, they did not update the capabilities and constraints document to maintain traceability between these two important requirements planning documents  Program officials stated that they learned during the AFG pilot that the use of a capabilities and constraints document for increment-level scope planning was not ideal and that they intended to change the process for how they documented planned requirements for future increments  However, program officials did not provide a time frame for when this change would be made  Until the program makes this change and then ensures it maintains traceability of increment-level requirements between requirements planning documents, it will continue to risk confusion among stakeholders about what is to be delivered  In addition, until recently, GMM’s planning documents were missing up- to-date information regarding when most of the legacy systems will be transitioned to GMM  Specifically, while the program’s planning documents (including the GMM roadmap) provided key milestones for the entire lifecycle of the program and high-level capabilities to be delivered in the AFG pilot, these documents lacked up-to-date time frames for when FEMA planned to transition the nine remaining legacy systems  For example, in May 2017, GMM drafted notional time frames for transitioning the legacy systems, including plans for AFG to be the seventh system replaced by GMM  However, in December 2017, the program decided to reprioritize the legacy systems so that AFG would be replaced first—yet this major change was not reflected in the program’s roadmap  Moreover, while AFG program officials were informed of the decision to transition the AFG program first, in June 2018 officials from other grant programs told us that they had not been informed on when their systems were to be replaced  As a result, these programs were uncertain about when they should start planning for their respective transitions  In August 2018, GMM program officials acknowledged that they were delayed in deciding the sequencing order for the legacy system transitions  Program officials stated that the delay was due to their need to factor the Agile development contractor’s perspective into these decisions; yet, at that time, the contract award had been delayed by approximately 8 months  Subsequently, in October 2018, program officials identified tentative time frames for transitioning the remaining legacy systems  Program officials stated that they determined the tentative time frames for transitioning the legacy systems based on key factors, such as mission need, cost, security vulnerabilities, and technical obsolescence, and that they had shared these new time frames with grant program officials  The officials also stated that, once the Agile contractor begins contract performance, they expect to be able to validate the contractor’s capacity and finalize these time frames by obtaining approval from the Grants Management Executive Steering Group  By taking steps to update and communicate these important time frames, FEMA should be better positioned to ensure that each of the grant programs are prepared for transitioning to GMM  According to leading practices, Agile programs should track their progress in delivering planned IT requirements within a sprint (i e , short iterations that produce working software)  Given that sprints are very short cycles of development (e g , 2 weeks), the efficiency of completing planned work within a sprint relies on a disciplined approach that includes using a fixed pace, referred to as the sprint cadence, that provides a consistent and predictable development routine  A disciplined approach also includes identifying by the start of a sprint which user stories will be developed, developing those stories to completion (e g , fully tested and demonstrated to, and accepted by, the product owner), and tracking completion progress of those stories  Progress should be communicated to relevant stakeholders and used by the development teams to better understand their capacity to develop stories, continuously improve on their processes, and forecast how long it will take to deliver all remaining capabilities  The GMM program did not effectively track progress in delivering IT requirements during the first nine sprints, which occurred from January to June 2018  These gaps in tracking the progress of requirements, in part, had an impact on the program’s progress in delivering the 19 AFG business functions that were originally planned by December 2018 and are now deferred to August 2019  However, beginning in July 2018, in response to our ongoing review, the program took steps to improve in these areas  Specifically, GMM did not communicate the status of its Agile development progress to program stakeholders, such as the grant programs, the regional offices, and the development teams, during most of the first nine sprints  Program officials acknowledged that they should use metrics to track development progress and, in July 2018, they began reporting metrics to program stakeholders  For example, they began collecting and providing data on the number of stories planned and delivered, estimated capacity for development teams, and the number of days spent working on the sprint, as part of the program’s weekly status reports to program stakeholders, such as product owners  Rather than using a fixed, predictable sprint cadence, GMM allowed a variable development cadence, meaning that sprint durations varied from 1 to 4 weeks throughout the first nine sprints  Program officials noted that they had experimented with the use of a variable cadence to allow more time to complete complex technical work  Program officials stated that they realized that varying the sprints was not effective and, in July 2018 for sprint 10, they reverted back to a fixed, 2 week cadence  GMM added a significant amount of scope during its first nine sprints, after the development work had already begun  For example, the program committed to 28 user stories at the beginning of sprint eight, and then nearly doubled the work by adding 25 additional stories in the middle of the sprint  Program officials cited multiple reasons for adding more stories, including that an insufficient number of stories had been defined in the backlog when the sprint began, the realization that planned stories were too large and needed to be decomposed into smaller stories, and the realization that other work would be needed in addition to what was originally planned  Program officials recognized that, by the start of a sprint, the requirements should be sufficiently defined, such that they are ready for development without requiring major changes during the sprint  The program made recent improvements in sprints 11 and 12, which had only five stories added after the start of a sprint  By taking these steps to establish consistency among sprints, the program has better positioned itself to more effectively monitor and manage the remaining IT development work  In addition, this improvement in consistency should help the program avoid future deferments of functionality  Leading practices state that programs should regularly collaborate with, and collect input from, relevant stakeholders; monitor the status of stakeholder involvement; incorporate stakeholder input; and measure how well stakeholders’ needs are being met  For Agile programs, it is especially important to track user satisfaction to determine how well the program has met stakeholders’ needs  Consistent stakeholder participation ensures that the program meets its stakeholders’ needs  FEMA implemented its responsibilities in this area through several means, such as stakeholder outreach activities; development of a strategic communications plan; and continuous monitoring, solicitation, and recording of stakeholder involvement and feedback  For example, the agency conducted nationwide outreach sessions from January 2016 through August 2017 and began conducting additional outreach sessions in April 2018  These outreach sessions involved hundreds of representatives from FEMA headquarters, the 10 regional offices, and state and local grant recipients to collect information on the current grants management environment and opportunities for streamlining grants management processes  FEMA also held oversight and stakeholder outreach activities and actively solicited and recorded feedback from its stakeholders on a regular basis  For example, GMM regularly verified with users that the new functionality met their IT requirements, as part of the Agile development cycle  Additionally, we observed several GMM biweekly requirements validation sessions where the program’s stakeholders were involved and provided feedback as part of the requirements development and refinement process  In addition, FEMA identified GMM stakeholders and tracked its engagement with these stakeholders using a stakeholder register  The agency also defined processes for how the GMM program is to collaborate with its stakeholders in a stakeholder communication plan and Agile development team agreement  Also, while several officials from the selected grant program and regional offices that we interviewed indicated that the program could improve in communicating its plans for GMM and incorporating stakeholder input, most of the representatives from these offices stated that GMM is doing well at interacting with its stakeholders  Finally, in October 2018, program officials reported that they had recently begun measuring user satisfaction by conducting surveys and interviews with users that have utilized the new functionality within GMM  The program’s outreach activities, collection of stakeholder input, and measurement of user satisfaction demonstrate that the program is taking the appropriate steps to incorporate stakeholder input  Reliable cost estimates are critical for successfully delivering IT programs  Such estimates provide the basis for informed decision making, realistic budget formulation, meaningful progress measurement, and accountability for results  GAO’s Cost Estimating and Assessment Guide defines leading practices related to the following four characteristics of a high-quality, reliable estimate  Comprehensive

ground_truth: Safety incidents in recent years on WMATA's rail system have raised questions about its processes for performing critical maintenance and replacing capital assets. WMATA initiated a new preventive maintenance program for its rail track in 2017, and is currently implementing a new capital planning process. GAO was asked to examine issues related to WMATA's capital funding and maintenance practices. This report examines: (1) how WMATA spent its capital funds from fiscal years 2011 through 2017, (2) how WMATA's new capital planning process addresses weaknesses it identified in the prior process, and (3) WMATA's progress toward its track preventive maintenance program's goals and how the program aligns with leading program management practices. GAO analyzed WMATA's financial and program information, interviewed officials of WMATA, the Federal Transit Administration, and five transit agencies selected for similarities to WMATA. GAO compared WMATA's capital planning process and track maintenance program with leading practices. From fiscal years 2011 through 2017, the Washington Metropolitan Area Transit Authority (WMATA) spent almost $6 billion on a variety of capital assets, with the largest share spent on improving its rail and bus fleet (see figure). Over this period, WMATA's capital spending was, on average, about $845 million annually. WMATA's new capital planning process could address some weaknesses it identified in the prior process. WMATA established a framework for quantitatively prioritizing capital needs (investments to a group of related assets) over a 10-year period. However, WMATA has not established documented policies and procedures for implementing the new process, such as those for selecting specific projects for funding in its annual capital budget. WMATA is currently using its new capital planning process to make fiscal year 2020 investment decisions. WMATA has proposed a fiscal year 2020 capital budget of $1.4 billion. Without documented policies and procedures for implementing the new planning process, WMATA's stakeholders do not have reasonable assurance that WMATA is following a sound process for making investment decisions. WMATA has made significant progress toward its track preventive maintenance program's goals, which are to reduce both track-defect and electrical-fire incidents by 50 percent in fiscal year 2019 compared with 2017. In fiscal year 2018, WMATA met its goal for reducing track defect incidents and reduced electrical fire incidents by 20 percent. However, in designing the program, WMATA did not fully assess risks. For example, WMATA did not quantitatively assess the impact of track defects or electrical fires on its ability to provide service, nor did it consider other risks such as non-electrical track fires, which represent about 30 percent of all fires on the system, or other factors, such as resources or track time. Without a comprehensive risk assessment, WMATA lacks reasonable assurance that the program is designed to address risks affecting the safety of the rail system or other risks that could hinder the new program's success. GAO is making five recommendations, including that WMATA establish documented policies and procedures for the new capital planning process and conduct a comprehensive risk assessment for the track preventive maintenance program. WMATA described actions planned or underway to address GAO's recommendations. GAO believes the recommendations should be fully implemented, as discussed in the report.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.



Report 2:  The estimate accounts for all possible costs associated with a program, is structured in sufficient detail to ensure that costs are neither omitted nor double counted, and documents all cost-influencing assumptions  Well-documented  Supporting documentation explains the process, sources, and methods used to create the estimate; contains the underlying data used to develop the estimate; and is adequately reviewed and approved by management  Accurate  The estimate is not overly conservative or optimistic, is based on an assessment of the costs most likely to be incurred, and is regularly updated so that it always reflects the program’s current status  Credible  Discusses any limitations of the analysis because of uncertainty or sensitivity surrounding data or assumptions, the estimate’s results are cross-checked, and an independent cost estimate is conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results  In May 2017, DHS approved GMM’s lifecycle cost estimate of about $251 million for fiscal years 2015 through 2030  We found this initial estimate to be reliable because it fully or substantially addressed all the characteristics associated with a reliable cost estimate  For example, the estimate comprehensively included government and contractor costs, all elements of the program’s work breakdown structure, and all phases of the system lifecycle; and was aligned with the program’s technical documentation at the time the estimate was developed  GMM also fully documented the key assumptions, data sources, estimating methodology, and calculations for the estimate  Further, the program conducted a risk assessment and sensitivity analysis, and DHS conducted an independent assessment of the cost estimate to validate the accuracy and credibility of the cost estimate  However, key assumptions that FEMA made about the program changed soon after DHS approved the cost estimate in May 2017  Thus, the initial cost estimate no longer reflects the current approach for the program  For example, key assumptions about the program that changed include: Change in the technical approach: The initial cost estimate assumed that GMM would implement a software-as-a-service model, meaning that FEMA would rely on a service provider to deliver software applications and the underlying infrastructure to run them  However, in December 2017, the program instead decided to implement an infrastructure-as-a-service model, meaning that FEMA would develop and deploy its own software application and rely on a service provider to deliver and manage the computing infrastructure (e g , servers, software, storage, and network equipment)  According to program officials, this decision was made after learning from the Agile prototypes that the infrastructure-as-a-service model would allow GMM to develop the system in a more flexible environment  Increase in the number of system development personnel: A key factor with Agile development is the number of development teams (each consisting of experts in software development, testing, and cybersecurity) that are operating concurrently and producing separate portions of software functionality  Program officials initially assumed that they would need three to four concurrent Agile development teams, but subsequently realized that they would instead need to expend more resources to achieve GMM’s original completion date  Specifically, program officials now expect they will need to at least double, and potentially triple, the number of concurrent development teams to meet GMM’s original target dates  Significant delays and complexities with data migration: In 2016 and 2017, GMM experienced various technical challenges in its effort to transfer legacy system data to a data staging platform  This data transfer effort needed to be done to standardize the data before eventually migrating the data to GMM  These challenges resulted in significant delays and cost increases  Program officials reported that, by February 2018—at least 9 months later than planned—all legacy data had been transferred to a data staging platform so that FEMA officials could begin analyzing and standardizing the data prior to migrating it into GMM  FEMA officials reported that they anticipated the cost estimate to increase, and for this increase to be high enough to breach the $251 million threshold set in GMM’s May 2017 acquisition program baseline  Thus, consistent with DHS’s acquisition guidance, the program informed the DHS acquisition review board of this anticipated breach  The board declared that the program was in a cost breach status, as of September 12, 2018  As of October 2018, program officials stated that they were in the process of revising the cost estimate to reflect the changes in the program and to incorporate actual costs  In addition, the officials stated that the program was applying a new cost estimating methodology tailored for Agile programs that DHS’s Cost Analysis Division had been developing  In December 2018, program officials stated that they had completed the revised cost estimate but it was still undergoing departmental approval  Establishing an updated cost estimate should help FEMA better understand the expected costs to deliver GMM under the program’s current approach and time frames  The success of an IT program depends, in part, on having an integrated and reliable master schedule that defines when the program’s set of work activities and milestone events are to occur, how long they will take, and how they are related to one another  Among other things, a reliable schedule provides a roadmap for systematic execution of an IT program and the means by which to gauge progress, identify and address potential problems, and promote accountability  GAO’s Schedule Assessment Guide defines leading practices related to the following four characteristics that are vital to having a reliable integrated master schedule  Comprehensive  A comprehensive schedule reflects all activities for both the government and its contractors that are necessary to accomplish a program’s objectives, as defined in the program’s work breakdown structure  The schedule also includes the labor, materials, and overhead needed to do the work and depicts when those resources are needed and when they will be available  It realistically reflects how long each activity will take and allows for discrete progress measurement  Well-constructed  A schedule is well-constructed if all of its activities are logically sequenced with the most straightforward logic possible  Unusual or complicated logic techniques are used judiciously and justified in the schedule documentation  The schedule’s critical path represents a true model of the activities that drive the program’s earliest completion date and total float accurately depicts schedule flexibility  Credible  A schedule that is credible is horizontally traceable—that is, it reflects the order of events necessary to achieve aggregated products or outcomes  It is also vertically traceable—that is, activities in varying levels of the schedule map to one another and key dates presented to management in periodic briefings are consistent with the schedule  Data about risks are used to predict a level of confidence in meeting the program’s completion date  The level of necessary schedule contingency and high-priority risks are identified by conducting a robust schedule risk analysis  Controlled  A schedule is controlled if it is updated regularly by trained schedulers using actual progress and logic to realistically forecast dates for program activities  It is compared to a designated baseline schedule to measure, monitor, and report the program’s progress  The baseline schedule is accompanied by a baseline document that explains the overall approach to the program, defines ground rules and assumptions, and describes the unique features of the schedule  The baseline schedule and current schedule are subject to a configuration management control process  GMM’s schedule was unreliable because it minimally addressed three characteristics—comprehensive, credible, and controlled—and did not address the fourth characteristic of a reliable estimate—well-constructed  One of the most significant issues was that the program’s fast approaching, final delivery date of September 2020 was not informed by a realistic assessment of GMM development activities, and rather was determined by imposing an unsubstantiated delivery date  Table 4 summarizes our assessment of GMM’s schedule  In discussing the reasons for the shortfalls in these practices, program officials stated that they had been uncertain about the level of rigor that should be applied to the GMM schedule, given their use of Agile development  However, leading practices state that program schedules should meet all the scheduling practices, regardless of whether a program is using Agile development  As discussed earlier in this report, GMM has already experienced significant schedule delays  For example, the legacy data migration effort, the AFG pilot, and the Agile development contract have been delayed  Program officials also stated that the delay in awarding and starting the Agile contract has delayed other important activities, such as establishing time frames for transitioning legacy systems  A more robust schedule could have helped FEMA predict the impact of delays on remaining activities and identify which activities appeared most critical so that the program could ensure that any risks in delaying those activities were properly mitigated  In response to our review and findings, program officials recognized the need to continually enhance their schedule practices to improve the management and communication of program activities  As a result, in August 2018, the officials stated that they planned to add a master scheduler to the team to improve the program’s schedule practices and ensure that all of the areas of concern we identified are adequately addressed  In October 2018, the officials reported that they had recently added two master schedulers to GMM  According to the statement of objectives, the Agile contractor is expected to develop an integrated master schedule soon after it begins performance  However, program officials stated that GMM is schedule-driven—due to the Executive Steering Group’s expectation that the solution will be delivered by September 2020  The officials added that, if GMM encounters challenges in meeting this time frame, the program plans to seek additional resources to allow it to meet the 2020 target  GMM’s schedule-driven approach has already led to an increase in estimated costs and resources  For example, as previously mentioned, the program has determined that, to meet its original target dates, GMM needs to at least double, and possibly triple, the number of concurrent Agile development teams  In addition, we have previously reported that schedule pressure on federal IT programs can lead to omissions and skipping of key activities, especially system testing  In August 2018, program officials acknowledged that September 2020 may not be feasible and that the overall completion time frames established in the acquisition program baseline may eventually need to be rebaselined  Without a robust schedule to forecast whether FEMA’s aggressive delivery goal for GMM is realistic to achieve, leadership will be limited in its ability to make informed decisions on what additional increases in cost or reductions in scope might be needed to fully deliver the system  NIST’s risk management framework establishes standards and guidelines for agencies to follow in developing cybersecurity programs  Agencies are expected to use this framework to achieve more secure information and information systems through the implementation of appropriate risk mitigation strategies and by performing activities that ensure that necessary security controls are integrated into agencies’ processes  The framework addresses broad cybersecurity and risk management activities, which include the following: Categorize the system: Programs are to categorize systems by identifying the types of information used, selecting a potential impact level (e g , low, moderate, or high), and assigning a category based on the highest level of impact to the system’s confidentiality, integrity, and availability, if the system was compromised  Programs are also to document a description of the information system and its boundaries and should register the system with appropriate program management offices  System categorization is documented in a system security plan  Select and implement security controls: Programs are to determine protective measures, or security controls, to be implemented based on the system categorization results  These security controls are documented in a system security plan  For example, control areas include access controls, incident response, security assessment and authorization, identification and authentication, and configuration management  Once controls are identified, programs are to determine planned implementation actions for each of the designated controls  These implementation actions are also specified in the system security plan  Assess security controls: Programs are to develop, review, and approve a security assessment plan  The purpose of the security assessment plan approval is to establish the appropriate expectations for the security control assessment  Programs are to also perform a security control assessment by evaluating the security controls in accordance with the procedures defined in the security assessment plan, in order to determine the extent to which the controls were implemented correctly  The output of this process is intended to produce a security assessment report to document the issues, findings, and recommendations  Programs are to conduct initial remediation actions on security controls and reassess those security controls, as appropriate  Obtain an authorization to operate the system: Programs are to obtain security authorization approval in order to operate a system  Resolving weaknesses and vulnerabilities identified during testing is an important step leading up to achieving an authorization to operate  Programs are to establish corrective action plans to address any deficiencies in cybersecurity policies, procedures, and practices  DHS guidance also states that corrective action plans must be developed for every weakness identified during a security control assessment and within a security assessment report  Monitor security controls on an ongoing basis: Programs are to monitor their security controls on an ongoing basis after deployment, including determining the security impact of proposed or actual changes to the information system and assessing the security controls in accordance with a monitoring strategy that determines the frequency of monitoring the controls  For the GMM program’s engineering and test environment, which went live in February 2018, FEMA fully addressed three of the five key cybersecurity practices in NIST’s risk management framework and partially addressed two of the practices  Specifically, FEMA categorized GMM’s environment based on security risk, implemented select security controls, and monitored security controls on an ongoing basis  However, the agency partially addressed the areas of assessing security controls and obtaining an authorization to operate the system  Table 5 provides a summary of the extent to which FEMA addressed NIST’s key cybersecurity practices for GMM’s engineering and test environment  Consistent with NIST’s framework, GMM categorized the security risk of its engineering and test environment and identified it as a moderate- impact environment  A moderate-impact environment is one where the loss of confidentiality, integrity, or availability could be expected to have a serious or adverse effect on organizational operations, organizational assets, or individuals  GMM completed the following steps leading to this categorization: The program documented in its System Security Plan the various types of data and information that the environment will collect, process, and store, such as conducting technology research, building or enhancing technology, and maintaining IT networks  The program established three information types and assigned security levels of low, moderate, or high impact in the areas of confidentiality, availability, and integrity  A low-impact security level was assigned to two information types: (1) conducting technology research and (2) building or enhancing technology; and a moderate- impact security level was assigned to the third information type: maintaining IT networks  The engineering and test environment was categorized as an overall moderate-impact system, based on the highest security impact level assignment  GMM documented a description of the environment, including a diagram depicting the system’s boundaries, which illustrates, among other things, databases and firewalls  GMM properly registered its engineering and test environment with FEMA’s Chief Information Officer, Chief Financial Officer, and acting Chief Information Security Officer  By conducting the security categorization process, GMM has taken steps that should ensure that the appropriate security controls are selected for the program’s engineering and test environment  Consistent with NIST’s framework and the system categorization results, GMM appropriately determined which security controls to implement and planned actions for implementing those controls in its System Security Plan for the engineering and test environment  For example, the program utilized NIST guidance to select standard controls for a system categorized with a moderate-impact security level  These control areas include, for example, access controls, risk assessment, incident response, identification and authentication, and configuration management  Further, the program documented its planned actions to implement each control in its System Security Plan  For example, GMM documented that the program plans to implement its Incident Response Testing control by participating in an agency-wide exercise and unannounced vulnerability scans  As another example, GMM documented that the program plans to implement its Contingency Plan Testing control by testing the contingency plan annually, reviewing the test results, and preparing after action reports  By selecting and planning for the implementation of security controls, GMM has taken steps to mitigate its security risks and protect the confidentiality, integrity, and availability of the information system  Consistent with NIST’s framework, in January 2018, GMM program officials developed a security assessment plan for the engineering and test environment  According to GMM program officials, this plan was reviewed by the security assessment team  However, the security assessment plan lacked essential details  Specifically, while the plan included the general process for evaluating the environment’s security controls, the planned assessment procedures for all 964 security controls were not sufficiently defined  Specifically, GMM program officials copied example assessment procedures from NIST guidance and inserted them into its security assessment documentation for all of its 964 controls, without making further adjustments to explain the steps that should be taken specific to GMM  Table 6 shows an example of a security assessment procedure copied from the NIST guidance that should have been further adjusted for GMM  In addition, the actual assessment procedures that the GMM assessors used to evaluate the security controls were not documented  Instead, the program only documented whether each control passed or failed each test  GMM program officials stated that the planned assessment procedures are based on an agency template that was exported from a DHS compliance tool, and that FEMA security officials have been instructed by the DHS OCIO not to tailor or make any adjustments to the template language  However, the assessment procedures outlined in NIST’s guidance are to serve as a starting point for organizations preparing their program specific assessments  According to NIST, organizations are expected to select and tailor their assessment procedures for each security control from NIST’s list of suggested assessment options (e g , review, analyze, or inspect policies, procedures, and related documentation options)  DHS OCIO officials stated that, consistent with NIST’s guidance, they expect that components will ensure they are in compliance with the minimum standards and will also add details and additional rigor, as appropriate, to tailor the planned security assessment procedures to fit their unique missions or needs  In November 2018, in response to our audit, DHS OCIO officials stated that they were meeting with FEMA OCIO officials to understand why they did not document the planned and actual assessment procedures performed by the assessors for GMM  Until FEMA ensures that detailed planned evaluation methods and actual evaluation procedures specific to GMM are defined, the program risks assessing security controls incorrectly, having controls that do not work as intended, and producing undesirable outcomes with respect to meeting the security requirements  In addition, the security assessment plan was not approved by FEMA’s OCIO before proceeding with the security assessment  Program officials stated that approval was not required for the security assessment plan prior to the development of the security assessment report  However, NIST guidance states that the purpose of the security assessment plan approval is to establish the appropriate expectations for the security control assessment  By not getting the security assessment plan approved by FEMA’s OCIO before security assessment reviews were conducted, GMM risks inconsistencies with the plan and security objectives of the organization  Finally, consistent with NIST guidance, GMM performed a security assessment in December 2017 of the engineering and test environment’s controls, which identified 36 vulnerabilities (23 critical- and high-impact vulnerabilities and 13 medium- and low-impact vulnerabilities)  The program also documented these vulnerabilities and associated findings and recommendations in a security assessment report  GMM conducted initial remediation actions (i e , remediation of vulnerabilities that should be corrected immediately) for 12 of the critical- and high-impact vulnerabilities and a reassessment of those security controls confirmed that they were resolved by January 2018  Remediation of the remaining 11 critical- and high-impact vulnerabilities and 13 medium- and low- impact vulnerabilities were to be addressed by corrective action plans as part of the authorization to operate process, which is discussed in the next section  The authorization to operate GMM’s engineering and test environment was granted on February 5, 2018  Among other things, this decision was based on the important stipulation that the remaining 11 critical- and high- impact vulnerabilities associated with multifactor authentication would be addressed within 45 days, or by March 22, 2018  However, the program did not meet this deadline and, instead, approximately 2 months after this deadline passed, obtained a waiver to remediate these vulnerabilities by May 9, 2019  These vulnerabilities are related to a multifactor authentication capability  Program officials stated that they worked with FEMA OCIO officials to attempt to address these vulnerabilities by the initial deadline, but they were unsuccessful in finding a viable solution  Therefore, GMM program officials developed a waiver at the recommendation of the OCIO to provide additional time to develop a viable solution  However, a multifactor authentication capability is essential to ensuring that users are who they say they are, prior to granting users access to the GMM engineering and test environment, in order to reduce the risk of harmful actors accessing the system  In addition, as of September 2018, the program had not established corrective action plans for the 13 medium- and low-impact vulnerabilities  Program officials stated that they do not typically address low-impact vulnerabilities; however, this is in conflict with DHS guidance that specifies that corrective action plans must be developed for every weakness identified during a security control assessment and within a security assessment report  In response to our audit, in October 2018, GMM program officials developed these remaining corrective action plans  The plans indicated that these vulnerabilities were to be fully addressed by January 2019 and April 2019  While the program eventually took corrective actions in response to our audit by developing the missing plans, the GMM program initially failed to follow DHS’s guidance on preparing corrective actions plans for all security vulnerabilities  Until GMM consistently follows DHS’s guidance, it will be difficult for FEMA to determine the extent to which GMM’s security weaknesses identified during its security control assessments are remediated  Additionally, as we have reported at other agencies, vulnerabilities can be indicators of more significant underlying issues and, thus, without appropriate management attention or prompt remediation, GMM is at risk of unnecessarily exposing the program to potential exploits  Moreover, GMM was required to assess all untested controls by March 7, 2018, or no later than 30 days after the approval of the authorization to operate; however, it did not meet this deadline  Specifically, we found that, by October 2018, FEMA had not fully tested 190 security controls in the GMM engineering and test environment  These controls were related to areas such as security incident handling and allocation of resources required to protect an information system  In response to our findings, in October 2018, GMM program officials reported that they had since fully tested 27 controls and partially tested the remaining 163 controls  Program officials stated that testing of the 163 controls is a shared responsibility between GMM and other parties (e g , the cloud service provider)  They added that GMM had completed its portion of the testing but was in the process of verifying the completion of testing by other parties  Program officials stated that the untested controls were not addressed sooner, in part, because of errors resulting from configuration changes in the program’s compliance tool during a system upgrade, which have now been resolved  Until GMM ensures that all security controls have been tested, it remains at an increased risk of exposing programs to potential exploits  Consistent with the NIST framework, GMM established methods for assessing and monitoring security controls to be conducted after an authorization to operate has been approved  GMM has tailored its cybersecurity policies and practices for monitoring its controls to take into account the frequent and iterative pace with which system functionality is continuously being introduced into the GMM environment  Specifically, the GMM program established a process for assessing security impact changes to the system and conducting reauthorizations to operate within the rapid Agile delivery environment  As part of this process, GMM embedded cybersecurity experts on each Agile development team so that they are involved early and can impact security considerations from the beginning of requirements development through testing and deployment of system functionality  In addition, the process involves important steps for ensuring that the system moves from development to completion, while producing a secure and reliable system  For example, it includes procedures for creating, reviewing, and testing new system functionality  As the new system functionality is integrated with existing system functionality, it is to undergo automated testing and security scans in order to ensure that the integrity of the security of the system has not been compromised  Further, an automated process is to deploy the code if it passes all security scans, code tests, and code quality checks  GMM’s process for conducting a reauthorization to operate within the rapid delivery Agile development environment is to follow FEMA guidance that states that all high-level changes made to a FEMA IT system must receive approval from both a change advisory board and the FEMA Chief Information Officer  The board and FEMA Chief Information Officer are to focus their review and approval on scheduled releases and epics (i e , collections of user stories)  Additionally, the Information System Security Officer is to review each planned user story and, if it is determined that the proposed changes may impact the integrity of the authorization, the Information System Security Officer is to work with the development team to begin the process of updating the system authorization  Finally, GMM uses automated tools to track the frequency in which security controls are assessed and to ensure that required scanning data are received by FEMA for reporting purposes  Program officials stated that, in the absence of department-level and agency-level guidance, they have coordinated with DHS and FEMA OCIO officials to ensure that these officials are in agreement with GMM’s approach to continuous monitoring  By having monitoring control policies and procedures in place, FEMA management is positioned to more effectively prioritize and plan its risk response to current threats and vulnerabilities for the GMM program  Given FEMA’s highly complex grants management environment, with its many stakeholders, IT systems, and internal and external users, implementing leading practices for business process reengineering and IT requirements management is critical for success  FEMA has taken many positive steps, including ensuring executive leadership support for business process reengineering, documenting the agency’s grants management processes and performance improvement goals, defining initial IT requirements for the program, incorporating input from end user stakeholders into the development and implementation process, and taking recent actions to improve its delivery of planned IT requirements  Nevertheless, until the GMM program finalizes plans and time frames for implementing its organizational change management actions, plans and communicates system transition activities, and maintains clear traceability of IT requirements, FEMA will be limited in its ability to provide streamlined grants management processes and effectively deliver a modernized IT system to meet the needs of its large range of users  While GMM’s initial cost estimate was reliable, key assumptions about the program since the initial estimate had changed and, therefore, it no longer reflected the current approach for the program  The forthcoming updated cost schedule is expected to better reflect the current approach  However, the program’s unreliable schedule to fully deliver GMM by September 2020 is aggressive and unrealistic  The delays the program has experienced to date further compound GMM’s schedule issues  Without a robust schedule that has been informed by a realistic assessment of GMM’s development activities, leadership will be limited in its ability to make informed decisions on what additional increases in cost or reductions in scope might be needed to achieve their goals  Further, FEMA’s implementation of cybersecurity practices for GMM in the areas of system categorization, selection and implementation, and monitoring will help the program  However, GMM lacked essential details for evaluating security controls, did not approve the security assessment plan before proceeding with the security assessment, did not follow DHS’s guidance to develop corrective action plans for all security vulnerabilities, and did not fully test all security controls  As a result, the GMM engineering and test environment remains at an increased risk of exploitations  We are making eight recommendations to FEMA: The FEMA Administrator should ensure that the GMM program management office finalizes the organizational change management plan and time frames for implementing change management actions  (Recommendation 1) The FEMA Administrator should ensure that the GMM program management office plans and communicates its detailed transition activities to its affected customers before they transition to GMM and undergo significant changes to their processes  (Recommendation 2) The FEMA Administrator should ensure that the GMM program management office implements its planned changes to its processes for documenting requirements for future increments and ensures it maintains traceability among key IT requirements documents  (Recommendation 3) The FEMA Administrator should ensure that the GMM program management office updates the program schedule to address the leading practices for a reliable schedule identified in this report  (Recommendation 4) The FEMA Administrator should ensure that the FEMA OCIO defines sufficiently detailed planned evaluation methods and actual evaluation methods for assessing security controls  (Recommendation 5) The FEMA Administrator should ensure that the FEMA OCIO approves a security assessment plan before security assessment reviews are conducted  (Recommendation 6) The FEMA Administrator should ensure that the GMM program management office follows DHS guidance on preparing corrective action plans for all security vulnerabilities  (Recommendation 7) The FEMA Administrator should ensure that the GMM program management office fully tests all of its security controls for the system  (Recommendation 8) DHS provided written comments on a draft of this report, which are reprinted in appendix IV  In its comments, the department concurred with all eight of our recommendations and provided estimated completion dates for implementing each of them  For example, with regard to recommendation 4, the department stated that FEMA plans to update the GMM program schedule to address the leading practices for a reliable schedule by April 30, 2019  In addition, for recommendation 7, the department stated that FEMA plans to ensure that corrective action plans are prepared by July 31, 2019, to address all identified security vulnerabilities for GMM  If implemented effectively, the actions that FEMA plans to take in response to the recommendations should address the weaknesses we identified  We also received technical comments from DHS and FEMA officials, which we incorporated, as appropriate  We are sending copies of this report to the Secretary of Homeland Security and interested congressional committees  In addition, the report will be available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-4456 or harriscc@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix V  Our objectives were to (1) determine the extent to which the Federal Emergency Management Agency (FEMA) is implementing leading practices for reengineering its grants management business processes and incorporating business needs into Grants Management Modernization (GMM) information technology (IT) requirements; (2) assess the reliability of the program’s estimated costs and schedule; and (3) determine the extent to which FEMA is addressing key cybersecurity practices for GMM  To address the first objective, we reviewed GAO’s Business Process Reengineering Assessment Guide and Software Engineering Institute’s Capability Maturity Model for Integration for Development to identify practices associated with business process reengineering and IT requirements management  We then selected six areas that, in our professional judgment, represented foundational practices that were of particular importance to the successful implementation of an IT modernization effort that is using Agile development processes  We also selected the practices that were most relevant based on where GMM was in the system development lifecycle and we discussed the practice areas with FEMA officials  The practices are: Ensuring executive leadership support for process reengineering Assessing the current and target business environment and business Establishing plans for implementing new business processes Establishing clear, prioritized, and traceable IT requirements Tracking progress in delivering IT requirements Incorporating input from end user stakeholders We also reviewed selected chapters of GAO’s draft Agile Assessment Guide (Version 6A), which is intended to establish a consistent framework based on best practices that can be used across the federal government for developing, implementing, managing, and evaluating agencies’ IT investments that rely on Agile methods  To develop this guide, GAO worked closely with Agile experts in the public and private sector; some chapters of the guide are considered more mature because they have been reviewed by the expert panel  We reviewed these chapters to ensure that our expectations for how FEMA should apply the six practices for business process reengineering and IT requirements management are appropriate for an Agile program and are consistent with the draft guidance that is under development  Additionally, since Agile development programs may use different terminology to describe their software development processes, the Agile terms used in this report (e g , increment, sprint, epic, etc ) are specific to the GMM program  We obtained and analyzed FEMA grants management modernization documentation, such as current and target grants management business processes, acquisition program baseline, operational requirements document, concept of operations, requirements analyses workbooks, Grants Management Executive Steering Group artifacts, stakeholder outreach artifacts, Agile increment- and sprint-level planning and development artifacts, and the requirements backlog  We assessed the program documentation against the selected practices to determine the extent to which the agency had implemented them  We then assessed each practice area as: fully implemented—FEMA provided complete evidence that showed it fully implemented the practice area; partially implemented—FEMA provided evidence that showed it partially implemented the practice area; not implemented—FEMA did not provide evidence that showed it implemented any of the practice area  Additionally, we observed Agile increment and sprint development activities at GMM facilities in Washington, D C  We also observed a demonstration of how the program manages its lower level requirements (i e , user stories and epics) and maintains traceability of the requirements using an automated tool at GMM facilities in Washington, D C  We also interviewed FEMA officials, including the GMM Program Executive, GMM Program Manager, GMM Business Transformation Team Lead, and Product Owner regarding their efforts to streamline grants management business processes, collect and incorporate stakeholder input, and manage GMM’s requirements  In addition, we interviewed FEMA officials from four out of 16 grant program offices and two out of 10 regional offices to obtain contextual information and illustrative examples of FEMA’s efforts to reengineer grants management business processes and collect business requirements for GMM  Specifically, We selected the four grant program offices based on a range of grant programs managed, legacy systems used, and the amount of grant funding awarded  We also sought to select a cross section of different characteristics, such as selecting larger grant program offices, as well as smaller offices  In addition, we ensured that our selection included the Assistance to Firefighters Grants (AFG) program office because officials in this office represent the first GMM users and, therefore, are more actively involved with the program’s Agile development practices  Based on these factors, we selected: Public Assistance Division, Individual Assistance Division, AFG, and National Fire Academy  Additionally, the four selected grant program offices are responsible for 16 of the total 45 grant programs and are users of five of the nine primary legacy IT systems  The four selected grant program offices also represent about 68 percent of the total grant funding awarded by FEMA from fiscal years 2005 through 2016  We selected two regional offices based on (1) the largest amount of total FEMA grant funding for fiscal years 2005 through 2016—Region 6 located in Denton, Texas; and (2) the highest percentage of AFG funding compared to the office’s total grant funding awarded from fiscal years 2005 through 2016—Region 5 located in Chicago, Illinois  To assess the reliability of data from the program’s automated IT requirements management tool, we interviewed knowledgeable officials about the quality control procedures used by the program to assure accuracy and completeness of the data  We also compared the data to other relevant program documentation on GMM requirements  We determined that the data used were sufficiently reliable for the purpose of evaluating GMM’s practices for managing IT requirements  For our second objective, to assess the reliability of GMM’s estimated costs and schedule, we reviewed documentation on GMM’s May 2017 lifecycle cost estimate and on the program’s schedule, dated May 2018  To assess the reliability of the May 2017 lifecycle cost estimate, we evaluated documentation supporting the estimate, such as the cost estimating model, the report on GMM’s Cost Estimating Baseline Document and Life Cycle Cost Estimate, and briefings provided to the Department of Homeland Security (DHS) and FEMA management regarding the cost estimate  We assessed the cost estimating methodologies, assumptions, and results against leading practices for developing a comprehensive, accurate, well-documented, and credible cost estimate, identified in GAO’s Cost Estimating and Assessment Guide  We also interviewed program officials responsible for developing and reviewing the cost estimate to understand their methodology, data, and approach for developing the estimate  We found that the cost data were sufficiently reliable  To assess the reliability of the May 2018 GMM program schedule, we evaluated documentation supporting the schedule, such as the integrated master schedule, acquisition program baseline, and Agile artifacts  We assessed the schedule documentation against leading practices for developing a comprehensive, well-constructed, credible, and controlled schedule, identified in GAO’s Schedule Assessment Guide  We also interviewed GMM program officials responsible for developing and managing the program schedule to understand their practices for creating and maintaining the schedule  We noted in our report the instances where the quality of the schedule data impacted the reliability of the program’s schedule  For both the cost estimate and program schedule, we assessed each leading practice as: fully addressed—FEMA provided complete evidence that showed it implemented the entire practice area; substantially addressed—FEMA provided evidence that showed it implemented more than half of the practice area; partially addressed—FEMA provided evidence that showed it implemented about half of the practice area; minimally addressed—FEMA provided evidence that showed it implemented less than half of the practice area; not addressed—FEMA did not provide evidence that showed it implemented any of the practice area  Finally, we provided FEMA with draft versions of our detailed analyses of the GMM cost estimate and schedule  This was done to verify that the information on which we based our findings was complete, accurate, and up-to-date  Regarding our third objective, to determine the extent to which FEMA is addressing key cybersecurity practices for GMM, we reviewed documentation regarding DHS and FEMA cybersecurity policies and guidance, and FEMA’s authorization to operate for the program’s engineering and test environment  We evaluated the documentation against all six cybersecurity practices identified in the National Institute of Standards and Technology’s (NIST) Risk Management Framework  While NIST’s Risk Management Framework identifies six total practices, for reporting purposes, we combined two interrelated practices—selection of security controls and implementation of security controls—into a single practice  The resulting five practices were: categorizing the system based on security risk, selecting and implementing security controls, assessing security controls, obtaining an authorization to operate the system, and monitoring security controls on an ongoing basis  We obtained and analyzed key artifacts supporting the program’s efforts to address these risk management practices, including the program’s System Security Plan, the Security Assessment Plan and Report, Authorization to Operate documentation, and the program’s continuous monitoring documentation  We also interviewed officials from the GMM program office and FEMA’s Office of the Chief Information Officer, such as the GMM Security Engineering Lead, GMM Information System Security Officer, and FEMA’s Acting Chief Information Security Officer, regarding their efforts to assess, document, and review security controls for GMM  We assessed the evidence against the five practices to determine the extent to which the agency had addressed them  We then assessed each practice area as: fully addressed—FEMA provided complete evidence that showed it fully implemented the practice area; partially addressed—FEMA provided evidence that showed it partially implemented the practice area; not addressed—FEMA did not provide evidence that showed it implemented any of the practice area  To assess the reliability of data from the program’s automated security controls management tool, we interviewed knowledgeable officials about the quality control procedures used by the program to assure accuracy and completeness of the data  We also compared the data to other relevant program documentation on GMM security controls for the engineering and test environment  We found that some of the security controls data we examined were sufficiently reliable for the purpose of evaluating FEMA’s cybersecurity practices for GMM, and we noted in our report the instances where the accuracy of the data impacted the program’s ability to address key cybersecurity practices  We conducted this performance audit from December 2017 to April 2019 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  The Federal Emergency Management Agency (FEMA) awards many different types of grants to state, local, and tribal governments and nongovernmental entities  These grants are to help communities prevent, prepare for, protect against, mitigate the effects of, respond to, and recover from disasters and terrorist attacks  Agile software development is a type of incremental development that calls for the rapid delivery of software in small, short increments  The use of an incremental approach is consistent with the Office of Management and Budget’s guidance as specified in its information technology (IT) Reform Plan, as well as the legislation commonly referred to as the Federal Information Technology Acquisition Reform Act  Many organizations, especially in the federal government, are accustomed to using a waterfall software development model, which typically consists of long, sequential phases, and differs significantly from the Agile development approach  Agile practices integrate planning, design, development, and testing into an iterative lifecycle to deliver software early and often  Figure 7 provides a depiction of software development using the Agile approach, as compared to a waterfall approach  The frequent iterations of Agile development are intended to effectively measure progress, reduce technical and programmatic risk, and respond to feedback from stakeholders in changes to IT requirements more quickly than traditional methods  Despite these intended benefits, organizations adopting Agile must overcome challenges in making significant changes to how they are accustomed to developing software  The significant differences between Agile and waterfall development impact how IT programs are planned, implemented, and monitored in terms of cost, schedule, and scope  For example, in waterfall development, significant effort is devoted upfront to document detailed plans and all IT requirements for the entire scope of work at the beginning of the program, and cost and schedule can be varied to complete that work  However, for Agile programs the precise details are unknown upfront, so initial planning of cost, scope, and timing would be conducted at a high level, and then supplemented with more specific plans for each iteration  While cost and schedule are set for each iteration, requirements for each iteration (or increment) can be variable as they are learned over time and revised to reflect experiences from completed iterations and to accommodate changing priorities of the end users  The differences in these two software development approaches are shown in figure 8  Looking at figure 8, the benefit provided from using traditional program management practices such as establishing a cost estimate or a robust schedule, is not obvious  However, unlike a theoretical environment, many government programs may not have the autonomy to manage completely flexible scope, as they must deliver certain minimal specifications with the cost and schedule provided  In those cases, it is vital for the team to understand and differentiate the IT requirements that are “must haves” from the “nice to haves” early in the planning effort  This would help facilitate delivery of the “must-haves” requirements first, thereby providing users with the greatest benefits as soon as possible  In addition to the contact named above, the following staff made key contributions to this report: Shannin G  O’Neill (Assistant Director), Jeanne Sung (Analyst in Charge), Andrew Beggs, Rebecca Eyler, Kendrick Johnson, Thomas J  Johnson, Jason Lee, Jennifer Leotta, and Melissa Melvin 
Report 2:  National Popular Vote opponents oppose the compact on various grounds  Some argue that it is unconstitutional or "anticonstitutional," that is, contrary to the Founders' intentions and the spirit of the nation's fundamental charter  It is also asserted that NPV would solve few of the electoral college system's alleged problem issues and would create some of its own  Finally, some observers note that the NPV compact is an interstate compact as defined in Article I, Section 10, clause 3 of the Constitution, and as such would be subject to congressional approval  This issue is examined in greater detail in a separate section of this report  On the most fundamental philosophical basis, opponents might argue that the NPV compact violates one of the basic principles of majoritarian democracy: it does not require that candidates win a majority of the popular vote in order to gain the presidency  Rather, it would anoint as winner the ticket that gains more popular votes than any other  A majoritarian democracy, it may be argued, should require a majority in order to elect; it may be further noted that the existing system, by comparison, requires a majority in the electoral college  As one commentary noted only the strictest of majoritarians desire a purely majoritarian presidential election system, and those individuals should be deeply troubled by the prospect of plurality presidencies, which the NPVC [sic] expressly countenances  Indeed, the NPVC promises to create more difficulties and "misfires" in its own way than the Electoral College system its proponents so earnestly seek to replace  Further, opponents might ask how the NPV compact would function in the event of a multicandidate election, a phenomenon that recurs from time to time in U S  presidential elections  One commentator posited the following problematic scenario under such circumstances: Under the compact, one can easily imagine a multi-candidate race in which a candidate would win, say, a thirty-four percent plurality of the popular vote nationwide while losing in every state and D C  If all of the states and D C  were signatories to the compact, all the electoral votes in such a hypothetical race would be awarded contrary to the will of voters choosing electors (still not voting directly for President under this plan)  Would the United States accept a President who wasn't the choice of sixty-six percent of those voting, nor even the choice of a single state? The existing electoral college system, NPV skeptics might also assert, is a fundamental element in the federal constitutional arrangements established by the Constitution  Fearing "the tyranny of the majority," the Founders established a system of government that provides checks and balances designed to restrain the majority and secure minority rights  These principles are also embedded in the structure of federal elections: the Senate, the House of Representatives, and the presidency were deliberately provided with different terms of office and different electorates, and the states were given an important role in the federal election process  In particular, through the electoral college the United States elects its national Presidents and Vice Presidents in a state-based federal election  Successful nominees are compelled under this system to present a broad political vision that commands nation-spanning "concurrent majorities" and appeals to the great variety of Americans  As in the case of the Senate, less populous states are accorded a small numerical advantage by assignment of two at-large electors reflecting the Constitution's equal apportionment of Senators to each state regardless of its population  The NPV initiative, they could claim, would discard the Founders' intentions in favor of what they consider to be a flawed "majoritarian" presidency that would ill-serve a continent-spanning and profoundly diverse republic  Another criticism centers on the use of the NPV compact to effect a fundamental change in the presidential election process and a de facto amendment to the Constitution, but without following the procedures set out in Article V  Critics may note that NPV's founders admit their plan is an "end run" around the Constitution  Proponents might counter with the argument that Article V presents too high a hurdle for what they consider a necessary reform of the system  Opponents, however, could respond that the Founders intended the various supermajority requirements in Congress and the states to ensure that successful constitutional amendments enjoy broad national support  The bare majority of electoral votes required to implement NPV, they might note, meets none of these supermajority requirements  As one study critical of the NPV initiative concluded, because the use of an interstate compact "does not conform to the constitutional means of changing the original decisions of the Framers, NPV could not [therefore] be a legitimate innovation " A final argument on this line might be that one "end run" around the amendment process might lead to others, setting a dangerous precedent for similar efforts in the future  Opponents might note that the NPV would eliminate the electoral college system's multiplier effect generated by the winner-take-all or general ticket system used in 48 states and the District of Columbia, which tends to magnify the winning ticket's margin of victory, and is said to confer greater legitimacy to the victors  For instance, in 2016, Republican nominee Donald Trump's clear electoral vote majority of 304 votes (56 5% of the total), compared with Democratic nominee Hillary Clinton's 227 (42 2% of the total) could be said to reinforce and confirm his victory, notwithstanding Clinton's plurality of the popular vote (48 2%), compared with Trump's 46 1%  From a practical standpoint, NPV opponents might argue that the NPV would actually lead to an increase in contested election results and legal challenges in the states, as the political parties maneuver to claim every possible vote  They assert that the existing tabulation of popular votes within each state reduces contested results and recounts  Under NPV, the incentive to gain every vote would arguably lead to far broader disputes and widespread recounts at every level of election administration  As a Heritage Foundation study concluded Under the NPV … any suspicions necessitating a recount in even a single district would be an incentive for a national recount     The prospect of a candidate challenging "every precinct, in every county, in every state of the Union" should be abhorrent to anyone who witnessed the drama, cost, delay, and undue litigation sparked by the Florida recount of 2000  Opponents might also assert that the increased incidence of recounts would be further complicated by wide-ranging disparities in state procedures, potentially leading to prolonged periods of uncertainty following close presidential elections  Critics may also note that the NPV plan contains no "statute of limitations," unlike proposed constitutional amendments, for which Congress typically sets a seven-year ratification period  Where, critics may ask, is a similar time limit that would "sunset" the NPV compact, after which it would expire or return to "square one"? According to its website, NPV was launched on February 23, 2006; if it were a constitutional amendment proposed by Congress, it would have expired on February 23, 2013, since by the end of the customary seven-year deadline it was "ratified" by only eight states and the District of Columbia  By what reasoning, they might ask, should the NPV be exempt from the standards of timeliness and contemporaneity Congress customarily sets for constitutional amendments? Opponents might reject claims that, under NPV, campaign spending and candidate appearances would be spread and scheduled more widely, beyond the current concentration of time and resources in battleground states  They might argue that spreading campaign resources and candidate events in non-battleground states is a questionable argument to justify a fundamental change in the presidential election process  Campaign appearances and spending, they could assert, should not be considered to be a local economic stimulus package, nor are the amounts in question sufficient to make much of a difference in the economic condition of most states  As one critical analysis notes, "    the nation does not hold presidential elections to foster local economic development " Moreover, they might continue, it is equally dubious to assert that nominees will slight the concerns of citizens of the non-battleground states from which they draw their greatest support, or that concentrated campaigning in the battleground states somehow "disenfranchises" voters in others  In the modern era, a small percentage of voters actually attends an in-person presidential or vice presidential candidate appearance  Television (especially broadcast and cable TV news networks), social media, the internet, and newspapers—not the traditional rallies, torchlight parades, and handbills—dominate presidential election campaigns in the 21 st century  In addition to policy issues discussed previously, some observers have also raised questions related to the NPV initiative based on the fact that it is an interstate compact as defined in the Constitution  Others have questioned whether NPV might conflict with some provisions of the Voting Rights Act  The NPV initiative has been described by its supporters variously as a bill, a state-level statute, and an interstate compact  The latter reference necessitates an analysis of whether the initiative complies with the Compact Clause of the Constitution  An interstate compact—under the broadest understanding—is a contract between two or more consenting states  The Supreme Court has further suggested that an interstate compact often requires reciprocal commitments between the governments of two or more states, such that one state's commitment is conditioned on the action of another state and no state can unilaterally repeal its commitment  The use of interstate compacts predates the Constitution, as the Articles of Confederation contained a similar Compact Clause that provided a qualified prohibition on states entering into any agreements between them without the consent of Congress  The chaos resulting from the disunity created by the Articles of Confederation prompted the Framers of the Constitution generally to "impose more uniformity" among the states, resulting in a Constitution that wholly prohibits states from entering into any treaties, alliances, and federations  Nonetheless, the Constitution maintained the Articles of Confederation's qualified prohibition on interstate compacts and agreements, allowing states to enter into an interstate compact so long as the participating states seek the consent of Congress  Specifically, the Compact Clause provides that "No State shall, without the Consent of Congress     enter into any Agreement or Compact with another State     " While the historical rationale for Article I's qualified prohibition on interstate compacts is unclear, the Compact Clause generally reflects the view of the Framers that states should be able to work cooperatively together, as well as the concern that unchecked interstate alliances might threaten the harmony of the Union or the authority vested by the Constitution in the federal government  As the Supreme Court noted in Cuyler v  Adams , "By vesting in Congress the power to grant or withhold consent, or to condition consent on the states' compliance with specified conditions, the Framers sought to ensure that Congress would maintain ultimate supervisory power over cooperative state action that might otherwise interfere with the full and free exercise of federal authority " The Compact Clause places no limits on what might be done through an interstate compact other than the requirement of congressional consent  In the early years of government under the Constitution, compacts were used almost exclusively to settle boundary disputes  Beginning with the establishment of the Port of New York Authority in 1921, however, compacts began to be used to address more complex, regional issues requiring intergovernmental cooperation  Some compacts are merely advisory in form, but others may be regulatory, with significant powers granted to multistate commissions  More recently, compacts have addressed such wide-ranging concerns as mental health treatment, law enforcement and crime control, education, driver licensing and enforcement, environmental conservation, energy, nuclear waste control, facilities operations, transportation, economic development, insurance regulation, placement of children and juveniles, disaster assistance, and pollution control  Approximately 200 interstate compacts are in effect today  Accordingly, the central legal issue with respect to the Compact Clause is whether a given interstate compact requires the consent of Congress  While a "literal" reading of the Compact Clause "would require the States to obtain congressional approval before entering into any agreement among themselves, irrespective of form, subject, duration, or interest to the United States [emphasis added]," the Supreme Court has repeatedly rejected such a reading  In 1893, in Virginia v  Tenness e e , Justice Stephen Field, writing for the Court, contended that a broad reading of the Compact Clause would "embrace all forms of stipulation, written or verbal, and relating to all kinds of subjects[,]" requiring congressional consent to agreements "which the United States can have no possible objection or have any interest in interfering with," as well as those that "may tend to increase     the political influence of the contracting states, so as to encroach upon or impair the supremacy of the United States     " Surmising that the Compact Clause could not have been intended to have such a broad reach, Justice Field concluded that the Clause prohibits states from entering into compacts without congressional consent only when the underlying compact is "directed to the formation of any combination tending to the increase of political power in the States, which may encroach upon or interfere with the just supremacy of the United States " The Supreme Court has subsequently reaffirmed Justice Field's "functional view of the Compact Clause," and, accordingly, generally where an agreement does not fall within the scope of the Compact Clause as envisioned by the Court in Virginia v  Tennessee , the agreement "will not be invalidated for lack of congressional consent " Whether the NPV initiative requires congressional consent under the Compact Clause first requires a determination as to whether NPV even constitutes an interstate compact  At times, its supporters have resisted framing the initiative as an interstate compact, arguably out of concern for running afoul of the Compact Clause's provisions  For example, Professor Akhil Amar has argued that because the initiative does not create a "new interstate governmental apparatus," the NPV should not be considered an interstate compact, as NPV compact signatory states are merely exercising power collectively that each state could exercise on its own  It is unclear, however, whether the creation of a new interstate governmental entity formed out of an agreement between two or more states is necessary, as opposed to sufficient, in order to deem an agreement as being an interstate compact subject to the Compact Clause  While the Supreme Court, in Northeast Bancorp , suggested that a "joint organization or body" formed out of an interstate agreement is a "classic indic[ium] of a compact," the Court has never adopted a definition of an interstate compact that solely rests on the existence of an interstate governmental body  Instead, the Court appears to have adopted a broader definition of what an interstate compact can entail  For example, in Virginia v  Tennessee , the Court noted that the words "compacts" and "agreements" are synonymous and "cover all stipulations affecting the conduct or claims of the parties " In other words, when two or more states enter into a stipulated agreement whereby one state agrees to perform an act in consideration for a reciprocal act by the other state(s), that agreement can be considered an interstate compact  This broad definition of a compact appears to encompass the NPV compact, as the initiative requires signatory states to agree mutually to appoint their electors to the winner of the national popular vote  Moreover, NPV binds each assenting state, as no member state can withdraw from it within six months or less of the end of a President's term  Because NPV prohibits states from freely "modify[ing] or repeal[ing] [the agreement] unilaterally" and requires "reciprocation" of mutual obligations, it appears that the initiative can be described as an interstate compact  Assuming the NPV initiative is an interstate compact, the question remains whether it is one that implicates the Compact Clause  The answer to that question primarily depends on whether NPV is "directed to the formation of any combination tending to the increase of political power in the States, which may encroach upon or interfere with the just supremacy of the United States  " In other words, the "test" for whether a particular interstate compact requires congressional consent is centrally concerned with vertical balances of power between the federal government and the states; namely, "whether the Compact enhances state power quoad the National Government " While the NPV arguably increases the political power of the states that have consented to it by ensuring that those states' desired outcome for the presidential election—the awarding of the majority of electoral votes to the presidential candidate supported by the majority of the voting populace—it is unclear how that increase in political power would be at the expense of the power of the federal government  After all, the Constitution provides the federal government with no role in determining the members of the electoral college  One scholar has suggested that the NPV initiative would lead to a vertical alteration of power by eliminating the possibility that the House of Representatives would resolve a presidential election in the absence of an electoral majority for a single candidate because it is premised on a majority of electoral votes going to a single candidate  The House of Representatives, however, has decided only two presidential elections in American history, and whether such an arguably hypothetical and de minimis diminishment of federal power through the NPV would be sufficient to require congressional consent under the Compact Clause is simply unresolved by the relevant case law  While the Supreme Court's case law interpreting the Compact Clause is centrally concerned with vertical federalism concerns (i e , the balance of power between the state and federal governments), the Court has recognized a potential secondary rationale suggested for the Compact Clause: to preserve the horizontal balance of powers among the various states  And horizontal federalism concerns could very well be implicated by the Compact Clause, as the provision appears to have been included in the Constitution out of concern both for the supremacy of the federal government and unity among the various states  Whether the NPV compact threatens the powers of nonconsenting states has been the subject of much debate among academics  Those in support of the initiative have contended that the nonconsenting states do not lose any power as a result of the NPV  According to this line of argumentation, even under the NPV, all states would retain their right to select the electors of their choosing, as nonmember state electors would still be counted in the electoral vote  Others, however, have pointed to the underlying premise of the NPV—to enhance the political power of more populous states in presidential elections—as evidence that the initiative diminishes the power of nonconsenting states  In other words, while non-compacting states would still retain the power to appoint electors, the influence that comes with that power would arguably be diminished because a state's role in the national election would be defined by its percentage of the popular vote and not by its percentage of electors, warranting congressional interest in approving a compact that effectuated such a change in national elections  Ultimately, however, whether the NPV actually threatens the power of nonconsenting states is a debate that remains active within academia but would likely be the source of considerable litigation if the initiative ever became effective  If congressional consent is needed for the NPV, that consent can take various forms  Usually congressional consent to an interstate compact takes the form of a joint resolution or act of Congress specifying its approval of the text of the compact and adding any conditions or provisions it deems necessary, often embodying the compact document  As with most congressional actions, consent to an interstate compact must occur with the approval of both houses and must be signed by the President before it becomes law  Rarely has the President vetoed or threatened to veto consent legislation by Congress  While congressional consent to an interstate compact is most often explicit, consent by Congress may also be implied by subsequent acts of Congress as "[a]n inference clear and satisfactory that Congress     intended to consent" to a compact may be sufficient  Congress may also delegate its power to approve a compact to a federal official so long as an "intelligible principle" against which approval can be measured is apparent  Ultimately, if congressional consent is truly needed for NPV to be effective, the initiative might have difficulty ever being enacted because the approval of both houses of Congress and the President would likely necessitate additional hurdles beyond the already challenging task of amassing support at the state level for the NPV  Beyond the legal issues raised with respect to the Compact Clause, the NPV initiative also potentially raises other broader constitutional concerns, including whether the states can functionally obviate the role of the electoral college through the NPV  Article II of the Constitution establishes that the election of the President should occur indirectly through the election by the electoral college  The choice of an indirect election for the President was a deliberate one by the Framers of the Constitution, because, while noting the importance that the "sense of the people" should influence the choice for President, they found it "equally desirable" for the "immediate election" of the President to be made by a body representative of distinct state interests and removed from the threat of unchecked majoritarianism  The result was that the Constitution established a presidential election process that was "manifestly nonmajoritarian," with the electoral college, a body established to represent the distinct views of each state, as the centerpiece of the election process  The central constitutional issue presented by the NPV, therefore, is whether the states, through an interstate compact, can functionally transform the presidential election system enshrined in the Constitution into a more majoritarian process  Supporters of the NPV argue that the Constitution provides the legal means for states to transform the presidential election system into one where the President is elected based solely on the result of the national popular vote  Specifically, clause 2 of Article II, Section 1 of the Constitution provides the states with the power to "appoint, in such Manner as the Legislature thereof may direct," the electors who represent the state in the electoral college  Facially, the Constitution's primary limitation on the power of a state to select its electors is the final number of electors awarded to each state  While perhaps an argument can be made that the structure, logic, and history of the Constitution place limits on the manner or method in which a state chooses its electors, the text of the Constitution simply does not impose any such limits  Supreme Court case law also supports reading Article II of the Constitution to broadly provide states with wide discretion as to the manner in which its electors are selected  Specifically, in 1892 in McPherson v  Blacker, a unanimous Supreme Court upheld a Michigan law providing for the election by individual congressional district of presidential electors against a challenge that the law violated Article II of the Constitution  In so holding, the Court placed great emphasis on a number of state laws that existed shortly after the ratification that provided a variety of "modes of choosing the electors," including selection by the legislature itself, by a "vote of the people for a general ticket," "by vote of the people in districts," or by some permutation of those methods  Viewing this evidence together with the text of Article II and the historical evidence from the Constitutional Convention led the Court to broadly conclude state legislatures have "conceded plenary power     in the matter of the appointment of electors," allowing the Michigan law to stand  Applying McPherson to the case of the NPV, the argument can and has been made that if the states have plenary power with respect to the manner of how electors are appointed, the power necessarily allows states to select electors in line with the results of the national popular vote  More recently, supporters of the NPV have relied on the Supreme Court's 2015 ruling in Arizona Legislature v  Arizona Independent Redistricting Commission (AIRC) —which held that the State of Arizona had wide discretion under the Elections Clause of the Constitution to select the method by which the state provided for redistricting —to argue that the states retain broad discretion in selecting electors under Article II, which uses similar language to the provision interpreted in AIRC  Others have argued that the structure of the Constitution and historical evidence suggest that the states do not have such vast discretion in appointing electors as to functionally transform the election for President into a national popular referendum  As noted elsewhere in this report, the electoral college was created by the Framers to ensure that states with the least population retained power in the selection of the President, providing a check against domination by the most populous states  The electoral college, being a product of the choices of individual state legislatures, was envisioned by the Framers as a body that would represent the specific interests of a given state, as opposed to the undifferentiated nation at large  Accordingly, it may be argued that allowing the most populous states to collude to ensure that the national popular vote, as opposed to the wishes of an individual state, dictates the results of a state's slate of electors, could arguably be irreconcilable with the Framers' intentions with respect to the electoral college  As such, for those who find the NPV compact constitutionally suspect under Article II, McPherson 's broad pronouncements about the nature of a state's power to appoint electors should be viewed in the context of that particular case, where the state of Michigan was attempting to appoint its electors based on the votes of an individual district in the state, as opposed to the state as a whole  In contrast to the law at issue in McPherson , with NPV, there appears to be no evidence contemporaneous with the ratification of the Constitution of a state selecting its electors in accordance with the results of the national popular vote  Unlike the State of Michigan in McPherson , an NPV state's electors might not be a product of the views of the state at the time of the election, but instead would reflect national popular sentiment about who should be the President  Moreover, the Supreme Court, in interpreting arguably analogous language from Article I of the Constitution allowing states to regulate the manner of the selection of the Members of the House of Representatives and Senate, concluded that the states cannot exercise their delegated authority in a way that would "effect a fundamental change in the constitutional structure " The question that remains is whether the Court in a future case challenging the NPV compact would interpret the states' authority under Article II to appoint electors to be broad enough to allow the President to be selected as a result of the national popular vote, a question that, given the lack of any precise precedent respecting the constitutionality of the NPV compact under Article II, will likely remain unresolved until such time  Other critics claim the NPV compact might violate Sections 2 and 5 of the Voting Rights Act (VRA)  Writing in Columbia Law Review , David Gringer invokes the voting power theory  He argues that the plan conflicts with Section 2 of the VRA because moving from "a state-based [vote] to a national popular vote dilutes the voting strength of a given state's minority population by reducing its ability [voting power] to influence the outcome of presidential elections " Gringer also asserts that the NPV compact may violate Section 5 of the act  In 2013, however, the U S  Supreme Court invalidated Section 4(b) of the VRA, which contained a formula prescribing which states and jurisdictions with a history of discrimination were required to obtain prior approval or "preclearance" under Section 5 before changing any voting standard, practice, or procedure  Although the Court invalidated only the coverage formula in Section 4, by extension, Section 5 has been rendered currently inoperable  Prior to the Supreme Court ruling, Gringer argued that the NPV compact would qualify as a covered practice under Section 5, and that the legislatures of all the "covered" states would have been required to obtain preclearance before implementing the compact  Responding to this point, National Popular Vote Inc  noted the following: The National Popular Vote bill manifestly would make every person's vote for President equal throughout the United States in an election to fill a single office (the Presidency)  It is entirely consistent with the goal of the Voting Rights Act  There have been court cases under the Voting Rights Act concerning contemplated changes in voting methods for various representative legislative bodies     However, these cases do not bear on elections to fill a single office (i e , the Presidency)  In 2012, the Justice Department's Civil Rights Division specifically declined to challenge California's accession to the NPV compact on VRA grounds  The states' authority to appoint electors by any method their legislatures choose is not absolute  Federal court decisions have struck down state laws concerning appointment of electors that were found to be in violation of the Fourteenth Amendment's guarantee of equal protection: Although Clause 2 (of Article II, Section 1 of the Constitution) seemingly vests complete discretion in the states, certain older cases had recognized a federal interest in protecting the integrity of the process  Thus, the Court upheld the power of Congress to protect the right of all citizens who are entitled to vote to lend aid and support in any legal manner to the election of any legally qualified person as a presidential elector     [I]n Oregon v  Mitchell (42 U S  112 (1970)), the Court upheld the power of Congress to reduce the voting age in presidential elections and to set a thirty-day durational residency period as a qualification for voting in presidential elections  Although the Justices were divided on the reasons, the rationale emerging from this case, considered with Williams v  Rhodes , (393 U S  20 1968)) is that the Fourteenth Amendment limits state discretion in prescribing the manner of selecting electors and that Congress in enforcing the Fourteenth Amendment may override state practices that violate that Amendment and may substitute standards of its own  Critics of the electoral college system have sought direct election of the President and Vice President without success for more than two centuries  The NPV initiative represents a novel effort to achieve this goal by use of an interstate compact that would circumvent the stringent requirements necessary for the proposal and ratification of constitutional amendments  Since its inception in 2006, NPV has achieved a degree of success: 14 states and the District of Columbia, controlling a total of 189 electoral votes, have joined the compact since 2008  Progress has arguably been sporadic, however, notwithstanding active campaigning by National Popular Vote Inc  Over the course of more than a decade, NPV has heretofore failed to develop a sustained momentum toward its stated goal of states controlling 270 electoral votes  The action of three states in joining NPV to date in 2019 marks the most activity in a single year since 2008; it remains to be seen whether this trend will continue  To date, certain Democratic-leaning states have joined the compact  The arguable lack of support in Republican-controlled state legislatures raises questions about further accessions to the compact in the immediate future, particularly given the fact that the GOP controlled both legislative chambers in 30 states following the 2018 elections  To date, while the NPV initiative has generated interest among supporters of direct popular election of the President, it does not appear to have gained widespread awareness among the public at large  The findings of the March 27, 2019, Politico/Morning Consult survey cited earlier in this report arguably suggest that greater public knowledge of NPV might spur popular support for the compact  This might then contribute to further momentum if additional states were to join, particularly populous ones like Florida (29 electoral votes), Georgia (16 electoral votes), and Ohio (18 electoral votes) where the compact was under active consideration in 2019  Under these circumstances, proponents might be energized and encouraged by a sense of progress for the initiative  At the same time, NPV opponents could be expected to coalesce around the issues identified earlier in this report, and renew and increase their efforts in defense of the electoral college system  The activities of both might ultimately bring the NPV initiative to the more immediate attention of Congress 

ground_truth: FEMA, a component of DHS, annually awards billions of dollars in grants to help communities prepare for, mitigate the effects of, and recover from major disasters. However, FEMA's complex IT environment supporting grants management consists of many disparate systems. In 2008, the agency attempted to modernize these systems but experienced significant challenges. In 2015, FEMA initiated a new endeavor (the GMM program) aimed at streamlining and modernizing the grants management IT environment. GAO was asked to review the GMM program. GAO's objectives were to (1) determine the extent to which FEMA is implementing leading practices for reengineering its grants management processes and incorporating needs into IT requirements; (2) assess the reliability of the program's estimated costs and schedule; and (3) determine the extent to which FEMA is addressing key cybersecurity practices. GAO compared program documentation to leading practices for process reengineering and requirements management, cost and schedule estimation, and cybersecurity risk management, as established by the Software Engineering Institute, National Institute of Standards and Technology, and GAO. Of six important leading practices for effective business process reengineering and information technology (IT) requirements management, the Federal Emergency Management Agency (FEMA) fully implemented four and partially implemented two for the Grants Management Modernization (GMM) program (see table). Specifically, FEMA ensured senior leadership commitment, took steps to assess its business environment and performance goals, took recent actions to track progress in delivering IT requirements, and incorporated input from end user stakeholders. However, FEMA has not yet fully established plans for implementing new business processes or established complete traceability of IT requirements. Until FEMA fully implements the remaining two practices, it risks delivering an IT solution that does not fully modernize FEMA's grants management systems. While GMM's initial May 2017 cost estimate of about $251 million was generally consistent with leading practices for a reliable, high-quality estimate, it no longer reflects current assumptions about the program. FEMA officials stated in December 2018 that they had completed a revised cost estimate, but it was undergoing departmental approval. GMM's program schedule was inconsistent with leading practices; of particular concern was that the program's final delivery date of September 2020 was not informed by a realistic assessment of GMM development activities, and rather was determined by imposing an unsubstantiated delivery date. Developing sound cost and schedule estimates is necessary to ensure that FEMA has a clear understanding of program risks. Of five key cybersecurity practices, FEMA fully addressed three and partially addressed two for GMM. Specifically, it categorized GMM's system based on security risk, selected and implemented security controls, and monitored security controls on an ongoing basis. However, the program had not initially established corrective action plans for 13 medium- and low-risk vulnerabilities. This conflicts with the Department of Homeland Security's (DHS) guidance that specifies that corrective action plans must be developed for every weakness identified. Until FEMA, among other things, ensures that the program consistently follows the department's guidance on preparing corrective action plans for all security vulnerabilities, GMM's system will remain at increased risk of exploits. GAO is making eight recommendations to FEMA to implement leading practices related to reengineering processes, managing requirements, scheduling, and implementing cybersecurity. DHS concurred with all recommendations and provided estimated dates for implementing each of them.