prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: Major disaster declarations can trigger a variety of federal response and recovery programs for government and nongovernmental entities, households, and individuals  FEMA’s Office of Response and Recovery manages the PA grant program, providing funds to states, territorial governments, local government agencies, Indian tribes, authorized tribal organizations, and certain private nonprofit organizations in response to presidentially declared disaster declarations to repair damaged public infrastructure such as roads, schools, and bridges  Figure 1 shows the total amount of PA funds obligated by county from January 2009 through February 2017 for federal disaster declarations  To implement the PA program, FEMA’s staff includes a mix of temporary, reservist, and permanent employees under two authorities, the Stafford Act and Title 5  Reservists make up the largest share of the PA workforce, which consisted of 1,852 employees––1,041 reservists, 634 full-time equivalents, and 177 temporary Cadre of On-Call Response/Recovery Employees––as of June 2017, according to PA officials  Figure 2 summarizes the key characteristics for each type of employee  After a disaster, FEMA sends PA program staff to the affected area to work with state and local officials to assess the damage prior to a disaster declaration  FEMA officials establish a temporary Joint Field Office (JFO) to house staff who will manage response and recovery functions after a declared disaster (including operations, emergency response and support teams, planning, administration, finance, and logistics)  Once the President has declared a disaster, PA staff work with grant applicants to help them document damages, identify eligible costs and work, and prepare requests for PA grant funds by developing project proposals  These proposals may include proposals for hazard mitigation if the hazard mitigation work is related to the repair of damaged facilities, referred to as permanent work projects  Immediate emergency measures, such as debris removal, are not eligible for hazard mitigation  Officials then review and obtain approval of the projects prior to FEMA obligating funds to state grantees  Figure 3 describes the process used to develop, review, and obligate PA projects  In addition to rebuilding and restoring infrastructure to its predisaster state, the PA program can be used to fund hazard mitigation measures that will reduce future risk to the infrastructure in conjunction with the repair of disaster-damaged facilities  There is no preset limit to the amount of PA funds a community may receive; however, PA hazard mitigation measures must be determined to be cost effective  Some examples of hazard mitigation measures that FEMA has predetermined to be cost effective, if they meet certain requirements, include installing shut-off valves on underground pipelines so that damaged sections can be isolated during or following a disaster; securing a roof using straps, clips, or other anchoring systems in locations subject to high winds; and installing shutters on windows or replacing glass with impact-resistant material  Applicants can also propose mitigation measures that are separate from the damaged portions of a facility, such as constructing floodwalls around damaged facilities to avoid future flooding  FEMA evaluates these proposals, considering how the proposed measure protects damaged portions of a facility and whether the measure is reasonable based on the extent of the damage, and determines eligibility on a case-by-case basis  FEMA’s Federal Insurance and Mitigation Administration (FIMA) deploys a cadre of mitigation staff to help coordinate and implement hazard mitigation activities during disaster recovery, including PA hazard mitigation  A primary task of these staff is to identify and assess opportunities to incorporate hazard mitigation into PA projects  Generally, if an applicant seeks to incorporate hazard mitigation measures into a PA project, FIMA’s hazard mitigation staff develop a hazard mitigation proposal  We, the DHS OIG, and others have reported past challenges with FEMA’s management of the PA program related to workforce management, information sharing, and hazard mitigation  For example, we reported in 2008 that the PA program had a shortage of experienced and knowledgeable staff, relied on temporary rotating staff, and provided limited training to their workforce, which impaired PA program delivery and delayed recovery efforts after Hurricanes Katrina and Rita  We found that staff turnover, coupled with information sharing challenges, delayed projects when applicants had to provide the same information each time FEMA assigned new staff and that poorly trained staff provided incomplete and inaccurate information during their initial meetings with applicants or made inaccurate eligibility determinations, which also caused processing delays  We recommended that FEMA strengthen continuity among staff involved in administering the PA program by developing protocols to improve information and document sharing among FEMA staff  In response, in 2013 FEMA instituted a PA Consistency Initiative, which included hiring new managers for FEMA regional offices, stakeholder training on PA program administration, and using a newly developed internal website to allow staff to post and share information to address continuity and knowledge sharing concerns during disaster operations  FEMA also developed the Public Assistance Program Delivery Transition Standard Operating Procedure to facilitate the transfer of responsibility for PA program activities during cases of staff turnover during recovery operations  Despite FEMA’s efforts to implement our recommendations, the DHS-OIG, in 2016, found continuing challenges after Hurricane Sandy with workforce levels, skills, and performance of reservists, who make up the majority of the PA workforce  Regarding information sharing, in 2008, we also identified difficulties sharing documents among federal, state, and local participants in the PA process and difficulties tracking the status of projects  We recommended that FEMA improve information sharing within the PA process by identifying and disseminating practices that facilitate more effective communication among federal, state, and local entities  In response, FEMA proceeded with the implementation of a grant tracking and management system, called EMMIE, which was used previously in 2007  However, in subsequent years we found weaknesses in how FEMA developed the system and the DHS-OIG found that information sharing problems similar to the ones identified in our 2008 report persisted  Regarding hazard mitigation, we reported in 2015 that state and local officials experienced challenges in using PA hazard mitigation during the Hurricane Sandy recovery efforts because PA officials did not consistently prioritize hazard mitigation, and in some cases discouraged mitigation projects during the PA grant application process, among other challenges  We recommended that FEMA assess the challenges state and local officials reported, including the extent to which they can be addressed, and implement corrective actions, as needed  In response to our recommendation, FEMA developed a corrective action plan that included actions and milestones for reviewing, updating, and implementing PA hazard mitigation policy  FEMA also identified the PA new delivery model as a solution for some of the challenges state and local officials reported  Previously, the OIG also reported that PA program officials did not consistently identify eligible PA hazard mitigation projects, and that PA officials did not prioritize the identification of PA hazard mitigation opportunities at the onset of recovery efforts after the 2005 Gulf Coast hurricanes  See appendix I for a summary of findings and the status of our past recommendations on challenges with workforce management, information sharing, and hazard mitigation related to the PA program since our last review in December 2008  FEMA’s own internal reviews and outreach efforts have also identified similar challenges  For example, at FEMA’s request the Homeland Security Studies and Analysis Institute assessed the effectiveness and efficiency of the PA program in 2011  The institute’s report outlined 3 key findings and 23 recommendations relating to the PA preaward process  For example, the report found that FEMA could enhance training programs to develop a skilled and experienced workforce; utilize technology and employ web-based tools to support centralized processing, transparency, and efficient decision making; and identify and address potential special considerations, such as hazard mitigation proposals, as early as possible in the preaward process to improve consistency  In 2014, PA program officials analyzed the PA grant process and used input from agency staff and officials involved in various aspects of the program to identify potential improvements  The resulting Public Assistance Program Realignment report found that challenges in workforce management, information sharing, and hazard mitigation continued, and included recommendations for improvement  For example, the report concluded that a shortage of qualified staff, high turnover, unclear organizational responsibilities, and inconsistent training were long-standing and continuing challenges that impaired the PA pre-award process  In addition, from January 2015 to April 2015, FEMA conducted extensive outreach with more than 260 stakeholders across FEMA headquarters, all 10 regions, 43 states, and 4 tribal nations to discuss challenges in the PA program and opportunities for improvement  For example, stakeholders identified challenges with ineffective information collection during the preaward process and suggested identifying special considerations, such as hazard mitigation, earlier in the PA process as an idea for improvement  In response, FEMA began redesigning the PA preaward process to operationalize the results of its 2014 report and address areas for improvement identified through its outreach efforts  FEMA awarded a contract for program support to help PA officials implement a redesigned PA program in 2015  This included a new process to develop and review grant applications, and obligate PA funds to states affected by disasters; new positions, such as a new program delivery manager who is the single point of contact throughout the grant application process; a new Consolidated Resource Center (CRC) to support field operations by supplementing project development, validation, and review of proposed PA project applications; and a new information system to maintain and share PA grant application documents  As part of the new process, PA program officials identified the need to ensure that staff emphasize special considerations, such as hazard mitigation, earlier in the process  Taken together, these efforts represent FEMA’s “new delivery model” for awarding PA program grants  Enhancements in the PA program under the new delivery model are presented in figure 4  Regarding the new delivery model process, FEMA introduced several changes to enhance outreach to applicants during the “exploratory call”— the first contact between FEMA and local officials—and during the first in- person meeting, called the “recovery scoping meeting ” FEMA also revised decision points during the process, when program officials can request more information from applicants, and applicants can review and approve the completion of project development steps  FEMA also incorporated special considerations, such as hazard mitigation, earlier in the new process during the exploratory calls and recovery scoping meetings  The changes and enhancements to the PA grant award process in the new delivery model are presented in figure 5  The new process divides proposed PA projects based on complexity and type of work into three categories—100 percent completed, standard, and specialized—that PA staff manage to expedite review or assign skilled staff to technical projects as needed  If the applicant has already completed work following a disaster, such as debris removal, it is considered “100 percent completed” and JFO staff collect the necessary documents and provide the information to the CRC staff who complete the development of project applications, validate the information, and complete all necessary reviews  Projects that require repairs and further assistance from PA program staff at the JFO include “standard” and “specialized” projects, which include a site inspection to document damages, before the JFO staff provide the information to the CRC  Further, PA program officials assign PA staff based on their skills and experience to standard projects, which are less technically complex to develop, and specialized projects, which are more technically complex and costly  We discuss the new workforce positions FEMA developed for JFOs and CRCs, the new information system FEMA developed to maintain and share PA grant documents with applicants, and FEMA’s efforts to incorporate hazard mitigation into PA projects later in this report  Since 2015, FEMA has invested almost $9 million to redesign the PA program through the reengineering and implementation of the new delivery model, including about $4 7 million for contract support for implementation, and $4 million for acquisition of the new information system  FEMA tested the new delivery model in a series of selected disasters, using a continuous process improvement approach to assess and improve the process, workforce changes, and information system requirements, prior to implementing the new model for all future disasters  For example, FEMA first tested the new process in Iowa in July 2015 and, in February 2016, PA program officials expanded their test to include all of the new staff positions  In October 2016, PA program officials added the new information system to achieve a comprehensive implementation of all of the elements of the new delivery model for the agency’s response to Hurricane Matthew in Georgia, two additional disasters in Georgia in January 2017, and in Missouri, North Dakota, Wyoming, Vermont, and two disasters in New Hampshire from June through August 2017  The timeline for PA’s implementation of the new delivery model is shown in figure 6  According to program officials, FEMA planned to implement the new model for all future disasters beginning in January 2018  However, historic disaster activity during the 2017 hurricane season accelerated full implementation  As a result, on September 12, 2017, FEMA officials announced that, unless officials determined it would be infeasible in an individual disaster, the program would use the new delivery model in all future disasters  According to FEMA’s 2014 PA Program Realignment report and other program documents, PA officials designed the new delivery model to respond to persistent workforce management challenges related to identifying the required number of staff and needed skills and training, among other things, to improve the efficiency and effectiveness of the PA preaward process  To address these challenges, PA program officials centralized much of the responsibility for processing PA projects in the CRCs, created additional new positions with specialized roles and responsibilities in JFOs, and established training and mentoring programs to help build the new staffs’ skills  In 2016, PA program officials centralized some of the project activities that otherwise were being carried out at individual JFOs at FEMA’s first new CRC in Denton, Texas  Officials did so by establishing 18 new positions, many of which directly correlated with positions that FEMA deployed to individual JFOs in the legacy PA delivery model  According to PA officials, centralizing positions will improve standardization in project processing, and result in a higher quality work product  As part of the new delivery model, PA program officials created a new hazard mitigation liaison position for PA program staff at the CRC that did not previously exist at individual JFOs  The other new positions that PA program officials either created or centralized at the CRC included two specialized positions responsible for costing and validating PA projects  Previously, the PA project specialist deployed to the JFO would complete these tasks and others; however, the consistency of project development varied across the regions and disasters  In contrast, CRC staff are full-time employees who receive training to specialize in completing standardized project development steps for PA projects from multiple disasters on an ongoing basis  Program officials anticipate that centralizing new specialized staff at the CRCs will also reduce PA administrative costs and staffing levels at the JFOs  For example, staff at the CRCs, such as the new hazard mitigation liaisons and insurance and costing specialists, could support project development for multiple disasters and regions simultaneously, whereas PA previously needed to deploy staff to each JFO to fulfill these roles  In addition, once JFOs operating under the new model send projects to the CRCs for processing and review, FEMA can more rapidly close its JFOs, reducing associated administrative costs  For example, following Hurricane Matthew, FEMA credited the new delivery model, in part, with its ability to close the JFO in Georgia sooner than several other JFOs in neighboring states not involved in the implementation of the new delivery model  PA program officials created new positions with more specialized roles and responsibilities to help PA staff at JFOs provide more consistency in the project development process and guidance to applicants  Program officials split the broad responsibilities previously managed at the JFOs by PA crew leaders and project specialists, into two new, specialized positions—the program delivery manager and site inspector  The program delivery manager serves as the applicant’s single point-of-contact throughout the preaward process, manages communication with the applicant, and oversees document collection  All three PA grant applicants we spoke to following Hurricane Matthew in Georgia greatly appreciated the knowledge and assistance provided by their program delivery managers  Site inspectors are responsible for conducting the site inspection to document all disaster-related damages; determining the applicant’s plans for recovery, coordinating with other specialists, and verifying the information collected with the applicant  Officials expect deployed staff at JFOs can complete the fieldwork faster and provide greater continuity of service to applicants  Further, program officials believe that specializing roles will enable them to provide more targeted training, and improve employee satisfaction  Site inspection, hazard mitigation, and environmental and historic preservation specialists, along with a new Public Assistance program mentor, conduct a site inspection with the applicant to document damages to a historic cemetery in Savannah, Georgia, following Hurricane Matthew in 2016  PA program officials designed new training and mentoring programs for the new positions at the CRCs and JFOs and used a continuous feedback process to update and improve the training, position guides, and task books throughout the implementation of the new delivery model, according to PA officials  According to a June 2017 update of the PA Cadre Training Plan, training for the new model has five major focuses: required training and skills for position qualification; on-site refresher training; mentor training; regional-based state, local, tribal, and territorial training; and training on the new information system  Specifically, officials developed six new training courses, and identified which are required for each position under the new delivery model  For example, a program delivery manager at the JFO is required to complete both the program delivery manager and site inspector specialist courses  As of June 2017, PA program officials had provided at least one new model training course to 93 percent of their cadre (including program delivery manager training to 366 individuals and site inspector training to 1,172 individuals) and planned to provide 28 additional courses through September 2017 to the PA cadre  According to regional and CRC officials, the training courses and mentoring from experienced staff helped maximize new staff’s capabilities in the new process  Throughout the third implementation of the new delivery model, JFO and CRC staff, as well as regional PA staff, stakeholders, and applicants, identified staff skills and training as a key area that needed more attention for full implementation of the new delivery model  Our work and FEMA’s after-action reports from the third test in Georgia identified problems with site inspector skills, which affected the timeliness and accuracy of projects  Specialists and managers at the CRC noted that poorly trained site inspectors did not consistently provide the necessary information from the field, which resulted in delays for the CRC staff to process projects, and after-action reports also identified challenges with site inspector skills  According to a PA applicant in Georgia, the inconsistency of skills and experience of their site inspector resulted in the need to conduct a “do-over” site inspection on one of the applicant’s projects, causing delays  PA staff and state officials attribute much of the site inspectors’ skill gaps to their lack of training and experience in the program  According to PA Region officials, providing timely training will be a resource-intensive challenge for implementing the new delivery model for all future disasters  For example, it can be difficult to train reservists before FEMA deploys them to disasters, and many of the program’s experienced reservists have retired or resigned, resulting in few mentors for the program and a high need to provide training to inexperienced and newly hired staff  PA officials and stakeholders also emphasized the need for FEMA to provide additional training for state and local officials to build capacity and support the goals of the new delivery model  For example, according to JFO officials at the third implementation, the new delivery model increases responsibilities for applicants, who will require more applicant training than FEMA currently provides  According to state officials, applicant capabilities vary, and FEMA should provide training to state and local officials on the new delivery model and the information system before a disaster  Skill gaps among applicants could result in inconsistent implementation of the new process, according to PA staff and stakeholders, and PA staff said that training was important to prevent applicants from reverting back to the legacy PA grant application process  To support full implementation of the new delivery model for all disasters, PA program officials have updated training courses for PA staff and applicants, and planned additional training to address these challenges and other lessons learned through the test implementation  For example, PA officials told us they updated the site inspector training program in May 2017 and scheduled a new site inspector training session in August 2017 to include more hands-on training to help address the skill gaps identified for site inspectors  PA officials created a new training course for FEMA’s regional offices, in part to enable regional PA staff to provide new delivery model training to state and local officials  PA officials also planned to develop a self-paced, online course for state and local officials by the end of 2017  PA officials have not fully assessed the workforce needed for JFO field operations, CRC staff, or FIMA’s hazard mitigation staff to support implementation of the new delivery model for all future disasters  PA program officials developed an initial assessment of the total number of staff needed in the field and the CRCs in 2016 to estimate cost savings associated with consolidating and specializing positions at the CRCs and deploying fewer staff to JFOs  However, the assessment did not identify the number of staff required to fill specific positions, including program delivery managers and hazard mitigation specialists, needed to support the new delivery model for full implementation  In reviewing the test implementations of the new delivery model, we found that inadequate staffing levels at the JFOs and CRCs, and with FIMA’s hazard mitigation staff, affected staffs’ ability to achieve the goals of the new delivery model  Staff levels at the JFO  We identified challenges with having the right number of program delivery managers and site inspection specialists to achieve program goals for customer satisfaction, efficiency, and quality in test implementations of the new delivery model  For example, in the second test implementation of the new delivery model in Oregon in 2016, PA did not deploy enough program delivery managers to the disaster, which resulted in unmanageable caseloads for program delivery managers, according to state and PA officials  PA program officials assigned program delivery managers an average caseload of 12 PA applicants, which was more than they could effectively manage, according to PA staff, and program officials aim for a caseload of 8 to 10 applicants  According to state officials, local officials reported they did not always receive the support they needed from program delivery managers who managed caseloads consisting of dozens of projects at multiple sites for each applicant during the Oregon implementation  As a result of overwhelmed program delivery managers, local officials faced challenges understanding their responsibilities, such as recognizing when they needed to provide information for the project development to proceed, according to state officials  PA staff involved with the third test implementation in Georgia in 2016 and 2017 said there were not enough site inspectors or program delivery managers to fully manage the workload at the JFO  Because of the specialization of roles, projects could not move forward when there were not enough staff to execute the next step in the process  For example, PA staff at the JFO said program delivery managers completed recovery scoping meetings rapidly, but faced a bottleneck in scheduling site inspections because there were more applicants awaiting site inspections than could be fulfilled by the number of site inspection specialists available  Staff levels at the CRC  Staff at the CRC reported challenges with staffing levels during the Oregon and Georgia test implementations, and expressed concerns about when PA officials will staff the CRCs to support full implementation of the new model for all disasters  During the Oregon test implementation, a CRC specialist said there were not enough technical specialists to manage the workload and, as a result, PA program officials had to redeploy site inspectors from their JFO field operations to the CRC to complete costing estimates  During the third test in Georgia, quality assurance specialists said that their workload resulted in added stress trying to complete the work in time while adhering to quality standards  According to CRC specialists in Denton, Texas, PA officials had not determined required staff levels for full implementation, but agreed that workload was too high and program officials needed to determine the appropriate staff levels for each CRC to support full implementation  PA officials were still evaluating CRC processing times and workload management from the Oregon and Georgia test implementations to determine staffing needs, according to PA officials  Further, PA program officials plan to establish a second CRC in Winchester, Virginia, before the end of 2017, but have not determined the number of additional permanent full-time staff needed to support the CRCs for full implementation of the new delivery model  Staff levels for the hazard mitigation specialists  PA officials have not identified the number of hazard mitigation specialists in FIMA’s hazard mitigation cadre needed for full implementation of the new delivery model  According to JFO staff, current hazard mitigation staff levels are insufficient to provide the desired in-person participation of hazard mitigation staff on all recovery scoping meetings to share information on hazard mitigation with applicants and help them identify potential mitigation opportunities  A PA program official said officials missed opportunities to pursue hazard mitigation during the test implementation after Hurricane Matthew in Georgia due to lack of hazard mitigation specialists  In addition, for the test implementation in Oregon, there were not enough hazard mitigation specialists to cover all site inspections and implement their new delivery model responsibilities, according to FEMA’s after-action reports  The absence of hazard mitigation specialists in the early stages of PA project development may cause delays in officials’ identifying hazard mitigation opportunities, according to a FIMA official  PA program officials said they did not work with FIMA to determine the appropriate levels of hazard mitigation staff under the new delivery model because they were refining the new process, but as of June 2017 were working with FIMA to do so  One of the key implementation activities in our Business Process Reengineering Assessment Guide includes addressing workforce management issues  Specifically, this includes identifying how many and which employees will be affected by the position changes and retraining  Further, our prior work has found that high-performing organizations identify their current and future workforce needs—including the appropriate number and deployment of staff across the organization— and address workforce gaps, to improve the contribution of critical skills and competencies needed for mission success  According to a PA program official, their initial workforce assessment was not comprehensive because they were still collecting data required to make informed decisions  PA officials agreed that updating their workforce assessments prior to full implementation could be helpful, and acknowledged that program officials needed to be more proactive applying the lessons learned as they pivot from testing to full implementation of the new delivery model in 2018  FEMA also conducts a standard agency wide workforce structure review every 2 to 3 years, which helps officials determine the appropriate disaster workforce levels  As of June 2017, PA officials were working with other offices within FEMA to expedite the agency-wide assessment of the PA and FIMA hazard mitigation cadres, but did not know when they would complete the assessment  PA officials also acknowledged that they faced an aggressive schedule to complete various planned activities for workforce management, training, and other efforts, in support of full implementation, and that they may not be able to complete all efforts as thoroughly as they would like in order to expedite the transition of the PA program to the new delivery model  The gaps in PA workforce assessment in the JFOs, CRCs, and for FIMA’s hazard mitigation cadre present a risk that PA program managers will not have a sufficient workforce to support the goals of the new delivery model  In addition, the timing and implementation of the hiring and training activities for new PA program staff could take multiple months, and program officials will need to know what staff levels are necessary for full implementation of the new delivery model to inform resource decisions for the program in coordination with other agency offices  According to PA program officials, workforce assessment efforts have been delayed as a result of disaster response and recovery efforts related to Hurricanes Harvey, Irma, and Maria  Completing a workforce assessment will help program officials identify gaps in their workforce and skills, which could help PA program officials minimize the effects of long- standing workforce staffing and training challenges on the PA program delivery and inform full implementation for all disasters  costs  For example, EMMIE does not collect information on all of the preaward activities that are part of the PA grant application process  As a result, PA program officials said they, and applicants, must use ad hoc reports and personal tracking documents to manage and monitor the progress of grant applications  PA officials added that EMMIE is not user- friendly and applicants often struggle to access the system  In response to these ongoing challenges, PA program officials developed FAC-Trax— a separate information system from EMMIE—with new capabilities designed to improve transparency, efficiency, and management of the PA program  Specifically, FAC-Trax allows FEMA staff (PA Grants Manager) and applicants (PA Grants Portal), to review, manage, and track current PA project status and documentation  For example, applicants can use FAC- Trax to submit requests for public assistance, upload required project documentation, approve grant application items, and send and receive notifications on grant progress and activities  In addition, the FAC-Trax system includes standardized forms, as well as required fields and tasks that PA program staff and applicants must complete before moving on to the next steps in the PA preaward process  According to PA officials, these capabilities increase transparency, encourage greater applicant involvement, and enhance collaboration and communication between FEMA and grant applicants, to improve efficiency in processing and awarding grant applications and enhance the quality of project development  Further, PA officials said that FAC-Trax could reduce challenges associated with staff turnover during the project development process because the system stores and maintains applicant information and project documentation, making it easier for transitioning staff to assist an applicant  They also said they use FAC-Trax to gather and analyze data that supports management of the PA process, including measuring the timeliness of the grant application process  For example, during the test implementation of the new delivery model in Georgia following Hurricane Matthew, officials were able to document that, on average, program delivery managers took 5 days to conduct the exploratory call and 14 days to hold the recovery scoping meeting with applicants, and CRC officials took 33 days to develop and review grant proposals  Managers use this data to assess staffing needs and identify bottlenecks in the PA process, according to PA officials  FAC-Trax is critical to the new PA delivery model and will be a primary means of sharing grant application documents, tracking ongoing PA projects, and ensuring that FEMA staff and applicants follow PA grant policies and procedures  Given the importance of developing and testing this new information sharing system, we evaluated its development against four key IT management controls—(1) project planning; (2) risk management; (3) requirements development; and (4) systems testing and integration  When implemented effectively, these controls provide assurance that IT systems will be delivered within cost and schedule and meet the capabilities needed by its users  We found that FEMA’s development of FAC-Trax fully satisfied best practices for project planning and risk management, but additional steps are needed to fully satisfy the areas of requirements development and systems testing and integration, as discussed below  See appendix II for the full assessment of each IT management control  PA program officials fully satisfied all five practices in the project planning control area, according to our assessment  Key project planning practices are (1) establishing and maintaining the program’s acquisition strategy, (2) developing and maintaining the overall project plan and obtaining commitment from relevant stakeholders, (3) developing and maintaining the program’s cost estimate, (4) establishing and maintaining the program’s schedule estimate, and (5) identifying the necessary knowledge and skills needed to carry out the program  To address the first and second practices, program officials established detailed plans that describe the acquisition strategy and objectives, the program’s scope, and its framework for using an Agile software development approach, among other key actions  Agile is a method of software development that utilizes an iterative process and constantly improves software based on user needs and feedback  Program officials also developed a plan detailing the program’s approach to deploy and maintain FAC-Trax and established stakeholder groups and an integrated product team to support and oversee the development of FAC-Trax  To address the third and fourth practices, they developed and maintained a master schedule of all implementation tasks and milestones through project completion, and developed a life-cycle cost estimate of over $19 million  Additionally, FAC-Trax’s acquisition performance baseline describes the system’s minimum acceptable and desired baselines for performance, schedule, and cost  Lastly, in regards to the fifth practice, program officials identified the knowledge and skills needed to carry out the program in the FAC-Trax Request for Proposal and FAC-Trax Capability Development Plan  PA program officials fully satisfied all four practices in the risk management control area, according to our assessment  Key risk management practices are (1) identifying risks, threats, and vulnerabilities that could negatively affect work efforts, (2) evaluating and categorizing each identified risk using defined risk categories and parameters, (3) developing risk mitigation plans for selected risks, and (4) monitoring the status of each risk periodically and implementing the risk mitigation plan as appropriate  To address the first and second practices, program officials identified key risks that could negatively affect FAC-Trax in a “risk register”—an online site used to track risks, issues, and mitigating actions  As of May 2017, officials had identified 13 risks in the risk register—four open and nine closed—and evaluated and categorized the identified risks based on the probability of occurrence and scope, schedule, and cost impacts  For example, program officials reported that two of its open risks have a “medium” risk rating—meaning the risk has the potential to slightly affect project cost, schedule, or performance  To address the third and fourth practices, program officials developed and documented risk mitigation plans for all identified risks  For example, program officials planned to mitigate the risk of limited engagement of subject matter experts by identifying and engaging with appropriate experts through workshops, and monitoring the capability development process to identify any issues that may cause project delays  In addition, PA program officials documented the responsible officials, reevaluation date, and risk status, among other things, for each risk in the register, and reviewed and updated risks during weekly and monthly program reviews with stakeholders throughout FEMA  PA program officials fully satisfied four out of five practices in the requirements development control area, according to our assessment  Key requirements development practices are (1) eliciting stakeholder needs, expectations, and constraints, and transforming them into prioritized customer requirements; (2) developing and reviewing operational concepts and scenarios to refine and discover requirements; (3) analyzing requirements to ensure that they are complete, feasible, and verifiable; (4) analyzing requirements to balance stakeholder needs and constraints; and (5) testing and validating the system as it is being developed  To address the first and second practices, program officials developed a requirements management plan outlining how officials capture, assess, and plan for FAC-Trax enhancements, and established a change control process to review, prioritize, and verify user requests for changes to the system and feedback  As of May 2017, the PA program office received 734 change requests related to FAC-Trax, of which program officials completed 420 changes and planned to address an additional 277 entries  Program officials also developed a functional requirements document outlining the high-level requirements for FAC- Trax and detailed operational concepts and scenarios for each phase of the preaward process in the system’s concept of operations  To address the fourth practice, program officials created a standard template to analyze and document the user needs and acceptance criteria for planned system capabilities in March 2017  In addition, PA program officials identified risks and dependencies for recommended changes to FAC-Trax, and balanced the cost and priority of system enhancements as part of the change control process  Lastly, regarding the fifth practice, program officials tested and evaluated FAC-Trax during development, which included validating system enhancements through user acceptance testing  However, program officials did not fully address the third practice— analyzing requirements to ensure they are complete, feasible, and verifiable—because they did not ensure detailed user requirements were necessary and sufficient by tracking them back to higher-level requirements  For example, although program officials reviewed change requests for completeness and followed up with users to verify requirements, officials did not track system enhancements, made in response to detailed user requirements (e g , allowing users to search PA projects by project number), back to the high-level requirements (e g , storing data and information provided by the applicant) identified in the FAC-Trax functional requirements document and performance work statement  Officials did not track system enhancements back to high-level requirements because they did not have a complete understanding of basic user needs and system requirements at the beginning of the FAC- Trax effort, according to the PA program manager  A PA official also said the change control process was a way to identify the basic capabilities FAC-Trax needed to have and that tracking enhancements back to high- level requirements could have made the change control process more difficult to manage, and reduced user participation if, for example, users needed to understand how their change requests related to high-level requirements  However, program officials could have tracked enhancements back to high-level requirements themselves using the change control process without putting any additional burden on users  Despite not having a complete understanding of user needs and system requirements at the beginning of the FAC-Trax effort, analyzing whether users’ change requests satisfy higher-level requirements identified in key design and planning documents would have provided officials with a basis for more detailed and precise requirements throughout project development and helped them better manage the project, according to IT management controls  Further, according to the PMBOK® Guide, tracking or measuring system capabilities against approved requirements is a key process for managing a project’s scope, measuring project completion, and ensuring the project meets user needs and expectations  Program officials acknowledged the importance of tracking system enhancements back to documented system requirements  Ensuring that FAC-Trax meets user needs and expectations is especially important because the information system is key to the success of the new delivery model, according to PA officials  By analyzing progress made on documented, high-level requirements, a step that reflects a key IT management control for requirements development, the PA program will have greater assurance that FAC-Trax will provide functionality that meets user needs and expectations  PA program officials did not fully satisfy either of the two practices in the systems testing and integration control area, according to our assessment  Key systems testing and integration practices are (1) developing test plans and test cases, which include a description of the overall approach for system testing, the set of tasks necessary to prepare for and perform testing, the roles and responsibilities for individuals or groups responsible for testing, and criteria to determine whether the system has passed or failed testing; and (2) developing a systems integration plan to identify all systems to be integrated, describe how integration problems are to be documented and resolved, define roles and responsibilities of all relevant participants, and establish a sequence and schedule for every integration step  In regards to the first practice, PA program officials and the FAC-Trax contractor established a test plan that identifies the method and strategy to perform testing, including the necessary tasks, such as responding to user feedback and testing errors, and incorporating necessary resolutions into future work, testing parameters, and the roles and responsibilities of the individuals responsible for testing  However, program officials have not developed system testing criteria to evaluate FAC-Trax, which would align with the practice described above of using criteria to determine whether the system has passed or failed testing  A key feature of Agile software development is the “definition of done”—a set of clear, comprehensive, and objective criteria, that the government should use to evaluate software after each iteration of development  PA program officials said they did not establish a definition of done because officials initially managing the FAC-Trax effort lacked familiarity with system development in the Agile environment  Officials acknowledged the importance of establishing a definition of done and said they are planning to develop one, but have not identified how or when to incorporate it into the development process  According to the TechFAR—the government’s handbook for procuring digital services using Agile processes—the government and vendor should establish this definition after contract award at the beginning of each cycle of software development  By establishing criteria, such as a definition of done, to evaluate the system—a step that reflects a key IT management control for system testing and is an effective practice for applying Agile to software development—the PA program will have greater assurance that FAC- Trax is usable and responsive to specified requirements  In regards to the second practice, PA program officials developed a systems integration plan in June 2017 that identified the potential for integration of FAC-Trax with four FEMA systems, including EMMIE  In addition, program officials included a description of how staff should document integration problems and the resolution of problems in FAC- Trax development and test plans  However, the systems integration plan does not define roles and responsibilities of all participants for system integration activities or establish a sequence and schedule for every integration step for the four FEMA systems  PA officials said that system integration planning for FAC-Trax is in the early stages, but acknowledged the importance of these elements of system integration planning  Officials plan to define roles and responsibilities of all participants for system integration activities and develop the sequence and schedule for every integration step as they add new systems to the FAC-Trax development plan and obtain funding needed for their integration  Nonetheless, FEMA has used FAC-Trax for selected PA disasters since October 2016 and plans to use FAC-Trax for all future disasters  According to IT management controls, agencies should establish the systems integration plan early in the project and revise it to reflect evolving and emerging user needs  By ensuring that the FAC- Trax systems integration plan defines the roles and responsibilities of relevant participants for all integration relationships and establishes a sequence and schedule for every integration step, the PA program will have greater assurance that FAC-Trax functions properly with other systems and meets user needs  FEMA’s new delivery model enhances participation of hazard mitigation staff with the goal of identifying opportunities for mitigation earlier in the PA preaward process, according to PA officials  Two key changes related to hazard mitigation under the new model include (1) an emphasis on engaging with hazard mitigation specialists at the JFO earlier in the PA process and involving them in specific PA preaward activities and (2) the establishment of the PA program’s hazard mitigation liaison at the CRC  For example, position guides direct program delivery managers to coordinate with FIMA’s hazard mitigation specialists prior to recovery scoping meetings, and site inspectors to coordinate with hazard mitigation specialists prior to site inspections to discuss a PA grant applicant’s damages and any potential mitigation opportunities  PA program officials also developed guidance for conducting the exploratory call and the recovery scoping meeting with applicants, which include questions for PA staff to ask on the applicant’s interest in or plans for incorporating hazard mitigation into potential projects  In addition, a new hazard mitigation liaison at the CRC is responsible for reviewing PA projects for hazard mitigation opportunities and serving as a mitigation subject matter expert for the PA program  According to data provided by FEMA, PA grant applicants incorporated hazard mitigation into approximately 18 percent of permanent work projects for all disasters nationwide from 2012 to 2015  During test implementation of the new delivery model, state, PA, and FIMA officials all reported an increase in the number of hazard mitigation activities on PA permanent work projects  For example, state officials who participated in the second new model test in Oregon said that effective communication and coordination between PA and hazard mitigation staff resulted in applicants incorporating hazard mitigation into over 60 percent of permanent work projects  Furthermore, PA officials reported an increase in hazard mitigation during the third test implementation of the new model in Georgia following Hurricane Matthew, where approximately 16 percent of permanent work projects included mitigation, as of June 2017  This represents an increase compared to the PA program’s estimate for the proportion of projects that incorporate hazard mitigation among previous PA hurricane disasters in Georgia, which was about 3 percent, according to PA officials  While PA officials are trying to increase hazard mitigation through the new delivery model, not all disasters present the same number of opportunities to incorporate hazard mitigation  First, the PA program only incorporates hazard mitigation measures for permanent work projects, such as repairs to roads, bridges, and buildings  For example, as of June 2017, approximately 60 percent of the projects FEMA funded in Georgia for the third test implementation after Hurricane Matthew were for emergency work, which is not eligible for hazard mitigation measures  Second, the PA program only funds mitigation measures that officials determine to be cost-effective  In addition, we have previously reported on other factors that affect whether applicants incorporate hazard mitigation into PA projects, such as their capacity to manage and ability to fund hazard mitigation projects  National Planning for Hazard Mitigation In our 2015 report on disaster resilience following Hurricane Sandy, we noted that disaster affected areas have different threats and vulnerabilities, and local stakeholders make the ultimate determination whether or not to incorporate hazard mitigation into a project  Further, without a strategic approach to making disaster resilience investments, the federal government and its nonfederal partners may be unable to fully capitalize on opportunities for mitigation on the greatest known threats and hazards  We recommended that the Mitigation Framework Leadership Group develop an investment strategy to help ensure that federal funds expended to enhance disaster resilience achieve the goal of reducing the nation’s fiscal exposure because of climate change and the rise in the number of federal major disaster declarations as effectively and efficiently as possible  In response, the Federal Emergency Management Agency (FEMA) plans to issue a final National Mitigation Investment Strategy in 2018  The goals of this strategy include increasing the effectiveness of investments in reducing disaster losses and increasing resilience, and improving coordination of disaster risk management among federal, state, local, tribal, territorial, and private entities  Although the new model establishes hazard mitigation activities for PA and FIMA staff in the preaward process, it does not standardize and prioritize hazard mitigation planning at JFOs in the way FEMA has done with prior PA program policy  Specifically, FEMA’s 2007 PA program policy standardized planning for hazard mitigation across PA recovery efforts by stating that agency and state officials should issue a memorandum of understanding (MOU) early in the disaster, outlining how PA hazard mitigation will be addressed for the disaster, including what mitigation measures will be emphasized, and identifying applicable codes and standards, and any potential integration with other mitigation grant programs  However, PA program officials did not include guidance that standardizes planning for hazard mitigation, such as encouraging the use of an MOU, in FEMA’s 2010 PA program policy, the most recent update to the Public Assistance Program and Policy Guide in April 2017, or the New Delivery Model Operations Manual  As a result, FIMA officials said FEMA and state officials do not consistently issue MOUs that outline how FEMA and the state plan to promote PA hazard mitigation during the recovery effort, explaining that the use of the MOU is based on the preferences and priorities of the FEMA officials involved  When not issuing an MOU, FIMA hazard mitigation staff and PA officials at the JFO meet to determine the extent which hazard mitigation staff interact directly with applicants regarding PA hazard mitigation during the recovery process, according to a FIMA official  Having a consistent approach to planning for and prioritizing hazard mitigation across all disasters is important for FEMA, given that FEMA experienced challenges consistently prioritizing and integrating hazard mitigation across PA recovery efforts, according to GAO and others  For example, in our 2015 report on resilience in the Hurricane Sandy recovery, we found that state and local officials experienced challenges maximizing disaster resilience in the recovery effort because PA officials did not consistently prioritize hazard mitigation during project development  According to FEMA’s National Mitigation Framework, planning is vital for mitigation efforts during disaster recovery, and federal, state, and local officials should establish procedures that emphasize a coordinated delivery of mitigation activities and capitalize on opportunities to reduce future disaster losses  Similarly, the Recovery Federal Interagency Operational Plan, which supports FEMA’s National Disaster Recovery Framework, identifies planning as a key task for identifying mitigation opportunities and integrating risk reduction considerations into decisions and investments during the recovery process  FIMA officials agreed that including the development of a formal plan, such as the historical 2007 PA program policy regarding the use of MOUs, for PA hazard mitigation in operations guidance would help program officials plan for and prioritize hazard mitigation  They noted that FIMA’s hazard mitigation field operations guide includes procedures for implementing proposed MOUs to achieve mitigation goals  PA program officials said that, in light of changes to the PA process under the new model and subsequent updates to program policies, the MOU policy from the 2007 PA program policy was outdated  But officials agreed that planning for and prioritizing hazard mitigation at the operational level is important and said they were examining additional ways to incorporate these activities early in the PA process  As FEMA continues to implement the new model, establishing procedures to standardize hazard mitigation planning for each disaster, as it did through prior policy, could improve the prioritization of hazard mitigation in PA recovery efforts and increase the effectiveness of mitigation for reducing disaster losses and increasing resilience  PA program officials developed performance objectives and measures for hazard mitigation in the new delivery model, but could add measures to better align performance assessment for the PA program with FEMA’s broader strategic goals for hazard mitigation  In its strategic plan for 2014–2018, FEMA established an agency-wide goal to increase the percentage of FEMA-funded disaster projects, such as those under the PA program, that provide mitigation above local, state, and federal building code requirements by 5 percentage points by the end of fiscal year 2018  For example, local building codes may require measures for new construction to mitigate against future damage  To align with FEMA’s strategic goal, PA officials would also need to measure the number of PA projects that included mitigation measures that bring any repaired infrastructure to a level above applicable building codes  However, under the new model, FEMA officials developed performance objectives (and associated measures) to increase the number of projects that include hazard mitigation by 5 percent, and increase the total dollars spent on hazard mitigation by 2 percent  While these measures could help to incentivize mitigation, they are not tied to building codes and do not include specific information that FEMA could use to continually assess the PA program’s contributions to meeting the agency’s strategic goal  According to Standards for Internal Control in the Federal Government, agency management should design control activities, such as establishing and reviewing performance measures, to achieve the agency’s objectives  In addition, our work on leading public sector organizations has found that such organizations assess the extent to which their programs and activities contribute to meeting their mission and desired outcomes, and strive to establish clear hierarchies of performance goals and measures  A clear connection between performance measures and program offices helps to both reinforce accountability and ensure that, in their day-to-day activities, managers keep in mind the outcomes their organization is striving to achieve  FEMA’s ability to evaluate and report on PA hazard mitigation data is constrained, but officials are addressing this challenge through the development of data reporting and analytics capabilities for the PA program’s new information system, according to PA officials  PA program officials developed measures they could use to evaluate the new model during test implementation and compare new model performance to the legacy PA process, and agreed that aligning PA program hazard mitigation goals with FEMA’s agency-wide strategic goals would be helpful  As FEMA continues to develop and implement the new model, developing performance measures and objectives to better inform and support the agency’s broader strategic goals could help to ensure that FEMA capitalizes on hazard mitigation opportunities in PA recovery efforts  FEMA’s Public Assistance grant program is a complicated, multi-billion dollar program that is critical to helping state and local communities rebuild and recover after a major disaster  In recent years, FEMA has undertaken a major reengineering effort to make the PA preaward process simpler and more efficient for applicants and to address challenges encountered during recovery from past disasters  FEMA’s new delivery model represents a significant opportunity to strengthen the PA program and address these past challenges, and growing pains are to be expected when implementing any large reengineering effort  Further, FEMA officials work to implement these changes while supporting response and recovery following disasters, including the catastrophic flooding from Hurricane Harvey in August 2017 and widespread damages from Hurricanes Irma and Maria in September 2017  As such, it is critical that feedback obtained and lessons learned while testing the new model inform decisions and actions as FEMA proceeds with full implementation for all disasters, including the complex recovery efforts in the states and territories affected by Hurricanes Harvey, Irma, and Maria  FEMA has redesigned the PA delivery model to address various challenges related to workforce management, information sharing with state and local grantees, and incorporating hazard mitigation into PA projects  FEMA has developed new workforce processes, training, and positions to address past challenges, but completing a workforce assessment that identifies the number of staff needed will inform workforce management and resource allocation decisions to help FEMA ensure a more successful implementation  This is particularly important as FEMA is using the new model for the long-term recovery from the 2017 hurricanes, and FEMA faces capacity challenges as its workforce is stretched thin  Further, FEMA’s new FAC-Trax information sharing system provides FEMA and state and local applicants and grantees with better capabilities to address past challenges in managing and tracking PA projects  In developing FAC-Trax, FEMA implemented many of the key IT management controls that can help ensure that new IT systems are implemented effectively  However, additional steps are needed to fully satisfy the areas of requirements development and systems testing and integration  Finally, FEMA has taken some actions to better promote hazard mitigation as part of its new PA model  However, more consistent planning for hazard mitigation following a PA disaster and developing specific performance measures and objectives that better align with and support the agency’s broader strategic goals related to hazard mitigation could help to ensure that mitigation is incorporated into recovery efforts, which presents an opportunity to encourage disaster resilience and reduce federal fiscal exposure from recurring catastrophic natural disasters  We are making the following five recommendations to FEMA’s Assistant Administrator for Recovery: The FEMA Assistant Administrator for Recovery should complete a workforce staffing assessment that identifies the appropriate number of staff needed at joint field offices, Consolidated Resource Centers, and in FIMA’s hazard mitigation cadre to implement the new delivery model nationwide  (Recommendation 1) The FEMA Assistant Administrator for Recovery should establish controls for tracking FAC-Trax capabilities to the system’s functional and operational requirements to more fully satisfy requirements development controls and ensure that the new information system provides capabilities that meets users’ needs and expectations  (Recommendation 2) The FEMA Assistant Administrator for Recovery should establish system testing criteria, such as a “definition of done,” to assess FAC- Trax as it is developed; define the roles and responsibilities of all participants; and develop the sequence and schedule for integration of other systems with FAC-Trax to more fully satisfy systems testing and integration controls  (Recommendation 3) The FEMA Assistant Administrator for Recovery, in coordination with the Associate Administrator of the Federal Insurance and Mitigation Administration, should implement procedures to standardize planning for addressing PA hazard mitigation at the joint field offices, for example, by requiring FEMA and state officials to develop a memorandum of understanding outlining how they will prioritize and address hazard mitigation following a disaster as it did through prior policy  (Recommendation 4) The FEMA Assistant Administrator for Recovery, in coordination with the Associate Administrator of the Federal Insurance and Mitigation Administration, should develop performance measures and associated objectives for the new delivery model to better align with FEMA’s strategic goal for hazard mitigation in the recovery process  (Recommendation 5) We provided a draft of this report to DHS and FEMA for review and comment  DHS provided written comments, which are reproduced in appendix III  In its comments, DHS concurred with our recommendations and described actions planned to address them  FEMA also provided technical comments, which we incorporated as appropriate  With regard to our first recommendation, that FEMA complete a workforce staffing assessment that identifies the number of staff needed at joint field offices, Consolidated Resource Centers, and FIMA’s hazard mitigation cadre, DHS stated that PA, in coordination with the Field Operations Directorate and FIMA, will continue to refine and evaluate staffing needs and update the cadre force structures under the new delivery model  DHS estimated that this effort would be completed by June 28, 2019  This action, if fully implemented, should address the intent of the recommendation  With regard to our second recommendation, that FEMA establish controls for tracking FAC-Trax capabilities to ensure the new information system meets users’ needs, DHS stated that Recovery program managers will update the FAC-Trax Requirements Management Plan and the FAC-Trax Release Plan to ensure the tracking and traceability of FAC-Trax functional and operational requirements  DHS estimated that this effort would be completed by January 31, 2018  This action, if fully implemented, should address the intent of the recommendation  With regard to our third recommendation, that FEMA establish systems testing criteria to assess the development of FAC-Trax; and define the roles and responsibilities, and sequence and schedule for system integration, DHS stated that Recovery program managers will update the FAC-Trax System Integration Plan to include integration with the Deployment Tracking System, Enterprise Data Warehouse, Preliminary Damage Assessment interface, and State Grants Management system interface  DHS estimated that this effort would be completed by June 29, 2018  This action, if fully implemented, should address the intent of the recommendation  With regard to our fourth recommendation, that FEMA implement procedures to standardize planning for addressing PA hazard mitigation at the JFO, DHS stated that PA will update current process documents or develop new documents to better incorporate mitigation into the operational planning phase of the new delivery model  DHS estimated that this effort would be completed by July 31, 2018  This action, if fully implemented, should address the intent of the recommendation  With regard to our fifth recommendation, that PA coordinate with FIMA to develop performance measures and associated objectives for the new delivery model that better align with FEMA’s strategic goals for hazard mitigation in the recovery process, DHS stated that PA will reconvene the PA-Mitigation working group to develop and refine PA related hazard mitigation performance measures  DHS estimated that this effort would be completed by June 29, 2018  This action, if fully implemented, should address the intent of the recommendation  We are sending copies of this report to the Secretary of Homeland Security and interested congressional committees  If you or your staff have any questions about this report, please contact me at (404) 679-1875 or CurrieC@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix IV  Appendix II: Assessment of Information Technology Management Controls for the FEMA Applicant Case Tracker (FAC-Trax) Table 2 shows details on the Federal Emergency Management Agency (FEMA) Public Assistance (PA) program office’s implementation of key practices across four information technology (IT) management control areas for its new information system, the FEMA Applicant Case Tracker (FAC-Trax)  PA developed FAC-Trax as a web-based project tracking and case management system to supplement the Emergency Management Mission Integrated Environment (EMMIE) and help resolve long-standing information sharing challenges  To determine the extent to which the FAC-Trax program office implemented IT management controls, we reviewed documentation from the FAC-Trax program and compared it to key management best practices, including the Software Engineering Institute’s Capability Maturity Model® Integration for Acquisition and Development, the Project Management Institute’s Guide to the Project Management Body of Knowledge (PMBOK® Guide), and the Institute of Electrical and Electronics Engineers’ Standard for Software and System Test Documentation  We assessed the program as having fully implemented a practice if the agency provided evidence that it fully addressed the practice; partially implemented if the agency provided evidence that it addressed some, but not all, portions of the practice; and not implemented if the agency did not provide any evidence that it addressed the practice  Table 2  Public Assistance (PA) Program Office’s Implementation of Key Information Technology Management Controls for FAC-Trax PA program officials developed an acquisition plan for FAC-Trax identifying the capabilities the system is intended to deliver, the acquisition approach, and acquisition objectives  Additionally, program officials developed a capability development plan outlining a strategy for the program to obtain approval to acquire FAC-Trax  Lastly, program officials developed a systems engineering plan describing the program’s scope and its framework for using an Agile development approach, as well as a deployment, support, and maintenance plan for FAC-Trax  PA program officials developed an acquisition program baseline detailing FAC-Trax’s cost parameters and a life-cycle cost estimate for the system  As of May 2017, the life- cycle cost estimate for FAC-Trax through fiscal year (FY) 2023 is approximately $19 3 million  PA program officials updated the life-cycle cost estimate for FYs 2016 and 2017 after price negotiations with the FAC-Trax contractor, and will continue to update the estimate as annual budgets are approved, according to the Integrated Logistic Support Plan  The contracting officer’s representative for FAC-Trax performs a cost review at the end of each month, according to program officials  Furthermore, the contractor’s weekly status report includes information on the number of hours worked and the percent of contract value spent  Program officials also review program costs with Office of Response and Recovery, PA, Office of the Chief Information Officer (OCIO), and other program office stakeholders during a weekly program review  PA program officials developed an acquisition program baseline detailing FAC-Trax’s schedule parameters, as well as an integrated master schedule for the system  The integrated master schedule identifies tasks, major milestones, and task dependencies  The PA program manager reviews and updates the integrated master schedule on a weekly basis  Program officials also review FAC-Trax’s schedule with Office of Response and Recovery, PA, OCIO, and other program office stakeholders during a weekly program review  PA program officials identified the knowledge and skills needed to carry out the program in FAC-Trax contract documentation and the capability development plan  Specifically, program officials included an attachment to the FAC-Trax contract listing the required labor categories and corresponding functional position descriptions  Program officials also described the role, position type, minimum grade, and minimum certification for required personnel resources for the acquisition, development, and implementation of FAC-Trax  PA program officials developed, reviewed, and maintained project planning documents and obtained commitment from relevant stakeholders  For example, program officials reviewed and updated the integrated master schedule and costs on a weekly and monthly basis, respectively  Further, program officials reviewed the status of project elements, such as the schedule, quality and technical issues, stakeholders, staffing, cost, and risks, with Office of Response and Recovery, PA, OCIO, and other program office stakeholders during a weekly program review  PA program officials also established tactical, functional, and stakeholder groups, as well as an Integrated Product Team to support and oversee the development of FAC-Trax  FEMA’s Recovery Technology Programs Division (RTPD) has a division-level risk management plan that serves as guidance for all Recovery systems, including FAC- Trax  Program officials identified key risks that could negatively affect FAC-Trax work efforts in RTPD’s “risk register”—an online site used to track risks, issues, and mitigating actions for the division and each program office  Program officials also identified five technical, cost, and schedule risks in the FAC-Trax acquisition plan  Program officials included one of these risks in the risk register, while the remaining four were managed outside of the register  As of May 2017, program officials had identified 13 risks in its risk register—four open and nine closed  The four open risks were (1) limited subject matter expert engagement during requirements development, (2) vacancies in program management office support positions, (3) unresolved service level agreement support and funding issues, and (4) the loss of the authority to operate due to a Trusted Internet Connection that is not compliant with Department of Homeland Security security policy  Program officials evaluated and categorized the identified risks based on the probability of occurrence and scope, schedule, and cost impacts  These four points of measurement are used to calculate an overall risk score  The risk score helps program officials determine a risk’s risk rating—low, medium, or high  For example, program officials reported that two of its open risks have a “medium” risk rating—meaning the risk has the potential to slightly impact project cost, schedule, or performance  In addition, program officials detailed the risk category, probability, and impact for the five risks identified in the FAC-Trax acquisition plan  Program officials developed risk mitigation and contingency plans for each risk in the risk register  For example, program officials planned to mitigate the open risk concerning subject matter expert engagement, by identifying and engaging with appropriate subject matter experts through requirements development workshops scheduled in advance of the sprint they are to support, and monitoring the development of user stories to identify any issues that may cause delays  In addition, program officials described the risk management plan and responsible officials for the five risks identified in the FAC-Trax acquisition plan  PA program officials review and update program risks during a monthly program meeting  Program officials also review program risks with Office of Response and Recovery, PA, OCIO, and other program office stakeholders during a weekly program review  Furthermore, the FAC-Trax contractor provides a weekly status update which includes a section on identified risks  Program officials established re-evaluation dates and recorded updates, including any actions taken, for each risk in the risk register  In addition, program officials were able to provide updates on the four risks identified in the FAC-Trax acquisition plan and managed outside of the register  According to PA officials, these risks were addressed and closed by the approval of program planning documents, such as the mission needs statement, concept of operations, and operational requirements document, following the solutions engineering review, which demonstrates the readiness of the program to proceed with the procurement, in September 2016  Program officials established a requirements management plan outlining how it captures, assesses, and plans for FAC-Trax enhancements, and established a change control process to review, prioritize, and verify user requests for changes to the system and feedback  As of May 2017, the PA program office received 734 change requests related to FAC-Trax, of which program officials completed 420 changes and planned to address an additional 277 entries  PA program officials also facilitated workshops to gather requirements for specific user groups and obtained additional requirements for FAC-Trax through customer feedback on a temporary technology tool— an Access database referred to as the Public Assistance Recovery Information System—used to support an early stage of the new model implementation  Further, program officials developed a functional requirements document outlining the high-level functional and operational requirements for FAC-Trax  PA program officials developed a concept of operations for FAC-Trax detailing operating concepts and scenarios for each phase of the PA preaward process  Program officials also detailed the workflow, phases, business functions, and data inputs and outputs for the re-engineered PA process in FAC-Trax’s functional requirements document  In March 2017, program officials developed a standard template to describe the process, tasks, and data inputs and outputs for specific system capabilities  As part of the change control process, PA program officials meet three times a week to discuss and prioritize change requests  Specifically, program officials review submissions to the change control form to ensure completeness, validate impacts and root cause, and research details for incoming requests  PA program officials also follow up with users to understand and verify requirements  In March 2017, program officials developed a standard template to capture acceptance criteria for specific requirements  However, PA program officials do not track system enhancements back to the high-level requirements identified in FAC-Trax’s operational and functional requirements documentation and performance work statement  PA program officials identified system requirements and constraints in the FAC-Trax concept of operations and functional and operational requirements documents  Further, through its change control process, program officials collect suggestions, issues, and feedback on FAC-Trax and system enhancements from stakeholders, identify risks for change requests, and balance prioritized requirements and estimated level of efforts with projected costs prior to each sprint  In March 2017, program officials developed a standard template to analyze and document the urgency and need for specific requirements  PA program officials and the FAC-Trax contractor established a testing and evaluation plan for the system, developed acceptance criteria for user stories, and obtained feedback from users during and after testing  The testing process concludes with user acceptance testing (UAT)  If a change request fails during UAT or a new requirement is discovered during development, the PA program will capture the failed request or new requirement in the product backlog for implementation in a future product release  Key practices Systems testing and integration Developing test plans and test cases PA program officials and the FAC-Trax contractor tested and evaluated the system during development  The FAC-Trax test plan identifies the method and strategy to perform the testing, including the necessary tasks, testing parameters, and the roles and responsibilities of the individuals responsible for testing  However, program officials did not develop system testing criteria to evaluate FAC-Trax  A key feature of Agile software development is the “definition of done”—a set of clear, comprehensive, and objective criteria, that the government should use to evaluate software after each iteration of development  PA program officials developed a systems integration plan in June 2017 that identifies potential integration of FAC-Trax and four FEMA systems, including the Emergency Management Mission Integrated Environment  Specifically, the plan includes data requirements and standards; descriptions of the four systems FEMA plans to integrate with FAC-Trax and the proposed relationship for each connection; and security and access management requirements  In addition, program officials included a description of how integration problems are to be documented and resolved in FAC-Trax development and test plans  However, the systems integration plan does not define roles and responsibilities of all participants for system integration activities or establish a sequence and schedule for every integration step for the four FEMA systems  ● Fully implemented: The agency provided evidence that it fully addressed this practice  ◐ Partially implemented: The agency provided evidence that it addressed some, but not all, portions of this practice  ◌ Not implemented: The agency did not provide any evidence that it addressed this practice  In addition to the contact named above, Chris Keisling (Assistant Director), Amanda R  Parker (Analyst-in-Charge), Mathew Bader, Allison Bawden, Anthony Bova, Eric Hauswirth, Susan Hsu, Rianna Jansen, Justin Jaynes, Tracey King, Matthew T  Lowney, Heidi Nielson, Claire Peachey, Brenda Rabinowitz, Ryan Siegel, Martin Skorczynski, Niti Tandon, Walter K  Vance, James T  Williams, and Eric Winter made key contributions to this report 


Report 2: The Secret Service plays a critical role in protecting the President, Vice President, their immediate families, and national leaders, among others  In addition, the component is responsible for safeguarding the nation’s currency and financial payment systems  To accomplish its mission, Secret Service officials reported that, as of June 2018, the component had approximately 7,100 employees (including the Uniformed Division, special agents, and administrative, professional, and technical staff)  These employees were assigned to the component’s headquarters in Washington, D C , and 133 field offices located throughout the world (including 115 domestic offices and 18 international offices)  The Secret Service’s employees are heavily dependent on the component’s IT infrastructure and communications systems to perform their daily duties  According to data reported on the Office of Management and Budget’s IT Dashboard, the component planned to spend approximately $104 8 million in fiscal year 2018 to modernize and maintain its IT environment  To manage this IT environment, the Secret Service hired a full-time CIO in November 2015  In addition, in an effort to improve its management structure, the component consolidated all IT staff and assets under this new CIO in March 2017  OCIO officials stated that these staff include the government employees who provide direct and indirect support of the day-to-day operations of the Secret Service’s enterprise systems and services  According to Secret Service officials, the component’s IT workforce included 190 staff, as of July 2018  These officials stated that 166 of these employees were located in the component’s headquarters in Washington, D C , and 24 were located in domestic field offices  The officials also reported that these July 2018 staffing levels were below their current approved staffing level of 220 staff (which included 44 positions in domestic field offices)  Secret Service IT staff also deploy to other locations, as necessary, to provide support for certain security activities  For example, the Secret Service reported that, in 2017, OCIO deployed over 79 staff to New York, N Y , to provide communications support during the United Nations General Assembly  As a component of DHS, the Secret Service must follow the department’s policies and processes for managing acquisitions, including IT acquisitions  DHS categorizes its acquisition programs according to three levels that are determined by the life cycle costs of the programs  These levels then determine the extent of required program and project management and the acquisition decision authority (the individual responsible for management and oversight of the acquisition)  The department also categorizes its acquisition programs as major or non- major based on expected cost  Table 1 describes the levels of DHS’s acquisition programs and their associated acquisition decision authorities  DHS’s policies and processes for managing major acquisition programs are primarily set forth in its Acquisition Management Directive 102-01 and Acquisition Management Instruction 102-01-001  In particular, these policies establish that a major acquisition program’s decision authority is to review the program at a series of predetermined acquisition decision events to assess whether the program is ready to proceed through the acquisition life cycle phases  Figure 1 depicts the acquisition life cycle established in DHS acquisition management policy  DHS’s Acquisition Management Directive and Instruction do not establish an acquisition life cycle framework for the department’s non-major acquisition programs  Instead, according to the Instruction, Component Acquisition Executives (i e , the senior acquisition official within a component that is responsible for implementation, management, and oversight of the component’s acquisition process) are required to establish component-specific non-major acquisition policies and guidance that support the “spirit and intent” of the department’s acquisition policies  To that end, the Secret Service developed a policy that establishes an acquisition life cycle framework for its non-major acquisition programs  This acquisition framework for the component’s non-major acquisition programs is consistent with the acquisition framework that DHS established for its major acquisition programs  In particular, the Secret Service’s framework includes the same phases and decision events as DHS’s framework (e g , acquisition decision event 2A, the point at which the acquisition decision authority determines whether a program may proceed into the obtain phase)  In addition, DHS’s Systems Engineering Life Cycle Instruction and Guidebook outline a framework of major systems engineering activities and technical reviews that are to be conducted by all DHS programs and projects, both major and non-major  This framework is intended to ensure that appropriate systems engineering activities are planned and implemented, and that a program’s development effort is meeting the business need  In particular, the systems engineering life cycle framework consists of nine major activities (e g , requirements definition, integration, and testing) and a set of related technical reviews (e g , preliminary design review) and artifacts (e g , requirements documents)  DHS policy allows programs to tailor these activities, technical reviews, and artifacts based on the unique characteristics of the program (e g , scope, complexity, and risk)  For example, a program may combine systems engineering technical reviews and artifacts, or add additional reviews  This tailored approach must be documented in a program’s systems engineering life cycle tailoring plan  The systems engineering technical reviews are intended to provide DHS the opportunity to determine how well a program has completed the necessary systems engineering activities  Each technical review includes a minimum set of exit criteria that must be satisfied before a program may move on to the next systems engineering activity  At the end of the technical review, the program manager must develop a technical review completion letter that documents the outcome of the review, including stakeholder concurrence that the exit criteria were satisfied  Moreover, DHS’s agile instruction, which was first issued in April 2016 and updated in April 2018, identifies agile as the preferred development approach for the department’s IT programs and projects  Agile is a type of incremental (i e , modular) development, which calls for the rapid delivery of software in small, short increments rather than in the typically long, sequential phases of a traditional waterfall approach  DHS’s agile instruction also states that component CIOs are to set modular (i e , incremental) outcomes and target measures to monitor progress in achieving agile implementation for IT programs and projects  To that end, the department identified core metrics that its agile IT programs are to use to monitor progress, including the number of story points completed per release and the number of releases per quarter  Further, DHS policy and guidance have established an acquisition (i e , contract) review process that is intended to enable the DHS CIO to review and effectively guide the department’s IT expenditures  According to the department’s IT acquisition review guidance, DHS components with a CIO (which includes the Secret Service) are to submit to DHS OCIO for review, IT acquisitions that (1) have total estimated procurement values of $2 5 million or more; and (2) are funded by a level 1, 2, or 3 program with a life cycle cost estimate of at least $50 million (i e , a major investment, as defined by DHS’s capital planning and investment control guidance)  DHS policies and guidance also establish numerous responsibilities for the department’s component-level CIOs that are aimed at ensuring proper oversight and management of the components’ IT investments  Among other things, these component-level CIO responsibilities relate to topics such as IT budgeting, portfolio management, and oversight of programs’ systems engineering life cycles  Table 2 identifies 14 selected IT oversight responsibilities for DHS’s component CIOs  The Secret Service acquires IT infrastructure and services that are intended to improve its ability to execute its investigation and protection missions  According to data reported on the Office of Management and Budget’s IT Dashboard, the Secret Service planned to spend about $104 8 million on IT in fiscal year 2018, which included approximately $34 6 million for the development and modernization of its IT infrastructure and services, and about $70 2 million for the operations and maintenance of this infrastructure (including 21 existing IT systems)  Also according to data reported on the IT Dashboard, as of April 2018, the Secret Service had one major IT investment (called the Information Integration and Technology Transformation and discussed in more detail later in this report), seven non-major IT investments, and one non- standard infrastructure investment  Figure 2 depicts the Secret Service’s planned IT spending for fiscal year 2018  The Secret Service has faced long-standing challenges in managing its IT infrastructure  For example, A National Security Agency audit of the Secret Service’s IT environment in 2008 identified network and system vulnerabilities that needed immediate remediation to protect the component’s systems and electronic information  The Secret Service determined in 2010 that it had IT capability gaps associated with three key areas: network security, information sharing and situational awareness, and operational communications  The component reported that it required a significant IT modernization effort with sustained investment of resources to replace dated and restrictive network and communications capabilities  The Secret Service also reported in 2010 that it had 42 mission- support applications that were operating on a 1980’s mainframe that lacked multi-level security (i e , the ability to view classified information from two security levels, such as secret and top secret, at the same time), was beyond its equipment life cycle, and was at risk of failing  Further, in 2011, DHS’s Office of Inspector General reported that the Secret Service’s existing infrastructure did not meet current operational requirements  According to the Secret Service, this dated infrastructure was unable to support newer technologies (e g , Internet protocol), share common DHS enterprise services, or migrate to the department’s consolidated data centers  To address challenges with its IT environment, in 2009, the Secret Service initiated the IITT investment, which is intended to modernize and enhance the component’s infrastructure, communications systems, applications, and processes  In particular, IITT is a portfolio of programs and projects that are meant to, among other things, improve systems availability in support of the Secret Service’s business operations, increase interoperability with other government systems and networks, enhance the component’s system and network security, and enable scalability to support growth  From 2010 to July 2018, according to OCIO officials, the Secret Service spent approximately $392 million on IITT  In fiscal year 2018, the component had planned to spend approximately $42 7 million on IITT (i e , about 40 percent of its total planned IT spending for the fiscal year), according to data reported on the Office of Management and Budget’s IT Dashboard  In total, the planned life cycle cost estimate for IITT is at least $811 million  As of June 2018, IITT was a major investment comprised of two programs (one of which included three projects) and one standalone project (i e , it was not part of another program) that had capabilities that were in planning or development and modernization  These programs and project were the Enabling Capabilities program, Enterprise Resource Management System program (which included three projects that were each being implemented using an agile methodology: Uniformed Division Resource Management System, Events Management, and Enterprise-wide Scheduling), and the Multi-Level Security project  Table 3 describes the IITT programs and projects that had capabilities that were in planning or development and modernization, as of June 2018  The table also includes the associated level, acquisition decision authority, estimated life cycle costs, and planned or actual dates of operational capability for each of the programs and projects  (Appendix II also provides additional information on these programs and projects ) The Enabling Capabilities program within IITT is designated as a major acquisition program  As such, its acquisition decision authority is the DHS Under Secretary for Management, and both DHS and the Secret Service provide oversight to this program  IITT’s other program and project—the Enterprise Resource Management System program (which includes three projects, as discussed earlier) and Multi-Level Security project—are designated non-major acquisition programs  In June 2011, DHS’s Under Secretary for Management delegated acquisition decision authority for this non-major program and project to the Secret Service Component Acquisition Executive  As such, oversight of the Enterprise Resource Management System program (including its three projects) and the Multi- Level Security project is conducted primarily at the component level  The Secret Service also implemented other capabilities that are now in operations and maintenance (i e , the capabilities have been fielded and are operational) as part of the IITT investment, such as a capability to move data between systems in separate classification levels (e g , top secret and secret) and communications interoperability  Table 4 describes IITT capabilities that are in operations and maintenance  DHS, including the Secret Service, has faced long-standing challenges in effectively managing its workforce  In January 2003, we designated the implementation and transformation of DHS as high risk, including its management of human capital, because it had to transform 22 agencies—several with major management challenges—into one department  This represented an enormous and complex undertaking that would require time to achieve in an effective and efficient manner  Since that time, the department has made important progress in strengthening and integrating its management functions  Nevertheless, we have continued to report that significant work remains for DHS to improve these management functions  Among other things, we previously reported that the department had lower average employee morale than the average for the rest of the federal government  We also reported that, in 2011, based on employee responses to the Office of Personnel Management’s Federal Employee Viewpoint Survey—a tool that measures employees’ perceptions of whether and to what extent conditions characterizing successful organizations are present in their agency—DHS was ranked 31st out of 33 large agencies on the Partnership for Public Service’s Best Places to Work in the Federal Government rankings  The most recent results of these surveys in 2017 showed that DHS continues to maintain its low rankings  DHS’s Office of Inspector General has reported on challenges that the Secret Service has faced in managing its IT workforce  Specifically, in October 2016, the Inspector General reported that the Secret Service CIO did not have oversight of, or authority over, all IT resources, including the workforce; in particular, almost all of the component’s IT employees were located in a division outside of OCIO; and the Secret Service had vacancies in key positions responsible for managing IT, including not having a full-time CIO from December 2014 through November 2015  As previously discussed, the Secret Service has taken actions to address these two issues with the management of its IT workforce  These actions included hiring its full-time CIO in November 2015 and consolidating the workforce and all IT assets under this CIO in March 2017  Of the 14 selected responsibilities established for component-level CIOs in DHS’s IT management policies, the Secret Service CIO had fully implemented 11 responsibilities and had partially implemented 3 responsibilities  Table 5 summarizes the extent to which the Secret Service CIO had implemented each of the 14 responsibilities  The Secret Service CIO fully implemented 11 of the 14 selected component-level CIO responsibilities  Examples of the responsibilities that the CIO fully implemented are as follows: Develop, implement, and maintain a detailed IT strategic plan  Consistent with DHS’s IT Integration and Management directive, in January 2017, the Secret Service CIO developed an IT strategic plan that outlined the CIO’s strategic IT goals and objectives, as well as tasks intended to meet the goals and objectives  The CIO maintained this strategic plan, to include updating it in January 2018  The CIO also took steps to implement the tasks identified within the strategic plan, such as working to develop an IT training program  In particular, as part of this effort to develop an IT training program, OCIO identified recommended training for the office’s various IT workforce groups (discussed in more detail later in this report)  Concur with each program’s and/or project’s systems engineering life cycle tailoring plan  In accordance with DHS’s Systems Engineering Life Cycle instruction, the Secret Service CIO concurred with the systems engineering life cycle tailoring plan for one program and three projects included in the Secret Service’s IITT investment  Specifically, the CIO documented his approval via his signature on the tailoring plans for IITT’s Enabling Capabilities program, and Multi-Level Security, Uniformed Division Resource Management System, and Events Management projects  Participate on DHS’s CIO Council, Enterprise Architecture Board, or other councils/boards as appropriate, and appoint employees to serve when necessary  As required by DHS’s IT Integration and Management directive, the Secret Service CIO participated on two required DHS-level councils/boards, and appointed a delegate to serve in his place, when necessary  Specifically, the Secret Service CIO or the CIO’s delegate—the Deputy CIO—attended bi-monthly meetings of the DHS CIO Council  In addition, another Secret Service CIO appointee—the component’s Chief Architect—attended an ad hoc meeting of the Enterprise Architecture Board in June 2017  In addition, the Secret Service CIO had partially implemented three component-level CIO responsibilities, as follows  Manage the component IT investment portfolio, including establishing a component-level IT acquisition review process that enables component and DHS review of component acquisitions (i e , contracts) that contain IT  As directed in DHS’s Capital Planning and Investment Control directive and guidebook, the Secret Service CIO took steps to manage the component’s IT investment portfolio, including reviewing certain contracts containing IT  For example, among our random sample of 33 IT contracts that the Secret Service awarded between October 1, 2016, and June 30, 2017, we found that the CIO or the CIO’s delegate had reviewed 31 of these contracts  However, the CIO had not established and documented a defined process for reviewing contracts containing IT, which may have contributed to why the CIO or the CIO’s delegate did not review 2 of the 33 contracts in our sample  OCIO officials were unable to explain why neither of these officials reviewed the 2 contracts, which had a combined planned total procurement value of approximately $1 75 million  In particular, one of the contracts, with a planned total procurement value of about $1,122,934, was to provide credentialing services for the 2017 Presidential Inauguration  The other contract, with a planned total procurement value of about $629,337, was to provide maintenance support for a logistics system  The OCIO officials acknowledged that both contracts should have been approved by one of these officials  Without establishing and documenting an IT acquisition review process that ensures that the CIO or the CIO’s delegate reviews all contracts containing IT, as appropriate, the CIO’s ability to analyze the contracts to ensure that they are a cost-effective use of resources and are aligned with the component’s missions and goals is limited  Ensure all component IT policies are in compliance and alignment with DHS IT directives and instructions  As required by DHS’s IT Integration and Management directive, the Secret Service CIO had ensured that certain component IT policies were in compliance and alignment with DHS IT directives and instructions  For example, in alignment with the department’s IT Integration and Management directive, the Secret Service’s Investment Governance for IT policy specifies that the component CIO (in conjunction with each Secret Service Office) is responsible for developing the component IT spend plan, as well as developing and maintaining an IT strategic plan  However, the Secret Service’s enterprise governance policy was not in compliance with DHS’s IT Integration and Management directive  Specifically, while the department’s policy states that the Secret Service CIO is responsible for developing and reviewing the component’s IT budget formulation and execution, the Secret Service’s enterprise governance policy does not specify this as the CIO’s responsibility  According to OCIO officials, the Secret Service CIO participates in the development and review of the IT budget formulation and execution as a member of the Executive Resources Board (the Secret Service’s highest-level governing body, which has the final decision authority and responsibility for enterprise governance), and the Secret Service Deputy CIO is a voting member of the Enterprise Governance Council (the Secret Service’s second-level governance body and advisory council to the Executive Resources Board)  However, the Secret Service’s enterprise governance policy has not been updated to reflect these roles  The Secret Service did not update its enterprise governance policy to properly reflect the CIO’s and Deputy CIO’s roles on the Executive Resources Board or Enterprise Governance Council because OCIO officials were not aware that these roles were not properly documented in the component’s policy until we identified this issue during our review  Further compounding the issue of the Secret Service’s enterprise governance policy not properly reflecting the CIO’s and Deputy CIO’s roles and responsibilities on the component’s governance boards is that the Secret Service has not developed a charter for its Executive Resources Board  We have previously reported that a best practice for effective investment management is to define and document the board’s membership, roles, and responsibilities  One such way to do so is via a charter  According to Secret Service officials, the component does not have a charter for the board because, while the Secret Service has established the board pursuant to law, there is little statutory guidance on how the board must be formalized, including whether a charter is required  The officials acknowledged that development of a board charter is a best practice  They stated that, in response to our review, the component has begun efforts to develop a charter for the Executive Resources Board, but they did not know when it would be completed  Until the Secret Service updates its enterprise governance policy to specify (1) the CIO’s current role and responsibilities on the Executive Resources Board, to include developing and reviewing the IT budget formulation and execution, and (2) the Deputy CIO’s role and responsibilities on the Enterprise Governance Council, the CIO’s ability to develop and review the component’s IT budget may be limited  Further, until the Secret Service develops a charter for its Executive Resources Board that specifies the roles and responsibilities of all board members, including the CIO, the Secret Service will not be effectively positioned to ensure that all members understand their roles and responsibilities on the board and will perform them as expected  Set modular outcomes and target measures to monitor the progress in achieving agile implementation for IT programs and/or projects within their component  Consistent with DHS policy, the Secret Service CIO has set modular outcomes and target measures to monitor the progress of two IITT projects that the component is implementing using an agile methodology—Uniformed Division Resource Management System and Events Management  For example, the modular outcomes set for these projects included measuring planned and actual burndown (i e , the number of user stories completed)  In addition, the projects were to measure their velocity (i e , the rate of work completed) for each sprint (i e , a set period of time during which the development team is expected to complete tasks related to developing a piece of working software)  However, the modular outcomes and target measures did not include product quality or post-deployment user satisfaction, although such measures are leading practices for managing agile projects  According to Secret Service OCIO officials, the component does not mandate the specific metrics that its agile projects are to use; instead, each project is to determine the metrics based on stakeholder requirements and unique project characteristics  The officials further stated that these metrics are to be documented in an acquisition program baseline and program management plan; this baseline and program management plan are then to be approved by the CIO  To its credit, the component’s one agile project that, as of May 2018, had deployed its system to users—the Uniformed Division Resource Management System—did measure product quality  OCIO officials also stated that they regularly receive verbal, undocumented feedback from users on the system and they plan to conduct a documented user satisfaction survey on this system by September 2018  Nevertheless, without ensuring that product quality and post- deployment user satisfaction metrics are included in the modular outcomes and target measures that the CIO sets for monitoring agile projects, the Secret Service lacks assurance that the Events Management project or other future agile projects will measure product quality or post-deployment user satisfaction  Without guidance specifying that agile projects track these metrics, the projects may not do so and the CIO may be limited in his knowledge of the progress being made on these projects  Workforce planning and management is essential for ensuring that federal agencies have the talent, skill, and experience mix they need to execute their missions and program goals  To help agencies effectively conduct workforce planning and management, the Office of Personnel Management, the Chief Human Capital Officers Council, DHS, the Secret Service, and we have identified numerous leading practices related to five workforce areas: strategic planning, recruitment and hiring, training and development, employee morale, and performance management  Table 6 identifies the five workforce areas and 15 selected leading practices associated with these areas (3 practices within each area)  Of the five selected workforce planning and management areas, the Secret Service had substantially implemented two of the areas and minimally implemented three of the areas for its IT workforce  In addition, of the 15 selected leading practices associated with these workforce planning and management areas, the Secret Service had fully implemented 3 practices, partly implemented 8 practices, and did not implement any aspects of 4 practices  Table 7 summarizes the extent to which the Secret Service had implemented for its IT workforce the five selected workforce planning and management areas and 15 selected leading practices associated with those areas, as of June 2018  Strategic workforce planning is an essential activity that an agency needs to conduct to ensure that its human capital program aligns with its current and emerging mission and programmatic goals, and that the agency is able to meet its future needs  We previously identified numerous leading practices related to IT strategic workforce planning, including that an organization should (1) establish and maintain a strategic workforce planning process, including developing all competency and staffing needs; (2) regularly assess competency and staffing needs, and analyze the IT workforce to identify gaps in those areas; and (3) develop strategies and plans to address gaps in competencies and staffing  The Secret Service minimally implemented the three selected leading practices associated with the IT strategic workforce planning area  Specifically, the component partly implemented two of the practices and did not implement one practice  Table 8 lists these selected leading practices and provides our assessment of the Secret Service’s implementation of the practices  Establish and maintain a strategic workforce planning process, including developing all competency and staffing needs—partly implemented  The Secret Service took steps to establish a strategic workforce planning process for its IT workforce  For example, the Secret Service CIO developed and maintained a plan that identified strategic workforce planning tasks, to include analyzing the staffing requirements of the IT workforce  In addition, the Secret Service defined general core competencies (e g , communication and customer service) for its workforce, including IT staff  However, OCIO did not identify all required knowledge and skills needed to support this office’s functions  In particular, while OCIO identified certain technical competencies that its IT workforce needs, such as cybersecurity, the office did not identify and document all of the technical competencies that it needs  OCIO officials stated that they did not identify and document the technical competencies that the office needs because the Secret Service was focused on reorganizing the IT workforce under a single, centralized reporting chain within the CIO’s office  Consequently, the officials stated that they had not completed the work to identify all required IT knowledge and skills necessary to support the office  Yet, the Secret Service completed the IT workforce reorganization effort over a year ago, in March 2017 and, since then, OCIO has not identified all of the required IT knowledge and skills that the office needs  OCIO officials told us that they plan to identify all of the technical competency needs for the IT workforce, but they were unable to specify a time frame for when these needs would be fully identified  Until OCIO identifies all of the required knowledge and skills for the IT workforce, the office will be limited in its ability to identify and address any competency gaps associated with this workforce  In addition, the Secret Service did not reliably determine the number of IT staff that it needs in order to support OCIO’s functions  Specifically, in January 2017, an independent review of the staffing model that the component used to identify its IT workforce staffing needs found that the model was not based on any verifiable underlying data  In late August 2018, Office of Human Resources officials reported that they had hired a contractor in early August 2018 to update the staffing model to improve the quality of the data  These officials expected the contractor to finish updating the model by August 2019  The officials plan to use the updated model to identify the Secret Service’s IT workforce staffing needs for fiscal year 2021  Updating the staffing model to incorporate verifiable workload data should increase the likelihood that the Secret Service is able to appropriately identify its staffing needs for its IT workforce  Regularly assess competency and staffing needs, and analyze the IT workforce to identify gaps in those areas—not implemented  The Secret Service regularly assessed the competency and staffing needs for 1 of the occupational series within its IT workforce (i e , the 2210 IT Specialist series)  However, it did not regularly assess the competency and staffing needs for the remaining 11 occupational series that are associated with the component’s IT workforce, nor identify any gaps that it had in those areas  OCIO officials stated that they had not assessed these needs or identified competency or staffing gaps because, among other things, the Secret Service was focused on reorganizing the IT workforce under a single, centralized reporting chain within the CIO’s office  However, as previously mentioned, the component completed this effort in March 2017, but OCIO did not subsequently assess its competency and staffing needs, nor identify gaps in those areas  OCIO officials reported that they plan to assess the competencies of the IT workforce to identify any gaps that may exist; however, they were unable to identify a specific date by which they expect to have the capacity to complete this assessment  Until OCIO regularly analyzes the IT workforce to identify its competency needs and any gaps it may have, OCIO will be limited in its ability to determine whether its IT workforce has the necessary knowledge and skills to meet its mission and goals  Further, Office of Human Resources officials reported that they plan to update the staffing model that they use to identify their IT staffing needs to include more reliable workload data  However, as discussed earlier, the Secret Service had not yet developed that updated model to determine its IT staffing needs  Office of Human Resources officials reported that once they update the staffing model they plan to re- evaluate the Secret Service’s IT staffing needs  The officials also stated that, going forward, they plan to reassess these needs each year as part of the annual budget cycle  Regular assessments of the IT workforce’s staffing needs should increase the likelihood that the Secret Service is able to appropriately identify the number of IT staff it needs to meet its mission and programmatic goals  Develop strategies and plans to address gaps in competencies and staffing—partly implemented  The Secret Service developed recruiting and hiring strategies to address certain competency and staffing needs (e g , cybersecurity) for its IT workforce  These strategies included, among other things, participating in DHS-wide recruiting events and using special hiring authorities  However, because OCIO did not identify all of its IT competency and staffing needs, and lacked a current analysis of its entire IT workforce, the Secret Service could not provide assurance that the recruiting and hiring strategies it developed were specifically targeted towards addressing current OCIO competency and staffing gaps  For example, without an analysis of the IT workforce’s skills, OCIO did not know the extent to which it had gaps in areas such as device management and cloud computing  As a result, the Secret Service’s recruiting strategies may not have been targeted to address any gaps in those areas  Until the Secret Service updates its recruiting and hiring strategies and plans to address all IT competency and staffing gaps identified (after OCIO completes its analysis of the entire IT workforce, as discussed earlier), the Secret Service will be limited in its ability to effectively recruit and hire staff to fill those gaps  According to the Office of Personnel Management, the Chief Human Capital Officers Council, and our prior work, once an agency has determined the critical skills and competencies that it needs to achieve programmatic goals, and identifies any competency or staffing gaps in its current workforce, the agency should be positioned to build effective recruiting and hiring programs  It is important that an agency has these programs in place to ensure that it can effectively recruit and hire employees with the appropriate skills to meet its various mission requirements  The Office of Personnel Management, the Chief Human Capital Officers Council, and we have also identified numerous leading practices associated with effective recruitment and hiring programs  Among these practices, an agency should (1) implement recruiting and hiring activities to address skill and staffing gaps by using the strategies and plans developed during the strategic workforce planning process; (2) establish and track metrics to monitor the effectiveness of the recruitment program and hiring process, including their effectiveness at addressing skill and staffing gaps, and report to agency leadership on progress addressing those gaps; and (3) adjust recruitment plans and hiring activities based on recruitment and hiring effectiveness metrics  The Secret Service minimally implemented the selected three leading practices associated with the recruitment and hiring workforce area  Specifically, the component partly implemented one of the three practices and did not implement the other two practices  Table 9 lists these selected practices and provides our assessment of the Secret Service’s implementation of the practices  Implement recruiting and hiring activities to address skill and staffing gaps by using the strategies and plans developed during the strategic workforce planning process—partly implemented  OCIO officials implemented the activities identified in the Secret Service’s recruiting and hiring plans  For example, as identified in its recruiting plan, OCIO participated in a February 2017 career fair to recruit job applicants at a technology conference  In addition, in August 2017, OCIO participated in a DHS-wide recruiting event  Secret Service officials reported that, during this event, they conducted four interviews for positions in OCIO  However, as previously discussed, OCIO did not identify all of its IT competency and staffing needs, and lacked a current analysis of its entire IT workforce  Without complete knowledge of its current IT competency and staffing gaps, the Secret Service could not provide assurance that the recruiting and hiring strategies that it had implemented fully addressed these gaps  Establish and track metrics to monitor the effectiveness of the recruitment program and hiring process, including their effectiveness at addressing skill and staffing gaps, and report to agency leadership on progress addressing those gaps—not implemented  The Secret Service had not established and tracked metrics for monitoring the effectiveness of its recruitment and hiring activities for the IT workforce  Officials in the Office of Human Resources attributed this to staffing constraints and said their priority was to address existing staffing gaps associated with the Secret Service’s law enforcement groups  In June 2018, Office of Human Resources officials stated that they plan to implement metrics to monitor the effectiveness of the hiring process for the IT workforce by October 2018  The officials also stated that they were in the process of determining (1) the metrics that are to be used to monitor the effectiveness of their workforce recruiting efforts and (2) whether they need to acquire new technology to support this effort  However, the officials did not know when they would implement the metrics for assessing the effectiveness of the recruitment activities and whether they would report the results to leadership  Until the Office of Human Resources (1) develops and tracks metrics to monitor the effectiveness of the Secret Service’s recruitment activities for the IT workforce, including their effectiveness at addressing skill and staffing gaps; and (2) reports to component leadership on those metrics, the Secret Service and the Office of Human Resources will be limited in their ability to analyze the recruitment program to determine whether the program is effectively addressing IT skill and staffing gaps  Further, Secret Service leadership will lack the information necessary to make effective recruitment decisions  Adjust recruitment plans and hiring activities based on recruitment and hiring effectiveness metrics—not implemented  While the Secret Service CIO stated in June 2018 that he planned to adjust the office’s recruiting and hiring strategies to focus on entry- level staff rather than mid-career employees, this planned adjustment was not based on metrics that the Secret Service was tracking  Instead, the CIO stated that he planned to make this change because his office determined that previous mid-career applicants were often unwilling or unable to wait for the Secret Service’s lengthy, required background investigation process to be completed  However, as previously mentioned, the Secret Service did not develop and implement any metrics for assessing the effectiveness of the recruitment and hiring activities for the IT workforce  As a result, the Office of Human Resources and OCIO were not able to use such metrics to inform adjustments to their recruiting and hiring plan and activities, thus, reducing their ability to target potential candidates for hiring  Until the Office of Human Resources and OCIO adjust their recruitment and hiring plans and activities as necessary, after establishing and tracking metrics for assessing the effectiveness of these activities for the IT workforce, the Secret Service will be limited in its ability to ensure that its recruiting plans and activities are appropriately targeted to potential candidates  In addition, the component will lack assurance that these plans and activities will effectively address skill and staffing gaps within its IT workforce  An organization should invest in training and developing its employees to help ensure that its workforce has the information, skills, and competencies that it needs to work effectively  In addition, training and development programs are an integral part of a learning environment that can enhance an organization’s ability to attract and retain employees with the skills and competencies needed to achieve cost-effective and timely results  DHS, the Secret Service, and we have previously identified numerous leading training and development-related practices  Among those practices, an organization should (1) establish a training and development program to assist the agency in achieving its mission and goals; (2) use tracking and other control mechanisms to ensure that employees receive appropriate training and meet certification requirements, when applicable; and (3) collect and assess performance data (including qualitative or quantitative measures, as appropriate) to determine how the training program contributes to improved performance and results  The Secret Service minimally implemented the selected three leading practices associated with the training and development workforce area  Specifically, the component partly implemented two of the three practices and did not implement one practice  Table 10 lists these selected leading practices and provides our assessment of the Secret Service’s implementation of the practices  Establish a training and development program to assist the agency in achieving its mission and goals—partly implemented  OCIO was in the process of developing a training program for its IT workforce  For example, OCIO developed a draft training plan that identified recommended training for the office’s various IT workforce groups (e g , voice communications employees)  However, the office had not defined the required training for each IT workforce group  In addition, OCIO officials had not yet determined which activities they would implement as part of the training program (e g , soliciting employee feedback after training is completed and evaluating the effectiveness of specific training courses), nor did they implement those activities  OCIO officials stated that they had not yet fully implemented a training program because their annual training budget for fiscal year 2018 was not sufficient to implement such a program  However, resource constrained programs especially benefit from identifying and prioritizing training activities to inform training budget decisions  Until OCIO (1) defines the required training for each IT workforce group, (2) determines the activities that it will include in its IT workforce training and development program based on its available training budget, and (3) implements those activities, the office may be limited in its ability to ensure that the IT workforce has the necessary knowledge and skills for their respective positions  Use tracking and other control mechanisms to ensure that employees receive appropriate training and meet certification requirements, when applicable—partly implemented  OCIO used a training system to track that the managers for IITT’s programs had met certain certification requirements for their respective positions  In addition, OCIO manually tracked the technical training that certain IT staff took  However, as discussed earlier, OCIO did not define the required training for each IT workforce group  As such, the office was unable to ensure that IT staff received the appropriate training relevant to their respective positions  Until it ensures that IT staff complete training specific to their positions (after defining the training required for each workforce group), OCIO will have limited assurance that the workforce has the necessary knowledge and skills  Collect and assess performance data (including qualitative or quantitative measures, as appropriate) to determine how the training program contributes to improved performance and results—not implemented  As previously discussed, OCIO did not fully implement a training program for the IT workforce; as such, the office was unable to collect and assess performance data related to such a program  OCIO officials stated that, once they fully implement a training program, they intend to collect and assess data on how this program contributes to improved performance  However, the officials were unable to specify a time frame for when they would do so  Until OCIO collects and assesses performance data (including qualitative or quantitative measures, as appropriate) to determine how the IT training program contributes to improved performance and results (once the training program is implemented), the office may be limited in its knowledge of whether the training program is contributing to improved performance and results  Employee morale is important to organizational performance and an organization’s ability to retain talent to perform its mission  We have previously identified numerous leading practices for improving employee morale  Among other things, we have found that an organization should (1) determine root causes of employee morale problems by analyzing employee survey results using techniques such as comparing demographic groups, benchmarking against similar organizations, and linking root cause findings to action plans; and develop and implement action plans to improve employee morale; (2) establish and track metrics of success for improving employee morale, and report to agency leadership on progress improving morale; and (3) maintain leadership support and commitment to ensure continued progress in improving employee morale, and demonstrate sustained improvement in morale  With regard to its IT workforce, the Secret Service substantially implemented the selected three practices associated with the employee morale workforce area  Specifically, the component fully implemented two of the selected practices and partly implemented one practice  Table 11 lists these selected practices and provides our assessment of the Secret Service’s implementation of the practices  Determine root causes of employee morale problems by analyzing employee survey results using techniques such as comparing demographic groups, benchmarking against similar organizations, and linking root cause findings to action plans  Develop and implement action plans to improve employee morale—fully implemented  The Secret Service used survey analysis techniques to determine the root causes of its low employee morale, on which we have previously reported  For example, the component conducted a benchmarking exercise where it compared the morale of the Secret Service’s employees, including IT staff, to data on the morale of employees at other agencies, including the U S  Capitol Police, U S  Coast Guard, and the Drug Enforcement Administration

ground_truth: FEMA, an agency of the Department of Homeland Security (DHS), has obligated more than $36 billion in PA grants to state, local, and tribal governments to help communities recover and rebuild after major disasters since 2009. Further, costs are rising with disasters, such as Hurricanes Harvey, Irma, and Maria in 2017. FEMA recently redesigned how the PA program delivers assistance to state and local grantees to improve operations and address past challenges identified by GAO and others. FEMA tested the new delivery model in selected disasters and announced implementation in September 2017. GAO was asked to assess the redesigned PA program. This report examines, among other things, the extent to which FEMA's new delivery model addresses (1) past workforce management challenges and assesses future workforce needs; and (2) past information sharing challenges and key IT management controls. GAO reviewed FEMA policy, strategy, and implementation documents; interviewed FEMA and state officials, PA program applicants, and other stakeholders; and observed implementation of the new model at one test location following Hurricane Matthew in 2016. The Federal Emergency Management Agency (FEMA) redesigned the Public Assistance (PA) grant program delivery model to address past challenges in workforce management, but has not fully assessed future workforce staffing needs. GAO and others have previously identified challenges related to shortages in experienced and trained FEMA PA staff and high turnover among these staff. These challenges often led to applicants receiving inconsistent guidance and to PA project delays. As part of its new model, FEMA is creating consolidated resource centers to standardize and centralize PA staff responsible for managing grant applications, and new specialized positions, such as hazard mitigation liaisons, program delivery managers, and site inspectors, to ensure more consistent guidance to applicants. However, FEMA has not assessed the workforce needed to fully implement the new model, such as the number of staff needed to fill certain new positions, or to achieve staffing goals for supporting hazard mitigation on PA projects. Fully assessing workforce needs will help to ensure that FEMA has the people and the skills needed to fully implement the new PA model and help to avoid the long-standing workforce challenges the program encountered in the past. FEMA designed a new PA information and case management system—called the FEMA Applicant Case Tracker (FAC-Trax)—to address past information sharing challenges, such as difficulties in sharing grant documentation among FEMA, state, and local officials and tracking the status of PA projects, but additional actions could better ensure effective implementation. Both FEMA and state officials involved in testing of the new model stated that the new information system allows them to better manage and track PA applications and documentation, which could lead to greater transparency and efficiencies in the program. Further, GAO found that this new system fully addresses two of four key information technology (IT) management controls—project planning and risk management—that are necessary to ensure systems work effectively and meet user needs. However, GAO found that FEMA has not fully addressed the other two controls—requirements development and systems testing and integration. By better analyzing progress on high-level user requirements, for example, FEMA will have greater assurance that FAC-Trax will meet user needs and achieve the goals of the new delivery model. GAO is making five recommendations, including that FEMA assess the workforce needed for the new delivery model and improve key IT management controls for its new information sharing and case management system, FAC-Trax. DHS concurred with all recommendations.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.



Report 2:  As part of this exercise, the Secret Service also compared its employee work-life offerings (e g , on-site childcare and telework program) to those available at other agencies  In addition, the Secret Service developed and implemented action plans for improving employee morale  Among these action plans, for example, the component implemented a student loan repayment program and expanded its tuition assistance program’s eligibility requirements  Establish and track metrics of success for improving employee morale, and report to agency leadership on progress improving morale—fully implemented  The Secret Service tracked metrics for improving employee morale and reported the results to leadership  For example, the component tracked metrics on the percentage of the workforce, including IT staff, that participated in the student loan repayment and tuition assistance programs  In addition, the Chief Strategy Officer reported to the Chief Operating Officer the results related to meeting those metrics  Maintain leadership support and commitment to ensure continued progress in improving employee morale, and demonstrate sustained improvement in morale—partly implemented  Secret Service leadership developed and implemented initiatives that demonstrated their commitment to improving the morale of the Secret Service’s workforce  For example, since 2014, the Secret Service had worked with a contractor to identify ways to improve the morale of its entire workforce, including IT staff  However, as of June 2018, the Secret Service was unable to demonstrate that it had sustained improvement in the morale of the component’s IT staff  In particular, the component was only able to provide IT workforce-specific results from one employee morale assessment that was conducted subsequent to the consolidation of this workforce into OCIO in March 2017  These results were from an assessment conducted by the component’s Inspection Division in December 2017 (the assessment found that the majority of the Secret Service’s IT employees rated their morale as “very good” or “excellent ”) While the component also provided certain employee morale results from the Office of Personnel Management’s Federal Employee Viewpoint Survey in 2017, these results were not specific to the IT workforce  Instead, this workforce’s results were combined with those from staff in another Secret Service division  According to OCIO officials, the results were combined because, at the time of the survey, the IT workforce was administratively identified as being part of that other division  OCIO officials stated that, going forward, they plan to continue to assess the morale of the IT workforce on an annual basis as part of the Federal Employee Viewpoint Survey  In addition, the officials stated that OCIO-specific results may be available as part of the 2018 survey results, which the officials expect to receive by September 2018  By measuring employee satisfaction on an annual basis, the Secret Service should have increased knowledge of whether its initiatives that are aimed at improving employee morale are in fact increasing employee satisfaction  Agencies can use performance management systems as a tool to foster a results-oriented organizational culture that links individual performance to organizational goals  We have previously identified numerous leading practices related to performance management that are intended to enhance performance and ensure individual accountability  Among the performance management practices, agencies should (1) establish a performance management system that differentiates levels of staff performance and defines competencies in order to provide a fuller assessment of performance, (2) explicitly align individual performance expectations with organizational goals to help individuals see the connection between their daily activities and organizational goals, and (3) periodically provide individuals with regular performance feedback  The Secret Service substantially implemented the selected three leading practices associated with the performance management workforce area  Specifically, the component fully implemented one of the three practices and partly implemented the other two practices  Table 12 lists these selected leading practices and provides our assessment of the Secret Service’s implementation of the practices  Establish a performance management system that differentiates levels of staff performance and defines competencies in order to provide a fuller assessment of performance—partly implemented  The Secret Service’s performance management process requires leadership to make meaningful distinctions between levels of staff performance  In particular, the component’s performance plans for IT staff, which are developed by the Office of Human Resources and tailored by OCIO, as necessary, specify the criteria that leadership use to determine if an individual has met or exceeded the expectations associated with each competency identified in their respective performance plan  The performance plans include pre-established, department-wide competencies that are set by DHS, as well as occupational series-specific goals that may be updated by the Secret Service  However, because OCIO did not fully define and document all of its technical competency needs for the IT workforce, as discussed earlier, the Secret Service’s performance plans for IT staff did not include performance expectations related to the full set of technical competencies required for their respective positions  In addition, because OCIO officials were unable to specify a time frame for when they will identify all of the technical competency needs for the IT workforce (as previously discussed), the officials were also unable to specify a time frame for when they would update the IT workforce’s performance plans to include those relevant technical competencies  Until OCIO updates the performance plans for each occupational series within the IT workforce to include the relevant technical competencies, once identified, against which IT staff performance should be assessed, the office will be limited in its ability to provide IT staff with a complete assessment of their performance  In addition, Secret Service management will have limited knowledge of the extent to which IT staff are meeting all relevant technical competencies  Explicitly align individual performance expectations with organizational goals to help individuals see the connection between their daily activities and organizational goals—partly implemented  The Secret Service’s performance plans for IT staff identified certain goals that appeared to be related to organizational goals and objectives  For example, the performance plan for the Telecommunications Specialist occupational series (which is one of the series included in OCIO’s IT workforce) identified a goal for staff to support the voice, wireless, radio, satellite, and video systems serving the Secret Service’s protective and investigative mission  This performance plan goal appeared to be related to the component’s strategic goal on Advanced Technology, which included an objective to create the infrastructure needed to fulfill mission responsibilities  However, the Secret Service was unable to provide documentation that explicitly showed how individual employee performance links to organizational goals, such as a mapping of the goals identified in employee performance plans to organizational goals  Specifically, while Office of Human Resources officials stated that each Secret Service directorate is responsible for ensuring that employee goals map to high-level organizational goals, OCIO officials stated that they did not complete this mapping  The officials were unable to explain why they did not align the goals in their employees’ performance plans to the component’s high-level goals  According to the officials, the Secret Service is in the process of implementing a new automated tool that will require each office to explicitly align individual performance expectations to organizational goals  The officials stated that OCIO plans to use this tool to create employees’ fiscal year 2019 performance plans  By explicitly demonstrating how individual performance expectations align with organizational goals, the Secret Service’s IT staff should have a better understanding of how their daily activities contribute towards achieving the Secret Service’s goals  Periodically provide individuals with regular performance feedback—fully implemented  Secret Service leadership periodically provided their IT staff with performance feedback  Specifically, on an annual basis, OCIO staff received feedback during a mid-year and end-of-year performance feedback assessment  In our prior work, we have stressed that candid and constructive feedback can help individuals maximize their contribution and potential for understanding and realizing the goals and objectives of an organization  Further, this feedback is one of the strongest drivers of employee engagement  According to leading practices of the Software Engineering Institute, effective program oversight includes monitoring program performance and conducting reviews at predetermined checkpoints or milestones  This is done by, among other things, comparing actual cost, schedule, and performance data with estimates in the program plan and identifying significant deviations from established targets or thresholds for acceptable performance levels  In addition, the Software Engineering Institute previously identified leading practices for effectively monitoring the performance of agile projects  According to the Institute, agile development methods focus on delivering usable, working software frequently; as such, it is important to measure the value delivered during each iteration of these projects  To that end, the Institute reported that agile projects should be measured on velocity (i e , number of story points completed per sprint or release), development progression (e g , the number of user stories planned and accepted), product quality (e g , number of defects), and post-deployment user satisfaction  DHS and the Secret Service had fully implemented the selected leading practice for monitoring the performance of one program and three projects within the IITT investment, and conducting reviews of this program and these projects at predetermined checkpoints  In addition, with regard to the selected leading practice for monitoring agile projects, the Secret Service had fully implemented this practice for one of its two projects being implemented using agile and had partially implemented this practice for the other project  Table 13 provides a summary of DHS’s and the Secret Service’s implementation of these leading practices, as relevant for one program and three projects within IITT  Monitor program performance and conduct reviews at predetermined checkpoints or milestones  Consistent with leading practices, DHS and the Secret Service monitored the performance of IITT’s program and projects by comparing actual cost, schedule, and performance information against planned targets and conducting reviews at predetermined checkpoints  For example, within the Secret Service: The Enabling Capabilities program and Multi-Level Security project monitored their contractors’ costs spent to-date on a monthly basis and compared them to the total contract amounts  OCIO used integrated master schedules to monitor the schedule performance of the Enabling Capabilities program and Multi-Level Security project  OCIO also monitored the cost, schedule, and performance of the Uniformed Division Resource Management System and Events Management projects during monthly status reviews  In addition, DHS and the Secret Service conducted acquisition decision event reviews and systems engineering life cycle technical reviews of IITT’s program and projects at predetermined checkpoints and, when applicable, identified deviations from established cost, schedule, and performance targets  For example: Secret Service OCIO met with DHS’s Office of Program Accountability and Risk Management in February 2017, and with DHS’s Acting Under Secretary for Management in June 2017, to discuss a schedule breach for the Enabling Capabilities program  In particular, the Enabling Capabilities program informed DHS that the program needed to change the planned date for acquisition decision event 3 (the point at which a decision is made to fully deploy the system) in order to conduct tests in an operational environment prior to that decision event  This delay was due to the Secret Service misunderstanding the tests that it was required to conduct prior to that decision event  Specifically, the Enabling Capabilities program had conducted tests on “production representative” systems, but these tests were not sufficient to meet the requirements for acquisition decision event 3  The project team for Multi-Level Security identified that certain technical issues they had experienced would delay system deployment and full operational capability (the point at which an investment becomes fully operational)  As such, in October 2017, the project notified the Secret Service Component Acquisition Executive of these expected delays  In particular, the web browser that was intended to provide users on “Sensitive But Unclassified” workstations the ability to view information from different security levels, experienced technical delays in meeting personal identity verification requirements  The project team also described for the executive how the schedule delay would affect the project’s performance metrics and funding, and subsequently updated the project plan accordingly  Measure and monitor agile projects on, among other things, velocity (i e , number of story points completed per sprint or release), development progression (e g , the number of features and user stories planned and accepted), product quality (e g , number of defects), and post-deployment user satisfaction  Secret Service OCIO measured its two agile projects—Uniformed Division Resource Management System and Events Management— using certain agile metrics  In particular, OCIO officials measured the Uniformed Division Resource Management System and Events Management projects using key metrics related to velocity and development progression  For example, the officials measured development progression for both projects on a daily basis  In addition, OCIO officials monitored each project’s progress against these metrics during bi-weekly reviews that they conducted with each project team  The OCIO officials also tracked product quality metrics for the Uniformed Division Resource Management System  For example, on a monthly basis, the officials tracked the number of helpdesk tickets that had been resolved related to the system  In addition, on a quarterly basis, they tracked the number of Uniformed Division Resource Management System defects that (1) had been fixed and (2) were in the backlog  However, while OCIO officials received certain post-deployment user satisfaction information from end-users of the Uniformed Division Resource Management System by, among other things, tracking the number of helpdesk tickets related to the system and via daily verbal, undocumented feedback from certain Uniformed Division officers, OCIO officials had not fully measured and documented post- deployment user satisfaction with the system, such as via a survey of employees who use the system  The officials stated that they had not conducted and documented a survey because they were focused on (1) addressing software performance issues that occurred after they deployed the system to a limited number of users, and (2) continuing system deployment to the remaining users after they addressed the performance issues  OCIO officials stated that they plan to conduct such a documented survey by the end of September 2018  The results of the user satisfaction survey should provide OCIO with important information on whether the Uniformed Division Resource Management System is meeting users’ needs  The Secret Service’s full implementation of 11 of 14 component-level CIO responsibilities constitutes a significant effort to establish CIO oversight for the component’s IT portfolio  Additional efforts to fully implement the remaining 3 responsibilities, including ensuring that all IT contracts are reviewed, as appropriate; ensuring that the Secret Service’s enterprise governance policy appropriately specifies the CIO’s role in developing and reviewing the component’s IT budget formulation and execution; and ensuring agile projects measure product quality and post-deployment user satisfaction, will further position the CIO to effectively manage the Secret Service’s IT portfolio  When effectively implemented, IT workforce planning and management activities can facilitate the successful accomplishment of an agency’s mission  However, the Secret Service had not fully implemented all of the 15 selected practices for its IT workforce for any of the five areas— strategic planning, recruitment and hiring, training and development, employee morale, and performance management  The Secret Service’s lack of (1) a strategic workforce planning process, including the identification of all required knowledge and skills, assessment of competency gaps, and targeted strategies to address specific gaps in competencies and staffing; (2) targeted recruiting activities, including metrics to monitor the effectiveness of the recruitment program and adjustment of the recruitment program and hiring efforts based on metrics; (3) a training program, including the identification of required training for IT staff, ensuring that staff take required training, and assessment of performance data regarding the training program; and (4) a performance management system that includes all relevant technical competencies, greatly limits its ability to ensure the timely and effective acquisition and maintenance of the Secret Service’s IT infrastructure and services  On the other hand, by monitoring program performance and conducting reviews at predetermined checkpoints for one program and three projects associated with the IITT investment, in accordance with leading practices, the Secret Service and DHS provided important oversight needed to guide that program and those projects  Measuring projects on leading agile metrics also provided the Secret Service CIO with important information on project performance  We are making the following 13 recommendations to the Director of the Secret Service: The Director should ensure that the CIO establishes and documents an IT acquisition review process that ensures the CIO or the CIO’s delegate reviews all contracts containing IT, as appropriate  (Recommendation 1) The Director should update the enterprise governance policy to specify (1) the CIO’s current role and responsibilities on the Executive Resources Board, to include developing and reviewing the IT budget formulation and execution; and (2) the Deputy CIO’s role and responsibilities on the Enterprise Governance Council  (Recommendation 2) The Director should ensure that the Secret Service develops a charter for its Executive Resources Board that specifies the roles and responsibilities of all board members, including the CIO  (Recommendation 3) The Director should ensure that the CIO includes product quality and post-deployment user satisfaction metrics in the modular outcomes and target measures that the CIO sets for monitoring agile projects  (Recommendation 4) The Director should ensure that the CIO identifies all of the required knowledge and skills for the IT workforce  (Recommendation 5) The Director should ensure that the CIO regularly analyzes the IT workforce to identify its competency needs and any gaps it may have  (Recommendation 6) The Director should ensure that, after OCIO completes an analysis of the IT workforce to identify any competency and staffing gaps it may have, the Secret Service updates its recruiting and hiring strategies and plans to address those gaps, as necessary  (Recommendation 7) The Director should ensure that the Office of Human Resources (1) develops and tracks metrics to monitor the effectiveness of the Secret Service’s recruitment activities for the IT workforce, including their effectiveness at addressing skill and staffing gaps; and (2) reports to component leadership on those metrics  (Recommendation 8) The Director should ensure that the Office of Human Resources and OCIO adjust their recruitment and hiring plans and activities, as necessary, after establishing and tracking metrics for assessing the effectiveness of these activities for the IT workforce  (Recommendation 9) The Director should ensure that the CIO (1) defines the required training for each IT workforce group, (2) determines the activities that OCIO will include in its IT workforce training and development program based on its available training budget, and (3) implements those activities  (Recommendation 10) The Director should ensure that the CIO ensures that the IT workforce completes training specific to their positions (after defining the training required for each workforce group)  (Recommendation 11) The Director should ensure that the CIO collects and assesses performance data (including qualitative or quantitative measures, as appropriate) to determine how the IT training program contributes to improved performance and results (once the training program is implemented)  (Recommendation 12) The Director should ensure that the CIO updates the performance plans for each occupational series within the IT workforce to include the relevant technical competencies, once identified, against which IT staff performance should be assessed  (Recommendation 13) DHS provided written comments on a draft of this report, which are reprinted in appendix III  In its comments, the department concurred with all 13 of our recommendations and provided estimated completion dates for implementing each of them  For example, with regard to recommendation 2, the department stated that the Secret Service would update its enterprise governance policy and related policies to outline the roles and responsibilities of the CIO and Deputy CIO, among others, by March 31, 2019  In addition, for recommendation 13, the department stated that the Secret Service OCIO will include relevant technical competencies in performance plans, as appropriate, in the next performance cycle that starts in July 2019  If implemented effectively, these actions should address the weaknesses we identified  The department also identified a number of other actions that it said had been taken to address our recommendations  For example, in response to recommendation 8, which calls for the Office of Human Resources to (1) develop and track metrics to monitor the effectiveness of the Secret Service’s recruitment activities for the IT workforce and (2) report to component leadership on those metrics, DHS stated that the Secret Service’s Office of Human Resources’ Outreach Branch provides to the department metrics on recruitment efforts toward designated priority mission-critical occupations  However, for fiscal year 2017, only 1 of the 12 occupational series associated with the Secret Service’s IT workforce was designated as a mission-critical occupation for the component (i e , the 2210 IT Specialist series)  The 11 other occupational series were not designated as mission- critical occupations  In addition, for fiscal year 2018, none of these 12 occupational series were designated as mission-critical occupations  As such, metrics on recruiting for these IT series may not have been reported to DHS leadership  Moreover, while we requested documentation of the recruiting metrics for the Secret Service’s IT workforce and, during the course of our review, had multiple subsequent discussions with the Secret Service regarding such metrics, the component did not provide documentation that demonstrated it had established recruiting metrics for its IT workforce  Tracking such metrics and reporting the results to Secret Service leadership, as we recommended, would provide management with important information necessary to make effective recruitment decisions  Further, in response to recommendation 10, which among other things, calls for the CIO to define the required training for each IT workforce group, the department stated that the Secret Service OCIO recently developed training requirements for each workforce group, which were issued during our audit  However, while during our audit OCIO provided a list of recommended training courses, the office did not identify them as being required courses  Defining training that is required for each IT workforce group, as we recommended, would inform OCIO of the necessary training for each position and enable the office to prioritize this training, to ensure that its staff have the needed knowledge and skills  In addition to the aforementioned comments, we received technical comments from DHS and Secret Service officials, which we incorporated, as appropriate  We are sending copies of this report to the appropriate congressional committees, the Secretary of Homeland Security, the Director of the Secret Service, and other interested parties  In addition, this report is available at no charge on the GAO website at http://www gao gov  Should you or your staffs have any questions on information discussed in this report, please contact me at (202) 512-4456 or HarrisCC@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made major contributions to this report are listed in appendix IV  Our objectives were to evaluate the extent to which: (1) the U S  Secret Service (Secret Service) Chief Information Officer (CIO) has implemented selected information technology (IT) oversight responsibilities, (2) the Secret Service has implemented leading workforce planning and management practices for its IT workforce, and (3) the Secret Service and the Department of Homeland Security (DHS) have implemented selected performance and progress monitoring practices for the Information Integration and Technology Transformation (IITT) investment  To address the first objective, we analyzed DHS’s policies and guidance on IT management to identify the responsibilities that were to be implemented by the component-level CIO related to overseeing the Secret Service’s IT portfolio, including existing systems, acquisitions, and investments  From the list of 33 responsibilities that we identified, we then excluded the responsibility that was associated with information security, which is expected to be addressed as part of a separate, subsequent GAO review  We also excluded those responsibilities that were significantly large in scope (e g , implement an enterprise architecture) or that, in our professional judgment, lacked specificity (e g , provide timely delivery of mission IT services)  As a result, we excluded from consideration for this review a total of 10 CIO responsibilities  For the 23 that remained, we then combined certain responsibilities that overlapped with other related responsibilities  For example, we combined related responsibilities on the component CIO’s review of IT contracts  As a result, we identified 14 responsibilities for review  We validated with the acting DHS CIO that these responsibilities were key responsibilities for the department’s component-level CIOs  We then included all 14 of the responsibilities in our review  The 14 selected component-level CIO responsibilities were: 1  Develop and review the component IT budget formulation and execution  2  Manage the component IT investment portfolio, including establishing an IT acquisition review process that enables component and DHS review of component acquisitions (i e , contracts) that contain IT  3  Develop, implement, and maintain a detailed IT strategic plan  4  Ensure all component IT policies are in compliance and alignment with DHS IT directives and instructions  5  Concur with each program’s and/or project’s systems engineering life cycle tailoring plan  6  Support the Component Acquisition Executive to ensure processes are established that enable systems engineering life cycle technical reviews and that they are adhered to by programs and/or projects  7  Ensure that all systems engineering life cycle technical review exit criteria are satisfied for each of the component’s IT programs and/or projects  8  Ensure the necessary systems engineering life cycle activities have been satisfactorily completed as planned for each of the component’s IT programs and/or projects  9  Concur with the systems engineering life cycle technical review completion letter for each of the component’s IT programs and/or projects  10  Maintain oversight of their component’s agile development approach for IT by appointing the responsible personnel, identifying investments for adoption, and reviewing artifacts  11  With Component Acquisition Executives, evaluate and approve the application of agile development for IT programs consistent with the component’s agile development approach  12  Set modular outcomes and target measures to monitor the progress in achieving agile implementation for IT programs and/or projects within their component  13  Participate on DHS’s CIO Council, Enterprise Architecture Board, or other councils/boards as appropriate, and appoint employees to serve when necessary  14  Meet the IT competency requirements established by the DHS CIO, as required in the component CIO’s performance plan  To determine the extent to which the Secret Service CIO has implemented these responsibilities, we obtained and assessed relevant component documentation and compared it to the responsibilities  Specifically, we obtained and analyzed documentation including evidence of the CIO’s participation on the Secret Service governance board that has final decision authority and responsibility for enterprise governance, including the IT budget; monthly program management reports showing the CIO’s oversight of IT programs, projects, and systems; monthly status reports on program spending; the Secret Service’s IT strategic plan; the Secret Service’s enterprise governance policy; meeting minutes from the DHS board and council on which the CIO participated (i e , the CIO Council and Enterprise Architecture Board); and documentation demonstrating whether the CIO met the IT competency requirements  In addition, we obtained and analyzed relevant documentation related to the CIO’s oversight of the major IT investments on which the Secret Service was spending development, modernization, and enhancement funds during fiscal year 2017  As of July 2017, the component had one investment—IITT—that met this criterion  IITT is a portfolio investment that, as of July 2017, included two programs (one of which included three projects) and one standalone project (i e , it was not part of another program) that had capabilities that were in planning or development and modernization: the Enabling Capabilities program, Enterprise Resource Management System program (which included three projects, called Uniformed Division Resource Management System, Events Management, and Enterprise-wide Scheduling), and Multi-Level Security project  In particular, we obtained and analyzed documentation related to the CIO’s oversight of the systems engineering life cycles for IITT’s Enabling Capabilities program and the Uniformed Division Resource Management System, Events Management, and Multi-Level Security projects  This documentation included acquisition program baselines, systems engineering life cycle tailoring plans, and systems engineering life cycle technical review briefings and completion letters  We then compared the documentation against the five selected systems engineering life cycle oversight responsibilities (responsibilities 5, 6, 7, 8, and 9)  We also obtained and analyzed documentation related to the CIO’s oversight of two projects that the Secret Service was implementing using an agile methodology—Uniformed Division Resource Management System and Events Management  Specifically, we obtained and assessed documentation of (1) the CIO’s approval for these projects to be implemented using an agile methodology and (2) the agile development metrics that the CIO established for each of these projects  We then compared this documentation to the three agile development-related component-level CIO responsibilities (responsibilities 10, 11, and 12)  Further, to determine the extent to which the Secret Service CIO had established an IT acquisition (i e , contract) review process that enabled component and DHS review of component contracts that contain IT (which is part of responsibility 2), we first asked Secret Service officials to provide us with a list of all new, unclassified IT contracts that the component awarded between October 1, 2016, and June 30, 2017  The Secret Service officials provided a list of 54 contracts  We validated that these were contracts for IT or IT services by: (1) searching for them in the Federal Procurement Data System – Next Generation; (2) identifying their associated product or service codes, as reported in that system; and (3) determining whether those codes were included in the universe of 79 IT product or service codes identified by the Category Management Leadership Council  In validating the list of 54 contracts provided by the Secret Service, we determined that 5 of the contracts were not associated with an IT product or service code  As such, we removed those contracts from the list  In addition, we found that three other items identified by the component were not in the Federal Procurement Data System – Next Generation  Secret Service officials subsequently confirmed that these three items were not contracts  We therefore removed these three items from the list  As such, the final list of validated contracts identified by the Secret Service included 46 IT contracts  In addition, to identify any IT contracts that were not included in the list provided by the Secret Service, we conducted a search of the Federal Procurement Data System – Next Generation to identify all unclassified contracts that (1) the component awarded between October 1, 2016, and June 30, 2017; (2) were not a modification of a contract; and (3) were associated with 1 of the 79 IT product or service codes identified by the Category Management Leadership Council  Based on these criteria, we identified 144 Secret Service IT contracts in the Federal Procurement Data System – Next Generation (these 144 contracts included the 46 contracts previously identified by Secret Service officials)  We then asked Secret Service officials to validate the accuracy, completeness, and reliability of these data, which they did  From each of these two lists of IT contracts (i e , the list of 46 IT contracts identified by the Secret Service and the list of 144 IT contracts that we identified from the Federal Procurement Data System – Next Generation), we then selected random, non-generalizable samples of contracts, as described below  First, from the list of 46 IT contracts identified by Secret Service officials, we removed 4 contracts that had total values of less than $10,000  To ensure that we selected across all contract sizes, we randomly selected 12 contracts from the remaining list of 42 contracts, using the following cost ranges: $10,000 to $50,000 (4 contracts), more than $50,000 to less than $250,000 (4 contracts), and more than $250,000 (4 contracts)  Second, from our list of 144 IT contracts that we identified from the Federal Procurement Data System – Next Generation, we removed the 46 contracts identified by Secret Service officials  We also removed 12 contracts that had total values of less than $10,000  To ensure that we selected across all contract sizes, we randomly selected 21 contracts from the remaining list of 86 contracts, using the following cost ranges: $10,000 to $50,000 (7 contracts), more than $50,000 to less than $250,000 (7 contracts), and more than $250,000 (7 contracts)  In total, we selected 33 IT contracts for review  We separated the contracts into the three cost ranges identified above in order to ensure that contracts of different value levels had been selected  This enabled us to determine the extent to which the CIO appropriately reviewed contracts of all values  To determine the extent to which the CIO had established an IT contract approval process that enabled the Secret Service and DHS, as appropriate, to review IT contracts, we first asked Secret Service Office of the CIO (OCIO) officials for documentation of their IT contract approval process  These officials were unable to provide such documentation  Instead, the officials stated that the Secret Service CIO or the CIO’s delegate approves all IT contracts prior to award  The officials also provided documentation that identified four staff to whom the CIO had delegated his approval authority  Further, the officials stated that, in accordance with DHS’s October 2016 IT acquisition review guidance, they submitted to DHS OCIO for approval any IT contracts that met DHS’s thresholds for review, including those that (1) had total estimated procurement values of $2 5 million or more, and (2) were associated with a major investment  Based on the IT acquisition review process that Secret Service OCIO officials described, we then obtained and analyzed each of the 33 selected IT contracts and associated approval documentation to determine whether or not the Secret Service CIO or the CIO’s delegate had approved each of the contracts  In particular, we (1) reviewed the name of the contract approver on the approval documentation, and (2) compared the signature dates that were on the contracts to the signature dates that were identified on the associated approval documentation  In addition, to determine whether or not the Secret Service CIO submitted to DHS OCIO for approval the IT contracts that (1) had total estimated procurement values of $2 5 million or more, and (2) were associated with major investments, we first analyzed the 144 Secret Service IT contracts that we had previously pulled from the Federal Procurement Data System – Next Generation to determine which contracts met the $2 5 million threshold  We identified 4 contracts that met this threshold  We then requested that OCIO identify the levels (i e , major or non-major) of the investments associated with these contracts  According to OCIO officials, 3 of the 4 contracts were associated with non-major investments and 1 was not associated with an investment  As such, based on DHS’s October 2016 IT acquisition review guidance, none of these contracts needed to be submitted to DHS OCIO for review  We also interviewed Secret Service officials, including the CIO and Deputy CIO, regarding the CIO’s implementation of the 14 selected component-level responsibilities  We assessed the evidence against the selected responsibilities to determine the extent to which the CIO had implemented them  To address the second objective—determining the extent to which the Secret Service had implemented leading workforce planning and management practices for its IT workforce—we first identified seven topic areas associated with human capital management based on the following sources: The Office of Personnel Management’s Human Capital Framework  Office of Personnel Management and the Chief Human Capital Officers Council Subcommittee for Hiring and Succession Planning, End-to-End Hiring Initiative  GAO, High-Risk Series: Progress on Many High-Risk Areas, While Substantial Efforts Needed on Others  GAO, IT Workforce: Key Practices Help Ensure Strong Integrated Program Teams; Selected Departments Need to Assess Skill Gaps  GAO, Department of Homeland Security: Taking Further Action to Better Determine Causes of Morale Problems Would Assist in Targeting Action Plans  GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government  GAO, Results-Oriented Cultures: Creating a Clear Linkage between Individual Performance and Organizational Success  DHS acquisition guidance  Secret Service acquisition guidance  Among these topic areas, we then selected five areas that, in our professional judgment, were of particular importance to successful workforce planning and management  They were also previously identified as part of our high-risk and key issues work on human capital management  These areas include: (1) strategic planning, (2) recruitment and hiring, (3) training and development, (4) employee morale, and (5) performance management  We also reviewed these same sources and identified numerous leading practices associated with the five topic areas  Among these leading practices, we then selected three leading practices within each of the five areas (for a total of 15 selected practices)  The selected practices were foundational practices that, in our professional judgment, were of particular importance to successful workforce planning and management  Table 14 identifies the five selected workforce areas and 15 selected associated practices  To determine the extent to which the Secret Service had implemented the selected leading workforce planning and management practices for its IT workforce, we obtained and assessed documentation and compared it against the 15 selected practices  In particular, we analyzed the Secret Service’s human capital strategic plan, human capital staffing plan, IT strategic plan, documentation of the component’s staffing model that it used to determine the number of IT staff needed, an independent verification and validation report on the component’s staffing models, documentation of the current number of IT staff, the Secret Service’s recruitment and outreach plans, documentation of DHS’s hiring authorities (which are applicable to the Secret Service), the Secret Service’s training strategic plan, IT workforce training plan, action plans for improving employee morale, and templates used for measuring and reporting employee performance  We also interviewed Secret Service officials—including the CIO, Deputy CIO, and workforce planning staff—about the component’s workforce- related policies and documentation  Further, we discussed with the officials the Secret Service’s efforts to implement the selected workforce practices for its IT workforce  Regarding our assessments of the Secret Service’s implementation of the 15 selected workforce planning and management practices, we assessed a practice as being fully implemented if component officials provided supporting documentation that demonstrated all aspects of the practice  We assessed a practice as not implemented if the officials did not provide any supporting documentation for that practice, or if the documentation provided did not demonstrate any aspect of the practice  We assessed a practice as being partly implemented if the officials provided supporting documentation that demonstrated some, but not all, aspects of the selected practice  In addition, related to our assessments of the Secret Service’s implementation of the five selected overall workforce areas, we assessed each area as follows, based on the implementation of the three selected practices within each area: Fully implemented: The Secret Service provided evidence that it had fully implemented all three of the selected practices within the workforce area; Substantially implemented: The Secret Service provided evidence that it had either fully implemented two selected practices and partly implemented the remaining one selected practice within the workforce area, or fully implemented one selected practice and partly implemented the remaining two selected practices within the workforce area; Partially implemented: The Secret Service provided evidence that it had partly implemented each of the three selected practices within the workforce area; Minimally implemented: The Secret Service provided evidence that it partly implemented two selected practices and not implemented the remaining one selected practice within the workforce area, or partly implemented one selected practice and not implemented the remaining two selected practices within the workforce area; or Not implemented: The Secret Service did not provide evidence that it had implemented any of the three selected practices within the workforce area  To address the third objective—determining the extent to which the Secret Service and DHS have implemented selected performance and progress monitoring practices for IITT—we reviewed leading project monitoring practices and guidance from the Software Engineering Institute  First, we reviewed the practices within the Project Monitoring and Control process area of the Institute’s Capability Maturity Model Integration® for Acquisition  Based on our review, we identified four practices associated with monitoring program performance and progress  In our professional judgment, all four of these practices were of significance to managing the IITT investment given the phase of the life cycle that the investment was in  As such, we elected to include all four of these practices in our review, and combined them into one practice, as follows: Monitor program performance and conduct reviews at predetermined checkpoints or milestones by, among other things, comparing actual cost, schedule, and performance data with estimates in the program plan and identifying significant deviations from established targets or thresholds for acceptable performance levels  Next, given the agile development methodology that the Secret Service was using for certain projects within IITT, we reviewed the Software Engineering Institute’s technical note on the progress monitoring of agile contractors  Based on our review, and in consultation with an internal expert, we selected four agile metrics that the Institute identified as important for successful agile implementations and that, in our professional judgment, were of most significance to monitoring the performance of IITT’s agile projects  We then combined these four metrics into one practice, as follows: Measure and monitor agile projects on velocity (i e , number of story points completed per sprint or release), development progression (e g , the number of features and user stories planned and accepted), product quality (e g , number of defects), and post-deployment user satisfaction  To determine the extent to which DHS and the Secret Service had implemented the first selected practice, we analyzed relevant program management and governance documentation for IITT’s Enabling Capabilities program, and Multi-Level Security, Uniformed Division Resource Management System, and Events Management projects  In particular, we analyzed acquisition program baselines, DHS acquisition decision event memorandums, artifacts from DHS and Secret Service program oversight reviews, cost monitoring reports, program integrated master schedules, and program status briefings, and compared this documentation to the selected practice  We also interviewed Secret Service OCIO officials regarding the Secret Service’s and DHS’s efforts to monitor the IITT investment’s performance and progress  To determine the extent to which the Secret Service had implemented the second selected practice related to measuring and monitoring agile projects on agile metrics (i e , velocity, development progression, product quality, and post-deployment user satisfaction), we obtained and analyzed agile-related documentation for the two projects that the Secret Service was implementing using an agile methodology—Uniformed Division Resource Management System and Events Management  Specifically, to determine the extent to which the Secret Service was measuring and monitoring these two projects on metrics for velocity and development progression, we obtained and analyzed documentation, such as sprint burndown charts and monthly program status reports, and compared it to the selected practice  In addition, the agile metrics for product quality and post-deployment user satisfaction were only applicable to projects that had been deployed to users  As such, these metrics were applicable to the Uniformed Division Resource Management System (which the Secret Service had deployed to users) and were not applicable to Events Management (which the Secret Service had not yet deployed to users, as of early May 2018)  We therefore obtained and analyzed documentation demonstrating that Secret Service OCIO measured product defects for the Uniformed Division Resource Management System  We also requested documentation demonstrating that OCIO had measured and monitored post-deployment user satisfaction for this project, including via a survey  OCIO officials stated that they had not conducted such a survey and were unable to provide documentation demonstrating they had measured post- deployment user satisfaction for the Uniformed Division Resource Management System  To assess the reliability of the cost, schedule, and agile-related data that were in DHS and the Secret Service’s program management and governance documentation for the IITT investment, we (1) analyzed related documentation and assessed the data against existing agency records to identify consistency in the information, and (2) examined the data for obvious outliers, incomplete, or unusual entries  We determined that the data in these documents were sufficiently reliable for our purpose, which was to evaluate the extent to which DHS and the Secret Service had implemented processes for monitoring the IITT investment’s performance and progress  We conducted this performance audit from May 2017 to November 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  As of June 2018, the Secret Service’s Information Integration and Technology Transformation (IITT) investment included two programs (one of which included three projects) and one project that had capabilities that were in planning or development and modernization, as described below: Enabling Capabilities  This program is intended to, among other things, (1) modernize and enhance the Secret Service’s information technology (IT) network infrastructure, including increasing bandwidth and improving the speed and reliability of the Secret Service’s IT system performance; (2) enhance cybersecurity to protect against potential intrusions and viruses; and (3) provide counterintelligence and data mining capabilities to improve officials’ ability to perform the Secret Service’s investigative mission  Enterprise Resource Management System  This program comprises three projects that are intended to provide: a system that will enable the Secret Service’s Uniformed Division to efficiently and effectively plan, provision, and schedule missions (this project is referred to as Uniformed Division Resource Management System), a system that will unify the logistical actions (e g , assigning personnel) surrounding special events that Secret Service agents need to protect, such as the United Nations General Assembly (this project is referred to as Events Management), and a capability for creating schedules for Secret Service agents and administrative, professional, and technical staff, as well as the ability to generate reports on information such as monthly hours worked (this project is referred to as Enterprise-wide Scheduling)  Multi-Level Security  This project is intended to enable authorized Secret Service users to view two levels of classified information on a single workstation  Previously, data at various security levels were contained and used in multiple disparate systems  Multi-Level Security is intended to streamline users’ access to information at different security levels in order to enable them to more quickly and effectively perform their duties  Table 15 provides the planned life cycle cost and schedule estimates (threshold values) for each IITT program and project that had capabilities in planning or development and modernization, as of June 2018  In addition, the table describes any changes in those cost and schedule estimates, as well as the key reasons for any changes, as identified by officials from the Secret Service’s Office of the Chief Information Officer  In addition to the contact named above, the following staff made key contributions to this report: Shannin O’Neill (Assistant Director), Emily Kuhn (Analyst-in-Charge), Quintin Dorsey, Rebecca Eyler, Javier Irizarry, and Paige Teigen 
Report 2: 0 million in prior-year advance procurement (AP) funding, and the Navy's proposed FY2020 budget requests the remaining $981 2 million in procurement funding needed to complete the two ships' estimated combined procurement cost  The Navy's proposed FY2020 budget also requests $73 0 million in AP funding for TAO-205s to be procured in future fiscal years, and $3 7 million in cost-to-complete procurement funding to cover cost growth on TAO-205s procured in prior fiscal years, bringing the total FY2020 procurement funding request for the TAO-205 program (aside from outfitting and post-delivery costs) to $1,057 9 million  One issue for Congress is whether to approve, reject, or modify the Navy's FY2020 procurement funding request for the TAO-205 program  In assessing this issue, Congress may consider, among other things, whether the Navy has accurately priced the work that it is requesting to fund in FY2020  Another issue for Congress concerns the number of oilers the Navy will require in coming years to support its operations  The Navy is implementing a new operational concept, called Distributed Maritime Operations (DMO), that could lead to the development of a fleet with larger numbers of individually smaller ships, and to more-widely dispersed Navy operations  DMO could affect requirements for Navy logistics, including oilers  The Navy states that Recapitalizing the auxiliary and sealift fleet in support of DMO has become a top priority  The initial reviews of the requirements to support this operational maritime concept indicate potential growth across the five lines of effort: refuel, rearm, resupply, repair, and revive  Coincident is the review of the level of effort needed to distribute logistics into a contested maritime environment following safe transfer by the logistics fleet—smaller, faster, multi-mission transports likely resident within the future battle force  The work to fully flesh out the requirement is ongoing, but the aggregate is expected to be no less than the current requirement, reinforcing the urgency to recapitalize the current fleet  An August 2017 GAO report states the following: The readiness of the surge sealift and combat logistics fleets has trended downward since 2012  For example, GAO found that mission-limiting equipment casualties—incidents of degraded or out-of-service equipment—have increased over the past 5 years, and maintenance periods are running longer than planned, indicating declining materiel readiness across both fleets     The Navy has not assessed the effects of widely distributed operations, which could affect the required number and type of combat logistics ships  The Navy released its new operational concept of more widely distributed operations—ships traveling farther distances and operating more days to support a more distributed fleet—in 2017  The Navy has not assessed the effects that implementing this concept will have on the required number and type of combat logistics ships  These effects could be exacerbated in the event that the Navy is less able to rely on in-port refueling—which has comprised about 30 percent of all refuelings over the past 3 years—placing greater demand on the combat logistics fleet  Given the fleet's dependence on the combat logistics force, waiting until 2019 or 2020 to conduct an assessment, as planned, could result in poor investment decisions as the Navy continues to build and modernize its fleet  Furthermore, without assessing the effects of widely distributed operations on logistics force requirements and modifying its force structure plans accordingly, the Navy risks being unprepared to provide required fuel and other supplies  Another issue for Congress is whether to encourage or direct the Navy to build TAO-205s with more ship self-defense equipment than currently planned by the Navy  The issue relates to how changes in the international security environment might affect how the Navy operates and equips its underway replenishment ships  During the Cold War, the Navy procured underway replenishment ships to support a two-stage approach to underway replenishment in which single-product "shuttle" ships (such as oilers, ammunition ships, and dry stores ships) would take their supplies from secure ports to relatively safe midocean areas, where they would then transfer them to multiproduct "station" ships called TAOEs and AORs  The TAOEs and AORs would then travel to Navy carrier strike groups operating in higher-threat areas and transfer their combined supplies to the carrier strike group ships  As a result, single-product shuttle ships were equipped with lesser amounts of ship self-defense equipment, and TAOEs and AORs were equipped with greater amounts of such equipment  When the Cold War ended and transitioned to the post-Cold War era, threats to U S  Navy ships operating at sea were substantially reduced  As a consequence, the amount of ship self-defense equipment on the TAOEs and AORs was reduced, and a single-stage approach to underway replenishment, in which oilers and dry stores ships took supplies from secure ports all the way to carrier strike group ships, was sometimes used  Now that the post-Cold War era has transitioned to a new strategic environment featuring renewed great power competition with countries like China and Russia, and a consequent renewal of potential threats to U S  Navy ships operating at sea, the question is whether TAO-205s should be equipped with lesser amounts of ship self-defense equipment, like oilers were during both the Cold War and post-Cold War eras, or with greater amounts of ship self-defense equipment, like TAOEs and AORs were during the Cold War  Building TAO-205s with more ship self-defense equipment than currently planned by the Navy could increase TAO-205 procurement costs by tens of millions of dollars per ship, depending on the amount of additional ship self-defense equipment  Section 1026 of the FY2016 National Defense Authorization Act ( S  1356 / P L  114-92 of November 25, 2015) required an independent assessment of the Navy's combat logistics force ships  The report was delivered to Congress in February 2016  A copy of the report was posted by the media outlet Politico on March 11, 2016  The report states the following: The T-AO(X) will only have a limited capability to defeat a submarine launched torpedo attack and no capability to defeat a missile attack  When delivered, the TAO(X) will have: —[the] NIXIE Torpedo Countermeasure System [for decoying certain types of torpedoes] —[the] Advanced Degaussing System (Anti-Mine) [for reducing the ship's magnetic signature, so as to reduce the likelihood of attack by magnetically fused mines] When required, the T-AO(X) will also have ability to embark Navy Expeditionary Combat Command Expeditionary Security Teams (EST)  The ESTs will embark with several crew served weapons and are designed to provide limited self-defense against a small boat attack  The T-AO(X) will have Space, Weight, Power and Cooling (SWAP-C) margins for future installations of the following systems: —[the] Close In Weapon System (CIWS) or SeaRAM (Rolling Airframe Missile) [for defense against missile attack] —[the] Anti-Torpedo Torpedo Defense System (ATTDS) [for destroying torpedoes] Even after the installation of a CIWS or ATTDS, if the T-AO(X) was to operate in anything other than a benign environment, the ship will require both air and surface escorts  The decision to rely on [other] Fleet assets to provide force protection [i e , defense against attacks] for the T-AO(X) was validated by the JROC [in June 2015]  Table 2 summarizes congressional action on the Navy's request for FY2020 procurement funding for the TAO-205 program  A May 2019 GAO report—the 2019 edition of GAO's annual report surveying DOD major acquisition programs—stated the following regarding the TAO-205 program: Technology Maturity and Design Stability The Navy has matured all Lewis class critical technologies and stabilized the ships' design  In 2014, the Navy identified three critical technologies for the Lewis class, all of which involved a new system for transferring cargo at sea  Prior to initiating detail design activities in June 2016, the Navy completed prototype tests of the critical technologies and found that they were fully mature—an approach consistent with shipbuilding best practices  In 2017, the Navy removed one critical technology—the Heavy e-STREAM cargo delivery system—from the Lewis class design  The Navy had intended to use this system to deliver F-35 Lightning II power modules  The Navy subsequently decided to deliver these by air, which precluded any need for the Heavy system  Lead ship construction began in September 2018 with 95 percent of the ship's total design effort complete  Program officials stated that this figure meant that 100 percent of the ship's basic and functional design were by then complete—an approach consistent with best practices  Throughout detail design and now into construction, the Navy has not changed the Lewis class program's performance requirements  The Navy also leveraged commercial vessel designs to minimize design and construction risks  The Lewis class features a modern double-hull construction, an environmental-based design standard for commercial tankers, to ensure the ships can dock at ports-of-call  This design was included in the final three Kaiser class oilers  Production Readiness The program office has largely kept to its construction schedule to date for the first ship, but a flooding incident at a NASSCO graving dock in July 2018 has affected the delivery of future ships  The program office stated that this incident has not affected current ship fabrication activities  However, the dock's unavailability while repairs are planned and implemented has disrupted the contractor's schedule for future ships  According to the program office, the incident has resulted in some delays to certain delivery dates for ships two through six  Other Program Issues As part of the Navy's plan to expand the fleet, the Navy concluded that it would need an additional three Lewis class ships  The Navy's budget request for fiscal year 2019 increased its planned one-ship-per-year buy to two for fiscal years 2019, 2021, and 2023  The Congress provided appropriations for the additional fiscal year 2019 ship in support of the Navy's request  To account for the additional ships in fiscal years 2019 and 2021, the Navy plans to add two more ships to the low-rate initial production phase  Subsequently, program officials stated that they plan to compete a new contract for the remaining 12 ships using the construction knowledge gained from efforts under the existing contract  Program Office Comments We provided a draft of this assessment to the program office for review and comment  The program office provided technical comments, which we incorporated where appropriate  The program office stated that it continues to follow GAO shipbuilding best practices and has leveraged commercial vessel design practices to minimize risk  The program office also stated that it is currently revising its acquisition baseline to reflect the update in total quantities to 20 ships  In addition, the program office noted that, in fiscal year 2019, it fully funded the third and fourth ships and funded advance procurement for the fifth ship 

ground_truth: Commonly known for protecting the President, the Secret Service also plays a leading role in investigating and preventing financial and electronic crimes. To accomplish its mission, the Secret Service relies heavily on the use of IT infrastructure and systems. In 2009, the component initiated the IITT investment—a portfolio of programs and projects that are intended to, among other things, improve systems availability and security in support of the component's business operations. GAO was asked to review the Secret Service's oversight of its IT portfolio and workforce. This report discusses the extent to which the (1) CIO implemented selected IT oversight responsibilities, (2) Secret Service implemented leading IT workforce planning and management practices, and (3) Secret Service and DHS implemented selected performance monitoring practices for IITT. GAO assessed agency documentation against 14 selected component CIO responsibilities established in DHS policy; 15 selected leading workforce planning and management practices within 5 topic areas; and two selected leading industry project monitoring practices that, among other things, were, in GAO's professional judgment, of most significance to managing IITT. The U.S. Secret Service (Secret Service) Chief Information Officer (CIO) fully implemented 11 of 14 selected information technology (IT) oversight responsibilities, and partially implemented the remaining 3. The CIO partially implemented the responsibilities to establish a process that ensures the Secret Service reviews IT contracts; ensure that the component's IT policies align with the Department of Homeland Security's (DHS) policies; and set incremental targets to monitor program progress. Additional efforts to fully implement these 3 responsibilities will further position the CIO to effectively manage the IT portfolio. Of the 15 selected practices within the 5 workforce planning and management areas, the Secret Service fully implemented 3 practices, partly implemented 8, and did not implement 4 (see table). Within the strategic planning area, the component partly implemented the practice to, among other things, develop IT competency needs. While the Secret Service had defined general core competencies for its workforce, the Office of the CIO (OCIO) did not identify all of the technical competencies needed to support its functions. As a result, the office was limited in its ability to address any IT competency gaps that may exist. Also, while work remains to improve morale across the component, the Secret Service substantially implemented the employee morale practices for its IT staff. Secret Service officials said the gaps in implementing the workforce practices were due to, among other things, their focus on reorganizing the IT workforce within OCIO. Until the Secret Service fully implements these practices for its IT workforce, it may be limited in its ability to ensure the timely and effective acquisition and maintenance of the component's IT infrastructure and services. Of the two selected IT project monitoring practices, DHS and the Secret Service fully implemented the first practice to monitor the performance of the Information Integration and Technology Transformation (IITT) investment. In addition, for the second practice—to monitor projects on incremental development metrics—the Secret Service fully implemented the practice on one of IITT's projects and partially implemented it on another. In particular, OCIO did not fully measure post-deployment user satisfaction with the system on one project. OCIO plans to conduct a user satisfaction survey of the system by September 2018, which should inform the office on whether the system is meeting users' needs. GAO is making 13 recommendations, including that the Secret Service establish a process that ensures the CIO reviews all IT contracts, as appropriate; and identify the skills needed for its IT workforce. DHS concurred with all recommendations and provided estimated dates for implementing each of them.