Information Technology: DHS Needs to Enhance Management of Cost and Schedule for Major Investments
Published: 2012-09-26T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DHS has responsibility for the development and management of the IT systems for the 22 federal agencies and offices under its jurisdiction. Of its 363 IT investments, 68 are in development and are classified by DHS as a major investment that requires special management attention because of its mission importance. Given the size and significance of these investments, GAO was asked to determine the (1) extent to which DHS IT investments are meeting their cost and schedule commitments, (2) primary causes of any commitment shortfalls, and (3) adequacy of DHSs efforts to address these shortfalls and their associated causes.
To address these objectives, GAO analyzed recent cost and schedule performance for DHSs major IT investments, as reported to OMB. To identify the primary cause(s) of any shortfalls and whether any corrective efforts were being taken to address them, GAO analyzed project plans and related documentation and interviewed responsible DHS officials and compared the corrective efforts to applicable criteria to assess their adequacy.
section_title: What GAO Found
paragraphs: Approximately two-thirds of the Department of Homeland Securitys (DHS) major information technology (IT) investments are meeting their cost and schedule commitments (i.e., goals). Specifically, out of 68 major IT investments in development, 47 were meeting cost and schedule commitments. The remaining 21which total about $1 billion in spendinghad one or more subsidiary projects that were not meeting cost and/or schedule commitments (i.e., they exceeded their goals by at least 10 percent, which is the level at which the Office of Management and Budget (OMB) considers projects to be at increased risk of not being able to deliver planned capabilities on time and within budget.)
The primary causes for the cost and schedule shortfalls were (in descending order of frequency):
inaccurate preliminary cost and schedule estimates,
technology issues in the development phase,
changes in agency priorities,
lack of understanding of user requirements, and
dependencies on other investments that had schedule shortfalls.
Eight investments had inaccurate cost and schedule estimates. For example, DHSs Critical Infrastructure Technology investment had a project where actual costs were about 16 percent over the estimated cost, due in part to project staff not fully validating cost estimates before proceeding with the project. In addition, six investments had technical issues in the development phase that caused cost or schedule slippages. For example, DHSs Land Border Integration investment had problems with wireless interference at certain sites during deployment of handheld devices used for scanning license plates, which caused a project to be about 2.5 months late. In past work on DHS investments, GAO has identified some of the causes of DHSs shortfalls and made recommendations to strengthen management in these areas (e.g., cost estimating, requirements), and DHS has initiated efforts to implement the recommendations.
DHS often did not adequately address shortfalls and their causes. GAOs investment management framework calls for agencies to develop and document corrective efforts to address underperforming investments. DHS policy requires documented corrective efforts when investments experience cost or schedule variances. Although 12 of the 21 investments with shortfalls had defined and documented corrective efforts, the remaining 9 did not. Officials responsible for 3 of the 9 investments said they took corrective efforts but were unable to provide plans or any other related documentation showing such action had been taken. Officials for the other 6 investments cited criteria in DHSs policy that excluded their investments from the requirement to document corrective efforts. This practice is inconsistent with the direction of OMB guidance and related best practices that stress developing and documenting corrective efforts to address problems in such circumstances. Until DHS addresses its guidance shortcomings and ensures each of these underperforming investments has defined and documented corrective efforts, these investments are at risk of continued cost and schedule shortfalls.
section_title: What GAO Recommends
paragraphs: GAO is recommending that the Secretary of Homeland Security direct the appropriate officials to address the guidance shortcomings and develop corrective actions for all major IT investment projects having cost and schedule shortfalls. In commenting on a draft of this report, DHS concurred with GAOs recommendations.

section_title: Letter
paragraphs: As part of its mission to protect the homeland, the Department of  Homeland Security (DHS) is responsible for 22 agencies and offices with  related missions and for the development and management of the  information technology (IT) investments and systems used by those  organizations. For its approximately 363 ongoing IT investments for fiscal  year 2012, DHS plans to spend about $5.6 billion. Of these 363  investments, 68 are under development and are classified by DHS as a  “major” investmentof its importance to DHS’s mission.    that requires special management attention because  Given the size and significance of these investments, you asked us to  determine the (1) extent to which DHS IT investments are meeting their  cost and schedule commitments, (2) primary causes of any commitment  shortfalls, and (3) adequacy of DHS’s efforts to address these shortfalls  and their associated causes. In addition to this report, we have also  recently issued a report focused on the management and performance of  DHS’s major acquisitions.
To address these objectives, we analyzed how each of these 68 major  investments was performing against its cost and schedules commitments as reported by the department to the Office of Management and Budget  (OMB) as of March 2012. In doing this, we identified investments that had  one or more projects exceeding 10 percent of their cost and schedule  commitments. We focused on these investments and on their subsidiary  projects because they are under development and because OMB  considers them to be at an increased level of risk of not being able to  deliver promised capabilities on time and within budget, and thus they  require special attention from management. For each of these  investments and associated projects experiencing such shortfalls, we  reviewed project plans and related documentation and interviewed  responsible DHS officials to identify the primary cause(s) for the shortfalls  and whether any corrective efforts had been developed and documented  to address the shortfalls. We also compared the corrective efforts to  applicable criteria to assess their adequacy.
Commitments are cost and schedule goals or targets that the department defined for the  investment in its investment planning and approval documents, such as cost and schedule  baselines. details on our objectives, scope, and methodology are included in  appendix I.
subsections: section_title: Background
paragraphs: DHS’s mission is to lead the unified national effort to secure America by  preventing and deterring terrorist attacks and protecting against and  responding to threats and hazards to the nation. DHS is also responsible  for ensuring that the nation’s borders are safe and secure, that they  welcome lawful immigrants and visitors, and that they promote the free  flow of commerce.
Created in 2002, DHS assumed control of about 209,000 civilian and  military positions from 22 agencies and offices that specialize in one or  more aspects of homeland security. The purpose behind the merger was  to improve coordination, communication, and information sharing among  these multiple federal agencies. Figure 1 shows DHS’s organizational  structure and table 1 identifies DHS’s principal organizations and  describes their missions.
Within the department’s Management Directorate, headed by the Under  Secretary for Management, is the Office of the Chief Information Officer  (CIO). The CIO’s responsibilities include setting departmental IT policies,  processes, and standards, and ensuring that IT acquisitions comply with  DHS IT management processes, technical requirements, and approved  enterprise architecture, among other things. Additionally, the CIO chairs  DHS’s Chief Information Officer Council (CIO Council), which is  responsible for ensuring the development of IT resource management  policies, processes, best practices, performance measures, and decision  criteria for managing the delivery of IT services and investments, while  controlling costs and mitigating risks.
subsections: section_title: DHS IT Investments Support Its Mission
paragraphs: DHS spends billions of dollars each year on IT investments to perform  both mission-critical and support functions that frequently must be  coordinated among components, as well as among external entities. Of  the $5.6 billion that DHS plans to spend on 363 IT-related investments in  fiscal year 2012, $4.4 billion is planned for the 83 the agency considers to  be a major investment; namely, costly, complex, and/or mission critical.
Of these 83 major IT investments, 68 are under development and have  planned fiscal year 2012 costs of approximately $4 billion. Examples of  major investments under development that are being undertaken by DHS  and its components include:
U. S. Customs and Border Protection—Automated Commercial  Environment/International Trade Data System will incrementally  replace existing cargo processing technology systems with a single  system for land, air, rail, and sea cargo and serve as the central data  collection system for federal agencies needing access to international  trade data in a secure, paper-free, web-enabled environment.
Immigration and Customs Enforcement —TECS Modernization is  to replace the legacy mainframe system developed by the U.S.  Customs Service in the 1980s to support its inspections and  investigations. Following the creation of DHS, those activities were  assigned to CBP and ICE, respectively. CBP and ICE are now  working to modernize their respective portions of the system in a  coordinated effort with separate funding and schedules. ICE’s portion  of the investment will include modernizing the investigative case  management and related support modules of the legacy system.
National Protection and Programs Directorate—National  Cybersecurity Protection System, also referred to as EINSTEIN, is  an integrated system that includes intrusion detection, analytics,  intrusion prevention, and information sharing capabilities that are used  to defend the federal executive branch civilian agencies’ IT  infrastructure from cyber threats. It consists of the hardware, software,  supporting processes, training, and services that are being developed  and acquired to support DHS’s mission requirements.
section_title: Best Practices and OMB Guidance Stress Importance of Keeping IT Investments within Cost and on Schedule
paragraphs: The success of major IT investments are judged by, among other things,  the extent to which they deliver promised system capabilities and mission  benefits on time and within cost. Consequently, our best practices  research and extensive experience at federal agencies, as well as OMB  guidance, stress the importance of federal IT investments meeting cost  and schedule milestones.
GAO’s Information Technology Investment Management guidancehighlights the need to regularly determine each IT project’s progress  toward cost and schedule milestones using established criteria and calls  for corrective efforts when milestones are not being met. The guidance  also calls for such corrective efforts to be defined and documented.   OMB plays a key role in helping federal agencies manage their  investments by working with them to better plan, justify, and determine  how much they need to spend on projects and how to manage their  approved projects. In December 2010, OMB issued its 25 Point  Implementation Plan to Reform Federal Information Technology  Management, a plan to change IT management throughout the federal  government by strengthening the role of investment review boards to  enable them to more adequately manage agency IT portfolios, redefining  the role of agency CIOs and the Federal CIO Council to focus on  portfolio management, and implementing face-to-face reviews to identify  IT investments that are experiencing performance problems and to select  them for a TechStat session—a review of selected IT investments  between OMB and agency leadership that is led by the Federal CIO.   In addition, OMB provides agencies with tools to measure how effectively  investments are meeting established cost and schedule parameters.  Specifically, OMB requires federal agencies to provide information on  their IT investments as a part of their yearly budget submissions, and to  do so using an exhibit 53, in which they list all of their IT investments and  their associated costs, and an exhibit 300, also called the Capital Asset  Plan and Business Case, which includes an investment’s cost and  schedule commitments.
Further, in June 2009, OMB deployed the IT Dashboard, a website that  displays near real-time information on, among other things, the cost and  schedule performance of all of an agency’s major IT investments. The IT  Dashboard provides, among other things, a cost and schedule  performance rating for each major IT investment’s subsidiary project.  These ratings are based on the extent to which the project is meeting its  cost and schedule commitments. For example, projects experiencing a 10  percent or greater cost and/or schedule variance are considered to be at  an elevated risk of not delivering promised capabilities on time and within  budget, and, as such, require management attention.
section_title: Prior GAO Reports Highlighted Cost and Schedule Shortfalls of Major DHS Investments
paragraphs: We have previously reported on the cost and schedule challenges  associated with major DHS IT investments, such as those with CBP’s  Secure Border Network (SBInet) and NPPD’s United States Visitor and  Immigrant Status Indicator Technology (US-VISIT). For example, in 2007  we reported that the Secure Border Network had experienced significant  cost and schedule shortfalls due, in part, to the project not having fully  defined activities. In addition, in May 2010, we reported that continued  delays to the investment were likely because, among other things, it had  not developed a reliable integrated master schedule and the schedule did  not adequately capture all necessary activities. In these reports, we  made recommendations to strengthen the program weaknesses to keep  the investment on schedule and within cost.
With regard to the US-VISIT investment, we noted in a November 2009  report that officials had not adopted an integrated approach to scheduling,  executing, and tracking the work that needed to be accomplished to  deliver the Comprehensive Exit project to more than 300 ports of entry on  schedule and within cost. Accordingly, we recommended that DHS  strengthen management of the project by ensuring that it develop and  maintain integrated scheduling plans in accordance with applicable key  practices; DHS concurred with the recommendations.
Further, in 2011, as a part of our High Risk series, because of acquisition weaknesses, major investments, such as the  recently canceled SBInet, continued to be challenged in meeting  capability, benefit, cost, and schedule expectations. Based on our prior  work, we identified and provided to DHS key actions and outcomes  critical to addressing this and other challenges.
Most recently, we reported  in July 2012 that DHS was making progress  in developing and implementing a new IT governance process. We found  that DHS had developed a new governance framework and that the  associated policies and procedures were generally consistent with recent  OMB guidance and with best practices for managing projects and  portfolios identified in GAO’s Information Technology Investment  Management framework; however, the agency had not yet finalized most  policies and procedures and was not fully using best practices for the  implementation. Accordingly, we made recommendations to DHS to,  among other things, strengthen its new governance process and related  IT management capabilities; the agency agreed to implement the  recommendations.
GAO, High Risk Series: An Update, GAO-11-278 (Washington, D.C.: February 2011).
GAO, Information Technology: DHS Needs to Further Define and Implement Its New  Governance Process, GAO-12-818 (Washington, D.C.: July 2012).
section_title: DHS Meeting Cost and Schedule Commitments for Most Major IT Investments
paragraphs: As discussed previously, our best practices research and experience at  federal agencies as well as OMB guidance stress the importance of  investments meeting their cost and schedule commitments. OMB requires  agencies to report to the IT Dashboard information on the cost and  schedule performance of all their major IT investments.
Our analysis of the cost and schedule performance for DHS’s 68 major IT  investments shows that approximately two-thirds of these investments  and their subsidiary projects were meeting cost and schedule  commitments; the remaining one-third had at least one subsidiary project  that was not meeting its commitments. Specifically, out of the 68 major  investments under development, 47 were meeting their cost and schedule  commitments. (See app. II for a listing of the 47 investments and  subsidiary projects that are meeting their commitments.) The remaining  21 investments had one or more subsidiary projects that were not  meeting cost and/or schedule commitments; the total planned cost for all  projects in development for the 21 investments is approximately $1 billion.  Table 2 lists the investments experiencing cost and/or schedule shortfalls,  and the total planned project cost for each investment. A list of the  investments and their subsidiary projects experiencing cost and/or  schedule shortfalls is included in appendix III.
Of the 21 investments with a shortfall, 5 had one or more subsidiary  project with a cost shortfall, 18 had one or more project with a schedule  shortfall, and 2 had a project with both a cost and schedule shortfall.  These shortfalls potentially impact the total cost of investments and can  delay the implementation of key systems. For example:
TSA’s Federal Air Marshal Service Mission Scheduling and  Notification System: project to modernize the core scheduling  software component of the system, which, among other things,  determines the allocation of federal air marshals to flights and  coordinates and communicates mission assignments, was delayed.
NPPD’s Critical Infrastructure Technology and Architecture  investment: project to develop an information-sharing application to be  used by federal, state, and local stakeholders to increase their  capability to combat terrorist use of improvised explosive devices had  cost overruns of approximately 16 percent ($296,000).
CBP’s Northern Border, Remote Video Surveillance System  investment: project to incorporate IT Security improvements to the  remote video surveillance systems in Buffalo, New York, and Detroit,  Michigan, was delayed by approximately 2 months.
FEMA’s Disaster Assistance Improvement Plan: a subsidiary  project—site usability enhancements—that included enhancements to  the DisasterAssistance.gov website to improve usability by making it  easier and more intuitive for users to apply for and find information  about disaster assistance from federal, state, local, tribal, and private  nonprofit organizations was delayed.
section_title: Causes of Investment Cost and Schedule Shortfalls Vary
paragraphs: The primary causes of the shortfalls in cost and schedule associated with  DHS’s 21 major IT investments were (in descending order of frequency):  inaccurate preliminary cost and schedule estimates, technical issues in  the development phase, changes in agency priorities, lack of  understanding of user requirements, and dependencies on other  investments that had schedule shortfalls. A summary of these causes by  investment and the associated component are shown in table 3 and are  followed by (1) our analysis of these causes by category and (2)  discussion of our past work on the department’s major investments and  related IT management processes where we identified some of these  same causes and made recommendations to strengthen management in  these areas.
Specifically, our analysis of these causes by category showed:  Inaccurate preliminary cost and schedule estimates: Inaccurate cost  and schedule estimates in eight investments resulted in significant cost  and schedule increases. For example:
Preliminary schedule estimates for a project under CBP’s Non- Intrusive Inspection Systems Program investment—which supports  the detection and prevention of contraband from entering the  country—were inaccurate due to underestimating the time needed to  complete a key task. Specifically, project officials did not accurately  estimate how long it would take to complete an environmental  assessment because they did not consider all requirements in their  initial planning, thus resulting in a schedule delay of approximately 2  months.
The NPPD investment called Critical Infrastructure Technology and  Architecture had a project—integral to developing an information  sharing application to be used by federal, state, and local  stakeholders to increase their capability to combat terrorist use of  improvised explosive devices—where actual costs for completing  critical tasks were about 16 percent over the cost estimated at project  initiation. According to investment officials, this was due in part to  project staff developing the cost estimates very quickly and not fully  validating them before proceeding with the project.
TSA’s Hazmat Threat Assessment Program (which performs a threat  assessment on commercial truck drivers who transport hazardous  materials to determine the threat status to transportation security) had  a schedule shortfall with a project, because, in part, the time needed  to modify a contract was not accurately estimated, which led to a  schedule delay of nearly 3 months.
Technology issues in the development phase: Technical issues in the  development phase caused cost or schedule slippages in six  investments. Examples include:
Changes made to one part of ICE’s Detention and Removal  Operations Modernization investment, which is designed to upgrade  IT capabilities to support efficient detention and removal of non-U.S.  citizens, created a cascading effect, leading to changes to other parts  of the system and contributed to delays of more than a month.
Issues in establishing a testing and development environment that  matched the production environment delayed project testing in several  projects under FEMA’s Disaster Assistance Improvement Plan  investment (which is to ease the burden on disaster survivors by  providing them with a mechanism to access and apply for disaster  assistance).
Technical complications during deployment caused the schedule to  slip by 79 days on a project under CBP’s Land Border Integration  investment, which assists with the processing of inbound and  outbound travel at border patrol checkpoints nationwide. Specifically,  the handheld devices used for scanning license plates used a  wireless spectrum that had interference problems at certain sites, and  resolving this issue took more time than had been planned for.
Changes in agency priorities: Four investments experienced cost and  schedule slippages due to changing priorities at the agency level. In  particular,
The schedules were delayed for two NPPD US-VISIT investments:  the Arrival and Departure Information System, which collects arrival  and departure information on non-U.S. citizens traveling to the United  States as well as current immigration status updates for each traveler,  and the Automated Biometric Identification System, a fingerprint  repository and biometric-matching system. Delays were due to a  management decision to focus on accelerating the development of  other investments or projects, which took resources (i.e., personnel)  away from the investment. Consequently, the Arrival and Departure  Information System’s fiscal year 2011 maintenance release project  was delayed approximately 3 months, and the Automated Biometric  Identification System’s fiscal year 2011 product support project was  delayed by approximately 7 months.
A critical subsidiary project to deliver predictive analytical capabilities  under USCG’s Business Intelligence investment, which is designed to  reduce organizational uncertainty and risk in decision making, had a  schedule delay of approximately 3 months due to changing priorities.  Project officials said that Coast Guard management directed  resources to other projects with a higher priority, thus limiting the  ability to work on the predictive analytics capability project.
Lack of understanding user requirements: Three investments had  slippages resulting from misunderstanding or inadequately developed  user requirements and expectations.
A project under USCIS’s Claims 4 investment, which is a processing  system for the adjudication of naturalization applications, was delayed  by 2 weeks because inadequate user requirements led to a design  flaw that required additional time to address.
Customer priorities and expectations for ICE’s Detention and Removal  Operations Modernization investment changed over time, which  contributed to schedule delays of more than a month.
The schedule for a CBP TECS Modernization investment project,  which supports the screening of travelers entering the United States,  was delayed due to users requesting that the application in  development interface with a separate system. The project was  delayed by 3 months while program officials developed new  requirements.
Dependencies on other component’s investments that had schedule  shortfalls: Investments also encountered schedule slippages when  interdependent investments encountered delays. For example:
USSS’s Information Integration and Technology Transformation  investment to provide advanced security measures to electronically  send, receive, and track access to USSS’s unclassified and classified  information was delayed approximately 6 months due to a  component’s project being delayed.
Costs for a project under FEMA’s Disaster Assistance Improvement  Plan investment rose approximately 27 percent ($210,000) due, in  part, to the delayed deployment of another investment.
Other causes of cost and schedule slippages that were cited by  department officials included delays in receiving funding and gaps in  leadership due to key management turnover. Specifically,
The schedule for three projects under TSA’s Federal Air Marshal  Service Mission Scheduling and Notification investment was delayed  due to delays in receiving full funding. DHS had provided the  investment with partial funding, and thus investment officials produced  an investment plan based on that funding level; when full funding was  subsequently restored, the plan had to be updated, which resulted in  delays.
The costs for a key subsidiary project of NPPD’s Infrastructure  Security Compliance, Chemical Security Assessment Tool, which is to  provide for the electronic submission of chemical facility data and  controlled use of such data, rose approximately 20 percent ($719,000)  due, in part, to multiple director-level program changes, which led to  corresponding changes in the investment’s vision and direction.
In our past work on DHS’s investments and related IT management  processes, we have identified some of these same causes and made  recommendations to strengthen management in these areas. For  example, with regard to cost estimating, we reported that forming a  reliable estimate of costs provides a sound basis for measuring against  actual cost performance and that the lack of such a basis contributes to  variances. To help agencies establish such a capability, we issued a  guide in March 2009 that was based on the practices of leading  organizations. In a July 2012 report examining how well DHS is  implementing these practices, we reported that the department had  weaknesses in cost estimating. Accordingly, we made  recommendations to DHS to strengthen its cost estimating capabilities,  and the department has plans and efforts under way to implement our  recommendations.
GAO, Department of Homeland Security: Assessments of Selected Complex  Acquisitions, GAO-10-588SP (Washington, D.C.: June 2010).
section_title: About Half of DHS’s Investments with Shortfalls Do Not Have Well-Developed Corrective Efforts
paragraphs: A variety of best practices exist to guide the successful acquisition of IT  investments, including how to develop and document corrective actions  for projects experiencing cost and schedule shortfalls. In particular,  GAO’s Information Technology Investment Management framework calls  for agencies to develop and document corrective efforts for  underperforming projects. It also states that agencies are to ensure that,  as projects develop and costs rise, the project continues to meet mission  needs at the expected levels of cost and risk; if projects are not meeting  expectations or if problems have arisen, agencies are to quickly take  steps to address the deficiencies. In addition, DHS policy requires  corrective actions when cost or schedule variances exceed 8 percent.
DHS developed and documented corrective efforts for 12 of the 21 major  investments with a shortfall, but the remaining 9 did not have documented  corrective efforts. Table 4 depicts the investments with shortfalls and  whether corrective efforts had been developed and documented.
DHS took corrective actions to address investment shortfalls in 12  investments. Actions for the 12 included:
CBP: Automated Commercial Environment /International Trade Data  System had schedule shortfalls due to an inadequate testing and  development environment; they were resolved by leveraging CBP’s  disaster recovery site to perform the testing.
CBP: Land Border Integration investment schedule delays due to  technical complications were resolved through the use of risk  management processes (e.g., identification, assessment, tracking,  and mitigation of risks) identified in the investment’s July 2011 risk  management plan, which addressed the cause of the investment  shortfalls.
CBP: Non-Intrusive Inspection Systems Program, which had schedule  shortfalls due to inaccurate estimates, tracked the project status and  risks via project health status reports and other mitigation strategies.
CBP: The Remote Video Surveillance System investment officials  developed and documented an investment rebaseline to address the  investment’s schedule shortfall, which was due to an inaccurate initial  project schedule estimate.
CBP: The TECS Modernization investment—which had schedule  shortfalls due to system development being delayed due to questions  about whether planned enhancements duplicated functions performed  by another agency system—program officials briefed key  management on the differences between the system functions and  development was allowed to continue.
FEMA: Disaster Assistance Improvement Plan cost and schedule  shortfalls were due to, among other things, dependencies on other  investments. DHS developed a remediation plan for each shortfall to  limit the negative impact.
ICE: Detention and Removal Operations Modernization investment  schedule shortfalls were due in part to a lack of understanding of user  requirements; to address these issues, investment officials worked  with key stakeholders to engage users to more thoroughly identify  user requirements.
NPPD: Critical Infrastructure Technology and Architecture  investment’s cost shortfalls, which were due to inaccurate initial cost  estimates, were resolved by investment officials through several  corrective efforts, including completing the project’s life cycle cost  estimate.
NPPD: Infrastructure Security Compliance, Chemical Security  Assessment Tool investment schedule was delayed due to multiple  changes in leadership and in the investment’s direction. Project  officials developed and documented an investment rebaseline, which  was approved in February 2012. It was intended to, among other  things, develop a more accurate schedule.
TSA: The Hazmat Threat Assessment Program investment schedule  was delayed because the time needed to adjust a contract had not  been accurately estimated. In response, investment officials  documented an investment rebaseline, which was approved in March  2012.
TSA: The Security Technology Integrated Program investment had  schedule shortfalls from inaccurate estimates of the time needed to  revise a contract. To resolve these issues, officials developed and  documented an initiative to improve methods used to identify and  track risks and resolve the schedule shortfalls. This effort is intended  to help the investment avoid additional schedule changes.
USSS: The Information Integration and Technology Transformation  investment had one project with schedule shortfalls due to  dependencies on another component’s investment that had schedule  slippages; issues were addressed by following the mitigation actions  detailed in the investment’s risk management plan.
With regard to the remaining nine investments, three were unable to  provide us with documentation, even though project officials stated that  they had developed some corrective efforts, and six did not engage in  corrective efforts to address shortfalls. Of the three investments, officials  from TSA’s Federal Air Marshal Service Mission Scheduling and  Notification System investment, for example, reported that they had  addressed the project’s schedule shortfall—which was due, in part, to a  support contractor not having adequate staffing—by performing the work  within the agency instead of relying on the contractor. Further, according  to TSA officials, the cost and schedule shortfalls on the Air Cargo Security  investment, which were due to technical complications and dependencies  on other investments, were addressed by establishing a new cost and  schedule baseline. Nonetheless, this lack of documentation is  inconsistent with the direction of DHS’s guidance and related best  practices, and it shows a lack of process discipline and attention to key  details, which raises concern about the thoroughness of corrective efforts.
Of the six investments without any corrective efforts, officials from these  investments (namely, the Office of the Chief Information Officer’s Human  Resources IT investment, NPPD’s US-VISIT Automated Biometric  Identification System and Arrival and Departure Information System  investments, USCG’s Business Intelligence investment, NPPD’s National  Cybersecurity Protection System, and USCIS’s Claims 4 investment),  stated that they did not develop and document corrective efforts because  they believed DHS’s guidance does not call for it in their circumstances.  Specifically, the officials said that although DHS’s guidance calls for  corrective actions to be developed and documented when an investment  or its projects experiences a life cycle cost or schedule variance of 8  percent or greater, the variances on their project activities thus far were  not large enough to constitute such a life cycle variance.
The impact of this is that multiple projects can continue to experience  shortfalls—which increases the risk that investments will experience  serious lifecycle cost and schedule variances—without having to develop  and document corrective actions and thus alert top management about  potential problems and associated risks. This is inconsistent with the  direction of OMB, which requires agencies to report (via the IT  Dashboard) on the cost and schedule performance of their projects and  considers those projects with a 10 percent or greater variance to be at an  increased level of risk of not being able to deliver promised capabilities on  time and within budget, and thus they require special attention from  management. It is also inconsistent with our best practices research and  experience at federal agencies, which stresses that agencies report to  management when projects are not meeting expectations or when  problems arise and quickly develop and document corrective efforts to  address the problems. Further, our research and work at agencies has  shown that waiting to act until significant life cycle variances occur can  sometimes be risky and costly, as life cycle schedules are typically for  multiyear periods, allowing the potential for underperforming projects to  continue to vary from their cost and schedule goals for an extended  amount of time without any requirement for corrective efforts.  Consequently, until these guidance shortcomings are addressed and  each underperforming project has defined and documented corrective  actions, the department’s major investments these projects support will  be at an increased risk of cost and schedule shortfalls.
section_title: Conclusions
paragraphs: Most of the projects comprising DHS’s 68 major IT investments are  meeting their cost and schedule commitments, but 21 major  investments—integral to DHS’s mission and costing approximately $1  billion—have projects that are experiencing significant cost and schedule  shortfalls. These shortfalls place these investments at increased risk of  not delivering promised capabilities on time and within budget, which, in  turn, pose a risk to DHS’s ability to fully meet its mission of securing the  homeland. DHS guidance does not require projects experiencing  significant cost and schedule shortfalls to develop and document  corrective efforts until they cause a life cycle cost and schedule variance.  This increases risk and is contrary to effective IT investment practices.  Given that DHS is currently establishing and implementing new IT  governance processes, the department is positioned to address the  guidance shortfalls.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Homeland Security direct the  appropriate officials to:
Establish guidance that provides for developing corrective efforts for  major IT investment projects that are experiencing cost and schedule  shortfalls of 10 percent or greater, similar to those identified in this  report.
Ensure that major IT investment projects with shortfalls of 10 percent  or greater have defined and documented corrective efforts.
section_title: Agency Comments and Our Evaluation
paragraphs: In its written comments signed by the Director for the Departmental GAO- OIG Liaison Office and reprinted in appendix IV, DHS concurred with our  recommendations and estimated that it would implement the first  recommendation by September 30, 2013, and the second one  immediately. It also commented that the department was pleased that the  report positively acknowledged that DHS (1) is meeting cost and schedule  commitments for most of its major IT investments and (2) has plans and  efforts under way to improve cost estimating capabilities and implement a  center of excellence for requirements engineering. The department also  provided technical comments, which we have incorporated where  appropriate.
We are sending copies of this report to interested congressional  committees and the Secretary of Homeland Security. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staffs have any questions on the matters discussed in this  report, please contact me at (202) 512-9286 or pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to determine the (1) extent to which  Department of Homeland Security (DHS) IT investments are meeting their  cost and schedule commitments, (2) primary causes of any commitment  shortfalls, and (3) adequacy of DHS’s efforts to address these shortfalls  and causes.
To address our first objective, we analyzed how each of DHS’s 68 major  investments under development was performing against its cost and  schedules commitments, as reported by the agency to the Office of  Management and Budget (OMB) for inclusion on OMB’s federal IT  Dashboard. More specifically, we analyzed the extent to which each of  these investments had met or exceeded, as of March 2012, cost and  schedule commitments established when the investment was initiated. In  doing this, we identified investments that had a project exceeding 10  percent of its cost and schedule commitments. We focused on these  investments and their subsidiary projects because OMB considers them  to be at an increased level of risk of not being able to deliver promised  capabilities on time and within budget, and thus requiring special attention  from management.
To assess the reliability of the IT Dashboard data we analyzed, we  corroborated the data by interviewing investment and other DHS officials  to determine whether the information on the dashboard was consistent  with that reported by DHS. In addition, we followed up on the status of  implementation of previous GAO recommendations to improve the quality   Specifically, we analyzed  of information on OMB’s federal IT Dashboard.plans and related documentation describing efforts by DHS to increase  the scrutiny and quality of data submitted to the IT Dashboard. As part of  this, we also interviewed department officials including those from the  Office of the Chief Information Officer who are responsible for reviewing  and submitting DHS’s investment cost and schedule data to the federal IT  Dashboard. The documentation and interviews provided us a level of  assurance that the data we used for this engagement were, in fact,  reliable.
For our second objective, we used a structured interview instrument to  survey the DHS and component officials responsible for the investments  experiencing cost and schedule shortfalls in order to identify the causes  of the shortfalls. As part of surveying these officials, we analyzed project  and related documentation to corroborate the causes reported to us via  the survey. We then analyzed these causes for commonalities, grouped  them accordingly, and tallied the frequency of each cause by investment.  In addition, we compared the causes to our prior reports on major DHS  investments and related IT management processes to identify the extent  to which we had made recommendations to address the causes  associated with the department’s investment cost and schedule shortfalls.
To address our third objective, we initially identified and reviewed relevant  criteria on developing and documenting corrective actions to address  investment shortfalls. Specifically, these criteria included
DHS’s Acquisition Directive 102 (AD-102),
DHS’s Capital Planning and Investment Control Guide, and
GAO’s Information Technology Investment Management guide.
We then used a structured interview instrument to survey DHS and  component officials responsible for those investments experiencing  shortfalls; we used the survey to identify whether any corrective actions  had been developed and documented to address investment shortfalls.  We also reviewed investment planning and execution documentation  (e.g., project plans, project status reports, program meeting minutes, and  acquisition program baselines) to corroborate information provided by the  officials during the survey process. We then compared these corrective  efforts to the criteria to identify any gaps and in those cases where there  were, we reviewed documentation and interviewed agency officials to  assess the reason for the gaps and any negative impacts.
GAO, Information Technology Investment Management: A Framework for Assessing and  Improving Process Maturity, GAO-04-394G (Washington, D.C.: Mar. 1, 2004). obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives.
section_title: Appendix II: List of DHS Major IT Investments that Are Meeting Cost and Schedule Commitments
paragraphs: Table 5 lists the DHS major IT investments that were meeting their cost  and schedule commitments.
section_title: Appendix III: List of DHS Major IT Investments and Subsidiary Projects with Shortfalls
paragraphs: Table 6 lists the DHS major IT investments with cost and/or schedule  shortfalls, their costs and subsidiary projects, whether or not they had a  cost and/or schedule shortfall, and their planned costs.
section_title: Appendix IV: Comments from the Department of Homeland Security
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, the following staff also made key  contributions to this report: Gary Mountjoy (assistant director), Scott  Borre, Camille Chaires, and Nancy Glover.