Information Technology: FBI Is Building Management Capabilities Essential to Successful System Deployments, but Challenges Remain
Released: 2005-09-14T00:00:00

Letter

Background

The FBI’s mission responsibilities include investigating serious federal crimes, protecting the nation from foreign intelligence and terrorist threats, and assisting other law enforcement agencies. Approximately 12,000 special agents and 16,000 analysts and mission support personnel are located in the bureau’s Washington, D.C., headquarters and in more than 450 offices in the United States and 45 offices in foreign countries.

Mission responsibilities at the bureau are divided among the following five major organizational components. ● Administration: manages the bureau’s personnel programs, budgetary and financial services, records, information resources, and information security. ● Counterterrorism and Counterintelligence: identifies, assesses, investigates, and responds to national security threats. ● Criminal Investigations: investigates serious federal crimes and probes federal statutory violations involving exploitation of the Internet and computer systems. ● Intelligence: collects, analyzes, and disseminates information on evolving threats to the United States. ● Law Enforcement Services: provides law enforcement information and forensic services to federal, state, local, and international agencies.

The components are further organized into subcomponents, such as divisions, offices, and other groups (hereafter referred to as “divisions”). Table 1 lists the components and briefly describes their respective divisions.

To execute its mission responsibilities, the FBI relies extensively on IT, and this reliance is expected to grow. For example, the bureau operates and maintains hundreds of computerized systems, networks, databases, and applications, such as ● the Combined DNA Index System, to support forensic ● the National Crime Information Center and the Integrated Automated Fingerprint Identification System, to help state and local law enforcement agencies identify criminals; ● the Automated Case Management System, to manage information collected on investigative cases; ● the Investigative Data Warehouse, to aggregate data in a standard format from disparate databases to facilitate content management and data mining; and ● the Terrorist Screening Database, to consolidate identification information about known or suspected international and domestic terrorists.

According to the FBI, it also has almost 500 systems, applications, databases, and networks that are in operation, undergoing enhancement, or being developed or acquired. In particular, it has identified 18 new or enhancement projects that support its intelligence, investigative, and analyst activities. Included in these 18 is its Sentinel program, the FBI’s effort to deliver—using commercially available software and hardware components—a modern automated capability for investigative case management and information sharing, with the goal of helping field agents and analysts to perform their jobs more effectively and efficiently.

As we have previously reported, these ongoing and planned IT programs and projects are part of the FBI’s systems modernization program. This program is based both on the bureau’s long-standing recognition of its antiquated, nonintegrated systems environment and its awareness of the importance of modern, integrated IT systems to its transformation efforts in the wake of the September 11 attacks. Currently, the FBI reports that it will spend approximately $484 million on modernization projects in fiscal year 2005 out of a total IT budget of $1.07 billion.

Effective IT Management Is Critical to FBI’s Ability to Successfully Transform

Technology can be a valuable tool in helping organizations transform and better achieve mission goals and objectives. Our research on leading private and public sector organizations, as well as our past work at federal departments and agencies, shows that successful organizations embrace the central role of IT as an enabler for enterprisewide transformation. These leading organizations develop and implement institutional or agencywide system modernization management controls to ensure that the vast potential of technology is effectively applied to achieving mission outcomes. Among these management controls are ● assigning IT responsibility and providing commensurate authority centrally with the agency’s CIO, ● using a well-defined enterprise architecture as a systems modernization blueprint, ● following a portfolio-based approach to selecting among competing IT programs and projects and controlling investment in each during their life cycles, ● adhering to a structured and disciplined system development and acquisition life cycle management methodology, and ● employing sufficient and qualified IT human capital.

We have observed that without these types of controls and capabilities, organizations increase the risk that system modernization projects will (1) experience cost, schedule, and performance shortfalls and (2) lead to systems that are redundant and overlap. They also risk not achieving their aim of increased interoperability and effective information sharing. All told, this means that technology will not effectively and efficiently support agency mission performance and help realize strategic mission outcomes and goals.

The FBI Director has recognized the importance of IT to transformation, and accordingly made it one of the bureau’s top 10 priorities. Consistent with this, the FBI’s strategic plan contains explicit IT-related strategic goals, objectives, and initiatives (near- term and long-term) to support the collection, analysis, processing, and dissemination of information.

However, as we have previously reported, the bureau’s long- standing approach to managing IT has not always been fully consistent with leading practices. The effects of this approach can be seen in, for example, the cost and schedule shortfalls experienced on a key infrastructure and applications modernization program, Trilogy, and particularly on one of its projects (the Virtual Case File), which was recently terminated by the bureau. Reviews of this project identified management weaknesses as the cause for its cost, schedule, and performance shortfalls. Among these weaknesses were lack of integration planning, inadequately defined requirements, project management deficiencies, and frequent turnover of key personnel.

In place of the Virtual Case File project, the FBI launched its Sentinel program in early 2005 to develop what the bureau describes as its next-generation electronic information management system. According to the FBI, the system is planned to consolidate and replace its existing case management capabilities with an integrated, paperless file management and workflow system.

FBI Is Making Progress in Establishing Key IT Modernization Management Capabilities

The FBI is making progress in establishing institutional IT modernization management capabilities. It has centralized IT responsibility and authority under the CIO, and it is establishing and beginning to implement management capabilities in the areas of enterprise architecture, IT investment management, systems development and acquisition, and IT human capital. Before it can effectively leverage technology to transform itself, the FBI will have to build on these capabilities and effectively implement them on its system investments.

FBI Has Centralized Responsibility and Authority for IT

Our research on leading private and public sector organizations, as well as our past work at federal departments and agencies, shows that successful organizations adopt a corporate, or agencywide, approach to managing IT under the leadership and control of a senior executive—commonly called a chief information officer— who operates as a full partner with the organizational leadership team in charting the strategic direction and making informed IT investment decisions. The Clinger-Cohen Act also mandates that major federal departments and agencies establish the position of CIO. As the focal point for IT management within an agency, the CIO is positioned to oversee the establishment and implementation of agencywide capabilities in IT management.

In the FBI, responsibility for managing IT was historically decentralized and diffused. For example, we testified in March 2004 that the FBI had not provided its CIO with bureauwide IT management authority and responsibility, vesting these instead in the bureau’s divisions. This is part of the reason that the FBI’s IT environment at the time consisted of nonintegrated applications residing on different servers, each of which had its own unique databases, unable to share information with other applications or with other government agencies. To address this, we discussed with the Director in 2003 the importance of centralizing IT management responsibility and authority under the CIO, and we subsequently recommended that the CIO be provided with the responsibility and authority for managing IT bureauwide, including budget management control and oversight of IT programs and initiatives.

The FBI has since taken steps to strengthen the scope and influence of the CIO Office. In particular, the CIO was assigned agencywide responsibility, authority, and control over IT resources, including responsibility for preparing the bureau’s IT strategic plan and operating budget; operating and maintaining existing systems and networks; developing and deploying new systems; defining and implementing IT management policies, procedures, and processes; and developing and maintaining the bureau’s enterprise architecture.

To fulfill these responsibilities, the CIO’s office has begun the process of developing and implementing a corporatewide approach to managing IT. For example, the FBI reorganized the CIO Office, establishing four offices to carry out key institutional management functions, and issued an IT strategic plan in September 2004 that outlined ongoing and planned efforts to strengthen policies and procedures by standardizing them across the bureau and incorporating best practices. Among other things, this plan provided for building capabilities in a number of key IT management areas, including the following four areas: enterprise architecture, IT investment management, systems development and acquisition, and IT human capital.

FBI Is Taking Steps to Develop an Enterprise Architecture, but Much Work Remains To Be Done

As our research and evaluations have shown, it is risky to attempt to modernize an IT environment without using an architecture, or blueprint, to guide and constrain the definition, design, and development of IT programs and projects. An enterprise architecture provides systematic structural descriptions—in useful models, diagrams, tables, and narrative—of how a given entity operates today and how it plans to operate in the future, and it includes a road map for transitioning from today to tomorrow. Our experience with federal agencies has shown that attempting to modernize systems without having an enterprise architecture often results in systems that are duplicative, not well integrated, unnecessarily costly to maintain, and limited in terms of optimizing mission performance.

To assist agencies in effectively developing, maintaining, and implementing an enterprise architecture, we published a framework for architecture management, grounded in federal guidance and recognized best practices. In 2002 and again in 2003, we reported that the FBI did not have either an architecture to guide and constrain its IT investments or the means in place to develop and implement one. We further reported that the development of an architecture was not being given the priority that it deserved. Accordingly, we recommended that the Director make it an institutional priority, and provided a series of recommendations for building an architecture management foundation, developing and completing the architecture, and using it to inform IT investment decision making.

In the last 12 months, the bureau has made important progress in developing its architecture. Last week we issued a congressionally mandated report on the state of the FBI’s enterprise architecture efforts. In summary, we found that the FBI is now managing its enterprise architecture program in accordance with many best practices, but it has yet to adopt others. Examples of best practices that the bureau has implemented include the following: ● the bureau has established a program office that is responsible for the development of the architecture; it has issued a written and approved policy governing architecture development; and it has ongoing efforts to complete a target architecture.

We ascribed this important progress, in part, to the demonstrated commitment of the FBI’s top management to the enterprise architecture program. Nonetheless, we recognized that much remains to be accomplished before the FBI’s enterprise architecture program will be mature. For example, we reported that the architecture program office did not yet have appropriate human resources with architecture expertise and that the bureau was not following a defined methodology for developing its architecture, both of which are foundational items. Also, the bureau’s current and target architectures were not yet complete. (For instance, the program office had not completed mapping FBI data structures, classifications, and exchanges to the business processes that use the data, nor has it finished defining how the various IT applications currently interrelate.) Further, the bureau had not yet begun to develop its investment plans for transitioning from the current to the target architectural states.

We also reported that the FBI had not employed effective contract management controls in developing its enterprise architecture, which is risky because the bureau is relying heavily on contractor support in this effort. (We discuss this contract management issue further in the section of this testimony dealing with system development and acquisition.)

Because we had already made comprehensive recommendations regarding the FBI’s enterprise architecture program, we made no additional recommendations in this area. However, because of the FBI’s heavy reliance on contractor assistance in developing its architecture and the state of its contract management controls, we recommended that the FBI employ performance-based contracting on all further architecture contract actions (to the maximum extent practicable) and follow effective contract tracking and oversight practices.

In response, the FBI stated that it would continue to strive to develop a robust enterprise architecture program supported by effective contract management practices and cited steps under way to strengthen its architecture management foundation. For example, since our report was issued, the FBI provided us with a document that the bureau stated defines its enterprise architecture methodology. In addition, the bureau reported that it is very close to hiring staff with architecture expertise (four senior level technologists) for the program office. Further, the FBI stated that it was taking steps to increase its use of performance-based contracting.

FBI Is Beginning to Apply Its New Investment Management Approach, but More Remains to Be Done

Clinger-Cohen Act of 1996, 40 U.S.C. §§11101-11703. ● planned (proposed systems or system enhancements), ● under way (systems being developed or acquired), and ● completed (existing systems being operated and maintained).

The FBI’s progress over the last 3 years to define and refine an IT investment approach has been slow. In 2002, the bureau first focused on developing an approach that addressed solely IT investments and in 2003 expanded the approach’s scope to include all capital investments. In 2004, under the leadership of the current CIO, the bureau redirected its investment selection, control, and evaluation activities back to include IT investments only. In September 2004, we reported that this redirected approach included one set of processes for new investments that are planned and under way and another set for the operation and maintenance of existing systems. At that time, the process for investments in new systems was still being defined, while a process for allocating operations and maintenance resources across existing systems had been developed. We also reported that the bureau was to pilot test its developed process on different types of investments (systems, applications, databases, and networks) with the goal of subsequently implementing the process enterprisewide. In our view, it was important that the implemented process be in accordance with key IT investment decision-making best practices (such as our ITIM framework). Accordingly, we made recommendations aimed at expediting implementation of ITIM-compliant policies and procedures.

Since then, the FBI has taken a number of steps to strengthen its capability to manage IT investments. For example, in November 2004, the FBI established an investment review board, composed of senior executives, that meets about every 2 weeks to review proposed and ongoing investments in new systems. The CIO stated that the board recently completed its first evaluation of the bureau’s 89 ongoing IT investments to, among other things, establish cost, schedule, and performance baselines and to begin the process of having the CIO and other senior executive review the projects at critical development milestones. The CIO also reported that the bureau has reviewed over 37 new proposals and is using the results in preparing its fiscal year 2007 IT budget request. Further, to establish a more defined structure to support the board’s activities, the CIO’s office recently issued an ITIM guide, which defines, among other things, the processes that the board is to follow in selecting and controlling these investments.

In addition, the CIO’s office is in the process of assessing the performance of existing systems (i.e., those in the operations and maintenance phase of their life cycle). Using cost and other criteria, these assessments are designed to determine which systems can be better used, replaced, outsourced, or retired. According to the CIO, the program recently completed a pilot assessment of projects in one FBI division, and it is currently preparing to perform similar assessments in the other divisions, which are scheduled to be completed by April 2006.

Notwithstanding these efforts, until the FBI fully implements processes for selecting, controlling, and evaluating all its IT investments, it will not be able to ensure that it is applying its resources to the best mix of investments to meet the goals of modernizing IT and transforming itself.

The Bureau Has Moved to Standardize System Development and Acquisition Life Cycle Processes that Were Inconsistent across FBI Components

Having rigorous and disciplined IT system development and acquisition life cycle processes is an important component of IT management. The Clinger-Cohen Act recognizes the importance of such effective processes, and the Software Engineering Institute’s (SEI) Capability Maturity Models™ define a suite of such processes. Five process areas associated with systems acquisition (which collectively are composed of 30 key practice areas) are configuration management, project management, quality assurance, requirements development and management, and risk management. In combination with other process areas, these five provide a foundation for managing software-intensive systems in a manner that minimizes risks and increases the chances of systems delivering required system capabilities and benefits on time and within budget.

In September 2004, we reported that the life cycle management policies and procedures then in place at the FBI for these five areas varied widely by division. On the one hand, for example, the policies and procedures for the six divisions that we examined generally addressed all the practices associated with the project management process area (see table 2); this process area involves management of project office activities so that projects are timely, efficient, and effective.

On the other hand, for example, the policies and procedures for these six divisions generally did not address the key practices associated with requirements development and management process area (see table 3); this process area involves establishing and maintaining agreement on system requirements. We would note that according to the CIO, it was a lack of bureau rigor and discipline in this area that in part caused the Virtual Case File project to be terminated.

Examples of requirements development and management practices that most divisions did not adequately address are (1) appraising changes to requirements for their impact on the project or the IT environment, which is important because it allows management and the project team to determine whether the benefits of changes to the requirements would be worth the likely cost and effect of making the changes, and (2) developing and baselining requirements and maintaining them under change control, which is important to ensuring that requirements are completely and correctly defined and that uncontrolled changes, commonly referred to as “requirements creep,” are avoided.

In our September 2005 report, we addressed another key process area associated with system acquisition life cycle management— contract management. Federal acquisition regulations and relevant IT acquisition management guidance recognize the importance of effectively managing contractor activities. According to the Federal Acquisition Regulation (FAR), for example, agencies are to use performance-based contracting to the maximum extent practicable when acquiring most services. Under the FAR, performance-based contracting includes, among other things, defining the work to be performed in measurable, results-oriented terms and specifying performance standards (quality and timeliness). The FAR and associated regulations also require government oversight of contracts to ensure that the contractor performs the requirements of the contract, and the government receives the service as intended. Although the regulations do not prescribe specific methods for this oversight, other acquisition management guidance describes a number of practices associated with this activity.

However, the FBI’s approach to managing its enterprise architecture contract did not include most of the performance-based contracting features described in the FAR. For example, the contract’s statement of work did not specify the products in results-oriented, measurable terms. In addition, the bureau did not have plans for assuring the quality of the contractor’s work; instead, according to bureau officials, they worked with the contractor to determine whether each deliverable was acceptable.

In addition, in overseeing its contractor, the FBI has not employed the kind of effective practices specified in relevant guidance. For example, the bureau does not have a written policy to govern its tracking and oversight activities, has not designated responsibility or established a group for performing contract tracking and oversight activities, and has not developed an approved contractor monitoring plan.

To address weaknesses in the FBI’s systems development and acquisition life cycle processes, we have recommended that the FBI establish effective policies and procedures for such systems acquisition and development areas as configuration management, project management, quality assurance, requirements development and management, risk management, and contract tracking and oversight.

Recognizing the need to strengthen and standardize its IT requirements and development management capabilities, the FBI has issued a bureauwide standard life cycle management directive with the aim of achieving consistent processes in the systems acquisition and development areas mentioned above. A second goal is to integrate these processes with other key IT disciplines, including those discussed in this testimony as well as others, such as information security management. CIO officials told us that they recently began implementing parts of the life cycle management directive across all projects. According to the CIO, the directive is to be fully defined and implemented by the end of 2006.

The FBI acknowledges that the directive needs to be enhanced and extended to adequately address all relevant process areas. For example, FBI officials stated that they are still working to define effective contract management controls, such as procedures for the use of performance-based contracting methods and the establishment of tracking and oversight structures, policies, and processes. For other key practices, procedures have been drafted but require further development.

FBI Has Developed Strategic IT Human Capital Management Policies and Procedures and Is Taking Steps to Implement Them

A strategic approach to human capital management includes viewing people as assets whose value to an organization can be enhanced by investing in them, and thus increasing both their value and the performance capacity of the organization. Based on our experience with leading organizations, we issued a model encompassing strategic human capital management, in which strategic human capital planning was one cornerstone. Strategic human capital planning enables organizations to remain aware of and be prepared for current and future needs as an organization, ensuring that they have the knowledge, skills, and abilities needed to pursue their missions. We have also issued a set of key practices for effective strategic human capital planning. These practices are generic, applying to any organization or component, such as an agency’s IT organization. They include involving top management, employees, and other stakeholders in developing, communicating, and implementing a strategic workforce plan; ● determining the critical skills and competencies needed to achieve current and future programmatic results; ● developing strategies tailored to address gaps between the current workforce and future needs; ● building the capability to support workforce strategies; and ● monitoring and evaluating an agency’s progress toward its human capital goals and the contribution that human capital results have made to achieving programmatic goals.

As we have reported, the FBI’s enterprisewide strategic human capital plan, issued in March 2004, includes policies and procedures for IT human capital. These IT policies and procedures are in alignment with the key practices discussed above. More specifically, they call for the following. ● Top management stakeholders (e.g., the CIO, the head of the Office of Strategic Planning, and the head of Administration) and other stakeholders (e.g., section and unit chiefs) are to be involved with the development, communication, and implementation of the policies and procedures. ● A detailed data bank is to be developed to store critical skills needed in the development and selection of personnel, including IT staff. ● Strategies are to be defined to address workforce gaps, including recruiting programs that provide for tuition assistance and cooperative education. ● An IT center is to be established to support workforce strategies and train existing personnel for future competencies and skills that will be needed. ● The agency’s progress is to be monitored and evaluated by tracking implementation plans to ensure that results are achieved on schedule.

Since that time, the CIO stated that his office is taking steps to enhance its IT human capital capability. For example, it is working with the bureau’s Training Division to identify the skills and abilities of the existing IT workforce and to provide training to enhance these skills and abilities, including having program and project managers work toward becoming certified in their respective disciplines. In addition, the CIO said that as part of reorganizing the CIO’s office, he has created 12 senior executive and 4 senior level technical positions and is in the process of filling them with experienced and qualified staff. According to the CIO, the bureau has hired 8 senior executives and is in the process of hiring the others as well as the 4 senior technical staff.

However, the bureau has yet to create an integrated plan of action that is based on a comprehensive analysis of the human capital roles and responsibilities needed to support the IT functions established under the CIO office’s reorganization. Such an analysis should include an assessment of core competencies and essential knowledge, skills, and abilities, as well as linking current human capital strengths and weaknesses to permit gaps to be identified between current capabilities and those needed to perform the established IT functions. The plan should then describe actions needed to fill the identified gaps (that is, the planned combination of hiring, training, contractor support, and so on), along with time frames, resources, performance measures, and accountability structures. According to the CIO, he is in the process of hiring a contractor with human capital expertise to help identify gaps between existing skills and abilities and those that will be needed to successful modernize the bureau’s IT. The CIO intends to have this effort completed, including the development of an implementation plan to address any gaps, by the end of calendar year 2005. As part of this effort, the CIO stated that he is planning to implement a formal management structure within the Deputy CIO’s office to monitor and evaluate human capital initiatives to ensure that results are achieved on schedule.

Notwithstanding the initiatives under way and planned, the FBI’s IT human capital situation remains a work in progress, and this is a significant challenge. As we have previously reported, when organizations implement a strategic approach to human capital management, how this is done, when it is done, and the basis on which it is done can make all the difference. With successful implementation, the bureau can better position itself to ensure that it has the right people, in the right place, at the right time to effectively modernize IT and transform the organization.

Success of New IT Investments, Like Sentinel, Will Depend on How Well the FBI Implements its New IT Management Approaches

The success of the FBI in using IT to support its transformation efforts and in achieving its mission goals and outcomes will depend on how well it actually implements and institutionalizes the IT management structures, processes, and controls that have been or are currently being put in place. When the bureau’s IT investments have been successfully delivered, and operational assets and tools are available to analysts and field agents to help them do their jobs better, only then can the mission value of technology be fully realized.

The FBI has identified several ongoing new or enhanced system projects that in our view will need to employ these kinds of IT management capabilities in order for each to be successfully defined, designed, developed or acquired, and deployed. For example, the FBI reports that it currently has 18 IT investments that support its “investigative, intelligence, and analytical” line of business, which is a major component of how the bureau accomplishes its mission. According to the bureau, each of these 18 investments is benefiting from the bureau’s newly established IT management approach and capabilities.

Included in these 18 investments is Sentinel, the FBI’s program to deliver an automated case management and information sharing capability; this is the successor to the Virtual Case File, the failed component of the Trilogy program. According to the FBI, Sentinel is to leverage commercially available technologies to consolidate and replace the bureau’s existing case management capabilities with an integrated, paperless file management and workflow system, and to enhance information access and promote information sharing with both the law enforcement and intelligence communities. Thus far, the bureau reports it has developed detailed system requirements, a concept of operations, an acquisition strategy and schedule, and a notional development and deployment strategy involving four increments delivered over 4 years. In August 2005, the FBI issued a request for vendor proposals to more than 40 eligible companies under a National Institutes of Health governmentwide contracting vehicle. According to the CIO, the request also was provided to over 500 eligible subcontractors. Vendor proposals are due later this month; the goal is to issue a contract in November 2005.

As an FBI flagship program, Sentinel can serve as a barometer of how well the FBI defines and implements its new IT management approaches and capabilities, particularly with regard to a system that is to rely extensively on commercially available components (software and hardware). As we discuss above (and have previously reported), there are a number of IT system management practices related to architecture, investment, acquisition/development, and human capital that are critical to delivering promised system capabilities and benefits, on time and within budget. Moreover, these include management practices that are critical to any system, whether custom-developed or built from commercial components, as well as certain practices unique to systems based on commercial components.

Although each of these practices is relevant to Sentinel, there are several that we believe to be especially germane given the FBI’s experience on the Virtual Case File, particularly with regard to requirements management and the bureau’s reported efforts and plans going forward. Specifically, it is critical for the FBI to examine and control its requirements in the context of what capabilities are to be addressed through enterprise-provided services (e.g., records management and security) and what capabilities are to be provided through Sentinel. At the same time, it is essential that the bureau examine its requirements in the context of which capabilities can be provided by commercially available products and which cannot, and for those that cannot, how such requirements will be satisfied, if at all. As we and others have reported, this examination involves continuous but controlled analyses of trade-offs among stated system requirements, commercial product availability, and enterprise architecture constraints; it also involves such practical constraints as human capital and financial resources.

Another area that is critical with respect to Sentinel is ensuring that decisions about the use of commercial components are based on an approach that includes deliberate and thorough research, analysis, and evaluation of components’ dependencies. In this regard, it will be important for the FBI to ensure that it understands the behavioral interaction and compatibility of commercial off-the-shelf (COTS) components in order to select components that can be integrated in a predictable and standard way. We have found based on our research and past work that doing so requires an effective methodology to gain and apply such knowledge; without such a methodology, building a COTS-based system can quickly lapse into trial and error, which is fraught with risks. For example, a trial and error approach can lead to expensive, ad hoc modifications, customized solutions, or unnecessary increases in the number and complexity of interfaces—all of which increases costs, delays delivery, and postpones realization of expected benefits. An effective approach would include (1) performing gap analysis between requirements and component capabilities, as mentioned above, (2) allocating requirements among the various products for a given system design option, (3) defining the interactions that need to occur among the components, (4) documenting decisions, and (5) using iterative prototyping to assess the interactions among the components.

Another very important area particularly relevant to Sentinel is ensuring that the project’s plans explicitly provide the necessary time and resources for (1) integrating the commercial components with the FBI’s existing systems and (2) preparing users for the impact that the business processes embedded in the COTS products will have on how the users will be expected to do their jobs, including potentially new roles and responsibilities. Available research suggests that insufficient attention to this organization change management issue has been a major cause of COTS solution implementations failing to live up their expectations.

Other management practices relevant to commercial component- based systems will be important on Sentinel, including (1) discouraging the modification of COTS products; (2) managing the systems configuration in a way that provides for evaluation, acquisition, and implementation of new, often frequent, releases of COTS products; and (3) ensuring that contractors are experienced in implementing COTS-based system solutions.

In light of the importance of these and other areas, we have just initiated a review of Sentinel at the request of the Chairman and Ranking Member of the House Judiciary Committee; as part of this review, we plan to address many of these keys to project success.

In closing, the FBI has made important progress, particularly in the last 12 months under the new CIO’s leadership, in establishing certain IT management and control capabilities that our research and evaluations show are key to exploiting technology to enable transformation. But although the bureau has come a long way from where it was just 18 months ago, establishing these capabilities is not enough. For the FBI to effectively use technology to transform itself and accomplish its goals, it will need to ensure that its capabilities are appropriately enhanced and extended and, most important, effectively implemented on all IT programs and projects. Nowhere will this be more crucial than on the Sentinel program. Because of the FBI’s stated approach to building Sentinel, it will be particularly important for the bureau to ensure that it follows the kind of acquisition management practices that our work has shown to be critical for commercial component-based systems to be successful. If it does not, the FBI increases the likelihood that Sentinel will encounter the same cost, schedule, and performance shortfalls as its predecessor, the Virtual Case File.

Mr. Chairman, this concludes our statement. We would be happy to answer any questions that you or members of the Subcommittee may have at this time.

Contact and Acknowledgments

If you should have any questions about this testimony, please contact Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. Other major contributors to this testimony included Gary Mountjoy, Assistant Director; Justin Booth; Barbara Collier; Kush Malhotra; Lori Martinez; Teresa Neven; Warren Smith; and Teresa Tucker.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.