Information Technology: FDA Needs to Establish Key Plans and Processes for Guiding Systems Modernization Efforts
Released: 2009-06-02T00:00:00

Letter

Background

FDA’s mission is to protect public health by ensuring the safety, efficacy, and security of human and veterinary drugs, biologic products, medical devices, our nation’s food supply, cosmetics, and products that emit radiation. The agency is also responsible for advancing public health by helping to speed innovations that make medicines and foods more effective, safer, and more affordable and by helping the public get the accurate, science-based information it needs to use medicines and foods to improve health.

FDA carries out its regulatory mission primarily through five main centers and its Office of Regulatory Affairs: Center for Biologics Evaluation and Research. Regulates and evaluates the safety and effectiveness of biological products, such as blood and blood products, vaccines and allergenic products, and protein-based drugs.

Center for Devices and Radiological Health. Ensures that new medical devices are safe and effective before they are marketed and that radiation- emitting products, such as microwave ovens, TV sets, cell phones, and laser products meet radiation safety standards.

Center for Drug Evaluation and Research. Promotes and protects the health of Americans by ensuring that all prescription and over-the-counter drugs are safe and effective.

Center for Food Safety and Applied Nutrition. Ensures the safety of 80 percent of food consumed in the United States (it is responsible for everything except meat, poultry, and some egg products, which are regulated by the U.S. Department of Agriculture).

Center for Veterinary Medicine. Helps to ensure that animal food products are safe; also evaluates the safety and effectiveness of drugs used to treat more than 100 million companion animals.

Office of Regulatory Affairs. Works to ensure that FDA’s health standards are properly implemented and adhered to through inspections, lab analysis, and public outreach.

The agency relies extensively on IT to fulfill its mission and to support related administrative needs. FDA has systems dedicated to supporting the following major mission activities: Reviewing and evaluating new product applications, such as for prescription drugs, medical devices, and food additives. These systems are intended to help FDA determine whether a product is safe before it enters the market. For example, the Document Archiving Retrieving and Regulatory Tracking System is intended to manage the drug and therapeutics review process.

Overseeing manufacturing sites and production supply chains to ensure that products comply with regulatory requirements. For example, the Field Accomplishments and Compliance Tracking System supports inspections, investigations, and compliance activities.

Monitoring the safety of products on the market by collecting and assessing adverse reactions to FDA-regulated products, such as illnesses due to food or negative reactions to drugs. For example, the Vaccine Adverse Event Reporting System accepts reports of adverse events that may be associated with U.S.-licensed vaccines from health care providers, manufacturers, and the public.

In addition, the agency has systems performing administrative processes, such as payroll administration and personnel systems.

All these systems are supported by an IT infrastructure that includes network components, critical servers, and multiple data centers. Appendix III provides additional details on the agency’s mission-critical systems and infrastructure.

The information that FDA receives is growing in volume and complexity. According to FDA, from 2001 to 2006, the number of import shipments that the agency inspected for admission into the United States increased from about 7 million imports reviewed annually to about 18 million. During this period, the number of adverse event reports and generic drug applications more than doubled. Advances in science and the increase in imports are also factors affecting the complexity of information that FDA receives. The ability of the agency’s IT systems and infrastructure to accommodate this growth will be crucial to FDA’s ability to accomplish its mission effectively.

Previous Studies Have Highlighted Limitations of FDA’s IT

FDA’s IT has been the subject of numerous reports and studies, both by the agency itself and by others (see app. IV for a list of major reports and studies related to limitations of the agency’s IT). These reports have noted limitations in a number of key areas, including data availability and quality, IT infrastructure, ability to use technology to improve regulatory effectiveness, and IT management.

Data availability and quality. Issues with the quality and availability of FDA’s data have been raised in several studies. In 2007, the FDA Science Board issued FDA Science and Mission at Risk, a broad assessment of challenges facing the agency. This study found that information was not easily and immediately accessible throughout the agency (including critical clinical trial data that were available only in paper form), hampering FDA’s ability to regulate products. Data and information exchange was impeded because information resided in different systems that were not integrated. The Science Board also reported that FDA lacked sufficient standards for data exchanges, both within the agency and between the agency and external parties, reducing its capability to manage the complex data and information challenges associated with rapid innovation, such as new data types, data models, and analytic methods.

In 2007, FDA commissioned Deloitte Consulting, LLP, to examine ways the agency could better meet increased demand for information and make decisions more quickly and easily. Deloitte noted that FDA’s former decentralized approach to IT, in which the centers developed their own systems, led to duplicative work efforts, tools, and information. Noting that the agency had begun moving toward a more enterprisewide approach, Deloitte recommended further steps, including establishing enterprisewide information standards and incorporating data exchange standards into its day-to-day processes and applications in order to achieve interoperability with external partners.

Our previous work also has identified issues related to the availability and quality of the agency’s data. For example, our 1998 study of FDA’s foreign drug inspection program cited evaluations that essential data for foreign inspections were not readily available, and that FDA did not have a comprehensive, agencywide, automated system for managing foreign inspection of manufacturers. Further, in a series of products (most recently in September 2008) on FDA inspections of foreign establishments, we reported that the agency’s databases on these establishments contained incorrect information and that different databases had differing information.

IT infrastructure. Issues raised regarding FDA’s infrastructure include aging and redundancy. According to the FDA Science Board’s 2007 report, the agency’s IT infrastructure was outdated and unstable, and it lacked sufficient controls to ensure continuity of operations or to provide effective disaster recovery services. For example, as many as 80 percent of the network servers were more than 5 years old and had exceeded their recommended service life. In addition, the report stated that outages were occurring in other systems as well; for example, e-mail problems occurred during an E. coli food contamination investigation. Further, critical network components did not reside in data centers that provided the necessary security, redundancy, and continuity of operations assurances.

In addition, after assessing the agency’s legacy applications, FDA’s contractor, High Performance Technologies, Inc., issued a report in 2008 that identified many systems that were redundant and could be combined with each other, as well as systems that could be retired.

Ability to use technology to improve regulatory effectiveness. According to the FDA Science Board report, advances in science and technology have been outpacing the capabilities of FDA’s IT infrastructure and systems. For example, although genetics and genome-wide association analyses are an increasingly important technique in drug reviews, the agency had minimal IT infrastructure to support genomics-focused efforts, which generate large data sets. To implement the real-time acquisition and sharing of genomics data would require the development of appropriate data storage, mining, analysis, and risk evaluation tools for FDA scientists.

IT management. Issues with FDA’s IT management have been found in several areas, including human capital, enterprise architecture, governance, and information security. In assessing IT human capital, the Science Board stated that the agency did not have sufficient IT staff with skills in such areas as capital planning/investment control and enterprise architecture, that processes for recruitment and retention of IT staff were inadequate, and that the agency did not invest sufficiently in professional development.

Deloitte’s study also commented on IT management, stating that FDA needed to develop both a common enterprise information management architecture and an IT architecture to facilitate both short-term operational gains such as improved information access, as well as long- term gains in strategic flexibility.

In another study, the Breckenridge Institute examined the process being used to develop requirements for the agency’s adverse event reporting system and found that FDA’s management of requirements development did not follow proper IT methodology, such as documenting the reasons for changes to system requirements.

Finally, in October 2008, an HHS inspector general report concluded that FDA had made progress implementing an infrastructure to support the security management program. However, the Inspector General also noted that the agency had not fully implemented a security program infrastructure and was not performing all the activities required to integrate security into applications.

FDA Has Been Moving toward an Enterprisewide Approach to IT

Driven in part by the various studies that the agency has performed or sponsored (as discussed previously), as well as legislative requirements, FDA has been transitioning to an enterprisewide approach to IT management. For example, in February 2006 the agency created the Bioinformatics Board to replace center-specific investment review boards, in order to better coordinate its IT investment decisions from an agencywide perspective. According to the agency’s Chief Information Officer (CIO), this broader perspective led to an increased emphasis on the need for FDA to treat its information as a strategic corporate asset and manage it accordingly. Among the steps taken to help achieve this goal were centralizing the IT organization and consolidating IT infrastructure.

In May 2008, the agency transferred responsibility for managing IT from individual components (centers and the Office of Regulatory Affairs) to a new centralized Office of Information Management (OIM), headed by the CIO. The CIO reports to the agency’s Chief Operating Officer.

As head of OIM, the CIO is responsible for managing IT, creating a foundation to enhance the interoperability of its systems, and managing more than 400 staff assigned to this office.

OIM has five divisions to carry out its responsibilities: Division of Business Partnership and Support. Acts as liaison and provides management and technical consultation resources regarding IT to FDA offices, centers, and other stakeholders, including parties outside the agency.

Systems Division. Manages design, development, implementation, and maintenance of agency software applications and systems, as well as their integration with other entities.

Infrastructure Division. Manages design, development, implementation, and maintenance of the agency’s IT infrastructure.

Division of CIO Support. Oversees internal IT management controls, such as its enterprise architecture, investment management, and human capital management.

Division of Technology. Reviews and evaluates the appropriateness of new and emerging information technologies for potential benefits.

As part of its centralization efforts, FDA is transferring IT staff and assets from its components to the new centralized organization, and it is consolidating its IT infrastructure. Under one initiative, Information and Computer Technology for the 21st Century (ICT21), the agency is, for example, consolidating its data into two new data centers, one to host its production and preproduction systems and information, and the other to host system testing, development, and scientific computing needs.

FDA’s IT Budget

FDA’s fiscal year 2009 budget totals about $2.67 billion and is derived both from the agency’s annual appropriations and user fees. The appropriated budget authority is about $2.05 billion or 77 percent of funding, and user fees account for about $613 million or 23 percent of funding. FDA collects user fees primarily from companies that produce certain human drug and biologic products, as authorized by the Prescription Drug User Fee Act of 1992 (PDUFA).

FDA’s fiscal year 2009 IT budget is approximately $364 million, which is about 14 percent of the agency’s total budget. The IT budget includes funds of $308.4 million for projects and systems and $55.2 million for federal employee salaries and expenses. The funding for projects and systems is derived from annual appropriations of $246.1 million and user fees of $62.3 million. The funding for federal employee salaries and expenses is derived from annual appropriations of $44.4 million and user fees of $10.8 million.

According to data provided by FDA officials, the portion of FDA’s fiscal year 2009 IT budget that funds IT projects and systems has increased from previous years. As shown in table 1, from fiscal year 2005 to fiscal year 2009, funding for projects and systems increased from $202.3 million in annual funding to $308.4 million.

According to the agency’s CIO, during fiscal years 2008 and 2009, IT expenditures have focused on addressing limitations, such as updating the infrastructure, and on problems that could be immediately addressed, such as eliminating duplicative databases related to adverse event reporting. He added that in the future, FDA plans to focus on more long- term modernization projects for supporting the agency’s regulatory responsibilities.

Effective IT Management Is Key to Successful Modernization

Key to an agency’s success in modernizing its IT systems, as our research and experience at federal agencies has shown, is institutionalizing a set of interrelated IT management capabilities, among which are strategic planning to describe an organization’s goals, strategies it will use to achieve desired results, and performance measures; developing and using an agencywide enterprise architecture, or modernization blueprint, to guide and constrain IT investments; establishing and following a portfolio-based approach to investment implementing information security management that ensures the integrity and availability of information; and building and sustaining an IT workforce with the necessary knowledge, skills, and abilities to execute this range of management functions.

Figure 1 shows these capabilities, which are critical to enable organizations to manage IT effectively.

The Congress and OMB have recognized the importance of these and other IT management controls. The Clinger-Cohen Act, for example, provides a framework for effective IT management that includes systems integration planning, human capital management, and investment management. In addition, the Paperwork Reduction Act requires that agencies have strategic plans for their information resource management, and the E- Government Act of 2002 contains provisions for improving the skills of the federal workforce in using IT to deliver government information and services. Further, OMB has issued guidance on integrated IT modernization planning and effective IT human capital and investment management.

Establishing IT management capabilities involves carrying out specific practices. For example, human capital management requires assessing present and future agency skills needs and making a plan to fill gaps. We have developed methods of evaluating agencies’ progress on these management capabilities, such as our IT Investment Management (ITIM) framework, Enterprise Architecture Management Maturity Framework, and framework for strategic human capital management. These frameworks list specific practices that an agency should use.

We have observed that without these types of capabilities, organizations increase the risk that system modernization projects will (1) experience cost, schedule, and performance shortfalls and (2) lead to systems that are redundant and overlap. They also risk not achieving such aims as increased interoperability and effective information sharing. As a result, technology may not effectively and efficiently support agency mission performance and help realize strategic mission outcomes and goals.

FDA Is Pursuing Systems Modernization, but It Has Not Developed an IT Strategic Plan to Guide Its Initiatives

FDA is pursuing numerous initiatives to modernize its IT systems and infrastructure, including at least 16 enterprisewide initiatives. However, it does not yet have a comprehensive IT strategic plan, with well-defined goals, strategies, milestones, and measures, to guide these efforts. According to the Chief Operating Officer, the agency must resolve many near-term planning activities and strategic investment decisions before it can complete long-term plans. Without a strategic plan to sequence and synchronize these initiatives based on a comprehensive picture of its strategic IT goals, the agency increases the risk that its modernization efforts will not be effective.

Of FDA’s numerous modernization initiatives, some began as a result of federal law and guidance (such as initiatives associated with PDUFA), and others in response to urgent mission requirements, including those pointed out in the various analyses of FDA’s IT systems and infrastructure previously described. Table 2 lists 16 major modernization projects with an enterprisewide focus that are under way or planned. As the table shows, many of these projects are still in the early stages of the life cycle (that is, planning and requirements development).

In addition to these system and infrastructure development projects, FDA is taking actions to develop and enhance its IT management capabilities. That is, the agency is taking actions such as beginning to develop its enterprise architecture, gathering information on needed IT skills, and seeking contract support to improve application security and to analyze skills gaps. (FDA’s IT management capabilities are further discussed later in this report.)

However, even as it undertakes these various initiatives and activities, FDA does not yet have the necessary planning in place to guide its efforts. Although agency officials identified two high-level planning documents that address different aspects of the agency’s IT environment, FDA lacks a comprehensive IT strategic plan, which is a foundation for effective modernization and is required by federal guidance. As we have previously reported, such a plan is to serve as the agency’s IT vision or roadmap and help align its information resources with its business strategies and investment decisions. The plan might include the mission o the agency, key business processes, IT challenges, and guiding principles. A strategic plan is important to enable an agency to consider the resources, including human, infrastructure, and funding, that are needed to manage, support, and pay for projects. For example, a strategic plan that identifies what an agency intends to accomplish during a given period helps ensure that the necessary infrastructure is put in place for new or improved capabilities. In addition, a strategic plan that identifies interdependencies within and across individual IT systems modernization projects helps ensure that the interdependencies are understood and managed, so that projects—and thus system solutions—are effectively integrated.

In summary, an IT strategic plan would provide a comprehensive picture of what the organization seeks to accomplish, identify the strategi use to achieve desired results, provide results-oriented goals andperformance measures that permit it to determine whether it is succeeding, and describe interdependencies that these can be understood and managed. within and across projects so However, FDA has not yet developed such a plan, although it does have two high-level planning documents—the agency’s Strategic Action Plan and the PDUFA IV IT Plan (PDUFA plan). Even in combination, however, the two plans do not have the scope and depth of an IT strategic plan: the first does not treat IT initiatives in depth, and the second is not an agencywide plan. Although these two plans include some elements of an IT strategic plan, they do not include all.

FDA’s Strategic Action Plan, approved in fall 2007, does not include all IT projects or their associated performance measures, milestones, and interdependencies, although it does include strategic goals and objectives. Specifically, the plan describes four major strategic goals for the agency along with subsidiary implementation objectives, some of which identify IT initiatives (table 3 shows these major goals, objectives, and initiatives). As an overall agency plan, the Strategic Action Plan includes initiatives related to the agency’s major strategic goals, but it does not include performance measures or milestones for those initiatives. In addition, it does not include certain IT initiatives; for example, the PREDICT initiative, described in table 2, is a major initiative not mentioned in the Strategic Action Plan. Further, it does not identify interdependencies within and across individual IT modernization projects to ensure that they are understood and managed appropriately. For example, FDA has several ongoing projects that are developing data standards, including Regulated Product Submission, Harmonized Inventory, and Automated Laboratory Management. A well-designed IT strategic plan would document any interdependencies in such related projects.

The PDUFA plan, published in July 2008, does focus on IT, and it provides details on goals, initiatives, and milestones, as well as performance measures. The plan includes several sections addressing current FDA IT goals and strategies. For example, it discusses detailed measures to create data standards to be used throughout the agency for regulatory submissions, and it describes the responsibilities of a Data Standards Council, which coordinates standards with data provider organizations.

However, this document is not a comprehensive plan for the agency’s IT because it addresses only those IT initiatives that are related to user fee programs (which cover drugs and biologics). Further, it does not include an assessment of interdependencies among projects.

Thus, although the Strategic Action Plan and PDUFA plan contain elements that would be included in an IT strategic plan, neither provides the comprehensive coverage of FDA’s goals and activities that a well- crafted IT strategic plan would provide.

FDA officials agreed that the current plans do not include all the elements required for an IT strategic plan. The CIO said that the agency is aware of the importance of having such a plan and intends to develop one. However, according to the Chief Operating Officer, the agency must resolve many near-term planning activities and strategic investment decisions before it can complete long-term systems development plans. He stated that FDA is still working on its vision for modernizing IT infrastructure and services and how to incorporate that vision into an IT strategic plan. Accordingly, FDA has not defined either milestones or a completion date for an IT strategic plan.

FDA’s Projects and Plans Are Intended to Address Most Previously Identified Limitations

As reflected by its projects and high-level plans, FDA intends to address most of the limitations in its IT systems and infrastructure that had been previously identified by the agency’s Science Board, its contractors, and us. Table 4 provides an overview of the limitations along with related projects and activities that the agency is planning or currently undertaking. The table also shows which identified limitations are discussed in the two high-level planning documents mentioned earlier (the agency’s Strategic Action Plan and the PDUFA plan). Addressing these limitations in plans and projects does not guarantee that the limitations will be successfully overcome, but it does indicate that they are receiving management attention.

As the table shows, FDA intends to address most of the previously identified limitations in its IT systems, infrastructure, and management. That is, of the 17 limitations in the table, 14 are associated with projects, activities, or plans. For example, to address IT infrastructure limitations, the ICT21 project is, among other things, replacing outdated data centers.

To address limitations in the agency’s ability to handle data and make the data available, the Common Electronic Document Room project is to digitize data formerly available only in paper form, as well as establish a single repository for all regulatory documents (replacing separate document repositories at FDA’s centers). Further, to increase the agency’s ability to use technology to improve regulatory effectiveness, the PREDICT project is to provide the capability for predictive, risk-based surveillance of imported food. That is, it is to assist FDA inspectors in deciding which shipments of imported food to inspect by using a rule- based expert system to assess information from multiple sources and determine which shipments carry the highest risk.

However, FDA is not addressing 3 of 17 limitations. For example, the agency does not have projects, activities, or plans to address its inability to perform inspections, remote monitoring, or sensing for contaminants in regulated products at manufacturing sites or in transportation vehicles. According to FDA officials, an initial investigation of the possible use of RFID (radio frequency identification) tags to allow remote monitoring to prevent drug counterfeiting was not successful. Agency officials indicated that remote sensing was currently not a high priority. In addition, the agency does not plan to address two previously identified limitations in IT management (this topic is discussed in the next section).

Further, although these projects, activities, and high-level plans are intended to address most of the limitations, successfully overcoming the limitations depends in part on the agency’s developing and implementing appropriately detailed plans. FDA is taking steps to respond to the need to modernize its IT systems and infrastructure, but the number and range of its activities are further evidence of the importance of a comprehensive IT strategic plan to guide and coordinate them. Such a plan would allow FDA to integrate the planning for all of its modernization projects, including setting priorities, allocating resources, and accounting for dependencies.

At the same time, it would provide a roadmap for improving FDA’s IT management capabilities, which would decrease the risk that the agency’s modernization initiatives will not achieve their goals or deliver planned capabilities on time and within budget.

FDA Has Made Mixed Progress in Key IT Management Practices

An agency’s chance of success in modernizing its IT systems is improved if it institutes critical IT management capabilities, including strategic planning (discussed in the previous section), investment management, information security, enterprise architecture, and human capital. Although FDA is making progress in these areas, it has considerable work to do. It is building necessary capabilities in investment management and information security, but it continues to have information security deficiencies, and important elements of its enterprise architecture are not in place. Finally, it is not effectively managing its IT human capital. Without these management capabilities in place, FDA increases the risk that its modernization efforts will not deliver required system capabilities and expected mission value on time and within budget.

FDA Has Implemented an Investment Management Structure and Processes

IT investment management links investment decisions to an organization’s strategic objectives and business plans. The Clinger-Cohen Act requires an agency to, among other things, select and control IT projects as investments in a manner that minimizes risks while maximizing the return. Projects are seen as investments and are selected and managed on the basis of cost, benefit, risk, and organizational priorities by an investment board made up of senior agency managers.

To select an investment, the organization (1) identifies and analyzes each project’s risks and returns before committing significant funds to any project and (2) selects those IT projects that will best support its mission needs. The selection process should take account of the specific business needs addressed by each project and should use the agency’s enterprise architecture.

Once a project is under way, the organization manages project schedules, costs, benefits, and risks to ensure that the project meets mission needs within cost and schedule expectations.

Our ITIM framework for assessing investment management maturity includes foundational processes for selecting projects and for managing them at the project level, such as establishing an investment review board, developing an investment selection process, and overseeing the progress of individual projects. FDA has made progress in implementing selected foundational processes, as described below.

Selecting IT investments. FDA has put in place several important practices cited in our ITIM framework, including establishing an investment review board and developing an investment selection process: In February 2006, the agency created an IT investment review board—the Bioinformatics Board. The board has broad responsibilities, including approving all IT budget execution decisions; overseeing business decisions on priority, planning, and execution of agency cross-cutting automation projects; directing the related business process analyses; and overseeing planning activities to ensure coordination. Members of the board are senior officials: It is co-chaired by two Deputy Commissioners—the Chief Operating Officer and the Chief Medical Officer.

FDA has established Business Review Boards, representing core agencywide business areas, as standing subcommittees of the Bioinformatics Board. The Business Review Boards, among other things, act as the agencywide “business sponsor” of new systems development, provide oversight and direction of the work being performed on IT systems and projects within their defined areas, and prepare and present proposals to the Bioinformatics Board for review and approval.

FDA has documented criteria for evaluating prospective projects, such as public health impact, cost savings, and whether the project is agencywide. Bioinformatics Board members told us that the Business Review Boards use these criteria and others specified by the Bioinformatics Board, such as budget considerations.

Oversight and project management. As part of an effective IT investment process, an agency must be able to control its investments—manage its projects—so that they finish predictably within established schedule and budget. To accomplish this, agencies should have policies and procedures for oversight and should provide adequate resources, such as managers and staff responsible for monitoring projects. In the absence of predictable, repeatable, and reliable investment control processes, investments will be subject to a higher risk of failure.

FDA’s Business Review Boards and Bioinformatics Board are responsible for overseeing projects. The Business Review Boards are responsible for day-to-day oversight of projects, for providing status reports, and for elevating problems to the Bioinformatics Board as needed. In the oversight area, the Bioinformatics Board reviews status reports and makes decisions on problems elevated by the Business Review Boards.

FDA also has put in place a policy framework to manage its projects effectively. For example: FDA has created a project management office to assess and improve project management, standardize project management practices, improve communication so that senior executives and stakeholders know program and project status, and centralize and coordinate the management of IT programs and projects. The agency also has a staff of trained project managers and has assigned project managers to most of its modernization projects.

FDA has a documented project monitoring and control process intended to track progress so that appropriate corrective actions can be taken when the project’s performance deviates significantly from the baseline project management plan. It defines tasks to be performed by the project manager—such as tracking progress and managing risk—and identifies supporting tools. This process, if appropriately implemented, provides FDA with a foundation for an effective project management capability.

FDA Is Making Progress on Addressing Information Security Issues, but Risks Remain

Information security is critically important for federal agencies, where the public’s trust is essential, and poor information security can have devastating consequences. Since 1997, we have identified information security as a governmentwide high-risk issue in each of our biennial reports to the Congress. Concerned by reports of significant weaknesses in federal computer systems, the Congress passed the Federal Information Security Management Act of 2002 (FISMA), which requires agencies to develop and implement an information security program, evaluation processes, and annual reporting.

FDA’s most recent FISMA results indicate that the agency has made progress on information security but that problems remain. The 2008 FISMA audit by the HHS Inspector General found that FDA continued to make progress in implementing an infrastructure to support security management. However, the report cited 78 deficiencies in seven categories, including infrastructure, integrating security into applications, network management, and personnel security.

In response to the Inspector General’s report, FDA’s CIO reported that the agency has conducted a comprehensive security review and made major changes to its information security program. According to the CIO, it has a new IT security program that is consolidated at the agency level and will provide consistent, centralized support across the agency. In addition, the agency has awarded a new contract for security services, and it is taking steps to address the Inspector General’s specific concerns. However, FDA is not addressing all of the Inspector General’s findings, because it believes it already meets the requirements for several of the controls found to be deficient.

Security issues could be a challenge for FDA’s modernization plans; the Common Electronic Document Room, for example, will need to securely keep confidential records, trade secrets, and classified materials. Effective information security is essential to prevent data tampering, disruptions in critical operations, fraud, and unauthorized access or disclosure of sensitive information.

FDA Has Not Developed an Architecture to Effectively Guide and Constrain Its Projects

An agency’s enterprise architecture describes both its business operations and the technology it uses to carry out those operations. It is a blueprint for organizational change defined in models that describe (in both business and technology terms) how an entity operates today and how it intends to operate in the future; it also includes a plan for transitioning to this future state. For example, it discusses interrelated business processes and business rules, information needs and flows, and work locations and users. Technical topics include hardware, software, data, communications, security attributes, and performance standards. It provides these perspectives both for the enterprise’s current or “as is” environment and for its target or “to be” environment, as well as a transition plan for moving from the “as is” to the “to be” environment.

We have developed our Enterprise Architecture Management Maturity Framework to provide federal agencies with a common benchmarking tool for planning and measuring their efforts to improve enterprise architecture management. Like the ITIM, it provides a five-stage hierarchy of core management elements that agencies should perform to manage enterprise architecture development, maintenance, and implementation. The initial core elements for building the enterprise architecture foundation focus on building a management foundation; for example, one of these core elements is the organization’s recognizing that an enterprise architecture is a corporate asset by vesting accountability for it in an executive body that represents the entire enterprise. At this stage, an organization also assigns management roles and responsibilities and establishes plans for developing enterprise architecture products and for measuring program progress and product quality; it also commits the resources necessary for developing an architecture—people, processes, and tools. In addition, the organization develops a documented enterprise architecture program management plan, describing in detail the steps to be taken and tasks to be performed in managing the program, including a detailed work breakdown and estimates for funding and staffing.

According to FDA, it has taken several initial steps toward building an enterprise architecture management foundation, such as establishing a committee or group representing the enterprise that is responsible for enterprise architecture, establishing a program office responsible for enterprise architecture, and designating a Chief Architect.

However, according to the chief architect, FDA has not developed the program management plan that our framework characterizes as essential to ensuring that the enterprise architecture is effectively and efficiently developed.

Beyond establishing an enterprise architecture management foundation, FDA has not yet developed architecture artifacts at the depth and breadth associated with a well-defined enterprise architecture. According to FDA’s Chief Architect and other officials, they are currently modeling the agency’s existing business processes and the data exchanges among existing processes as part of an HHS-wide modeling effort. Further, the agency has a listing of its current systems and the business processes that they support. However, no other “as is” artifacts were available. For the “to be,” the Chief Architect stated that they have developed an initial version of the “to be” architecture and have completed a transition plan for moving from the “as is” to the “to be.” However, they could not provide either the “to be” architecture artifacts that we requested or the enterprise transition plan. According to relevant guidance and best practices, the transition plan should provide a road map for moving from the “as is” to the “to be” environment.

To facilitate its enterprise architecture efforts, FDA is using an approach called segment architecture. A segment architecture allows for the details needed to implement an enterprise architecture to be built in pie by piece. First a corporate layer of architecture is built that sufficient ly reflects, among other things, those policies, rules, and standards that apply across the whole enterprise; then the more specific content needed to implement the enterprise architecture on a segment-by-segment basis is added. The segment architecture extends the enterprisewide layer, providing additional detail and depth needed to implement project and IT solutions. Accordingly, segment architectures do not stand alone.

FDA has begun building segments before it has a well-defined enterprise architecture and before it has prioritized its segments. According to the Federal Enterprise Architecture Practice Guide, prioritizing segments should precede building them. Once prioritization is completed, the agency should define (1) the scope and strategic intent of each segment, (2) business and information requirements, and (3) the conceptual solution architecture. FDA has identified 26 segments in all (for example, product safety, risk analysis, scientific analysis, and external partnerships), but it has not yet prioritized them. According to FDA, its enterprise architecture staff are currently working to define a standard set of criteria that the Bioinformatics Board is to use to set priorities for the remaining segments.

Although FDA has not prioritized its segments, it has, according to officials, completed the architecture for one segment—product safety— including an “as is,” “to be,” and transition plan. According to the Chief Architect, the completed product safety segment architecture describes the scope and strategic intent of the segment, defines business and information requirements, and includes a description of the solutions architecture. According to FDA officials, this architecture has been sent to HHS for approval. However, they could not provide documentation of the completed segment.

Attempting to define and build major IT systems without first completing either an enterprisewide architecture and, where appropriate, the relevant segment architecture is risky. According to the Federal Enterprise Architecture Practice Guide, prioritizing segments should precede building them, and developing the segment architecture should take place before an agency executes projects. FDA has identified three modernization projects as being within the product safety segment: MedWatch Plus, FAERS, and Harmonized Inventory. Thus, the other 13 major modernization projects are proceeding without the guidance and constraint of an enterprise or segment architecture. For example, some projects outside the product safety segment—such as the Common Electronic Document Room and PREDICT—that will need to use data from multiple sources may not be able to exchange data seamlessly with future systems. Similarly, a recent FDA study to identify existing applications with potential for agencywide use said it could not make definitive recommendations without a “to be” architecture. Also, going forward, further development of a “to be” enterprise architecture could be hindered by the lack of an IT strategic plan, since an enterprise architecture must align with an organization’s strategic planning. As long as the architectural context for its enterprise architecture and segment architectures lags behind its modernization projects, FDA increases the risk that its modernization solutions will not be defined, developed, and deployed in a way that promotes interoperability, maximizes shared reuse, and minimizes overlap and duplication.

FDA Has Begun Steps for Strategically Managing IT Human Capital, but Critical Activities Remain

The success or failure of federal programs, like those of other organizations, depends on having the right number of people with the right mix of knowledge and skills. In our past work, we have found that strategic human capital management is essential to the success of any organization.

Strategic human capital management focuses on two principles that are critical in a modern, results-oriented management environment: People are assets whose value can be enhanced through investment.

An organization’s human capital approaches must be aligned to support the mission, vision for the future, core values, goals and objectives, and strategies by which the organization has defined its direction.

In our model of strategic human capital management and our report on principles for strategic workforce planning, we lay out principles for managing human capital. Strategic workforce planning involves determining the critical skills and competencies needed to achieve current and future program results (these should be linked to long-term goals), analyzing the gaps between current skills and future needs, and developing strategies for filling gaps. Figure 2 shows the process of planning for workforce needs and the need for ongoing gap analyses based on program goals.

Inventory of existing workforce capabilities FDA is not yet strategically managing its IT workforce, although it is taking some steps to address its IT human capital limitations. (As described in table 4, previously identified limitations include insufficient IT workforce and lack of investment in staff development.) For example, officials told us they have substantially increased the training budget this year for IT staff, although they could not provide actual dollar figures. Further, because the centers’ IT staffs have been centralized into the new Office of Information Management, IT human capital planning can be done centrally by the CIO.

However, FDA has not yet inventoried the IT skills of its current IT workforce, determined present or future skills needs, or analyzed gaps. (A senior official said these activities were not undertaken because the centralization was too recent.) The CIO said that the agency is drafting a work order for an IT skills gap analysis, and agreed that the IT function is still understaffed. Even in the absence of an inventory, FDA officials were able to cite some skills areas as currently in short supply, such as project managers and network engineers. Finally, as mentioned earlier, the agency does not yet have an IT strategic plan; having a plan that describes future activities would improve the agency’s ability to accurately project its future staff and skill needs. Until it begins managing IT human capital strategically, FDA cannot be assured that it will have the workforce it needs to carry out its modernization projects.

Conclusions

FDA is undertaking a variety of activities to address IT limitations that have hampered its mission, many of which the agency describes as urgent and some (such as PDUFA investments) as a result of federal laws and guidance. To help ensure that these important efforts are successful, the agency would be assisted by the kind of strategic view of its modernization initiatives provided by an appropriately comprehensive IT strategic plan. However, FDA does not have such a plan guiding its modernization efforts. FDA’s current agencywide plans lack many of the elements associated with a comprehensive IT strategic plan, such as strategies for managing the interdependencies among projects.

In its modernization initiatives, FDA is taking steps to improve IT management. That is, it has begun implementing an enterprisewide approach to IT management, and it has put into place a foundation for investment management. However, FDA has weaknesses in certain IT management capabilities, including enterprise architecture, human capital, and security. Unless it further develops its enterprise architecture, the agency increases the risk that projects will not fully meet its strategic mission requirements, will be duplicative, and will not be integrated. In addition, the lack of a developed IT human capital management process increases the risk that projects will fail and that activities will continue to be hampered by a shortage of appropriately skilled staff. Finally, to address information security risks, the agency will need to ensure that it responds appropriately to the recommendations made by the HHS Inspector General.

Recommendations for Executive Action

To help ensure the success of FDA’s modernization efforts, we recommend that the Commissioner of FDA require the CIO to take expeditious actions to set milestones and a completion date for developing a comprehensive IT strategic plan, including results-oriented goals, strategies, milestones, performance measures, and an analysis of interdependencies among projects and activities, and use this plan to guide and coordinate its modernization projects and activities; develop a documented enterprise architecture program management plan that includes a detailed work breakdown of the tasks, activities, and time frames associated with developing the architecture, as well as the funding and staff resources needed; complete the criteria for setting priorities for the segment architecture and accelerate development of the segment and enterprise architecture, including “as is,” “to be,” and transition plans, and in the meantime develop plans to manage the increased risk to modernization projects of proceeding without an architecture to guide and constrain their development; and develop a skills inventory, needs assessment, and gap analysis, and develop initiatives to address skills gaps as part of a strategic approach to IT human capital planning.

Agency Comments and Our Evaluation

The Acting Commissioner of Food and Drugs provided written comments on a draft of this report (the comments are reproduced in app. II). In the comments, FDA generally agreed with our recommendations and identified actions initiated or planned to address them. On developing a comprehensive IT strategic plan, for example, the agency stated that its efforts included performing a high-level analysis of FDA’s most immediate needs and priorities, and taking a longer-range view of the functionalities and capabilities it will need in the coming years. The agency added that it intends to complete a draft plan by the end of fiscal year 2009. In addition, with regard to its enterprise architecture, the agency stated that it was currently documenting a program management plan. It also indicated that it will use its ITIM processes to identify risks to its projects and programs and help ensure that they adhere to the agency’s “to be” architecture. Further, on developing a strategic approach to IT human capital planning, FDA stated that it plans to assess workforce needs, develop hiring plans based on the needs, and survey staff to identify their concerns with the organizational environment.

The agency’s completion of the activities described, as well as other necessary actions to implement our recommendations, should increase the likelihood that FDA’s modernization projects and activities will accomplish their intended goals.

In addition, the agency provided technical comments to clarify our discussion of its IT budget, which we have incorporated as appropriate.

We are sending copies of this report to the Commissioner of the Food and Drug Administration, appropriate congressional committees, and other interested parties. In addition, the report is available at no charge on the GAO Web site at http://www.gao.gov.

Should you or your staffs have questions on matters discussed in this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) evaluate the Food and Drug Administration’s (FDA) overall plans for modernizing its systems, including the extent to which the plans address identified limitations or inadequacies in the agency’s information technology (IT) capabilities, and (2) assess to what extent the agency has put in place key IT management policies and processes to guide the implementation of its modernization projects.

To evaluate FDA’s overall plans for modernizing its IT systems, we examined criteria for strategic plans in guidance from the Office of Management and Budget (OMB), legislation (the Clinger-Cohen Act), and our previous reports. We analyzed studies of FDA’s IT conducted in the last several years to identify core limitations. We requested and received documentation from FDA on its agencywide modernization projects, including descriptions of their purpose and project summary status reports showing their expected completion dates and other milestones. We then analyzed these documents to determine which IT limitations these projects were intended to address. We analyzed the agency’s two main high-level planning documents that address IT, the agency’s Strategic Action Plan and the Prescription Drug User Fee Act (PDUFA) IV IT Plan, to determine whether they included elements of an IT strategic plan. We also assessed whether these plans were addressing IT limitations by analyzing whether they included strategies to address each limitation, and whether the plan included one or more projects intended to address each limitation. However, we did not assess the degree to which each limitation was addressed by FDA’s activities. Finally, we attended information sessions given by a contractor and an FDA inspector on one of the agency’s major initiatives—the Predictive Risk-based Evaluation for Dynamic Import Compliance Targeting (PREDICT) system—to gain understanding of the methodology and plans for implementing the system.

To assess the IT management guiding the implementation and management of FDA’s modernization projects, we focused on key areas— investment management (including project management), information security, enterprise architecture development, and human capital management. We looked at whether policies or processes were in place for IT investment management, enterprise architecture, and human capital. We based our analysis on three frameworks: our Information Technology Investment Management (ITIM) framework, our Enterprise Architecture Management Maturity Framework, and our framework for strategic human capital management.

The ITIM framework is a maturity model composed of five progressive stages of maturity that an agency can achieve in its IT investment management capabilities. Each stage specifies critical processes as well as specific key practices within each process. Stage 2 critical processes lay the foundation for sound IT investment management. We examined FDA’s implementation of three critical stage 2 processes (Instituting the Investment Board, Selecting an Investment, and Providing Investment Oversight). Within each process, we looked for the existence of policies, procedures, and organizational entities that would enable effective investment management and oversight. We did not do a complete ITIM assessment or audit specific IT projects to analyze how well the policies and procedures were implemented.

Our Enterprise Architecture Maturity Framework (EAMMF) describes stages of maturity in managing enterprise architecture. Each stage includes core elements—descriptions of a practice or condition that is needed for effective enterprise architecture management. We evaluated FDA’s implementation of four core elements from stage 2 (Building the Enterprise Architecture Management Foundation). We did not do a complete EAMMF assessment, and we did not audit specific IT projects to analyze how well the policies and procedures were implemented. To supplement the EAMMF criteria, we used criteria from the Federal Enterprise Architecture Practice Guide issued by OMB and compared FDA’s progress on its architecture with these criteria.

Our framework for strategic human capital management lays out principles for managing human capital. We evaluated FDA’s policies and procedures against this framework.

To assess the agency’s management of information security, we analyzed the HHS Inspector General’s fiscal year 2009 FISMA report, which assessed FDA’s compliance with FISMA information security provisions. We did not do an independent review of the agency’s information security.

In addition, we interviewed FDA officials, including the Chief Operating Officer, the Chief Information Officer (CIO), and officials from the new Office of Information Management and its five subdivisions. We also interviewed officials from the Office of Budget Presentation and Formulation, the Center for Biologics Evaluation and Research, and the Center for Drug Evaluation and Research. Further, we interviewed officials outside FDA, including a member of the Science Board study and a former FDA regulatory official to obtain additional perspectives on IT issues and proposed solutions at FDA. Finally, we obtained the perspectives of the Acting Commissioner regarding the IT issues identified in our review.

We conducted this performance audit at FDA headquarters in Rockville, Maryland, from May 2008 through June 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective.

Appendix II: Comments from the Food and Drug Administration

Appendix III: FDA’s Mission-Critical Systems and Infrastructure

According to FDA’s CIO, the agency defines mission-critical systems as those that support its centers and offices in accomplishing their mission. According to FDA, there are currently about 47 of these mission-critical systems. FDA’s CIO stated that the number of mission-critical systems is subject to change as legacy systems are retired and modernization projects create new systems to take their place.

Mission-Critical Systems

Mission-critical systems can be grouped by the key mission areas that they support: reviewing and evaluating applications for new products, overseeing manufacturing and production supply chains, and monitoring the safety of products on the market.

In tables 5 to 7, we provide examples of systems that are currently in use and support a variety of internal users from each of FDA’s main centers and the Office of Regulatory Affairs (ORA).

Systems to Review and Evaluate Applications for New Products

Regulatory tracking systems are currently used by each center for the day- to-day business activities supporting FDA’s regulatory review processes. These systems are used in the receipt and storage of externally generated applications, submissions, or other information for FDA’s regulatory review processes.

Compliance systems are used to process or assess data used by FDA when overseeing conformance to regulatory requirements of an external entity or marketed product. These systems are generally used in the inspection of an FDA-regulated product or its manufacturing facilities.

Adverse event reporting and analysis systems are used to process and/or assess data related to adverse reactions to FDA-regulated products. An adverse event could be illness due to food, injury caused by a device, or negative reaction to a drug or vaccine.

Mission-Critical Infrastructure

FDA has defined its mission-critical infrastructure as IT equipment that must be available full time (24 hours a day, 7 days a week) in order for the agency to accomplish its mission. FDA identified the following infrastructure components as mission critical: Network components, which consist of Internet connectivity, domain name servers, active directory, e-mail, single sign on, and the routing infrastructure.

Critical servers to run systems needed for operations that must run full time, such as the Prior Notice Center, which must be available full time for FDA to receive prior notice before food is imported into the United States. Other examples are servers to support Mission Accomplishments and Regulatory Compliance Services, Operational and Administrative System for Import Support, and Electronic Submission Gateway.

Security components, such as the firewalls that protect the network from unauthorized users.

Secure Remote Access infrastructure, which provides the ability for authorized users to securely access FDA computing resources from a non- FDA remote location.

In addition to its mission-critical infrastructure, FDA provides other infrastructure services that support its mission, including telecommunications and help desk services.

Appendix IV: Studies That Identify FDA’s Information Technology Limitations

Reason study performed Main IT-related findings Undertaken to examine the effectiveness of the process used to develop requirements for a replacement for the agency’s dysfunctional AERS I system.

FDA’s management of requirements development did not follow proper IT methodology; the Office of IT had poor procedures in the areas of procurement and communication with end users.

Endorsed by FDA Management Council to ensure that FDA’s mission- critical IT activities are driven by proper business planning procedures.

According to a survey of participants from FDA’s business centers done to understand the state of FDA business processes for use in FDA’s business process strategies, FDA’s IT capability to support processes needed significant improvement.

Requested by members of the Congress to determine FDA’s ability to manage postmarket drug safety issues and assess the steps FDA is taking in this area.

FDA databases cannot perform some actions needed to make postmarket drug safety decisions, and different types of data are not available to FDA.

Requested by FDA to assess whether the agency’s science and technology can support current and future regulatory needs; to identify the broad categories of scientific and technologic capacities that FDA needs to fully support its core regulatory functions and decision making.

FDA’s resources have not increased in proportion to the scientific demands on the agency, resulting in demand that far exceeds its capacity to respond. FDA cannot fulfill many of its core regulatory functions because its IT infrastructure is obsolete, unstable, and inefficient.

Contracted by FDA to identify IT applications performing premarket processes, as defined by the Business Process Framework, with potential for agencywide use; also to find which applications were redundant, to retire them.

Significant overlap exists among the IT applications assessed— opportunities exist to streamline these applications; 16 of 54 premarket applications had high enterprise potential for functionality, 25 were rated medium, and 13 were rated low.

Reason study performed Main IT-related findings Requested by the Congress to investigate concerns regarding FDA’s foreign drug inspection program and make recommendations.

FDA’s databases do not provide an accurate count of foreign establishments subject to inspection and do provide widely divergent counts. Because FDA does not know the number of establishments subject to inspection, the percentage of those inspected also cannot be calculated with certainty. Inconsistencies in its databases such as these have prevented FDA from ensuring compliance with corrective items from inspections that highlighted serious deficiencies.

Required by OMB to determine FDA’s compliance with the Federal Information Security Management Act of 2002 (FISMA) in accordance with the OMB’s guidance; to determine if the FDA’s security program encompasses a risk-based life cycle approach to improving information security.

Among other things, FDA did not fully implement a security program infrastructure to support its overall security program, and FDA did not conduct all required system development life cycle activities.

Undertaken to allow FDA to better meet increased demand for information, and to make decisions more quickly and easily.

Among other things, recommendations included development of information standards at an agency level, and use of these standards within a common enterprise information model within 7 to 10 years.

Appendix V: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the contact person named above, key contributors to this report were Cynthia Scott, Assistant Director; Shaun Byrnes; Barbara Collier; Neil Doherty; Rebecca Eyler; Anh Le; Glenn Spiegel; Shawn Ward; and Daniel Wexler.