Information Technology: FDA Has Taken Steps to Address Challenges but Needs a Comprehensive Strategic Plan
Released: 2015-12-17T00:00:00

Letter

Background

Within the U.S. Department of Health and Human Services, FDA is responsible for the protection of public health by ensuring the safety, efficacy, and security of human and veterinary drugs, biological products, and medical devices; and ensuring the safety and security of our nation’s food supply, cosmetics, and products that emit radiation. The agency is also responsible for ensuring the proper labeling of foods, drugs, medical devices, tobacco, and cosmetics. Further, its work includes advancing public health by facilitating innovations and promoting public access to science-based information on medicines, devices, and foods.

As stated in FDA’s agency-wide strategy for fiscal years 2014 through 2018, FDA Strategic Priorities, four goals support five organizational cross-cutting priorities—regulatory science, globalization, safety and quality, smart regulation, and stewardship. The four supporting goals are to enhance oversight of FDA regulated products, improve and safeguard access to FDA-regulated products to benefit health, promote better informed decisions about the use of FDA-regulated products, and strengthen organizational excellence and accountability.

FDA exercises its core functions through four directorates: the Offices of Medical Products and Tobacco; Foods and Veterinary Medicine; Global Regulatory Operations and Policy; and Operations. These offices, along with the Chief Scientist, report to the FDA Commissioner and carry out their missions through seven centers and through FDA’s Office of Regulatory Affairs.

Office of Medical Products and Tobacco:

Center for Biologics Evaluation and Research. Regulates and evaluates the safety and effectiveness of biological products, such as blood and blood products, vaccines and allergenic products, and cells, tissues, and gene therapy products.

Center for Drug Evaluation and Research. Promotes and protects the public health by ensuring that all prescription and over-the-counter drugs are safe, as well as by reviewing and regulating clinical research.

Center for Devices and Radiological Health. Promotes and protects the public health by ensuring the safety and effectiveness of medical devices and preventing unnecessary human exposure to radiation from radiation-emitting products.

Center for Tobacco Products. Educates the public on the dangers of tobacco use; develops the science needed for tobacco regulation; and develops and enforces regulations on the manufacture, marketing, and distribution of tobacco products.

Office of Foods and Veterinary Medicine:

Center for Food Safety and Applied Nutrition. In conjunction with FDA’s field staff, promotes and protects the public health, in part by ensuring the safety of the food supply and that food products are properly labeled, and ensuring that cosmetics are safe and properly labeled.

Center for Veterinary Medicine. Protects and promotes the health of humans and animals by ensuring the safety and effectiveness of animal drugs, as well as the safety of animal food and devices.

Office of the Commissioner:

National Center for Toxicological Research. Conducts peer-reviewed scientific research and provides expert technical advice and training to support FDA’s science-based regulatory decisions.

Office of Global Regulatory Operations and Policy:

Office of Regulatory Affairs. Leads FDA field activities and provides FDA leadership on imports, inspections, and enforcement policy. This office supports the FDA product centers by inspecting regulated products and manufacturers, conducting sample analysis on regulated products, and reviewing imported products offered for entry into the United States. The office also develops FDA-wide policy on compliance and enforcement and executes FDA’s Import Strategy and Food Protection Plans.

Office of Operations. Ensures the timely and effective delivery of high quality and cost effective mission support services, including IT support, across the FDA and its centers, and coordinates emergency preparedness and response activities for incidents involving FDA- regulated products across FDA and its stakeholders.

FDA’s Reliance on IT

FDA relies extensively on the use and outcomes of IT to achieve its strategic priorities, fulfill its mission, and support related administrative needs. Among others, the agency has implemented systems and other technology dedicated to supporting activities, such as:

Reviewing and evaluating new product applications, such as for prescription drugs, medical devices, and food additives. These systems are intended to help FDA determine whether a product is safe before it enters the market. One such system—the Document Archiving, Reporting, and Regulatory Tracking System—is intended to manage the drug and therapeutics review process.

Monitoring the safety of products on the market by collecting and assessing adverse reactions to FDA-regulated products, such as illnesses due to food or negative reactions to drugs. This activity is supported by systems such as the Vaccine Adverse Event Reporting System, which accepts reports of adverse events that may be associated with U.S.-licensed vaccines from health care providers, manufacturers, and the public.

Screening imported food to detect and prevent entry into the United States of adulterated, misbranded, or potentially spoiled food. For example, the Predictive Risk-based Evaluation for Dynamic Import Compliance Targeting is a tool used as part of the system that provides entry reviewers with information to help target high-risk shipments for examination and expedite the clearance of lower-risk cargo.

Surveilling and promoting the safety of FDA-regulated products. With the exception of laboratories, systems developed and managed by the Mission Accomplishments and Regulatory Compliance Services (MARCS) investment are used to support all field activities, including domestic and foreign inspections, imports, compliance, and enforcement activities.

Conducting post-market safety surveillance for drug and therapeutic biologic products that have been approved by FDA. The FDA Adverse Event Reporting System is used to help identify new safety concerns that might be related to a marketed product, evaluate a manufacturer’s compliance with reporting regulations, and respond to outside requests for information.

In addition, FDA relies on various systems that support its administrative processes, such as payroll administration and personnel systems. All of the agency’s systems are supported by an IT infrastructure that includes network components, critical servers, and multiple data centers.

FDA’s budget for the development, operations, and maintenance of its IT investments was approximately $419 million for fiscal year 2013 and approximately $566 million for fiscal year 2014. IT spending for fiscal year 2014 was categorized into three portfolios: $287.4 million for IT investments that support mission delivery and $260 million for IT infrastructure, security, office automation, and $18.72 million for investments in enterprise architecture capital planning and chief information officer functions.

The agency received about $578 million in IT funding for fiscal year 2015.

FDA IT Governance

Within FDA, overall responsibility for managing IT resides in the Office of Information Management and Technology (OIMT), which was established within the Office of Operations as part of an agency-wide reorganization in January 2014. OIMT is headed by the agency’s Chief Operating Officer, and its mission is to ensure the timely and effective delivery of high quality, innovative and cost-effective mission- related information technology support services across the FDA and its centers. The office’s major functions include information security, business and customer assurance, technology, and informatics and technology innovation.

The Office of Information Management (OIM) is a component of OIMT and is led by the CIO. OIM’s mission includes, among other things, setting goals for improving customer focus and developing new IT solutions that support changing business needs. Within OIM, and under the direction of the Chief Operating Officer, the CIO is responsible for managing and overseeing the design and development of IT systems to support FDA’s mission, and creating a foundation to enhance the interoperability of the agency’s systems. As such, this official is charged with providing services and support to approximately 20,000 federal personnel and contractors throughout the United States and the international FDA community.

Our Prior Work Has Included Recommendations to Address FDA’s Challenges with IT Strategic Planning and Management

In previously reporting on FDA’s IT challenges, we highlighted deficiencies in a number of the agency’s management processes and system modernization efforts. Accordingly, our two reports—issued in June 2009 and March 2012—included a total of nine recommendations aimed at improving key IT areas, such as IT strategic planning, enterprise architecture development, human capital planning, investment management, information sharing, and project management. (We provide a detailed discussion of the agency’s actions to address the recommendations later in this report.)

Specifically, in 2009 we reported that FDA did not have a comprehensive IT strategic plan to coordinate and manage the numerous modernization projects that it was pursuing in response to federal law and guidance and the agency’s urgent mission needs. We noted that the agency had developed planning documents that included some, but not all of the elements of a comprehensive plan, and stressed that, lacking such a plan, the agency’s ability to meet its strategic goals was at risk. Thus, we recommended that the Commissioner of FDA require the CIO to set milestones and a completion date for developing a comprehensive IT strategic plan that included results-oriented goals, strategies, milestones, performance measures, and an analysis of interdependencies among projects and activities. We further recommended that the agency use such a plan to guide and coordinate its modernization projects and activities.

Our 2009 report also noted that FDA had made mixed progress in establishing important IT management capabilities that are essential for helping to ensure a successful modernization, particularly in the areas of enterprise architecture and related segment development, and in IT human capital planning and management. We recommended additional actions to be taken by the CIO to improve these areas, which we discuss in detail later in this report. The Department of Health and Human Services, in commenting on our 2009 report, agreed with our recommendations and identified actions FDA had initiated or planned to address them.

We again reported on the agency’s IT management capabilities in 2012, noting that FDA lacked a comprehensive IT inventory—a tool for managing its investments that identifies and provides key information about the systems it uses and is developing. We also noted that much work remained to be done on the agency’s largest and most costly system modernization effort— the MARCS program—and that FDA’s Center for Food Safety and Applied Nutrition (CFSAN) had not assessed information sharing needs and capabilities. We recommended that FDA take actions regarding these issues, which we discuss in detail later in this report. Further, we pointed out that FDA still did not have an actionable IT strategic plan that identified specific goals and corresponding tasks to guide its overall modernization efforts and support agency-wide goals, as we had recommended in our earlier report. In commenting on our 2012 report, the Department of Health and Human Services neither agreed nor disagreed with the recommendations, but stated that FDA was taking actions to address many of the issues we identified.

Changes in FDA’s IT Management Structure and Strategy

Our prior work stressed that one of the reasons FDA had not established key management capabilities for the areas we found to be deficient was because the agency’s IT management structure had been in flux. Specifically, the agency had appointed five CIOs since 2008—each with differing priorities for managing IT, including the modernization of the agency’s infrastructure. Two of those CIOs served in an acting capacity during 2011, with the agency hiring a permanent CIO in October 2011. However, that official left FDA in March 2013, and the agency’s Chief Operating Officer took on additional responsibilities as an acting CIO until May 2015, when the agency hired its current CIO.

Subsequent to our 2012 report, FDA’s CIO that was hired in October 2011 led the development of an IT strategic plan that was to be used to support the agency’s mission and goals. The plan, released in September 2012 and entitled Information Management Strategic Plan, Version 1.1, defined goals and objectives to be addressed from fiscal year 2012 through fiscal year 2016. The documented intent of the plan was to set the path forward for modernization of the agency’s IT infrastructure to more effectively support the agency’s mission, goals, and objectives. The plan included four goals and related objectives, major activities to fulfill these objectives and to be implemented within specified time frames, and performance indicators for determining whether outcomes resulting from the implementation of the plan were effective toward achieving goals. While FDA implemented some of the activities from 2012 through 2015, according to the CIO and OIM officials who participated in its development and implementation, the plan was not used, as intended, to manage and measure outcomes of activities to support the agency’s mission, goals, and objectives. OIMT officials told us that the plan was not used because the CIO who was present when the plan was finalized left the agency and the Chief Operating Officer, who resumed responsibilities of an acting CIO, managed IT from an operational perspective. According to these officials, the Chief Operating Officer worked with the agency’s IT organization to identify and incorporate key initiatives of the Information Management Strategic Plan into the Office of Operations’ strategic planning activities.

FDA Has Developed a New IT Strategic Plan, but It Lacks Key Elements

To be effective as a guide for managing IT investments and better address agency-wide business challenges, best practices show that an IT strategic plan should define the agency’s vision and provide a road map to help align information resources with business strategies and investment decisions. Accordingly, best practices call for a comprehensive and effective plan to (1) be aligned with the agency’s overall strategy; (2) identify the mission of the agency, results-oriented goals, and performance measures that permit the agency to determine whether implementation of the plan is succeeding; (3) include strategies the governing IT organization will use to achieve desired results; and (4) provide descriptions of interdependencies within and across projects so that they can be understood and managed.

FDA’s new CIO, who was hired in May 2015, has taken steps to develop a new IT strategic plan. The CIO has stated that the intent of the plan, entitled Information Technology Strategic Plan, Version 1.0 and dated September 2015, is to allow OIMT to address business challenges facing the agency. The plan describes the current state and customer perception of the IT environment, along with OIMT’s mission, vision, and objectives of three strategic themes—quality service, security, and efficiency.

To support the office’s strategic themes, the CIO defined in the plan objectives, performance measures, and 15 initiatives. For example, the plan defines objectives for supporting stakeholders, such as “improve customer satisfaction” and “increase security,” as well as performance measures and targets for meeting the objectives. Specifically, a target and performance measure for “improve customer satisfaction” is that at least 80 percent of help desk requests are to be resolved on the first call. The plan also defines 15 categories of initiatives that are intended to support OIMT’s efforts to meet the objectives. These categories of initiatives are to address areas such as business continuity, contract and project management, customer service, IT security, infrastructure, and application development.

The 2015 IT strategic plan also includes a table of specific milestone activities that are mapped to OIMT’s categories of initiatives, and identifies responsible parties and target dates for completing them. For example, information included in the table describes a milestone to “improve knowledge and use of telework tools” that is to support the business continuity initiative. It also defines a target date for accomplishing the milestone, along with a responsible entity within FDA. The table also provides information about the status of the milestones’ implementation.

Nevertheless, in its current state, the document that FDA has developed does not include the key elements of a comprehensive IT strategic plan. First, the new plan does not align OIMT’s objectives with the agency-wide strategies and goals that were defined by the Commissioner in FDA Strategic Priorities: 2014-2018. Specifically, the plan identifies objective areas aimed at accomplishing OIMT’s mission that include, among other things, information security, training of staff, and project management. However, the plan does not address or align with the agency’s five strategic priorities—regulatory science, globalization, safety and quality, smart regulation, and stewardship—and the goals for achieving them: enhance oversight of FDA regulated products, improve and safeguard access to FDA-regulated products to benefit health, promote better informed decisions about the use of FDA-regulated products, and strengthen organizational excellence and accountability.

Further, the plan does not identify goals and performance measures for determining whether its implementation is successful in supporting the agency’s mission. Rather than emphasizing the agency-wide mission to protect public health, the plan reflects an office-level (OIMT) mission to “provide effective and fiscally responsible technology services in a manner that promotes high standards.” Thus, the performance measures and milestones for achieving goals were defined to be used to determine the success of initiatives such as applications development, staffing, and training. In addition, because the plan does not align with agency-wide goals and mission performance, it cannot provide information the agency needs to define appropriate performance measures and mechanisms for monitoring the outcomes of any IT objectives or initiatives to determine whether they contribute to FDA’s progress toward meeting its mission goals.

The 2015 IT strategic plan also does not identify initiatives that can be aligned with agency-wide strategies described in FDA’s strategic priorities document. Rather, it identifies specific IT initiatives that can be linked to OIMT’s mission, such as contract and project management, customer service, IT security, infrastructure, and application development. For example, the IT strategic plan includes a “cybersecurity” initiative and identifies activities for securing internal and external data. While this initiative could help support one of the strategies for achieving the fourth goal of the agency-wide plan—secure mission critical and sensitive assets and information—the IT strategic plan does not link this initiative to the agency-wide strategy.

Finally, the plan does not identify interdependencies among planned and established IT initiatives. Further, it does not identify specific initiatives that support strategies and goals for fulfilling the agency’s public health mission. For example, the agency-wide plan identifies a strategic goal to enhance oversight of FDA-regulated products. However, the IT strategic plan does not identify any planned OIMT initiatives or ongoing modernization projects that could be aligned with this goal, such as the MARCS program—a collection of subprojects intended to enhance existing applications and develop new systems in support of the surveillance and safety of FDA-regulated products. Given the limitations noted, FDA’s current IT strategic plan is not a comprehensive tool for providing its officials the information they need to determine and manage interdependencies across IT projects throughout the agency.

In discussing our concerns, the CIO stated that the intent of the first version of the September 2015 IT strategic plan was to identify improvements, based on an assessment of FDA’s IT environment, that needed to be made to existing IT processes, technologies, roles, functions, and capabilities within OIMT. In October 2015, the CIO documented a process for maintaining milestones in the plan on a monthly basis, and for adding initiatives and milestones to updated versions that the agency intends to release on an annual basis. However, according to OIM officials and the CIO, they have not yet established or documented schedules or milestones for managing and completing the development and implementation of the next version of the IT strategic plan. As such, it is not known if and when FDA will have a comprehensive plan that can be implemented to provide the guidance and direction needed for effectively managing IT initiatives and ensuring that outcomes of the initiatives help the agency meet its overall mission and goals.

Until FDA incorporates into its IT strategic plan necessary elements that align the goals and objectives with its overall mission, the agency will continue to lack critical information needed to ensure that information resources support business strategies and investment decisions. In addition, without defined performance measures that support the goals of the agency, along with mechanisms for assessing system outcomes against them, FDA will be hampered in its ability to determine whether its IT initiatives are successful in supporting agency-wide goals. Further, until the plan identifies key IT initiatives and describes interdependencies among them, FDA will also remain without critical information it needs to guide and coordinate its modernization projects and other IT initiatives, and ensure that they are implemented in support of the agency’s strategic goals.

FDA Has Made Progress in Implementing Prior Recommendations on IT Planning and Management

FDA has implemented six of the nine recommendations included in the two reports that we issued in 2009 and 2012. Specifically, the agency took actions that resulted in the implementation of four of the five recommendations that we made in 2009, and two of the four recommendations we made in 2012. As a result of its actions, FDA has improved its ability to plan and manage the agency’s IT initiatives. Implementing the remaining recommendations we made regarding needed improvements in the areas of IT strategic planning, and scheduling and monitoring the progress of key projects is important for ensuring that ongoing and planned IT initiatives and modernization projects are successful in supporting agency-wide goals and business needs.

FDA Implemented Four of Five Recommendations Related to IT Planning

With regard to the recommendations included in our 2009 report, FDA has taken actions that led to the implementation of four of the five recommendations for improving IT planning. The lone outstanding recommendation is to complete a comprehensive IT strategic plan. As previously discussed, FDA has developed a plan, but it is neither comprehensive nor linked to the overall agency strategic plan.

Table 1 summarizes the status of FDA’s actions to implement the five recommendations.

We reported in 2009 that FDA had significant work remaining with regard to its enterprise architecture development, including developing a program management plan for ensuring that an enterprise architecture is effectively and efficiently developed. We noted that, without an enterprise architecture, FDA did not have the assurance it needed to enable effective modernization of its IT environment. Accordingly, we recommended that the agency develop an enterprise architecture program management plan that includes a detailed work breakdown of the tasks, activities, and time frames associated with developing the architecture, as well as the funding and staff resources needed.

FDA has implemented this recommendation. In particular, FDA completed its Enterprise Architecture Program Management Plan, dated July 2012, that outlines the agency’s key enterprise architecture initiatives and program management functions, including a work breakdown of the major tasks, activities, time frames, and key staff responsible for developing the enterprise architecture. The plan’s key initiatives include adopting enterprise architecture processes and a management approach, developing the architecture, and developing and implementing an approach for operating and maintaining the architecture.

The agency also included in its Segment Architecture Funding and Resource Plan, dated September 2013, an estimate of $3.6 million for funding needed to accomplish enterprise architecture activities. The plan called for the establishment of a Program Support Office made up of OIM staff and business stakeholders from FDA centers. The office and its staff were to be responsible for developing and managing the enterprise architecture. As a result, FDA should be better positioned to manage its enterprise architecture development and ensure the effectiveness of its IT modernization efforts.

Agency Officials Defined Criteria for Prioritizing Segment Architectures

In 2009, we also reported that FDA had begun building the segments of an enterprise architecture before it had established priorities for doing so. However, the Federal Enterprise Architecture Practice Guidance states that prioritizing segments, based on business needs, should precede building them. Accordingly, we recommended that FDA define criteria for setting priorities for the segment architecture and then prioritize building of the segments accordingly.

In response, FDA identified relevant segment scoring criteria for use in setting priorities for the architecture segments. Further, as of September 2013, the agency had completed architecture development for its two highest priority segments—regulatory science and post-market safety and surveillance—and planned to prioritize remaining segments, such as scientific computing. These actions better positioned the agency to develop its segment architectures based on priorities, and reduce risks that its systems modernization efforts may not be successful in supporting FDA’s business needs.

FDA Accelerated the Development of the Enterprise Architecture and Managed Interim Risks

Our report also noted that the agency’s major modernization projects were proceeding without the guidance and constraint of an enterprise or segment architecture. Accordingly, we recommended that FDA accelerate development of the enterprise architecture, including “as is” and “to be” architectures and transition plans, and, in the meantime, develop plans to manage the increased risks to modernization projects of proceeding without an architecture to guide and constrain their development.

Consistent with this recommendation, FDA took steps to accelerate the development of its segment and enterprise architectures. For example, in fiscal year 2013, the agency developed “as is” and “to be” enterprise architectures in terms of business and applications, and proposed solution architectures for certain segments, including Adverse Events Analysis and Reporting and Regulatory Science. Further, FDA reported in its July 2012 Enterprise Architecture Program Management Plan that it had plans to complete the development of its enterprise architecture and transition plan and, in an update to the plan, stated that it intended to complete the segment architectures.

In conjunction with these efforts, FDA took steps to manage the increased risks of proceeding without the guidance that would be provided by a complete architecture. For example, in August 2013 it formed an Engineering Review Board to discuss and reach mutual agreement on architectural solutions for modernization projects that promote interoperability among systems. Further, the FDA centers jointly conducted reviews of solutions already in progress to identify IT capabilities that could be reused to avoid duplication. As a result of its actions, the agency should be better positioned to proceed with modernization projects and reduce risks of introducing inefficiencies, such as the lack of interoperability and duplication of efforts, that could occur without a complete enterprise architecture.

The Agency Addressed IT Human Capital Planning

In our 2009 report, we also noted that FDA had not inventoried the skills of its current IT workforce, determined present and future skills needs, or analyzed gaps—steps it needed to take to determine critical skills and competencies for achieving IT program results. Accordingly, we recommended that the agency develop a skills inventory, needs assessment, and gap analysis, and develop initiatives to address any identified gaps as part of a strategic approach to IT human capital planning.

Consistent with this recommendation, OIM officials implemented a strategic approach to IT human capital planning. Specifically, in September 2013, the office identified critical skill shortages, including insufficient cybersecurity skills. It also concluded that network, engineering, and infrastructure operations skills were outdated. In addition, OIM made plans to use a tool that allowed it to collect skills information about its staff on an ongoing basis, and conducted a skills gaps analysis, which identified gaps relating to eight job descriptions in two of its office components. FDA officials also took actions to address the skills gaps, including offering project management training and hiring staff, such as database administrators. As a result of its actions, the agency made important progress toward a strategic approach for managing IT human capital to more effectively achieve program results.

FDA Implemented Two of Four Recommendations Regarding IT Management Challenges

FDA has implemented two of the four recommendations we made in 2012 regarding challenges in key areas of IT management. Specifically, the agency developed a systems inventory and improved the sharing of information within CFSAN. However, it has taken some actions, but has not yet fully implemented two recommendations regarding the development and use of schedules for managing and monitoring the progress of IT initiatives.

Table 2 summarizes the status of the four recommendations, and a discussion of each recommendation follows the table.

Our 2012 report noted that FDA did not have a complete list of IT systems that it was using or developing. We recommended that the agency develop an inventory that includes key information for each of its systems.

In response, FDA took actions to implement our recommendation. Specifically, in May 2015, FDA developed an inventory that identified and described 205 systems that were reported in the agency’s fiscal year 2016 budget estimates for IT systems to the Office of Management and Budget. The inventory includes, for each system, data such as the system’s name; description; annual costs of development, maintenance, and operations; effective date the system became operational; and a unique identifier that links each system to a corresponding investment reported to the Office of Management and Budget. Further, in June 2015, the agency issued a draft procedure for maintaining the inventory, which it intends to follow to help ensure that the inventory is consistently updated.

In addition, FDA incorporated use of the system inventory into the agency’s IT investment management process. For example, FDA identified for each system the business capabilities that they support, such as strategic planning, emergency response, and product review and approval. These data enable the agency to organize systems and investments into portfolios according to business capabilities and to view systems as they relate to other systems that perform similar functions, rather than as unrelated IT systems. This portfolio view is intended to be used to help FDA better manage its IT investments as it allows agency officials to identify any gaps in its ability to support specific business capabilities that may arise after a system is retired. According to OIM officials, when such gaps are identified, the agency may then identify and appropriately fund IT investments that are needed to fill the gaps and ensure continued support of the agency’s mission.

FDA Assessed Information-sharing Needs and Capabilities for CFSAN

We reported in 2012 that, although we had previously identified deficiencies in CFSAN’s ability to effectively share critical information, such as information on recalls of contaminated food, FDA had not performed a comprehensive review to identify opportunities for improving data sharing within the center. We therefore recommended that the agency perform a full assessment of the center’s data-sharing needs and capabilities, and develop a plan to address such needs and provide any additional capabilities.

In response, FDA implemented our recommendation. Specifically, FDA assessed and documented CFSAN’s information-sharing needs and capabilities. The resulting document, entitled Office of Foods and Veterinary Medicine Data Management Strategy and Roadmap (OFVM Roadmap), identifies IT shortfalls that contribute to information-sharing issues, such as the lack of integration between systems, and cases when systems do not store data in a central warehouse that could be used to share information. The document includes plans, with milestones and dates for addressing the center’s information-sharing needs, such as access to data needed to conduct food facilities and import inspections.

The roadmap also describes an existing capability for data sharing, the Online Reporting and Analysis Decision Support System (ORADSS) Data Warehouse. It noted that the warehouse was being used to store significant amounts of data used by CFSAN, such as data provided by FDA’s Office of Regulatory Affairs, but that most data from the center’s systems were not stored in a data warehouse. Specifically, the roadmap identified 17 systems that were assessed as “likely candidates” for integration with ORADSS, but did not store data in the warehouse. The roadmap also identified systems from other centers that could share data via the warehouse, and further stated that incorporating data from these systems into the warehouse would support data sharing by allowing access to information from one location, thereby improving collaboration within and between FDA’s centers.

In addition, consistent with plans described in the OFVM Roadmap, the agency documented a plan for sharing scientific computing data among CFSAN and its partners. This plan is focused on systems for sharing genomics data that are used to identify and track bacteria that contaminate food products. OFVM officials determined that scientific data should be maintained and stored separately from other data because, unlike the transactional data that are structured and can be stored in a data warehouse, scientific computing data are less structured and can be more voluminous, and require additional computational and data transfer capabilities. To address the need for additional data storage and sharing capabilities for these data, the agency established two scientific databases—the Bionumerics SQL and the Next Generation Sequencing databases.

By taking these actions, FDA has completed important steps toward developing an IT environment in which data from multiple systems and databases can be more efficiently stored and accessed.

FDA Has Begun but Has Not Completed Efforts to Develop an IMS for MARCS

In 2012 we noted that considerable work remained to be completed on the MARCS program, one of FDA’s largest and most costly IT modernization efforts. We recommended that the agency develop an IMS to assist in managing and monitoring projects that are part of the MARCS program. We stressed that such a schedule should (1) identify which legacy systems will be replaced and when, (2) identify all current and future tasks to be performed by contractors and FDA, (3) define and incorporate information reflecting resources, and (4) define critical dependencies between tasks.

FDA took a number of steps to implement the recommendation, but there are shortfalls in its approach for identifying tasks and resources. For example, FDA developed an IMS that identified legacy systems and included associated tasks and time frames for updating or maintaining the systems from fiscal year 2014 through fiscal year 2018. In addition, the agency identified detailed tasks to be completed in fiscal years 2014 and 2015, and also included in the schedule some summary-level tasks to be completed in future years, such as those for maintaining legacy systems.

However, we could not determine if the tasks in the schedule included all tasks that needed to be completed to implement the MARCS projects. We could not make this determination because the agency had not documented the information that it needed to identify all tasks that should be included in the schedule—i.e., it had not developed a work breakdown structure that could be used to identify all tasks needed to be completed for the program.

In addition, the schedule did not identify all resources needed to complete certain tasks—i.e., it did not identify specific FDA staff or the skills needed to complete tasks. Instead, the schedule identified responsible entities within the agency, thus increasing the risk that needed staff and skills could be assigned to another task and, therefore, would not be available to work on others.

Lastly, we found errors in the way dependencies between tasks were defined. Specifically, the MARCS IMS did not identify external dependencies between activities, such as projects to be completed by contractors, and dependencies that were identified between tasks contained errors. For example, there are a significant number of tasks in the schedule for which there are no identified dependencies, and some remaining tasks have restrictions on their start date that are not justified— i.e., the restrictions are not based on the start or finish of an activity on which the tasks are dependent.

Agency officials from the Office of Regulatory Affairs and the MARCS program said that the IMS identifies detailed tasks for the current fiscal year because FDA develops its schedule based on work that is to be accomplished during that year. According to the officials, this is done because projects are funded on an annual basis; therefore, they cannot know what projects will be conducted in future years. The officials further explained that the IMS is updated at least annually based on changes in business priorities and funding availability, and as the planning horizon advances. While this approach can be taken for developing schedules, according to best practices defined in our schedule assessment guide, all tasks needed to complete the program should be identified within a work breakdown structure to ensure that the total scope of work is accounted for within the schedule.

The officials also noted that they do not identify specific FDA staff or estimate the percentage of time required of those resources because they assume that the responsible entities will dedicate one staff person to work full time on each project. However, a schedule that does not identify specific resources assumes unlimited availability and, thus, the risk of the program’s schedule slipping significantly increases if needed staff or skills are not available when needed.

In addition, officials said that, while they identify some dependencies between contractors’ projects, they do not identify all of them because some of the projects are not tightly integrated. The program officials further stated that the objectives and scope of the MARCS program today are much different from those at the time we recommended the development of an IMS, and that the program does not currently have the cross-task dependencies that existed in the prior scope that would be addressed by such an integrated schedule. However, the identification of interdependencies between all activities is important to allow program officials to recognize delays that could occur when activities that slip early in the schedule impact concurrent or future activities.

Lacking certainty that the IMS includes all short- and long-term tasks, reliable estimates of resources, and dependencies between individual tasks and contractors’ activities, the value of the schedule as a critical management tool is diminished—i.e., it cannot be used to accurately gauge the level of effort required to implement the MARCS program, determine whether staff will be available when needed, or predict the effect of factors such as delayed contractor activities on the schedule. Thus, the MARCS schedule cannot be used as an effective tool for managing and monitoring the progress of the project.

FDA Has Not Yet Used the IMS to Monitor Progress of MARCS

In our 2012 report, we also noted that the agency lacked an IMS for MARCS that provided a summary of progress on lower-level tasks and effects of changes to lower-level schedules and tasks on the overall project. Thus, we reported that the agency did not have the information it needed to gauge the progress of the entire project. We recommended that, in addition to developing a complete IMS, agency officials use it to monitor progress of the MARCS program.

Agency officials have implemented certain activities that could help them use the IMS to monitor the progress of MARCS; however, they have not developed the schedule to include all the information needed to do so. First, agency officials began to conduct monthly updates that, for example, reflected actual progress from individual system subproject schedules. In addition, the program office developed a baseline schedule that could be used for measuring and monitoring project performance and progress. These actions are important steps to be taken toward addressing our recommendation that agency officials use the MARCS IMS to monitor the program’s progress.

Yet, because of deficiencies in the agency’s practices for conducting these activities, along with other deficiencies noted previously, the usefulness of the schedule for monitoring the program is limited. For example, although program officials update the IMS on a monthly basis, it includes activities with projected start or finish dates that are prior to the date the schedule was updated. However, according to best practices defined in our schedule assessment guide, projected start and finish dates should never be prior to the date the schedule was updated; that is, dates cannot be projected to start in the past. Agency officials within OIMT stated that they manually update the schedule each month, and do not use a tool for automated updating. As a result, the updates often are not accurate or complete, which could account for projected dates for activities not being modified during the monthly updates.

MARCS program officials also developed a baseline schedule that is aligned to the MARCS IMS, and is intended to be used to monitor the progress of the program. For example, data in a baseline schedule represent important information that program management could use to assess the validity of the program’s initial duration estimates and the program’s progress toward beginning and finishing work on time and according to plan—i.e., trend data. While program officials stated that they planned to analyze trend data provided by the baseline schedule at the end of fiscal year 2015, and use the results of the analysis to manage and monitor the progress of MARCS, they have not yet done so.

While FDA officials have taken some steps to use the IMS and a baseline schedule to monitor the progress of the MARCS implementation, they are doing so with inaccurate or incomplete information. Therefore, the agency may not have necessary information to monitor and make any needed changes to program plans to ensure that the schedule for implementing MARCS remains on track and the project is providing results that meet the agency’s needs.

The fiscal year 2016 business case that was reported to the Office of Management and Budget indicates that funding for MARCS is planned through fiscal year 2019. However, officials with the Office of Regulatory Affairs told us that the agency has chosen to restructure the MARCS investment to better reflect current business needs and objectives. As such, it is important that FDA continue to take steps toward implementing the recommendations that we made in our 2012 report regarding the agency’s need to develop an IMS that can be used to manage the implementation of MARCS and ensure its success.

Conclusions

Although FDA has developed an IT strategic plan, it does not include elements that best practices reflect are associated with a comprehensive plan—that is, it is not aligned with agency-wide strategic priorities, and does not include results-oriented goals and performance measures that support the agency’s mission, identify key IT initiatives that support agency-wide goals and objectives, or describe interdependencies among the initiatives. Until it incorporates these elements into and fully implements the plan, the agency will continue to lack critical information needed to align IT initiatives with business strategies and investment decisions, and to determine whether outcomes of the IT initiatives are succeeding in supporting the agency-wide mission, goals, and objectives.

In addition, FDA has implemented most of our prior recommendations for improving IT management practices. Taking further actions to fully implement the remaining recommendations will improve the agency’s ability to successfully monitor the progress of one of FDA’s most critical and costly IT modernization projects throughout its remaining life cycle.

Recommendations for Executive Action

To help ensure that FDA’s IT strategic planning activities are successful in supporting the agency’s mission, goals, and objectives, we recommend that the Commissioner of FDA require the CIO to take the following two actions: establish schedules and milestones for completing a version of an IT strategic plan that incorporates elements to align the plan’s strategies with agency-wide priorities; includes results-oriented goals and performance measures that support the agency’s mission, along with targets for measuring the extent to which outcomes of IT initiatives support FDA’s ability to achieve agency-wide goals and objectives; identifies key IT initiatives that support the agency’s goals; and describes interdependencies among the initiatives; and implement the plan to ensure that expected outcomes of the agency’s key IT initiatives are achieved.

Agency Comments and Our Evaluation

We received written comments on a draft of this report, signed by the Department of Health and Human Services’ Assistant Secretary for Legislation. In the comments (reprinted in appendix II), the department stated that it concurred with our recommendations. Further, the department stated that FDA is committed to continuing the evolution and implementation of its IT strategic plan and has established a process for the annual review and updating of the plan. It added that FDA intends to follow this process to incorporate the alignment of strategic initiatives with the agency’s mission and business strategies. If effectively implemented, such actions could help assure that FDA develops a comprehensive plan that can be used to provide the guidance and direction needed to ensure that outcomes of key IT initiatives help the agency meet its overall mission and goals. The Department of Health and Human Services also provided technical comments, which we incorporated into the report as appropriate.

We are sending copies of this report to the Secretary of Health and Human Services, Commissioner of the Food and Drug Administration, appropriate congressional committees, and other interested parties. This report is also available at no charge on our website at http://www.gao.gov.

If you or your staff have any questions on matters discussed in this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to (1) determine the status of the Food and Drug Administration’s (FDA) efforts to develop and implement an information technology (IT) strategic plan that includes results-oriented goals, activities, milestones, and performance measures; and (2) assess the extent to which FDA has addressed our prior IT-related recommendations.

For the first objective, our scope was focused on FDA’s efforts to develop an IT strategic plan to be used to manage and guide key initiatives, including modernization projects, in support of agency-wide goals and objectives. To that end, we assessed FDA’s IT strategic plan, Information Technology Strategic Plan, Version 1.0, dated September 2015, by comparing its contents to key practices for IT strategic planning that we have previously identified. Those practices include developing a plan that (1) is aligned with the agency’s overall strategy; (2) identifies the mission of the agency, results-oriented goals, and performance measures that permit the agency to determine whether implementation of the plan is succeeding; (3) includes strategies the governing IT organization will use to achieve desired results; and (4) provides descriptions of interdependencies within and across projects so that they can be understood and managed. We compared the goals and objectives in the IT strategic plan to agency-wide goals and objectives defined in the September 2014 FDA Strategic Priorities document, which was released by the Commissioner to establish key strategies for fulfilling FDA’s public health mission. Finally, we examined the IT strategic plan to identify any key IT initiatives and modernization projects and determine whether interdependencies with any such projects were defined.

To address our second objective, we focused on actions FDA took to implement recommendations that we made in 2009 and 2012 as a result of studies related to IT management challenges that FDA faced. Specifically, to determine actions taken by FDA to implement the five recommendations related to IT planning that we made in our 2009 report, we collected and examined agency documentation related to its IT strategic planning, enterprise architecture development, and human capital planning. Specifically:

For the recommendation to develop a comprehensive IT strategic plan, we took the various actions described in the preceding paragraphs that pertained to objective 1.

For the recommendation to develop an enterprise architecture program management plan, we assessed FDA’s July 2012 Enterprise Architecture Program Management Plan against criteria defined in GAO’s enterprise architecture management maturity framework, such as the inclusion of detailed tasks and time frames necessary to develop the architecture. We also examined the plan to determine whether it identified the organizational unit and its staffing that would implement the plan, and whether it included detailed costs for implementing tasks described in the plan.

For the recommendation that FDA define criteria for setting priorities for developing the segment architectures, we analyzed documentation, such as the FDA Architecture Segment Architecture, Segment Scoring, that described the agency’s method for establishing priorities, to determine whether the approach for prioritizing segments development considered mission-based criteria. We also reviewed FDA’s prioritized list of segments and plans for completing segments, documented in its September 2013 FDA Segment Architecture Prioritization and List of Segments, to determine the agency’s progress and status of its effort to develop segments.

To address the recommendation that FDA accelerate development of the segment and enterprise architectures and develop plans to manage the increased risk of proceeding without an architecture, we reviewed documentation on the status of FDA’s architecture development initiative to determine whether progress had been made toward developing “as is” and “to be” architectures. Specifically, we reviewed segment architecture documentation to identify whether the “as is” architecture included components such as business processes and applications. We also reviewed documentation describing the agency’s “to be” architecture to identify the architectural components and transition steps that were included in the conceptual view. We also interviewed FDA officials to determine whether they had planned an approach to mitigate the risk of developing systems prior to completing the architecture. Finally, we assessed FDA’s enterprise architecture governance plan to determine whether the agency had established an organizational structure and designated responsibilities that supported a risk mitigation approach.

For the final recommendation regarding human capital planning, we assessed actions taken by FDA to implement a strategic approach to IT human capital planning, including developing a skills inventory, needs assessment, and gap analysis, and addressing skills gaps. To do so, we reviewed FDA’s assessment of the agency’s IT skills and gaps and its assessment of critical skill shortages. We also reviewed a template that OIM used to collect and inventory information on employee skills, to verify that FDA had an approach in place for collecting this information.

To assess FDA’s progress in addressing the four recommendations regarding IT management from our 2012 report, we collected and examined documentation that provided information relevant to the actions the agency had taken. Specifically:

For the recommendation that FDA develop a complete IT systems inventory, we compared the contents of its inventory to systems included in the IT portfolio summary that accompanied the agency’s fiscal year 2016 budget request to the Office of Management and Budget. We also examined artifacts that provided evidence of steps taken by FDA during its IT investment management process, including reports generated from the systems inventory that were used to align systems with business capabilities and help to identify versions of technologies to be retired. We assessed a document that described technologies used to support various systems, along with schedules for maintaining them, to identify information from the inventory that was used to make funding decisions.

To determine whether FDA had taken steps to assess information- sharing needs and capabilities of CFSAN, we reviewed an information-sharing assessment and roadmap for addressing business needs of the Office of Foods and Veterinary Medicine. Through our review, we identified the approach that was used to assess data-sharing needs, the capabilities that were identified, and the underlying reasons for information-sharing problems. In addition, we analyzed sections of the roadmap and compared the systems FDA was assessing for inclusion in a central data warehouse against the systems we identified in our prior report. We also examined FDA’s plan for addressing scientific computing information-sharing needs of CFSAN by assessing FDA’s Planned Scientific Data Sharing Capabilities and identifying the approaches that were planned to address data sharing, the agencies that were included as data-sharing partners, the types of information planned for sharing, and the schedule and steps that were planned to address scientific data-sharing needs.

To assess actions the agency had taken to address our recommendation that it develop an integrated master schedule (IMS) for the Mission Accomplishments and Regulatory Compliance Services (MARCS), we obtained the most current version of the schedule and determined whether it was complete by comparing it to best practices defined in our schedule guide. For each of the practices, we collected and examined documentation, such as descriptions of projects included in the IMS and their supporting schedules. In addition, we compared the tasks identified in the IMS against a list of MARCS legacy systems that FDA planned to replace, maintain, or update, to determine whether the schedule identified all legacy systems to be replaced and time frames for replacing them. We also used a software tool that analyzes project schedules to determine whether all tasks had dependencies identified and were sequenced logically, the number of tasks with date constraints, and whether all tasks had resources identified to complete the tasks. We inspected the IMS to determine whether it identified specific staff or skills needed to complete tasks. Finally, to establish whether the integrated schedule was aligned with other schedules on which it was dependent, we compared it to the project’s contractors’ schedules and looked for evidence of linkages. We also conducted a structured interview with agency officials responsible for developing the IMS, based on questions from our schedule guide, to obtain FDA’s explanation of practices that they were conducting or were conducting using a different approach than specified in best practices.

Finally, to determine whether FDA had implemented our recommendation that it monitor the progress of MARCS using the IMS, we obtained and examined documents that described the agency’s approach for updating the schedule to reflect the progress made toward completing specific tasks. We compared its approach to best practices in our schedule guide for updating a schedule to be used for monitoring the progress of projects. We analyzed the most recently updated MARCS IMS and subproject schedules, and compared them to earlier versions to identify evidence of recent updates. In addition, we analyzed the schedule using a software tool to determine whether individual task dates were updated, and whether dependencies between tasks were logical.

For each of the recommendations, we supplemented the information we collected from examining and analyzing agency documentation by interviewing FDA officials knowledgeable of actions taken to address the recommendations.

We conducted this performance audit from March 2015 to December 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusion based on our audit objectives.

Appendix II: Comments from the Department of Health and Human Services

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Teresa F. Tucker, Assistant Director; Melina I. Asencio; Elena P. Epps; Rebecca E. Eyler; Jason T. Lee; Jennifer V. Leotta; Thomas E. Murphy; Edward G. Varty; Daniel K. Wexler; and Charles E. Youman made key contributions to this report.