Information Technology: FDA Has Taken Steps to Address Challenges but Needs a Comprehensive Strategic Plan
Published: 2015-12-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: IT systems are critical to FDA's ability to achieve its mission. GAO previously reported on limitations in a number of FDA's key IT areas, including data availability and quality, information infrastructure, the ability to use technology to improve regulatory effectiveness, and investment management. GAO recommended FDA take actions to address these limitations, including the development of a comprehensive IT strategic plan to provide direction for modernizing the agency's IT environment.
The Food and Drug Administration Safety and Innovation Act of 2012 included a provision for GAO to report on FDA's progress regarding an IT strategic plan and implementation of GAO's prior recommendations. This report provides an assessment of the (1) status of FDA's efforts to develop and implement an IT strategic plan that includes results-oriented goals, activities, milestones, and performance measures; and (2) extent to which FDA has addressed GAO's prior IT-related recommendations.
To do so, GAO assessed the agency's 2015 IT strategic plan against best practices for IT management. GAO also reviewed supporting documents regarding FDA's actions on prior recommendations.
section_title: What GAO Found
paragraphs: As of September 2015, the Food and Drug Administration (FDA), an agency within the Department of Health and Human Services (HHS), had developed and released a new information technology (IT) strategic plan, entitled Information Technology Strategic Plan, Version 1.0 . The plan, according to the agency's Chief Information Officer (CIO), was developed to help FDA's Office of Information Management and Technology (OIMT) address business challenges facing the agency through the implementation of IT. The plan describes the current state of the agency's IT environment, along with OIMT's mission, vision, and the objectives of three strategic themes—quality service, security, and efficiency. The plan also defines performance measures and initiatives intended to support the office's strategic themes.
Nevertheless, the plan lacks key elements that GAO previously recommended be included in a comprehensive strategy to align with the agency-wide mission and goals, and allow the plan to be used for managing IT investments to more effectively address business challenges. For example, FDA's IT strategic plan does not align with strategic priorities and goals that the agency defined for 2014 through 2018. Further, it does not identify results-oriented goals and performance measures and milestones, or targets for measuring the extent to which outcomes of IT initiatives support FDA's ability to achieve agency-wide goals and objectives; strategies that the governing IT organization will use to support agency-wide goals and objectives; and key IT initiatives and interdependencies to be managed. The agency's CIO stated that this version of the strategic plan was developed to address challenges related to processes, technologies, roles, functions, and capabilities for improving the operations of OIMT, which has the responsibility for managing IT. However, FDA has not yet defined schedules or milestones for managing and completing the development and implementation of future versions of the plan that would reflect actions intended to address the agency-wide mission and goals. Until FDA incorporates these key elements of comprehensive IT strategic planning into its plan and fully implements the plan, it will lack critical information needed to align information resources with business strategies and investment decisions, and be hindered in determining whether outcomes of its IT initiatives are succeeding in supporting agency-wide goals.
FDA has made progress in implementing GAO's prior IT-related recommendations. Although it has not yet developed a comprehensive IT strategic plan, the agency has improved enterprise architecture development and IT human capital planning by implementing four of nine prior recommendations. FDA implemented two recommendations to develop an IT systems inventory that can be used to help manage IT investments and to improve information-sharing capabilities of one of its centers, and took steps toward implementing the remaining two recommendations related to improvements in scheduling and monitoring progress of a key IT modernization initiative. However, the agency did not complete all actions needed to implement these two recommendations. Specifically, it did not develop project schedules or conduct IT project monitoring in accordance with best practices. FDA's continued efforts to implement the remaining recommendations are critical to assuring that the agency's ability to manage IT investments and resources will meet its overall mission and goals.
section_title: What GAO Recommends
paragraphs: GAO recommends that FDA define schedules and milestones for incorporating into its IT strategic plan elements that align with the agency's mission and business strategies, and fully implement the plan. HHS agreed with the recommendations.

section_title: Letter
paragraphs: Within the Department of Health and Human Services, the Food and Drug  Administration (FDA) is responsible for ensuring the safety of a wide  range of consumer products, including approximately 80 percent of our  nation’s food supply. Information technology (IT) systems are critical to  FDA’s ability to conduct activities to achieve its mission, including product  review, adverse event reporting, and compliance activities, among others.
We have previously noted limitations in a number of the agency’s key IT  areas, including data availability and quality, information infrastructure,  the ability to use technology to improve regulatory effectiveness, and  investment management. In 2009, we reported that FDA lacked guidance  and direction to address such limitations, which could be provided by  having a comprehensive IT strategic plan and by establishing key IT  management capabilities. We subsequently reported in 2012 that the  agency continued to lack IT management capabilities needed to support  and guide its modernization initiatives. The two reports, collectively,  contained nine recommended actions that FDA could take to address the  challenges it faced, including a recommendation to develop a  comprehensive IT strategic plan that defined guidance and direction for  modernizing the agency’s IT environment.
The Food and Drug Administration Safety and Innovation Act of 2012  included a provision that GAO report on FDA’s progress in developing  and implementing an IT strategic plan, and to report on actions taken by  the agency to address the other IT-related recommendations resulting  from our prior reports in 2009 and 2012. Accordingly, our objectives for  this study were to (1) determine the status of FDA’s efforts to develop and  implement an IT strategic plan that includes results-oriented goals,  activities, milestones, and performance measures; and (2) assess the  extent to which FDA has addressed our prior IT-related  recommendations.
To address the first objective, we analyzed the agency’s Information  Technology Strategic Plan, Version 1.0, dated September 2015, to  determine if it was aligned with strategic goals and priorities of the  agency. We assessed the contents of the plan by comparing them to best  practices for IT strategic planning that we have previously identified. We  supplemented the information we received by interviewing FDA officials  who were involved in the development of the 2015 IT strategic plan,  including the agency’s Chief Information Officer (CIO).
For the second objective, we assessed FDA’s actions taken in response  to the IT-related recommendations that we made in our reports published  in 2009 and 2012. To do so, we reviewed the agency’s internal tracking  reports and documents related to each recommendation and summarized  the actions taken to implement each recommendation. We collected and  analyzed documentation related to IT investment management by, for  example, comparing the contents of FDA’s system inventory with other  supporting documentation, such as the agency’s IT portfolio summary. In  addition, we assessed FDA’s efforts to develop a schedule for one if its  key modernization projects by comparing contents of the schedule to best  practices defined in our schedule assessment guide. We supplemented  the information we received by interviewing FDA officials who were  involved in implementing practices to address the recommendations we  made in prior reports.
We conducted this performance audit from March 2015 to December  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives. Further discussion of our  objectives, scope, and methodology can be found in appendix I.
subsections: section_title: Background
paragraphs: Within the U.S. Department of Health and Human Services, FDA is  responsible for the protection of public health by ensuring the safety,  efficacy, and security of human and veterinary drugs, biological products,  and medical devices; and ensuring the safety and security of our nation’s  food supply, cosmetics, and products that emit radiation. The agency is  also responsible for ensuring the proper labeling of foods, drugs, medical  devices, tobacco, and cosmetics. Further, its work includes advancing  public health by facilitating innovations and promoting public access to  science-based information on medicines, devices, and foods.
As stated in FDA’s agency-wide strategy for fiscal years 2014 through  2018, FDA Strategic Priorities, four goals support five organizational  cross-cutting priorities—regulatory science, globalization, safety and  quality, smart regulation, and stewardship. The four supporting goals are  to enhance oversight of FDA regulated products, improve and safeguard access to FDA-regulated products to benefit  health, promote better informed decisions about the use of FDA-regulated  products, and  strengthen organizational excellence and accountability.
FDA exercises its core functions through four directorates: the Offices of  Medical Products and Tobacco; Foods and Veterinary Medicine; Global  Regulatory Operations and Policy; and Operations. These offices, along  with the Chief Scientist, report to the FDA Commissioner and carry out  their missions through seven centers and through FDA’s Office of  Regulatory Affairs.
Office of Medical Products and Tobacco:
Center for Biologics Evaluation and Research. Regulates and  evaluates the safety and effectiveness of biological products, such as  blood and blood products, vaccines and allergenic products, and cells,  tissues, and gene therapy products.
Center for Drug Evaluation and Research. Promotes and protects the  public health by ensuring that all prescription and over-the-counter  drugs are safe, as well as by reviewing and regulating clinical  research.
Center for Devices and Radiological Health. Promotes and protects  the public health by ensuring the safety and effectiveness of medical  devices and preventing unnecessary human exposure to radiation  from radiation-emitting products.
Center for Tobacco Products. Educates the public on the dangers of  tobacco use; develops the science needed for tobacco regulation; and  develops and enforces regulations on the manufacture, marketing,  and distribution of tobacco products.
Office of Foods and Veterinary Medicine:
Center for Food Safety and Applied Nutrition. In conjunction with  FDA’s field staff, promotes and protects the public health, in part by  ensuring the safety of the food supply and that food products are  properly labeled, and ensuring that cosmetics are safe and properly  labeled.
Center for Veterinary Medicine. Protects and promotes the health of  humans and animals by ensuring the safety and effectiveness of  animal drugs, as well as the safety of animal food and devices.
Office of the Commissioner:
National Center for Toxicological Research. Conducts peer-reviewed  scientific research and provides expert technical advice and training to  support FDA’s science-based regulatory decisions.
Office of Global Regulatory Operations and Policy:
Office of Regulatory Affairs. Leads FDA field activities and provides  FDA leadership on imports, inspections, and enforcement policy. This  office supports the FDA product centers by inspecting regulated  products and manufacturers, conducting sample analysis on regulated  products, and reviewing imported products offered for entry into the  United States. The office also develops FDA-wide policy on  compliance and enforcement and executes FDA’s Import Strategy  and Food Protection Plans.
Office of Operations. Ensures the timely and effective delivery of high  quality and cost effective mission support services, including IT  support, across the FDA and its centers, and coordinates emergency  preparedness and response activities for incidents involving FDA- regulated products across FDA and its stakeholders.
subsections: section_title: FDA’s Reliance on IT
paragraphs: FDA relies extensively on the use and outcomes of IT to achieve its  strategic priorities, fulfill its mission, and support related administrative  needs. Among others, the agency has implemented systems and other  technology dedicated to supporting activities, such as:
Reviewing and evaluating new product applications, such as for  prescription drugs, medical devices, and food additives. These  systems are intended to help FDA determine whether a product is  safe before it enters the market. One such system—the Document  Archiving, Reporting, and Regulatory Tracking System—is intended to  manage the drug and therapeutics review process.
Monitoring the safety of products on the market by collecting and  assessing adverse reactions to FDA-regulated products, such as  illnesses due to food or negative reactions to drugs. This activity is  supported by systems such as the Vaccine Adverse Event Reporting  System, which accepts reports of adverse events that may be  associated with U.S.-licensed vaccines from health care providers,  manufacturers, and the public.
Screening imported food to detect and prevent entry into the United  States of adulterated, misbranded, or potentially spoiled food. For  example, the Predictive Risk-based Evaluation for Dynamic Import  Compliance Targeting is a tool used as part of the system that  provides entry reviewers with information to help target high-risk  shipments for examination and expedite the clearance of lower-risk  cargo.
Surveilling and promoting the safety of FDA-regulated products. With  the exception of laboratories, systems developed and managed by the  Mission Accomplishments and Regulatory Compliance Services  (MARCS) investment are used to support all field activities, including  domestic and foreign inspections, imports, compliance, and  enforcement activities.
Conducting post-market safety surveillance for drug and therapeutic  biologic products that have been approved by FDA. The FDA Adverse  Event Reporting System is used to help identify new safety concerns  that might be related to a marketed product, evaluate a  manufacturer’s compliance with reporting regulations, and respond to  outside requests for information.
In addition, FDA relies on various systems that support its administrative  processes, such as payroll administration and personnel systems. All of  the agency’s systems are supported by an IT infrastructure that includes  network components, critical servers, and multiple data centers.
FDA’s budget for the development, operations, and maintenance of its IT  investments was approximately $419 million for fiscal year 2013 and  approximately $566 million for fiscal year 2014. IT spending for fiscal year  2014 was categorized into three portfolios: $287.4 million for IT investments that support mission delivery and $260 million for IT infrastructure, security, office automation, and $18.72 million for investments in enterprise architecture capital  planning and chief information officer functions.
The agency received about $578 million in IT funding for fiscal year 2015.
section_title: FDA IT Governance
paragraphs: Within FDA, overall responsibility for managing IT resides in the Office of  Information Management and Technology (OIMT), which was established  within the Office of Operations as part of an agency-wide reorganization  in January 2014. OIMT is headed by the agency’s Chief Operating  Officer, and its mission is to ensure the timely and effective delivery of  high quality, innovative and cost-effective mission- related information  technology support services across the FDA and its centers. The office’s  major functions include information security, business and customer  assurance, technology, and informatics and technology innovation.
The Office of Information Management (OIM) is a component of OIMT  and is led by the CIO. OIM’s mission includes, among other things,  setting goals for improving customer focus and developing new IT  solutions that support changing business needs. Within OIM, and under  the direction of the Chief Operating Officer, the CIO is responsible for  managing and overseeing the design and development of IT systems to  support FDA’s mission, and creating a foundation to enhance the  interoperability of the agency’s systems. As such, this official is charged  with providing services and support to approximately 20,000 federal  personnel and contractors throughout the United States and the  international FDA community.
section_title: Our Prior Work Has Included Recommendations to Address FDA’s Challenges with IT Strategic Planning and Management
paragraphs: In previously reporting on FDA’s IT challenges, we highlighted  deficiencies in a number of the agency’s management processes and  system modernization efforts. Accordingly, our two reports—issued in  June 2009 and March 2012—included a total of nine recommendations  aimed at improving key IT areas, such as IT strategic planning, enterprise  architecture development, human capital planning, investment  management, information sharing, and project management. (We provide  a detailed discussion of the agency’s actions to address the  recommendations later in this report.)
Specifically, in 2009 we reported that FDA did not have a comprehensive  IT strategic plan to coordinate and manage the numerous modernization  projects that it was pursuing in response to federal law and guidance and  the agency’s urgent mission needs. We noted that the agency had  developed planning documents that included some, but not all of the  elements of a comprehensive plan, and stressed that, lacking such a  plan, the agency’s ability to meet its strategic goals was at risk. Thus, we  recommended that the Commissioner of FDA require the CIO to set  milestones and a completion date for developing a comprehensive IT  strategic plan that included results-oriented goals, strategies, milestones,  performance measures, and an analysis of interdependencies among  projects and activities. We further recommended that the agency use  such a plan to guide and coordinate its modernization projects and  activities.
Our 2009 report also noted that FDA had made mixed progress in  establishing important IT management capabilities that are essential for  helping to ensure a successful modernization, particularly in the areas of  enterprise architecture and related segment development, and in IT  human capital planning and management. We recommended additional  actions to be taken by the CIO to improve these areas, which we discuss  in detail later in this report. The Department of Health and Human  Services, in commenting on our 2009 report, agreed with our  recommendations and identified actions FDA had initiated or planned to  address them.
We again reported on the agency’s IT management capabilities in 2012,  noting that FDA lacked a comprehensive IT inventory—a tool for  managing its investments that identifies and provides key information  about the systems it uses and is developing. We also noted that much  work remained to be done on the agency’s largest and most costly  system modernization effort— the MARCS program—and that FDA’s  Center for Food Safety and Applied Nutrition (CFSAN) had not assessed  information sharing needs and capabilities. We recommended that FDA  take actions regarding these issues, which we discuss in detail later in  this report. Further, we pointed out that FDA still did not have an  actionable IT strategic plan that identified specific goals and  corresponding tasks to guide its overall modernization efforts and support  agency-wide goals, as we had recommended in our earlier report. In  commenting on our 2012 report, the Department of Health and Human  Services neither agreed nor disagreed with the recommendations, but  stated that FDA was taking actions to address many of the issues we  identified.
section_title: Changes in FDA’s IT Management Structure and Strategy
paragraphs: Our prior work stressed that one of the reasons FDA had not established  key management capabilities for the areas we found to be deficient was  because the agency’s IT management structure had been in flux.  Specifically, the agency had appointed five CIOs since 2008—each with  differing priorities for managing IT, including the modernization of the  agency’s infrastructure. Two of those CIOs served in an acting capacity  during 2011, with the agency hiring a permanent CIO in October 2011.  However, that official left FDA in March 2013, and the agency’s Chief  Operating Officer took on additional responsibilities as an acting CIO until  May 2015, when the agency hired its current CIO.
Subsequent to our 2012 report, FDA’s CIO that was hired in October  2011 led the development of an IT strategic plan that was to be used to  support the agency’s mission and goals. The plan, released in September  2012 and entitled Information Management Strategic Plan, Version 1.1,  defined goals and objectives to be addressed from fiscal year 2012  through fiscal year 2016. The documented intent of the plan was to set  the path forward for modernization of the agency’s IT infrastructure to  more effectively support the agency’s mission, goals, and objectives. The  plan included four goals and related objectives, major activities to fulfill  these objectives and to be implemented within specified time frames, and  performance indicators for determining whether outcomes resulting from  the implementation of the plan were effective toward achieving goals.  While FDA implemented some of the activities from 2012 through 2015,  according to the CIO and OIM officials who participated in its  development and implementation, the plan was not used, as intended, to  manage and measure outcomes of activities to support the agency’s  mission, goals, and objectives. OIMT officials told us that the plan was not  used because the CIO who was present when the plan was finalized left  the agency and the Chief Operating Officer, who resumed responsibilities  of an acting CIO, managed IT from an operational perspective. According  to these officials, the Chief Operating Officer worked with the agency’s IT  organization to identify and incorporate key initiatives of the Information  Management Strategic Plan into the Office of Operations’ strategic  planning activities.
section_title: FDA Has Developed a New IT Strategic Plan, but It Lacks Key Elements
paragraphs: To be effective as a guide for managing IT investments and better  address agency-wide business challenges, best practices show that an IT  strategic plan should define the agency’s vision and provide a road map  to help align information resources with business strategies and  investment decisions. Accordingly, best practices call for a  comprehensive and effective plan to (1) be aligned with the agency’s  overall strategy; (2) identify the mission of the agency, results-oriented  goals, and performance measures that permit the agency to determine  whether implementation of the plan is succeeding; (3) include strategies  the governing IT organization will use to achieve desired results; and (4)  provide descriptions of interdependencies within and across projects so  that they can be understood and managed.
FDA’s new CIO, who was hired in May 2015, has taken steps to develop  a new IT strategic plan. The CIO has stated that the intent of the plan,  entitled Information Technology Strategic Plan, Version 1.0 and dated  September 2015, is to allow OIMT to address business challenges facing  the agency. The plan describes the current state and customer perception  of the IT environment, along with OIMT’s mission, vision, and objectives  of three strategic themes—quality service, security, and efficiency.
To support the office’s strategic themes, the CIO defined in the plan  objectives, performance measures, and 15 initiatives. For example, the  plan defines objectives for supporting stakeholders, such as “improve  customer satisfaction” and “increase security,” as well as performance  measures and targets for meeting the objectives. Specifically, a target  and performance measure for “improve customer satisfaction” is that at  least 80 percent of help desk requests are to be resolved on the first call.  The plan also defines 15 categories of initiatives that are intended to  support OIMT’s efforts to meet the objectives. These categories of  initiatives are to address areas such as business continuity, contract and  project management, customer service, IT security, infrastructure, and  application development.
The 2015 IT strategic plan also includes a table of specific milestone  activities that are mapped to OIMT’s categories of initiatives, and  identifies responsible parties and target dates for completing them. For  example, information included in the table describes a milestone to  “improve knowledge and use of telework tools” that is to support the  business continuity initiative. It also defines a target date for  accomplishing the milestone, along with a responsible entity within FDA.  The table also provides information about the status of the milestones’  implementation.
Nevertheless, in its current state, the document that FDA has developed  does not include the key elements of a comprehensive IT strategic plan.  First, the new plan does not align OIMT’s objectives with the agency-wide  strategies and goals that were defined by the Commissioner in FDA  Strategic Priorities: 2014-2018. Specifically, the plan identifies objective  areas aimed at accomplishing OIMT’s mission that include, among other  things, information security, training of staff, and project management.  However, the plan does not address or align with the agency’s five  strategic priorities—regulatory science, globalization, safety and quality,  smart regulation, and stewardship—and the goals for achieving them:  enhance oversight of FDA regulated products, improve and safeguard  access to FDA-regulated products to benefit health, promote better  informed decisions about the use of FDA-regulated products, and  strengthen organizational excellence and accountability.
Further, the plan does not identify goals and performance measures for  determining whether its implementation is successful in supporting the  agency’s mission. Rather than emphasizing the agency-wide mission to  protect public health, the plan reflects an office-level (OIMT) mission to  “provide effective and fiscally responsible technology services in a  manner that promotes high standards.” Thus, the performance measures  and milestones for achieving goals were defined to be used to determine  the success of initiatives such as applications development, staffing, and  training. In addition, because the plan does not align with agency-wide  goals and mission performance, it cannot provide information the agency  needs to define appropriate performance measures and mechanisms for  monitoring the outcomes of any IT objectives or initiatives to determine  whether they contribute to FDA’s progress toward meeting its mission  goals.
The 2015 IT strategic plan also does not identify initiatives that can be  aligned with agency-wide strategies described in FDA’s strategic priorities  document. Rather, it identifies specific IT initiatives that can be linked to  OIMT’s mission, such as contract and project management, customer  service, IT security, infrastructure, and application development. For  example, the IT strategic plan includes a “cybersecurity” initiative and  identifies activities for securing internal and external data. While this  initiative could help support one of the strategies for achieving the fourth  goal of the agency-wide plan—secure mission critical and sensitive  assets and information—the IT strategic plan does not link this initiative to  the agency-wide strategy.
Finally, the plan does not identify interdependencies among planned and  established IT initiatives. Further, it does not identify specific initiatives  that support strategies and goals for fulfilling the agency’s public health  mission. For example, the agency-wide plan identifies a strategic goal to  enhance oversight of FDA-regulated products. However, the IT strategic  plan does not identify any planned OIMT initiatives or ongoing  modernization projects that could be aligned with this goal, such as the  MARCS program—a collection of subprojects intended to enhance  existing applications and develop new systems in support of the  surveillance and safety of FDA-regulated products. Given the limitations  noted, FDA’s current IT strategic plan is not a comprehensive tool for  providing its officials the information they need to determine and manage  interdependencies across IT projects throughout the agency.
In discussing our concerns, the CIO stated that the intent of the first  version of the September 2015 IT strategic plan was to identify  improvements, based on an assessment of FDA’s IT environment, that  needed to be made to existing IT processes, technologies, roles,  functions, and capabilities within OIMT. In October 2015, the CIO  documented a process for maintaining milestones in the plan on a  monthly basis, and for adding initiatives and milestones to updated  versions that the agency intends to release on an annual basis. However,  according to OIM officials and the CIO, they have not yet established or  documented schedules or milestones for managing and completing the  development and implementation of the next version of the IT strategic  plan. As such, it is not known if and when FDA will have a comprehensive  plan that can be implemented to provide the guidance and direction  needed for effectively managing IT initiatives and ensuring that outcomes  of the initiatives help the agency meet its overall mission and goals.
Until FDA incorporates into its IT strategic plan necessary elements that  align the goals and objectives with its overall mission, the agency will  continue to lack critical information needed to ensure that information  resources support business strategies and investment decisions. In  addition, without defined performance measures that support the goals of  the agency, along with mechanisms for assessing system outcomes  against them, FDA will be hampered in its ability to determine whether its  IT initiatives are successful in supporting agency-wide goals. Further, until  the plan identifies key IT initiatives and describes interdependencies  among them, FDA will also remain without critical information it needs to  guide and coordinate its modernization projects and other IT initiatives,  and ensure that they are implemented in support of the agency’s strategic  goals.
section_title: FDA Has Made Progress in Implementing Prior Recommendations on IT Planning and Management
paragraphs: FDA has implemented six of the nine recommendations included in the  two reports that we issued in 2009 and 2012. Specifically, the agency  took actions that resulted in the implementation of four of the five  recommendations that we made in 2009, and two of the four  recommendations we made in 2012. As a result of its actions, FDA has  improved its ability to plan and manage the agency’s IT initiatives.  Implementing the remaining recommendations we made regarding  needed improvements in the areas of IT strategic planning, and  scheduling and monitoring the progress of key projects is important for  ensuring that ongoing and planned IT initiatives and modernization  projects are successful in supporting agency-wide goals and business  needs.
subsections: section_title: FDA Implemented Four of Five Recommendations Related to IT Planning
paragraphs: With regard to the recommendations included in our 2009 report, FDA  has taken actions that led to the implementation of four of the five  recommendations for improving IT planning. The lone outstanding  recommendation is to complete a comprehensive IT strategic plan. As  previously discussed, FDA has developed a plan, but it is neither  comprehensive nor linked to the overall agency strategic plan.
Table 1 summarizes the status of FDA’s actions to implement the five  recommendations.
We reported in 2009 that FDA had significant work remaining with regard  to its enterprise architecture development, including developing a  program management plan for ensuring that an enterprise architecture is  effectively and efficiently developed. We noted that, without an enterprise  architecture, FDA did not have the assurance it needed to enable  effective modernization of its IT environment. Accordingly, we  recommended that the agency develop an enterprise architecture  program management plan that includes a detailed work breakdown of  the tasks, activities, and time frames associated with developing the  architecture, as well as the funding and staff resources needed.
FDA has implemented this recommendation. In particular, FDA completed  its Enterprise Architecture Program Management Plan, dated July 2012,  that outlines the agency’s key enterprise architecture initiatives and  program management functions, including a work breakdown of the major  tasks, activities, time frames, and key staff responsible for developing the  enterprise architecture. The plan’s key initiatives include adopting  enterprise architecture processes and a management approach,  developing the architecture, and developing and implementing an  approach for operating and maintaining the architecture.
The agency also included in its Segment Architecture Funding and  Resource Plan, dated September 2013, an estimate of $3.6 million for  funding needed to accomplish enterprise architecture activities. The plan  called for the establishment of a Program Support Office made up of OIM  staff and business stakeholders from FDA centers. The office and its staff  were to be responsible for developing and managing the enterprise  architecture. As a result, FDA should be better positioned to manage its  enterprise architecture development and ensure the effectiveness of its IT  modernization efforts.
subsections: section_title: Agency Officials Defined Criteria for Prioritizing Segment Architectures
paragraphs: In 2009, we also reported that FDA had begun building the segments of  an enterprise architecture before it had established priorities for doing so.  However, the Federal Enterprise Architecture Practice Guidance states  that prioritizing segments, based on business needs, should precede  building them. Accordingly, we recommended that FDA define criteria for  setting priorities for the segment architecture and then prioritize building  of the segments accordingly.
In response, FDA identified relevant segment scoring criteria for use in  setting priorities for the architecture segments. Further, as of September  2013, the agency had completed architecture development for its two  highest priority segments—regulatory science and post-market safety and  surveillance—and planned to prioritize remaining segments, such as  scientific computing. These actions better positioned the agency to  develop its segment architectures based on priorities, and reduce risks  that its systems modernization efforts may not be successful in supporting  FDA’s business needs.
section_title: FDA Accelerated the Development of the Enterprise Architecture and Managed Interim Risks
paragraphs: Our report also noted that the agency’s major modernization projects  were proceeding without the guidance and constraint of an enterprise or  segment architecture. Accordingly, we recommended that FDA accelerate  development of the enterprise architecture, including “as is” and “to be”  architectures and transition plans, and, in the meantime, develop plans to  manage the increased risks to modernization projects of proceeding  without an architecture to guide and constrain their development.
Consistent with this recommendation, FDA took steps to accelerate the  development of its segment and enterprise architectures. For example, in  fiscal year 2013, the agency developed “as is” and “to be” enterprise  architectures in terms of business and applications, and proposed  solution architectures for certain segments, including Adverse Events  Analysis and Reporting and Regulatory Science. Further, FDA reported in  its July 2012 Enterprise Architecture Program Management Plan that it  had plans to complete the development of its enterprise architecture and  transition plan and, in an update to the plan, stated that it intended to  complete the segment architectures.
In conjunction with these efforts, FDA took steps to manage the increased  risks of proceeding without the guidance that would be provided by a  complete architecture. For example, in August 2013 it formed an  Engineering Review Board to discuss and reach mutual agreement on  architectural solutions for modernization projects that promote  interoperability among systems. Further, the FDA centers jointly  conducted reviews of solutions already in progress to identify IT  capabilities that could be reused to avoid duplication. As a result of its  actions, the agency should be better positioned to proceed with  modernization projects and reduce risks of introducing inefficiencies, such  as the lack of interoperability and duplication of efforts, that could occur  without a complete enterprise architecture.
section_title: The Agency Addressed IT Human Capital Planning
paragraphs: In our 2009 report, we also noted that FDA had not inventoried the skills  of its current IT workforce, determined present and future skills needs, or  analyzed gaps—steps it needed to take to determine critical skills and  competencies for achieving IT program results. Accordingly, we  recommended that the agency develop a skills inventory, needs  assessment, and gap analysis, and develop initiatives to address any  identified gaps as part of a strategic approach to IT human capital  planning.
Consistent with this recommendation, OIM officials implemented a  strategic approach to IT human capital planning. Specifically, in  September 2013, the office identified critical skill shortages, including  insufficient cybersecurity skills. It also concluded that network,  engineering, and infrastructure operations skills were outdated. In  addition, OIM made plans to use a tool that allowed it to collect skills  information about its staff on an ongoing basis, and conducted a skills  gaps analysis, which identified gaps relating to eight job descriptions in  two of its office components. FDA officials also took actions to address  the skills gaps, including offering project management training and hiring  staff, such as database administrators. As a result of its actions, the  agency made important progress toward a strategic approach for  managing IT human capital to more effectively achieve program results.
section_title: FDA Implemented Two of Four Recommendations Regarding IT Management Challenges
paragraphs: FDA has implemented two of the four recommendations we made in 2012  regarding challenges in key areas of IT management. Specifically, the  agency developed a systems inventory and improved the sharing of  information within CFSAN. However, it has taken some actions, but has  not yet fully implemented two recommendations regarding the  development and use of schedules for managing and monitoring the  progress of IT initiatives.
Table 2 summarizes the status of the four recommendations, and a  discussion of each recommendation follows the table.
Our 2012 report noted that FDA did not have a complete list of IT systems  that it was using or developing. We recommended that the agency  develop an inventory that includes key information for each of its systems.
In response, FDA took actions to implement our recommendation.  Specifically, in May 2015, FDA developed an inventory that identified and  described 205 systems that were reported in the agency’s fiscal year  2016 budget estimates for IT systems to the Office of Management and  Budget. The inventory includes, for each system, data such as the  system’s name; description; annual costs of development, maintenance,  and operations; effective date the system became operational; and a  unique identifier that links each system to a corresponding investment  reported to the Office of Management and Budget. Further, in June 2015,  the agency issued a draft procedure for maintaining the inventory, which it  intends to follow to help ensure that the inventory is consistently updated.
In addition, FDA incorporated use of the system inventory into the  agency’s IT investment management process. For example, FDA  identified for each system the business capabilities that they support,  such as strategic planning, emergency response, and product review and  approval. These data enable the agency to organize systems and  investments into portfolios according to business capabilities and to view  systems as they relate to other systems that perform similar functions,  rather than as unrelated IT systems. This portfolio view is intended to be  used to help FDA better manage its IT investments as it allows agency  officials to identify any gaps in its ability to support specific business  capabilities that may arise after a system is retired. According to OIM  officials, when such gaps are identified, the agency may then identify and  appropriately fund IT investments that are needed to fill the gaps and  ensure continued support of the agency’s mission.
section_title: FDA Assessed Information-sharing Needs and Capabilities for CFSAN
paragraphs: We reported in 2012 that, although we had previously identified  deficiencies in CFSAN’s ability to effectively share critical information,  such as information on recalls of contaminated food, FDA had not  performed a comprehensive review to identify opportunities for improving  data sharing within the center. We therefore recommended that the  agency perform a full assessment of the center’s data-sharing needs and  capabilities, and develop a plan to address such needs and provide any  additional capabilities.
In response, FDA implemented our recommendation. Specifically, FDA  assessed and documented CFSAN’s information-sharing needs and  capabilities. The resulting document, entitled Office of Foods and  Veterinary Medicine Data Management Strategy and Roadmap (OFVM  Roadmap), identifies IT shortfalls that contribute to information-sharing  issues, such as the lack of integration between systems, and cases when  systems do not store data in a central warehouse that could be used to  share information. The document includes plans, with milestones and  dates for addressing the center’s information-sharing needs, such as  access to data needed to conduct food facilities and import inspections.
The roadmap also describes an existing capability for data sharing, the  Online Reporting and Analysis Decision Support System (ORADSS) Data  Warehouse. It noted that the warehouse was being used to store  significant amounts of data used by CFSAN, such as data provided by  FDA’s Office of Regulatory Affairs, but that most data from the center’s  systems were not stored in a data warehouse. Specifically, the roadmap  identified 17 systems that were assessed as “likely candidates” for  integration with ORADSS, but did not store data in the warehouse. The  roadmap also identified systems from other centers that could share data  via the warehouse, and further stated that incorporating data from these  systems into the warehouse would support data sharing by allowing  access to information from one location, thereby improving collaboration  within and between FDA’s centers.
In addition, consistent with plans described in the OFVM Roadmap, the  agency documented a plan for sharing scientific computing data among  CFSAN and its partners. This plan is focused on systems for sharing  genomics data that are used to identify and track bacteria that  contaminate food products. OFVM officials determined that scientific data  should be maintained and stored separately from other data because,  unlike the transactional data that are structured and can be stored in a  data warehouse, scientific computing data are less structured and can be  more voluminous, and require additional computational and data transfer  capabilities. To address the need for additional data storage and sharing  capabilities for these data, the agency established two scientific  databases—the Bionumerics SQL and the Next Generation Sequencing  databases.
By taking these actions, FDA has completed important steps toward  developing an IT environment in which data from multiple systems and  databases can be more efficiently stored and accessed.
subsections: section_title: FDA Has Begun but Has Not Completed Efforts to Develop an IMS for MARCS
paragraphs: In 2012 we noted that considerable work remained to be completed on  the MARCS program, one of FDA’s largest and most costly IT  modernization efforts. We recommended that the agency develop an IMS  to assist in managing and monitoring projects that are part of the MARCS  program. We stressed that such a schedule should (1) identify which  legacy systems will be replaced and when, (2) identify all current and  future tasks to be performed by contractors and FDA, (3) define and  incorporate information reflecting resources, and (4) define critical  dependencies between tasks.
FDA took a number of steps to implement the recommendation, but there  are shortfalls in its approach for identifying tasks and resources. For  example, FDA developed an IMS that identified legacy systems and  included associated tasks and time frames for updating or maintaining the  systems from fiscal year 2014 through fiscal year 2018. In addition, the  agency identified detailed tasks to be completed in fiscal years 2014 and  2015, and also included in the schedule some summary-level tasks to be  completed in future years, such as those for maintaining legacy systems.
However, we could not determine if the tasks in the schedule included all  tasks that needed to be completed to implement the MARCS projects. We  could not make this determination because the agency had not  documented the information that it needed to identify all tasks that should  be included in the schedule—i.e., it had not developed a work breakdown  structure that could be used to identify all tasks needed to be completed  for the program.
In addition, the schedule did not identify all resources needed to complete  certain tasks—i.e., it did not identify specific FDA staff or the skills needed  to complete tasks. Instead, the schedule identified responsible entities  within the agency, thus increasing the risk that needed staff and skills  could be assigned to another task and, therefore, would not be available  to work on others.
Lastly, we found errors in the way dependencies between tasks were  defined. Specifically, the MARCS IMS did not identify external  dependencies between activities, such as projects to be completed by  contractors, and dependencies that were identified between tasks  contained errors. For example, there are a significant number of tasks in  the schedule for which there are no identified dependencies, and some  remaining tasks have restrictions on their start date that are not justified— i.e., the restrictions are not based on the start or finish of an activity on  which the tasks are dependent.
Agency officials from the Office of Regulatory Affairs and the MARCS  program said that the IMS identifies detailed tasks for the current fiscal  year because FDA develops its schedule based on work that is to be  accomplished during that year. According to the officials, this is done  because projects are funded on an annual basis; therefore, they cannot  know what projects will be conducted in future years. The officials further  explained that the IMS is updated at least annually based on changes in  business priorities and funding availability, and as the planning horizon  advances. While this approach can be taken for developing schedules,  according to best practices defined in our schedule assessment guide,  all tasks needed to complete the program should be identified within a  work breakdown structure to ensure that the total scope of work is  accounted for within the schedule.
The officials also noted that they do not identify specific FDA staff or  estimate the percentage of time required of those resources because they  assume that the responsible entities will dedicate one staff person to work  full time on each project. However, a schedule that does not identify  specific resources assumes unlimited availability and, thus, the risk of the  program’s schedule slipping significantly increases if needed staff or skills  are not available when needed.
In addition, officials said that, while they identify some dependencies  between contractors’ projects, they do not identify all of them because  some of the projects are not tightly integrated. The program officials  further stated that the objectives and scope of the MARCS program today  are much different from those at the time we recommended the  development of an IMS, and that the program does not currently have the  cross-task dependencies that existed in the prior scope that would be  addressed by such an integrated schedule. However, the identification of  interdependencies between all activities is important to allow program  officials to recognize delays that could occur when activities that slip early  in the schedule impact concurrent or future activities.
Lacking certainty that the IMS includes all short- and long-term tasks,  reliable estimates of resources, and dependencies between individual  tasks and contractors’ activities, the value of the schedule as a critical  management tool is diminished—i.e., it cannot be used to accurately  gauge the level of effort required to implement the MARCS program,  determine whether staff will be available when needed, or predict the  effect of factors such as delayed contractor activities on the schedule.  Thus, the MARCS schedule cannot be used as an effective tool for  managing and monitoring the progress of the project.
section_title: FDA Has Not Yet Used the IMS to Monitor Progress of MARCS
paragraphs: In our 2012 report, we also noted that the agency lacked an IMS for  MARCS that provided a summary of progress on lower-level tasks and  effects of changes to lower-level schedules and tasks on the overall  project. Thus, we reported that the agency did not have the information it  needed to gauge the progress of the entire project. We recommended  that, in addition to developing a complete IMS, agency officials use it to  monitor progress of the MARCS program.
Agency officials have implemented certain activities that could help them  use the IMS to monitor the progress of MARCS; however, they have not  developed the schedule to include all the information needed to do so.  First, agency officials began to conduct monthly updates that, for  example, reflected actual progress from individual system subproject  schedules. In addition, the program office developed a baseline schedule  that could be used for measuring and monitoring project performance and  progress. These actions are important steps to be taken toward  addressing our recommendation that agency officials use the MARCS  IMS to monitor the program’s progress.
Yet, because of deficiencies in the agency’s practices for conducting  these activities, along with other deficiencies noted previously, the  usefulness of the schedule for monitoring the program is limited. For  example, although program officials update the IMS on a monthly basis, it  includes activities with projected start or finish dates that are prior to the  date the schedule was updated. However, according to best practices  defined in our schedule assessment guide, projected start and finish  dates should never be prior to the date the schedule was updated; that is,  dates cannot be projected to start in the past. Agency officials within  OIMT stated that they manually update the schedule each month, and do  not use a tool for automated updating. As a result, the updates often are  not accurate or complete, which could account for projected dates for  activities not being modified during the monthly updates.
MARCS program officials also developed a baseline schedule that is  aligned to the MARCS IMS, and is intended to be used to monitor the  progress of the program. For example, data in a baseline schedule  represent important information that program management could use to  assess the validity of the program’s initial duration estimates and the  program’s progress toward beginning and finishing work on time and  according to plan—i.e., trend data. While program officials stated that  they planned to analyze trend data provided by the baseline schedule at  the end of fiscal year 2015, and use the results of the analysis to manage  and monitor the progress of MARCS, they have not yet done so.
While FDA officials have taken some steps to use the IMS and a baseline  schedule to monitor the progress of the MARCS implementation, they are  doing so with inaccurate or incomplete information. Therefore, the agency  may not have necessary information to monitor and make any needed  changes to program plans to ensure that the schedule for implementing  MARCS remains on track and the project is providing results that meet  the agency’s needs.
The fiscal year 2016 business case that was reported to the Office of  Management and Budget indicates that funding for MARCS is planned  through fiscal year 2019. However, officials with the Office of Regulatory  Affairs told us that the agency has chosen to restructure the MARCS  investment to better reflect current business needs and objectives. As  such, it is important that FDA continue to take steps toward implementing  the recommendations that we made in our 2012 report regarding the  agency’s need to develop an IMS that can be used to manage the  implementation of MARCS and ensure its success.
section_title: Conclusions
paragraphs: Although FDA has developed an IT strategic plan, it does not include  elements that best practices reflect are associated with a comprehensive  plan—that is, it is not aligned with agency-wide strategic priorities, and  does not include results-oriented goals and performance measures that  support the agency’s mission, identify key IT initiatives that support  agency-wide goals and objectives, or describe interdependencies among  the initiatives. Until it incorporates these elements into and fully  implements the plan, the agency will continue to lack critical information  needed to align IT initiatives with business strategies and investment  decisions, and to determine whether outcomes of the IT initiatives are  succeeding in supporting the agency-wide mission, goals, and objectives.
In addition, FDA has implemented most of our prior recommendations for  improving IT management practices. Taking further actions to fully  implement the remaining recommendations will improve the agency’s  ability to successfully monitor the progress of one of FDA’s most critical  and costly IT modernization projects throughout its remaining life cycle.
section_title: Recommendations for Executive Action
paragraphs: To help ensure that FDA’s IT strategic planning activities are successful  in supporting the agency’s mission, goals, and objectives, we recommend  that the Commissioner of FDA require the CIO to take the following two  actions: establish schedules and milestones for completing a version of an IT  strategic plan that incorporates elements to align the plan’s strategies  with agency-wide priorities; includes results-oriented goals and  performance measures that support the agency’s mission, along with  targets for measuring the extent to which outcomes of IT initiatives  support FDA’s ability to achieve agency-wide goals and objectives;  identifies key IT initiatives that support the agency’s goals; and  describes interdependencies among the initiatives; and  implement the plan to ensure that expected outcomes of the agency’s  key IT initiatives are achieved.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report, signed by the  Department of Health and Human Services’ Assistant Secretary for  Legislation. In the comments (reprinted in appendix II), the department  stated that it concurred with our recommendations. Further, the  department stated that FDA is committed to continuing the evolution and  implementation of its IT strategic plan and has established a process for  the annual review and updating of the plan. It added that FDA intends to  follow this process to incorporate the alignment of strategic initiatives with  the agency’s mission and business strategies. If effectively implemented,  such actions could help assure that FDA develops a comprehensive plan  that can be used to provide the guidance and direction needed to ensure  that outcomes of key IT initiatives help the agency meet its overall  mission and goals. The Department of Health and Human Services also  provided technical comments, which we incorporated into the report as  appropriate.
We are sending copies of this report to the Secretary of Health and  Human Services, Commissioner of the Food and Drug Administration,  appropriate congressional committees, and other interested parties. This  report is also available at no charge on our website at  http://www.gao.gov.
If you or your staff have any questions on matters discussed in this report,  please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to (1) determine the status of the Food  and Drug Administration’s (FDA) efforts to develop and implement an  information technology (IT) strategic plan that includes results-oriented  goals, activities, milestones, and performance measures; and (2) assess  the extent to which FDA has addressed our prior IT-related  recommendations.
For the first objective, our scope was focused on FDA’s efforts to develop  an IT strategic plan to be used to manage and guide key initiatives,  including modernization projects, in support of agency-wide goals and  objectives. To that end, we assessed FDA’s IT strategic plan, Information  Technology Strategic Plan, Version 1.0, dated September 2015, by  comparing its contents to key practices for IT strategic planning that we  have previously identified. Those practices include developing a plan  that (1) is aligned with the agency’s overall strategy; (2) identifies the  mission of the agency, results-oriented goals, and performance measures  that permit the agency to determine whether implementation of the plan is  succeeding; (3) includes strategies the governing IT organization will use  to achieve desired results; and (4) provides descriptions of  interdependencies within and across projects so that they can be  understood and managed. We compared the goals and objectives in the  IT strategic plan to agency-wide goals and objectives defined in the  September 2014 FDA Strategic Priorities document, which was released  by the Commissioner to establish key strategies for fulfilling FDA’s public  health mission. Finally, we examined the IT strategic plan to identify any  key IT initiatives and modernization projects and determine whether  interdependencies with any such projects were defined.
To address our second objective, we focused on actions FDA took to  implement recommendations that we made in 2009 and 2012 as a  result of studies related to IT management challenges that FDA faced.  Specifically, to determine actions taken by FDA to implement the five  recommendations related to IT planning that we made in our 2009 report,  we collected and examined agency documentation related to its IT  strategic planning, enterprise architecture development, and human  capital planning. Specifically:
For the recommendation to develop a comprehensive IT strategic  plan, we took the various actions described in the preceding  paragraphs that pertained to objective 1.
For the recommendation to develop an enterprise architecture  program management plan, we assessed FDA’s July 2012  Enterprise Architecture Program Management Plan against  criteria defined in GAO’s enterprise architecture management  maturity framework, such as the inclusion of detailed tasks and  time frames necessary to develop the architecture. We also  examined the plan to determine whether it identified the  organizational unit and its staffing that would implement the plan,  and whether it included detailed costs for implementing tasks  described in the plan.
For the recommendation that FDA define criteria for setting  priorities for developing the segment architectures, we analyzed  documentation, such as the FDA Architecture Segment  Architecture, Segment Scoring, that described the agency’s  method for establishing priorities, to determine whether the  approach for prioritizing segments development considered  mission-based criteria. We also reviewed FDA’s prioritized list of  segments and plans for completing segments, documented in its  September 2013 FDA Segment Architecture Prioritization and List  of Segments, to determine the agency’s progress and status of its  effort to develop segments.
To address the recommendation that FDA accelerate  development of the segment and enterprise architectures and  develop plans to manage the increased risk of proceeding without  an architecture, we reviewed documentation on the status of  FDA’s architecture development initiative to determine whether  progress had been made toward developing “as is” and “to be”  architectures. Specifically, we reviewed segment architecture  documentation to identify whether the “as is” architecture included  components such as business processes and applications. We  also reviewed documentation describing the agency’s “to be”  architecture to identify the architectural components and transition  steps that were included in the conceptual view. We also  interviewed FDA officials to determine whether they had planned  an approach to mitigate the risk of developing systems prior to  completing the architecture. Finally, we assessed FDA’s  enterprise architecture governance plan to determine whether the  agency had established an organizational structure and  designated responsibilities that supported a risk mitigation  approach.
For the final recommendation regarding human capital planning,  we assessed actions taken by FDA to implement a strategic  approach to IT human capital planning, including developing a  skills inventory, needs assessment, and gap analysis, and  addressing skills gaps. To do so, we reviewed FDA’s assessment  of the agency’s IT skills and gaps and its assessment of critical  skill shortages. We also reviewed a template that OIM used to  collect and inventory information on employee skills, to verify that  FDA had an approach in place for collecting this information.
To assess FDA’s progress in addressing the four recommendations  regarding IT management from our 2012 report, we collected and  examined documentation that provided information relevant to the actions  the agency had taken. Specifically:
For the recommendation that FDA develop a complete IT systems  inventory, we compared the contents of its inventory to systems  included in the IT portfolio summary that accompanied the  agency’s fiscal year 2016 budget request to the Office of  Management and Budget. We also examined artifacts that  provided evidence of steps taken by FDA during its IT investment  management process, including reports generated from the  systems inventory that were used to align systems with business  capabilities and help to identify versions of technologies to be  retired. We assessed a document that described technologies  used to support various systems, along with schedules for  maintaining them, to identify information from the inventory that  was used to make funding decisions.
To determine whether FDA had taken steps to assess information- sharing needs and capabilities of CFSAN, we reviewed an  information-sharing assessment and roadmap for addressing  business needs of the Office of Foods and Veterinary Medicine.  Through our review, we identified the approach that was used to  assess data-sharing needs, the capabilities that were identified,  and the underlying reasons for information-sharing problems. In  addition, we analyzed sections of the roadmap and compared the  systems FDA was assessing for inclusion in a central data  warehouse against the systems we identified in our prior report.  We also examined FDA’s plan for addressing scientific computing  information-sharing needs of CFSAN by assessing FDA’s Planned  Scientific Data Sharing Capabilities and identifying the  approaches that were planned to address data sharing, the  agencies that were included as data-sharing partners, the types of  information planned for sharing, and the schedule and steps that  were planned to address scientific data-sharing needs.
To assess actions the agency had taken to address our  recommendation that it develop an integrated master schedule  (IMS) for the Mission Accomplishments and Regulatory  Compliance Services (MARCS), we obtained the most current  version of the schedule and determined whether it was complete  by comparing it to best practices defined in our schedule guide.  For each of the practices, we collected and examined  documentation, such as descriptions of projects included in the  IMS and their supporting schedules. In addition, we compared the  tasks identified in the IMS against a list of MARCS legacy systems  that FDA planned to replace, maintain, or update, to determine  whether the schedule identified all legacy systems to be replaced  and time frames for replacing them. We also used a software tool  that analyzes project schedules to determine whether all tasks  had dependencies identified and were sequenced logically, the  number of tasks with date constraints, and whether all tasks had  resources identified to complete the tasks. We inspected the IMS  to determine whether it identified specific staff or skills needed to  complete tasks. Finally, to establish whether the integrated  schedule was aligned with other schedules on which it was  dependent, we compared it to the project’s contractors’ schedules  and looked for evidence of linkages. We also conducted a  structured interview with agency officials responsible for  developing the IMS, based on questions from our schedule guide,  to obtain FDA’s explanation of practices that they were conducting  or were conducting using a different approach than specified in  best practices.
Finally, to determine whether FDA had implemented our  recommendation that it monitor the progress of MARCS using the  IMS, we obtained and examined documents that described the  agency’s approach for updating the schedule to reflect the  progress made toward completing specific tasks. We compared its  approach to best practices in our schedule guide for updating a  schedule to be used for monitoring the progress of projects. We  analyzed the most recently updated MARCS IMS and subproject  schedules, and compared them to earlier versions to identify  evidence of recent updates. In addition, we analyzed the schedule  using a software tool to determine whether individual task dates  were updated, and whether dependencies between tasks were  logical.
For each of the recommendations, we supplemented the information we  collected from examining and analyzing agency documentation by  interviewing FDA officials knowledgeable of actions taken to address the  recommendations.
We conducted this performance audit from March 2015 to December  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusion based on our audit objectives.
section_title: Appendix II: Comments from the Department of Health and Human Services
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Teresa F. Tucker, Assistant  Director; Melina I. Asencio; Elena P. Epps; Rebecca E. Eyler; Jason T.  Lee; Jennifer V. Leotta; Thomas E. Murphy; Edward G. Varty; Daniel K.  Wexler; and Charles E. Youman made key contributions to this report.