Food and Drug Administration: Comprehensive Strategic Planning Needed to Enhance Coordination between Medical Product Centers
Published: 2016-05-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: FDA—an agency within the Department of Health and Human Services (HHS)—has faced challenges in carrying out its responsibilities to ensure the safety and efficacy of medical products sold in the United States. In 2012, Congress required FDA to develop a SIMP for the three centers overseeing medical products that identifies initiatives for improving efficiency, initiatives for workforce development, and measures for assessing the progress of these initiatives. FDA issued the SIMP in July 2013.
GAO was asked to examine FDA's implementation of the SIMP. In this report, GAO (1) evaluates the extent to which the SIMP serves as a strategic planning document, (2) describes the types of plan initiatives, and (3) describes the mechanisms FDA has to evaluate the effectiveness of its plan initiatives. GAO analyzed FDA documents and spoke to FDA officials to assess the SIMP's development and use, along with the implementation status and evaluation mechanisms used for the SIMP's initiatives. GAO also assessed FDA's plan against leading practices for strategic planning. Finally, GAO analyzed FDA workforce data on hiring and attrition for fiscal years 2012 to 2015.
section_title: What GAO Found
paragraphs: The Food and Drug Administration (FDA) developed a strategic integrated management plan (SIMP) for its three centers that oversee medical products (biologics, drugs, and medical devices); however, GAO found that the plan does not incorporate leading practices for strategic planning or document a comprehensive strategy for the centers. FDA officials explained that circumstances at the time of the SIMP's development, including leadership gaps, limited FDA's ability to structure the plan into an effective strategic planning document. While officials said they use a variety of other key documents for strategic planning—such as agency-level and initiative-specific plans—these other plans also do not describe a long-term strategy for addressing key issues that cut across medical product centers. For example, these other FDA documents do not describe the agency's plans for collaboration between the centers that could benefit certain initiatives, improve their decision-making, and improve the quality of evidence and clarity of guidance. FDA officials acknowledged the growing need for strategic planning across the medical product centers to improve center collaboration and address emerging issues. The absence of a comprehensive long-term plan for medical product oversight may hinder FDA's efforts to address emerging issues that require center collaboration, such as access to quality data. Fully documenting such a strategy, either in a separate plan or through existing documents, would help the agency identify measurable goals and objectives for the centers that align with its mission and help communicate its priorities to key stakeholders.
In the SIMP, FDA compiled mostly preexisting initiatives to improve the efficiency of each center's activities and develop its workforce. GAO found that for improving efficiency, FDA selected 30 initiatives that it grouped into three different themes—smarter regulation, process improvement, and business modernization. FDA had fully implemented a third of the initiatives prior to the SIMP's issuance in 2013; another half were implemented by March 2016. As of this date, the remaining initiatives had yet to be fully implemented. For workforce development, FDA included 19 recruitment, retention, and training initiatives, which generally reflected differences in center activities. FDA implemented 15 initiatives prior to the SIMP's issuance and 2 additional initiatives since then. Of the remaining initiatives, 1 was terminated and, as of March 2016, FDA was in the process of implementing the other initiative.
Although not generally reported in the SIMP, FDA officials identified mechanisms to assess the effectiveness of the majority of the initiatives included in the plan. Of the 30 efficiency initiatives, FDA officials identified 8 that have formal evaluations (such as third-party assessments) and 9 that are assessed informally (such as by gathering feedback). For the remaining 13, officials said they are either exploring effectiveness measures or have no plans to assess them because they consider it to be unnecessary or impractical. FDA identified mechanisms to assess 12 of the 19 workforce development initiatives, including through recruitment performance metrics and surveys of training participants. For 4 initiatives, the centers each use different approaches to assess training. For the remaining 3 initiatives, FDA either is developing a mechanism or described past assessment activities.
section_title: What GAO Recommends
paragraphs: GAO recommended that the Secretary of Health and Human Services direct FDA to engage in a strategic planning process to identify challenges that cut across the medical product centers, and document how it will achieve measurable goals and objectives in these areas. HHS agreed with the recommendation.

section_title: Letter
paragraphs: The Food and Drug Administration (FDA)—an agency within the  Department of Health and Human Services (HHS)—is responsible for  ensuring the safety and effectiveness of the millions of medical products  sold in the United States that Americans use daily. FDA must ensure the  safety and effectiveness of various types of medical products—drugs,  biological products, and medical devices—and must do so throughout the  products’ lifecycle, including before and after they are brought to market.  However, FDA has faced challenges in carrying out the many  responsibilities necessary for this oversight.
Our prior work and other studies have identified key management  challenges FDA faces in order to successfully fulfill its mission. In 2009,  we reported on FDA’s challenges managing its growing medical product  oversight responsibilities and recommended that the agency develop a  more complete estimate of its resource needs. In 2010, we identified  significant management challenges at FDA and made recommendations  to improve its strategic management and planning. Among other  challenges, our work, as well as the work of the Partnership for Public  Service, identified challenges FDA experienced with recruiting, retaining,  and developing its workforce, and coordinating internally. In addition,  HHS’s Office of Inspector General found management challenges at FDA  that weakened its ability to oversee regulated products, such as by  preventing medication imports from foreign and unlicensed suppliers.
The Food and Drug Administration Safety and Innovation Act (FDASIA),  in addition to providing FDA with various responsibilities related to the  oversight of medical products, required FDA to report on its  implementation of management improvements for this oversight.  Specifically, FDASIA required FDA to develop and submit to Congress a  strategic integrated management plan (SIMP) to identify (1) initiatives for  improving efficiency, (2) initiatives for workforce development, and (3)  measures for assessing the progress of these initiatives. To satisfy this  requirement, FDA issued the SIMP in July 2013. In light of the  documented management challenges within FDA and the importance of  effective medical product oversight, you asked us to study FDA’s  implementation of the SIMP. In this report we   1.  evaluate the extent to which FDA’s SIMP serves as a strategic  planning document for medical product oversight,  2.  describe the types of initiatives that FDA chose to include in the SIMP,  3.  describe the mechanisms FDA has to evaluate the effectiveness of  the efficiency and workforce initiatives described in the SIMP.
To evaluate the extent to which FDA’s SIMP serves as a strategic  planning document for medical product oversight, we examined the plan  for strategic planning activities, and spoke to officials from FDA’s three  medical product centers, which are responsible for overseeing medical  products, and the agency’s planning offices to understand how FDA  developed the SIMP. We also spoke with officials from five industry  groups that together represent each of the three medical product areas to  obtain their views on the SIMP. We reviewed relevant criteria from GAO’s  body of work on effectively managing performance under the Government  Performance and Results Act of 1993 (GPRA), as enhanced by the  GPRA Modernization Act of 2010. Among other things, GPRA requires  that federal agencies develop long-term strategic plans containing nine  key elements, such as agency-wide goals and strategies for achieving  those goals. We have previously reported that these elements serve as  the foundation for effective strategic planning, and therefore can serve as  leading practices at lower levels within federal agencies. As such, we  assessed the SIMP against the seven relevant strategic planning  elements identified in GPRA to determine whether the plan contained  these key elements.
To describe the types of initiatives FDA chose to include in the SIMP, we  reviewed the plan and identified the initiatives FDA selected for improving  efficiency and developing its workforce at the medical product centers.  We also reviewed separate FDA medical product guidance and workforce  planning documents related to the initiatives to further describe each  initiative’s purpose. We spoke with officials from each medical product  center, as well as FDA human resources officials to learn more about the  initiatives and determine each initiative’s implementation status. We then  compared the characteristics of these initiatives and their implementation  status across centers to describe key differences.
To describe the mechanisms FDA has to evaluate the effectiveness of the  efficiency and workforce initiatives described in the SIMP, we reviewed  the plan and identified measures of effectiveness. We also analyzed  related FDA guidance documenting performance results to compare  differences in monitoring activities across medical product centers. We  interviewed officials from each medical product center, as well as the  agency’s human resources office to assess if the agency had additional  mechanisms in place to measure the effectiveness of the initiatives in the  SIMP. We did not assess each mechanism to determine if the approach  was effective or appropriate for monitoring and evaluation. Finally, we  analyzed workforce data for fiscal years 2012 through 2015—from the  year FDASIA was enacted to the most recently available year—to assess  hiring and attrition trends for FDA and each of the medical product  centers. We interviewed officials on their use of the agency’s hiring and  personnel databases, and on potential issues associated with the  completeness, accuracy, and timeliness of the data. We assessed the  reliability of FDA workforce data by interviewing agency officials  knowledgeable about the data, reviewing related documentation, and  performing electronic testing for obvious errors and accuracy and  completeness, where applicable. We determined that the data were  sufficiently reliable for the purposes of this report.
We conducted this performance audit from July 2015 to May 2016 in  accordance with generally accepted government auditing standards.  These standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: FDA Medical Product Oversight Structure
paragraphs: Within FDA, the Office of Medical Products and Tobacco is responsible  for providing leadership for the medical product centers and coordinating  their plans, strategies, and programs. Under the office’s direction, three  FDA centers have primary responsibility for overseeing medical products  and developing strategic plans to guide their activities:
The Center for Biologics Evaluation and Research (CBER) is  responsible for overseeing most biologics, such as blood, vaccines,  and human tissues.
The Center for Drug Evaluation and Research (CDER) is responsible  for overseeing drugs and certain therapeutic biologics.
The Center for Devices and Radiological Health (CDRH) is  responsible for overseeing devices and for ensuring that radiation- emitting products, such as microwaves and x-ray machines, meet  radiation safety standards.
Several offices within FDA provide additional oversight and management  support to assist the three medical product centers. FDA’s Office of  Policy, Planning, Legislation, and Analysis supports strategic planning at  the agency-wide, program-specific, and center levels across FDA, which  included coordinating the development of the SIMP and FDA’s agency- wide strategic priorities document. FDA’s Office of Human Resources  supports recruitment and workforce management activities. Finally, FDA’s  Office of Regulatory Affairs conducts field activities for all of FDA’s  medical product centers, such as inspections of domestic and foreign  establishments involved in medical products.
section_title: Medical Product Oversight Activities
paragraphs: The centers conduct pre- and post-market oversight of medical products,  as well as formulate guidance, perform research, communicate  information to industry and the public, and set priorities. Premarket  oversight comprises review activities to ensure that medical products are  safe and effective for use before they can be marketed in the United  States. FDA’s premarket oversight typically begins when companies— known as sponsors—develop a medical product. Before beginning  clinical trials (studies involving humans) for a new medical product,  sponsors must submit an application so that FDA can preliminarily assess  the product for safety. As part of its premarket oversight, FDA may also  choose to inspect establishments producing medical products to ensure  their manufacturing processes meet quality standards.
Postmarket oversight includes review activities to both provide certainty  that medical products are safe and effective after they have been  marketed, and to enable FDA to take regulatory actions if a safety issue is  identified, such as requiring that sponsors communicate new safety  information to the public and health care providers or withdraw the  product from the market. Examples of postmarket oversight include  reviewing reports of adverse events to monitor the safety of marketed  medical products and examining advertising and other promotional  materials to ensure they are not false or misleading. FDA may require  sponsors to provide additional information both before and after a product  has been approved. For example, FDA may require medical product  manufacturers to create a Risk Evaluation and Mitigation Strategy to  ensure that the benefits of a medical product outweigh its risks.
section_title: Funding for Medical Product Oversight
paragraphs: A significant portion of FDA’s annual appropriation consists of amounts  derived from user fees paid by the medical products industry. Beginning  in 1992 with prescription drugs, Congress has authorized the collection of  user fees from the medical products industry to provide additional  resources for certain FDA oversight activities. Each user fee program is  subject to reauthorization every 5 years and supports different oversight  activities across each of the centers, as illustrated in figure 1. In 2012,  FDASIA reauthorized or authorized four user fee programs for medical  products. It included the fifth reauthorization of the Prescription Drug  User Fee Act of 1992 (PDUFA), which allows FDA to collect user fees  from manufacturers of prescription drugs. It also included the third  reauthorization of the Medical Device User Fee and Modernization Act  (MDUFA), which allows FDA to collect user fees from manufacturers of  medical devices. Congress also authorized two new user fee programs  in FDASIA: the Biosimilar User Fee Act (BsUFA), and the Generic Drug  User Fee Amendments Act (GDUFA). BsUFA authorizes FDA to collect  user fees from manufacturers of biosimilars, which FDA may approve  based on a sponsor’s ability to show that the product is highly similar to  an FDA-approved biological product and has no clinically meaningful  differences in terms of safety and effectiveness. GDUFA authorizes  FDA to collect user fees from manufacturers of generic drugs.
Prior to each user fee program reauthorization, FDA negotiates with  representatives of each medical products industry to identify goals for  how FDA should spend those user fees over the next 5-year authorization  period. Once FDA and the industry reach agreement, the Secretary of  Health and Human Services submits letters containing these  commitments to Congress. The user fee commitments contain  performance goals for FDA’s review activities, such as reviewing and  acting upon a certain number of received medical product applications  within certain time frames. User fee commitments may also require FDA  to undertake certain actions, such as implementing agreed upon  efficiency enhancements by a given date. FDA reports annually to  Congress on progress made in achieving performance goals identified in  each of the user fee commitments. These reports contain both  descriptions of each center’s relevant oversight activities over the  previous year, and data on its performance toward meeting user fee  commitments.
section_title: FDA Did Not Fully Document a Comprehensive Strategy for Medical Product Oversight in its Strategic Integrated Management Plan
paragraphs: We found that the SIMP does not contain key elements of strategic  planning and therefore does not present a comprehensive strategy across  the medical product centers. Our previous work has shown that strategic  planning for activities below the agency-wide level is a leading practice for  successful agencies, and can help agencies integrate activities, align  goals, and coordinate performance management across different parts of  their organization. However, the SIMP does not fully contain several of  these leading practices. Of the seven relevant strategic planning  elements from GPRA, the SIMP fully contains two elements, partially  contains four elements, and does not contain one element. In particular,  we found that the SIMP contains a mission statement and describes how  FDA incorporated input from Congress; it partially contains a description  of its general goals and objectives, the strategies needed to achieve its  goals and objectives, how its performance goals related to its general  goals and objectives, and program evaluations used to review its goals  and objectives; and it does not identify external factors that could  significantly affect the achievement of its goals and objectives.
Specifically, the SIMP presents high-level information on goals and  performance measures for medical product oversight, but lacks detail on  how it will be used or implemented. Each of the SIMP’s first two sections  describes a goal—improving efficiency and developing the workforce,  respectively—and lists planned or ongoing initiatives to achieve that goal.  For most of these initiatives, rather than describe the necessary steps,  planned accomplishments, or time frames for implementation, the SIMP  provides a high-level description of what FDA expects to achieve. In  addition, the SIMP’s summary states that the plan reflects coordination  and cooperation among the centers to address their program-specific  needs, share best practices, and share common solutions. However, FDA  officials told us that they do not use the SIMP to address issues requiring  center collaboration, and acknowledged that the plan did not represent  the full range of working relationships among the centers. Moreover, the  SIMP does not fully link its performance goals to its general goals and  objectives. The SIMP instead describes performance measures related to  FDA’s user fee commitments, even though several of the initiatives  included in the plan are unrelated to these commitments. FDA officials  explained that they focused the SIMP’s performance measures on user  fee commitments rather than, for example, tying performance measures  to each initiative, because user fee commitments are the main vehicle by  which FDA assesses the efficiency of each medical product center’s  premarket review.
Additionally, groups we spoke with that represent the medical products  industry did not view the SIMP as an effective strategic planning  document for FDA. Of the five industry groups we interviewed, two were  unfamiliar with the SIMP and the others did not see how its contents  related to strategic planning. For example, representatives from one  industry group said that the SIMP was neither integrated nor strategic,  because it merely described the different activities of the centers rather  than establishing one overarching strategic approach for all of the  centers. Additionally, representatives from another industry group said  that the SIMP lacked detail on how FDA would use it or implement the  initiatives it described.
FDA officials said that due to the circumstances around FDASIA’s  enactment in 2012, they chose to develop the SIMP as a point-in-time  document to address legislative requirements rather than as a strategic  plan for medical product oversight. For example, agency officials said  FDASIA required FDA to submit the SIMP within a year of enactment,  during which time FDA was also developing its agency-wide strategic  priorities document. Officials said that more time would have better  enabled FDA to align the SIMP with agency-wide goals, and helped the  agency to structure the plan as a strategic planning document. Officials  also told us that leadership gaps in the Office of Medical Products and  Tobacco, caused in part by turnover in the Deputy Commissioner  position, created challenges when developing the SIMP. Officials said  that, given these factors, the agency chose to develop a more limited  document. Despite acknowledging that the SIMP was not intended to be  an effective strategic planning document, FDA officials said that the  SIMP’s development process was useful because it facilitated  coordination and information sharing between the centers on how to  achieve certain user fee goals.
Nonetheless, FDA officials acknowledged the growing need for strategic  planning across the medical product centers to improve center  collaboration and address emerging issues, but said that it may not  require a separate strategic plan. Officials said that some issues, such as  staffing vacancies and coordination with other agencies, were better  addressed at an agency-wide level. However, they indicated that  integration and collaboration across the medical product centers are  important for other issues that the agency is working to address, such as  data sharing, evidence generation, biomarker integration, combination  products, consistent terminology, patient engagement, and the medical  product review process. FDA officials also said that these types of  issues have become more important as the complexity of medical  products has increased, and that coordination can help the centers share  leading practices to address these issues. For example, officials said that  collaboration could help the centers develop more effective clinical trials,  improve their decision-making, and improve the quality of evidence and  clarity of guidance. For these issues, FDA officials said that they  continue to strategically plan across the centers without a written  document specifically for medical products by using other planning  documents. Although they noted that the agency’s resources have been  better spent working toward goals in existing plans, rather than putting  together a new strategic plan specific to medical product oversight, they  indicated that more formal planning in the future may be useful as  resources become available.
FDA officials said that they did not structure the SIMP as a strategic plan,  because they thought it would be duplicative of other FDA strategic plans;  however, we found that none of these other plans comprehensively  describes FDA’s long-term plan for addressing key issues amongst the  centers, as summarized below:
FDA has an overarching strategic priorities document that includes  strategic goals and objectives for medical product activities. This  document describes a broad level of activities, but does not  specifically discuss strategies across the centers. For example, one  FDA goal is partially aimed at improving coordination within FDA, and  the agency also describes some activities that may require the  centers’ collaboration, such as developing comprehensive regulatory  approaches for integrating approval and compliance functions.
FDA officials said that they use the annual budget process as an  opportunity for strategic planning. While FDA’s fiscal year 2017  budget justification describes planned activities specific to each  center, its planning across the centers is limited to a few specific  initiatives, such as developing scientific workshops to advance the  development of pediatric therapeutic products.
FDA officials identified strategic plans for specific initiatives that  involve each center, such as FDA’s strategic plan for advancing  regulatory science and FDA’s strategic plan for information  technology. However, we recently reported on FDA’s strategic plan  for information technology, finding a lack of goals and performance  measures for determining whether its implementation is successful in  supporting FDA’s mission.
Each center also has its own strategic plan, but they differ in structure  and content. While the center-specific plans include activities, goals,  and objectives relevant to each individual center, they do not describe  crosscutting issues or include plans for collaboration across the  centers to address them. Officials from each center said that they also  relied on performance measures in other documents, such as user fee  commitments, to plan their activities and measure their  performance.
The growing importance of areas that cut across medical product centers  highlights the importance of FDA’s strategic planning for medical product  oversight. The absence of a documented long-term plan for medical  product oversight may hinder FDA’s efforts to address emerging issues  that require center collaboration, such as access to quality data and  developing requirements for combination products. Also, the absence of a  documented strategy is inconsistent with leading practices for strategic  planning based on prior GAO work. These practices indicate that formal  strategic planning is needed for medical products by identifying  crosscutting issues and ensuring that collaborative center goals,  measures, and activities are effectively integrated with FDA’s overall  organizational mission and goals. Documenting a strategic plan for  medical products—whether it occurs in a freestanding document or as  part of existing documents the centers are already using—would also  enable FDA to oversee its activities in a consistent and transparent  manner, help the agency communicate its priorities to key stakeholders,  and help align its activities to support mission-related outcomes.
section_title: FDA Included Mostly a Compilation of Preexisting Efficiency and Workforce Development Initiatives in its Strategic Integrated Management Plan
paragraphs: In FDA’s SIMP, the agency compiled 30 efficiency initiatives under three  different themes and included 19 different types of workforce  development initiatives for each center on training, recruitment, and  retention. FDA had fully implemented about a third of the efficiency  initiatives and most of the workforce development initiatives prior to the  SIMP’s issuance in 2013.
subsections: section_title: FDA’s Strategic Integrated Management Plan Included User Fee and Center Initiatives to Improve Efficiency, About a Third of Which Were Implemented Prior to the Plan’s Issuance
paragraphs: We found that FDA grouped the SIMP’s 30 efficiency initiatives into three  themes: (1) business modernization, (2) process improvement, and (3)  smarter regulation. (See appendix I for a full description of each efficiency  initiative.)
Under business modernization, FDA included 3 initiatives on each  center’s workload measurement activities, 3 initiatives focused on  data standards efforts, and 2 initiatives specific to staff location and  ability to use electronic functions to complete their work. For the  initiatives on the centers’ workload measurement activities, the  centers each updated their time reporting systems to record user fee  activities, which employees are required to do in 2-week increments  four times during the fiscal year.
Under process improvement, FDA included 11 efficiency initiatives  specific to an agency-wide or center-specific need. CBER included  initiatives to improve its review mechanisms and move to more  electronic processes. CDER included efforts to streamline processes  for its formal communication mechanisms with the industry and  manufacturing facilities. CDRH included pilot programs for certain  device types and manufacturers, and a postmarket program for  identifying new device risks.
Under smarter regulation, FDA included 11 initiatives—8 initiatives  that stem from each user fee program, as well as 3 initiatives for  medical devices that respond to other statutory requirements. The  majority of the 11 initiatives are focused on the premarket review  process of medical products. Specifically, the initiatives are related to  improving communication between FDA and the industry, providing  additional guidance to industry for how FDA will assess medical  products, providing its plans for health information technology, and  defining FDA’s approach to and requirements for facilities that  manufacture drug products.
The SIMP notes that these three themes reflect the strategic goals and  priorities that the medical product centers are all pursuing to improve  efficiency. FDA officials further explained that the three themes helped to  connect seemingly unrelated center-specific and user fee program  responsibilities and initiatives presented in the SIMP.
We found that FDA fully implemented about a third of the 30 efficiency  initiatives within the 12 to 18 months prior to the SIMP’s issuance in July  2013, and implemented another half of the initiatives since then. As of  March 2016, the remaining initiatives had yet to be fully implemented, the  majority of which are related to developing data standards for electronic  submissions or efforts to move to an electronic review process. For  example, CDRH specified that its initiative to establish a unique device  identification system started with the highest risk medical devices and will  be fully implemented in 2020 once all medical devices have identifiers in  electronic health records. (See table 1.)
section_title: FDA Included Different Types of Workforce Development Initiatives for Each Center, and Most Initiatives Were Implemented Prior to the Strategic Integrated Management Plan’s Issuance
paragraphs: We found that FDA included 19 workforce development initiatives in the  SIMP—11 training initiatives, 7 recruitment initiatives, and 1 retention  initiative. (See appendix II for a full description of each workforce  development initiative.) FDA officials told us that the majority of the  workforce development initiatives are specific to each center’s activities,  reflecting differences in program responsibilities and procedures. Industry  officials we spoke with emphasized the importance of recruitment,  retention, and training efforts on the agency’s ability to meet user fee  commitments. (For more information on the size and characteristics of  FDA’s overall and center-specific workforce, see appendix III.)
The 11 training initiatives FDA included in the SIMP describe multiple  training courses or programs. As part of these initiatives, FDA included  programs for the new reviewer trainings offered by each of the medical  product centers and initiatives covering training for each of the user fee  programs, which may be taken by staff from multiple centers. The  initiatives also included training courses dedicated to specific topics for  each medical product center. For example, CBER included training  courses covering medical device review and project management, and  CDRH included two leadership experience programs for future and  current managers. The first CDRH program gives certain staff an  opportunity to explore a supervisory career path; the second is to help  staff in management positions learn about CDRH’s management  competencies and satisfy federal supervisory training requirements.
We found that the seven recruitment initiatives FDA included in the SIMP  are intended to streamline recruitment processes at both the agency and  center levels. For example, CDER included initiatives to manage and fill  vacancies in executive-level positions and critical occupations, such as  chemists and project managers. Each of the centers also included  initiatives to improve outreach to potential job candidates, such as  through job fairs, alumni networks, and institutional partnerships.
For retention, we found that FDA included a single initiative in the SIMP— CDRH’s efforts to address the center’s high attrition rate by reducing  individual workloads, decreasing staff-to-manager ratios, and providing  employees with a better work environment. To reduce staff workloads and  decrease staff-to-manager ratios, CDRH increased the number of review  and management staff. To provide a better work environment, CDRH  developed and improved performance evaluation tools and employee  recognition processes. For example, CDRH created a resource guide to  educate staff on the center’s performance management system. FDA did  not include retention initiatives for CBER or CDER in the SIMP; however,  officials from both centers told us that each center uses some retention  tools and processes.
Among the 19 workforce development initiatives included in the SIMP, 15  initiatives were implemented prior to the plan’s issuance in July 2013. By  March 2016, FDA implemented 2 additional workforce development  initiatives, bringing the total to 17 initiatives. Of the remaining 2 initiatives,  1 is still being implemented. CDRH is in the process of reducing staff  workloads as part of the center’s retention initiative—an activity related to  hiring plans that are to be phased in through fiscal year 2017. The final  one, CDER’s alumni network initiative, was terminated. CDER planned to  pilot the initiative in four of its offices beginning in 2013, but it was never  piloted or implemented due to a lack of employee activity in alumni  associations. (See table 2.)
section_title: FDA Identified Mechanisms for Evaluating the Effectiveness of the Majority of Its Strategic Integrated Management Plan Initiatives
paragraphs: We found that FDA had already established or has plans to establish  formal and informal mechanisms to assess the effectiveness of just over  half of the 30 efficiency initiatives in the SIMP. For the SIMP’s workforce  development initiatives, FDA identified mechanisms to assess most of the  19 initiatives, and each center’s approach to assess training is different.
subsections: section_title: FDA Identified Formal and Informal Mechanisms to Evaluate the Effectiveness of Just Over Half of Its Efficiency Initiatives
paragraphs: FDA stated that the agency had assessed or has plans to assess just  over half of the 30 efficiency initiatives for effectiveness, although these  plans are generally not described in the SIMP. In its plan, FDA identified  formal measures of effectiveness for 3 initiatives, each of which is based  on a MDUFA or PDUFA commitment, but does not specify any additional  measures in the plan itself for the remaining 27 initiatives. (See table 3.)  However, we found that FDA has formal or informal measures that do not  appear in the SIMP for a majority of these initiatives.
For five initiatives, FDA officials identified formal measures of  effectiveness that were not described in the SIMP. The officials explained  that these initiatives are assessed through periodic user fee program  reports or center strategic goals. For example
CDER officials told us that the GDUFA initiative on commitments,  complete review, and easily correctable deficiencies is assessed  against the user fee commitments. For example, FDA committed to  review and act on 90 percent of complete, electronic abbreviated new  drug applications within 10 months after the date of submission. FDA  does not have to meet some of these commitments until 2017, but the  agency indicated that it faces challenges meeting them due to a large  backlog of applications.
CDRH officials told us that they assessed the investigational device  exemption decision program using center-specific strategic goals  related to reducing the number of review cycles needed before full  approval, and reducing the overall median time to full approval. CDRH  met each of these goals in fiscal year 2015.
For nine initiatives, officials from each center described efforts they took  to informally examine effectiveness. For example, CBER uses staff  feedback to assess implementation of its electronic review templates, and  incorporates revisions as appropriate. For CDRH’s initiative to establish a  unique device identification system, officials said they track certain  metrics, such as numbers of vendors certified to participate in the  program and visits to the program’s website.
FDA officials told us that, for the remaining 13 effectiveness initiatives in  the SIMP, they are either exploring effectiveness measures or do not  have plans to measure effectiveness. In some cases, officials described  ways in which effectiveness could be measured or efforts to develop  assessments. For example, CDRH officials told us that they did not  currently have, but were exploring, ways to measure the impact of its  signal management program initiative through industry responses or  actions taken. In other instances, such as with CBER’s two initiatives on  improving its managed review process tool, officials indicated that they  were unclear about the best way to measure effectiveness. Additionally,  FDA does not have current plans to measure effectiveness of some  initiatives and officials noted that such measurement would be either  unnecessary or impractical. For example, FDA is not measuring  effectiveness for the PDUFA meeting minutes initiative, because officials  said it would be a challenge to survey sponsors and the agency wants to  be selective about choosing that option.
section_title: FDA Identified Mechanisms to Assess Most of the Workforce Development Initiatives, including Centers’ Different Approaches to Assess Training
paragraphs: FDA identified mechanisms to assess the effectiveness of 12 of the 19  workforce development initiatives. Specifically, the agency identified  mechanisms to assess 4 of 7 recruitment initiatives, the 1 retention  initiative, and 7 of 11 training initiatives. In the SIMP, FDA generally did  not describe assessments for specific initiatives, but rather described  each user fee program’s hiring and training commitments as broad  measures of the agency’s workforce development efforts. For example, in  order to reach the committed GDUFA level of 923 full-time equivalent  staff by the end of fiscal year 2015, FDA committed to hire and then train  at least 25 percent of staff in fiscal year 2013 and 50 percent in fiscal year  2014. FDA reported that it met this commitment by October 2014, 11  months ahead of schedule. (As previously noted, appendix III provides  additional information on the size and characteristics of FDA’s overall and  center-specific workforce.)
FDA officials described the mechanisms in place to assess the  effectiveness of 4 of the 7 recruitment initiatives described in the SIMP.  For the two that are FDA-wide recruitment initiatives, FDA uses agency-  and department-wide tools to measure the overall effectiveness.  Specifically, FDA developed the FDA Accelerated Staffing Track 80-day  hiring metric in early fiscal year 2015 to measure the time it takes to hire a  new employee once the need is identified. However, officials said that  data quality and data entry issues limited the accuracy and validity of the  data available at the time of our review. In addition, FDA uses HHS  personnel information systems to track monthly and quarterly hire and  separation data for each medical product center. Officials also  described performance metrics that CBER and CDRH track to assess  effectiveness for two center-specific recruitment initiatives. For CBER’s  comprehensive recruitment strategy, the center tracks the number of  resumes received and hires from targeted populations. For example,  CBER hired four veteran; five minority; and 31 science, technology,  engineering, and mathematics candidates during fiscal year 2015. For  CDRH’s initiative on strategic communication and outreach for  recruitment, the center uses monthly reports to track the number of  applicants responding to the center’s job postings, including data on the  number of applicants that apply to and are eligible for each position. For  the three recruitment initiatives that do not currently have mechanisms to  assess effectiveness, CDER officials described the center’s current plans  or what it had already done. For one initiative, officials said that the center  is developing an automated project management and tracking tool.  Officials expect that the tool will be implemented in spring 2016. For  another initiative, CDER met its overall hiring objectives, but did not  measure the number of selections made as a result of the initiative itself.  Finally, CDER’s alumni network initiative was never implemented, and  thus FDA did not put in a place a mechanism to assess its effectiveness.
To assess the effectiveness of the one retention initiative in the SIMP,  CDRH officials told us they measure the number of full-time equivalent  staff supporting MDUFA activities, changes in staff-to-manager ratios,  and survey results. CDRH’s total full-time equivalent staff supporting  MDUFA increased from 1,133 in fiscal year 2013 to 1,293 in fiscal year  2015. At the same time, CDRH reduced the staff-to-manager ratio in its  two offices with medical device review responsibilities. CDRH also  analyzes changes in federal Employee Viewpoint Survey responses to  assess its efforts to provide a better workplace for its employees. From  2011 to 2014, CDRH observed positive changes for three of six critical  indicators it identified for providing recognition and all six critical indicators  it identified for performance evaluation.
Of the 11 training initiatives, CBER, CDER, and CDRH officials each  identified mechanisms to assess the effectiveness of 7 initiatives.  Specifically, each center indicated that it uses participant surveys to  assess effectiveness. CBER also delivers a test at the conclusion of  some, but not all, of the programs included in its training initiative.Furthermore, as described in the SIMP, CDRH conducts an audit process  for its Reviewer Certification Program through which new reviewers are  evaluated by an experienced reviewer. During the audit process, new  reviewers are rated against six criteria, including the appropriate use of  guidance and strength of final review decision analyses.
The four remaining training initiatives described in the SIMP were related  to each user fee program and the centers use different approaches to  assess the extent to which all reviewers required to complete training  have done so. CBER and CDER track the names of staff who register for  training and do not measure the number of medical product reviewers  required to complete the trainings. CBER and CDER officials said that  FDA was not required to report on training completion rates, and they  assume that required staff completed user fee training, because it is  made available in different settings, such as CBER’s review management  updates and CDER’s new employee orientation. For example, CDER  officials told us that 99 percent of the staff hired under the GDUFA  commitment had completed training, as all hired staff take mandatory  online training once hired. Training completion rates are not included in  GDUFA performance reports. In contrast, CDRH measures user fee  training completion rates among its required staff and reports on these  rates in MDUFA quarterly performance reports, as required by their user  fee commitments. CDRH reported a 99 percent staff completion rate  among its review staff required to complete MDUFA training.
Center officials did not identify any mechanisms to assess how effective  participants were in applying the information learned during these user  fee trainings (known as training comprehension). CBER officials said its  user fee trainings were delivered and recorded in special training  sessions, such as in monthly review management updates, and that  these trainings do not have mechanisms to assess comprehension.  CDER officials were unable to show that staff who took user fee trainings  were given post-completion tests. CDRH officials told us that a post- completion test was not disseminated for the initial MDUFA trainings.  However, CDRH has since incorporated the user fee trainings into the  center’s Reviewer Certification Program, which has multiple mechanisms  for assessment.
section_title: Conclusions
paragraphs: Emerging issues—including increasingly complex medical products such  as combination products, the need for integrated information systems,  and the increased hiring demands for specific scientific knowledge—go  beyond the expertise of a single medical product center and highlight the  growing importance of strategic planning across medical products.  Advances involving new diagnostic tools, treatments, and cures require  collaboration in order to be successful. However, FDA has faced  longstanding challenges in carrying out the many responsibilities  necessary for the oversight of medical products. While FDA engaged  each of the medical product centers in the development of the SIMP, this  narrowly focused plan is not used by the agency or centers. Moreover, it  highlights gaps in the agency’s management across FDA’s medical  product centers by not fully linking its performance goals to its general  goals and objectives, and having limited information on implementation  time frames. While FDA has various other strategic planning documents  for medical product oversight, these documents also do not set a long- term strategy for the centers, because they are focused on narrower  issues or do not have details specific to center-level collaboration. Using  leading practices identified as essential for strategic planning can help  ensure the agency is prepared to address challenges requiring  coordination across the centers in a consistent and transparent manner.  Documenting measurable goals, objectives, and a long-term strategy for  areas resulting from this planning—whether it is through a freestanding  document or as part of existing documents—can help the agency ensure  its priorities are communicated among key stakeholders, even in times of  leadership turnover.
section_title: Recommendation for Executive Action
paragraphs: To ensure that FDA can effectively coordinate and integrate its medical  product centers’ programs and emerging issues, we recommend that the  Secretary of Health and Human Services direct the Commissioner of FDA  to engage in a strategic planning process to identify challenges that cut  across the medical product centers and document how it will achieve  measurable goals and objectives in these areas.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to HHS. The agency agreed with our  recommendation and provided written comments, which are reprinted in  appendix IV. In its written comments, HHS described the context  surrounding the development of the SIMP and the progress FDA has  made regarding its medical product review activities under its four user  fee programs. It noted the importance of coordinating and integrating the  activities that are common among FDA’s medical product centers. In  agreeing with our recommendation, HHS indicated that FDA has already  started a process to identify key crosscutting themes for the medical  products centers, which it will then use to develop an overarching  strategic planning framework to guide the work of these centers. We  encourage FDA to use leading practices to ensure this framework has  measurable goals and objectives.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution of it until 30 days from its  date. At that time, we will send copies to the Secretary of Health and  Human Services. In addition, the report will be available at no charge on  the GAO website at http://www.gao.gov.
If you or your staffs have any questions about this report, please contact  me at (202) 512-7114 or crossem@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix V.
section_title: Appendix I: Description of Efficiency Initiatives in FDA’s Strategic Integrated Management Plan
paragraphs: Table 4 shows each efficiency initiative that the Food and Drug  Administration (FDA) included in its strategic integrated management  plan. FDA described 30 efficiency initiatives in its plan, including those  specific to a medical product center or to a user fee program. FDA also  grouped the initiatives into three themes: (1) business modernization, (2)  process improvement, and (3) smarter regulation.
section_title: Appendix II: Description of Workforce Development Initiatives in FDA’s Strategic Integrated Management Plan
paragraphs: Table 5 shows each workforce development initiative the Food and Drug  Administration (FDA) included in its strategic integrated management plan.  FDA described 19 workforce development initiatives in its plan specific to  recruitment, retention, or training.
section_title: Appendix III: Size and Characteristics of FDA and Medical Product Center Workforce, Fiscal Years 2012 to 2015
paragraphs: We analyzed Food and Drug Administration (FDA) data on the agency’s  workforce population and attrition for fiscal years 2012 to 2015. Our  analysis includes detail on the three medical product centers: the Center  for Biologics Evaluation and Research (CBER), the Center for Drug  Evaluation and Research (CDER), and the Center for Devices and  Radiological Health (CDRH). FDA’s total workforce grew from 16,716  employees in 2012 to 19,043 employees in 2015—a 14 percent increase.  FDA measures year-to-year changes in its total workforce by subtracting  the employee losses from the employee gains of permanent and non- permanent staff. Figure 2 shows the number of medical product center  employees—permanent and non-permanent—for each fiscal year.
Some losses and gains reported by the centers are due to employees that  transferred within the agency, such as from one center to another. Tables  6, 7, and 8 show information on transfers within FDA for each medical  product center in fiscal years 2012 to 2015.
FDA also tracks the percentage of retirement-eligible staff. In fiscal year  2015, 12.4 percent of FDA’s overall permanent workforce was retirement- eligible. In the same fiscal year, the retirement eligibility for each medical  product center was 15.9 percent for CBER, 10.9 percent for CDER, and  11.8 percent for CDRH.
Figure 3 shows the FDA-wide and center-specific attrition rates from fiscal  year 2012 to 2015. FDA calculates attrition rates by dividing the number of  voluntary personnel losses by the average number of employees for each  fiscal year. Voluntary personnel losses include retirements, resignations,  and employees who transfer externally to another federal agency or  internally to a different center or office within FDA.
The following tables show the number of employees, personnel gains, and  attrition rates for FDA and each medical product center. The tables also  include information on mission-critical occupations, which may vary by  center.
section_title: Appendix IV: Agency Comments from the Department of Health and Human Services
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, William Hadley, Assistant  Director; George Bogart; Jennel Lockley; Drew Long; Matt Lowney; Dan  Powers; and E. Jane Whipple made key contributions to this report.