# Presentation: 232238

## PDUFA, Drug Approval Times, Drug Safety Withdrawal Rates, and the Drug Development Process:  Empirical Findings

**Presentation by Prof. E. R. Berndt to FDA Public Meeting:**
**“****Prescription Drug User Fee Act Public Meeting”  Natcher Conference Center, National Institutes of Health  Bethesda, MD  20815**
**November 14, 2005  10:00 – 10:50 am**

***Collaborators:****  *
*Ernst Berndt, Ph.D., **MIT Sloan School and National Bureau of Economic Research*
*Adrian Gottschalk, **Biogen Idec (formerly of MIT/Harvard)*
*Tomas Philipson, Ph.D.,** University of Chicago*
*Matthew Strobeck, Ph.D., **Westfield Capital (formerly of MIT/Harvard)*

**Correspondence:  eberndt@mit.edu**

## Project Support and Acknowledgements

- We gratefully acknowledge research funds and support from general funds at Harvard-MIT Division of Health Sciences and Technology and the MIT Sloan School of Management
- We thank the following individuals for their support, assistance, and insight
- Dr. Mark McClellan – CMS (formerly FDA)
- Ed Hass – FDA
- Roy Gross – MIT
- Dr. Joseph Dimasi – Tufts CSDD
- Dr. Lee Simon – Harvard Medical School (formerly FDA)
- We thank the FDA as well as all the companies and their employees who participated in our confidential and anonymous survey of the drug development process.

## Publications Underlying This Presentation

- Ernst R. Berndt, Adrian H. B. Gottschalk, Tomas J. Philipson and Matthew W. Strobeck, “Industry Funding of the FDA: Effects of PDUFA on Approval Times and Withdrawal Rates,” *Nature Reviews: Drug Discovery*, 4(7), July 2005: 545-554.
- Ernst R. Berndt, Adrian H. B. Gottschalk and Matthew W. Strobeck, “Opportunities for Improving the Drug Development Process: Results from a Survey of Industry and the FDA,” Cambridge, MA: National Bureau of Economic Research, Working Paper #11425, June 2005, available from _[http://www.nber.org](http://www.nber.org/)_.  Forthcoming in Adam Jaffe, Joshua Lerner and Scott Stern, eds., *Innovation Policy and the Economy*, Vol. 6, Cambridge, MA: MIT Press for the National Bureau of Economic Research, late 2005
- For further information, contact eberndt@mit.edu

## NDA approval times soared from 22.7 months in 1970 to over 32 months by 1990

**Framing the Issues**

**Source: Reichert, Janice M. “Trends in Development and Approval Times for New Therapeutics in the United States,” **_**Nature Reviews: Drug Discovery**_**, Sept. 2003, Vol. 2.**

## In response to public pressure, Congress addressed the issue of lengthy approval times via  the Prescription Drug User Fee Acts in 1992 (renewed in 1997 and 2002)

**Framing the Issues**

**Source: Reichert, Janice M. “Trends in Development and Approval Times for New Therapeutics in the United States,” **_**Nature Reviews: Drug Discovery**_**, Sept. 2003, Vol. 2.**

- NDA approval times declined substantially from 1992 to 2001

## The PDUFA Acts I, II, and III have legislated specific action dates for reviewing NDAs within 6 or 10 months

**PDUFA**

**Source:  FDA Website - http://www.fda.gov/oc/pdufa3/2003plan/default.htm#update**

## In aggregate from 1993 to 2003, for ALL NDAs, BLAs, product fees, and establishment fees, the FDA has collected just over $1.2 billion in user fees

**PDUFA**

**Sources:  FDA Website - http://www.fda.gov/oc/pdufa/finreport2003/financial-fy2003.html#totalcosts**
**“****Budget in Brief FY 2004” and “Budget in Brief FY 2005” U.S. Department of HHS**
**“****FY 2003 PDUFA Financial Report.”  FDA, March 2004.**

- FY 2002 and 2003  budget for entire FDA was **$1.55 B** and **$1.65 B**
- PDUFA accounted for **9.2%** and **12.7%** of budget respectively
- FY 2002 and 2003 budget for drug and biologic agencies totaled $540 M and$ 624 M
- PDUFA accounted for **26.3%** and **33.7%** of amounts respectively
- PDUFA monies in 2002 and 2003 make up roughly ***50%*** of monies spent on review of human drug applications

## Similar user fees exist within the EMEA 

**PDUFA**

**Sources:  Ines M. Vilas-Boas, C. Patrick Tharp, “The Drug Approval Process in the U.S., Europe, and Japan:  Some Marketing and Cost Implications”, J. Managed Care Pharm 3, 1997, 459-465.**
**http://www.mhra.gov.uk/aboutmhra/aboutmhra.html**

- Current application free for single strength and one pharmaceutical form is €232,000 with an additional €23,200 for each additional strength and/or form
- Annual fee of €75,600 is assessed with a five year renewal fee of €11,600.
- EMEA goal of 75% funding from industry fees and 25% from European Commission
- In comparison ...
- UK’s Medicines and Healthcare products Regulatory Agency is funded entirely through user fees
- Japan’s regulatory agency, Koseisho, has no user fee program

## De facto, NME NDA approval times have been reduced ... but was this truly a result of PDUFA?

- Research Issues:
- Have PDUFA-I (1992) and PDUFA-II (1997) been associated with reductions in NDA/BLA review time, controlling for confounders?
- What is the effect of PDUFA on safety withdrawal rates?
- What would NME approvals have looked like in a world without PDUFA? Would the R&D slowdown have improved or would it have been exacerbated?

**PDUFA**

## Research methods for the PDUFA analysis – we ...

- Evaluated a sample of 662 New Molecular Entities (NMEs) using data provided by the FDA*, 1979 - 2004
- Performed least squares regression analysis on continuous and binary dummy variables to quantify factors affecting the reduction in NDA review times
- Used the estimated regression equation to determine what approval times would have been in the world where PDUFA did not exist
- Performed analysis in aggregate and across therapeutic areas
- Used sample with updates for 2004 approvals to perform preliminary safety withdrawal analysis

**PDUFA**

- Source:  Ed Hass – FDA, PhRMA – Biologic NMEs

## Specification of the multivariate least squares linear regression equation

- Dependent variable is logarithm of NDA/BLA approval time
- Explanatory Variables Include
- Time Trend counter (1 to 23)
- Measured annual progression of time (proxy for improvements in technology, medical advances)
- Priority Binary
- Is the NDA review a standard review or a priority review
- PDUFA1 Binary x Time ( 1 to 5 )
- Is time period during PDUFA 1 (Yes, No) x time trend
- PDUFA2 Binary x Time ( 6 to 10)
- Is time period during PDUFA 2 (Yes, No) x time trend
- Orphan Binary
- Is the drug filed for an orphan drug indication (Yes, No)
- Nation Binary
- Is the sponsoring drug developer foreign (Yes, No)
- IND-NDA Time (months)
- Logarithm of the IND to NDA time
- Therapeutic Class
- Thirteen major therapeutic areas and biologics evaluated (“biologics” excluded)

**PDUFA**

## Least squares regression indicates that PDUFA did indeed have a statistically significant effect in reducing approval times

**PDUFA**

**Pre-PDUFA = 2% annual decline**
**PDUFA I = 10% annual decline**
**PDUFA II = 5% annual decline**

## We used the estimated regression equation to characterize the worlds with and without PDUFA , with predicted approval times based on explanatory variables evaluated at their sample means

**PDUFA**

***World without PDUFA***

***PDUFA World***

**Bump in line due to PDUFA II coefficient change**

## Based on our regression analysis ... a world without PDUFA would have shifted the peak of NME approvals out at least two years ...

**PDUFA**

## Based on our regression analysis ... a world without PDUFA would have shifted the peak of NME approvals out at least two years ...

- The productivity growth “slowdown” would have been less severe - a high of 55 in 1998 to 32 in 2002, compared to a high of 62 in 1996 and 26 in 2002.
- The years from 1999 forward are relatively similar in terms of NME approvals
- Up through the end of PDUFA I (end of 1997), 33 fewer drugs would have been approved cumulatively (15% less)
- Through the end of PDUFA II (end of 2002), 13 fewer drugs would have been approved cumulatively (3.3% less)
- Effectively, many patients would not have had access to numerous innovative treatments as quickly

**PDUFA**

## What about safety?  Safety withdrawal rate comparisons for pre- and post-PDUFA depend critically on three variables

- Numerator
- what should be considered a safety withdrawal
- Denominator
- what sample of drugs should be included
- i.e. biologics, vaccines, NMEs only ...
- Time period definition and differential exposure
- what defines pre-PDUFA vs. post-PDUFA
*approved* during time period or s*ubmitted* during time period
- Note: withdrawals are relatively rare events

**Withdrawals**

## Some may say the FDA is approving drugs too rapidly and there are more safety withdrawals post-PDUFA 

**GAO (2002) study**
- only NCEs, not BLAs
- no significant difference 1986-1992 vs. 1993-2000 (p = 0.9615) – calculated by current authors
**Internal FDA Analysis***
- Thirteen of 477 (2.7%) of NMEs ***approved*** from 1971 to 1993 were withdrawn
- Seven of 303 (2.3% of NMEs ***approved*** from 1994 to 2004 were withdrawn
- P-value of 0.9104 (cannot reject null hypothesis of no difference) – calculated by current authors
**MIT Analysis (calendar years)**
- Nine of 320 (2.81%) of NMEs ***approved *** from 1980 to 1992
- Eight of 361 (2.21%) of NMEs ***approved *** from 1993 to 2002
- Includes chemical and biologic NMEs
- P-value of 0.8011 (cannot reject null hypothesis of no difference)

**Withdrawals**

- * Source: “Center for Drug Evaluation and Research 2003 Report to the Nation:  Improving Public Health through Human Drugs.” U.S. Department of Health and Human Services, Food and Drug Administration, April 23, 2004.

## In addition to simple comparison of proportions, a Kaplan-Meier survival curve illustrates that the pre-PDUFA and PDUFA samples almost converge for submitted NMEs

**In addition to simple comparison of proportions, a Kaplan-Meier survival curve illustrates that the pre-PDUFA and PDUFA samples almost converge for *****submitted NMEs***

**Withdrawals**

**n = 320 – Jan 1, 1980 to Aug 31, 1992**
**7 safety withdrawals**

**n = 368 – Sept 1, 1992 to Dec 31, 2003**
**8 safety withdrawals**

**Statistical comparisons of groups by log-rank and Wilcoxon tests indicate that we cannot reject the null hypothesis that the curves are the same  (p-value = 0.39)**

**Cumulative probability = 2.3%**

**Cumulative probability = 2.6%**

## Kaplan-Meier curve and simple histogram analysis reveal that pre-PDUFA drugs with safety issues remained on the market much longer than post-PDUFA approved drugs

**Withdrawals**

## However, although drug approval times have declined, there has been little change to clinical development times for several decades

**Clinical**

- 0

- 10

- 20

- 30

- 40

- 50

- 60

- 70

- 80

- 90

- 100

**1979**

**-**

**1985**

**1986**

**-**

**1991**

**1992**

**-**

**1997**

**1997**

**-**

**2002**

**Mean Approval time (Months)**

**FDA & Industry**

**Mean Clinical Development Time (months)**

**Clinical Development Time: IND to NDA Submission**

## We interviewed 50 senior R&D employees within the drug development industry at public companies with at least one product on the market – Additionally, we interviewed 8 senior staff members at the FDA  (all in the first half of 2004, pre-Vioxx)

**FDA & Industry**

**Confidential & Anonymous**

## Industry believes that the FDA is effective at keeping unsafe therapeutics from the public  

**FDA & Industry**

**General Findings **

**However, the agency has also delayed a significant number of therapeutics that ultimately have had a substantial positive impact on the public health**

**Note:  We did not ask the CEO or any finance people what they thought of the FDA**

## Industry believes additional communication and interaction with the FDA would be very valuable across all phases of development

**FDA & Industry**

**Communication - 1**

**In contrast, in four of the five stages the FDA rated the value of additional interaction much lower than industry.  FDA and Industry agreed that additional communication in Phase II would be very valuable.**

## Quantitative survey results indicate that Industry is willing to put its money where its mouth is

**FDA & Industry**

**Communication - 2**

**As costly as clinical development is, industry believes that funds spent on enhancing the interactions with the FDA would be moneys well spent, provided appropriate metrics and objectives were put in place**

- Industry interviewees indicated that their company would be willing to pay PDUFA type fees (no significant difference between biotech & pharma)
**Phase I **
- 70% 100K to 500K
-  30% 500K to 1M)
**Phase II **
- 10% - Would not pay
- 38% - 100K to 500k,  42% 500K to 1M
- 10% - Greater than 5 M
**Phase III**
- ~30% - 1M to 5M
- 40% - 100K to 1 M
- 30% - 0 or Other

**Notes:**
Would your company be willing to pay additional user fees to hire FDA staff to help with clinical development reviews, discussion, and issues during the following phases?  

## Among recommendations to improve the drug development process...

***Institute better metrics and goals for development in exchange for PDUFA-like fees***
***Increase interactions prior to Phase III, funded by user fees***
- Implement and monitor best practices across divisions
- Establish an oversight board with industry, agency, and premier scientists/physicians to evaluate a random sample of completed/terminated drug projects retrospectively
- Establish a more robust industry/FDA/NIH exchange program
- Create a more structured dispute resolution process
- Create a drug development knowledge database (FDA is the “custodian of the knowledge base”)

**Conclusions & Recs**

## Additional Research Quantifying Benefits and Costs of PDUFA

- Tomas J. Philipson, Ernst R. Berndt, Adrian H. B. Gottschalk and Matthew W. Strobeck, *“Assessing the Safety and Efficacy of the FDA: The Case of the Prescription Drug User Fee Acts”,* Cambridge, MA: National Bureau of Economic Research, Working Paper 11724, October 2005.  Available at www.nber.org/papers/w11724.