Department  of Health  and Human  Services 
OFFICE OF 
INSPECTOR GENERAL 
POSTMARKETING  STUDIES  OF 
PRESCRIPTION  DRUGS  OFFICE OF INSPECTOR GENERAL 
The mission  of the  Office of Inspector  General  (OIG),  as mandated  by Public  Law 95-452, as 
amended,  is to protect  the  integrity  of the Department  of Health  and Human  Services (HHS) 
programs  as well as the  health  and welfare  of beneficiaries  served by those  programs.  This 
statutory  mission is carried  out  through  a nationwide  network  of audits,  investigations,  and 
inspections  conducted  by three  OIG  operating  components:  the  Office of Audit  Sefices,  the 
Office of Investigations,  and the  Office of Evaluation  and Inspections.  The  OIG  also informs 
the Secretary  of HHS  of program  and management  problems  and recommends  courses  to 
correct  them. 
OFFICE OF AUDIT  SERVICES 
The  OIGS  Office of Audit  Services  (OAS)  provides  all auditing  services for HHS,  either  by 
conducting  audits with its own audit  resources  or by overseeing  audit work done  by others. 
Audits  examine  the performance  of HHS  programs  and/or  its grantees  and contractors  in 
carrying out  their  respective  responsibilities  and are intended  to provide  independent 
assessments  of HHS  programs  and operations  in order  to reduce  waste,  abuse,  and 
mismanagement  and to promote  economy  and efficiency throughout  the Department. 
OFFICE OF INVESTIGATIONS 
The  OIGS Office of Investigations  (01)  conducts  criminal,  civil, and administrative 
investigations  of allegations  of wrongdoing  in HHS  programs  or to HHS  beneficiaries  and of 
unjust  enrichment  by providers.  The  investigative  efforts  of 01 lead to criminal  convictions, 
administrative  sanctions,  or civil money  penalties.  The  01  also oversees  State  Medicaid  fraud 
control  units which investigate  and prosecute  fraud  and patient  abuse  in the  Medicaid  program. 
OFFICE OF EVALUATION  AND  INSPECIIONS 
The  OIGS  Office of Evaluation  and Inspections  (OEI)  conducts  short-term  management  and 
program  evaluations  (called  inspections)  that  focus on issues of concern  to the Department, 
the  Congress,  and the  public.  The  findings  and recommendations  contained  in these  inspection 
reports  generate  rapid,  accurate,  and up-to-date  information  on the  efficiency, vulnerability, 
and effectiveness  of departmental  programs. 
This report  was prepared  in the  Philadelphia  regional  office of OEI,  under  the  direction  of 
Robert  A  Vito,  Regional  Inspector  General.  Project  staffi 
REGIONAL  HEADQUARTERS 
Isabelle  Buonocore,  Project  Leader  Mary Beth  Clarke,  Program  Specialist

Nancy J. Molyneaux,  Program  Analyst

WonJong  Oh, Intern

Emily Tseng,  Intern

For  additional  copies  of this report,  please  call the Philadelphia  office (215-596-0606).
 Department  of Health  and Human  Services 
OFFICE OF

INSPECTOR GENERAL

POSTMARKETING  STUDIES  OF

PRESCRIPTION  DRUGS

# SERVICE$ 
J+  %

t

w  JUNE  GIBBS BROWN
$ 
<  Inspector  General
2
% 
%++,# 
 Mi4Y 1996 
%>dga  0EI-03-94-O0760  EXECUTIVE  SUMMARY

PURPOSE 
Our  report  describes  the  effectiveness  of the  Food  and  Drug  Administrations  (FDA) 
monitoring  of postmarketing  studies  for  prescription  drugs. 
BACKGROUND 
Postmarketing  studies--investigational  drug  studies  conducted  after  FDA  marketing 
approval--can  have  an  impact  on  how  drugs  are  prescribed  and  used.  A  companys 
agreement  to  conduct  an  FDA-requested  postmarketing  study  is called  a 
postmarketing  or  phase  iv commitment. 
Responsibility  for  monitoring  phase  iv commitments  lies  with  the  Offices  of Drug 
Evaluation  in  the  Center  for  Drug  Evaluation  and  Research  (CDER).  A division  in 
the  CDERS  Office  of Management  was  assigned  the  responsibility  of identi&ing 
commitments  for  all new  drugs  and  tracking  their  status.  The  tracking  results  are 
published  for  CDERS  internal  use  as the  Phase IV Postapproval  Research  List. 
Our  report  provides  FDA  management  with  an  overview  of monitoring  procedures 
and  outcomes,  focusing  on  new  molecular  entities. 
FINDINGS 
%  number of new dregs with postmarking  commitmerm  k increasing.  In the 1970s, 
the highest annual percentage  of such dregs was 33 percent  compared  to a high of 70 
percent  in the XWOs. 
l%ere are no formal  standurds and procedures for monitotig  or for establishing whether a 
postmurkdng  commitment  k nu%. 
W%ik the PhaseIV PostapprovalResearchList contains important  and usefid 
informutio~  it is not a@@  flective  management  tool in iti cument form  Its fonnut 
preclidx  summarization  of records, it k not u.ted  regukdy,  and s&nificant data 
elementi  are not captured cons&mtly. 
%  FDA k taking some steps to improve the tracking of phase  iv comrnitmeruk 
RECOMMENDATIONS 
We  recognize  there  are  limited  resources  and  FDAs  priority  is the  review  of 
premarketing  studies.  This  is especially  true  with  the  shorter  time  frames  in which 
FDA  reviews  premarketing  studies  under  the  Prescription  Drug  User  Fee  Act  of  1992. 
i  We  also  recognize  FDA  is interested  in improving  data  management  for 
postmarketing  commitments  and  is taking  steps  in  that  direction. 
As  our  report  indicates,  FDA  is requesting  postmarketing  studies  for  an  increasing 
number  of  drugs.  We  believe  it is reasonable  to  expect  FDA  to  determine  whether  or 
not  submitted  studies  fulfill  the  commitments.  Until  commitments  are  more 
systematically  monitored  and  brought  to  closure,  FDA  is not  availing  itself  of  a 
management  tool  with  which  to  assess  the  appropriateness  of its  requests  and  plan  for 
the  future. 
We  recommend  that  FDA: 
establish stanch-..,  prvcedums,  or guidelines for canying out monitoring and 
F
tracking objecthm;  and 
 establkh  accountability for monitoring  tracking  and btiging  comrnitmen@ to 
cihure. 
Below  are  some  ideas  for  FDA  to  consider  in  order  to  streamline  data  management: 
. Assign  authority  to  specific  personnel  for  carrying  out  managements  objectives. 
Establish  a coordinator  with  authority  to  (1)  represent  all  offices  involved  in 
monitoring,  tracking,  and  bringing  commitments  to  closure;  and  (2)  ensure  that 
there  is a quality  control  system  for  the  commitments  database. 
. Reduce  the  number  of  documents  that  have  to  be  researched  in  order  to 
identi~  the  status  of commitments;  and  use  standard  notations  on  these 
documents  to  signify  how  each  document  affects  the  status  of the  commitment. 
. Establish  descriptive  categories  for  (1)  types  of studies  requested  and  (2)  status 
of commitments. 
. Put  the  database  of postmarketing  commitments  on-line  in  the  review  divisions 
and  allow  authorized  individuals  to  update  records  as  soon  as  an  action  occurs 
or  a determination  is made. 
FDA  COMMENTS 
The  FDA  agrees  with  this  reports  findings  and  recommendations.  A  draft  directive 
that  establishes  procedures  and  provides  guidance  is undergoing  review  within  CDER 
is expected  to  be  finalized  by  the  end  of June  1996.  (Appendix  C contains  FDAs 
comments  in full.) 
11  TABLE  OF  CONTENTS

EXECUTIVE  SUMMARY

INTRODUCTION  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1

FINDINGS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...5

  Quantity  and  status  ofpostmarketing  commitments  . . . . . . . . . . . . . . . . .  5

  Monitoring  and  bringing  commitments  to  closure  . . . . . . . . . . . . . . . . . . .  7

c  The  Phase IVList  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...9

10

  Improvements  to  the  tracking  system  . . . . . . . . . . . . . . . . . . . . . . . . . . . 
RECOMMENDATIONS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  11

ENDNOTES  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  13

APPENDICES

k.  Categories  for  the  status  ofcommitments  . . . . . . . . . . . . . . . . . . . . . . . . . ..A-1

B:  Number  of commitments  in each  status  as of 1995  . . . . . . . . . . . . . . . . . . . .  B-1

C  Comments  from  the  Food  and  Drug  Administration  . . . . . . . . . . . . . . . . . . .  C-1
 INTRODUCTION

PURPOSE 
This  report  describes  the  effectiveness  of the  Food  and  Drug  Administrations  (FDA) 
monitoring  of postmarketing  studies  for  prescription  drugs. 
BACKGROUND 
Definition  of Postmarketirw  Studies 
Before  drug  companies  can  market  new  drugs  in the  United  States,  they  must  apply  to 
FDA  and  receive  approval.  Normally,  drug  manufacturers  test  new  drugs  in three 
phases  of studies  prior  to  applying  for  marketing  approval.  The  premarketing  studies 
determine  the  drugs  safety  and  efficacy  under  controlled  conditions. 
Prior  to  or  after  granting  marketing  approval,  FDA  may  ask  the  manufacturer  to 
conduct  a phase  iv or  postmarketing  study--synonymous  terms  in  this  report.  This 
request  is made  if FDA  concludes  that  additional  information,  while  not  essential  for 
approval,  will  improve  the  prescribing  and  use  of the  drug.  Postmarketing  studies  may 
confirm  existing  data,  raise  questions,  or  provide  new  data.  Drug  companies  may 
initiate  postmarketing  studies  without  an  FDA  request,  but  in  this  report  we  are  only 
addressing  studies  requested  by FDA. 
As  of the  early  1990s,  new  regulations  at  21 CFR  314.500,  make  it possible  for  FDA 
to  grant  marketing  approval  under  an  accelerated  review  process  for  a small  number 
of  drugs  that  treat  serious  and  life-threatening  illnesses.l  When  FDA  uses  the 
accelerated  review  process,  the  drug  is available  sooner  but  with  less  immediate 
clinical  and  safety  information  than  the  normal  review  process  requires.  When  using 
the  accelerated  process,  FDA  can  require  postmarketing  studies  and  can  withdraw 
marketing  approval  if the  studies  are  not  completed  with  due  diligence. 
For  most  new  drugs,  FDA  does  not  have  the  same  kind  of legal  authority  to  require 
completion  of postmarketing  studies,  Nevertheless,  there  is a tradition  of FDA 
requesting  studies  and  companies  agreeing  to  conduct  them.  A  companys  agreement 
to  conduct  a study  is called  a phase  iv or  postmarketing  commitment.  In  this  report 
we  use  the  term  commitment  when  discussing  agreed-upon  phase  iv studies  for  drugs 
approved  under  the  normal  and  accelerated  processes. 
Who  Does  the  Monitoring 
Monitoring  includes  following-up  on  a phase  iv commitment,  reviewing  any  data  the 
company  submits,  and  tracking  the  status  of the  commitment  until  closure  is reached.2 
1
 Primary  responsibility  for  monitoring  commitments  lies  with  the  reviewers  handling  a 
drug  companys  application  for  marketing  approval.  The  reviewers  are  located  in 
divisions  of the  Offices  of Drug  Evaluation  in FDAs  Center  for  Drug  Evaluation  and 
Research.  (Hereinafter  the  name  of  the  Center  is abbreviated  as the  Center  for 
Drugs).  In  the  course  of  analyzing  a drug  companys  application,  a reviewer  may 
request  that  the  company  conduct  one  or  more  postmarketing  studies.  If the  drug 
under  review  is a not  a new  molecular  entity,3  the  request  is cleared  by  the  review 
division  director.  Requests  relative  to  a new  molecular  entity  are  cleared  at  the  level 
of office  director.  The  company  also  enters  into  discussions  about  the  request  before 
making  a commitment  to  conduct  any  studies. 
The  FDA  usually  enumerates  a companys  postmarketing  commitments  in  a letter  to 
the  company.  Then,  as the  company  submits  data  relative  to  its  commitments, 
reviewers  (ideally  the  same  ones  that  requested  the  phase  iv study)  determine  if any 
action  is necessary.  The  kind  of action  depends  on  the  submission  and  might  include 
a labeling  change. 
While  reviewers  keep  track  of the  phase  iv commitments  for  drugs  they  reviewed,  a 
division  in the  Office  of Management,  Center  for  Drugs,  was  given  the  responsibility 
for  tracking  the  status  of commitments  for  all  drugs  approved  in the  Center.  This  is 
the  Division  of Drug  Information  Resources  (DDIR).  Periodically,  DDIR  staff 
research  documents  in  the  review  divisions  to  identi@  commitments  and  their  status. 
Data  extracted  from  the  documents  are  organized  in  a word-processing  file,  and  the 
file  is published  for  the  Centers  internal  use  under  the  title,  Phase IV Pmtappmval 
Research  list  (hereinafter  abbreviated  as Phase IV List). 
What  the  Phase IV List  Contains 
The  Phase IV List  contains  a record  for  each  drug  (both  prescription  and  over-the-
counter)  that  has  postmarketing  commitments.  Records  are  only  of  commitments 
made  on  original  drug  applications.  There  is no  Centerwide  tracking  of commitments 
for  supplementary  applications  or  for  postmarketing  studies  that  were  not  requested 
by FDA. 
Each  drug  record  briefly  describes  the  commitments  and  their  status  as  of the  last  data 
collection  period,  and  cites  the  documentary  sources  of that  information.  Sources  used 
to  identify  the  commitments  are  generally  one  of two  types  of form  letters  review 
divisions  send  to  drug  companies,  but  other  documents  are  also  used.  The  status  is 
derived  from  drug  company  correspondence,  supplements,  progress  reports,  periodic 
reports,  annual  reports,  and  from  FDA  correspondence,  reviewer  reports,  meeting 
minutes,  and  telephone  or  electronic  contact  reports. 
There  may  be  any  number  of commitments  per  drug,  and  they  are  listed  together  in 
the  drugs  record.  When  all  the  commitments  for  a particular  drug  are  met,  the 
record  for  that  drug  moves  from  the  Phase IV Lists Pending  Commitments  List  to  its 
Commitments  Met  or  Released  List. 
2
 In  1988,  DDIRconducted  the  first  andonly  internal  analysis  ofinforrnation  in  the 
Phase IV List.  The  result  was  a description  of drugs  with  postmarketing  commitments 
between  1972  and  1988,  types  of studies  agreed  to,  and  drug  company  compliance.4 
SCOPE AND  METHODOLOGY 
We  conducted  our  data  collection  and  analysis  between  February  and  October  of

1995.  All  data  is from  FDAs  Center  for  Drugs  in Rockville,  MD.

We  focused  on  all new  molecular  entities  (NMEs)  approved  from  January  1, 1987  to

December  31,  1993,  including  NMEs  approved  under  the  accelerated  review  process.

We  limited  our  analysis  to  NMEs  to  focus  on  drugs  most  likely  to  have  postmarketing

commitments.  New  molecular  entities  are  usually  under  30 percent  of  all new  drug

approvals,  and  they  are  prescription  drugs  which  have  never  been  marketed  as  a single

entity  or  as  part  of  a combination  product.  We  selected  1993  as the  cutoff  year  for

drug  approval  because  it left  2 years--from  1993  to  1995--to  show  if companies  took

action  on  their  commitments.

We  used  the  most  recent  version  (1993)  of the  Phase IV List  to  determine  which  of

the  NMEs  had  phase  iv commitments.  Our  identifications  were  verified  by DDIR.

Since  DDIR  was  planning  to  update  the  Phase IV List  in  1995,  it agreed  to  update  the

status  of the  commitments  in  our  studys  universe  and  send  them  to  us  in  advance  of

its  completing  the  update  for  all its  records,  As  a result,  we  were  able  to  analyze  the

status  of  commitments  as  of August  1995.  The  status  categories  we  created  for  the

analysis  are  similar  to  those  used  by DDIR  in  1988  but  are  somewhat  more

explanatory.  Both  sets  of status  categories  are  in Appendix  A.

We  (1)  quantified  the  number  and  type  of postmarketing  studies  for  NMEs;  (2)

identified  the  pharmacologic-therapeutic  classification  of the  NMEs;  (3)  categorized

each  phase  iv study  according  to  its  current  status;  and  (4)  when  possible,  determined

the  length  of time  between  a companys  submission  of a study  and  FDAs

determination  of closure.  The  tables  and  figure  in  this  report  are  the  result  of our

analysis  of data  in  FDAs  Phase IV List  and  Management  Information  System.

In  addition  to  the  above,  we  conducted  45 on-site  interviews  with  FDAs  professional,

scientific,  and  technical  staff  in the  areas  of drug  review,  drug  information  resources,

document  management,  and  computer  design.  The  respondents  included  office

directors,  division  directors,  drug  reviewers,  project  management  staff  (also  known  as

consumer  safety  officers),  drug  information  officers,  and  technical  information

specialists.  Ninety-three  percent  of the  respondents  were  in the  review  divisions  and

DDIR.  Each  of those  respondents  received  the  interview  questions  in  advance.  .

When  we  began  our  study  there  were  10 review  divisions,  each  specializing  in  certain

drugs.5  We  conducted  interviews  in  6 of the  10 review  divisions,  including  the  two

3
 that  have  used  the  accelerated  approval  process.  The  Center  for  Drugs  has  since 
undergone  a reorganization,  and  the  number  of review  divisions  has  changed. 
Under  separate  cover,  we  gave  the  Center  for  Drugs  our  data  analysis  that  serves  as 
an  extension  of DDIRs  1988  analysis  but  which  is not  directly  related  to  findings 
herein. 
Our  study  was  conducted  in  accordance  with  the  Qualiy  Standards for Inspections 
issued  by  the  Presidents  Council  on  Integrity  and  Efficiency. 
4
 FINDINGS

THE NUMBER  OF NEW  DRUGS  WITH POSTMARKEmG  Co~~ 
IS INCREASING. 
Percentage  of Drum  with  Postmarketin~  Commitments 
While  the  annual  percentage  of new  molecular  entities  (NMEs)  with  commitments 
varies,  there  has  been  an  upward  trend  as  shown  in Figure  A.  In  the  1970s,  the 
highest  percentage  of  such  drugs  was  33 percent  compared  to  a high  of  70 percent  in 
the  1990s. 
Figure  A 
Annual Percent of NMEs With 
Phase IV Commitments,  1972-1993 
76-

71-

66-

61-

56-

51-

a-

41-

36-

31

26  -

21

16 lb  I 
11)
 ,
1 1 I  1 t  I  1 I 1 1 1 1 !  I ,  1  1 ,  1  I 
72  74  76  78  80  82  64  86  88  90  92 
Year of Approval 
More  than  half  of the  NMEs  in  our  study  period,  1987-1993,  have  postmarketing 
commitments.  For  these  years,  FDA  approved  a total  of  169 NMEs  including  2 under 
the  accelerated  approval  process.  Fifty-three  percent,  or  90  of the  169 drugs  had 
commitments.  Table  1 on  the  next  page  shows  the  percent  by year. 
There  was  a total  of  385  commitments  for  the  90  drugs,  and  an  average  of 4 
commitments  per  drug  as  shown  in Table  2.  Twenty-five  of the  90 drugs  had  only  one 
5  commitment  each.  Of  the  385  commitments,  35 were  for  the  2 NMEs  approved  under
the  accelerated  process. 
Table  1 
PHASE IV NMIZ.s*AS A PERCENTAGE  OF TOTAL NMEs 
1987-1993 
! 
Year  Total NMEs  Phase IV NMEs 
I  I 
1987  21  9  (43%) 
I  I 
1988  20  6  (30%) 
1989  24  12  (50%) 
1990  23  12  (52%) 
1991  30  21  (70%) 
1992  25  13  (52%) 
1993  26  17  (65%) 
TOTAL  169  90  (53%) 
~Mti  wtth phase w (postmarketmg) commitments 
Table  2 
AVERAGE  NUMBER OF PHASE IV COMMITMENTS 
PER PHASE IV NME* 
1987-1993 
Year of  Number of  Number of Phase  Average Number 
Drug  Phase IV NMEs  IV Commitments  of Commitments 
Approval  Per NME 
1987  9  37  4 
I  I 
1989  12  36  3 
1 
1990  I  12  I  44  14 
1991  21  96  5 
1992  13  80  6 
Status  of PostmarketinE  Commitments 
The  FDA  data  indicates  that  out  of the  385  phase  iv studies  it  asked  companies  to 
conduct,  150  are  currently  in progress  and  146 were  completed  and  submitted  to  FDA. 
6  In  other  words,  296  studies,  or77percent  of thestudies,  areeither  inprogress  or  have

been  submitted.  Thetable  in Appendti  Bshowsthe  status  ofcommitments  as  of

August  1995  for  all NMEs  approved  between  1987  and  1993,  including  NMEs

approved  under  the  accelerated  review  process.

Fifty-seven  percent  or880f  the  150studies  inprogress  are  actually  underway.  The

remaining  62 studies  break  out  as follows.  One  study  is perpetual  (i.e.,  an  ongoing

surveillance  study);  11 were  halted;  and  50 were  completed  but  a report  has  not  yet

been  submitted  to  FDA.

Of  the  146 studies  that  were  submitted  to  FDA  FDA  reviewed  40 and  found  39

acceptable  and  only  1 not  acceptable.  It  appears  to  have  taken  FDA  an  average  of  16

months  to  determine  that  25 studies  were  acceptable.  Dates  that  would  have  allowed

us  to  calculate  an  average  time  for  all  39  studies  are  missing  from  the  Phase IV List.

Nine  of the  39 studies  FDA  found  acceptable  were  studies  for  accelerated  NMEs.

Five  of these  nine  studies  had  determinations  within  11 days  to  21 months  after  the

date  of the  studys  submission.G

The  FDA  data  does  not  indicate  whether  106 out  of the  146 submitted  studies  are

acceptable  or  not.  Dates  in  the  Phase IV Lz3t indicate  that  30 of  the  106 studies  have

been  without  a determination  for  2 to  7-1/2  years.  Dates  for  the  remaining  76 studies

are  missing.  One  study  for  an  accelerated  NME  has  not  had  a determination  for

almost  3 years.7

Thirty-five  studies  out  of the  total  385  have  not  yet  begun,  either  because  the  company

(a)  is postponing  action  until  FDA  reviews  a supplement  application,  (b)  did  not  agree

to  conduct  the  study,  or  (c)  plans  to  start  the  study  in  the  future.

The  FDA  released  companies  from  23 of the  385  commitments.  Twelve  of these

released  commitments  were  for  drugs  that  are  not  marketed.  For  the  remaining  11

studies  in  this  status  catego~,  either  (a)  the  studies  were  not  feasible,  (b)  FDAs

questions  were  answered  by  other  studies,  or  (c)  the  reason  for  release  was  not  clearly

explained  in the  Phase IV List,

There  was  no  status  information  at  FDA  for  31  (eight  percent)  of the  385  studies.  All

31 studies  were  for  drugs  approved  from  1990  through  1993.

THERE  ARE  NO FORMAL  STANDARDS  AND  PROCEDURES  FOR 
MONITORING  OR ESTABLISHING WHETHER  A COMMITMENT  IS MET. 
Review  Divisions 
There  are  no  formal  Centerwide  standards  and  procedures  for  monitoring 
postmarketing  studies.  Monitoring  in the  review  divisions  is up  to  individuals. 
Reviewers  request  postmarketing  studies  and  review  company  submissions  in their 
normal  scope  of work  which  is reviewing  investigational  new  drug  applications,  new  drug  applications,  and  efficacy  and  labeling  supplements.  However,  following-up  on a

commitment  and  reviewing  a phase  iv submission  are  done  as time  allows  and  are

dependent  on  the  commitments  relative  importance.

There  is no  formal  mechanism  for  establishing  whether  a commitment  is met  and  then

taking  the  commitment  out  of the  pending  section  of the  Phase IV List.  There  are

indications  that  in  some  cases  someone  has  determined  that  phase  iv studies  were

acceptable  and  the  commitments  were  met.  For  example,  we  found  such

determinations  for  39  of the  146 submitted  studies.  During  our  inteniews,  most

review  division  respondents  said  the  reviewer  would,  at  least  initially,  decide  whether  a

commitment  was  met.  However,  no  one  has  the  formal  responsibility  of determining

and  recording  whether  or  not  a companys  phase  iv submission  meets  the  commitment.

This  may  explain  why  data  in  the  Phase IV List  indicates  that  106 submitted  studies  for

NMEs  approved  from  1987  through  1993  are  still without  determinations.

The  review  divisions  have  a procedure  for  advising  drug  companies  of their

commitments  around  the  time  the  new  drug  receives  market  approval.  But,  they  do

not  have  a procedure  for  notifying  a company  when  and  whether  their  phase  iv

submissions  have  been  reviewed  and  accepted  as fulfillment  of the  commitments.

While  individual  reviewers  may  relay  this  to  drug  companies,  the  records  do  not

indicate  that  such  notification  occurs.

Under  the  current  monitoring  which  relies  on  individuals,  reviewers  may  or  may  not

have  ways  of reminding  themselves  to  contact  a company  about  a commitment.

Reviewers  either  rely  on  reminders  from  project  management  staff,  have  their  own

tickler  system,  or  respond  when  a question  arises.  But  reviewers  and  review  division

directors  alike  believe  the  important  studies  are  memorable  and  most  companies  meet

their  commitments.  Some  respondents  said  relying  on  individuals  within  an  informal

system  could  be  a problem.  They  cited  that  a reviewer  could  leave  the  division  and

remaining  staff  would  not  be  aware  of pending  commitments.

While  some  review  division  respondents  believe  there  should  be  guidelines  regarding

phase  iv monitoring,  a greater  number  feel  the  current  way  things  are  done  is good

enough  given  limited  resources.  Most  of these  respondents  feel  it is unnecessary  to

have  formal  procedures  because  in  most  cases  it is extremely  difficult  for  FDA  to

enforce  compliance.  They  feel  once  the  drug  is approved,  it is up  to  the  company  to

take  the  next  step  regarding  their  commitment.

Phase IV List 
While  the  Phase IV List  is a tool  the  review  divisions  could  use  as  a reminder  of which 
commitments  are  pending,  most  reviewers  and  division  directors  were  not  aware  of its 
existence.  The  office  directors  and  project  management  staff  knew  of the  Phase IV 
List  but  they  did  not  make  full  use  of it. 
8
 The  informality  of phase  iv tracking  in  the  review  divisions  hampers  DDIR  in its 
ability  to  collect  complete  and  timely  phase  iv data  for  the  Centerwide  F%ase IV L&. 
All  sources  of postmarketing  data  are  located  in review  division  documents.  But 
because  documents  containing  phase  iv information  are  so numerous  and  are  not 
indexed,  identi&ing  them  is very  time-consuming.  In  addition,  there  is no  way  for  the 
tracking  staff  to  be  sure  they  have  located  all the  relevant  documents.  It is only  when 
documents  indicate  that  a phase  iv study  fulfilled  the  commitment  for  a particular  drug 
that  DDIR  removes  it from  the  pending  section  of the  Phase IV L&. 
Currently,  there  is no  formal  mechanism  by which  DDIR  can  receive  feedback  from 
the  review  divisions  about  the  Phase IV List.  Respondents  in  DDIR  say updating  the 
Phase IVList  requires  a great  deal  of work  and  they  hope  their  product  is useful.  But 
without  feedback  from  end-users  they  do  not  know  if the  Phase IV List  is addressing 
user  needs. 
WHILE  THE Phase lVL&  CONTAINS  IMPORTANT  INFORMATION,  IT IS NOT 
A FULLY  EFFECTIVE  MANAGEMENT  TOOL  IN ITS CURRENT  FORM. 
Important  Information 
The  Phase IV LA  is the  only  instrument  containing  postmarketing  commitment  data 
on  every  drug  (including  NMEs)  approved  by the  Center  for  Drugs.  The  Phase IV 
List  is made  up  of one  record  for  each  drug  with  commitments,  and  the  records  are 
grouped  according  to  the  review  division  handling  the  drug.  A  record  contains 
identifiers  for  the  drug,  the  drugs  approval  date,  a narrative  description  of the 
commitments,  a narrative  description  of the  status,  the  name  and  date  of the  sources 
of information,  and  the  name  of a contact  person  in  the  review  division.  We were 
able  to  use  the  Phase IV List  for  most  of  our  analyses. 
Timeliness  and  Completeness 
The  Phase IV List  is not  updated  on  a regular  or  frequent  basis,  and  certain  data 
elements  are  not  captured  consistently.  At  the  time  of our  interviews  in mid-1995,  the 
Phase IV List  was  already  2 years  old.  Many  of the  dates  that  would  give  management 
time  frames  for  company  or  FDA  activity  were  missing.  Also,  drugs  approved  under 
the  accelerated  review  process  were  not  identified  as  such.  However,  the  timeliness 
and  completeness  of  data  is affected  by  the  lack  of standards  and  procedures  for 
monitoring  phase  iv studies  in  the  review  divisions.  Without  procedures  to  keep 
abreast  of commitments  and  to  record  that  activity  in  a standard  way,  the  same  phase 
iv data  cannot  be  captured  consistently  Centerwide. 
Format 
The  Phase IV List  does  not  contain  summarizations  of the  numerous  records.  Two 
factors  can  explain  the  absence  of summaries.  One  factor  is that  extended  narratives 
are  used  in  each  record  to  describe  a commitment  and  its  status.  The  narrative 
9  format  makes  it hard  to  see  patterns  or  anomalies  in the  types  of studies  requested  or 
in  company  compliance.  While  an  individual  can  manually  go  through  each  record  in 
the  Phase IV List,  create  categories,  and  then  count  up  items  in  the  categories  (as we 
did),  this  is very  time-consuming.  Furthermore,  narratives  are  subject  to  interpretation 
and  categorizing  them  for  analysis  is a matter  of individual  judgment. 
The  second  factor  is the  type  of  computer  software  in which  the  information  is stored. 
Currently,  the  information  is in  a word-processing  program  which  cannot  automatically 
summarize  data.  Given  the  nature  of the  data,  the  need  for  quick  turn-around 
reporting,  and  the  need  for  ease  of data  entry  and  changes,  we  believe  more 
convenient  software,  such  as  a relational  database,  may  be  more  appropriate. 
THE  FDA IS TAKING  SOME STEPS TO IMPROVE  THE TRACKING  OF 
COMMITMENTS. 
In  late  1994,  the  Center  for  Drugs  established  a work  group  to  look  at  tracking 
problems.  As  of September  1995  at  least  two  improvements  were  being  implemented. 
The  first  is that  postmarketing  commitments  will be  formally  listed  in  a standard  letter 
to  the  company.  This  will  reduce  the  number  of documents  DDIR  staff  have  to 
research  in  order  to  identify  a new  commitment.  The  second  improvement  is that  the 
Phase IV List  will  be  transferred  from  its  current  word-processing  program  to  a 
database  program. 
The  Center  plans  to  continue  its  efforts  to  improve  the  tracking  system.  One  goal  is 
to  have  the  Phase IV List  on-line  for  greater  accessibility.  Another  is to  make 
company  submissions  more  easily  identifiable  for  tracking  purposes. 
10
 RECOMMENDATIONS 
We  recognize  that  FDA  has  limited  resources  and  its  priority  is the  review  of 
premarketing  drug  studies.  This  is especially  true  with  the  shorter  time  frames  FDA 
has  to  review  premarketing  studies  and  new  drug  applications  under  the  Prescription 
Drug  User  Fee  Act  of  1992  (P.L.  102-571).  Wealsorecognize  that  the  Center  for 
Drugs  is interested  in improving  the  data  management  system  for  postmarketing 
commitments  and  has  started  to  do  so.  Since  FDA  work  groups  are  currently  meeting 
to  improve  tracking,  this  is an  opportune  time  to  make  data  collection  less  labor 
intensive  and  to  create  a system  that  meets  the  informational  needs  of senior 
management  in  the  Center  for  Drugs. 
As  our  report  indicates,  FDA  is requesting  postmarketing  studies  for  an  increasing 
number  of new  drugs,  and  most  of the  commitments  are  long-term  in nature. 
Furthermore,  for  drugs  approved  under  the  accelerated  review  process,  FDA  can 
move  to  withdraw  the  drugs  from  the  market  if the  postmarketing  studies  are  not 
completed  with  due  diligence. 
Until  the  monitoring  system  is made  less  dependent  on  individual  memory,  FDA  risks 
losing  track  of the  commitments.  Unless  time  frames  are  tracked  and  monitored, 
FDA  cannot  uniformly  determine  diligence  on  the  part  of drug  companies.  Until  the 
commitments  are  monitored  and  tracked  systematically,  FDA  is not  availing  itself  of a 
management  tool  with  which  to  assess  the  appropriateness  of its  requests  and  plan  for 
the  future. 
Lastly,  FDAs  requests  for  postmarketing  studies  put  added  responsibilities  on  the 
drug  companies.  Therefore,  it is reasonable  to  expect  FDA  to  determine  whether  or 
not  the  studies  fulfill  the  commitments  and  to  noti~  the  companies  of the 
determination. 
We  recommend  that  FDA: 
	 estabtih  standkrds, procedures,  or guidelines for individuals responsible for canying 
out trwnito~g  and tracki%gobjectivm;  and 
F	 establikh accountability for monitoring  tracking  and bringing comtments  to 
cihsure. 
Below  are  some  ideas  for  FDA  to  consider  in  order  to  streamline  data  management: 
. Assign  authority  to  specific  personnel  for  carrying  out  management  objectives. 
Establish  a coordinator  with  authority  to  (1)  represent  all  offices  involved  in 
monitoring,  tracking,  and  bringing  commitments  to  closure;  and  (2)  ensure  that 
there  is a quality  control  system  for  the  commitments  database. 
11  . Reduce  thenumber  ofdocuments  that  have  to  beresearched  in  order  to 
identify  the  status  of commitments;  and  use  standard  notations  on  these 
documents  to  signify  how  each  document  affects  the  status  of the  commitment. 
. Establish  descriptive  categories  for  (1)  types  of studies  requested  and  (2)  status 
of  commitments. 
. Put  the  database  of postmarketing  commitments  on-line  in the  review  divisions 
and  allow  authorized  individuals  to  update  records  as  soon  as  an  action  occurs 
or  a determination  is made. 
FDA COMMENTS 
The  FDA  agrees  with  this  reports  findings  and  recommendations.  A  draft  directive 
that  establishes  procedures  and  provides  guidance  is undergoing  review  within  CDER 
is expected  to  be  finalized  by  the  end  of June  1996.  (Appendix  C contains  FDAs 
comments  in full.) 
4.-I 
1A  ENDNOTES

1.  Accelerated  approvals  are possible  in only two situations: 
(1)	 When  approval  can be based  on a drugs effect  on a surrogate  endpoint 
that  is reasonably  likely to predict  clinical benefit,  or on a drugs effect 
on a clinical endpoint  other  than  survival or irreversible  morbidity. 
After  approval,  the  a~plicant  would be required  to complete  adequate 
and well-controlled  clinical studies  alreadv underwav  at the  time  of 
aumoval,  to confirm  the  predictive  value  of the  surro~ate  end~oint  or 
other  indicator.  (21 CFR  314.510) 
(2)	 When  FDA  determines  that  an effective  but  highly toxic drug can be 
used safely only if distribution  or use is modified  or restricted.  (21 CFR 
314.520) 
2.  Closure  is defined  by FDAs  Division  of Drug  Information  Resources  as: 
(1)	 the  drug product  was not  marketed  or it was withdrawn  from the  market 
(closure  would be void if the drug were re-marketed); 
(2)  the  drug company  was released  from all phase  iv commitments  by FDA; 
(3)	 the  research  was terminated  for reasons  mutually  agreed  to by the  drug 
company  and FDA  or 
(4) the  research  was completed  and accepted  by FDA  as scientifically valid 
and in fulfillment  of the  commitment(s). 
3.	 Definition  of new molecular  entity:  The  active moiety  (component  or molecule)  has 
not  previously  been  approved  or marketed  in the  United  States  by any drug 
manufacturer  either  as a single entity  or as part  of a combination  product. 
4.	 The  analysis was performed  by staff in the  Division  of Drug  Information  Resources, 
Office of Management,  Center  for Drug  Evaluation  and Research.  It was used in 
papers  presented  at (1) the  23rd Annual  Meeting  of the  U.S. Public  Health  Service 
Professional  Association  in Scottsdale,  AZ,  May 1988, and (2) the  25th Annual 
Meeting  of the  Drug  Information  Association  in Boston,  ~  June  1989. 
5. Review  Divisions  (as of May 1995):  (1) Cardio-Renal,  (2) Neuropharmacological,  (3) 
Oncology  and Pulmonary,  (4) Medical  Imaging  and Surgical-Dental,  (5) 
Gastrointestinal  and Coagulation,  (6) Metabolism  and Endocrine,  (7) Anti-infective,  (8) 
Anti-Viral,  (9) Topical  Drug  Products,  and (10) Pilot  Drug  Evaluation  Staff. 
6.	 We used the  following method  to calculate  the  time it took  FDA  to review and 
determine  that  studies  were  acceptable:  Out  of the  39 studies  found  acceptable,  25 
had submission  and determination  dates.  We calculated  the  difference  between  the 
dates.  The  time ranged  from 5 days to nearly 7 years, for an average  of a little  over  16 
months. 
13  7.	 We used  the  following  method  to calculate  the  time that  studies  have been  without  a 
determination:  Out  of 106 studies  without  a determination,  30 had submission  dates. 
We calculated  the  difference  between  those  dates  and August  31, 1995 which was when 
the  studies status  was most recently  updated.  The  time periods  for the  30 studies 
ranged  from 2 to 7-1/2 years. 
14
 APPENDIX  A

CATEGORIES  FOR THE STATUS OF POSTMARKETING  CO~NTS 
On  the  left  are  the  15 categories  we  established  in  order  to  analyze  status  information 
in FDAs  Phase IV Postapproval  Research  List.  On  the  right  are  the  eight  categories 
established  by FDA  in  1988. 
See  Appendix  B for  the  number  of commitments  in  each  status  category  as  of August 
1995. 
OIG  -1995  FDA  -1988 
StudyNot Begun  A. Study not begun  or status  could 
1.1Will begin in the future  not be determined 
1.2 Company  did not agree  to 
conduct  study  B. Study in progress 
1.3 Company  awaiting approval 
of a supplement  application  C. Study is perpetual 
studiynProgrW  D. Study submitted  and FDA 
2.1 Underway  determined  commitment  is met 
2.2 Completed  but not yet 
submitted  to FDA  E. Study submitted  and was 
2.3 Halted  rejected  by FDA 
2.4 Perpetual 
F. Study submitted  and FDA  has not 
Study Submittedto FDA
 determined  if commitment  is met 
3.1 Accepted  by FDA

3.2 Not accepted  by FDA
 G. Company  released  from 
3.3 No determination  made
 commitment 
by FDA

H. Study not completed  due to drug 
ReleasedfromCommitment
 not being marketed 
4.1 Drug  not marketed

4.2 Questions  answered  by

other  studies

4.3 Reason  unclear

4.4 Study not feasible

statuunsknown 
5.1 No information 
A-1
 APPENDIX  B

NUMBER OF PHASE lV COMMHMENR  IN EACH STATUS 
II 
STATUS AS OF AUGUST 1995  \  YEAR OF DRUG APPROVAL 
StudyNot Begun  I 87 
1.1  Will begin in future  11 
1.2 Company did not agree to  3 
conduct study 
1.3  Company awaiting approval of  O 
supplement 
I 
study in Progress 
2.1  Underway 
2.2 Completed but not yet  I  1 
submitted to FDA 
2.3  Halted  10 
2.4  Perpetual  I o 
StudySubmitted 
I 
3.1  Accepted by FDA  \  10 
3.2  Not accepted by FDA  11 
4.1  Drug not marketed  18 
4.2 Question answered by other  1 
studies 
4.3  Reason unclear  o 
1 
4.4  Study not feasible  10 
status unknown  I 
B-1
 APPENDIX  C 
COMMENTS FROM THE FOOD AND DRUG ADMINISTRATION 
c-1  %
* 
5WOC,* 
*4+  $ PubIii  HealttI Service
:-*Y   DEPARTMENT OF HliiMTH  & HUMAN SERVICES  Foodand  Drug Administradon 
:
z 
%
%
++2.,w  Memorandum 
28D%
.~ 
Date 
From 
Deputy  Commissioner  for Management  and Systems  (Acting) 
Subject  Food  and Drug  Administrations  (FDA)  Comments  on the OffIce  of Inspector 
Generals  (OIG)  Draft  Report,  Postmarketing  Studies  of Prescription  Drugs,   
(OEI-03-94-O0760),  -- ACTION 
To 
Deputy  Inspector  General  for  Evaluation  and Inspections 
We reviewed  the  referenced  draft  report  and prepared  the attached  comments. 
The  FDAs  Center  for Drug  Evaluation  and Research  has  agreed  with  the  reports 
recommendation  and  is beginning  to implement  it. 
If your  staff  has  any  questions,  please  have  them  contact  Jim  Dillon  on (301)  443-6392. 
Attachment 
[C-2]  COM  MFNTS  OF  THF  FOO17  AND  DRUG  A13MIN ISTRATION  (FDA)  ON  THF  OF FICF 
s! 
OF  INSPECTOR  GFNFRAI  (OICJ) DRAF T RFPORT,  OSTMARKF  TING  STUDIFS  OF 
PRFSCRIP TION  DRUGS.   OFI-03-94-00760,  FFBRUAR Y 1996 
General  Commen& 
We  appreciate  the opportunity  to review  and comment  on the  referenced  OIG  draft  report. 
The  FDAs  Center  for Drug  Evaluation  and  Research  (CDER)  agrees with  the reports 
findings  and  recommendation.  CDER  wishes  to express  its gratitude  to the OIG  evaluators 
for considering  their  comments  and suggestions  on the working  draft  report. 
OIG  RecQmmendatior1 
We  recommend  that  FDA  establish  standards,  procedures,  or guidelines  for individuals 
responsible  for carrying  out monitoring  and tracking  objectives;  and,  establish 
accountability  for monitoring,  tracking,  and  bringing  commitments  to closure. 
FDA  comment 
FDA  concurs.  CDER  has formed  a working  group that  is canying  out the  intent  of the 
recommendation.  This working  group  has drafted  a directive  that establishes  procedures 
and provides  guidance  for resolving  various  issues.  The draft directive  tentatively  entitled, 
Tracking  Phase 4 Commitments,  is undergoing  review  within  CDER  and should  be 
finalized  by end  of June.  CDER  expects  implementation  shortly  thereafter.  The  directive 
will  become  part of the  Manual  of Policies  and Procedures  and will  be available  for use by 
all CDER  employees. 
[C-3]